The practice of using covariates is widespread in both experimental and correlational research, yet controversy remains on whether (and when) it is appropriate. Traditionally touted by statisticians as a useful method to soak up noise in a dependent variable and boost power, the practice recently has been recast in a negative light because of Type I error inflation (e.g., Simmons, Nelson, & Simonsohn, 2011). To make informed decisions about using covariates, researchers need to know more about the actual size of the associated costs and benefits.

In a series of simulations, I quantified the tradeoff between power and Type I error rates from flexible, post-hoc covariate use in experimental designs, and we found that including a single, strong covariate, even when unplanned, can substantially boost power while only slightly inflating Type I error (Wang, Sparks, Hess, Gonzales, & Ledgerwood, 2017, JESP). When more than one covariate is flexibly included or when the covariate is weakly correlated with the dependent variable, however, the tradeoff becomes more unfavorable. I provide researchers guidance on how to appropriately incorporate such exploratory analyses.