The practice of using covariates is widespread in both experimental and correlational research, yet controversy remains on whether (and when) it is appropriate. Traditionally touted by statisticians as a useful method to soak up noise in a dependent variable and boost power, the practice recently has been recast in a negative light because of Type I error inflation (e.g., Simmons, Nelson, & Simonsohn, 2011). To make informed decisions about using covariates, researchers need to know more about the actual size of the associated costs and benefits. In a series of simulations, I quantified the tradeoff between power and Type I error rates from flexible, post-hoc covariate use in experimental designs, and we found that including a single, strong covariate, even when unplanned, can substantially boost power while only slightly inflating Type I error (Wang, Sparks, Hess, Gonzales, & Ledgerwood, 2017, JESP). When more than one covariate is flexibly included or when the covariate is weakly correlated with the dependent variable, however, the tradeoff becomes more unfavorable.
My subsequent work on covariate use extended the scope to correlational study designs. One common goal for using covariates in correlational studies is to make incremental validity claims (e.g., “X predicts Y over and above Z”). These claims are common in many research domains—for example, by our estimate, more than half of empirical work in relationship science contains them—yet they are also fraught with problems (Wang & Eastwick, invited revision under review; Ledgerwood, Wang, & da Silva Frost, invited chapter in prep), including overemphasis of counterintuitive findings, neglect of measurement issues, and inflation of Type I error rates. We offered solutions to these problems: to critically examine the value of incremental validity claims, and to use latent variable approaches that model measurement properties of constructs and more accurately estimate the effects of interest. My research on covariate use demonstrates how considerations of both the conceptual value and the practical implications of statistical procedures can guide researchers toward better inferences and more robust findings.