What happens if our balance test tells us that randomization has failed? If our balance test tells us that randomization has failed, it means that the treatment and control groups are not comparable on all of the important confounding variables.

Is everything lost? Fortunately, not everything is lost! Multiple OLS regression can be used to control for imbalanced confounders. Multiple OLS regression is a statistical method that estimates the relationship between a dependent variable and one or more independent variables. The Frisch-Waugh Theorem gives us an intuitive understanding of how OLS controls for imbalanced confounders. The theorem states that the OLS estimator is the same as the least squares estimator that is obtained by regressing the outcome of interest on the confounders and then using the residuals from this regression as the dependent variable in a second regression with the treatment as the independent variable.

OLS regression is a powerful tool for controlling for imbalanced confounders, but it is not a perfect fix. If our balance test tells us that randomization has failed, we can use OLS regression to control for imbalanced confounders. However, it is important to remember that OLS is not a perfect fix. OLS can only control for confounders that we have observed in the data.

Lecture slides.

Reading: Veaux, Velleman, and Bock (2021, Ch. 9.1-9.4).