Multiple OLS Regression

Adjusting for the confounders you can see

Merlin Schaeffer · Department of Sociology

2026-07-28

By the end of today you can …

1. explain omitted variable bias — why leaving a confounder out of a regression shifts a coefficient, and in which direction;

2. read a multiple OLS coefficient as “holding the other variables constant” — and show what that means with the Frisch–Waugh three-step;

3. estimate an adjusted effect in R — and say honestly what multiple OLS does not fix.

One part per goal. Our running question: back to Legewie’s Bali attack — but this time the natural experiment is not quite as-if random.

Set the stakes: they can already run lm() with two predictors — nobody needs a lecture to add a +. What they cannot yet do is say why the number moved, by how much, and in which direction. That is today. Flag goal 3 as the honest one: today’s tool is weaker than a coin flip, and we will say exactly how.

Where we are

Three lectures, three ways to earn a causal claim:

• L6 — randomise it. A coin balances everything, observed and not. Gold standard, rarely available.
• L7 — find a natural experiment. Let the world randomise for you; use an instrument if compliance is broken.
• Today — adjust for it. No coin, no instrument: measure the confounder and hold it constant.

Beware: these are in descending order of credibility. Today’s tool is the weakest of the three — and the one you will use most often.

This slide replaces about ten slides of IV recap from the old deck — they did Minneapolis and the Wald estimator properly last week, so do not re-teach it. Just place today in the sequence and be blunt about the ranking: randomisation, then natural experiments, then statistical adjustment. Today’s tool is the weakest and the one they will reach for constantly, which is exactly why it needs a careful lecture.

The crack in last week’s natural experiment

Part 1 of 3

Legewie’s Bali design assumed interview date was as-if random. The balance table says: almost, but not quite.

Almost random is not random

Last week we treated the day of a respondent’s ESS interview as as-if random — so the people interviewed after the Bali attack should look like those interviewed before.

Legewie checked. They do not quite:

“The treatment group is on average slightly younger, and the proportion of people who are retired and who work from home is lower.”

Legewie (2013, p. 1211)

Discuss: why would people interviewed early in a survey period differ systematically from those interviewed late?

Do not re-explain the Bali design — one sentence of reminder, then straight to the crack. The mechanism behind the imbalance is reachability bias: people who are easy to contact get interviewed early, and being easy to contact correlates with being retired, at home, older. So the “randomisation” leaked. Let them work that out from the blue box before you say it.

Age, the confounder

1. Xenophobia tends to increase with age.

2. The younger a respondent, the more likely they were still to be interviewed after the attack.

\[\Rightarrow \text{Avg}[Y_{0i} \mid D = 0] \;\color{#901A1E}{>}\; \text{Avg}[Y_{0i} \mid D = 1]\]

The treatment group had a lower baseline to begin with — so the raw comparison understates the attack’s effect.

In Portugal the imbalance runs the other way — older respondents were more likely to land in the treatment group. Same logic, opposite sign.

Discuss: does this imbalance make our estimate of the Bali effect too large or too small?

The two bullets are the two arrows — walk them one at a time, then trace the backdoor path. Getting the direction right is the hardest exam skill in this block, so make them commit out loud before you move on. Do not resolve it yet; the toy simulation is where they will see it happen. And flag the Portugal footnote, because their exercise uses Portugal and the sign is reversed there.

The problem, stated plainly

We cannot re-run the survey. The confounder is already baked into the data.

Discuss: age is measured — it is sitting right there in the data set. Can we use that somehow to repair the comparison after the fact?

Yes — that is exactly what multiple OLS does. It compares people of the same age who differ only in whether they were interviewed after the attack. The rest of today is how, and how much you should trust it.

The pivot of the lecture. The key contrast: last week the design fixed the problem before the data existed; today we fix it afterwards, with a variable we happened to measure. Put weight on “happened to measure” — it sets up the honesty slide in Part 3.

A laboratory where we know the truth

Real data never tells you the right answer. So let’s simulate data where we set the true effect — then check whether OLS finds it.

• Packages
• Step 1: age
• Step 2: who gets treated
• Step 3: the outcome

pacman::p_load(
  tidyverse,    # Data manipulation and visualization
  estimatr,     # OLS with robust standard errors
  modelr,       # Residuals and predictions from a model
  modelsummary  # Nicely formatted regression tables
)

set.seed(831983) # So everyone gets the SAME "random" numbers

toydat <- tibble(
  age = rnorm( # 500 fictitious people
    n = 500,
    mean = 40, # Average age 40
    sd = 5     # Standard deviation 5 years
  )
)

# Older people are interviewed EARLIER, so they less often
# end up in the post-attack (treatment) group.
toydat <- toydat %>%
  mutate(
    int_day = (-0.1 * age) + rnorm(n = 500, mean = 0, sd = 1),
    bali    = if_else(int_day >= mean(int_day), 1, 0)
  )

# WE decide the truth: the Bali effect is exactly 1,
# and each year of age adds 0.3 to xenophobia.
toydat <- toydat %>%
  mutate(
    xeno = (1 * bali) + (0.3 * age) + #<<
      rnorm(n = 500, mean = 0, sd = 1)
  )

The true causal effect of Bali is 1. Remember that number — it is the only time in this course you will know it.

Sell the trick before the code: in real research we never know the right answer, so we cannot tell a good method from a bad one. In a simulation we write down the truth and then check whether our method recovers it. Walk the three steps as a story — people have ages, age pushes them into one group, and then we build xenophobia by hand out of the treatment plus age plus noise. Point hard at the highlighted line: the truth is 1. They must carry that number into the next slide.

Which model is right?

# Bivariate: the treatment only
ols_bi <- lm_robust(xeno ~ bali, data = toydat)

# Multiple: add the confounder
ols_mult <- lm_robust(xeno ~ bali + age, data = toydat)

modelsummary(
  list("Bivariate" = ols_bi, "Multiple" = ols_mult),
  coef_rename = c("bali" = "Exposed to Bali", "age" = "Age"),
  stars = TRUE, gof_map = c("nobs", "r.squared"),
  output = "kableExtra"
)

Discuss: we know the true effect is 1. Which of these two models found it?

	Bivariate	Multiple
(Intercept)	12.504***	−0.499
	(0.113)	(0.428)
Exposed to Bali	0.174	1.135***
	(0.154)	(0.096)
Age		0.313***
		(0.010)
Num.Obs.	500	500
R2	0.003	0.649
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

The bivariate model is badly wrong — it misses the truth by a wide margin. Adding one variable, age, recovers ≈ 1. Same data, same outcome, one extra column.

This is the money slide of Part 1. Show only the first column, ask them to compare it to 1, and let the size of the miss land. Then reveal the second column. The lesson to state explicitly: nothing about the data changed and nothing about the outcome changed — the only difference is which variables we let the model see. Do not explain why yet; that is Part 2, and the suspense is useful.

Omitted variable bias

Part 2 of 3

The gap between those two numbers is not random noise. It has a formula.

Where the gap comes from

When we compare xenophobia between the exposed and the unexposed, we are comparing people who differ in two ways:

1. They differ in exposure — that is the causal effect we want, 1.14 in our simulation.

2. They also differ in age. How much? Regress the confounder on the treatment:

Discuss: the exposed are on average 3.1 years younger. Given that each year of age adds ≈ 0.3 xenophobia, how much xenophobia does that age gap alone account for?

# How much do the two groups differ in AGE?
ols_age <- lm_robust(age ~ bali, data = toydat)

modelsummary(
  list("Age" = ols_age),
  coef_rename = c("bali" = "Exposed to Bali"),
  stars = TRUE, gof_map = c("nobs"),
  output = "kableExtra"
)

	Age
(Intercept)	41.546***
	(0.289)
Exposed to Bali	−3.070***
	(0.395)
Num.Obs.	500
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

Build the arithmetic with them rather than showing it. Two groups, two differences: the one we want and the one that is contaminating it. Get the age gap on the board, remind them each year is worth about 0.3, and let someone multiply. When they produce the number they will recognise it as almost exactly the gap between the two columns on the previous slide — that recognition is the point.

The formula

\[\underbrace{\tilde\beta_{D \rightarrow Y}}_{\text{what the biased model gives}} = \underbrace{\beta_{D \rightarrow Y}}_{\text{the truth}} + \underbrace{(\beta_{C \rightarrow Y} \times \beta_{D \rightarrow C})}_{\textbf{omitted variable bias}}\]

Put our simulated numbers in — the two paths, multiplied:

\[\underbrace{0.17}_{\text{bivariate}} \;\approx\; \underbrace{1.14}_{\text{multiple}} \;+\; (\underbrace{0.31}_{C \rightarrow Y} \times \underbrace{-3.07}_{D \rightarrow C})\]

It adds up. The bias is not a mystery: it is the confounder’s effect on the outcome, times the imbalance in the confounder between the groups.

Let them watch the arithmetic close. Two things make this slide work: the bias is the product of the two dashed arrows, and it is a product — so if either arrow is zero there is no bias at all. Ask what that means practically: a variable is only a confounder if it affects the outcome and is imbalanced across the groups. That is the test they should apply for the rest of the course.

Which way does the bias run?

The formula also tells you the sign — before you run anything:

\(\beta_{C \rightarrow Y}\)	\(\beta_{D \rightarrow C}\)	Bias
\(+\)	\(+\)	too large
\(+\)	\(-\)	too small
\(-\)	\(+\)	too small
\(-\)	\(-\)	too large

Same rule as multiplying two signed numbers — because that is exactly what it is.

Discuss: in Legewie’s data, age raises xenophobia (\(+\)) and the exposed group is younger (\(-\)). Was his raw estimate too large or too small?

Too small. The bias is negative, so the unadjusted estimate understates the effect of the attack. Adjusting for age should push it up.

The most exam-relevant slide in the deck. Make them do the sign reasoning out loud before revealing — and note it is just multiplying two signs, nothing more. The payoff is that they can predict the direction of a bias without any data, which is what a good discussant does in a seminar. Tell them explicitly that “controlling for it will lower the estimate” is a claim they can now check rather than guess.

Naming things

You will meet the same idea under several names. They are the same problem seen from different angles:

• Selection bias (L5) — the general problem: groups differ at baseline.
• Confounder bias — named after the variable that causes it.
• Omitted variable bias — named after the fact that it is missing from your model.

“Omitted variable bias” is the useful name today, because it points at the fix: the variable is omitted, so put it in.

Beware: that only works for variables you actually measured. There is no formula for the confounders you never recorded.

Students get genuinely confused by the three names, so spend thirty seconds tidying the vocabulary. The distinction worth making: selection bias is the general condition, omitted variable bias names one specific culprit that you could have included. Then plant the red box — it is the seed of the honesty slide later in Part 3.

Break

Your turn: exercise 1

You estimate the adjusted Bali effect for Portugal — controlling for age, gender and employment status — and compare it to the unadjusted one.

Open exercise 1 in a new tab ↗

How OLS does it — and what it can’t do

Part 3 of 3

“Holding age constant” sounds like a metaphor. Frisch and Waugh (1933) showed it is an arithmetic procedure.

Frisch–Waugh, step 1

Regress the outcome on the confounder, keep the residuals — the part of xenophobia that age does not explain.

toydat <- toydat %>%
  add_residuals(
    model = lm(xeno ~ age, data = .),
    var   = "e_xeno"
  )

Each red line is one person’s residual: how far they sit from what their age alone predicted.

Announce the three steps before starting, so they know where this is going. Step 1 strips age out of the outcome. Ask what the leftovers mean: e_xeno is xenophobia for someone of that age — above or below what age alone would predict. That phrasing is what “holding age constant” actually means, and it is worth saying in those words.

Frisch–Waugh, step 2

Regress the treatment on the confounder, keep the residuals — the part of Bali exposure that age does not explain.

toydat <- toydat %>%
  add_residuals(
    model = lm(bali ~ age, data = .),
    var   = "e_bali"
  )

Discuss: what kind of person has a large positive e_bali?

Exactly the same operation, now on the treatment — the symmetry is the point, both variables cleaned with the same rag. The blue box has a nice concrete answer: someone older than you would expect given that they were exposed, i.e. a surprising case. Those surprising people are precisely the ones carrying the information, which sets up the “why it works” slide.

Frisch–Waugh, step 3

Regress the two sets of residuals on each other. Its slope is identical to the multiple-regression coefficient.

ols_resid <- lm_robust(e_xeno ~ e_bali, data = toydat)

modelsummary(
  list("Multiple OLS" = ols_mult, "Residualised" = ols_resid),
  coef_rename = c("bali" = "Exposed to Bali", "age" = "Age",
                  "e_bali" = "Exposed to Bali (resid.)"),
  coef_omit = "(Intercept)",
  stars = TRUE, gof_map = c("nobs"),
  output = "kableExtra"
)

	Multiple OLS	Residualised
Exposed to Bali	1.135***
	(0.096)
Age	0.313***
	(0.010)
Exposed to Bali (resid.)		1.135***
		(0.096)
Num.Obs.	500	500
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

Same number, two routes. That is all “controlling for age” does — it removes age from both variables and fits a line to what is left.

We meet this again in Lecture 9, on real cross-country data.

Let them see the two coefficients match before you say anything. The demystification is the whole point: multiple regression is not a black box that “adjusts” in some vague way, it subtracts the confounder out of both variables and runs the same bivariate OLS they already know. Flag the forward-reference — they will do this again in Lecture 9 on real data, so a flicker of recognition then is intended.

Why it works

Look at step 2 again: after removing age, the people with the most residual variation in treatment are the untypical ones — the older person who was still interviewed late, the younger one interviewed early.

Multiple OLS quietly down-weights typical cases and up-weights untypical ones. The comparison is carried by people who broke the pattern.

Beware: if nobody breaks the pattern — if age predicted exposure perfectly — there would be no residual variation left, and nothing to estimate from.

A genuinely useful intuition that most textbooks skip. The estimate is carried by the surprising cases, the people whose treatment status you could not have guessed from their age. Then the warning, which matters practically: the more strongly a control predicts the treatment, the less variation survives and the noisier the estimate gets. That is the honest reason you cannot just throw in every control you can think of.

Still just OLS

Nothing new is being minimised. The best-fitting plane still minimises the sum of squared residuals:

\[\min \sum_{i=1}^{n} \left(y_i - (\alpha + \beta_1 x_{1i} + \dots + \beta_k x_{ki})\right)^2\]

\[y_{i} = \alpha + \beta_{1}x_{1} + \beta_{2}x_{2} + \ldots + \beta_{k}x_{k} + \epsilon_{i}\]

Beware — two things that follow:

1. Multiple OLS assumes a linear relation for every continuous predictor, not just the one you care about.

2. Every coefficient is adjusted for all the others. There is no such thing as adding a control “just for one variable”.

Keep this brief — it is reassurance, not new machinery. Same loss function they met in Lecture 2, just more slopes. The two warnings are what actually matters: the linearity assumption now applies to every continuous control, and adjustment is mutual, so a badly chosen control can distort the coefficient they care about. Point three arrives on the next slide.

What multiple OLS cannot fix

It can close a backdoor path through a confounder you measured — even long after the data were collected.

It cannot do anything about a confounder you never measured. And no test will tell you one is there.

If the confounders you can see turned out to be imbalanced — is it likely that the ones you cannot see are perfectly balanced?

The double circle is a closed backdoor path — we controlled for it. The red dashed one stays wide open.

The honesty slide, and the one to slow down for. The double circle means closed; the red path is still open and no diagnostic will ever flag it. Then put the rhetorical question to them properly, because it is the strongest argument in the lecture: if the confounders you could see were imbalanced, what are the odds the invisible ones happen to be perfectly balanced? This is precisely why L7’s natural experiment and L6’s coin outrank today’s tool.

Back to Bali: the Portuguese sample

The same wrangling as last week — now keeping age as well, and restricting to Portugal.

• Balance
• R code

	Before Bali (N=187)		After Bali (N=104)
	Mean	Std. Dev.	Mean	Std. Dev.	Diff. in Means	Std. Error
age	45.8	19.4	48.5	17.6	2.6	3.0
anti_immi	3.6	0.9	4.0	0.8	0.3	0.1

ESS %>%
  select(treat, age, anti_immi, pspwght) %>%
  rename(weights = pspwght) %>% # Used as survey weights
  datasummary_balance(~ treat, data = .,
                      output = "kableExtra")

Discuss: in Portugal, which group is older — and so, by the sign rule, is the raw estimate too large or too small here?

Careful: the direction is the opposite of the all-country pattern Legewie reports. Portugal is its own case.

Now the real data, where nobody knows the truth. Point out that the balance table is the empirical counterpart of \(\beta_{D \rightarrow C}\) from the formula — the imbalance term, measured. In Portugal the treatment group is older, which is the reverse of the pooled pattern, so the sign rule now predicts the adjustment will push the estimate down. Make them commit to a direction before the next slide.

The adjusted effect

ols_pt_bi   <- lm_robust(anti_immi ~ treat,
                         weights = pspwght, data = ESS)
ols_pt_mult <- lm_robust(anti_immi ~ treat + age,
                         weights = pspwght, data = ESS)

modelsummary(
  list("Bivariate" = ols_pt_bi, "Adjusted" = ols_pt_mult),
  coef_rename = c("treatAfter Bali" = "After Bali", "age" = "Age"),
  stars = TRUE, gof_map = c("nobs", "r.squared"),
  output = "kableExtra"
)

	Bivariate	Adjusted
(Intercept)	3.621***	3.328***
	(0.082)	(0.175)
After Bali	0.330**	0.313**
	(0.116)	(0.118)
Age		0.006+
		(0.003)
Num.Obs.	291	291
R2	0.032	0.050
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

The same decomposition, now on real data:

\[\underbrace{0.33}_{\text{bivariate}} \;\approx\; \underbrace{0.313}_{\text{adjusted}} \;+\; (\underbrace{0.006}_{C \rightarrow Y} \times \underbrace{2.62}_{D \rightarrow C})\]

Adjusting for age moves the estimate by only -0.017 SD. Why so little? The bias is a product — and in Portugal age barely predicts xenophobia (0.006 per year). A big imbalance times a tiny effect is still a tiny bias.

The payoff, and the honest one: the formula they verified in a toy simulation also works on real survey data — walk the arithmetic once more. But do not oversell the shift, because it is small, and that is the lesson. Ask them why before you reveal: the imbalance is real, 2.6 years, yet age turns out to barely move xenophobia in Portugal. A product with one near-zero factor is near zero. This is exactly the point from the formula slide, now confirmed on real data — and a useful corrective to the instinct that every control must be added.

It’s real research

Source: Legewie (2013, p. 1210)

Legewie’s published models adjust for a whole battery of observed confounders — age, gender, education, employment, and more.

Alternative names for the same thing, all of which you will meet in papers: controlling for \(X\) · partialling out \(X\) · adjusted for \(X\) · conditional on \(X\).

Show them the published table so today’s move looks like normal scholarly practice rather than a classroom exercise. The vocabulary box is genuinely useful — all four phrases mean the identical operation, and papers switch between them without warning. Mention that they will produce a version of this table themselves in exercise 1.

Your turn: exercise 2

You become the simulator: set the confounder’s strength and sign yourself, predict the bias from the formula, then check whether OLS agrees.

Open exercise 2 in a new tab ↗

Today’s general lessons

1. A confounder you leave out of your model creates omitted variable bias: \(\tilde\beta = \beta + (\beta_{C \rightarrow Y} \times \beta_{D \rightarrow C})\). It is the same problem as selection bias, named after the missing variable.

2. Because the bias is a product of two effects, you can predict its sign before running anything — and it is zero if either effect is zero.

3. Frisch–Waugh: “controlling for \(C\)” means residualise \(Y\) on \(C\), residualise \(D\) on \(C\), and regress the leftovers. Nothing more mysterious than that.

4. Multiple OLS is still OLS — one plane, minimising squared residuals, with every coefficient adjusted for all the others.

5. It can repair imbalance in observed confounders. It can do nothing about unobserved ones — which is why a good design still beats a good regression.

Run these fast, they are tired. If one sentence survives the day, make it point five: regression fixes the confounders you thought to measure, and nothing else. That is precisely why Lecture 6 randomised and Lecture 7 went hunting for natural experiments — and why Lecture 9 will push on how far clever controls can take you.

Check yourself: today’s goals

• Write down the omitted variable bias formula, and say what makes the bias zero.
• Given a DAG and the signs of two arrows, say whether an unadjusted estimate is too large or too small.
• Explain in the three Frisch–Waugh steps what “controlling for age” does — and name one thing multiple OLS cannot fix.

Shaky on any of these? That is what this week’s Absalon quiz and the Friday exercise class are for.

Today’s important functions

• lm_robust(y ~ d + c, ...): multiple OLS — the coefficient on d holds c constant.
• modelr::add_residuals(model = ..., var = ...): keep a model’s residuals (the Frisch–Waugh workflow).
• modelsummary::datasummary_balance(~ group, data = ...): the balance table — your empirical read on \(\beta_{D \rightarrow C}\).
• modelsummary(list("Bivariate" = ..., "Adjusted" = ...)): put the unadjusted and adjusted models side by side.
• set.seed() + rnorm(): simulate data where you know the true effect.

References

Frisch, R. and F. V. Waugh (1933). “Partial Time Regressions as Compared with Individual Trends”. In: Econometrica, pp. 387-401.

Legewie, J. (2013). “Terrorist Events and Attitudes toward Immigrants: A Natural Experiment”. In: American Journal of Sociology, pp. 1199-1245.