Multiple OLS Regression

Adjusting for the confounders you can see

Merlin Schaeffer · Department of Sociology

2026-10-26

By the end of today you can …

1. explain omitted variable bias: why leaving a confounder out of a regression shifts a coefficient, and in which direction;

2. read a multiple OLS coefficient as “holding the other variables constant”, and show what that means with the Frisch–Waugh three-step;

3. estimate an adjusted effect in R, and say honestly what multiple OLS does not fix.

Set the stakes: they can already run lm() with two predictors — nobody needs a lecture to add a +. What they cannot yet do is say why the number moved, by how much, and in which direction. That is today. Flag goal 3 as the honest one: today’s tool is weaker than a coin flip, and we will say exactly how.

Closer to home: Copenhagen, February 2015

A gunman attacked a debate on free speech at Krudttønden, then the Great Synagogue in Krystalgade. Two people were killed.

Discuss: did the attacks make people in Denmark more hostile towards immigrants? How could you find out?

Krudttønden, 15 February 2015.
Photo: Kim Bach.

Great Synagogue, 22 February 2015.
Photo: Kim Bach.

Why not for Denmark? Of everyone the ESS interviewed in the five weeks around the Bali attack, only 2 were living in Denmark; interviewing only got going at the very end of October, after the window had closed. Natural experiments need lucky timing.

Open close to home. Many of them will remember February 2015: the attack on the free-speech debate at Krudttønden in Østerbro, then the shooting at the synagogue in Krystalgade; Finn Nørgaard and Dan Uzan were killed. Sensitive topic, so keep it factual. Ask the question and collect a few designs before you click. Click: last week’s answer, the natural experiment. Click: the Danish twist, which is a real number from Legewie’s data. Denmark simply was not in the field when Bali happened, so the design cannot say anything about Danes. That is the point: you cannot plan a natural experiment, you can only be lucky. And today we will see that even a lucky one can have a crack.

Where we are

Three lectures, three ways to estimate causal effects:

Beware: read from left to right in descending order of credibility. Today’s tool is the weakest of the three, and the one you will use most often.

The map of the causal half of the course, rebuilt from the old deck’s design tree. Walk it top-down: the only question that matters is who decided who got treated. A researcher’s coin (L6): an RCT, plain bivariate OLS, the average causal effect κ. Nature’s coin (L7): the intention-to-treat comparison ρ, and when compliance is broken the instrument, λ = ρ/φ, the LATE for compliers. Today’s branch, in red: nobody flipped a coin at all, so we measure the confounders and adjust for them. Be blunt about the ranking and stress the “if” in the last red box; it is today’s whole honesty problem in one word.

The crack in last week’s natural experiment

Part 1 of 3

Legewie’s Bali design assumed interview date was as-if random. The balance table says: almost, but not quite.

The Bali bombing memorial, Kuta. Photo: Jorge Láscar, CC BY 2.0.

Almost random is not random

Last week we treated the day of a respondent’s ESS interview as as-if random: the people interviewed after the Bali attack should look like those interviewed before.

Legewie did not just assume it. The paper checks it, and honestly reports that they do not quite:

“The treatment group is on average slightly younger, and the proportion of people who are retired and who work from home is lower” Legewie (2013, p. 1211).

Discuss: why would people interviewed early in a survey period differ systematically from those interviewed late?

Do not re-explain the Bali design: one sentence of reminder, then straight to the crack. Make it personal: Joscha is a friend, and the paper is a model of good conduct, because it runs the balance check on its own design and prints the uncomfortable answer instead of hiding it. That is what we want from them in their own papers. The mechanism behind the imbalance is reachability bias. Let them work it out from the blue box before you move on; the next slide shows the answer as a story.

Who opens the door first?

An ESS interviewer rings at two doors on the first day of fieldwork.

Reachability bias: the easy to reach (retired, at home, older) are interviewed early, before the attack. The hard to reach (working, out, younger) only after several visits, often after it.

So the interview day is not quite a coin flip: it depends on age.

The answer to the last slide’s question, as a story. Inge is retired and at home, so the interviewer gets her on the very first visit, mid-September: she lands in the control group. Mikkel works shifts and is out three times; the interviewer leaves a card each time and only catches him on the fourth visit, after the attack: he lands in the treatment group. Nobody did anything wrong, and still the two groups now differ in age. Inge and Mikkel are invented, but the pattern is exactly what Legewie’s balance table shows.

Age, the confounder

1. Xenophobia tends to increase with age.

2. The younger a respondent, the more likely they were still to be interviewed after the attack.

\[\Rightarrow \text{Avg}_n[Y_{0i} \mid D_i = 0] \;\color{#901A1E}{>}\; \text{Avg}_n[Y_{0i} \mid D_i = 1]\]

In Portugal the imbalance runs the other way. Older respondents were more likely to land in the treatment group. Same logic, opposite sign.

Discuss: does this imbalance make our estimate of the Bali effect too large or too small?

The treatment group had a lower baseline xenophobia to begin with \(\rightarrow\) the raw comparison understates the attack’s effect.

The two bullets are the two arrows — walk them one at a time, then trace the backdoor path. Getting the direction right is the hardest exam skill in this block, so make them commit out loud before you move on. Do not resolve it yet; the toy simulation is where they will see it happen. And flag the Portugal footnote, because their exercise uses Portugal and the sign is reversed there.

The hidden term, selection bias, is back

Lecture 5 split group comparisons into two parts:

\[\underbrace{\text{Avg}_n[Y_{1i} \mid D_i = 1] - \text{Avg}_n[Y_{0i} \mid D_i = 0]}_{\text{what we compare}} = \text{ATT} \; + \; \underbrace{\text{Avg}_n[Y_{0i} \mid D_i = 1] - \text{Avg}_n[Y_{0i} \mid D_i = 0]}_{\text{Selection Bias } = \; 0 \text{ if exposure is as-if random (L6, L7)}}\]

\[\underbrace{\text{Avg}_n[Y_{1i} \mid D_i = 1] - \text{Avg}_n[Y_{0i} \mid D_i = 0]}_{\text{what we compare}} = \text{ATT} \; + \; \color{#901A1E}{\underbrace{\text{Avg}_n[Y_{0i} \mid D_i = 1] - \text{Avg}_n[Y_{0i} \mid D_i = 0]}_{< \; 0 \text{: the treated are younger}}}\]

Last week: we assumed the interview day was as-if random, and that both groups shared the same baseline \(Y_0\). We thought the selection-bias term was zero and the raw difference was the effect.

Beware: in Legewie’s data the term is not quite zero. The treated are younger, so their baseline \(Y_0\) is lower, and the raw comparison understates the effect.

Today: for a confounder we measured, we can put a number on this hidden term (Part 2) and remove it (Part 3).

The bridge to Lecture 5. Put the decomposition back on the screen: what we compare equals the effect on the treated plus the hidden selection-bias term. Remind them that randomisation, by a researcher or by nature, is what set that term to zero. Click: in Legewie’s data it is not zero any more; the treated are younger, so their baseline is lower and the term is negative. Click: today’s promise. For a confounder that happens to be in the data set, the hidden term stops being hidden: we can compute it, and we can take it out.

A laboratory where we know the truth

Like engineers who crash a car with a dummy before they trust it with people, we simulate data where we set the true effect, then check how OLS works.

• Packages
• Step 1: age
• Step 2: who gets treated
• Step 3: the outcome

pacman::p_load(
  tidyverse,    # Data manipulation and visualization
  estimatr,     # OLS with robust standard errors
  modelr,       # Residuals and predictions from a model
  modelsummary  # Nicely formatted regression tables
)

set.seed(831983) # So everyone gets the SAME "random" numbers

toydat <- tibble(
  age = rnorm( # Random age
    n = 500,   # for 500 fictitious people
    mean = 40, # Average age 40
    sd = 5     # Standard deviation 5 years
  )
)

# Older people are interviewed EARLIER, so they less often
# end up in the post-attack (treatment) group.
toydat <- toydat %>%
  mutate(
    int_day = (-0.1 * age) + rnorm(n = 500, mean = 0, sd = 1),
    bali    = case_when(
      int_day < mean(int_day) ~ 0,   # before the attack
      int_day >= mean(int_day) ~ 1)  # after the attack
  )

The older someone is, the less likely they land in the post-attack group.

# In this simulation, 
# WE decide the truth: the Bali effect is exactly 1,
# and each year of age adds 0.3 to xenophobia.
toydat <- toydat %>%
  mutate(
    xeno = (1 * bali) + (0.3 * age) +
      rnorm(n = 500, mean = 0, sd = 1)
  )

Two overlapping clouds. The red ones sit a little further left: they are younger.

Sell the trick before the code, with the crash-test dummy: engineers do not test a new car on real people, they crash it with a dummy whose injuries they can measure exactly. A simulation is our crash test. In real research we never know the right answer, so we cannot tell a good method from a bad one; here we write down the truth and check whether our method recovers it. Walk the three steps as a story, and use the plots: people have ages (step 1), age pushes them into one group, older people less often after the attack (step 2), and then we build xenophobia by hand out of the treatment plus age plus noise (step 3). Point hard at the highlighted line: the truth is 1. They must carry that number into the next slide.

Which model is right?

# Bivariate: the treatment only
ols_bi <- lm_robust(xeno ~ bali, data = toydat)

# Multiple: add the confounder
ols_mult <- lm_robust(xeno ~ bali + age, data = toydat)

modelsummary(
  list("Bivariate" = ols_bi, "Multiple" = ols_mult),
  coef_rename = c("bali" = "Exposed to Bali", "age" = "Age"),
  stars = TRUE, gof_map = c("nobs", "r.squared"),
  output = "kableExtra"
)

	Bivariate	Multiple
(Intercept)	12.504***	−0.499
	(0.113)	(0.428)
Exposed to Bali	0.174	1.135***
	(0.154)	(0.096)
Age		0.313***
		(0.010)
Num.Obs.	500	500
R2	0.003	0.649
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

The bivariate model is badly wrong. It misses the truth by a wide margin. Adding one variable, age, recovers ≈ 1.

This is the money slide of Part 1. Show only the first column, ask them to compare it to 1, and let the size of the miss land. Then reveal the second column. The lesson to state explicitly: nothing about the data changed and nothing about the outcome changed — the only difference is which variables we let the model see. Do not explain why yet; that is Part 2, and the suspense is useful.

Omitted variable bias

Part 2 of 3

The gap between those two numbers has a formula.

Where the gap comes from

The exposed and the unexposed differ in two ways:

1. In exposure: the causal effect. We know it is 1; the multiple model’s 1.135 is 1 plus a little sampling error (Lecture 3).

2. In age: the exposed are on average 3.07 years younger (the table).

Discuss: each year of age adds ≈ 0.313 xenophobia. How much xenophobia does that age gap alone account for?

Take 1 and 2 together: 1.135 + (-0.961) = 0.174, exactly the bivariate estimate.

# How much do the two groups differ in AGE?
ols_age <- lm_robust(age ~ bali, data = toydat)

modelsummary(
  list("Age" = ols_age),
  coef_rename = c("bali" = "Exposed to Bali"),
  coef_omit = "(Intercept)",
  stars = TRUE, gof_map = c("nobs"),
  output = "kableExtra"
)

	Age
Exposed to Bali	−3.070***
	(0.395)
Num.Obs.	500
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

The age gap alone: \(-3.07 \times 0.313 = -0.961\) xenophobia.

Build the arithmetic with them rather than showing it, one click at a time, as in the old deck. Two groups, two differences: the one we want and the one that is contaminating it. Start with the first: we know the truth is 1, and the multiple model’s 1.135 is 1 plus sampling error, a callback to Lecture 3. Click: the age gap, from the regression of the confounder on the treatment. Click: ask them to multiply the gap by the age effect. Click: the product. Click: add the two pieces and they get the bivariate estimate from the previous slide, to the third decimal. That recognition is the point.

Seeing the gap

1. Compare the two group averages: they are only 0.174 apart. That is the bivariate estimate.

2. But the groups are not similarly old: the exposed are 3.07 years younger on average.

3. Now compare people of the same age. Multiple OLS fits two parallel lines; at any age they are 1.135 apart. That is the multiple OLS estimate, holding age constant.

4. Walk along the red line from the exposed group’s average age to the other group’s: it climbs 0.313 × 3.07 = 0.961. That is omitted variable bias: 1.135 − 0.961 = 0.174.

Zoomed in on the middle of the toy data.

The same numbers as the last slide, as a picture; this is the slide that makes “holding constant” visible. Step 1: the two group averages, the dashed lines, are only 0.17 apart, the bivariate estimate. Click: but the big dots show the groups sit at different ages, the exposed about three years further left. Click: multiple OLS fits one line per group with the same slope; at any given age the lines are 1.14 apart, and that vertical gap between people of the same age is what “holding age constant” means. Click: the triangle. Walking along the red line from the exposed group’s average age to the unexposed group’s, the line climbs slope times age gap, 0.96. That climb is the omitted variable bias, and the bivariate gap is just the same-age gap minus the climb. This identity is exact, not an approximation.

The formula

\[\underbrace{\tilde\beta_{D \rightarrow Y}}_{\text{what the biased model gives}} = \underbrace{\beta_{D \rightarrow Y}}_{\text{the truth}} + \underbrace{(\beta_{C \rightarrow Y} \times \beta_{D \rightarrow C})}_{\textbf{omitted variable bias}}\]

Put our simulated numbers in: the two paths, multiplied:

\[\underbrace{0.174}_{\text{bivariate}} \;=\; \underbrace{1.135}_{\text{multiple}} \;+\; (\underbrace{0.313}_{C \rightarrow Y} \times \underbrace{-3.07}_{D \rightarrow C})\]

It adds up, exactly. The bias is the confounder’s effect on the outcome, times the imbalance in the confounder between the groups. It is the size of Lecture 5’s hidden term, for one confounder.

Generalise what they just computed and saw. Two things make this slide work: the bias is the product of the two dashed arrows, and it is a product, so if either arrow is zero there is no bias at all. Ask what that means practically: a variable is only a confounder if it affects the outcome and is imbalanced across the groups. That is the test they should apply for the rest of the course. Close the loop to Lecture 5: this product is the hidden selection-bias term, now with a name and a size.

Which way does the bias run?

The formula also tells you the sign:

\(\beta_{C \rightarrow Y}\)	\(\beta_{D \rightarrow C}\)	Bias
\(+\)	\(+\)	too large
\(+\)	\(-\)	too small
\(-\)	\(+\)	too small
\(-\)	\(-\)	too large

The most exam-relevant slide in the deck. Make them do the sign reasoning out loud before revealing — and note it is just multiplying two signs, nothing more. The payoff is that they can predict the direction of a bias without any data, which is what a good discussant does in a seminar. Tell them explicitly that “controlling for it will lower the estimate” is a claim they can now check rather than guess.

Naming things

These terms name the same problem, but from different angles:

• Selection bias (L5): the general problem, groups differ at baseline.
• Confounder bias: named after the variable that causes it.
• Omitted variable bias: named after the fact that it is missing from your model.

“Omitted variable bias” is the useful name today, because it points at the fix: the variable is omitted, so put it in.

Beware: that only works for variables you actually measured. There is no formula for the confounders you never recorded.

Students get genuinely confused by the three names, so spend thirty seconds tidying the vocabulary. The distinction worth making: selection bias is the general condition, omitted variable bias names one specific culprit that you could have included. Then plant the red box — it is the seed of the honesty slide later in Part 3.

Break

Your turn: exercise 1

You estimate the adjusted Bali effect for Portugal, controlling for age, gender and employment status. And compare it to the unadjusted one.

Open exercise 1 in a new tab ↗

How multiple OLS solves omitted variable bias — and what it can’t do

Part 3 of 3

“Holding age constant”: Frisch and Waugh (1933) showed it is an arithmetic procedure.

Ragnar Frisch (1895–1973), a Norwegian, coined the word econometrics, was the first editor of the journal Econometrica, where this theorem appeared in volume 1, and shared the very first Nobel prize in economics, 1969. His co-author Frederick Waugh was an agricultural economist in the US.

Photo: Borgens Atelier.

Give the theorem a face, and a Scandinavian one. Frisch invented the word econometrics, edited Econometrica from its first issue, and won the first ever Nobel prize in economics. The theorem they are about to learn was published in the journal’s very first volume, in 1933. Then the promise of Part 3: “holding constant” is not a metaphor, it is three regressions they can run themselves.

“Holding age constant”

“comparing people with the same values on the other variables, …”. That is what holding constant, controlling for and adjusted for mean.

The multiple model gave 1.135 for Exposed to Bali. And 0.313 for Age: the slope of both lines.

Discuss: say in one sentence what each of the two coefficients means.

“Comparing people of the same age, the ones interviewed after the attack score 1.14 points higher on xenophobia.”

“Among people interviewed before/after the attack, each extra year of age goes with 0.31 points more.”*

The reading skill from learning goal 2, practised before the machinery. Point at the picture: the vertical gap between the two parallel lines is the treatment coefficient, and the common slope of both lines is the age coefficient. Make two or three students say the sentence out loud before you click, and insist on the opening words “comparing two people of the same age”. Then the template. Every multiple OLS coefficient they ever read, in any paper, gets this sentence.

Frisch–Waugh, step 1

Frisch and Waugh (1933) demonstrated that “controlling” for a confounder is mathematically equivalent to a three-step procedure.

Regress the outcome on the confounder, keep the residuals: the part of xenophobia that age does not explain.

(toydat <- toydat %>%
   add_residuals(
     model = lm(xeno ~ age, data = .),
     var   = "e_xeno" # A new column
   ))
# # A tibble: 500 × 5
#     age int_day  bali  xeno   e_xeno
#   <dbl>   <dbl> <dbl> <dbl>    <dbl>
# 1  42.5   -3.90     1  15.2  1.97   
# 2  45.3   -5.10     0  13.3 -0.764  
# 3  42.7   -2.89     1  12.8 -0.532  
# 4  37.1   -4.40     0  11.8 -0.00133
# # ℹ 496 more rows

Each red line is one person’s residual: how far they sit from what their age alone predicted.

The double circle: we control for age, which closes the backdoor path.

Announce the three steps before starting, so they know where this is going, and introduce the double circle: from now on it means “we controlled for this variable”, which closes the backdoor path through it. Step 1 strips age out of the outcome. Point at the printout: the data set has a new column, e_xeno. Ask what the leftovers mean: e_xeno is xenophobia for someone of that age, above or below what age alone would predict. That phrasing is what “holding age constant” actually means, and it is worth saying in those words.

Frisch–Waugh, step 2

Regress the treatment on the confounder, keep the residuals: the part of Bali exposure that age does not explain.

(toydat <- toydat %>%
   add_residuals(
     model = lm(bali ~ age, data = .),
     var   = "e_bali" # Another new column
   ))
# # A tibble: 500 × 6
#     age int_day  bali  xeno   e_xeno e_bali
#   <dbl>   <dbl> <dbl> <dbl>    <dbl>  <dbl>
# 1  42.5   -3.90     1  15.2  1.97     0.582
# 2  45.3   -5.10     0  13.3 -0.764   -0.318
# 3  42.7   -2.89     1  12.8 -0.532    0.590
# 4  37.1   -4.40     0  11.8 -0.00133 -0.609
# # ℹ 496 more rows

Discuss: what kind of person has a large positive e_bali?

Exactly the same operation, now on the treatment — the symmetry is the point, both variables cleaned with the same rag. The blue box has a nice concrete answer: someone older than you would expect given that they were exposed, i.e. a surprising case. Those surprising people are precisely the ones carrying the information, which sets up the “why it works” slide.

Frisch–Waugh, step 3

Regress the two sets of residuals on each other. Its slope is identical to the multiple OLS coefficient.

ols_resid <- lm_robust(e_xeno ~ e_bali, data = toydat)

modelsummary(
  list("Multiple OLS" = ols_mult, "Residualised" = ols_resid),
  coef_rename = c("bali" = "Exposed to Bali", "age" = "Age",
                  "e_bali" = "Exposed to Bali (resid.)"),
  coef_omit = "(Intercept)",
  stars = TRUE, gof_map = c("nobs"),
  output = "kableExtra"
)

	Multiple OLS	Residualised
Exposed to Bali	1.135***
	(0.096)
Age	0.313***
	(0.010)
Exposed to Bali (resid.)		1.135***
		(0.096)
Num.Obs.	500	500
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

Same number, two routes. That is what “controlling for age” does: it removes age from both variables and fits a line to what is left.

Let them see the two coefficients match before you say anything. The demystification is the whole point: multiple regression is not a black box that “adjusts” in some vague way, it subtracts the confounder out of both variables and runs the same bivariate OLS they already know. Flag the forward-reference — they will do this again in Lecture 9 on real data, so a flicker of recognition then is intended.

Why it works

Look at step 2 again: after removing age, who still has a lot of unexplained exposure? The untypical cases: the older people who were still interviewed after the attack, and the younger ones interviewed before it.

Multiple OLS quietly down-weights typical cases and up-weights untypical ones. The comparison is carried by people who broke the pattern.

Beware: if nobody breaks the pattern (if age predicted exposure perfectly) there would be no residual variation left, and nothing to estimate from.

A genuinely useful intuition that most textbooks skip, now with both pictures side by side as in the old deck. Left: step 2 again, with the untypical people in red, the quarter with the largest residuals. Older people who were still interviewed after the attack, younger people interviewed before it: in the story, a retiree who happened to be away, a student who opened the door on day one. Right: the same red people in step 3. They are exactly the points far out on the x-axis, so they pull hardest on the slope. Then the warning, which matters practically: the more strongly a control predicts the treatment, the less variation survives and the noisier the estimate gets. That is the honest reason you cannot just throw in every control you can think of.

Still just OLS

Nothing new is being minimised. The best-fitting plane still minimises the sum of squared residuals:

\[\min \sum_{i=1}^{n} \left(y_i - (\alpha + \beta_1 x_{1i} + \dots + \beta_k x_{ki})\right)^2\]

\[y_{i} = \alpha + \beta_{1}x_{1} + \beta_{2}x_{2} + \ldots + \beta_{k}x_{k} + \epsilon_{i}\]

With a 0/1 treatment and one control, the “plane” is simply the two parallel lines from Seeing the gap.

Beware — two things that follow:

1. Multiple OLS assumes a linear relation for every continuous predictor, not just the one you care about.

2. Every coefficient is adjusted for all the others. There is no such thing as adding a control “just for one variable”.

Keep this brief — it is reassurance, not new machinery. Same loss function they met in Lecture 2, just more slopes. The two warnings are what actually matters: the linearity assumption now applies to every continuous control, and adjustment is mutual, so a badly chosen control can distort the coefficient they care about. Point three arrives on the next slide.

What multiple OLS cannot fix

It can close a backdoor path through a confounder you measured.

It cannot do anything about a confounder you never measured. And no test will tell you one is there.

The streetlight effect. A man searches for his lost keys under a street lamp. Did he lose them here? “No, but this is where the light is.” We control for what we happened to measure, not necessarily for what matters.

The double circle is a closed backdoor path: we controlled for it. The red dashed one stays wide open.

Discuss: if the confounders you can see turned out to be imbalanced, is it likely that the ones you cannot see are perfectly balanced?

The honesty slide, and the one to slow down for. Tell the streetlight joke first: the man searches under the lamp not because he lost his keys there, but because that is where he can see. Regression adjustment is searching under the lamp: we adjust for the confounders our survey happened to measure. The double circle means closed; the red path is still open and no diagnostic will ever flag it. Then put the question to them properly, because it is the strongest argument in the lecture: if the confounders you could see were imbalanced, what are the odds the invisible ones happen to be perfectly balanced? This is precisely why L7’s natural experiment and L6’s coin outrank today’s tool.

Back to Bali: the Portuguese sample

The same wrangling as last week. Now keeping age as well, and restricting to Portugal.

• Balance
• R code

	Before the attack (N=187)		After the attack (N=104)
	Mean	Std. Dev.	Mean	Std. Dev.	Diff. in Means	Std. Error
age	45.8	19.4	48.5	17.6	2.6	3.0
anti_immi	3.6	0.9	4.0	0.8	0.3	0.1

ESS %>%
  select(treat, age, anti_immi, pspwght) %>%
  rename(weights = pspwght) %>% # Used as survey weights
  datasummary_balance(~ treat, data = .,
                      output = "kableExtra")

Discuss: in Portugal, is the raw estimate too large or too small?

Careful: the direction is the opposite of the all-country pattern Legewie reports. Portugal is its own case.

Now the real data, where nobody knows the truth. Point out that the balance table is the empirical counterpart of \(\beta_{D \rightarrow C}\) from the formula — the imbalance term, measured. In Portugal the treatment group is older, which is the reverse of the pooled pattern, so the sign rule now predicts the adjustment will push the estimate down. Make them commit to a direction before the next slide.

The adjusted effect

ols_pt_bi   <- lm_robust(anti_immi ~ treat,
                         weights = pspwght, data = ESS)
ols_pt_mult <- lm_robust(anti_immi ~ treat + age,
                         weights = pspwght, data = ESS)

modelsummary(
  list("Bivariate" = ols_pt_bi, "Adjusted" = ols_pt_mult),
  coef_rename = c(
    "treatAfter the attack" = "After the attack",
    "age" = "Age"),
  stars = TRUE, gof_map = c("nobs", "r.squared"),
  output = "kableExtra"
)

\[\underbrace{0.330}_{\text{bivariate}} \;\approx\; \underbrace{0.313}_{\text{adjusted}} \;+\; (\underbrace{0.0064}_{C \rightarrow Y} \times \underbrace{2.62}_{D \rightarrow C})\]

	Bivariate	Adjusted
(Intercept)	3.621***	3.328***
	(0.082)	(0.175)
After the attack	0.330**	0.313**
	(0.116)	(0.118)
Age		0.006+
		(0.003)
Num.Obs.	291	291
R2	0.032	0.050
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

Adjusting for age moves the estimate by only -0.017 SD. Why so little? The bias is a product, and in Portugal age barely predicts xenophobia (0.0064 per year).

The payoff, and the honest one: the formula they verified in a toy simulation also works on real survey data — walk the arithmetic once more. But do not oversell the shift, because it is small, and that is the lesson. Ask them why before you reveal: the imbalance is real, 2.6 years, yet age turns out to barely move xenophobia in Portugal. A product with one near-zero factor is near zero. This is exactly the point from the formula slide, now confirmed on real data — and a useful corrective to the instinct that every control must be added.

The adjusted effect, drawn

• The plot
• … its R code

plotdata <- bind_rows( # Stack the two tidy tables on top of each other
  tidy(ols_pt_bi),     # Model 1: bivariate
  tidy(ols_pt_mult),   # Model 2: adjusted for age
  .id = "model") %>%   # ... and remember which row came from which model
  filter(term != "(Intercept)") %>%
  mutate(model = case_when(model == "1" ~ "Bivariate",
                           model == "2" ~ "Adjusted for age"),
         term = case_when(term == "treatAfter the attack" ~ "After the attack",
                          term == "age" ~ "Age"))

ggplot(data = plotdata, aes(y = estimate, x = term, color = model)) +
  geom_hline(yintercept = 0, color = "#901A1E", lty = "dashed") +
  geom_pointrange(aes(ymin = conf.low, ymax = conf.high),
                  position = position_dodge(width = 0.5)) + # Side by side
  coord_flip() +
  labs(x = "", y = "Difference in xenophobia (SD)", color = "") +
  theme_minimal(base_size = 15)

The table from the previous slide as a picture, which is how papers show it. Two new tools, both in the code tab: bind_rows() stacks the two tidy tables, and .id remembers which model each row came from; position_dodge() puts the two estimates side by side instead of on top of each other. They need both in exercise 1.

It’s real research

Source: Legewie (2013, p. 1210)

Legewie’s published models adjust for a whole battery of observed confounders: age, gender, education, employment, and more.

Alternative names for the same thing, all of which you will meet in papers: controlling for \(X\) · partialling out \(X\) · adjusted for \(X\) · conditional on \(X\).

Show them the published table so today’s move looks like normal scholarly practice rather than a classroom exercise. The vocabulary box is genuinely useful — all four phrases mean the identical operation, and papers switch between them without warning. Mention that they will produce a version of this table themselves in exercise 1.

Your turn: exercise 2

You become the simulator: set the confounder’s strength and sign yourself, predict the bias from the formula, then check whether OLS agrees.

Open exercise 2 in a new tab ↗

Today’s general lessons

1. A confounder you leave out of your model creates omitted variable bias: \(\tilde\beta = \beta + (\beta_{C \rightarrow Y} \times \beta_{D \rightarrow C})\). It is the same problem as selection bias, named after the missing variable.

2. Because the bias is a product of two effects, you can predict its sign before running anything. And it is zero if either effect is zero.

3. Holding \(C\) constant means comparing people with the same \(C\): the gap between two parallel lines. Frisch–Waugh shows how OLS does it: residualise \(Y\) on \(C\), residualise \(D\) on \(C\), and regress the leftovers.

4. Multiple OLS is still OLS: minimising squared residuals, with every coefficient adjusted for all the others.

5. It can repair imbalance in observed confounders. It can do nothing about unobserved ones, which is why a good design still beats a good regression.

Check yourself: today’s goals

• Write down the omitted variable bias formula, and say what makes the bias zero.
• Given a DAG and the signs of two arrows, say whether an unadjusted estimate is too large or too small.
• Say in one sentence what a multiple OLS coefficient means: “comparing people with the same …”.
• Explain in the three Frisch–Waugh steps what “controlling for age” does, and name one thing multiple OLS cannot fix.

Shaky on any of these? That is what this week’s Absalon quiz and the Friday exercise class are for.

Today’s important functions

• lm_robust(y ~ d + c, ...): multiple OLS — the coefficient on d holds c constant.
• modelr::add_residuals(model = ..., var = ...): keep a model’s residuals (the Frisch–Waugh workflow).
• modelsummary::datasummary_balance(~ group, data = ...): the balance table — your empirical read on \(\beta_{D \rightarrow C}\).
• modelsummary(list("Bivariate" = ..., "Adjusted" = ...)): put the unadjusted and adjusted models side by side.
• set.seed() + rnorm(): simulate data where you know the true effect.
• bind_rows(tidy(m1), tidy(m2), .id = "model") + position_dodge(): two models in one coefficient plot.

References

Frisch, R. and F. V. Waugh (1933). “Partial Time Regressions as Compared with Individual Trends”. In: Econometrica, pp. 387-401.

Legewie, J. (2013). “Terrorist Events and Attitudes toward Immigrants: A Natural Experiment”. In: American Journal of Sociology, pp. 1199-1245.