Polynomials & Transformations

Fitting curves with a linear model

Merlin Schaeffer · Department of Sociology

2026-07-28

By the end of today you can …

1. spot when OLS’s linearity assumption fails, and use LOESS to see the true shape;

2. fit and interpret a polynomial — a curve built from a straight-line model — and know when higher orders overfit;

3. transform a predictor with log₂ and read it as “per doubling” — then say which model fits best.

One part per goal. Our running question: does life expectancy simply rise in a straight line with health spending?

Frame the day as a repair job. Every model so far has assumed a straight line; today the data bends, and we learn two ways to fit a curve while still using ordinary OLS. Stress that neither trick leaves OLS behind — a polynomial and a log transform are both just linear regression on cleverly built predictors. That reassurance is the through-line.

When the line bends

Part 1 of 3

More health spending buys more life — but surely not forever, and not at a constant rate. Let’s look.

The research question of the day

Richer countries spend more on health and live longer. But is the relationship a straight line — each extra dollar worth the same everywhere?

Research question: how does life expectancy at birth relate to health spending per person — linearly, or as a bending curve?

Life expectancy over time. Source: Our World in Data.

Let the Our World in Data chart breathe — it is the motivating hook, and it already hints at the shape: fast early gains, then a plateau. Pose the question sharply: is each extra dollar of health spending worth the same in Nigeria as in Denmark? Everyone’s intuition says no, and that intuition is exactly the linearity assumption we are about to test.

Preparation

• Packages
• Get the data
• A panel, not a cross-section

pacman::p_load(
  tidyverse,    # Data manipulation and visualization
  estimatr,     # OLS with clustered / robust standard errors
  modelsummary, # Regression tables
  wbstats,      # World Bank data
  vdemdata,     # V-Dem (civil liberties)
  scales        # Log axis helpers
)

# World Bank: life expectancy + health spending per capita (PPP $), 2000-
life_raw <- wb_data(c("SP.DYN.LE00.IN", "SH.XPD.CHEX.PP.CD"),
                    start_date = 2000, end_date = 2025)

Dat_life <- life_raw %>%
  rename(LifeExpect = SP.DYN.LE00.IN, HealthExp = SH.XPD.CHEX.PP.CD,
         year = date, country_text_id = iso3c) %>%
  select(country_text_id, country, year, LifeExpect, HealthExp) %>%
  drop_na(LifeExpect, HealthExp)

# V-Dem: civil liberties, joined on country + year, as a control
Dat_vdem <- vdem %>%
  as_tibble() %>%
  select(country_text_id, year, civ_liberties = v2xcl_rol)

Dat <- inner_join(Dat_life, Dat_vdem, by = c("country_text_id", "year")) %>%
  drop_na()

This is panel data: many countries, each observed over many years — so the rows are not independent. Every model today uses clusters = country so the standard errors account for repeated observations of the same country.

Two things to flag and then move on. First, civil liberties rides along as a single control — the same V-Dem measure from Lectures 2, 9 and 11. Second, and more important: this is a panel, so observations repeat within countries and are not independent. clusters = country is not optional decoration; without it the standard errors are badly overconfident. Say that plainly and keep going.

A straight line through a curve

• The picture
• The model

The line misses the shape — linearity is violated.

# I() lets us rescale inside the formula: 1 unit = $1,000
ols_lin <- lm_robust(
  LifeExpect ~ I(HealthExp / 1000) + year + civ_liberties,
  data = Dat, clusters = country
)

modelsummary(
  list("Life expectancy" = ols_lin),
  coef_rename = c("I(HealthExp/1000)" = "Health spend ($1,000s)",
                  "year" = "Year", "civ_liberties" = "Civil liberties"),
  stars = TRUE, gof_map = c("nobs", "r.squared"),
  output = "kableExtra"
)

	Life expectancy
(Intercept)	−129.922*
	(55.077)
Health spend ($1,000s)	2.877***
	(0.408)
Year	0.095***
	(0.027)
Civil liberties	6.274**
	(2.079)
Num.Obs.	4070
R2	0.455
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

Show the picture first — the straight black line slicing through an obvious curve is more persuasive than any statistic. Name the tell-tale pattern: the line sits below the cloud in the middle and above it at both ends, which is exactly what a mis-specified linear fit looks like. Point at the labelled countries: the United States spends the most of anyone (far right) yet lives shorter than Switzerland or Norway — it sits well below the curve, a vivid reminder that beyond a point more money buys little life. Each faint coloured thread is one country tracked across years. The \(R^2\) of about 0.45 is not terrible, which is the trap — a mediocre linear fit can still look respectable in a table while being the wrong shape.

LOESS: let the data draw the curve

Before modelling a shape, see it. LOESS (locally estimated scatterplot smoothing) fits a little regression around each point using only its nearest neighbours, then stitches them into one smooth curve.

Beware: LOESS is descriptive — it wiggles to the data and easily overfits. Use it to spot the shape, not to test a hypothesis.

• The idea
• On our data

LOESS answers “what shape is this?” without committing to a formula. The mechanism is worth one sentence: a moving local regression, each point informed only by its neighbours, smoothed into a curve. Point at our fit — a steep early rise that flattens hard — that is the shape both of today’s tools will try to capture. And repeat the warning: LOESS is a describing tool, not a modelling one; it will happily trace noise.

Two ways to fit a curve — with a straight-line model

The shape is clear. Now we fit it — without leaving OLS. Two tricks: polynomials and transformations.

Set up the rest of the lecture as a choice between two tools that solve the same problem. The headline that reassures: both keep us inside ordinary least squares. A polynomial adds a squared predictor; a transformation reshapes a predictor. Neither is a new estimator — just a cleverer set of columns handed to the same lm.

Polynomials

Part 2 of 3

A polynomial is a variable interacted with itself — last week’s idea, pointed inward.

Remember interactions?

In Lecture 11, an interaction let one variable’s slope depend on another variable: \[Y = \alpha + \beta_1 X + \beta_2 Z + \beta_3 (X \times Z)\]

A polynomial is the same move with \(Z = X\) — a variable interacted with itself: \[Y = \alpha + \beta_1 X + \beta_2 (X \times X) = \alpha + \beta_1 X + \beta_2 X^2\]

So the health-spending slope now depends on the level of health spending itself — it can start steep and flatten out, exactly the shape LOESS showed.

Everything you learned about reading interaction terms carries straight over: the effect of \(X\) is no longer one number.

This is the conceptual bridge from last week, and it is genuinely clarifying — students who found interactions hard suddenly see polynomials as the easy case, because there is only one variable. Say it as a slogan: a polynomial is a variable interacted with itself, so its own slope depends on its own level. That is precisely why it can bend. No need to re-run last week’s models; just make the link.

The quadratic model

ols_poly <- lm_robust(
  LifeExpect ~ poly(I(HealthExp / 1000), 2, raw = TRUE) +
    year + civ_liberties,
  data = Dat, clusters = country
)

\[Y_i = \alpha + \beta_1 X + \beta_2 X^2 + \dots + e_i\]

poly(x, 2, raw = TRUE) adds both \(X\) and \(X^2\) in one clean term. raw = TRUE keeps the original units so the coefficients are interpretable.

One squared term, and the straight line becomes a curve that follows the bend.

Keep the code moment short — the point is that adding a curve costs exactly one extra term, poly(x, 2, raw = TRUE). Explain raw = TRUE once: without it R uses orthogonal polynomials whose coefficients are uninterpretable, and we want plain units. Then the picture: the red line now hugs the cloud. The interpretation of the two coefficients is the next slide, and it is the part they must get right.

Reading a quadratic

	Life expectancy
(Intercept)	−101.184*
	(46.208)
Health spend	7.472***
	(0.792)
Health spend²	−0.676***
	(0.129)
Year	0.081***
	(0.023)
Civil liberties	2.840
	(1.727)
Num.Obs.	4070
R2	0.588
+ p < 0.1, * p < 0.05, ** p < 0.01, *** p < 0.001

The slope of health spending is no longer one number — it changes as spending rises:

• \(\beta_1 = 7.47\) — near $0, each extra $1,000 adds about 7.5 years.
• \(\beta_2 = -0.68\) — but every $1,000 makes the next $1,000 worth less.

From $1,000 to $2,000: about \(7.47 + 2 \times (-0.68) = \mathbf{6.12}\) years. The gains flatten — diminishing returns, quantified.

The negative squared term is the whole story: it is what bends the line down. Walk the arithmetic slowly, because students reliably misread it — the effect of health spending is \(\beta_1 + 2\beta_2 X\), so it shrinks as \(X\) grows. Compute one concrete step, a poor country going from $1,000 to $2,000, and contrast it with what the same $1,000 buys a rich country: almost nothing. That is diminishing returns made numerical, and it is a genuinely sociological point about where a health dollar does the most good.

Higher orders — and the overfitting trap

• The temptation
• The warning

Beware overfitting. A high-degree polynomial bends to chase every wiggle — including noise. It fits this sample better and the next one worse. It also implies absurd swings (life expectancy shooting up or down at the extremes).

Rule of thumb: stop at the lowest degree that captures the shape. Here, a quadratic is plenty — degree 2, rarely more than 3.

The blue degree-9 curve is the cautionary tale — let them watch it thrash near the edges where data are thin, even swinging in directions no theory would predict. The lesson is not “more terms, better fit”; a high-degree polynomial fits this sample and generalises worse, the same overfitting idea as LOESS. Give them the discipline: choose the lowest degree that captures the shape you can defend, which is almost always two.

Break

Your turn: exercise 1

You fit a polynomial yourself — does xenophobia rise and then fall with age? Back to the ESS.

Open exercise 1 in a new tab ↗

Transformations

Part 3 of 3

A polynomial adds a term. A transformation reshapes the variable itself — and our favourite is the logarithm.

What is a logarithm?

A logarithm is just a power question, asked backwards.

Raising 2 to a power means repeated doubling — multiply 2 by itself that many times: \[2^{\color{#901A1E}{3}} = 2 \times 2 \times 2 = 8\]

The base-2 logarithm asks the reverse: “2 raised to which power gives this number?” — and the answer is exactly that exponent: \[\log_2(8) = \color{#901A1E}{3} \quad\text{because}\quad 2^{\color{#901A1E}{3}} = 8\]

exponent	power of 2	value	log₂(value)
1	\(2^1\)	2	1
2	\(2^2\)	4	2
3	\(2^3\)	8	3
4	\(2^4\)	16	4
10	\(2^{10}\)	1,024	10

Read a row left to right: multiply 2 by itself exponent times to get the value. log₂ hands that count back to you.

Assume they have genuinely forgotten what a logarithm is — most have, and that is fine. Anchor everything on one sentence: a logarithm is a power question asked backwards. Powers run forwards — 2 to the 3rd is 8 — and logs run back — the log base 2 of 8 is 3. Walk the table one row at a time, out loud: “two, times two, times two, is eight; so log base two of eight is three.” Do not move on until the exponent-in / value-out / count-back-out loop feels automatic. This is the slide the whole transformation idea rests on.

…so log₂ counts doublings

Because the exponent counts doublings, moving +1 on the log₂ scale = one doubling of the original value: \[\log_2(2) = 1, \quad \log_2(4) = 2, \quad \log_2(8) = 3\]

Perfect for health spending: $500 → $1,000 is one doubling, and so is $4,000 → $8,000. If each doubling buys the same extra years, a curve on the raw dollar scale becomes a straight line on the log₂ scale.

Now connect the backwards-power idea to why it helps us model. The exponent is literally a count of doublings, so a one-step move on the log scale always means “twice as much”, whether that is 500 to 1,000 or 4,000 to 8,000. The modelling payoff, in the green box: if life expectancy gains a fixed amount per doubling rather than per dollar, then against log-spending the curve becomes a straight line — and straight lines are exactly what OLS fits. Point at the plot: equal steps up the y-axis correspond to ever-bigger jumps in dollars.

The same fit, two ways to draw it

We fit life expectancy on log₂(spending). Drawn back on the normal dollar axis, that fit is a curve — rising fast, then flattening. Exactly the shape we needed.

Same model, same fit. The next slide shows it on a log₂ axis — where the very same curve becomes a perfectly straight line. Flip back and forth.

On the normal dollar axis, the log fit is a curve.

This is the first half of a back-and-forth pair — flip between this slide and the next to make the point land. Here the axis is ordinary dollars, and the log fit is a curve that hugs the data: steep among poor countries, flat among rich ones. On the next slide the axis becomes log₂ and that identical fit straightens into a line. Same model both times; only the ruler on the x-axis changed. That is the whole idea of a transformation.

The log₂ model

ols_log <- lm_robust(
  LifeExpect ~ log2(HealthExp) + year + civ_liberties,
  data = Dat, clusters = country
)

modelsummary(
  list("Life expectancy" = ols_log),
  coef_rename = c("log2(HealthExp)" = "Health spend (log2)",
                  "year" = "Year",
                  "civ_liberties" = "Civil liberties"),
  stars = TRUE, gof_map = c("nobs", "r.squared"),
  output = "kableExtra"
)

One coefficient, and it reads beautifully: every doubling of health spending is associated with about 3.63 more years of life expectancy.

On a log₂ axis, that same curve is a straight line.

The payoff of the transform is interpretability: one number, and it means “years gained per doubling of spending” — about 3.6 here. Contrast that with the quadratic, which needed two coefficients and arithmetic to interpret. Show the log-scaled x-axis: the same data that curved is now a clean straight line, which is the visual proof that the log captured the shape. This “per doubling” reading is exactly how health economists talk, so it is worth drilling.

Which fits better — polynomial or log₂?

• Fit
• R code

Model	R²	Adj. R²
Linear	0.455	0.455
Quadratic	0.588	0.588
log₂	0.710	0.710

# Pull R² and adjusted R² from each fitted model
gof <- function(m) c(R2 = m$r.squared, adj = m$adj.r.squared)

rbind(
  Linear    = gof(ols_lin),
  Quadratic = gof(ols_poly),
  log2      = gof(ols_log)
) %>% round(3)

Both curves beat the straight line — but here the log₂ model wins on fit and is simpler to read (one coefficient, not two).

Discuss: the log needs one coefficient, the quadratic two. When might you still prefer the polynomial — for example, if the curve went up and then down?

Read the fit statistics across the three models — linear worst, quadratic better, log best here. Then resist the temptation to declare a universal winner. The log wins on this shape because the relationship is monotone and flattening. The blue box is the important nuance: a log can only bend one way, so if a relationship rises and then falls — a genuine hump, like age and earnings — a quadratic is the right tool and the log cannot do it. Match the tool to the shape you expect.

One more possibility, for later

Polynomials and transformations keep a linear model and bend the predictor.

A third route bends the link between predictors and outcome — generalised linear models (logistic, Poisson …), for binary or count outcomes.

You will meet GLMs later in your studies. Today’s lesson is the foundation: even a “linear” model can fit deeply non-linear relationships once you reshape what goes into it.

A brief signpost, not a lesson — no need to teach GLMs. The point is to place today’s tools in a bigger map: polynomials and logs bend the predictor while keeping a linear link; GLMs bend the link itself for outcomes that are binary or counts. Reassure them that everything today transfers. Then move to the summary.

Today’s general lessons

1. When a scatter bends, the linearity assumption fails. LOESS reveals the shape — but only describes it, so beware overfitting.

2. A polynomial (\(X + X^2\)) is a variable interacted with itself: its slope depends on its own level, so it can curve. Read \(\beta_1 + 2\beta_2 X\); watch for overfitting at high degrees.

3. A log₂ transformation turns “per dollar” into “per doubling” — one interpretable coefficient, and it straightens a flattening curve.

4. Match the tool to the shape: a log bends one way (monotone, flattening); a quadratic can rise then fall. Compare fits, but let theory choose.

5. It is still OLS — we only reshaped the predictors. That is what makes a “linear” model so powerful.

Run through quickly. If one idea survives, make it point five: a linear model is far more flexible than its name suggests, because linearity is a claim about the coefficients, not about the shape of the curve. Everything today was ordinary OLS fed a cleverer set of columns.

Check yourself: today’s goals

• Look at a scatter and say whether a straight line is defensible — and add a LOESS to check.
• Fit a quadratic, and interpret both coefficients as a slope that changes with the predictor’s level.
• Fit a log₂ model and read the coefficient as “per doubling”; say when you’d use a polynomial instead.

Shaky on any of these? That is what this week’s Absalon quiz and the Friday exercise class are for.

Today’s important functions

• lm_robust(y ~ ..., clusters = country): OLS with cluster-robust SEs for panel data.
• I(x / 1000) and I(x^2): build a rescaled or squared predictor inside the formula.
• poly(x, 2, raw = TRUE): add \(X\) and \(X^2\) in one term, in original units.
• log2(x): base-2 log — a one-unit rise is a doubling.
• geom_smooth(method = "lm", formula = y ~ poly(x, 2)) / y ~ log2(x): draw the fitted curve.
• scale_x_continuous(trans = scales::log2_trans()): plot on a log₂ axis.

References

NULL