CMH MATH PATH — LESSON 4: LINEAR REGRESSION

Finding the Line That Predicts the Future

PART 1: THE CORE IDEA — DRAWING THE “BEST GUESS” LINE

The Grade Prediction Problem

Imagine you notice a pattern in your class:

  • Students who study 5 hours for a test score around 70%
  • Students who study 10 hours score around 80%
  • Students who study 15 hours score around 90%

The Question: If a new student studies 12 hours, what score should we predict?

The Answer: Draw a straight line through the existing dots, then read off the prediction!

This is linear regression — finding the straight line that best fits your data.

PART 2: VISUALIZING REGRESSION — THE “LINE OF BEST FIT”

The Scatter Plot with a Magic Line

Let’s plot “Study Hours” vs. “Test Score”:

Test Score (%)
   ↑
100 |                     •
    |                 •
 90 |             •
    |         •
 80 |     •
    | •
 70 |
    |
    +---------------------→ Study Hours
     0   5   10   15   20

Now draw the best straight line through these dots:

Test Score (%)
   ↑
100 |                     •
    |                 •  /
 90 |             •  /
    |         •  /
 80 |     •  /
    | •  /
 70 |/
    |
    +---------------------→ Study Hours
     0   5   10   15   20

This line is our prediction machine!

How to Use the Prediction Line:

  1. Find 12 hours on the bottom (X-axis)
  2. Go straight up to hit the line
  3. Go straight left to read the score: About 86%!

The Equation of This Magic Line:

Predicted Score = 64 + (1.8 × Study Hours)

Let’s check:

  • 5 hours: 64 + (1.8 × 5) = 64 + 9 = 73% (close to 70!)
  • 10 hours: 64 + (1.8 × 10) = 64 + 18 = 82% (close to 80!)
  • 15 hours: 64 + (1.8 × 15) = 64 + 27 = 91% (close to 90!)
  • 12 hours: 64 + (1.8 × 12) = 64 + 21.6 = 85.6% (our prediction!)

PART 3: THE TWO MAGIC NUMBERS IN EVERY REGRESSION

Every straight line has just two numbers that define it:

1. The Intercept (b₀ “b-zero” or “b-naught”)

  • Where the line starts on the left (when X = 0)
  • In our example: 64%
  • Meaning: “With ZERO study hours, we’d predict a 64% score”
  • (This might be base knowledge or guessing!)

2. The Slope (b₁ “b-one”)

  • How steep the line is
  • In our example: 1.8
  • Meaning: “For each extra hour studied, score increases by 1.8%”

The Universal Formula:

ŷ = b₀ + b₁x
(Say: “y-hat equals b-zero plus b-one times x”)

Where:

  • ŷ (y-hat) = Predicted Y value
  • x = Actual X value we know
  • b₀ = Intercept (starting point)
  • b₁ = Slope (rate of change)

PART 4: CMH CASE STUDY — EDUCATION VS. INCOME

The Real-World Relationship

Let’s use real(ish) data:

  • High school diploma: Average income = $35,000
  • Bachelor’s degree: Average income = $55,000
  • Master’s degree: Average income = $70,000

Plot it:

Income ($000)
   ↑
70 |                 •
    |             •
60 |         •
    |     •
50 | •
    |
40 |
30 |
    +---------------------→ Education (Years)
    12   14   16   18

Draw the best line:

Income=−50+(7.5×EducationYears)Income = -50 + (7.5 × Education Years)

Check:

  • 12 years (high school): -50 + (7.5 × 12) = -50 + 90 = $40K (close!)
  • 16 years (college): -50 + (7.5 × 16) = -50 + 120 = $70K (close!)
  • 18 years (master’s): -50 + (7.5 × 18) = -50 + 135 = $85K (a bit high, but close!)

CMH Insight: Each additional year of education predicts $7,500 more income!

PART 5: HOW COMPUTERS FIND THE “BEST” LINE (THE SIMPLE VERSION)

The “Least Squares” Method

Computers don’t guess the line — they calculate it by minimizing errors.

What’s an error? The vertical distance from each dot to the line:

Test Score
   ↑
   |                     •
   |                 •  /
   |               ↗    (This vertical distance
   |             •      is the "error" for this dot)
   |         •  /
   |     •  /
   | •  /
   |/
   +---------------------→ Study Hours

The Goal: Find the line that makes the sum of squared errors as small as possible.

Why squared errors?

  1. Makes all errors positive (so they don’t cancel)
  2. Punishes big errors more than small ones (squaring 10 = 100, squaring 2 = 4)

The Math (Simplified):

b1=Correlation×(SDofY/SDofX)b₁ = Correlation × (SD of Y / SD of X)
b0=AverageofY−b1×(AverageofX)b₀ = Average of Y – b₁ × (Average of X)

Don’t memorize — just know computers do this instantly!

PART 6: HANDS-ON REGRESSION LAB

Exercise 1: Manual Best-Fit Line

Data:

Study Hours (X):  5,  7,  9, 11, 13
Test Scores (Y): 60, 70, 75, 85, 90

Step 1: Plot dots on graph paper
Step 2: Try drawing 3 different lines
Step 3: For each line, calculate errors:

  • Error = Actual Y – Predicted Y (from line)
  • Square each error
  • Sum all squared errors

Step 4: Which line has smallest sum? That’s closest to “best fit”!

Exercise 2: Spreadsheet Regression

In Google Sheets/Excel:

Column A (X): 5, 7, 9, 11, 13
Column B (Y): 60, 70, 75, 85, 90

Method 1: Scatter plot + Trendline

  1. Insert → Chart → Scatter
  2. Double-click chart → Customize → Series
  3. Check “Trendline”
  4. Show equation (magic appears!)

Method 2: LINEST function

=LINEST(B1:B5, A1:A5, TRUE, TRUE)

This gives slope and intercept directly!

Your result should be: ŷ = 40 + 4x

  • Slope = 4: Each study hour adds 4 points
  • Intercept = 40: Zero hours → 40% (base knowledge/guessing)

Exercise 3: Make Predictions & Calculate Error

Using ŷ = 40 + 4x:

Hours (X)Actual ScorePredicted (40+4X)ErrorError²
56040+20=6000
77040+28=6824
97540+36=76-11
118540+44=8411
139040+52=92-24

Sum of Squared Errors = 10 (This is as low as possible for straight line!)

Predict for new student:

  • Studies 10 hours: ŷ = 40 + 4×10 = 80%
  • Studies 20 hours: ŷ = 40 + 4×20 = 120% ← Impossible! Shows extrapolation danger!

PART 7: REGRESSION IN CMH — BEYOND SIMPLE LINES

Multiple Regression: Adding More Predictors

Real CMH models rarely use just one variable. We add multiple X’s:

Simple: ŷ = b₀ + b₁×(Education)
Better: ŷ = b₀ + b₁×(Education) + b₂×(Wealth) + b₃×(Parent Education) + …

Example: Predicting Protest Size

Expected Protesters = 1000 + 50×(Unemployment%) + 30×(Youth Bulge%) - 20×(Trust in Gov) + ...

Each b tells us: “Holding everything else constant, how much does changing this one variable affect the outcome?”

Key Insight: Multiple regression helps isolate direct effects while controlling for other factors!

The “Regression Coefficient” Interpretation

Remember: Correlation tells us “they move together”
Regression tells us “how much” one moves when the other changes

If: Income = -50 + 7.5×(Education)
Then: Each additional year of education predicts $7,500 higher annual income

Important: This is still predictive, not necessarily causal! (Remember Lesson 3!)

PART 8: LIMITATIONS & DANGERS

1. The Extrapolation Trap

  • Safe: Predicting within your data range (educated 12-18 years → income $35-85K)
  • Dangerous: Predicting outside range (educated 30 years → income $175K? Probably not!)

CMH Example: Predicting revolution risk for regimes more oppressive than any in your data.

2. The Linearity Assumption

Regression assumes a straight line relationship. What if it’s curved?

Example: Anxiety vs. Performance

  • Some anxiety → Better performance (focused!)
  • Too much anxiety → Worse performance (panicked!)
  • This is an inverted U shape, not a straight line!

Solution: Add squared terms: ŷ = b₀ + b₁x + b₂x² (makes curve!)

3. The Outlier Problem

One weird dot can drag the whole line:

   • • • • • • • • • •
                     \
                      \
                       • (This ONE dot pulls the line way off!)

Solution: Check residuals (errors)! Big residual = potential outlier.

4. The “All Linear Relationships” Fallacy

Not everything is straight!

  • Diminishing returns: First $1,000 reduces poverty a lot, millionth $1,000 does little
  • Threshold effects: Below 5% unemployment → stable, above 15% → revolt
  • Network effects: First 100 users worthless, millionth user priceless

PART 9: YOUR CMH REGRESSION FIELD KIT

Quick Decision Guide

When to Use Linear Regression:

  • Predicting a number (not yes/no)
  • Relationship looks roughly straight on scatter plot
  • You want to know “how much” not just “if”

When NOT to Use Linear Regression:

  • Predicting categories (war/peace, win/lose) → Use logistic regression (next lesson!)
  • Curved relationships → Add polynomial terms
  • Extreme outliers present → Clean data or use robust regression

The Regression Report Card

Every regression should report:

  1. Equation: ŷ = b₀ + b₁x
  2. R-squared: How much variation is explained (0-1, like correlation²)
  3. P-values: Is the relationship real or just chance? (p < 0.05 = probably real)
  4. Prediction Examples: Show what model predicts for realistic inputs

PART 10: REAL CMH REGRESSION EXAMPLE

Predicting State Stability (Simplified)

Data from 100 countries:

  • X₁ = Youth Unemployment (%)
  • X₂ = Gini Coefficient (inequality, 0-1)
  • X₃ = Trust in Institutions (1-10 scale)
  • Y = Social Unrest Index (0-100)

Regression Result:

Unrest=20+2.5×(YouthUnemp)+40×(Gini)−3×(Trust)Unrest = 20 + 2.5×(YouthUnemp) + 40×(Gini) – 3×(Trust)

Interpretation:

  1. Youth Unemployment: Each 1% increase → +2.5 points unrest
  2. Inequality: Each 0.1 Gini increase → +4 points unrest (big effect!)
  3. Trust: Each 1-point increase → -3 points unrest

Make a prediction:
Country A: 25% youth unemp, Gini=0.35, Trust=6
Unrest = 20 + 2.5×25 + 40×0.35 – 3×6
= 20 + 62.5 + 14 – 18 = 78.5 (High risk!)

Policy test: What if we boost trust from 6 to 7?
Unrest = 20 + 62.5 + 14 – 21 = 75.5 (Small improvement)

YOUR MATH HOMEWORK

Exercise 1: Build a Mini-CMH Model

Find data online (World Bank, UN) or make reasonable numbers:

  • X = “% Urban Population” for 10 countries
  • Y = “Carbon Emissions per capita” for same countries
  1. Plot scatter plot
  2. Add trendline, get equation
  3. Interpret: “For each 1% increase in urbanization…”
  4. Predict for a country with 80% urban population
  5. Calculate R²: What % of emissions variation does urbanization explain?

Exercise 2: The “Happiness vs. GDP” Puzzle

Common finding: ŷ(Happiness) = 4 + 0.3×log(GDP)
Notice: log(GDP), not GDP! (Because money has diminishing returns for happiness)

  1. Why use log? What does this mean in human terms?
  2. Calculate: Going from $10K to $20K GDP/capita → Happiness increases how much?
  3. Going from $100K to $200K → Happiness increases how much?
  4. Insight: Why is economic growth less urgent in rich countries?

Exercise 3: Spot Regression Abuse

Find a news article claiming: “Study shows [X] causes [Y]” with regression.

Analyze:

  1. What’s the regression equation (if given)?
  2. What third variables might be missing?
  3. Could causation be reversed?
  4. Is prediction outside data range (extrapolation)?

Exercise 4: Design a CMH Intervention

Using our education-income regression: Income = -50 + 7.5×Education

You have $1 million to boost incomes in a community.
Option A: Give 100 people $10,000 each (direct cash)
Option B: Fund extra education (costs $10,000/person for 2 more years)

Calculate:

  1. Option A: Direct $10K boost × 100 people = $1M immediate
  2. Option B: 2 years education × $7,500/year boost = $15K/year extra income
    Over 40 working years: $15K × 40 = $600,000 lifetime boost × 100 people = $60M total!

Question: Which is better? What assumptions is this making? What might regression be missing?

KEY TAKEAWAYS:

  1. Regression finds the best straight line through data points
  2. ŷ = b₀ + b₁x — Intercept (start) + Slope (rate)
  3. Use for prediction: “If X changes by 1, Y changes by b₁”
  4. Multiple regression adds more predictors (X₁, X₂, X₃…)
  5. Danger zones: Extrapolation, outliers, non-linear relationships
  6. CMH power: Quantify “how much” not just “if”

The CMH Mantra:

“Correlation tells us if two things dance together.
Regression tells us the dance steps:
For every step X takes, how many steps does Y take?
But remember — just because they step together doesn’t mean one leads.
And not all dances are simple two-steps; some are tangos with many partners.”

NEXT LESSON PREVIEW:

Lesson 5: Logistic Regression — Predicting Yes/No Events

Where we’ll learn:

  • How to predict probabilities (40% chance of revolution) not just numbers
  • The S-curve that goes from 0% to 100%
  • Why this is essential for CMH (revolutions, wars, elections are yes/no!)
  • How to interpret “odds ratios” in human terms

Thought to Ponder: Linear regression predicts things like “GDP will be $53,247 next year” — overly precise and often wrong. Logistic regression predicts “60% chance GDP will grow” — humble, probabilistic, and more useful for decision-making. In CMH, we’re often predicting risks, not certainties…

Remember: In CMH, we’re not just describing the world — we’re building prediction machines. Linear regression is your first real prediction tool. Use it wisely, understand its limits, and always ask: “What’s missing from this equation?” The most important variable is often the one you didn’t think to measure.

Similar Posts