MODEL VALIDATION · 2018 AND 2022

Would MP-26 have got it right?

MP-26 was replayed on the last two midterms as it runs now: each race's own starting margin (presidential lean, incumbency and the incumbent's personal vote where it is known) with the national environment set to the generic-ballot average on the eve of each election, through the same 50,000-run simulation. No race polls go in: the repository has none for those years. Then it was compared with what actually happened and with FiveThirtyEight's final forecasts, which used every poll.

4/4chamber results inside the 80% interval940 races scored

Races called right95%79% in the closest races

Brier score (lower is better)0.036FiveThirtyEight's own odds: 0.030

Typical race miss±6.1points of margin, root mean square

National missR+1.2 / R+0.9average miss in 2018 / 2022: the error every race shares

CHAMBERS

Seat intervals against the result

MP-26 80% interval FiveThirtyEight 80% median actual

2018

HouseD majority 73% · actual 235 D seats

218235

Inside the 80% intervalFavourite won

SenateD majority 2% · actual 47 D seats

5147

Inside the 80% intervalFavourite won

2022

HouseD majority 32% · actual 213 D seats

218213

Inside the 80% intervalFavourite won

SenateD majority 51% · actual 51 D seats

5051

Inside the 80% intervalFavourite won

YEAR BY YEAR

Each chamber, scored

	Races	Called right	Brier MP-26	Brier 538	Close races (MP-26 / 538)	Seats: MP-26 (80%)	538	Actual
2018 · House	435	95%	0.039	0.027	0.148 / 0.108	225 (211–242)	233	235
2018 · Senate	35	91%	0.085	0.056	0.226 / 0.151	47 (44–49)	—	47
2022 · House	435	95%	0.030	0.032	0.135 / 0.141	213 (200–225)	204	213
2022 · Senate	35	97%	0.026	0.035	0.084 / 0.110	50 (47–52)	49	51

Brier: the squared gap between each probability and what happened, averaged (lower is better). Close races: those FiveThirtyEight had between 5% and 95%. Seats: Democratic seats, with the 80% interval.

CALIBRATION

Do 30% races happen 30% of the time?

Each dot groups the competitive races (5–95%) by forecast. Dots below the diagonal on the left and above it on the right mean the engine was too cautious: favourites won more often than it said.

RACE BY RACE

Expected margin vs. result

Each dot is a contested race: the final expected margin (across) against the actual one (up). Gold rings mark races the favourite lost. 2018 2022

WHAT THIS REPLAY COVERS

Tested here, and not

Replayed

The starting-margin formula: presidential lean, incumbency and the incumbent's personal vote
The national environment, taken from the generic-ballot average on the eve
The ±7.5-point race error, the shared national error and the 50,000-run simulation
Every race's probability and the seat totals

Not replayed

Race polls and how they blend in: the repository has none for 2018 or 2022
The national environment read from the Senate polls
The partisan-poll corrections and the 14-day recency weighting

In sample: the coefficients are fitted on these midterms (and 2014), so the scores flatter the model. Fitted without 2018, the incumbent's personal vote scored worse on 2018's House than the plain formula (Brier 0.044 against 0.035; 223 seats against 236, actual 235).

WHAT WAS LEARNED

Findings and limits

Chambers: 4 of 4 inside the interval. The favourite won 4 of 4 chambers. In 2018 the House median fell short of the Democratic gain (225 against 235 seats).
Races: behind FiveThirtyEight, without polls. Race-level Brier 0.036 against 0.030 for FiveThirtyEight's own odds, which used every poll; this replay uses none. Of the per-race errors tried, ±7.5 points, the one the live model uses, scores best.

Log loss at ±7.5: 0.129, the best of the errors tried. Inputs: MP-26's own baseline with the election-eve generic ballot (FiveThirtyEight's average), no race polls. Comparison: FiveThirtyEight final 2018/2022 forecasts (classic), CC BY 4.0, via the Internet Archive. Outcomes: FiveThirtyEight election-results (CC BY 4.0).

How the model works →Sources and methodology →