World Cup Predictions: Most models underestimate the chance of a tie.

Methodology Sports
  1. March Madness Bracketeering
  2. Applied Bracketeering: Modeling March Madness
  3. Bracketeering update: Mascot randomness is beating the pants off RPI after round 2
  4. Applied Bracketeering: So, who saw that final four coming?
  5. Applied bracketeering wrapup: Highly-rated team wins in shocking finale
  6. Applied Bracketeering: Does our model also work for the NCAA Women’s tournament?
  7. Applied Bracketeering, 2018 Edition: Do streaks matter?
  8. Applied Bracketeering, 2018: Streaky Clean
  9. Bracketeering Sweet 16 update: The Infallible Braculator agrees to never speak of this past weekend again
  10. Bracketeering Final Four update: Round of the Usual Suspects (and Loyola)
  11. Bracketeering Finale: Much ado about nothing or A tale of four regions
  12. What countries punch above their demographic weight at the World Cup (and can this be predictive)?
  13. World Cup Predictions: in a bonkers first round of games, even the best models get just over 50%
  14. World Cup Predictions: Most models underestimate the chance of a tie.
  15. World Cup Predictions: Knockout round madness
  16. World Cup Predictions: The final countdown
  17. World Cup predictions wrap-up: Vive le France!
  18. The Insufferable Braculator™ Strikes Again. Can your NCAA Women’s Tourney predictions beat it?
  19. The Insufferable Braculator models NCAA Women’s basketball, chapter 2: Concerning chalk

By: Patrick W. Zimmerman

Soccer has a lot of ties.  This is not a mystery.

So why do most models severely under-account for them?

Looking at the modeling experts at thefivethirtyeight, the composite betting line from oddsportal, and Principally Uncertain’s own model all treat ties as rare events, even though more than a quarter of World Cup games all time have ended in draws after 90 minutes.  Through games of Monday 6/25, this World Cup looks as if ties are a little down from their normal rate (probably related to the ridiculous number of late late late goals), but at 19.44% (vs. 25.7% all time, 26.6% in 2014), that’s not really way out of the normal with only 36 games played so far.

Ties are a thing.  Get used to it.

The question

Are we, collectively as soccer analysts, under-weighting the possibility of a tie? 

The short-short version

Oh, yeah we are.  Big time.

If one looks at only results after 90 minutes (to compare games on an even playing field), models based on head-to-head matchups produce predictions that are way below the historical rate for ties.  Even if you separate out knock-out games (where extra time and penalty wins technically count as 90min draws), it’s still way way low. 

Principally Uncertain’s model suspected this might be the case, so we doubled the weight on ties.  Nnnoppppeeee.  We’re still at around 50% of what should be the expected historical result.

Ties over time

Mouseover for details.

Here is just group games (1934 & 1938 did not have a group stage, 1950 had only group stages).

Mouseover for details.

So how are we doing compared to the smart people elsewhere in the world?  Not bad!  Much better than the FIFA rankings, with the betting line in the lead, FiveThirtyEight a tick behind, and us just behind that.  Remember, the FIFA rankings’ “tie” result is us eyeballing a 150pt difference in FIFA team rating and calling that a draw (which apparently was pretty close).

Model comparisons after 36 group games

Model scoreboard
Model Points Points % Correct results Correct % Ties predicted Tie %
Betting Markets 24⅓ 0.676 22 0.611 0 0.00%
The FiveThirtyEight 23⅔ 0.657 21 0.583 1 2.78%
Principally Uncertain 23 0.639 20 0.556 4 11.11%
FIFA rankings 20 0.556 16 0.444 7 19.44%

Checking your work: Comparing models to reality

This is endemic.  While it’s understandable that the betting line would tend to eschew ties, as it’s based on the collective wisdom of the betting public (and the public clearly expects draws less often than it should).  That makes sense.

FiveThirtyEight, though, has the same problem we do.  They’re actually even lower, only forecasting one single tie during the group stages as a max-probability result from any game (Uruguay v. Russia.  Which they got wrong). 

This brings up the issue of tournament aggregate patterns v individual game forecasting.  If one is producing results based on a game-by-game basis, a tie that is only a little less probable than one or another team winning isn’t counted, whereas over the course of a whole tournament, that result will happen a fair amount.  It’s just obscured at the single-game level.

Stage Game P? 538 Odds FIFA Actual result
Group A Russia v. Saudi Arabia RUS RUS RUS TIE RUS, 5-0
Group A Egypt v. Uruguay URU URU URU URU URU, 1-0
Group A Uruguay v. Saudi Arabia URU URU URU URU URU, 1-0
Group A Russia v. Egypt RUS RUS RUS EGY RUS, 3-1
Group A Uruguay v. Russia URU TIE URU URU URU, 3-0
Group A Saudi Arabia v. Egypt RSA EGY EGY EGY RSA, 2-1
Group B Portugal v. Spain TIE ESP ESP TIE TIE, 3-3
Group B Morocco v. Iran MAR MAR MAR TIE IRN, 1-0
Group B Portugal v. Morocco POR POR POR POR POR, 1-0
Group B Iran v. Spain ESP ESP ESP ESP ESP, 1-0
Group B Iran v. Portugal POR POR POR POR TIE, 1-1
Group B Spain v. Morocco ESP ESP ESP ESP TIE, 2-2
Group C France v. Australia FRA FRA FRA FRA FRA, 2-1
Group C Peru v. Denmark DEN DEN DEN TIE DEN, 1-0
Group C France v. Peru FRA FRA FRA TIE FRA, 1-0
Group C Denmark v. Australia DEN DEN DEN DEN TIE, 1-1
Group D Argentina v. Iceland ARG ARG ARG ARG TIE, 1-1
Group D Croatia v. Nigeria CRO CRO CRO CRO CRO, 1-0
Group D Argentina v. Croatia ARG ARG ARG ARG CRO, 3-0
Group D Nigeria v. Iceland ISL ISL ISL ISL NIG, 2-0
Group E Brazil v. Switzerland BRA BRA BRA BRA TIE, 1-1
Group E Costa Rica v. Serbia TIE SRB SRB TIE SRB, 1-0
Group E Brazil v. Costa Rica BRA BRA BRA BRA BRA, 2-0
Group E Serbia v. Switzerland TIE SUI SUI SUI SUI, 2-1
Group F Germany v. Mexico GER GER GER GER MEX, 1-0
Group F Sweden v. South Korea SWE SWE SWE SWE SWE, 1-0
Group F Germany v. Sweden GER GER GER GER GER, 2-1
Group F South Korea v. Mexico TIE MEX MEX MEX MEX, 2-1
Group G Belgium v. Panama BEL BEL BEL BEL BEL, 3-0
Group G Tunisia v. England ENG ENG ENG TIE ENG, 2-1
Group G Belgium v. Tunisia BEL BEL BEL BEL BEL, 5-2
Group G England v. Panama ENG ENG ENG ENG ENG, 6-1
Group H Colombia v. Japan COL COL COL COL JPN, 2-1
Group H Poland v. Senegal POL POL POL POL SEN, 2-1
Group H Poland v. Colombia POL COL COL POL COL, 3-0
Group H Japan v. Senegal SEN JPN SEN SEN TIE, 2-2

What’s next?

Soccer, coffee, and beer.  Probably not in that order. 

Also, our favorite mantra: observe and refine, test and tweak.  Keep working on this, because we honestly didn’t think that it would hold up so well this far (except against the FIFA rankings.  We saw that one coming.

About The Author

Architeuthis Rex, a man of (little) wealth and (questionable) taste. Historian and anthropologist interested in identity, regionalism / nationalism, mass culture, and the social and political contexts in which they exist. Earned Ph.D. in social and cultural History with a concentration in anthropology from Carnegie Mellon University and then (mostly) fled academia to write things that more than 10 other people will actually read. Driven to pursue a doctorate to try and answer the question, "Why do they all hate each other?" — still working on it. Plays beer-league hockey, softball, and soccer. Professional toddler wrangler. Likes dogs, good booze, food, and horribly awesome kung-fu movies.

No Comments on "World Cup Predictions: Most models underestimate the chance of a tie."

Leave a Comment