By: Richard W. Sharp

The question

So, after a very chalk first round and a bonkers-insane second round, how are our models doing vs. the NCAA’s officially-sanctioned rankings (now powered by magical numbers!)?

Quite good!

Early returns

• RPI is as terrible as we thought it was.
• The mascot noise models are doing substantially better in terms of peak performance (remember, we added noise to maximize our chances to win pools, not just to do respectably on average).

Here’s how the brackets we highlighted before the tourney are doing after 2 rounds of games. Remember, we’re using a simple scoring system, where each 1st round game is worth 1 point, each 2nd round game 2, 3rd round game 4, etc.

Stand-alone brackets

Here are several single brackets (as opposed to models run many times and averaged) in order of decreasing score. An upset is any game in which the team with the better seed loses. 

1. Seeds (although RPI is officially used “to aid in the selecting and seeding” process, apparently cooler heads prevailed)
   • Correct: 38
   • Points: 50
   • Upsets called: None, by definition
2. Sagarin with mascot weighting #14, (bracket.ranker_SAG.mascot_14.csv – this is the one that really counts. We entered it in an actual pool)
   • Correct: 37
   • Points: 47
   • Upsets called: 1
3. RPI
   • Correct: 35
   • Points: 46
   • Upsets called: 0
4. Obama’s bracket
   • Correct: 33
   • Points: 43
   • Upsets called: 3

Randomized bracket sets

The brackets we posted on GIT were created by adding either random noise or mascot-weighted noise to some standard model. For each set of brackets listed below, we report the performance of the brackets at or above the 90^th percentile in each group. If we play random brackets from these strategies year after year, we should expect to generate one this good in every ten entries.

1. Mascot-weighted composite
   • 90^th pct. Correct: 39
   • 90^th pct. Points: 51
   • Most common Sweet Sixteen at 90^th pct. 
     • East
       • Villanova (1) vs. Florida (4) | Duke (2) vs. Baylor (3)
     • Midwest
       • Kansas (1) vs. Purdue (4) | Louisville (2) vs. Oregon (3)
     • South
       • North Carolina (1) vs. Butler (4) | Kentucky (2) vs. UCLA (3)
     • West
       • Gonzaga (1) vs. Notre Dame (5) | Arizona (2) vs. Maryland (6)
2. Mascot-weighted Sagarin
   • 90^th pct. Correct: 38
   • 90^th pct. Points: 49
   • Most common Sweet Sixteen at 90^th pct.
     • East
       • Villanova (1) vs. Florida (4) | Duke (2) vs. Baylor (3)
     • Midwest
       • Kansas (1) vs. Iowa St. (5) | Louisville (2) vs. Oregon (3)
     • South
       • North Carolina (1) vs. Butler (4) | Kentucky (2) vs. UCLA (3)
     • West
       • Gonzaga (1) vs. West Virginia (4) | Arizona (2) vs. Florida St. (3)
3. Sagarin with standard deviation 5 noise
   • 90^th pct. Correct: 38
   • 90^th pct. Points: 49
   • Most common Sweet Sixteen at 90^th pct.
     • East
       • Villanova (1) vs. Florida (4) | Duke (2) vs. Baylor (3)
     • Midwest
       • Kansas (1) vs. Purdue (4) | Louisville (2) vs. Oregon (3)
     • South
       • North Carolina (1) vs. Butler (4) | Kentucky (2) vs. UCLA (3)
     • West
       • Gonzaga (1) vs. West Virginia (4) | Arizona (2) vs. Florida St. (3)
4. RPI with standard deviation 5 random noise
   • 90^th pct. Correct: 36
   • 90^th pct. Points: 48
   • Most common Sweet Sixteen at 90^th pct.
     • East
       • Villanova (1) vs. Florida (4) | Duke (2) vs. Baylor (3)
     • Midwest
       • Kansas (1) vs. Purdue (4) | Louisville (2) vs. Oregon (3)
     • South
       • North Carolina (1) vs. Butler (4) | Kentucky (2) vs. UCLA (3)
     • West
       • Gonzaga (1) vs. West Virginia (4) | Arizona (2) vs. Florida St. (3)
5. Model composite with standard deviation 5 random noise
   • 90^th pct. Correct: 36
   • 90^th pct. Points: 46
   • Most common Sweet Sixteen at 90^th pct.
     • East
       • Villanova (1) vs. Florida (4) | Duke (2) vs. Baylor(3)
     • Midwest
       • Kansas (1) vs. Purdue (4) | Louisville (2) vs. Oregon (3)
     • South
       • North Carolina (1) vs. Butler (4) | Kentucky (2) vs. UCLA (3)
     • West
       • Gonzaga (1) vs. West Virginia (4) | Arizona (2) vs. Florida St. (3)

Coming up

We all take a deep breath, re-introduce ourselves to our families and pets, and then prep for the Sweet 16!

We’ll post another update after the elite 8 games are finished on March 28th.