March Madness statistical analysis does NOT guaranty good pool performance

After all of the statistical analysis from 24 years of the NCAA Basketball tournament why is my pool doing so poorly? The pool type favors upsets, the analysis says to pick upsets, but it doesn't say which to pick. To win the bracket must contain the correct upsets. Mine, however, does not.

Potential reasons for this:

• I don't know anything about college basketball. I admit to this as a first principle
• My technique picked every upset beyond a certain threshold of difference in the Sagarin ratings. This is probably too aggressive and resulted in two 6 seeds in the Final Four, which I should have corrected as this is too unlikely.
• The average number of upsets is different from the fluctuations in upsets from year to year. Picking the average number is picking a certain outcome from the many different outcomes over the years. Average value is different from most likely value in a histogram.
  

For fluctuations vs. average it is instructive to look at the outcome of the round of 64. Over 24 years of tournaments the graph below (reproduced from an earlier post) shows the fraction of times a given matchup resulted in either the expected outcome or an upset where the lower seed wins.
As an example, 55 out of 96 historical matchups between 8 and 9 seed teams result in upsets where the 9 seed wins. That is more than half the time. That does not mean that every year there are two upsets, it means that on average over all the years roughly half of the outcomes are upsets. This has implications for what we can expect each year.

Each year in the round of 64 a given matchup occurs 4 times, one for reach region. A different presentation of the round of 64 data from the past 24 years shows that for each given matchup different years have different numbers of upsets. The possible range is from no upsets to four upsets. The chart tallies the number of years with each particular combination of upsets for each matchup. Comparing this variability data to the outcome chart shows that while more than half the time 9 seeds beat 8 seeds, in any give year every possibility has occurred. In fact only 9 years in 24 opportunities (~38%) have there been exactly two upsets. It is the most common value, but still more likely to be wrong than right.

I am not sure how to present a similar analysis for the later rounds, since the number of opportunities is determined by the outcome of the round before. It is just as important in those rounds to realize that the average outcome over 24 years is not the same as the most likely outcome from those 24 years.

Perhaps a more systematic process that seeks to maximize the number of points even in the face of these uncertainties is needed.

NCAA March Madness bracket submission

A commenter asked to see the bracket produced after all of the machinations and analysis of the statistics of the past history of the NCAA March Madness basketball tournament. Here is is. Click for larger.


I am already in last place in my pool after 16 games. This allows the illustration of an important point. There will always be upsets in the NCAA tournament, the key to winning this particular office pool is to pick the correct upsets, and sometimes to be the only one that picked a particular correct upset. More games await.

Picking the 2009 March Madness Basketball Brackets with Statistics

I don't know anything about college basketball (unlike President Obama). I don't mean that I don't know the main rules, or how many players there are, or what's allowed and not allowed. I mean that I don't know what teams are good or bad, which players are destined for the NBA, which coaches are the best or who won what game in 1985. I am not even sure of what teams are in what division so I always check. I suspect that this ignorance is exactly the correct approach to take when picking the game winners in the NCAA Division I playoffs, March Madness.

Instead of learning the teams and the players, I have explored the statistics of the past 24 years (data can be found here) (Last years picks, Round of 64 and 32 upsets, Final Four and Championship probabilities) combined with other's specialized knowledge like the Sagarin ratings. This year I am updating some of my charts to include data from 2006, 2007 and 2008.

The pool I enter favors upsets. The points for each round are the round multiplier times the seed of the winning team that you picked. Thus if a 10 seed wins Round 2 and you pick it you get 2*10 for points. To win this type of pool it is imperative that you pick upsets. Game results for 24 years of Round 1 are shown graphically below.

Some quick points for Round 1, the round of 64:

• No 16 seed team has ever won in the first round. Don't be the first to pick one.
• 15 seeds are also very safe and normally win their games.
• History shows that there will be at least one, and in some years two upsets favoring a 10, 11, 12 seed.
• One could make the case for one upset a year favoring a 13 and 14 seed as well.
• 9 seeds win against 8 seeds more than half of the time. Pick two upsets.

Even in Round 2 with 32 teams, upset picking is important as well. 24 years of Round of 32 matchups are shown below with expected and upset outcomes tabulated.

Because this round depends on the outcome of the first round the number of opportunities is different for each matchup. In the most extreme case, no 16 seed team has ever beaten a 1 seed to advance to this round, so there is no data for that matchup. Only once has a 15 seed beaten 2 seed and then played a 7 seed, thus there is only one occurrence on the chart.

Some lessons from the Round of 32 chart:

• 1 seed teams typically win in this round as well, rarely being beaten by 8 or 9 seeds.
• Matchups with 5 vs 4 seeds, 6 vs 3 seeds and even 10 vs 2 seeds and 12 vs 4 seeds (surprisingly) seem to be toss-ups over the 24 years of data. Almost half the time there is an upset and the lower seed wins. If you have them in your bracket pick the correct underdog half of the time.
• Matchups with 7 vs 2 seeds do result in upsets about a third of the time. Look for opportunities to pick one.

The results of this chart show what teams advanced to the Sweet Sixteen Round and should help to determine which upsets to pick according to past history.

Below is a matchup outcome chart for the Sweet Sixteen round which is similar to the earlier charts, but much more complicated.
As each round progresses there are more combinations of possible matchups, though most of them have never actually occurred in history of the tournament in its modern form. No 16 seed has ever advanced so those matchups are not represented. 15 seeds rarely advance, so many of those matchups also have no data.

Some lessons gleaned from the Round of 16 outcome chart:

• 1 seeds usually win. They always beat 12 seeds that make it through.
• The closer the distance between seeds the more the outcome is a tossup. This is true for all of these charts.
• In the three times that 11 seeds have made it to this round they have beaten the 7 seed they played. Whether that is statistically significant or not is the question.

On the other side of the range, rather than add combinatorial complexity, it is easier to compile the results of past years for the late rounds to see how likely it is that certain seeds reach the Sweet Sixteen, Elite Eight, Final Four, The Championship Game and finally win the championship. These frequency charts are easier to read than matchup charts at these rounds because the combinations of matchups grow large as the tournament progresses.

Sweet Sixteen frequency chart below
These frequencies are determined by who succeeds in the Round of 32 and are reflected in the Round of 32 outcomes chart above. Look at the lump for the 10, 11, and 12 seeds. In years where these teams move forward knowing to pick them results in a large multiplicative effect on your score. Correctly picking #10 Davidson last year won me the pool.

Elite Eight seed frequency chart below.

Final Four seed frequency chart below.
Championship game seed frequency chart below.Championship winner seed frequency chart below.Some points for the Final Four, Championship game, and winner:

• Every other year or so a 5, 6, 8, 10, 11 seed makes it to the Elite Eight.
  
• One 11 seed, three 8 seeds, three 6 seeds and four 5 seeds have appeared in the Final Four in 96 opportunities over 24 years, choose these upsets sparingly, but if you get them right you might just win the pool.
• In the Championship game, one 8 seeds and two of 4, 5, and 6 seeds have made it that far. use sparingly.
• No team lower seeded than 8 has won the whole Championship. A 6, 8 and 4 seed have won it once each. The Final winner has been a 1 seed more than half of the time.

I have also taken the point totals for the past 24 years assuming a perfect sheet and plotted them.

I try to make sure that the potential points on my Playoff sheet add up to a reasonable number based on the past history of the tournament. The histogram below is a simple way to compare the past data to a current bracket selection.

It provides a way of ensuring that I haven't picked to many upsets, or worse, been too cautious and picked too few. Last year this method caused me to adjust my sheet to have more upsets and pick #10 Davidson to make it to the Elite Eight. I won the pool so handily that I was already uncatchable at that round.

After all of this discussion of picking upsets and examination of the data indicates that upsets happen and are the key to winning the pool, but which upsets and where. This is where we resort to the expertise of others. I use the Sagarin ratings (click on 2008-09 NCAA men's ratings by team) which are essentially a least squares ranking of all of the teams, based on all of the games that a team has played in the year. He suggests using the Predictor ranking to predict the outcome of a game rather than the ranking itself. Every year I match the teams to their rankings, the rankings represent the number of points a team is expected to score in a game so the difference of these rankings is the difference in the game. Since there is some error in the rankings I choose a value below which I will pick the lower seated team to win (picking upsets) and generate my bracket.

This year I automated the process in Excel. If a Predictor difference fell below the chosen factor I set the lower ranked team as the winner. Only for the final four does the model let the best team (higher Predictor score) win regardless of seed. A plot of the resulting expected points versus this factor shows some interesting cutoffs. Realize also that the home advantage for the Sagarin ratings this year is 3.79, almost two baskets. So the factors listed below are not out of the question. Always assuming that they fault to the upset is unreasonable, but called for to maximize point possibilities for this particular bracket.

In a similar manner to the inflection points from my earlier football simulations, certain values for the factor make the potential points jump between values as teams losing teams win and winning teams lose at certain rounds, only to be swept away at higher rounds. This leads to a high sensitivity of the final potential points to small changes in the game spread factor. An earlier plot shows that the 50% median value for the potential points was 631 and that 90% of the years had total pool points of less than 796. With this in mind I set the factor to 2.33, just below the first step change from 665 to 845 and then I examined the pool for reasonableness according to the statistics shown above. One caution with this model is that it might allow improbable events like too low a seed to make it through to a high round, so I used it merely to cause me to push the limits on upsets.

All that being said, be aware that on any day, any given team can beat any other, thus the format of March Madness is given to upsets and surprises and picking a bracket is still as much luck as skill. These models are an attempt to quantify this uncertainty and use it to drive bracket picks that will take advantage of luck, upsets and surprises when they occur.

March Madness for 2008 - reviewing the links

This week I have gotten a lot of search traffic based on search terms like "march madness statistics" and "march madness stats". Welcome to you all. I have in the past tried to use analysis of the past NCAA College basketball tournaments to try to improve my March Madness picks. In the past several years I still haven't won, but at least I know that I had a statistically good picks. To help you make your picks this year I suggest the following past posts:

• For the round of 64 and the round of 32 you should be picking upsets. I analyzed the data to find the range where the upsets occur.

• For the Final Four and the Championship only certain seeds have ever made it that far. These frequency charts might help you to ensure your picks are not outrageously different from the past history.

• Our March Madness pool gives points based on the seed of the team. If the team you picked wins then the points you get are the seed multiplied by the factor for the round (1,2,4,8,16,32 for the Round of 64, Round of 32, Sweet 16, Elite 8, Final Four and the Championship). I analyzed the data from 1985 to 2005 to find what the maximum points a perfect winner could get to help see if my picks were optimistic or pessimistic.

• The final useful detail is that though their will always be upsets, you can win a pool by picking which teams will upset. I use the Sagarin ratings to get some idea of which teams are seeded correctly and which are over- or underestimated. It is surprising how the seeds often don't follow the ranks or the ratings. Also remember that any given day any team can beat any other team no matter the rating.
  

All of the posts above are chuck full of statistical analysis, charts and data. I have even offered some tentative advice. Good luck with your pool, but hurry tomorrow is the start. Perhaps this year I will update two years more data and finally analyze the Sweet 16 and Elite 8. The data is in a spreadsheet just calling to me.

tags: March Madness, statistics, math, basketball

Can't get enough statistical analysis of March Madness upset picks

I decided to go back through the data I have accumulated from NCAA tournaments from 1985 to 2005 in order to see just how many points one could achieve by picking the winners with 100% accuracy. The point total scheme to be used is based on the sum of the seeds multiplied by a Round factor:

points = sum of winning seeds of 1st round * 1
+ sum of winning seeds of 2nd round * 2
+ sum of winning seeds of Sweet 16 Round * 4
+ sum of winning seeds of Elite 8 Round * 8
+ sum of winning seeds of Final Four Round * 16
+seed of the Champion * 32

As mentioned in earlier analyses this scheme encourages picking upsets since you get correspondingly more points. The big problem remains: which upsets to pick, and did I pick the right number of upsets. The following chart shows 21 year of data and the point totals expected for each. (click to enlarge)

This chart lets us answer some questions about which round is the most important for points and whether the tournament is the same from year to year or very different. Most years have around 185 points for Round 1, and 150 points for Round 2 and have the variability in the number of points increasing in later Rounds.

The highest point total year was 1985 (909 points) when #8 seed Villanova won the entire tournament and the #8 factor multiplied through every round. 1988 (799 points) had #6 seed Kansas win the entire tournament with similar effect. Lowest point year 1993 (only 486 points) had the opposite effect with almost no upsets the entire tournament and 3 #1 seeds and a #2 seed in the Final Four.

The point totals are high in 2000 for a different reason. In 2000 (789 points) two #8 seeds, North Carolina and Wisconsin made it to the Final Four. It is apparent and maybe obvious that years with upsets have correspondingly higher potential for points based on the upset scoring format, but that these points show up more in later rounds than in Rounds 1 and 2.

The next chart shows the distribution of the points for the past 21 years. (Click to enlarge) Half of the totals are less than 632 and 90% are less than 789. After the bracket is together for a particular year you can calculate the potential points assuming that all of the picks are right. It seems to me that this potential point total should reflect the distribution of past tournaments.

Thus, potential point totals of 600 to 650 reflect the most common potential point totals from the past. If you have chosen a bracket that falls out side of the ranges above they are statistically less likely. This type of analysis should allow you to check a bracket once it is complete to ensure that you haven't chosen too wildly or too conservatively. My potential point total this year was around 450, so I now feel that that was too conservative.

The next analysis needs to get at the difficult point of exactly which upsets to pick and how to perform the above analysis taking into account that no one gets all of the picks correctly. What is the correct way to make picks that ensures I get the bounce in the points in later rounds?

Some last March Madness NCAA Basketball pool advice.

The NCAA Basketball March Madness pool that I play in encourages upset picks by weighting the points by the seed in the tournament. You get the seed for the win. If #13 Davidson beats #4 Maryland you get 13 points, otherwise you get 4 points. Each round is multiplied by a factor, first round is 1, second is 2, third is 4 and so on to 32 points multiplied by the seed of the overall winner. It is ingenious because being safe doesn't translate into as many points as picking upsets, and that's the key to winning the whole thing.

Last year I did some extensive upset probability analysis of upsets in the round of 64 and round of 32. I also separately analyzed the final four and the winner. This year I am spending my time on my picks so I will just use the analysis from last year, which does neglect the 2006 results, but that is a small effect over the 21 years of data already in the analysis.

My advice:

• Pick the #1 team in the first round. They have never lost, and you are not going to be the lucky one who finally picks it when they do.

• For almost the same reason as above, pick the #2 team to win in the first round also.

• Pick upsets in the first round for the other matchups. #8 vs. #9 seeds are worse than toss ups, more than half of the time the #9 seed wins. Even #12 beats #5 one in three times.

• Pick more upsets in the second round. Almost half the time, 12 beat 4, 5 beat 4, 6 beat 3, 10 beat 2. A quarter of the time 8 beat 1, 7 beat 2, and 11 beat 4. Some of those matchups are rare, so take the statistics with a grain of salt.

• The worst seed to win the tournament was #8 and this happened only once. The worst seed to make it to the Final Four was ranked #11, which also only happened once. Since these are highly improbably events your bracket should avoid them.

• Get some more information about the individual teams, but don't rely too heavily on it (see upsets above). I am using the Sagarin ratings for some extra information on where teams were ranked over the year and as a substitute for my utter lack of knowledge about college basketball. This gives me some way to guess where the upsets that are expected above will happen. My excitement is in the math more than the sport.

Finally, on any given day any team can beat any other team. The key to winning is picking which upsets will happen and that takes some knowledge of the teams. Good luck.

tags: March Madness, statistics, math, basketball