Jump to content
Sign in to follow this  
maligned

You asked for it; here it is...Analysis into 12 years of unbalanced brackets.

Recommended Posts

OK, here’s what you’ve been asking for: genuine, history-proven statistics into the effects of the unbalanced random bracket.

Using 12 years of U.S. wrestling world-level performance, we can take the number of wrestlers on our side of the bracket, incorporate an element for how many matches we needed to win to reach the finals, and use statistical analysis software to come to some very strong estimates of the impact of various half-bracket sizes.

For a wrestler of average U.S. world-level quality (85 weight classes over 12 years), these are the expected probabilities for earning a medal based on the random brackets into which we’re drawn:

32-man half-bracket: 13.2% chance to medal
24-man half-bracket (big quarter): 17.5%
24-man half-bracket (small quarter): 23.9%
16-man half-bracket: 28.9%
12-man half-bracket (big quarter): 31.7%
12-man half-bracket (small quarter): 39.9%
8-man half-bracket: 42.6%
 

Share this post


Link to post
Share on other sites

There are 26 different combinations of path strength from among the 85 I compiled.  Most percentages of those are therefore meaningless.  These numbers represent what we can pull from building a mathematical model that combines all 85 performances into one expected performance trend for any randomly selected United States wrestler.

 

I say "average quality" because it considers each performance from among the 85 as one event.  Therefore, the expectation trend listed as probabiities above represents an average U.S. performance level.

Share this post


Link to post
Share on other sites

This may not seem like much, but take 65kg.  If you're on the bottom, you have a 13.2% chance to medal (31 participants on bottom).  If you're on the top, you have a 28.9% chance to medal (16 participants on top).

 

Just by being drawn in the top half, where roughly 1/3 of the participants reside, you've increased your statistical chance to medal by 219%.  This is a problem IF we care about fairness.

Share this post


Link to post
Share on other sites

I think it's a little more than 100%, rather than more than 200% (3 times as much as something) right?

 

Anyway, I'd rather see the actual numbers here, rather than the adjusted numbers.  I don't understand what has been adjusted.

 

This goes back to several long threads this week where a number of posters argued (effectively) that the thicker side of the brackets are mathematically twice as difficult as the shallower side.  I can't tell from maligned's numbers if this is the case in reality or not.

Share this post


Link to post
Share on other sites

I think it's a little more than 100%, rather than more than 200% (3 times as much as something) right?

 

Anyway, I'd rather see the actual numbers here, rather than the adjusted numbers.  I don't understand what has been adjusted.

 

This goes back to several long threads this week where a number of posters argued (effectively) that the thicker side of the brackets are mathematically twice as difficult as the shallower side.  I can't tell from maligned's numbers if this is the case in reality or not.

 

I think the point of his data is just that US wrestlers, as expected, are about twice as likely to medal when they are on the small side of the bracket (which is about half as big) compared to when they are on the large side. 

 

 

Could you also go through the champions from these 12 tournaments. Take the percentage of champions from the Small bracket, vs the Large bracket, and then normalize for what you would expect based on the skew of athletes?  

 

I.E. Is the champion total closer to 50%  from top bracket half, while the total athletes on top is actually closer to 33%?

 

If there is a skewing with statistical significance, this would suggest that the easier side also makes you more likely to win the tournament, despite not necessarily being the best wrestler on that particular day.  

Edited by Billyhoyle

Share this post


Link to post
Share on other sites

I think the point of his data is just that US wrestlers, as expected, are about twice as likely to medal when they are on the small side of the bracket (which is about half as big) compared to when they are on the large side. 

 

 

Could you also go through the champions from these 12 tournaments. Take the percentage of champions from the Small bracket, vs the Large bracket, and then normalize for what you would expect based on the skew of athletes?  

 

I.E. Is the champion total closer to 50%  from top bracket half, while the total athletes on top is actually closer to 33%?

 

If there is a skewing with statistical significance, this would suggest that the easier side also makes you more likely to win the tournament, despite not necessarily being the best wrestler on that particular day.  

This particular item: "what percentage of winners should we expect from the bottom bracket vs. the top?" is being thought of incorrectly in a number of discussions I've seen.  Just because we may be able to reach the conclusion that the bottom half winner will be better than the top half winner 2/3 of the time, does not mean that the expected winning percentage of the "better wrestler" will be 67%. 

If we assume that the average quality level of the winner of the top bracket ends up being roughly the quality level of the 3rd best wrestlers in the world and we assume that the bottom bracket winner typically is at an average quality level of the 2nd best wrestlers in the world, for example, the question becomes, "What is the typical expected win percentage of the 2nd best vs. the 3rd best?"  I'm guessing it's more like 55%, not 67%.

To test the "attrition" effect of the two brackets, we could potentially try to use a method for determining expected performance level for the two bracket winners independent of bracket effects--which we could then test against real-life performance in the championship matches.  This would be tricky though.

One thing I had thought of doing that was just taking too much time for data collection was to examine each half bracket.  You could regress a dummy of winning/losing on dummies for the presence of all teams in that half bracket AND a variable for how many teams there are.  The nation dummies should control for the impact of various quality levels in each bracket and intrinsically the quantity of expected performance level coming out of it--meaning that if the resulting coefficient on the "size of bracket" variable is significantly negative, we can assume that there is an attrition factor that is dragging expected performance level down from bigger brackets.

Edited by maligned

Share this post


Link to post
Share on other sites

This particular item: "what percentage of winners should we expect from the bottom bracket vs. the top?" is being thought of incorrectly in a number of discussions I've seen.  Just because we may be able to reach the conclusion that the bottom half winner will be better than the top half winner 2/3 of the time, does not mean that the expected winning percentage of the "better wrestler" will be 67%. 

If we assume that the average quality level of the winner of the top bracket ends up being roughly the quality level of the 3rd best wrestlers in the world and we assume that the bottom bracket winner typically is at an average quality level of the 2nd best wrestlers in the world, for example, the question becomes, "What is the typical expected win percentage of the 2nd best vs. the 3rd best?"  I'm guessing it's more like 55%, not 67%.

To test the "attrition" effect of the two brackets, we could potentially try to use a method for determining expected performance level for the two bracket winners independent of bracket effects--which we could then test against real-life performance in the championship matches.  This would be tricky though.

One thing I had thought of doing that was just taking too much time for data collection was to examine each half bracket.  You could regress a dummy of winning/losing on dummies for the presence of all teams in that half bracket AND a variable for how many teams there are.  The nation dummies should control for the impact of various quality levels in each bracket and intrinsically the quantity of expected performance level coming out of it--meaning that if the resulting coefficient on the "size of bracket" variable is significantly negative, we can assume that there is an attrition factor that is dragging expected performance level down from bigger brackets.

You are overthinking it a bit.  For my suggestion to look at the finals results, I am going on a simple assumption:

 

The best wrestler on a particular day should win the tournament.  

 

The basis for this is that while Varner may not have been the best wrestler overall at 96kg, we like to say that on one particular day in London, he was, because he was able to win the tournament.

 

The entire goal and purpose of bracketing/seeding first and foremost is to do everything it can to ensure that the best wrestler on a particular day is able to win the tournament.  It is impossible to put the bracket together perfectly, because we do not know ahead of time who that person happens we be (we have a good idea based on prior results, but sometimes people wake up with fire in them).

 

If it is assumed that the best wrestler wins on a particular day (i.e. what i would consider the ideal bracketing system), the bottom half of the bracket should win 2/3 of the time in the finals (or be proportional to the split in participants). 

 

I agree with you that a matchup between the 2nd and 3rd ranked wrestler will not lead to the 2nd best wrestler winning 100% of the times..because in some matches one person wrestles better than the other, while this can be reversed in the next matchup...But that is part of the point I am trying to make.  Being on the top of the bracket also gives you an unfair advantage to win the entire tournament (as opposed to only giving you an advantage in winning a medal) because you have an easier road to the finals than you otherwise should.  

Edited by Billyhoyle

Share this post


Link to post
Share on other sites

But that's just it...better wrestlers DON'T win 100% of the time, no matter the circumstance.  And this is especially true at this high of a level among closely ranked opponents, such as they are in the finals, even with uneven bracketing.  It's like randomly selecting 10 NFL teams to play one side of a bracket and 5 to play the other side then expecting that the team coming out of the 10-team bracket will win two-thirds of the time.  It just ain't gonna happen.  They'll win more than half the time, but they won't win close to 2/3 of the time because of the parity among good teams that will survive those two brackets.  The same is true here.  I've only looked myself at the last couple years to know that the bottom half of the bracket is only 8-8 in the last two years.  If we look back at a lot of years, it will be different than that, but the bottom bracket won't win much more than half and it's tough to attribute that fact automatically to attrition.

Edited by maligned

Share this post


Link to post
Share on other sites

But that's just it...better wrestlers DON'T win 100% of the time, no matter the circumstance.  And this is especially true at this high of a level among closely ranked opponents, such as they are in the finals, even with uneven bracketing.  It's like randomly selecting 10 NFL teams to play one side of a bracket and 5 to play the other side then expecting that the team coming out of the 10-team bracket will win two-thirds of the time.  It just ain't gonna happen.  They'll win more than half the time, but they won't win close to 2/3 of the time because of the parity among good teams that will survive those two brackets.  The same is true here.  I've only looked myself at the last couple years to know that the bottom half of the bracket is only 8-8 in the last two years.  If we look back at a lot of years, it will be different than that, but the bottom bracket won't win much more than half and it's tough to attribute that fact automatically to attrition.

I'm not only attributing it to attrition.  I'm trying to say that the unbalanced bracket impacts the champion just like it impacts the overall medalists.  Part of it is fatigue, while part is also (as you point out), the fact that the final result will still be partly random.  

 

My point is that any deviation of champions from the ratio of participants unfairly skews the overall winner, which people may find more outrageous than simply skewing the overall medalists.    

 

In short:

Easier path to finals=more likely to win tournament.   

Edited by Billyhoyle

Share this post


Link to post
Share on other sites

If the brackets were equally balanced size and quality, each side of the bracket would produce 50% of the winners, given enough events.

 

If the half brackets are unequal in size (and therefore in quality), the results will not be 50/50.  They should be proportional to bracket size.  If they aren't, something is wrong.

Share this post


Link to post
Share on other sites

If the brackets were equally balanced size and quality, each side of the bracket would produce 50% of the winners, given enough events.

 

If the half brackets are unequal in size (and therefore in quality), the results will not be 50/50.  They should be proportional to bracket size.  If they aren't, something is wrong.

Exactly.  By something is wrong, you mean that it is unfair to the wrestlers who get put in the side of the bracket where it makes them less likely to win.  

 

Maligned, when he says it shouldn't be balanced, is pointing out the fact that the finals match shouldn't necessarily be proportional to the difficulty of the side of the bracket because it is almost like a coin flip.  

 

I agree with maligned in that the coin flip impacts the outcome, but like you I agree that this is unfair (whether it is from fatigue or other reasons).  

Share this post


Link to post
Share on other sites

 

Maligned, when he says it shouldn't be balanced, is pointing out the fact that the finals match shouldn't necessarily be proportional to the difficulty of the side of the bracket because it is almost like a coin flip.  

It's not a coin flip - coins are balanced 50/50.  Brackets are unbalanced.  Results of the finals should mirror the shape of the brackets, given enough events.

Share this post


Link to post
Share on other sites

It's not a coin flip - coins are balanced 50/50.  Brackets are unbalanced.  Results of the finals should mirror the shape of the brackets, given enough events.

They should, if the assumption held that all wrestlers were of different skills and the better wrestler won every time.  This assumption never actually holds though...

 

take the example of a 12 man bracket where everyone is of equal talent.  Each match has a 50% chance of going to either wrestler.  

 

Wrestled enough times, the results of the finals match will still be 50% for top bracket 50% for bottom bracket, even if the split is 4 top to 8 bottom.  

Share this post


Link to post
Share on other sites

They should, if the assumption held that all wrestlers were of different skills and the better wrestler won every time.  This assumption never actually holds though...

 

take the example of a 12 man bracket where everyone is of equal talent.  Each match has a 50% chance of going to either wrestler.  

 

Wrestled enough times, the results of the finals match will still be 50% for top bracket 50% for bottom bracket, even if the split is 4 top to 8 bottom.  

We're not dealing with a 12 man bracket of equal talent.  We're dealing with brackets of variable size with huge variations in talent.  In a bracket like that, the size of the half-brackets matters a lot, and it's going to influence who wins.

Share this post


Link to post
Share on other sites

We're not dealing with a 12 man bracket of equal talent.  We're dealing with brackets of variable size with huge variations in talent.  In a bracket like that, the size of the half-brackets matters a lot, and it's going to influence who wins.

We aren't dealing with the 12 man bracket of equal talent, but we also aren't dealing with the situation where the better wrestler wins every time.  It's something in between the two....Anyway there's no point going on about this much longer.  Either way, if the results are skewed from the ratio of participants it shows the bracket is inherently unfair for determining the champion.  

Share this post


Link to post
Share on other sites

but we also aren't dealing with the situation where the better wrestler wins every time.

This doesn't matter, because it will balance out over time.  Sometimes the best guy will be on the top, sometimes on the bottom, sometimes he won't win -- whatever.  All of that balances out.  Imagine if the brackets were always randomly filled, but there were always 32 men in the bracket.  If you had a tournament for one hundred years, you would expect half the champions to come from the top and half to come from the bottom.  If the results were badly skewed, you would know that something was wrong.

 

In our situation, the brackets are not equal.  Given enough events, the champion should come out of the thick sides of the brackets more often in proportion to how unbalanced the brackets are.  Given some preliminary counting, that does not seem to be happening, which suggests that something is wrong.

Share this post


Link to post
Share on other sites

The brackets being unbalanced may not affect who wins the gold medal as much as who gets a silver or bronze.

 

If you have the top three wrestlers in the world (yeah, I know... rankings are rather subjective) all in the same half bracket... or even quarter of bracket, you'll likely see a strong potential medalist...a #1,2 or 3 ranked guy, get knocked out earlier than he might have, had brackets been more balanced. 

 

I wonder, given the difficulties in ranking guys throughout the world, if it would be possible to do an abbreviated version of seeding.  Try to select the top 4 guys in the world, and spread them out throughout brackets, and then everyone else get placed by luck.

Share this post


Link to post
Share on other sites

My thought would be to take returning medalists (and given the seeming youth movement perhaps Junior Champions) and then randomly assign them to the 'normal' seeded locations (top, bottom, bottom of top half, top of bottom half, etc). Then everyone else randomly gets assigned to the remaining spots. Nothing subjective, no ranking of 'seedables' just placement.

Share this post


Link to post
Share on other sites

This doesn't matter, because it will balance out over time.  Sometimes the best guy will be on the top, sometimes on the bottom, sometimes he won't win -- whatever.  All of that balances out.  Imagine if the brackets were always randomly filled, but there were always 32 men in the bracket.  If you had a tournament for one hundred years, you would expect half the champions to come from the top and half to come from the bottom.  If the results were badly skewed, you would know that something was wrong.

 

In our situation, the brackets are not equal.  Given enough events, the champion should come out of the thick sides of the brackets more often in proportion to how unbalanced the brackets are.  Given some preliminary counting, that does not seem to be happening, which suggests that something is wrong.

Quanon, it's true that over time the winners will come from the bottom bracket slightly more often.  But it won't be by much. 

 

Consider: If we generally end up with two top 5 guys (which will be true the vast majority of the time), we might theoretically expect for the best guy--even with full rest--will only win a matchup of randomly selected Top 5 guys about 55-75% of the time (we'll say 75% to stay on the very generous side, so I can illustrate my point).

 

Now, if we have the best guy on a full bracket on the bottom only 2/3 of the time, the math problem for predicting the expected bottom-half win% goes like this: (better guy 2/3 of the time) TIMES (75% win percentage) PLUS (worse guy 1/3 of the time) TIMES (25% win percentage) EQUALS 58% expected win percentage for the bottom half.

 

But here's the thing: we don't have full double-sized brackets on the bottom very often.  Over the last two years, the ratio of bottom to top is only 1.38 to 1 (22:16 if we have a full 16-man top bracket).  That means the bottom half only gets the better guy from the above description 58% of the time to begin with--lowering our expected bottom-half win percentage all the way down to 54%.  This is a much more realistic starting point for the expected long-term win% of the bottom-half representative if we had hundreds of brackets available to analyze.  We will never approach anything near 2/3.

Share this post


Link to post
Share on other sites

Ok, I counted up the last 12 years: the top bracket (smaller bracket, typically 40-45% of competitors) actually holds the edge in that time period, 46-40 (53%).

 

I'm sure this difference is mostly just luck and we would see the larger bracket side have more winners with more data points available.  But it does go to show the point that a couple of us were making: it's way less simple than "this bracket has 2/3 of the wrestlers so it will produce 2/3 of the champions."

Share this post


Link to post
Share on other sites

Ok, I counted up the last 12 years: the top bracket (smaller bracket, typically 40-45% of competitors) actually holds the edge in that time period, 46-40 (53%).

 

I'm sure this difference is mostly just luck and we would see the larger bracket side have more winners with more data points available.  But it does go to show the point that a couple of us were making: it's way less simple than "this bracket has 2/3 of the wrestlers so it will produce 2/3 of the champions."

I think the point is that in a fair system, the bottom would win 2/3 of the time if it has 2/3 of the participants. Would you mind posting the exact average for participants instead of the range?

 

From your data, simply being in the top bracket makes you 20-30% more likely to win than you were going into the tournament, and about 50% more likely to win than if you had been placed in the bottom bracket.  

 

Think about that...the bracket system alone has that large of an effect on the overall outcome, not just the medal count.  

Share this post


Link to post
Share on other sites

Ok, I counted up the last 12 years: the top bracket (smaller bracket, typically 40-45% of competitors) actually holds the edge in that time period, 46-40 (53%).

 

I'm sure this difference is mostly just luck and we would see the larger bracket side have more winners with more data points available.  But it does go to show the point that a couple of us were making: it's way less simple than "this bracket has 2/3 of the wrestlers so it will produce 2/3 of the champions."

If I flipped a coin a thousand times and it came up heads 700 times, wouldn't you begin to suspect that the coin was not fair?  Or would you say it's a fair coin, but since you never know whether the coin will be heads or tails, it's not surprising that it's landing on heads 70% of the time (this is like your argument about uneven brackets producing winners out of proportion to the brackets).

 

A bracket with 2/3 of the wrestlers will produce 2/3 of the champions on average, if they are randomly distributed.  It has nothing to do with who is "the best" wrestler.  Whether the best guy wins or the 10th best guy wins any particular event is irrelevant given enough events -- the skill levels will be distributed randomly each time, in proportion to bracket size.

 

More data would be great, but this difference between expectations and results is not luck.  I counted up the results between 2010 and 2014 a few days ago -- there was never a bracket that didn't have more champions with byes than it should have.  This is a structural problem.

Share this post


Link to post
Share on other sites

 

 

A bracket with 2/3 of the wrestlers will produce 2/3 of the champions on average, if they are randomly distributed.  It has nothing to do with who is "the best" wrestler.

 

This is absolutely, positively a wrong statement, for all the reasons I laid out in the last post on page one of this thread.  The only way this can be true is if there is always one best wrestler who wins 100% of the matches and will therefore provide 2/3 winning odds for the bigger bracket and 1/3 winning odds for the smaller bracket. 

Edited by maligned

Share this post


Link to post
Share on other sites

If I flipped a coin a thousand times and it came up heads 700 times, wouldn't you begin to suspect that the coin was not fair?  Or would you say it's a fair coin, but since you never know whether the coin will be heads or tails, it's not surprising that it's landing on heads 70% of the time (this is like your argument about uneven brackets producing winners out of proportion to the brackets).

My calculations show a 5x10-38 chance of getting 700 or more heads with a thousand flips of an unbiased coin.

There is a one in million chance to get 571 or more heads in the same situation.

Sorry about nerdng out!

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  

×
×
  • Create New...