Jump to content
Ching

Rank the Rankers - 2019

Recommended Posts

Copy/past from last year

Quote

 

Which ranking service is the most predictive?

At the conclusion of the NCAA's this year, I'm going to score each of the ranking services with a very simple metric (result relative to rank) to determine which service is the best. The following table will be used to score each of the top 8 ranked wrestlers at each weight class based on their ranking and placement for each service:

2h2ebur.jpg

 

Each service will have a score per weight class and those will aggregate up to their total score.

Initially, I considered looked at each head-to-head matchup and scoring points based on the ranker having the correct wrestler winning. I thought this would be a good way to deal with the seeding having an outsized impact on the final results. Unfortunately (or maybe good for me), this doesn't work very well. After looking at last year, there are not enough individual matches that actually test the rankings to be useful. See examples:

Let's say Joe beats Bob in some round of the tournament

Example 1) Ranking A has #13 Joe upsetting #7 Bob and Ranking B has #14 Joe upsetting #5 Bob

Example 2): Ranking A has #5 Joe upsetting #4 Bob and Ranking B has #4 Joe beating #5 Bob

Example 1 is common enough (basically any upset), but none of the rankings services differ enough to score that result differently. Directionally, all ranking services are going to have Bob ranked above Joe. Example 2 happens, but is not common enough to separate the ranking services.

I plan to score the actual seedings as its own ranking system. If seeding has a significant effect, it should do better than the rankings services.

The scoring grid I'm using (see graphic above) is non-linear. I've created some cutoffs to reward getting a ranking close to the result, and punish as you get further away. R12 through 0-2 have a bigger step down because those are shared places with other individuals (there are 4 R12's and 8 R16's, etc). I also had a rule for no positive points for 0-2.

 

These are the ranking services I will include:

  • Trackwrestling
  • Seeding
  • Intermat
  • Amateur Wrestling News
  • WrestlingByPirate Dual Impact Index
  • Flowrestling
  • The Open Mat
  • WIN Magazine
  • WrestleStat

Any others you would like to see?

Last year, the top three were:

  1. Trackwrestling
  2. Seeding
  3. Intermat

I will grab all the rankings as soon as the seeds come out. Any changes after the seeds will not be considered.

Share this post


Link to post
Share on other sites

Thinking about dropping AWN from this list, they don't have updated rankings. I could use their Feb 25th rankings, any opinions on this quandary? The rest of the services all published within a day of each other. 

Share this post


Link to post
Share on other sites
14 hours ago, pish6969 said:

Can’t wait for the results. Flo stuck with Shak at 8 or so. That’s gonna hurt

They have him at #6 and everyone else has him at #2. 184 is the least correlated of all the weight classes:

1988124887_Screenshotfrom2019-03-1508-48-34.png.a43ca88d3a8a0ef24be4029213f94967.png

Screenshot-from-2019-03-15-08-48-21.png

Flo has the highest correlation with the seedings. Overall, the average of all rankings is a little less than last year.

WrestleStat and Pirate are more highly correlated to the group this year. I know WrestleStat ( @andegre ) changed his algo in the offseason and it has some weights that are in line with the crowd and a few headscratchers (such as 141 and 157). We will find out if he is an evil genius or just plain evil.

Share this post


Link to post
Share on other sites

Are you trying to look at prediction or rankings? I could be way off but a ranking is not necessarily where a service thinks a guy will finish right? Like I think a lot of rankers would still pick Lee, Zhahid and Gable to win it all. It is just hard to rank those guys No. 1. 

Edited by osufan12
added thoughts

Share this post


Link to post
Share on other sites
Are you trying to look at prediction or rankings? I could be way off but a ranking is not necessarily where a service thinks a guy will finish right? Like I think a lot of rankers would still pick Lee, Zhahid and Gable to win it all. It is just hard to rank those guys No. 1. 

 

He’s looking at rankings. Correct they are not necessarily predictions but it’s not a stretch that your year end rankings should be close to what you predict

Share this post


Link to post
Share on other sites

I recommended in another thread that the winner of your yearly contest should be the media outlet invited to next year’s seeding meetings!

You want to help with seedings? Than win this contest (you have to beat the seeders). Last year Track was only one that was better

Share this post


Link to post
Share on other sites
3 hours ago, Ching said:

They have him at #6 and everyone else has him at #2. 184 is the least correlated of all the weight classes:

1988124887_Screenshotfrom2019-03-1508-48-34.png.a43ca88d3a8a0ef24be4029213f94967.png

Screenshot-from-2019-03-15-08-48-21.png

Flo has the highest correlation with the seedings. Overall, the average of all rankings is a little less than last year.

WrestleStat and Pirate are more highly correlated to the group this year. I know WrestleStat ( @andegre ) changed his algo in the offseason and it has some weights that are in line with the crowd and a few headscratchers (such as 141 and 157). We will find out if he is an evil genius or just plain evil.

Ironic that Flo is the one most loudly complaining about the seedings, yet Flo is the most correlated to the seedings.

Share this post


Link to post
Share on other sites

Ching, appreciate your laughter, but it's true. Flo could run a $10,000,000,000,000 dollar giveaway to anybody who could come up with the perfect bracket, and at least half of the entries would say that they were pretty confident Wednesday afternoon that their brackets were all perfect...but, let's face it, nobody is going 640-0!

Share this post


Link to post
Share on other sites
Thinking about dropping AWN from this list, they don't have updated rankings. I could use their Feb 25th rankings, any opinions on this quandary? The rest of the services all published within a day of each other. 

AWN did post them in the mag and in their email, but haven’t on the site. DM me an email address and i’ll forward them to you.

Share this post


Link to post
Share on other sites
20 minutes ago, SetonHallPirate said:

Ching, appreciate your laughter, but it's true. Flo could run a $10,000,000,000,000 dollar giveaway to anybody who could come up with the perfect bracket, and at least half of the entries would say that they were pretty confident Wednesday afternoon that their brackets were all perfect...but, let's face it, nobody is going 640-0!

Right! I remember when Warren Buffett did his "billion dollar bracket" challenge for March Madness, which was a good promotion because the chances of having a perfect bracket is so infinitesimally small. I think he stopped doing it when people started organizing crowdsourced brackets and prediction markets like "take buffett's billion" - which leads me to wonder if it would be helpful to add a crowdsourced "people's ranking" for wrestling that's also considered in the mix for seeding.

Share this post


Link to post
Share on other sites
24 minutes ago, JasonBryant said:


AWN did post them in the mag and in their email, but haven’t on the site. DM me an email address and i’ll forward them to you.

Thanks Jason, I reached out the other day and they responded pretty quickly. I've updated them to their March 12 rankings.

Share this post


Link to post
Share on other sites
15 hours ago, pamela said:

Right! I remember when Warren Buffett did his "billion dollar bracket" challenge for March Madness, which was a good promotion because the chances of having a perfect bracket is so infinitesimally small. I think he stopped doing it when people started organizing crowdsourced brackets and prediction markets like "take buffett's billion" - which leads me to wonder if it would be helpful to add a crowdsourced "people's ranking" for wrestling that's also considered in the mix for seeding.

Standard practice for those marketing schemes is to purchase an insurance policy that actually pays if there is a winner.  It is surprisingly affordable.  I wonder if the crowdsourcing issue voided the policy, thus Buffett was forced to stop?

Share this post


Link to post
Share on other sites

Doesn't look good for WrestleStat already this year, not just because of the rankings, but how the wrestlers are SEEDED... Check out what my algorithm guy sent me explaining why we'll do poorly this year...

Quote

Hey Greg,

   We are in big trouble in the RTR contest.  9 times in the 10 weights, 2 of our top 8 guys meet each other in the round of 16 or 32 whereas this happens only one time for SHP.  Weight 184 looks bad also. This is serious because the chance of placing for the loser of the match is only 12.5% = 0.5 x 0.5 x 0.5 = 3 wins (16-32, 16-16  16-Q ) to place in the top 8 if you lose in the Rd. of 16 and 6.25% chance to place if you lose in the Rd. of 32. See below for details:

 

     133:   Our 5 and 8, seeded 7 and 10, meet in Rd. of 16   (7+10=17)

     141:   Our 7 and 5, seeded 9 and 24, meet in Rd. of 32   (9+24=33) 

               Our 4 and 6, seeded 3 and 14, meet in Rd. of 16   (3+14=17)

     149:   Our 6 and 8, seeded 7 and 10, meet in Rd. of 16   (7+10=17)

     157:   Our 3 and 7, seeded 3 and 14, meet in Rd. of 16   (3+14=17)

     165:   Our 6 and 8, seeded 6 and 11, meet in Rd. of 16   (6+11=17) 

     174:   Our 1 and 7, seeded 1 and 17, meet in Rd. of 16    if our # 7 seeded 17 beats 16 seed in Rd. 32 (16 +17=33)

     197:   Our 7 and 5, seeded 8 and 9  , meet in Rd. of 16    (8+9 =17)

     285:   Our 5 and 8, seeded 6 and 11, meet in Rd. of 16   (6 +11=17)

 

     184:   Our 5 meets 9th seed in Rd. of 16

               Our 6 meets 6th seed in Rd. of 16

               Our 7 meets 4th seed in Rd. of 16

               Our 8 meets our 10     in Rd. of 16

 

He's sent subsequent emails clarifying that @SetonHallPirate is affected by this same scenario at 4 different weights (149, 184, 197, 285). Will be interesting now to see how the rankings hold up, though I'm not as confident as I was previously.

Share this post


Link to post
Share on other sites
59 minutes ago, andegre said:

Doesn't look good for WrestleStat already this year, not just because of the rankings, but how the wrestlers are SEEDED... Check out what my algorithm guy sent me explaining why we'll do poorly this year...

 

He's sent subsequent emails clarifying that @SetonHallPirate is affected by this same scenario at 4 different weights (149, 184, 197, 285). Will be interesting now to see how the rankings hold up, though I'm not as confident as I was previously.

I'm actually affected by it at 141, 149, 157, 184, and 285, for what it's worth. I wonder what weights the human rankers are affected.

Share this post


Link to post
Share on other sites
19 hours ago, SetonHallPirate said:

I'm actually affected by it at 141, 149, 157, 184, and 285, for what it's worth. I wonder what weights the human rankers are affected.

And here's his response:

 

Quote

     To answer SHP's question, for some of the human rankers, InterMat is affected 6 times, The Open Mat 6 times and Flo 7 times. For the computers, SHP is affected 5 times and Wrestlestat is still hurt the worst at 9 times.

            Intermat at         133,157,184 twice and 285 twice = 6 Total

            The Open Mat at    133,149,157,184,197 and 285 = 6 Total

            Flo at          125, 133, 141, 149, 157, 184 and 285 = 7 Total

 

FYI, going back over a number of past Nat'l tournaments and averaging the data, here is where the 8 placers come from:  From the quarter finals, 4 winners place.

          From the quarter finals, 2 of the 4 losers place ( 4 x 0.5 = 2 since losing Q must win 16 - Q match to place).

          From  Rd. of 16  losers, 1 places ( 8 x 0.5 cubed = 8 x 0.125 = 1 since must win 32-16, 16-16 and 16-Q).

          From  Rd. of 32  losers, 1 places (16 x 0.5 to the 4th power = 16 x 0.0625 = 1, 32-32, 32-16, 16-16 and 16-Q).

That last section shows the significance of this....quarters 2 of 4 place, rd16 losers only 1, and rd32 only 1.

Share this post


Link to post
Share on other sites
On 3/16/2019 at 11:45 AM, pish6969 said:

Ching. I was just thinking about this. How do you account for seeds higher than 20 that end up higher since most of the outlets don’t rank past 20?

I only score the top eight ranked wrestlers.

Edited by Ching

Share this post


Link to post
Share on other sites
On 3/16/2019 at 3:24 PM, SetonHallPirate said:

Thinking we should be allowed to use our algorithm's projected brackets, rather than projected seeds. Mine can be found at https://wrestlingbypirate.wordpress.com/2019/03/16/dual-impact-index-ncaa-championships-bracket-projections/

I agree with this.  The seeds have an unfair advantage vs the rankings in this comparison since they set the brackets and thus help to determine the outcome.  

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×