Rank the Rankers

[Ching 180]

Which ranking service is the most predictive?

 

At the conclusion of the NCAA's this year, I'm going to score each of the ranking services with a very simple metric (result relative to rank) to determine which service is the best. The following table will be used to score each of the top 8 ranked wrestlers at each weight class based on their ranking and placement for each service:

 

 

Each service will have a score per weight class and those will aggregate up to their total score.

 

Initially, I considered looked at each head-to-head matchup and scoring points based on the ranker having the correct wrestler winning. I thought this would be a good way to deal with the seeding having an outsized impact on the final results. Unfortunately (or maybe good for me), this doesn't work very well. After looking at last year, there are not enough individual matches that actually test the rankings to be useful. See examples:

 

Let's say Joe beats Bob in some round of the tournament

 

Example 1) Ranking A has #13 Joe upsetting #7 Bob and Ranking B has #14 Joe upsetting #5 Bob

Example 2): Ranking A has #5 Joe upsetting #4 Bob and Ranking B has #4 Joe beating #5 Bob

 

Example 1 is common enough (basically any upset), but none of the rankings services differ enough to score that result differently. Directionally, all ranking services are going to have Bob ranked above Joe. Example 2 happens, but is not common enough to separate the ranking services.

 

I plan to score the actual seedings as its own ranking system. If seeding has a significant effect, it should do better than the rankings services.

 

The scoring grid I'm using (see graphic above) is non-linear. I've created some cutoffs to reward getting a ranking close to the result, and punish as you get further away. R12 through 0-2 have a bigger step down because those are shared places with other individuals (there are 4 R12's and 8 R16's, etc). I also had a rule for no positive points for 0-2.

 

These are the ranking services I will include:

• Flowrestling
• Intermat
• The Open Mat
• WrestleStat
• NCAA Seedings

[Plasmodium 1,312]

Excellent!

[lu_alum 647]

Which ranking service is the most predictive?

 

At the conclusion of the NCAA's this year, I'm going to score each of the ranking services with a very simple metric (result relative to rank) to determine which service is the best. The following table will be used to score each of the top 8 ranked wrestlers at each weight class based on their ranking and placement for each service:

 

 

Each service will have a score per weight class and those will aggregate up to their total score.

 

Initially, I considered looked at each head-to-head matchup and scoring points based on the ranker having the correct wrestler winning. I thought this would be a good way to deal with the seeding having an outsized impact on the final results. Unfortunately (or maybe good for me), this doesn't work very well. After looking at last year, there are not enough individual matches that actually test the rankings to be useful. See examples:

 

Let's say Joe beats Bob in some round of the tournament

 

Example 1) Ranking A has #13 Joe upsetting #7 Bob and Ranking B has #14 Joe upsetting #5 Bob

Example 2): Ranking A has #5 Joe upsetting #4 Bob and Ranking B has #4 Joe beating #5 Bob

 

Example 1 is common enough (basically any upset), but none of the rankings services differ enough to score that result differently. Directionally, all ranking services are going to have Bob ranked above Joe. Example 2 happens, but is not common enough to separate the ranking services.

 

I plan to score the actual seedings as its own ranking system. If seeding has a significant effect, it should do better than the rankings services.

 

The scoring grid I'm using (see graphic above) is non-linear. I've created some cutoffs to reward getting a ranking close to the result, and punish as you get further away. R12 through 0-2 have a bigger step down because those are shared places with other individuals (there are 4 R12's and 8 R16's, etc). I also had a rule for no positive points for 0-2.

 

These are the ranking services I will include:

• Flowrestling
• Intermat
• The Open Mat
• WrestleStat
• NCAA Seedings

Track also has rankings

 

 

Sent from my iPhone using Tapatalk

[Ching 180]

Track also has rankings

 

 

Sent from my iPhone using Tapatalk

 

Thank you, I will add them also.

[Zebra 545]

Wrestlestat is not a "ranking" but a career points earned. 

 

The rest are all useless in general and generally suck. 

[Plasmodium 1,312]

WIN magazine does a good job with the rankings

[Ching 180]

WIN magazine does a good job with the rankings

 

WIN has not updated their rankings since Dec 4th.

[Plasmodium 1,312]

They do a weekly ranking that themat.com posts on Mondays. It's sort of hidden on their site because they want it to be behind a paywall.

[Ching 180]

Wrestlestat is not a "ranking" but a career points earned. 

 

The rest are all useless in general and generally suck. 

 

Regardless of what they are called, I do want to see how WrestleStat will do.

[Ching 180]

They do a weekly ranking that themat.com posts on Mondays. It's sort of hidden on their site because they want it to be behind a paywall.

 

Got it, I will add them to the list also.

[Show_Me 258]

Thanks for doing this.

 

I offer wonder about the differences between FLO, Intermat, Track & TOM when compared to NCAA Seeds.  Your study should shed some light on this.

 

Look forward to seeing your results.  I would recommend doing this another year or two before drawing any conclusions as one season does not constitute a meaningful sample size.

[Zebra 545]

Regardless of what they are called, I do want to see how WrestleStat will do.

 

And that's fine but I just wanted to make my point about the ranchers all sucking in general while Wrestlestat is not a ranking. 

[potentiallydangerous 176]

Gregg Henry was great at Wrestling Report but I think he stopped when Allen Brown died.

Edited March 7, 2018 by potentiallydangerous

[Witherman 350]

Oh boy I can’t wait for the results of this!!!

 

Please keep wrestlestat included, I’d love to see how math and statistics (objectivity) stacks up to expert panels (subjectivity)

 

Also - pending results of this I can’t want to hear FRLs response lol

[Husker_Du 480]

rankings are not predictions.

[OrangeEffect 18]

rankings are not predictions.

 

It will be interesting anyway and I am looking forward to it.

[Ching 180]

rankings are not predictions.

 

I didn't call them predictions, but I do wonder if they are predictive. I assume a rankings process attempts to identify who is the best at a point in time. That just so happens to match up nicely with the tournament that objectively tells us who the best is at a point time.

[paboom 204]

Regardless of what they are called, I do want to see how WrestleStat will do.

They will have to remove Ashnault first.

 

Sent from my SM-G950U using Tapatalk

[Zebra 545]

Regardless of what they are called, I do want to see how WrestleStat will do.

And if you do you should select the "Starters only" option since RS is included in the career points. 

[Husker_Du 480]

agree on it being interesting.

 

looking forward to the results. 

[TobusRex 1,659]

Sounds like a great idea, Ching!

[mlbruem 123]

Amateur Wrestling News also does rankings last update 3/5/18

please include them if possible.

[Ching 180]

Amateur Wrestling News also does rankings last update 3/5/18

please include them if possible.

 

 

Will do, thanks.

[Jaroslav Hasek 1,800]

this is cool, thank you for doing this. 

[Billyhoyle 1,796]

I like this idea, but you should not word what you are doing as a measure of which service is best.  Imagine this scenario:  Myles Martin beats Bo Nickal in the finals.  If a ranking service had Myles Martin ranked #1, it would get the most points.  However, we all know this would be a ridiculous ranking to give him at the moment.  

 

Instead, simply stick to saying you are looking at which ranking service is most predictive.  This does not necessarily it is a better ranking service.  

Maybe throw in SHP's dual impact rankings?  He has some mystery formula to determine those.  Also, a ranking based solely on win/loss percentage that ignores strength of schedule could be an interesting control to throw in there.