Jump to content
Ching

Rank the Rankers

Recommended Posts

Which ranking service is the most predictive?

 

At the conclusion of the NCAA's this year, I'm going to score each of the ranking services with a very simple metric (result relative to rank) to determine which service is the best. The following table will be used to score each of the top 8 ranked wrestlers at each weight class based on their ranking and placement for each service:

 

2h2ebur.jpg

 

Each service will have a score per weight class and those will aggregate up to their total score.

 

Initially, I considered looked at each head-to-head matchup and scoring points based on the ranker having the correct wrestler winning. I thought this would be a good way to deal with the seeding having an outsized impact on the final results. Unfortunately (or maybe good for me), this doesn't work very well. After looking at last year, there are not enough individual matches that actually test the rankings to be useful. See examples:

 

Let's say Joe beats Bob in some round of the tournament

 

Example 1) Ranking A has #13 Joe upsetting #7 Bob and Ranking B has #14 Joe upsetting #5 Bob

Example 2): Ranking A has #5 Joe upsetting #4 Bob and Ranking B has #4 Joe beating #5 Bob

 

Example 1 is common enough (basically any upset), but none of the rankings services differ enough to score that result differently. Directionally, all ranking services are going to have Bob ranked above Joe. Example 2 happens, but is not common enough to separate the ranking services.

 

I plan to score the actual seedings as its own ranking system. If seeding has a significant effect, it should do better than the rankings services.

 

The scoring grid I'm using (see graphic above) is non-linear. I've created some cutoffs to reward getting a ranking close to the result, and punish as you get further away. R12 through 0-2 have a bigger step down because those are shared places with other individuals (there are 4 R12's and 8 R16's, etc). I also had a rule for no positive points for 0-2.

 

These are the ranking services I will include:

  • Flowrestling
  • Intermat
  • The Open Mat
  • WrestleStat
  • NCAA Seedings

Share this post


Link to post
Share on other sites

Which ranking service is the most predictive?

 

At the conclusion of the NCAA's this year, I'm going to score each of the ranking services with a very simple metric (result relative to rank) to determine which service is the best. The following table will be used to score each of the top 8 ranked wrestlers at each weight class based on their ranking and placement for each service:

 

2h2ebur.jpg

 

Each service will have a score per weight class and those will aggregate up to their total score.

 

Initially, I considered looked at each head-to-head matchup and scoring points based on the ranker having the correct wrestler winning. I thought this would be a good way to deal with the seeding having an outsized impact on the final results. Unfortunately (or maybe good for me), this doesn't work very well. After looking at last year, there are not enough individual matches that actually test the rankings to be useful. See examples:

 

Let's say Joe beats Bob in some round of the tournament

 

Example 1) Ranking A has #13 Joe upsetting #7 Bob and Ranking B has #14 Joe upsetting #5 Bob

Example 2): Ranking A has #5 Joe upsetting #4 Bob and Ranking B has #4 Joe beating #5 Bob

 

Example 1 is common enough (basically any upset), but none of the rankings services differ enough to score that result differently. Directionally, all ranking services are going to have Bob ranked above Joe. Example 2 happens, but is not common enough to separate the ranking services.

 

I plan to score the actual seedings as its own ranking system. If seeding has a significant effect, it should do better than the rankings services.

 

The scoring grid I'm using (see graphic above) is non-linear. I've created some cutoffs to reward getting a ranking close to the result, and punish as you get further away. R12 through 0-2 have a bigger step down because those are shared places with other individuals (there are 4 R12's and 8 R16's, etc). I also had a rule for no positive points for 0-2.

 

These are the ranking services I will include:

  • Flowrestling
  • Intermat
  • The Open Mat
  • WrestleStat
  • NCAA Seedings
Track also has rankings

 

 

Sent from my iPhone using Tapatalk

Share this post


Link to post
Share on other sites

Wrestlestat is not a "ranking" but a career points earned. 

 

The rest are all useless in general and generally suck. 

 

Regardless of what they are called, I do want to see how WrestleStat will do.

Share this post


Link to post
Share on other sites

Thanks for doing this.

 

I offer wonder about the differences between FLO, Intermat, Track & TOM when compared to NCAA Seeds.  Your study should shed some light on this.

 

Look forward to seeing your results.  I would recommend doing this another year or two before drawing any conclusions as one season does not constitute a meaningful sample size.

Share this post


Link to post
Share on other sites

Regardless of what they are called, I do want to see how WrestleStat will do.

 

And that's fine but I just wanted to make my point about the ranchers all sucking in general while Wrestlestat is not a ranking. 

Share this post


Link to post
Share on other sites

Oh boy I can’t wait for the results of this!!!

 

Please keep wrestlestat included, I’d love to see how math and statistics (objectivity) stacks up to expert panels (subjectivity)

 

Also - pending results of this I can’t want to hear FRLs response lol

Share this post


Link to post
Share on other sites

rankings are not predictions.

 

I didn't call them predictions, but I do wonder if they are predictive. I assume a rankings process attempts to identify who is the best at a point in time. That just so happens to match up nicely with the tournament that objectively tells us who the best is at a point time.

Share this post


Link to post
Share on other sites

I like this idea, but you should not word what you are doing as a measure of which service is best.  Imagine this scenario:  Myles Martin beats Bo Nickal in the finals.  If a ranking service had Myles Martin ranked #1, it would get the most points.  However, we all know this would be a ridiculous ranking to give him at the moment.  

 

Instead, simply stick to saying you are looking at which ranking service is most predictive.  This does not necessarily it is a better ranking service.  

Maybe throw in SHP's dual impact rankings?  He has some mystery formula to determine those.  Also, a ranking based solely on win/loss percentage that ignores strength of schedule could be an interesting control to throw in there. 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...

×
×
  • Create New...