Ching 180 Report post Posted March 7, 2018 Which ranking service is the most predictive? At the conclusion of the NCAA's this year, I'm going to score each of the ranking services with a very simple metric (result relative to rank) to determine which service is the best. The following table will be used to score each of the top 8 ranked wrestlers at each weight class based on their ranking and placement for each service: Each service will have a score per weight class and those will aggregate up to their total score. Initially, I considered looked at each head-to-head matchup and scoring points based on the ranker having the correct wrestler winning. I thought this would be a good way to deal with the seeding having an outsized impact on the final results. Unfortunately (or maybe good for me), this doesn't work very well. After looking at last year, there are not enough individual matches that actually test the rankings to be useful. See examples: Let's say Joe beats Bob in some round of the tournament Example 1) Ranking A has #13 Joe upsetting #7 Bob and Ranking B has #14 Joe upsetting #5 Bob Example 2): Ranking A has #5 Joe upsetting #4 Bob and Ranking B has #4 Joe beating #5 Bob Example 1 is common enough (basically any upset), but none of the rankings services differ enough to score that result differently. Directionally, all ranking services are going to have Bob ranked above Joe. Example 2 happens, but is not common enough to separate the ranking services. I plan to score the actual seedings as its own ranking system. If seeding has a significant effect, it should do better than the rankings services. The scoring grid I'm using (see graphic above) is non-linear. I've created some cutoffs to reward getting a ranking close to the result, and punish as you get further away. R12 through 0-2 have a bigger step down because those are shared places with other individuals (there are 4 R12's and 8 R16's, etc). I also had a rule for no positive points for 0-2. These are the ranking services I will include: Flowrestling Intermat The Open Mat WrestleStat NCAA Seedings 4 LaxHawk174, GoNotQuietly, GranbyTroll and 1 other reacted to this Quote Share this post Link to post Share on other sites
Plasmodium 1,596 Report post Posted March 7, 2018 Excellent! Quote Share this post Link to post Share on other sites
lu_alum 708 Report post Posted March 7, 2018 Which ranking service is the most predictive? At the conclusion of the NCAA's this year, I'm going to score each of the ranking services with a very simple metric (result relative to rank) to determine which service is the best. The following table will be used to score each of the top 8 ranked wrestlers at each weight class based on their ranking and placement for each service: Each service will have a score per weight class and those will aggregate up to their total score. Initially, I considered looked at each head-to-head matchup and scoring points based on the ranker having the correct wrestler winning. I thought this would be a good way to deal with the seeding having an outsized impact on the final results. Unfortunately (or maybe good for me), this doesn't work very well. After looking at last year, there are not enough individual matches that actually test the rankings to be useful. See examples: Let's say Joe beats Bob in some round of the tournament Example 1) Ranking A has #13 Joe upsetting #7 Bob and Ranking B has #14 Joe upsetting #5 Bob Example 2): Ranking A has #5 Joe upsetting #4 Bob and Ranking B has #4 Joe beating #5 Bob Example 1 is common enough (basically any upset), but none of the rankings services differ enough to score that result differently. Directionally, all ranking services are going to have Bob ranked above Joe. Example 2 happens, but is not common enough to separate the ranking services. I plan to score the actual seedings as its own ranking system. If seeding has a significant effect, it should do better than the rankings services. The scoring grid I'm using (see graphic above) is non-linear. I've created some cutoffs to reward getting a ranking close to the result, and punish as you get further away. R12 through 0-2 have a bigger step down because those are shared places with other individuals (there are 4 R12's and 8 R16's, etc). I also had a rule for no positive points for 0-2. These are the ranking services I will include: Flowrestling Intermat The Open Mat WrestleStat NCAA Seedings Track also has rankings Sent from my iPhone using Tapatalk Quote Share this post Link to post Share on other sites
Ching 180 Report post Posted March 7, 2018 Track also has rankings Sent from my iPhone using Tapatalk Thank you, I will add them also. Quote Share this post Link to post Share on other sites
Zebra 545 Report post Posted March 7, 2018 Wrestlestat is not a "ranking" but a career points earned. The rest are all useless in general and generally suck. Quote Share this post Link to post Share on other sites
Plasmodium 1,596 Report post Posted March 7, 2018 WIN magazine does a good job with the rankings Quote Share this post Link to post Share on other sites
Ching 180 Report post Posted March 7, 2018 WIN magazine does a good job with the rankings WIN has not updated their rankings since Dec 4th. Quote Share this post Link to post Share on other sites
Plasmodium 1,596 Report post Posted March 7, 2018 They do a weekly ranking that themat.com posts on Mondays. It's sort of hidden on their site because they want it to be behind a paywall. 1 Ching reacted to this Quote Share this post Link to post Share on other sites
Ching 180 Report post Posted March 7, 2018 Wrestlestat is not a "ranking" but a career points earned. The rest are all useless in general and generally suck. Regardless of what they are called, I do want to see how WrestleStat will do. Quote Share this post Link to post Share on other sites
Ching 180 Report post Posted March 7, 2018 They do a weekly ranking that themat.com posts on Mondays. It's sort of hidden on their site because they want it to be behind a paywall. Got it, I will add them to the list also. Quote Share this post Link to post Share on other sites
Show_Me 273 Report post Posted March 7, 2018 Thanks for doing this. I offer wonder about the differences between FLO, Intermat, Track & TOM when compared to NCAA Seeds. Your study should shed some light on this. Look forward to seeing your results. I would recommend doing this another year or two before drawing any conclusions as one season does not constitute a meaningful sample size. 1 Ching reacted to this Quote Share this post Link to post Share on other sites
Zebra 545 Report post Posted March 7, 2018 Regardless of what they are called, I do want to see how WrestleStat will do. And that's fine but I just wanted to make my point about the ranchers all sucking in general while Wrestlestat is not a ranking. Quote Share this post Link to post Share on other sites
potentiallydangerous 194 Report post Posted March 7, 2018 (edited) Gregg Henry was great at Wrestling Report but I think he stopped when Allen Brown died. Edited March 7, 2018 by potentiallydangerous Quote Share this post Link to post Share on other sites
Witherman 359 Report post Posted March 7, 2018 Oh boy I can’t wait for the results of this!!! Please keep wrestlestat included, I’d love to see how math and statistics (objectivity) stacks up to expert panels (subjectivity) Also - pending results of this I can’t want to hear FRLs response lol Quote Share this post Link to post Share on other sites
Husker_Du 569 Report post Posted March 7, 2018 rankings are not predictions. Quote Share this post Link to post Share on other sites
OrangeEffect 18 Report post Posted March 7, 2018 rankings are not predictions. It will be interesting anyway and I am looking forward to it. Quote Share this post Link to post Share on other sites
Ching 180 Report post Posted March 7, 2018 rankings are not predictions. I didn't call them predictions, but I do wonder if they are predictive. I assume a rankings process attempts to identify who is the best at a point in time. That just so happens to match up nicely with the tournament that objectively tells us who the best is at a point time. Quote Share this post Link to post Share on other sites
paboom 204 Report post Posted March 7, 2018 Regardless of what they are called, I do want to see how WrestleStat will do.They will have to remove Ashnault first. Sent from my SM-G950U using Tapatalk Quote Share this post Link to post Share on other sites
Zebra 545 Report post Posted March 7, 2018 Regardless of what they are called, I do want to see how WrestleStat will do. And if you do you should select the "Starters only" option since RS is included in the career points. 1 andegre reacted to this Quote Share this post Link to post Share on other sites
Husker_Du 569 Report post Posted March 7, 2018 agree on it being interesting. looking forward to the results. 1 Witherman reacted to this Quote Share this post Link to post Share on other sites
TobusRex 1,836 Report post Posted March 7, 2018 Sounds like a great idea, Ching! 1 andegre reacted to this Quote Share this post Link to post Share on other sites
mlbruem 138 Report post Posted March 8, 2018 Amateur Wrestling News also does rankings last update 3/5/18 please include them if possible. 1 Ching reacted to this Quote Share this post Link to post Share on other sites
Ching 180 Report post Posted March 8, 2018 Amateur Wrestling News also does rankings last update 3/5/18 please include them if possible. Will do, thanks. Quote Share this post Link to post Share on other sites
Jaroslav Hasek 1,888 Report post Posted March 8, 2018 this is cool, thank you for doing this. 1 Ching reacted to this Quote Share this post Link to post Share on other sites
Billyhoyle 1,988 Report post Posted March 8, 2018 I like this idea, but you should not word what you are doing as a measure of which service is best. Imagine this scenario: Myles Martin beats Bo Nickal in the finals. If a ranking service had Myles Martin ranked #1, it would get the most points. However, we all know this would be a ridiculous ranking to give him at the moment. Instead, simply stick to saying you are looking at which ranking service is most predictive. This does not necessarily it is a better ranking service. Maybe throw in SHP's dual impact rankings? He has some mystery formula to determine those. Also, a ranking based solely on win/loss percentage that ignores strength of schedule could be an interesting control to throw in there. 3 OrangeEffect, GoNotQuietly and SetonHallPirate reacted to this Quote Share this post Link to post Share on other sites