Comparing Rivals rankings to Sagarin rankings

[AndyDufresne]

Comparing Rivals Rankings to Sagarin Rankings

 

For this particular analysis, I decided to average the Rivals rankings for every division IA team from 2002-2008. I also averaged the 2006-2008 Sagarin rankings for each team. It was necessary to use the Sagarin poll because each team is ranked every year.

 

Here are the top 25 recruiting teams and the the 7 year class average:

 

1. USC - 4.1

2. Georgia - 6.4

3. Florida - 7.1

4. Oklahoma - 7.3

5. LSU - 8.9

6. Miami - 8.9

7. Florida State - 9.0

8. Texas - 10.0

9. Michigan - 11.3

10. Auburn - 12.6

11. Tennessee - 13.7

12. Ohio State - 14.0

13. Notre Dame - 18.0

14. South Carolina - 18.4

15. Alabama - 19.1

16. Texas A&M - 20.0

17. UCLA - 25.0

18. Nebraska - 25.3

19. California - 26.4

20. Oregon - 27.7

21. Arkansas - 27.7

22. Arizona State - 28.1

23. Oklahoma State - 28.3

24. Maryland - 28.9

24. Clemson - 28.9

 

Here are the top 25 teams according to the Sagarin rankings and the 3 year average:

 

1. USC - 2.7

2. Florida - 4.0

3. Oklahoma - 8.0

4. LSU - 8.3

5. Ohio State - 9.7

6. Texas - 11.7

7. Georgia - 13.0

7. West Virginia - 13.0

9. Virginia Tech - 16.3

10. Oregon State - 16.3

11. Penn State - 17.3

12. California - 17.7

13. Oregon - 18.0

14. BYU - 20.7

15. Boise State - 22.0

16. Missouri - 22.3

17. Texas Tech - 23.0

18. Boston College - 25.3

19. Wake Forest - 26.7

20. Clemson - 27.0

21. TCU - 27.7

22. Auburn - 28.0

23. Utah - 28.3

24. Rutgers - 29.0

25. Tennessee - 29.3

25. Oklahoma State - 29.3

(38. Nebraska - 38.0)

 

As you can see, for the top 7 teams in the Sagarin rankings (including Georgia but not West Virginia) there is a high correlation with the Rivals rankings In fact, the average absolute value of the differential between the Sagarin and Rivals ranking is 2.6 which means that each team's average Rivals ranking was within 2.6 places of the average Sagarin ranking. After that, things get quite a bit murkier. For the remainder of the top 25, the average absolute value differential is 24.4. The average differential is -22.5, which means that on average, teams 8 through 25 performed better in the Sagarin poll than the recruiting rankings by 22.5 places. This is quite a differential. Even removing the non-BCS teams from the top 25 results in an average differential of -13.8 for positions 8-25.

 

Looking at all BCS teams only, the average absolute value differential is 20.4. This means that on average BCS teams average Sagarin finish was either 20.4 spots better or 20.4 spots worse than the average recruiting ranking. Keep in mind that these numbers are certainly skewed by non-BCS conference teams, even though the Sagarin computer rankings account for strength of schedule.

 

Here are the big BCS winners (Sagarin rankings outperformed recruiting rankings) and the average differential:

 

1. Cincinnati - 57.1

2. Wake Forest - 47.0

3. Connecticut - 31.9

4. Oregon State - 29.1

5. South Florida - 28.0

6. Kentucky - 27.8

7. West Virginia - 26.7

8. Vanderbilt - 22.9

9. Rutgers - 19.6

10. Texas Tech - 17.6

11. Penn State - 15.0

12. Kansas - 14.0

13. Virginia Tech - 13.1

14. Boston College - 11.2

15. Missouri - 10.5

 

And the big losers:

1. Washington - 49.0

2. Iowa State - 47.1

3. Miami - 44.1

4. Syracuse - 43.8

5. Colorado - 43.1

6. Texas A&M - 36.7

6. Notre Dame - 36.7

8. Duke - 36.0

9. North Carolina - 34.4

10. Mississippi State - 33.9

11. Illinois - 33.7

12. Minnesota - 33.3

13. NC State - 33.1

14. Michigan - 29.7

15. Kansas State - 28.6

 

So as you can see, the evidence from this rather limited study does show that at the very top (approximately the top 6%), the recruiting rankings have been a very good indicator of future performance during the last 3 years. After that, even the most diehard recruiting junkie would have to admit that the correlation is very weak.

[Overland Park Husker]

How many of these threads are we going to have?

[AndyDufresne]

How many of these threads are we going to have?

 

Apologies, but I didn't see anything like this currently posted. I'm not trying to argue for or against the recruiting services or take sides, just showing the results of a study over a period of time.

[huskerswrkhavoc]

I don't think I've seen a thread like this.. That is actually pretty good analysis taking the personal emotions out of any possible comparisons.. Good 1st post, welcome to the board, and thanks for the info..

[caveman99]

Good post, I like the analysis.

[robsker]

How many of these threads are we going to have?

 

Apologies, but I didn't see anything like this currently posted. I'm not trying to argue for or against the recruiting services or take sides, just showing the results of a study over a period of time.

 

 

No need to apologize. In fact, I sincerely thank you for the time, effort and energy you put into this analysis. It is an outstanding contribution and is very interesting in terms of implications.

 

This seems to support what is obvious. To consistently be an elite program, a top 6% program as you say --- to be numbered along with USC, Florida --- the absolute top --- you have to recruit the absolute top athletes. All the coaching and all the effort and all the intangibles will not be sufficient to CONSISTENTLY compete with the biggest of the big dogs. To play with USC and Florida you must have comparable recruiting prowess. If you can not recruit like a big dog --- you will not be a big dog. period. At least not consistently.

 

Also from your data, it is clear that after the top big dogs, the rest of the nation gets athletes of sufficiently similar quality that the effort, coaching, the intangibles and other criteria make the recruiting correlation essentially non-correlative. That is, if you cannot recruit with the big dogs you will not be a big dog --- but you can still be good because after the big dogs take the top athletes, what is left over for others (like NU) is distributed such that your coaching, intangibles, heart, effort, etc. can enable you to compete well with anyone other than the big dogs --- even if they have slightly higher rated recruits.

 

It is really simple --- at the extremes of recruiting --- when you are at the absolute tops --- you'll be at the absolute top of the BCS title hunt. If you recruit at the other extreme --- really badly --- you can not compete no matter what the heart, coaching is. In between --- where most teams reside --- like NU --- recruiting ranking is numbered among an array of other contributors and anything can happen --- here coaching and heart, etc. is largely what dictates the pecking order.

[AndyDufresne]

How many of these threads are we going to have?

 

Apologies, but I didn't see anything like this currently posted. I'm not trying to argue for or against the recruiting services or take sides, just showing the results of a study over a period of time.

 

 

No need to apologize. In fact, I sincerely thank you for the time, effort and energy you put into this analysis. It is an outstanding contribution and is very interesting in terms of implications.

 

This seems to support what is obvious. To consistently be an elite program, a top 6% program as you say --- to be numbered along with USC, Florida --- the absolute top --- you have to recruit the absolute top athletes. All the coaching and all the effort and all the intangibles will not be sufficient to CONSISTENTLY compete with the biggest of the big dogs. To play with USC and Florida you must have comparable recruiting prowess. If you can not recruit like a big dog --- you will not be a big dog. period. At least not consistently.

 

Also from your data, it is clear that after the top big dogs, the rest of the nation gets athletes of sufficiently similar quality that the effort, coaching, the intangibles and other criteria make the recruiting correlation essentially non-correlative. That is, if you cannot recruit with the big dogs you will not be a big dog --- but you can still be good because after the big dogs take the top athletes, what is left over for others (like NU) is distributed such that your coaching, intangibles, heart, effort, etc. can enable you to compete well with anyone other than the big dogs --- even if they have slightly higher rated recruits.

 

It is really simple --- at the extremes of recruiting --- when you are at the absolute tops --- you'll be at the absolute top of the BCS title hunt. If you recruit at the other extreme --- really badly --- you can not compete no matter what the heart, coaching is. In between --- where most teams reside --- like NU --- recruiting ranking is numbered among an array of other contributors and anything can happen --- here coaching and heart, etc. is largely what dictates the pecking order.

 

You're very welcome. Based on the sample and the numbers, I think that your deductions are spot on. Perhaps a larger sample size would provide different results, but unfortunately Rivals database only goes back to 2002.

[dylan]

what your analysis fails to take into account is that the ranking of recruits by the recruiting services is largely influenced by the schools recruiting them. in other words, if a top team (say texas or USC) offers a kid, that is then used as justification for a high recruiting ranking for the kid (and hence the school), ensuring that the top glamor teams always have top recruiting rankings. this partially accounts for how teams that consistently get supposedly great recruits can manage to fail (see ND or USC and texas prior to their current coaching staffs) and teams that live off of supposedly middling recruits can succeed (VT, TCU, wake, boise, etc.). as teams move up in national esteem, so does the recruiting services view of their recruits.

 

to a lesser extent, this can be seen with nebraska recruits as well. a lot of our recruits over the years that were considered somewhat unknown or borderline by the recruiting services prior to our offer, will later get a bump in rankings. again, our program prestige plays a role in this, but then so does the marketing aspect of keeping an active and profitable fan base shelling out cash for news on hot prospects.

 

i'll add that i do think it's necessary that you get some difference making athletes in your program to be a top school. i just don't think that companies out to make a buck are an ideal way to judge the success of that effort.

[AndyDufresne]

what your analysis fails to take into account is that the ranking of recruits by the recruiting services is largely influenced by the schools recruiting them. in other words, if a top team (say texas or USC) offers a kid, that is then used as justification for a high recruiting ranking for the kid (and hence the school), ensuring that the top glamor teams always have top recruiting rankings. this partially accounts for how teams that consistently get supposedly great recruits can manage to fail (see ND or USC and texas prior to their current coaching staffs) and teams that live off of supposedly middling recruits can succeed (VT, TCU, wake, boise, etc.). as teams move up in national esteem, so does the recruiting services view of their recruits.

 

to a lesser extent, this can be seen with nebraska recruits as well. a lot of our recruits over the years that were considered somewhat unknown or borderline by the recruiting services prior to our offer, will later get a bump in rankings. again, our program prestige plays a role in this, but then so does the marketing aspect of keeping an active and profitable fan base shelling out cash for news on hot prospects.

 

i'll add that i do think it's necessary that you get some difference making athletes in your program to be a top school. i just don't think that companies out to make a buck are an ideal way to judge the success of that effort.

 

I think that you are discounting the importance of talent and the methods the recruiting services use to rate it.

 

I think it is fairly safe to assume that the best evaluators of high school talent work for the highest bidders. While I obviously don't have payroll data available to me, I would guess that the biggest paychecks come from the top teams and not Rivals or Scout. Basing at least part of their player ratings on offers only seems like a sound business move to me. If Rivals or Scout lose whatever credibility that they do have with their subscribers, they will not remain in business. It is within their own best interests to publish the most accurate ratings that they can by using whatever information is available to them.

 

As for talent, I like to think of talent as raw material. As a bad analogy, I will use mountain bikes. An expert frame builder can turn aluminum into a very good bike frame, but can really create something special with titanium. Someone who doesn't know what they are doing is going to end up with a shoddy finished product no matter what material they use.

 

I do see your point though and am certainly intrigued by some of the implications. I think too often that we who follow recruiting like to lump recruits into specific star categories and treat one 2-star player just like the next. We will somewhat discount a specific recruit because they are a 2-star and speculate on their future based on what we know about the entire pool of 2-star recruits. Based on this study, http://www.omninerd.com/articles/Follow_up...l_All_Americans, 30% of a compiled list of 2007 All-Americans (excluding kickers and punters) were 2-stars. Sure, the percentage chance of any random 2-star attaining All-American status is much lower than that of a 5 star, but are there certain teams that have a knack for identifying these future impact players, thus increasing the percentages exponentially? Not every 2-star is created equally.

 

On the opposite end of the spectrum, some say that Bill Callahan had a knack for recruiting the "wrong" 4-stars. If some of these same players had gone elsewhere, how would this have changed their fates? Florida State's 2004 recruiting class had 12 players with Florida offers. What if these players would have signed with Florida instead? Would Urban Meyer have been able to win 2 out of the last 3 national championships with these players? All of these are obviously impossible questions to answer, but it certainly makes for interesting discussion.

[HuskerJen]

AndyDufresne,

 

Great posts! I agree and disagree with you all at once.

 

I certainly agree that to be successful in college football a team needs to recruit the best players. The current schools dujour: USC, Oklahoma, Texas and Florida all recruit some of the best talent in the nation. It's no coincidence that those schools all are smack in the middle of, or very proximal to, the most fertile recruiting hotbeds this nation has to offer. Combine great talent with superior and established coaching staffs, which those schools have, and over time you will see sustained top level success.

 

Where I disagree with you is on the number of elite recruits a given school needs. Certain people think a team needs to be 3 deep with 4 and 5 's just to have talent. I think that a school merely needs to have difference makers at a few positions. For ease and convenience I'll talk about Nebraska under Tom Osborne. Even during the glory years, '90's, there were times when the opposing team, recruiting wise, was flat out better across the board than Nebraska, to wit Florida, Tennessee, Miami, etc. And yet, Nebraska beat all those teams to win national championships.

 

Basically on offense you need a difference maker at QB, RB and have maybe one go-to WR. On defense, you need a top notch DT, LB and one DB-preferably a CB who can lock down the other team's best WR. Then, as long as the other players are solid and the team is solid in the kicking game, that team should be able to notch 9-10 wins consistently. That's the way the Huskers did it under TO and that's the way it can still be done.

 

Plus, these recruiting rankings don't consider: 1) what type of offense or defense the player's team ran in high school and 2) does that player "fit" with the college he's going to. For example, Marlon Lucky versus Roy Helu. In high school Lucky's team ran a more traditional I-formation based offense while Helu's squad was more of a shotgun spread set. At Nebraska, Lucky seemed to run the best when he was in the I and struggled when in the shotgun. Helu on the other hand, thrived while in the shotgun and eventually supplanted Lucky as the starter once OCSW made the decision to move to a more shotgun spread type of offense.

 

In conclusion, stars do matter absolutely. But, stars only get a team so far. I'll take a three star kid who will bust his arse on every play and put forth maximum effort in to the off-season program before I'll take a lazy, coast on talent, five star.

[dylan]

 

I think that you are discounting the importance of talent and the methods the recruiting services use to rate it.

 

I think it is fairly safe to assume that the best evaluators of high school talent work for the highest bidders. While I obviously don't have payroll data available to me, I would guess that the biggest paychecks come from the top teams and not Rivals or Scout. Basing at least part of their player ratings on offers only seems like a sound business move to me. If Rivals or Scout lose whatever credibility that they do have with their subscribers, they will not remain in business. It is within their own best interests to publish the most accurate ratings that they can by using whatever information is available to them.

 

talent is the single most important factor in football, but only if you are able to harness it within your system. that requires both coaching ability, as well as an eye for which talented players will actually fit your system (from a skills standpoint as well as from the player "buying in").

 

now, recruiting services rate the kids based upon pure "talent", largely discounting things like fit, attitude, etc. that's fine, and we can address their efforts at judging solely that "talent", but it does lessen their value in determining which teams were "successful" in recruiting. it's lessened even more when you consider their bias for top teams.

 

while i don't agree that the free market for talented recruiters/assistants is as efficient as you seem to think it is, i do think that offers is one of the best ways to determine how good a recruit is. i certainly don't have any faith in the ability of these marketing/recruiting sites to break down film (other than for the adrian petersons and aaron greens of the world). it is a problem though to use the circular logic that a kid must be good if a top recruiting team recruits him. one of the reasons that this is generally effective though is that the top teams DO get a lot of their top choices. with those recruiting hauls, having just some of the kids live up to the hype will often allow them to stay near the top. as was pointed out above, you don't need a whole line up of difference makers to be a top team. you just can't afford glaring weaknesses.

 

finally, i agree that it is in the services best interests to be accurate, but i strongly disagree that their sustainability is threatened by being inaccurate. first and foremost they are "news" services and people (to include myself to some extent) obviously clamor for that "news". credibility is way down the list of their business needs. i don't know if it's still the case, but it used to be that rivals folks would talk about openly lobbying for extra stars.

 

most of their followers just aren't going to tune them out over weak evaluations, and those fans will be easily replaced if they do by all of the fans that grow up with EA sports games emphasizing recruiting ratings and updates as a primary part of the sport. besides, it isn't the rating services that are considered busts when a kid doesn't live up to the hype they assigned him.

[Ohio Pete]

It is widely known that services in the 80s overrated Notre Dame and under rated Miami because ND sold subscriptions. Do they still do that? Maybe to some extent and maybe they get lazy, but there are definitely recruits who get boosted in the ratings apparently based on offers.