I have an annual hobby of taking the list of college players invited to the NFL Combine (http://www.nfl.com/combine/players), and noting what they were rated as recruits out of high school or junior college. I think the resulting data is very revealing about talent acquisition and development. I look at the Combine list as the best metric available for measuring individual collegiate talent. It has these positive traits:
1. It is objective. The list is developed by NFL scouts. Any biases for particular programs are diversified out by the numbers of scouts making inputs. The Combine invites All-Americans and no-name players from Division 2 that were unranked out of high school.
2. It discriminates. I mean the good definition of "discriminates" here. College football all-star ratings are dominated by multi-year starters at successful programs. Terrific players at low profile programs are often ignored. The Combine tries to move past all of that. They invite the guys they are interested in, no matter how wretched their program, and ignore those they don’t want.
3. It sets a constant "bar". Too often, when reviewing recruit classes five years later, success or failure is determined by counting lettermen and starters. The problem is that somebody has to start at North Texas, but that doesn’t make them a good player- they may just be the least worst player. Conversely, the guy backing up Darren McFadden might actually be pretty talented, even if he never had a chance to show it. The Combine list provides a measure that ignores depth charts and college program quality.
There are also problem with using the Combine list for rating the quality of college players:
1. The list is a pass/fail list. It does not rate or rank players. Sure, there are informal rankings- some players are so highly considered that they can set the terms of their workouts. Others had better do all of the drills or they will be sent home. Now, being a pass/fail test is not all bad. A highly touted 5-star recruit may not have become the big time star he expected, but can’t be said to be a bust if he gets invited to the Combine- he cleared a major hurdle on the way to being a pro. Meanwhile, a non-ranked recruit from a Division 2 school with an invite has made a great achievement.
2. The list focuses on NFL needs in talent, and ignores unique college needs. For example, Pat White was not invited to the Combine. Stephen McGee was. The Combine invites a bunch of kickers and deep snappers every year, too.
Overall, I think being invited to the Combine is the best gauge we have of whether a player was a difference-maker or not. That’s useful, because we can then measure how good the recruiting services are at rating recruits’ potential. Are the blue chip recruits really worth all the attention they get?
Yes, they are. The NFL invited 330 players to the 2009 Combine. 13 of them were 5-star recruits, from any of the 2004 (RS senior Bomar) thru 2006 (early entrants like Matt Stafford) classes. Over the last three years, 51 of the players at the Combine were 5-star recruits. Given that only about 30 – 35 players per year are rated as 5-stars, you can conclude that a 5-star recruit has a 50% chance of being invited to the Combine in the next few years. The 2009 Combine is inviting 66 players who were 4-star recruits. That represents about 25% of all 4-star recruits in a given year.
Damn. This post is long. It needs a picture. Here's a guy who transferred from Texas to get out of a depth chart logjam.
After three years of reviewing this data, I’m coming to the opinion that the recruiting services really don’t do as good of a job in differentiating between 2-star and 3-star talent. Here’s why- in 2004, Rivals rated 33 recruits as 5-stars, 239 as 4-stars, 1725 as 2-stars, and the rest were either 1-star (rare) or unranked. Of that class, 14 of the 5-stars were invited to a combine from 2006 – 2009 (42.4%), 60 of the 4-stars were invited (25.1%), 94 of the 3-stars were invited (15.5%), 94 of the 2-stars (5.45%), and 45 unranked recruits ended up at a combine. Yes, a smaller percentage of 2-stars make it to the NFL Combine than 3-stars. Given the numbers involved, shouldn’t the difference be larger if the rankers really knew what they were doing? Should the 45 unranked players really be that anonymous after high school? After all, those 45 guys (granted- about a fifth were kickers) had real NFL potential.
It seems clear to me that the data supports the idea that you want a bunch of 5-stars and 4-stars in your class. Some will point out that 50% of 5-stars and 75% of 4-stars wash out or have mediocre careers. I counter that by pointing out that 85% of 3-stars and 95% of 2-stars wash out or have mediocre careers.
I realize this is a little inexact. There will be some guys make the NFL as undrafted free agents who never went to the Combine. There always have been. That said, the odds are not with the guys who don’t go to the Combine. The Combine guys are the NFL equivalent of the 4-star and 5-star recruits. They have a significantly better chance of making the NFL than the guys who have to work their way up as undrafted free agents.
To me, the best measure of a recruiting class is how many difference makers it produced. The NFL Combine list is the best metric for counting elite contributors. As evidence, here is a list of the programs that produce the most NFL Combine participants for the last three years combined-
USC (28), LSU (25), Texas (21), OU (21), VT (20), Tenn (20), tOSU (19), Florida (19), Georgia (18), Clemson (17), and Michigan (17). These were the teams with the most talent the last few years. They either had the best teams (all of the MNC game teams for the last few years are represented), or had coaches fired for underperforming (Tennessee, Clemson, and Michigan). Beamer at VT doesn’t come under fire because his recruiting classes weren’t ranked as elite. They just turned out that way, which was nice (more on him later).
I have used this data in the past to develop my own recruit class ranking method. I use this formula:
(Expected Difference-makers) = (# of 5-stars)*.5 + (# of 4-stars)*.25 + (Remainder)*.1
It’s a simple formula. Basically, it ignores all qualitative assessment except for that which went into the Rivals rating. Texas’ 2003 class had (1) 5-star, (11) 4-stars, and (6) 3-stars. Per my formula, this class was expected to yield 3.85 Combine-worthy players. It actually produced 7 (Crowder, Sweed, Pittman, Hill, Griffin, Griffin, and Brown). This was a good outcome. The 2003 USC and OU classes were each expected to yield 4.8 Combine players. USC produced 12 and OU produced one. This shows the variability possible in recruiting classes. But still, consider what this tool does- it discards half of the 5-stars, ¾ of the 4-stars, and allows for a few of the lower rated players to succeed.
Usually, this tool is a good estimator of overall talent. It predicted that TAMU would have experienced elite talent comparable to Texas in 2006 and 2007 (7.45 predicted combine players from 2003 and 2004 classes; 7 actual Combine players). It predicted that LSU would be one of the most talented teams in 2007 and 2008.
Here’s how it rated the Big 12 south classes from 2009-
UT: 4.85
OU: 3.95
TAMU: 3.8
TT: 3.35
OSU: 3.25
BU : 3.15
Now, scouting ability is just one part of the process that makes a successful recruit class. The other aspect is, let’s say, "development". What happens with recruits in their 3 – 5 years on campus? Do they get stronger? Do they become technicians at their positions? Do they improve in ways that make them elite playmakers?
I don’t know. And really, I don’t care. To me, it doesn’t matter why a recruit class exceeds or falls short of expectations. It could be exceptional scouting, or exceptional "development". If the end result is more NFL-caliber players than reasonably could be expected, who can carp? This tool is a great way to predict a class’ quality on signing day, and to grade it years later.
Time for another picture. Here's a guy who never even played college ball.
I’ve got data from the past three combines. Most of the final results of the 2003 and 2004 recruiting classes are in, except for a handful of Utah recruits that have a couple more years of eligibility. I have enough data to answer some interesting questions-
1. Does the quality of recruits vary any from year to year? Yes, it does, among the elite players. Remember, when we’re looking at the NFL combine players, we’re looking at the right tail of the bell curve of talent. The 2003 recruit class sent 283 players to an NFL Combine (2006, 2007, 2008, or 2009). The 2004 recruit class sent 321 to date, with a few stragglers (like Jordan Shipley) still out there. That’s quite a difference. In other words, when choosing players to invite, the NFL scouts decided they like the 320th best player from the 2004 class more than the 285th best player from the 2003 class. If talent can vary that much, year to year, on a national level, it’s safe to assume it fluctuates even more on a state level.
2. Do any coaches show the ability to consistently outperform expectations with their recruit classes? Yes. Pete Carroll, Mack Brown, Joe Paterno, Nick Saban, and Lloyd Carr all outperformed expectations (yielding Combine-invited players) with each of their 2003 and 2004 classes. Bob Stoops’ 2003 class dramatically underperformed, and his 2004 class slightly overperformed. Franchione, Tressell, Richt, Tuberville, Meyer, Willingham, Bowden and Beamer overperformed one year and underperformed one year. Leach underperformed both years. Note to conspiracy theorists in Brazos County- if Texas recruits were rated higher than they deserve, the proof would be in the final outcomes- you would expect Texas to underperform its projections. It has done the opposite instead.
An interesting observation- when looking at the 2003-2004 classes combined, Brown, Carroll, Tressell, Richt, Carr, Paterno, Beamer, Carr, Saban, and Willinghham all out-performed expectations. This makes me think that the coaches at the flagship schools actually do get to be more discerning. They aren’t lying when they claim to recruit based on their own evaluations, rather than those of ratings services. This leads to…
3. Fans of major BCS programs should never fret when their school signs a 2-star recruit. A review of the data reveals that although a 2-star recruit has less chance than a 3-star recruit of becoming elite, the 2-star recruits from the major programs do about as well as the 3-stars, percentage-wise. I think the thing to remember here is that a major program never offers a 2-star because they ran out of 3-stars to recruit. Rather, they offer the 2-star in lieu of score of 3-stars they could sign.
4. Funny thing I’ve noted- I also track the Texans (played HS ball in Texas) invited to the Combine. In 2007, 40 Texans were invited. In 2008, 41 were invited. This year, 42 are invited. What does it mean that there is so little variability in Texans invited each year? I don’t know, but it looks interesting. Note that only about 12 of those Texans each year were ever on a Fab 55 list. Several are guys like Robert Myers (Utah State TE), or Johnnie Lee Higgins (UTEP WR)- unranked guys who went to small colleges with no fanfare, redshirted and played four years. The key is that they went to places where they could play. If I could give any advice to a HS player considering offers, it’s this- Go somewhere you will play. If you find yourself buried on a depth chart after a year or two, and you still want to pursue a football career- transfer. Every year at the Combine, there are 5 or so players who started at big programs, and finished at smaller ones. Last year, Joe Flacco was one (was behind Tyler Palko at Pitt).
5. Are there any positions which seem to be scouted worse? Yes. The two positions where an unranked player has the best chance of making it to the Combine are QB and WR. Over the last three years, 11 out of 61 QBs and 25 out of 147 WRs were not ranked as even 2-star prospects out of high school. The positions where the rankings matter the most? RB (5 unranked prospects out of 92 at the Combine the last three years) and DL (10 out of 154). OL is close (13 out of 158), but five of those 13 unranked OL prospects were guys from schools in the northwest who graduated high school seven years ago. The vast majority of the unranked guys who make it use redshirt seasons in their collegiate careers. The typical low-ranked NFL prospect is a WR that went to a non-power program, redshirted, and started three or four years.
6. To re-emphasize the correlation between recruit ranks and Combine invites one more time- let’s look again at the last three years of data. 1.3% of ranked recruits are 5-stars. 5.2% of the guys at the Combine were 5-stars. 11.6% of recruits were 4-stars or higher. 28.4% of Combine players were 4-stars or higher. 30% of recruits were ranked as 3-stars or higher and 60.4% of combine players were 3-stars or higher. 57% of recruits were 2-stars or higher, and 87.7% of Combine-invited players were 2-stars or higher as recruits. Convinced?
I hope this has helped you understand the odds in recruiting a little better. Thoughts?