Find a Coach by the Numbers, Part III

Before we dig into the numbers, allow me a minor indulgence:

Thank you, Mack Brown.

We in the BC community have been consistent critics of Mack's work for the last few years. But at least on my part, no one should mistake the consistency of my criticism for the entirety of my opinion.

If y'all read that Bryan Curtis essay in Grantland to which Sailor linked...oh man, what a beautiful piece of sportswriting. Read it if you haven't yet. I'll wait.

I'm the same age as Curtis, a little older maybe. I remember the McWilliams years, the Mackovic years.

We were underdogs then - maybe not always in a literal sense, but certainly in regard to expectations. Ironically enough, it was that underdog status that first drew me to the Longhorns. I liked to root against the sneering strutting brutes to the north and east - the Boz, the Wrecking Crew. But it wasn't always very fun, expecting my boys to get regularly kicked in the pants by Switzer's OU and Sherrill's A&M, with another routine gunt punt (or two, or three) every season by the likes of Arky, Tech and Houston.

That dynamic changed when Mack Brown arrived. Mack may not be the best X's and O's guy out there, but he did accomplish something truly remarkable at Texas: he convinced our fans and the general public to raise their regard for the Texas program. We're a sleeping giant, he said, and it's time to wake up.

And he asked my generation - the recruits, the players, the students when he arrived - to completely reinvent our opinions, since we had no memories of a dominant Texas. And everyone bought in. Suddenly Texas' natural advantages kicked in, and though the coaching was never what I'd call "brilliant" we still won a bunch of games, including a title. Incredible. And more importantly, fun.

But...

...within Curtis' warm memories of an ascendant Texas, he's making an implicit argument: Mack Brown is losing his job because he's too nice. Goshdurn it, Mr. Football can't help it if Saban's greyshirting half his classes and Stoops is wearing the tread off his kids' tires.

That's a neat story - and not wholly mistaken - but it's way too simplistic.

I can buy Mack Brown as an upstanding guy. But let's not make that out to be a work handicap. Lawful Good may be Mack's preferred moral alignment but it's also a smart public stance to take at a flagship program in one of the Big Three football states.

The advantage at schools like Oklahoma or Alabama is different: those fans and those schools will push the rules to their limit and occasionally beyond, because if they try to keep up with the top Texas or Florida teams on even terms, their natural disadvantages become fixed.

So sure, let's embrace the upstanding self-respecting rule-abiding character as part of Mack's legacy, and part of Texas Football.

But the cold truth is, Mack Brown is retiring due to what Curtis handwavingly calls "football things". And there's no reason that a Texas coach can't be a decent human being while doing the "football things" well.

As reference, I point you to every article this site has produced for lo, these four years.

Last year our defense bit the turf repeatedly against the run game. That performance ruined the season. Is that because our defenders were too busy curtsying, laying down their capes, heroically refusing to affront the honour of the enemy in battle? First glance at the game film says "maybe" but the BC record of analysis shouts "no". They just weren't prepared to do the job.

The Texas coaching gig comes down to this: find able kids, bring them to Texas and prepare them to win games. Mack's staff has been doing poorly in at least one aspect of that job every year for six, seven years running. Given ample opportunity to fix the problems, he instead showed that he doesn't know how. So it's time to shake hands, thank him for the memories, and move on.

Thank you, Mack. Sincerely and gratefully.

Moving on...

CxT and the Replacement Suitability Index

Here are my last two pieces in this series. Check them out if you need a primer:

Find a Coach by the Numbers, Part II

Find a Coach by the Numbers, Part I

In a nutshell, here's what I've done so far:

• Following RedmondLonghorn's lead (Part I | Part II), I start with the general assumption:

Team Quality = Team Talent +/- "Coaching Effect" (the latter includes all environmental effects, including measurement error)

• I compiled two Talent (Rivals, Scout) and two Quality (F/+, Sagarin Predictor) components, rescaled them all from 40-100, and then averaged them to create Talent ("T") and Quality ("Q") measures for each FBS team/season from 2005-2013;
• Then I find the difference between the Quality and Talent scores for each team/season in the dataset, and rescale that range of raw Coaching Effect numbers ("C") to the same 40-100 range;
• Finally, I took the C and T measures - whose unscaled versions are added to make Q - and multiplied the scaled versions together to make what I call "CxT". Scaling the factors first ensures that both measures receive equal consideration, and multiplying the factors together rather than adding them ensures that the best scores go to coaches with decent C and T scores, rather than one or the other. Of these four measures, CxT should be the most appropriate for evaluating a coaching candidate for a top job like Texas.

Since writing Part II, I've made a few significant changes to the analysis. Smaller changes include:

• Re-scaling the CxT measure so it has the same distribution as the other variables, i.e., has the highest score at 100 and the lowest score at 40, with the median score around 70ish.
• Updating the Q metric (and hence the C and CxT measures) to include team performance data through the end of the 2013 regular season.
• Lots of structural fixes so that the spreadsheet will be easier to update moving forward.

However, the main change to the analysis is the introduction of a new element: the Replacement Suitability Index.

The RSI is intended to solve the confusion surrounding how to evaluate "C".

C is simply a mathematical remainder that describes overachievement or underachievement of Q related to T. Problem is, coaches with low T scores have little room to underachieve, and top talent doesn't need to overachieve to be successful. So you can't readily compare C scores between candidates unless their T scores are already similar.

Those discussions in comments lead me to one fruitful insight, however: while CxT may the best of the four measures for evaluating candidates for top coaching positions, C is actually the best map for success for a coaching position near the lowest rung. That insight led to the creation of the Replacement Suitability Index.

What is the Replacement Suitability Index? It's a combination of the CxT score and C score.

However, it's not a static number - you have to provide variable input, in this case a T score for the coach for whom you're trying to find a replacement.

Feed the RSI the highest possible T score (100), and it returns the CxT score. Feed it the lowest (40), and it returns the C score. For all the T scores in between, it combines the CxT and C scores in a very simple, linear fashion; a T score of 85 should have an RSI that's 75% CxT, a T score around 70 should produce a RSI that's about 50% CxT, a T score of 65 should return an RSI that's about 75% C.

Another thing the RSI gave me a chance to do is to weight the cumulative career and single-tenure numbers to make them more biased toward recent performance. Those CxT and C measures treat every year equally; for a final metric I wanted something that would give marginally more weight to recent performance.

Let's look at the RSI in action by evaluating the single-school coaching tenures of a handful of possible replacements for Mack Brown (average T score: 86.52):

By comparison, here is the exact same list, but with the RSI evaluated for Art Briles' Baylor tenure (average T score: 68.58):

Note that when comparing the Briles RSI to the Brown RSI, the candidates from major programs tend not to fall that much while the guys who win at little schools tend to make bigger jumps. That's the conceit of the analysis at play: big-school successes are presumed to be able to handle smaller schools, while it's a little more unsure the other way around.

So when looking at the big school openings, small school coaches get discounted; as the talent level associated with the job opening drops, that discount dwindles.

Of course, Baylor wouldn't have access to all those big school candidates if Briles stepped down. Here's the list of Baylor-appropriate candidates if you take out all the schools that have greater or comparable talent, as well as the other coaches who for one reason or another aren't applicable or realistic:

Not coincidentally, we have DeRuyter - a guy I think should be a dark-horse for the Texas job - and eight guys who collectively compose the remaining names on my "Ten Guys to Keep an Eye On" list.

Whazza? A list on the Internet? Indeed. I mentioned it in prior comment threads. The other two members are Bryan Harsin and Dave Clawson, who've taken jobs at Boise State and Wake Forest since I last mentioned it.

Doc Holliday's inclusion is a little tongue-in-cheek; he's a old West Virginny boy who'd probably be pleased to stay at Marshall as long as they let him. So if I were consulting with a Baylor-ish team, I'd suggest replacing Holliday on the list with either Jim McElwain, the former Saban OC who's now the HC at Colorado State and seems to be turning things around, or Jeff Quinn, the HC at Buffalo.

Or, if Baylor had the gumption and complete lack of standards I think they do: Bobby Petrino.

As in previous Find a Coach threads, if you have any graphic requests leave 'em in comments.

A copy of the new spreadsheet can be downloaded HERE.

A New Project: The Talent Megadatabase

One final word before I finish about where this is headed. The biggest limitation of this approach is that it's stupid. Not "stupid" as in dumb, but "stupid" as in not discerning.

Most of that fault lies with the Talent calculation. It assumes a four-year rolling average of recruiting scores is a decent proxy for team talent. It ignores redshirts, injury, attrition, scheme, player development, S&C, depth chart weakness, and the outsized effect of transcendental players at key positions. I believe all of these things are pretty consequential issues and if this system were truly robust it'd account for them. And if you COULD account for them, you'd have a system that not only told you something about "coaching" but more specifically "scouting", "recruiting", "developing", and maybe even "scheming". This is completely doable...in theory.

Problem is, there's no repository of player-level data good enough to support that kind of analysis. At least, nothing like what we have available for the NFL, where fantasy football has ripped the lid open on player-level statistical analysis.

Instead we have a bunch of analysis on NCAA team performance and very little about individual players. The NCAA has some rosters and box scores, the recruiting sites have some stuff and so does cfbstats.com and the College Football Reference. That's pretty much it and every source by itself is pretty incomplete. And the cfbstats/CFR material is the only stuff that's remotely user-friendly; the rest of it needs to be scraped off of websites, cleaned, and reassembled before we can even find out how useful it is.

So that's what I'm doing. It'll take awhile I'm guessing before I have anything too insightful to work with, but here's the booty from my first scraping expedition: a spreadsheet containing all the tabled player info that's available for free from one of the major recruiting databases.

Want to know how many 3-star players Texas has signed since 2009? Here ya go:

Need to know how may A&M Consolidated High School players have received FBS schollies since 2002? My treat:

(yes, the database includes high school names, and city, and state; I cut it off so you can't see it in the graphic)

Have a desperate urge to know how many players have either a J-apostrophe at the beginning of their first names, or call themselves "Fudge"? Take a gander:

(yes, in 2012 Tulane signed a QB named "Fudge Van Hooser", and he runs a sub-4.4 forty)

All this and more - 32,141 players, 16 columns of info, over half a million datapoints - can be downloaded HERE. If you have any questions about my compilation methods or whatnot, ask me in comments.

Next up, I'd like to add the other major recruiting scores, plus for each player at a minimum:

• NCAA games played,
• matching side coordinator,
• whether the player was eventually nominated for or won conference/national awards, and
• NFL draft/FA info, if applicable.

Those can all be obtained but it'll take some time to write the scripts and clean the data. This will give us some additional insight into player development and also give us a way to evaluate coordinator performance. In addition I'd love to find data for player game starts, snap counts, position coach names, injuries and transfers. Just simple lists will do, but I can't find any web sources for more than one team.

Regardless, during this offseason I'd just like to get my hands on everything I can and put it in a single database and worry about how to use it when I'm done. So if you find any of this stuff available on the Internet, let me know.