clock menu more-arrow no yes

Filed under:

Is Harsin Really An Improvement?

New, 97 comments

bryan harsin

I have hope for Harsin, but he showed he's not the miracle worker we were all hoping for.

Like the effort. Just not impressed with the results. You'll see when I put the post together.

(Excerpts from my text messages to Sailor Ripley on November 25th.)

Just one day after Texas narrowly escaped Kyle Field with a bizarre win driven by defense, special teams, and the cosmic hand of Karma, I was convinced. I had taken a quick look at the statistic for a Cliff's Notes style judgment of offensive coordinators -- yards-per-play -- and determined that the 2011 Texas Longhorn offense was almost as putrid as its anemic 2010 counterpart. Despite a new coordinator, new position coaches, and an infusion of freshman playmaking talent, the Longhorns' offensive output had barely grown from 5.23 YPP to 5.34 YPP.

That's a meager 2% improvement over a season so inept that it caused Mack Brown to fire his best friend of 20 years.

Jesus, I thought. Harsin isn't any better than Greg Davis.

So I fired off a couple of brash texts to the owner of this esteemed publication, and set about collecting enough data to prove my surprising claim.

But it turns out that I was wrong. As much as I would love to, I can't back up my counterintuitive bravado. My initial back-of-the-envelope analysis, based on non-adjusted YPP, implied stagnation.* But the deeper I dove into more sophisticated (and fairer) statistics, the more impressive Harsin's offense looked compared to last season's. I now fully believe that Harsin has been a huge improvement over the previous regime. And I expect a very bright future, as the talent on hand matures and absorbs his system.

Now is the part where I tell you why. Your time is valuable and you can't be bothered with detailed statistical mumbo-jumbo. We understand. You, Barking Carnival reader, are a critical member of the Carl's Jr. / Spencer Gifts / Lane Bryant customer service team. So here's an executive summary of my argument that Harsin is the shit (or, at least, better than the last chump):

  1. Compared to 2010, the Longhorn offense improved slightly (5%) in terms of adjusted yards-per-play, which factors in the strength of the opposing defenses.
  2. The 2011 offense was far more efficient at its primary job: scoring points. An expected value analysis reveals that, if the 2010 offense was given the same number of drives, with the same starting field position, as the 2011 offense, it would have scored 13.5% fewer points in conference play.
  3. The statistical improvement over 2010 is even more impressive when you consider that: (a) in 2010, a disproportionate amount of Texas's offensive points were scored in garbage time, against unmotivated, conservative defenses; (b) the offense actually lost a number of experienced players from the 2010 team; and (c) just when the 2011 Texas offense was finding its legs, four of its most prolific players lost theirs. (Literally. Wait, no - figuratively!).

And now, the deets.

Well Adjusted Yards-Per-Play

My gut reaction after the A&M game was based on emotion and a single statistic: yards-per-play. YPP is my favorite basic statistic by which to judge an offense. It measures performance over the smallest measurable unit of offensive opportunity (the play), and thus provides the most granular measure of an offense's ability to accomplish its goals. But raw YPP numbers are deceptive, because they measure only absolute performance with no accounting for quality of opponent. As a result, comparisons based on YPP can be misleading. Just think about comparing UH's offense to Arkansas's based solely on flat YPP. Think the Cougs' stats might suffer if they had to face LSU and Alabama?

Fortunately, Barking Carnival statistician extraordinaire Huckleberry has solved this problem for us. According to his adjusted stats, which account for the strength of opposing defenses, here's how the 2010 and 2011 Texas offenses compare:

2010: 5.18 aYPP (76th nationally)

2011: 5.44 aYPP (46 64th nationally)

Harsin improved our aYPP by 5%. That doesn't seem incredibly impressive, but consider this. In terms of national ranking on aYPP, the Horns moved from well within the bottom half of D1 offenses to just barely outside the top third half. (NOTE: that's not nearly as impressive as it was before I corrected the error. Dyslexia bumped the offense 18 spots in the rankings!) As I'll discuss in more detail below, Harsin achieved this improvement with the football equivalent of duct tape and paperclips.

Unexpected Value

aYPP is a nice statistic. But it also has limitations. In particular, aYPP measures only the yardage gained. It does not directly measure the offense's ability to achieve its ultimate goal: to score points. Other statistics, such as points-per-play and points-per-drive fare better on this measure. But these statistics do not factor out the role special teams and defense play in setting up an offense for success or failure.

I prefer to judge an offense by looking at its expected value profile - a set of numbers representing the average points scored by an offense as a result of drives starting at various points on the field. Now, by "prefer," what I mean to say is that I would love to rely on this statistic, if only it was available. Unfortunately, it's not. I have to piece EV information together by laboriously cutting and pasting drive charts into Excel and then post-processing to filter out meaningless drives and transform the raw data into expected values.

The work is worth it, at least to me. Once two teams' respective EV profiles have been calculated, it is possible to compare apples to apples. By inserting the EV profile of Team A into the drive profile of Team B (and vice-versa), the two offenses can be compared on a level playing field. Basically, it becomes possible to answer the question "Would Team A's offense have done better than Team B's offense, if it played on Team B?"

The first step is to determine the teams' respective EV profiles. Before we get down into the weeds (too late?), a few notes are in order. First, there are some basic assumptions about special teams play built into the model. A touchdown is assigned 7 points. Missed PATs are ignored. So are 2- point conversions. Second, some assumptions that should be made about special teams play are not. A missed field goal yields 0 points, even if the miss is the fault of the kicker. A made field goal, regardless of distance, yields 3 points. I'd prefer to incorporate a model for expected value of a field goal attempt, based on field position, but haven't gotten around to this yet. Third, the model collects drives into starting field position ranges. Behind the 50 yard line, these ranges are in 10-yard increments. In order to capture (relatively) statistically significant data about favorable field position drives, which are rare, I created two ranges beyond the 50 (50-31, and 30-goal line). Obviously, this sacrifices precision for the sake of statistical significance. Given the statistically scant number of total drives in a season, this trade-off turns out to be a fundamental limitation of the method.

The following chart presents a comparison of the 2010 and 2011 offenses' EV profiles during Big 12 play:

Just looking at the chart, it should be obvious that the 2011 offense was more efficient at scoring points compared to the 2010 offense. With only one exception, the 2011 offense was superior - often by a large margin - from every range of starting field position. The one saving grace for the 2010 squad is that the one range in which is was superior to the 2011 team (drives initiating somewhere between the 20 and 29 yard lines) happens to be the range in which more drives start than any other. Approximately 35% of Texas's drives started between the 20 and 29 yardlines in both 2010 and 2011.

And now for the money shot. The 2011 offense scored 210 points in Big 12 play (subject to the assumptions discussed above). When the 2010 offense is inserted into the 2011 offense's drive profile, its expected total score is only 181.6 points. In other words, had Davis (and his offense) stuck around for this season, Texas would have been expected to score 13.5% fewer points on offense. That works out to about a 3 points-per-game difference.

Harsin's ability to outperform last year's offense may seem slight, but I believe it is significant. Especially in light of the following considerations.

Garbage In? Garbage Out!

If you thought the 2010 Longhorn offense's statistics were bad, consider this: those putrid stats are bloated by meaningless scores late in already-decided games. In its 2010 Big 12 losses, 40% of Texas's offensive points were scored in the 4th quarter. Another 31% were scored in the 3rd. Essentially, the 2010 Longhorns were racking up points after the other team had begun celebrating. Remember Iowa State, Kansas State, etc.?

Removing garbage time drives from the analysis yields the following EV profile:

After removing garbage time drives from the 2010 EV profile and rerunning the plug-and-play 2011 prediction analysis, the result is shocking. If the 2010 "game still in doubt" offense was substituted for Harsin's 2011 offense, the team would be expected to score a mere 130.4 points in its nine conference games. At 210 points, Harsin's squad was around 60% better than the performance we should have expected with Greg Davis at the helm.

Given the number of inexperienced young players filling out the starting lineup and the injury swarm that descended on the skill position players, Harsin's performance astounds.

Heeeeeeeeeeeere They Come to Snuff the Roster

In 2011, Harsin faced two personnel hurdles. The first was Greg Davis's Old Mother Hubbard act. Harsin inherited almost nothing on offense. In fact, I'd argue that the 2010 team had more experienced talent to work with. But, instead, I'll just let this handy chart argue on my behalf:

[table id=1 /]

At a few positions, Harsin had more to work with than Davis did. Running back jumps out at me as the most glaring example. Right guard is another. But at the two most crucial positions on the field, QB and LT, Harsin had basically nothing to work with. And we replaced a decent, senior-heavy group of receivers with a true freshman phenom and a couple of underachievers.

The second hurdle Harsin had to overcome was an improbable rash of injuries to his best players. Through the Texas Tech game, Fozzy Whittaker, Malcolm Brown, Joe Bergeron, and Jaxson Shipley collectively accounted for approximately 55% of the Horns' total offense. Then, over a two-game period, they all disappeared from the lineup.

Losing that much firepower from a very thin offensive roster was a huge hit and a big reason for the drop-off in offensive performance late in the season. The offense steadily improved throughout its Big 12 schedule, hitting a peak against Tech (gaining 52% above Tech's average YPP allowed), before falling off a cliff and, in each of the next three games, gaining a remarkably consistent 28% below our opponent's average YPP allowed. Harsin basically lost everything he'd built over the course of the season, and limped to the finish line.


Brian Harsin has done more in 2011 with less than Greg Davis had in 2010. At times during the season, it felt like Harsin wasn't achieving anything worth noting. He's clearly not a miracle worker (that's the one thing I got right in my texts to Sailor Ripley). But when viewed in context, Harsin has done a pretty good job manufacturing points out of thin air. He really had very little to work with this season. I'm anxious to see what he can do with some experienced talent on the roster.

*Including the Baylor game, the team improved to 5.46 YPP. For those scoring at home, that's a 4.4% improvement over 2010.