PLAYING THE NUMBERS GAME: Measuring Running Back Efficiency
by Peter Bean on Sep 12, 2007 6:27 AM CDT
in Texas Longhorns Football
Email
PrintBillyzane returns with another of his weekly columns looking at the numbers of the game we love. --PB--
 Obligatory picture of Mike Leach. Pirates! |
Claiming that total rushing yards by a team or by an individual back is a dispositive indication of the strength of a running back or rushing attack is ludicrous, and any learned football fan will tell you so. If a team runs every down of every game, of course they will end up with more net rushing yards than, say, Texas Tech. But does that mean that this run-only team is a better at rushing than our pirate friends to the West? Not necessarily.
We need to know more than just raw rushing statistics. We need to know something about the efficiency of that rushing. So what do we have? The most common efficiency statistic is, quite predictably, the easiest to calculate: yards per carry. Divide total yards by number of carries and voila! Efficiency! And sure, this is a lot better than total yards because it doesn’t care how many carries you get per game – only what you do when you get those carries.
And that is where most commentaries on the efficiency of running backs end. But is that enough?
Click here to continue reading this entry.
You know me well enough by now to realize that I believe the answer is no. Why not? Because some yards are inherently more important than other yards. For instance, if it’s 4th down and 2, and a running back rushes for 2 yards, that’s a pretty successful run. But if in that same situation a running back rushes for 4 yards, is it twice as important as rushing for 2 yards? Of course not. It’s certainly marginally better, but on fourth down, all that really matters is getting a first down, which 2 yards accomplishes just as well as 4 yards. But on first down and 10, 4 yards is much more valuable than 2 yards. Yards have different values depending on the situation.
Armed with all these seemingly brilliant thoughts, I decided to construct my own equally brilliant metric to accurately determine the efficiency of a running back (or team rushing attack) based on the principle that not every yard is equal in value. In an effort to figure out a way of determining the values to be assigned to the yards gained in certain situations, I did a google search to see if anyone had done something remotely similar to this before.
Much to my dismay, I found the oldest tenet of my intellectual life proved yet again: everything I ever wanted to do has already been done. I subsequently wrote in my notes: "Football Outsiders knows fucking everything." Here’s a link to an article of theirs titled Introducing Running Back Success Rate and dated 8/16/04. Three freaking years ago! What a bunch of jerks. [As a side note, and hurt feelings aside, Football Outsiders is the absolute most cutting edge football website there ever was or ever will be. It’s geared mostly towards the NFL, but if you care at all about understanding the game beyond the plays themselves, you can’t NOT read it.]
The Running Back Success Rate Statistic
Here’s how Football outsiders breaks down "Running Back Success Rate":
• A play counts as a "Success" if it gains 40% of yards to go on first down, 60% of yards to go on second down, and 100% of yards to go on third or fourth down.
• If the team is behind by more than a touchdown in the fourth quarter, the benchmarks switch to 50%/65%/100%.
• If the team is ahead by any amount in the fourth quarter, the benchmarks switch to 30%/50%/100%.
For instance, regularly, if it’s 1st and 10, a rush of 4 yards or more is a success and less than 4 is a failure. On 2nd and 10, a rush of 6 yards or more is a success, and less is a failure. For a rush on 3rd or 4th down, anything less than a first down (or a touchdown in an "and goal" situation) is considered a failure. The metric also takes into consideration and accounts for the fact that yards become more or less important not just based on the situation of that specific play but also based on the situation of the game as a whole. For instance, when you’re ahead in the 4th quarter, yards on the early downs don’t matter as much as killing time. Thus, the definition of "success" is less stringent: 30% of yards to go on 1st down, etc. The inverse is the case if your team is behind in the fourth quarter.
 The bane of my existence. |
Thus, as Football Outsiders notes, when it’s 3rd and 1 and a running back dives for 2 yards, that’s a very successful run and is rewarded as such by this metric whereas he is penalized by the "yards per carry" metric. This statistic, then, will give you a much better idea than yards per carry of how efficient a running back is at achieving "success" in the situations in which he finds himself within a game.
But Football Outsiders designed this metric very specifically to measure NFL running backs based on the intricacies of the pro game, which are often times quite divergent from those of the college game. So to reconfigure this statistic for college football, I needed to take a look at what’s different between college and pro football. Yardage is the same, so I think I’m fine leaving "success" as based on the 40/60/100 splits. I’m also fine with the other splits (50/65/100 for a team trying to come back and 30/50/100 for a team trying to protect a lead). But what’s different about college football from the pros is 1) how much more easily points are scored and 2) the circumstances under which a team’s predominant goal is to kill time.
As to difference 1), I have decided to change the success splits to 50/65/100 not when a team is behind by 7+ points in the fourth quarter, but rather when a team is behind by 11-21 points in the fourth quarter or 7+ points on any drive that begins more than halfway through the 4th quarter. I think this better jibes with the relative ease of scoring points in college as opposed to the pros (due in part to college clock rules and in part to the comparative lack of consistency displayed by most college teams). I don’t think being 10 points down in the first half of the fourth quarter is cause for a full-on "comeback mentality." Ten points in a quarter isn’t an unreasonable expectation for a team using regular standards for what constitutes "success."
Regarding difference 2), the nature of college football (great talent disparities between many teams) leads to many more blowouts than in the NFL and a need for college teams to scale back the offensive attack and try to kill the clock much earlier by using plain vanilla rushing plays. So I think we should use to 30/50/100 success splits when a team is up 7+ points in the 4th quarter AND we should use it in the 3rd quarter when a team has certifiably blown out the other and has obviously let up (by, say, inserting the backups and not passing at all).
Now, I’m trying to formulate this statistic with your collective wisdom, so any thoughts on whether you think this metric is properly constructed or not are welcome. Are the splits for what constitutes "success" an accurate representation of what "success" should be? How about when the different splits are used? Should there be any other splits (perhaps based on which yard line the play starts on)?
Beyond Running Back Success Rate
There’s definitely some value to this success rate because it tells us one specific thing about a running back (or team rushing attack). To wit, how effective is a running back at accomplishing the very specific, bare-minimum goals of each play? But I still thought to myself, "That’s not all there is, though. What about someone who runs for 20 yards on 3rd and 1 as opposed to 2 yards? Sure, they were both equally successful at gaining the first down in front of them, but the one guy ran for 18 more yards! That’s worth something. I’ve got you, Football Outsiders!"
As it turns out, I didn’t have Football Outsiders. Aaron Schatz is clearly smarter than me and has more free time than me, meaning he’s thought of everything I will ever think of (I’m not bitter though – not at all). They equate Running Back Success Rate with batting average in baseball, which as any baseball fan knows tells you one important thing about the effectiveness of a hitter, but which tells you far from the whole story. The trick is trying to develop a statistic that’s closer to the football equivalent of OPS than batting average.
Football Outsiders did just that, creating "Defense-adjusted Value Over Average," or DVOA. This is too complicated to explain here and it’s so heavily NFL-centric as to be virtually useless for college football. If you’re interested, here’s a short explanation and a long explanation. But what I’m going to attempt to do is create something similar (though admittedly taking into account fewer variables) for college football. And I’m going to need your help.
Here’s what I’ve decided to do so far in constructing this statistic:
(1) I’ve decided to measure efficiency based on a play-by-play basis rather than a drive-by-drive basis because I think it affords more flexibility in isolating certain situations to see, for instance, how one particular player rushes by down or by other game situation.
(2) For each rushing attempt, 1 point is awarded for a "success" using the above metric and 0 points for anything other than success.
(3) When using the 40/60/100 or 50/65/100 split in the success rate statistic (i.e. normal situations), each yard gained on a play is worth 1/10 of a point (1 point for every 10 yards), positive or negative, which is added to the success point (if there is one) to get a value for the run.
(4) When using the 40/60/100 or 50/65/100 split, the maximum yardage points that can be accumulated are 3 (equivalent to 30 yards). Any rush beyond 30 yards still nets only 3 yardage points.
(5) When using the 30/50/100 split (i.e. trying to hold a lead and/or kill time), each yard gained on a play is worth 1/20 of a point (1 point for every 20 yards), positive or negative, which is added to the success point (if there is one) to get a value for the run.
(6) When using the 30/50/100 split, the maximum yardage points that can be accumulated is 1 (equivalent to 10 yards). Any rush beyond 10 yards still nets only 1 yardage point.
Thus, on a normal 1st and 10, a rush for 4 yards will garner 1 success point and .4 yardage points, such that the run will have a value of 1.4. Fleshing this out, remember our example of the difference between a 4-yard run and a 2-yard run on 3rd down and 1? I said that a 4-yard run isn’t anywhere close to twice as important as a 2-yard run because getting the first down is what’s most important on that play. With this system, a 2-yard rush is worth 1.2 points and a 4-yard run is worth 1.4 points, which I think is a much more accurate representation of the value differential of these two runs.
The reason for number (6) is that, when you’re trying to kill clock, yardage beyond a first down is virtually irrelevant and perhaps even counter-productive (because there’s less yards with which to eat up time). The reason for number (4) is that beyond 30 yards any yards gained are not the product of running back capability, but rather of the yard line on which the play starts. I’m willing to change this rule for some other rule that accounts for this problem of the varying lines of scrimmage, but I haven’t figured out how to do it yet. That’s where you guys come in. Any ideas?
Additionally, are there any other things I should be taking into account? What about fumbles? Touchdowns? I’m inclined to not give a bonus for touchdowns because there’s nothing special about being the person who physically gets the ball in the endzone. If you’re a short-yardage/goal-line specialist RB, the measure of your value is pretty adequately measured by your success rate statistic. But I’m open to arguments for why I’m wrong.
One other possibility: yards in excess of what is required for "success" are more important on 1st and 2nd down than they are on 3rd and 4th. Should the value of yards beyond what’s needed for success be weighted differently according to what down it is? How so?
A Plea for Help
 |
This running back statistic-thing is going to be a season-long project. I hope to come up with an adequate metric soon, which I will find interesting to no end. But the intellectual exercise of coming up with a statistic isn’t all that interesting to most people (we call these people "normal"). All of us want to see a practical application, so I want to compare Texas backs to each other, perhaps over several years, and I want to compare all the national standouts for this season using the statistic.
Sadly, this requires work. Even sadder, I just started my big firm job and will have much less time to do stuff like this. So the moral of the story is, I need help. If any of you guys (particularly those of you still in school with lots of free time – don’t pretend you’re busy, I’m not too old to remember college) have any interest in this sort of thing, please let me know. What’s in it for you? Um...satisfaction; your name on the main column of BON (very exciting for me the first time); you can help me write the follow-up piece to this. Hopefully, at least 1 or 2 of you can help me out.
Even if you can't help out with your time, please give your thoughts in the comments. I both want and need them.
--BZ--
17 comments
|
0 recs |
Do you like this story?
Comments
Touchdowns
Shouldn't receive any extra reward, as you mention. Rather, it should be a "success" just like any other third and short would be. If a running back scores on 1st and goal from the 3, it should be treated as a "success" and nothing more.
This is awesome BZ. I hope the army of smart readers out there steps up with some data help and comments. No doubt they will.
My two cents...
So under normal conditions (starting at 1st and 10), let's take player A, who goes for 2 runs of 0 yards and 1 30 yard run, versus player B, who goes for 3 10 yard runs.
Player A would garner 1 success point and 3 yardage points, while player B would garner 3 success points and 3 yardage points. As such, player B would be considered more successful. The FSN announcers were talking about how crucial "explosive plays" were (insert crude joke here). I don't remember how many rushing yards were needed to define an explosive play, but I believe it was more than 10 (somewhere in the neighborhood of 11-16, perhaps?). So in the above situation, player A had the explosive play, and the John Madden-esque announcer would have praised him as such.
What I'm wondering is whether or not player B is considered more "successful" than player A, and if he is, is he considered 2 points more successful as demonstrated by your point system? I would argue that player B is more successful, simply because of baseball OBP argument. In baseball, there are a limited number of outs, so anytime you are able to get on base, you're not wasting a limited resource. Therefore, in a large sample size, a player who goes 3-3 with three singles is more valuable than a player who goes 1-3 with a triple, even if their slugging percentages are equal. While the only limited resource in football is the time on the clock, I believe that first downs serve the same purpose. While both player A and player B gained the same amount of yardage (slugging percentage), the fact that player B was able to accrue 3 first downs is somehow more valuable than player A's ability to gain 1. Whether this is true or not is entirely debatable.
As for the touchdown issue, scoring points in football is akin to scoring runs in baseball; if you don't score points, you won't win the game (blatantly obvious, I know). So it makes little sense that a 5 yard run on 1st and 10 from the opposition 10 yard line (which is worth 1 success point and .5 yardage points) against a touchdown run from the 2 yard line (which is worth 1 success point and .2 yardage points). This effect is similarly shown in any normal 1st and 10 v. 3rd and 2 situation. This, in effect, downplays the added advantage of a power running back (such as a Chris Ogbannaya), who is in the game to gain the hard yards but has little chance of breaking off a long gain.
Again, the debate comes down to which is more valuable: first downs or yards to go to the end zone. To reiterate, my argument is predicated on the belief that first downs are more important than yards to go, simply because first downs are a more precious commodity. The yards can always be regained on a next down, whereas it's harder to replenish the first down. And touchdowns should be given extra weight, since the primary goal of an offense is to score points. Yardage is nice, but if it doesn't lead to points, yardage is inconsequential.
And if I've misconstrued any of the math or reasoning behind your argument, please let me know.
i'll take these thoughts 1 by 1
So under normal conditions (starting at 1st and 10), let's take player A, who goes for 2 runs of 0 yards and 1 30 yard run, versus player B, who goes for 3 10 yard runs.
Player A would garner 1 success point and 3 yardage points, while player B would garner 3 success points and 3 yardage points. As such, player B would be considered more successful......What I'm wondering is whether or not player B is considered more "successful" than player A, and if he is, is he considered 2 points more successful as demonstrated by your point system?
These points aren't mean to be accumulated over the course of a game. Accumulating them defeats the whole purpose of creating this statistic, because all you're left with is a gross amount of "rush value," which is no more valuable than gross yardage and far more difficult to understand. To be an efficiency statistic, you have to average the rush value based on the number of rushes. Similarly, for the Success Rate, you have to look at the actual rate of success, not just the number of times the RB was successful.
As such, Player A would have 1 run worth 4 points and Player B would have 3 runs worth 2 points each. Certainly, per rush, player A is more valuable. If he can rush for 30 yards and a first down every play (making this his average rush value), he's more valuable than player B.
What you're trying to do is measure value in terms of drives rather than in terms of plays. Is getting 30 yards in one play better than doing it over the course of 3 consecutive plays? I don't know the answer to that question, and I think, whatever that answer is, it depends on the game situation. But as I said in the column, I'm trying to create this measure of efficiency based on plays, not drives. So that's why I did what I did.
You should totally try to come up with a drive efficiency statistic though. That'd be awesome.
second issue - touchdowns
As for the touchdown issue, scoring points in football is akin to scoring runs in baseball; if you don't score points, you won't win the game (blatantly obvious, I know). So it makes little sense that a 5 yard run on 1st and 10 from the opposition 10 yard line (which is worth 1 success point and .5 yardage points) against a touchdown run from the 2 yard line (which is worth 1 success point and .2 yardage points). This effect is similarly shown in any normal 1st and 10 v. 3rd and 2 situation. This, in effect, downplays the added advantage of a power running back (such as a Chris Ogbannaya), who is in the game to gain the hard yards but has little chance of breaking off a long gain.
See, I think your argument is backwards here. The whole point of having a power back is very specifically to accomplish a "success," not to gain a bunch of yards. That's why this system is so much better at accounting for value than yards per rush. When a power back is put in specifically to gain a first down or get a touchdown and does just that, that's just as important as an every down back gaining 6 or 7 yards on 1st or 2nd down.
Now, it's true that for a power back, touchdowns and first downs are decent ways to measure effectiveness, but again, that's just a cumulative measure that has nothing to do with efficiency. Additionally, there's no way to compare the efficiency of short-yardage power backs and regular RB's. That's what this statistic gives us.
As to the addition for touchdowns, I'll answer that below in response to Longhorn in Canada's post.
Smart Football
Smart Football has an article about the Walsh system that breaks plays down according to situation, taking into account down, position on the field, etc. I think an excellent measure of running back effectiveness would be the percentage of times that the situational need is met.
Long runs and touchdowns
I think you are off to a really good start here. there are two points I would like to make that are related. First, I think that runs over 30 yards should count for more. Possibly you could continue to add points, but at a reduced rate for longer runs. Second, there should be some points awarded for touchdowns. Fourth and goal from the 1 yard line is more difficult than fourth and 1 from the 30. Your idea of weighting the points based on the line of scrimmage is good, but I think scoring a touchdown should count for something.
Here's an example of both points. Back in the Seventies, I was working in Dallas, and had the pleasure of going to see Texas play SMU at the Cotton Bowl. The play as I remember it happened when Texas had 3rd and short from inside its own 30. Everyone in the stadium knew what the play call would be. SMU put 10 in the box. Earl broke through and headed for the end zone with one safety down field. Even the Texas fans felt sorry for that lone SMU player. As anticipated, Earl put the poor guy on his butt on his way to score. The lift a play like that gives your team, and the demoralizing effect it has on your opponent, should count for something. Anyone could look at Earl and see he would be hard to bring down in short yardage. It was after he outran the fastest guys on your team that you knew you were in real trouble.
A long run where you are caught from behind and tackled at the 2 yard line doesn't compare with the long touchdown run. Those last 2 yards mean something.
by Longhorn in Canada on Sep 12, 2007 12:24 PM CDT reply actions
the problem with that example
is that i'm not in the business of quantifying emotional reactions. something being demoralizing in one instance isn't demoralizing in another. you just can't take an anecdote and quantify it.
regardless, you make some good other points. as for touchdowns counting for something, as i said, i tend to disagree. the whole point of getting a success point is that you are given the ball with a specific minimum goal that varies with the situation. the measure of success is whether you accomplish that minimum goal. Perhaps 4th and goal on the 1 is harder than 4th and 1 from the 30, but how we define success doesn't vary with how hard it is to accomplish the goal.
as you note, the only reason i'm contemplating adding extra points for touchdowns is because i'm struggling with the line of scrimmage problem. if the ball is on the 20 and you rush 20 yards for a touchdown, is that any appreciably different in terms of efficiency than if the ball is on the 30 and you rush 30 yards for a touchdown? or is it instead merely a product of the fact that the line of scrimmage on that run happened to be 10 yards further away from a touchdown than on the 20 yard run? this is the same reason that i decided to cap yardage points at 3 (30 yards).
i've got some vague ideas, but i need to think about this more. any more ideas would be appreciated.
Think about...
... the actual decision-making process of a running back. I've just run 20 yards, and a tackler is closing in on me.
I have two basic choices:
1.) Run over him and (likely) get a couple of extra yards as I fall
2.) Try to evade him and break away for many more yards. (If I fail, I give up those couple "power" yards I would have had otherwise)
Now, if I'm at the fifty yard line, it seems to me to be worth the risk to try and escape the tackler, because taking a big run to the 50 is hardly different from a big run to the 48.
If I'm at the goal line, however (this could even be argued of the first down marker), everything changes. Now, it's pointless to try and run around. There are no more yards to run, so I sacrifice my ability to continue the run, and plow through the poor bastard for a TD (or FD).
That's why I propose only adding a bonus to TDs which occur after, or at a certain number of yards after, success. You've already done your job, and have only one more possible goal, so by accomplishing it, you attain maximum value from the run.
This adjusts for field position on its own, because if you think about it, a graph of value vs. yard line would look like a bowl. The first few yards from your own goal line would be important, because each one is breathing room for your QB or punter, the middle of the field is roughly the same between field goal range and your own 10, within field goal range, yards are worth increasingly more, but the value does not increase at an incredible rate, and then the last yard is super important. (While getting 2 yards on 1st and goal from the 4 counts as success, it really isn't that much different from a touchdown than 3 yards.) This implies that the "slope" of the graph here would be super-steep, and is probably best approximated by a TD bonus.
Long runs and touchdowns
I agree that there is not much difference between 20 and 30 yard runs for touchdowns. I do believe, however, that there is a significant difference between a 30 yard touchdown run and a 28 yard run to the 2. (Particularly this year, when the run to the 2 is likely to result in 3 points, at best.)
I understand your reluctance to try to quantify emotional reactions. Otherwise, you might find yourself as beloved as Lou Holtz! I don't want to overstate the emotional importance, but think of the impact JC's long TD rush had on both sides in the OU game in 2005. If a Sooner had been able to run him down from behind, it would not have been quite the game changing moment that it was. It's hard to quantify the impact of a run like that, but the fact that he got in the end zone seems to be worth something. That's why I recommend adding something for a touchdown.
I certainly don't want to be giving style points, where we use French figure skating judges to determine who is the most efficient running back. Since efficiency and greatness are not quite the same, maybe we can use statistics to determine efficiency, but have to actually watch the games to determine greatness.
by Longhorn in Canada on Sep 12, 2007 11:32 PM CDT up reply actions
Good point, LHiC.
Maybe you can give a bonus for touchdowns if they are more than what is required for success? Say, 3rd and goal TD receives normal point score, but 3rd and 2 from the 10 TD receives a bonus. Maybe the bonus should be fixed, but I think it should come as a multiplier of the yardage bonus. Instead of 1/10th of a point, maybe give between 1/10th and 1/5th of a point for each yard gained on a TD run that was more than required?
does this calculation...
only apply to rushing yards?
Wouldn't a Running Backs efficiency also improve if he was a receiving threat as well?
How about the RBs overall efficiency for "Plays in the backfield"? I'm not even sure if obtaining that stat (without video review) is even possible since you would not easily know if a RB was even lined up on a play in which he didnt touch the ball.
I only bring this up, because of a RBs effectiveness in situations other than carries. Is a RB being left in the game on 3rd and long for pass protection or even receiving options? Is a RB taken out of the game during short yardage situations?
Shouldn't all of this be taken into account for the RBs efficiency or would that fall into a completely different category? I know this is a screw ball, and I'm not even requesting that these stats be included in the formula. I was just wondering what the options were and if any of these stats are readily available for the sake of statistical analysis.
by bleed burnt orange on Sep 12, 2007 3:04 PM CDT reply actions
we're talking rushing efficiency specifically
not running back efficiency generally. we'd have to include both receiving and blocking to get the full picture. i should have made that a bit more clear.
as far as availability, this info is not readily available to the public as far as i know (other than by reviewing the video), but i'm sure athletic departments keep track of it.
yeah...
with the stats geared toward rushing yards only, I'd assumed that was all that was included.
"Running Back Success/Efficiency" imo would almost def have to include some of those stats that we could never get.
by bleed burnt orange on Sep 12, 2007 4:18 PM CDT reply actions
you're right
from here on out, i'll try to call it "running back rushing efficiency" or something like that.
Rushing on 3rd Down
I thought of one situation where a run on 3rd down that doesnt give you 100% of the yards might be considered a success.
If you were facing a 3rd and 10 from your OWN 1 yard line, a running back who takes the handoff and runs for 9 yards still puts your team in a better situation. In this case, the running back's objective was to provide as much room as possible for the punter to get the kick away. Running for 9 yards, the back failed to make the 1st down and did not accomplish his goal in the system proposed. However, he did give his punter enough room to get the kick away safely and hence the play was a "Success".
My point is, shouldn't a teams position on the field also count towards judging "Success"? Any thoughts guys?
by LonghornForLife on Sep 12, 2007 4:41 PM CDT reply actions
you're absolutely right
this just further drives home to me that i have to come up with another variable for line of scrimmage. to many things are dependent on this. in the situation you mention, it's probably true that the definition of success is something less than a 1st down.
any ideas on how to incorporate this?
maybe end result?
if the RB gained yardage on 3rd and whatever to make it 4th and shorter which resulted in a 4th down conversion, then you would have to consider both the 3rd down and the 4th down run a success.
Now if it was like 3rd and 1, and the RB gets stuffed, then converts on 4th and 1, then 3rd down would be fail and 4th down would be success. The preceding 3rd down should only get credit if it makes the distance shorter?
I dont know, just throwing out ideas.
As far as giving your punter extra room which IS very critical to the variables of the game (field position, no safeties, successful punt, etc) I would say that if the end result is a Punt or turnover of any sort (success - probably better specified as 1st down or TD on 3rd and 4th down attempts), the rush attempt is still a fail.
by bleed burnt orange on Sep 12, 2007 5:14 PM CDT reply actions
Comments For This Post Are Closed
Secondary Sidebar
User Tools
FanPosts
Community blog posts and discussion.
Certain photos copyright © 2012 by Associated Press or Getty Images. Any commercial use or distribution without the express written consent of Associated Press and Getty Images is strictly prohibited.
