Why Passer Rating is not a Good Stat

Passer rating is kind of a bad stat. It is outdated at best, it is confusing, and I'm not sure that it is worth the trouble. I am going to explain passer rating's shortcomings, but first a word on what stats should do.

Reactive vs. Predictive Statistics

Stats are only interesting if they tell us what we don't already know. Did you know that scoring more points than your opponent correlates very strongly with wins? How interesting is that?!! I believe good stats tell us something interesting, usually in one of two categories:

• They accurately represent an individual performance or an aspect of the game that has already occurred, or
• They predict future performance.

I will refer to metrics that accomplish these tasks particularly well as "narrative" or "explanatory" stats and "predictive" stats.

Passer rating is decent as an explanatory stat. It is far from perfect as I'll discuss later, but in general passer rating goes up as the passer's performance improves and vice versa. Passer rating is not so good at predicting future performance. No individual metric is, and the game would be a whole lot less fun if we could predict the future, so I'm glad. More on this in a future post.

Limits

Passer rating is made up of four component stats, with limits and different weighted values on each. These limits and scales are one of my objections to passer rating.

Each of the component stats in passer rating are scaled and assigned a value between 0 and 2.375, with 1.0 being statistically average...based on the league data between 1960 and 1970. That's right, passer rating is scaled to data that is at least 43 years old. I think it is safe to say that the league has changed a little since then.

For whatever reason, these limits and values were scaled so that a 66.7 passer rating was statistically average. In 2008, the league average was 83.2, a nearly 20 point swing on a 158.3 point scale.

The 0 ≤ x ≤ 2.375 limitations are not often problems, at least not yet. A QB would have to throw for over a 77.5% completion rate, 12.5 yards/attempt, TDs on more than 11.875% of his passes, or less than zero INTs to actually hit the limits. (Though if the league continues its trajectory as a passing league, some of those limits may become a factor soon). But the fact that these numbers are scaled but not to any sort of sensible number is arbitrary and unnecessarily complicated at best.

Seriously, are you confused about how and why this statistic is calculated yet? Why 158.3, why not 100? Why would 66.7 be average? At least the lower limit of a 0 passer rating makes sense. Though as Browns fans know there are always more interceptions that can be thrown, and maybe taking the limit off of that statistic makes sense: you keep throwing INTs, your rating should keep plummeting even if you've passed zero.

Next, we look at the component stats of passer rating: yards per attempt, completions, touchdowns and interceptions.

Component stats of passer rating

Yards/Attempt

This is the least objectionable component of the bunch. Yards per attempt is a fairly good way to measure a quarterback's success. It correlates well with winning games, and it's a pretty good rate stat to begin with...

Completion percentage

...the problem is that when you count both completion percentage AND yards per attempt, you are effectively "counting completions twice." Completions are already factored in to the rate stat of yards per attempt because if you throw a pass and it falls incomplete you threw for zero yards on that attempt.

Is a complete pass for no gain more effective than an incomplete pass? Perhaps, as the quarterback threw a ball that was accurate enough to be caught, and at least gave his receiver a chance to break a tackle and gain yardage. However, the incomplete pass may have also been a good throw by the quarterback which gave his receiver a similar chance to make a play. Furthermore if a quarterback were really giving his receivers chances to make plays they would eventually actually make plays. And even if this was only once in a while, it would show up in a QB's y/a.

In the end, I think there is some value to consistency. Reliability, consistency, and steady play seem to have some value in NFL coaching circles, and offenses who can consistently get in short yardage conversion situations would seem to have an advantage over those who generate big plays at lower rates. This premise is at least plausible.

What doesn't make sense, however, is the weight that passer rating gives to merely completing passes. Consider the following stat lines:

Attempts Completions Yards TDs INTs
QB A 40 40 100 0 0
QB B 40 20 325 0 0

Who would you rather take? Give me those 325 yards, and it is no contest. QB A has completed 100% of his passes, but for 2.5 yards per attempt (i.e. worse than Charlie Frye's career average). QB B might have only completed 50% of his passes, but he's still geting 8.125 yards every time he throws the ball, meaning completions are going for 16.25 a pop!

Passer rating still rates QB A as the better passer by about a point and a half, equating a mere completed pass (and no yards) to 20 yards of offense. This should sound ridiculous to every fan. While there may be some value to consistency, these examples show how completion percentage is over-emphasized in the formula.

INTs

Interceptions are another statistic that seem fair to use to evaluate QBs. Turning the ball over hurts your chances of winning, and if the quarterback is responsible for INTs he should be accountable in any explanatory stat.

However, interception numbers are highly volatile, especially with regard to game situations. For instance, Russell Wilson has thrown an interception on only 0.79% of his attempts this year while leading but on 3.3% of his attempts while trailing*, meaning Wilson throws over four times as many picks while trailing. This phenomenon makes sense according to game theory as well, as a defense will be geared up to stop the pass in situations where passes are expected. A quick look at how those situations can fluctuate from team to team and you can see why one QB might have a tougher job of avoiding turnovers than another.

*stat includes week 1 through week 12 in 2013.

Remember that the goal with any "advanced" quarterback statistic should be to isolate the quarterback's performance or to predict his future performance, not to give him credit for team accomplishments. If a team's performance is a big factor in an individual stat, one must wonder about including it at all.

INTs likely fall at least somewhat on the quarterbacks' shoulders and a case can be made for them to be included in any statistic that evaluates QB performance. What I do take issue with is the absurd value that passer rating assigns to avoiding interceptions. According to passer rating, INTs are worth about 100 yards. Think about that for a second: is a turnover really worth 100 yards of field position? Think about the average field position differential between an incompletion and an interception, does it seem like 100 yards of difference?

The value of interceptions obviously varies highly by situation: consider an intercepted hail-mary into the endzone to end the first half. Now contrast that with the value of a turnover in field goal range while down 3 points late in the 4th quarter.

Interceptions are returned to within 8 yards of the line of scrimmage on average (depending on your source). Considering the difference between an 8 yard net gain via the interception or a 35-40 yard net gain via a punt, and you're looking at a difference of about 30 yards. If you throw in a yardage penalty for any chance at a first down, 45 yards sounds reasonable (and this was the conclusion reached by the authors of the groundbreaking work The Hidden Game of Football in 1988). Accomplished statistician Brian Burke says the modern figure should be 60 yards. Others claim that the "point of indifference" to possession is the 15 yard line, meaning that in an average game you would give equal value to having the ball on your own 15 or giving your opponent the ball at his 15. In any event, 100 yards is too severe a penalty.

Personally, I side with the lower end of the figures in this debate. Regardless, a more appropriate penalty would be somewhere between 45 and 60 yards for interceptions.

Touchdowns

Much like the out-of-whack weight given to interceptions, passer rating gives touchdowns a bonus equivalent to 80 yards. Remember that all actual yardage gained is accounted for in yards per attempt, so this bonus is "extra" yardage awarded solely for crossing the goal line. This bonus should be equivalent to having the ball at your own 1" line vs. having 7 points and kicking the ball off. According to The Hidden Game of Football, this bonus was equivalent to about 10 yards. Others have more recently calculated the value at around 20 yards. Regardless, an 80 point bonus is way too high.

Next up

As you can see, passer rating is flawed as a reactive statistic. The values it assigns to its component stats are odd at best, and it could do a much better job of illustrating QB performance with some tweaks.

In my next post I will explain passer rating's performance as a predictive statistic and suggest alternatives and fixes.