Quarterback (and other player) Rating Systems

[Nebraska55fan]

Not everything a QB does can be measured to determine how effective he really is. 

The QB sack, where he could have thrown out of bounds

The inopportune fumble that leads to loss of momentum, points, losing the game

Missing a chance to run with plenty of daylight for a first down, instead holding onto the ball for a late sack, late covered throw, or short gain

Missing wide open receivers for game winning throws even with time

In the clutch being able to get a key first down so your team down the field to secure the win

In the clutch being able to lead your team to the tying or winning score on the last possession- I think he is 1-14 in that regard and we haven't gotten a first down in 3 Overtime games. 

No winning season in 3 1/2 years. 

So yes, he does a lot of things well and has made progress this season so far. But pure numbers don't tell the entire story. I like how he's trending, time to pull one of these games out. 

 

[Mavric]

Not everything a QB does can be measured to determine how effective he really is. 

The QB sack, where he could have thrown out of bounds

The inopportune fumble that leads to loss of momentum, points, losing the game

Missing a chance to run with plenty of daylight for a first down, instead holding onto the ball for a late sack, late covered throw, or short gain

Missing wide open receivers for game winning throws even with time

In the clutch being able to get a key first down so your team down the field to secure the win

In the clutch being able to lead your team to the tying or winning score on the last possession- I think he is 1-14 in that regard and we haven't gotten a first down in 3 Overtime games. 

No winning season in 3 1/2 years. 

So yes, he does a lot of things well and has made progress this season so far. But pure numbers don't tell the entire story. I like how he's trending, time to pull one of these games out. 

Actually, yes, all of that can be measured by numbers.  I'm sure he gets a poor grade for those plays.  That's what goes into the QBR number.  They grade each play and give it a value - higher value for a good play, lower number for a bad play.  It is perfect?  I'm sure it isn't.  Is it a good indication and a way to compare different players in different systems with different talent around them?  I tend to think it is.

 

[Nebraska55fan]

No metric is going to measure the plays he could have made or should have made. We all saw a significantly different AM in 2019 and 2020. The 2021 version is certainly more decisive and similar to the 2018 version than the other years. 

 

[Mavric]

I'm seeing Martinez's freshman QBR at 139.5, and last year's at 135.  That would be slightly worse rather than significantly better. I believe QBR is heavily influenced by yards per attempt and TD/INT ratios. Maybe you found a different metric. He definitely had a better per carry rushing average last year, but in the spirit of this discussion, wouldn't leading the NCAA in fumbles two years in a row be a pretty huge factor in figuring a player's progress?  

That's not QBR.  That's the Quarterback Rating which only looks at the actual outcome of passing plays - complete, incomplete, touchdowns, interceptions, etc.  He was slightly down on that last year but that would also obviously be affected by who he has to throw to.

QBR is a completely different metric.  It assigns a weighted grade to all plays - passes, runs, scrambles, whatever - and gives an overall rating that takes everything a quarterback does into account.  By that metric Martinez was much better last year than he was his freshman year.

 

[ZRod]

That's not QBR.  That's the Quarterback Rating which only looks at the actual outcome of passing plays - complete, incomplete, touchdowns, interceptions, etc.  He was slightly down on that last year but that would also obviously be affected by who he has to throw to.

QBR is a completely different metric.  It assigns a weighted grade to all plays - passes, runs, scrambles, whatever - and gives an overall rating that takes everything a quarterback does into account.  By that metric Martinez was much better last year than he was his freshman year.

It should be noted that nobody knows the formula for QBR, except ESPN I think, but it's generally reliable comparison tool.

 

[Mavric]

It should be noted that nobody knows the formula for QBR, except ESPN I think, but it's generally reliable comparison tool.

Yeah, that part seems a little odd.  Although I suppose it is basically considered proprietary and they don't want others copying it.

But as far as I know that and the PFF ratings are the best objective measures to compare players.

 

[RedDenver]

Yeah, that part seems a little odd.  Although I suppose it is basically considered proprietary and they don't want others copying it.

But as far as I know that and the PFF ratings are the best objective measures to compare players.

Not to derail this thread too much, but what makes you say that? I'm interested in what makes QBR or PFF grades any more objective or any better of a comparison tool than anything else? The lack of transparency really hurts the "objective" claim IMO, but I'm interested in what I might be missing. (Can also move to a different thread.)

 

[Mavric]

Not to derail this thread too much, but what makes you say that? I'm interested in what makes QBR or PFF grades any more objective or any better of a comparison tool than anything else? The lack of transparency really hurts the "objective" claim IMO, but I'm interested in what I might be missing. (Can also move to a different thread.)

Because they look at each play and assign a grade to that play then compile the results for an overall grade.  I'm not aware of any other system that does that.

 

[Nebraska55fan]

That's not QBR.  That's the Quarterback Rating which only looks at the actual outcome of passing plays - complete, incomplete, touchdowns, interceptions, etc.  He was slightly down on that last year but that would also obviously be affected by who he has to throw to.

QBR is a completely different metric.  It assigns a weighted grade to all plays - passes, runs, scrambles, whatever - and gives an overall rating that takes everything a quarterback does into account.  By that metric Martinez was much better last year than he was his freshman year.

  It does not take everything into account. 

There are many nuances to the game that a statistical model will not be able to account for.

Missing an opportunity to run early with lots of green and pick up a first down- instead throwing an incompletion or taking a sack. 

Holding onto the ball long instead of hitting an open receiver early. 

Making a last possession drive count that can tie or win the game. 

 

[Archy1221]

Not to derail this thread too much, but what makes you say that? I'm interested in what makes QBR or PFF grades any more objective or any better of a comparison tool than anything else? The lack of transparency really hurts the "objective" claim IMO, but I'm interested in what I might be missing. (Can also move to a different thread.)

For QB’s specific, unless that system’s grade for each passing play takes into account whether or not the QB missed a first read or second read, it wouldn’t amount to much of a grading system. 
 

QB could get a good score on a scramble play but in reality should have been dinged for missing the wide open read and therefor not have to scramble. 

 

[Jason Sitoke]

For QB’s specific, unless that system’s grade for each passing play takes into account whether or not the QB missed a first read or second read, it wouldn’t amount to much of a grading system. 
 

QB could get a good score on a scramble play but in reality should have been dinged for missing the wide open read and therefor not have to scramble. 

My understanding is that at the root of PFFs grading system is a qualitative assessment of each player for each play, which means that it theoretically should be taken into account and hurt the QB's passing score for that play.  The value here is in the granularity of the assessment, rather than some super advanced quantitative metrics.  I would argue that scouts and evaluators likely measure players in a similar way.  Qualitative scrutiny using a player's film.  

 

[Nebraska55fan]

Actually, yes, all of that can be measured by numbers.  I'm sure he gets a poor grade for those plays.  That's what goes into the QBR number.  They grade each play and give it a value - higher value for a good play, lower number for a bad play.  It is perfect?  I'm sure it isn't.  Is it a good indication and a way to compare different players in different systems with different talent around them?  I tend to think it is.

 That isn't really true- its a statistical model- and many are skeptical about the results. The model grades plays- no human interface. Misses a deep open route for a TD and throws to the short guy or 5 yard run instead. Yips on an easy throw that could have been a game winning throw. All you need is a TD to win or tie- drive stalls. Clutch plays. Tough to put that into a SAS algorithm. Like everyone Im rooting for him and he looks improved but youre putting the cart before the horse here. 

https://www.hogshaven.com/2018/3/13/16839982/5-oclock-club-difference-nfl-passer-rating-and-quarterback-rating-redskins-alex-smith-kirk-cousins

 

[RedDenver]

Because they look at each play and assign a grade to that play then compile the results for an overall grade.  I'm not aware of any other system that does that.

So not objective (grading is still subjective) but rather granularity of grading individual plays as @Jason Sitoke said. It's certainly different but not sure that makes it any better although I see the appeal of thinking that the graders can be less biased by looking at every play individually.

My understanding is that at the root of PFFs grading system is a qualitative assessment of each player for each play, which means that it theoretically should be taken into account and hurt the QB's passing score for that play.  The value here is in the granularity of the assessment, rather than some super advanced quantitative metrics.  I would argue that scouts and evaluators likely measure players in a similar way.  Qualitative scrutiny using a player's film.  

That's my understanding as well. I just don't think it's any less biased or more useful than a grader watching the game and giving an overall grade for a player. But PFF does grade a whole lot more players every week than traditional pundits/analysts, so there's not really anything else like it.

 

[Mavric]

  It does not take everything into account. 

There are many nuances to the game that a statistical model will not be able to account for.

Missing an opportunity to run early with lots of green and pick up a first down- instead throwing an incompletion or taking a sack. 

Holding onto the ball long instead of hitting an open receiver early. 

Making a last possession drive count that can tie or win the game. 

You don't know what they do or don't take into account.  You like to make a ton of assumptions that fit your narrative.

 

[Mavric]

So not objective (grading is still subjective) but rather granularity of grading individual plays as @Jason Sitoke said. It's certainly different but not sure that makes it any better although I see the appeal of thinking that the graders can be less biased by looking at every play individually.

Yeah, I didn't word that very precisely.  There is obviously subjectivity in the grading.  But there is (presumably) as much control of that subjectivity as can be attained to accomplish the task that is needed.  And as long as that subjectivity is as consistant as possible across different teams and different years, they are by far the best grading systems available. 

I meant objective in the sense that they aren't a fan of any one team and have their grades affected by the outcome of the game.  And they are looking at all teams and players through (roughly) the same lens.  And also objective in that each play gets it's own weight.  Not trying to down-play a bunch of good plays too much because of one bad play that cost the team the game.

That's my understanding as well. I just don't think it's any less biased or more useful than a grader watching the game and giving an overall grade for a player.

How can a specific grade being given to each play then totaled be seen as less comprehensive than a general grade given for the the overall play?  And who is doing this that wouldn't have bias because they are closely associated with a given team?

But PFF does grade a whole lot more players every week than traditional pundits/analysts, so there's not really anything else like it.

This is the point.  It is comprehensive across all teams for multiple years and not driven by coverage a specific team.