Quarterback (and other player) Rating Systems

[Mavric]

 That isn't really true- its a statistical model- and many are skeptical about the results. The model grades plays- no human interface. Misses a deep open route for a TD and throws to the short guy or 5 yard run instead. Yips on an easy throw that could have been a game winning throw. All you need is a TD to win or tie- drive stalls. Clutch plays. Tough to put that into a SAS algorithm. Like everyone Im rooting for him and he looks improved but youre putting the cart before the horse here. 

https://www.hogshaven.com/2018/3/13/16839982/5-oclock-club-difference-nfl-passer-rating-and-quarterback-rating-redskins-alex-smith-kirk-cousins

It is actually true.  It is possible to assign a numeric value to the performance on each play and aggregate that across whatever timeframe is desired.

And, again, you don't actually know what they are doing or not doing.   You're making more assumptions to fit your narrative.  All the examples you listed are almost assuredly taken into account - it would be stupid to go to all that work and not take them in to account.  But it makes you feel better to assume you know more than they do so you assume that you are the only one capable of seeing those things.

You may disagree with the accuracy.  That's fine.  But you continue to insist things are "wrong" or "not true" when what you really mean is "I don't like it."  That's a pretty straight-forward distinction that you refuse to acknowledge.

 

[Nebraska55fan]

It is actually true.  It is possible to assign a numeric value to the performance on each play and aggregate that across whatever timeframe is desired.

And, again, you don't actually know what they are doing or not doing.   You're making more assumptions to fit your narrative.  All the examples you listed are almost assuredly taken into account - it would be stupid to go to all that work and not take them in to account.  But it makes you feel better to assume you know more than they do so you assume that you are the only one capable of seeing those things.

You may disagree with the accuracy.  That's fine.  But you continue to insist things are "wrong" or "not true" when what you really mean is "I don't like it."  That's a pretty straight-forward distinction that you refuse to acknowledge.

 You don't know what is in the algorithm. Would be extremely difficult to assign some arbitrary number to a play where an early receiver open is passed up or the countless nuances of a game that would take endless hours to analyze. You are making assumptions you can't. YOU are making assumptions to fit your narrative- you don't know all that is what is measured in model. Rules for thee not me again. 

 

[Mavric]

 You don't know what is in the algorithm. Would be extremely difficult to assign some arbitrary number to a play where an early receiver open is passed up or the countless nuances of a game that would take endless hours to analyze. You are making assumptions you can't. YOU are making assumptions to fit your narrative- you don't know all that is what is measured in model. Rules for thee not me again. 

So you tell me I don't know what's in the algorithm but you purport to know because you can say what's not included?  That doesn't seem accurate.

It is not tough at all.  If the QB throws a perfect pass for a 75 yard touchdown, that's 100.  If he throws a terrible pass for a pick six, that's a zero.  He completes a simple screen pass where there's no pressure, that's maybe a 50.  Everything else is graded somewhere along that spectrum.  It's not at all difficult.  But you want to discredit it because you don't like what the numbers show so you're making any possible attempt to make it sound like it's impossible when it absolutely isn't.

 

[RedDenver]

Yeah, I didn't word that very precisely.  There is obviously subjectivity in the grading.  But there is (presumably) as much control of that subjectivity as can be attained to accomplish the task that is needed.  And as long as that subjectivity is as consistant as possible across different teams and different years, they are by far the best grading systems available.

I'm more concerned about quality than consistency (although both are important). How do we know these graders are actually any good at evaluating plays or players? I mean, even I can go through and mark grades for every player for each play, but I'm just some guy that grew up watching Huskers football. I know enough to assign reasonable grades that wouldn't probably alert much suspicion but I definitely am not qualified to provide real insight.

I meant objective in the sense that they aren't a fan of any one team and have their grades affected by the outcome of the game.

Do we know that? There's no transparency as to who the graders are or what the individual grades per play per grader were, so I don't think you can make that statement with anything other than hoping it's true.

 How can a specific grade being given to each play then totaled be seen as less comprehensive than a general grade given for the the overall play?

I'm not saying it's less comprehensive, rather that there's no way to know if it's any more comprehensive. It's entirely conceivable that one analyst could watch the game and come up with a more accurate evaluation of a player than another analyst who writes down numbers for every play. If there was some metric that could be used to measure the quality of the analysis, then we could say something about it, but it's really just a bunch of opinions that are difficult to sort out.

  And who is doing this that wouldn't have bias because they are closely associated with a given team?

Yes, exactly. The same applies to all subjective analysis including PFF's graders.

This is the point.  It is comprehensive across all teams for multiple years and not driven by coverage a specific team.

Yeah, I get that appeal. But again, we don't know who the graders are, so they could be people who actually cover a specific team. There's no way any of the graders could cover all the plays of all the players of every game each week.

To sum up, I think PFF is interesting but have strong doubts that it's any more or less accurate or meaningful than any other analyst like Kirk Herbstreit or Jerry Dinardo (just examples, there are many, many others).

 

[Nebraska55fan]

So you tell me I don't know what's in the algorithm but you purport to know because you can say what's not included?  That doesn't seem accurate.

It is not tough at all.  If the QB throws a perfect pass for a 75 yard touchdown, that's 100.  If he throws a terrible pass for a pick six, that's a zero.  He completes a simple screen pass where there's no pressure, that's maybe a 50.  Everything else is graded somewhere along that spectrum.  It's not at all difficult.  But you want to discredit it because you don't like what the numbers show so you're making any possible attempt to make it sound like it's impossible when it absolutely isn't.

Where did I say anything like that? Of course you can quantify a 75 yard pass. They can't quantify the pass that should have been thrown for a game winning TD that instead was thrown for a short gain or a sack. If it is somehow accounted for- what is the number minus 100 or should it be minus 200, or does the analyst have leeway or is he even looking and counting that stuff at all?  They cant quantify a run that should have happened to get a first down if run early but instead was run late and got a 5 yard gain. Do they have the same graders grading every single QB?  120 CFB teams- same guy analyzing every snap of every game to the nth degree- looking at all of the coulda shoulda's of every single snap- I seriously doubt it. Those ratings usually come out, what the Monday after the Saturday games? 

YOu are assuming they go to that level- which would be nearly impossible to do or quantify. Again- you are assuming they can- rules for thee not for me- I get it. 

 

[Mavric]

I tried to separate out the discussion over how QBR and PFF come up with their rating from the rest of the Martinez discussion.  Kind of hard to tell which posts to pull and which to leave but I think it's fairly close now.

Hopefully that way the Martinez thread isn't as jumbled.

 

[Mavric]

To sum up, I think PFF is interesting but have strong doubts that it's any more or less accurate or meaningful than any other analyst like Kirk Herbstreit or Jerry Dinardo (just examples, there are many, many others).

I guess there are enough football people who put enough weight on their rankings to give them the benefit of the doubt.  You seem to apply much more scrutiny to them then anyone else.  I don't think that's probably warranted.  You basically have to assume the least amount of competence for everything they're doing to judge them as worse than anything else (or even on the same level as anything else). 

No one else is claiming or representing to do anything nearly that comprehensive.  They are just giving their own gut feeling by what they saw, often just watching the game live, not even going back and studying the film.

I'm pretty sure Kirk Herbstreit and Jerry Dinardo are not watching and grading every play for any one player every week, let along doing it for a bunch of players.

And it's not like their ratings are so far out of the realm that they're wrong at face value.  According the ESPN's QBR, CJ Stroud, Adrian Martinez and Sean Clifford are the top three in the B1G.  Michael Penix, Graham Mertz and Hunter Johnson are at the bottom.  Based on everything else, that seems to be pretty legit.  Yeah, we can quibble with how they fall exactly but generally speaking it seems like their system works.  Again, not perfect but much more thorough and, thus, like more accurate than anything else out there.