Ryan Tannehill

"The Guy wrote:

But the quarterback's surroundings don't typically change significantly on a game-by-game basis, whereas they can on a year-by-year basis (through player acquisitions and losses, coaching changes, etc.).

Note for example the huge increase in Tom Brady's season passer rating in 2007, presumably as a function of the receiving trio of Randy Moss, Wes Welker, and Donte Stallworth. His passer rating probably didn't vary from game to game any more that season than it has from game to game in any other season of his career, but I suspect it varied from game to game at a level significantly higher that year than it has in any other year of his career.

So if it's true that a QB's performance is determined more by his surroundings than by his own ability (or vice-versa), shouldn't a year-by-year comparison, across multiple QBs and teams, provide the best test of that hypothesis?

There's nothing inherently stopping us from finding elevated standard deviations analogous to the ones you pointed out above, but on a within-group basis in the comparison I outlined in the post you quoted. Whether those would outweigh the between-group variation would be the question.

I agree with you that a QB's surroundings are likely to change more across years than within each year, but given that the passer rating formula keeps the QB constant, any within-year variation in surrounding cast is being ignored if you just look at across-year variation. In particular, you're ignoring the effect of within-season injuries and all the variation in ability from game to game of the other players (and there are many more of them!). So ignoring game-by-game variation is artificially making the QB more important than he actually is.

Also.. to address the issue about looking at multiple QB's, the similarity in surrounding cast for a given QB decreases as a function of time so you'd actually be better off looking at N consecutive years for a single QB than N different years for N different QB's, or N/K sets of K consecutive years for N/K QB's because you'd have more data on games farther apart in time.

The data you really want is how that QB plays when he changes teams. For example, if you do an ANOVA on Fitzpatrick's adjusted ratings with the 6 different teams he's been on previous to Miami with 300+ passing attempts you see no significant difference:

This year in Miami might be the lone statistically significant season once it's over. That's the kind of data, at least for one QB, that suggests the QB may be the more influential component. But again this is just one QB.

Also.. while you're right that some of Brady's years are significantly different than the others (p-value is less than 0.05), note how similar many of the years are (the blue "box" indicates the middle 25-75% range of adjusted ratings in that year):

I mean.. there's hardly any real difference from 2003-2006 for example (red line is median). So it wouldn't surprise me if within-year variance in surrounding cast during those years was actually larger than across-year variance. But there's no way to really know because all this is confounded by variance introduced by opponent strength.

Anyway.. best not to artificially remove sources of variance, especially if it biases the results. So I'd stick with game-by-game ratings.

So something else to consider (or at least another way to look at it) is what was the most likely outcome from Tannehill using the assumption that he is an average QB ( or slightly less then average for a few posters on this forum).

So we have a QB that is coming in cold against teams in midseason form, who did not get an offseason with the A team or practices with the A team right up to the week he became the starter.

The highest percentage outcomes up to Cbrads 120 passes would easily be as follows:

1st Slightly below average play as he gets accustomed to the new environment.
2nd Average play across the board
3rd slightly above average play
4th bad play, clear signs of struggling
5th good play, looks competent
6th horrible play, disasterous outcome
7th Amazing play looks like a legit top 5 QB

I think its pretty clear that it becomes considerably less likely with each step down the chart. number 7 should be practically impossible, in the .001 to .0001 percentage range. In perfect conditions it would be nearly impossible to hit 7th option, and I don't think you can point to anything about Tannehills conditions that are "perfect", and yet here we are and he is clearly up to this point playing in the 7th option category.

So you really have 2 choices right now. You can stick by the notion that we just saw the lottery get won by the 1 guy we ran out of here with pitch forks.... or we have to re-evaluate our original hypothesis that he is in fact JUST an average QB.

"Phins_to_Win wrote:

So something else to consider (or at least another way to look at it) is what was the most likely outcome from Tannehill using the assumption that he is an average QB ( or slightly less then average for a few posters on this forum).

So we have a QB that is coming in cold against teams in midseason form, who did not get an offseason with the A team or practices with the A team right up to the week he became the starter.

The highest percentage outcomes up to Cbrads 120 passes would easily be as follows:

1st Slightly below average play as he gets accustomed to the new environment.
2nd Average play across the board
3rd slightly above average play
4th bad play, clear signs of struggling
5th good play, looks competent
6th horrible play, disasterous outcome
7th Amazing play looks like a legit top 5 QB

I think its pretty clear that it becomes considerably less likely with each step down the chart. number 7 should be practically impossible, in the .001 to .0001 percentage range. In perfect conditions it would be nearly impossible to hit 7th option, and I don't think you can point to anything about Tannehills conditions that are "perfect", and yet here we are and he is clearly up to this point playing in the 7th option category.

So you really have 2 choices right now. You can stick by the notion that we just saw the lottery get won by the 1 guy we ran out of here with pitch forks.... or we have to re-evaluate our original hypothesis that he is in fact JUST an average QB.

Yeah, this is another good example of where statistical analysis matters. People don't have good intuitions about what the probability of a series of events is given past data. Here's the comparable graph to that for Brady in my post above:

The p-value here is 0.2557 which is still well above 0.05 meaning it's not yet statistically significant, and that's precisely because he's only played 6 games. If that graph is exactly the same at year's end with 11 games I guarantee you that p-value is well less than 0.05 and we will definitely have to adjust our assumptions about Tannehill. Specifically, we'll know with certainty he can play at an elite level (if this continues) with proper surrounding cast over 11 games, which was not statistically likely by any stretch before.

btw.. 0.2557 means 25.57% likely Tannehill's ratings in 2019 can be explained by random variation alone (ignoring factors we don't know how to adjust for like starting for a new team midseason etc..). So you can see just how far off your intuition about probabilities is.

With Tannehill's 155.8 rating in his last game he now has his best 6 game streak in his career, but before that there were multiple 5-game streaks that were comparable, so it's not anywhere near as unlikely as you think it is. Anyway, like I've said many times before, let's let this play out. If he keeps this up, the stats will show it.

Oh.. btw, the threshold for passing attempts I use is 150 not 120 (and Tannehill does have more than 150 now).

"cbrad wrote:

Yeah, this is another good example of where statistical analysis matters. People don't have good intuitions about what the probability of a series of events is given past data. Here's the comparable graph to that for Brady in my post above:

The p-value here is 0.2557 which is still well above 0.05 meaning it's not yet statistically significant, and that's precisely because he's only played 6 games. If that graph is exactly the same at year's end with 11 games I guarantee you that p-value is well less than 0.05 and we will definitely have to adjust our assumptions about Tannehill. Specifically, we'll know with certainty he can play at an elite level (if this continues) with proper surrounding cast over 11 games, which was not statistically likely by any stretch before.

btw.. 0.2557 means 25.57% likely Tannehill's ratings in 2019 can be explained by random variation alone (ignoring factors we don't know how to adjust for like starting for a new team midseason etc..). So you can see just how far off your intuition about probabilities is.

With Tannehill's 155.8 rating in his last game he now has his best 6 game streak in his career, but before that there were multiple 5-game streaks that were comparable, so it's not anywhere near as unlikely as you think it is. Anyway, like I've said many times before, let's let this play out. If he keeps this up, the stats will show it.

Oh.. btw, the threshold for passing attempts I use is 150 not 120 (and Tannehill does have more than 150 now).

Isn't that probability of him having a 5 game stretch like this vs having that exact 5 games from a given point? If he played the next 5 years, then I assume the likelihood he could have this 5 game stretch goes up, but to have him do it immediately after switching teams with no cherry picking (you HAVE TO USE his first 5 games as a Titan) it seems like the odds would go astronomically against you.

So if you told me I was going to flip a coin 100 times and sometime in that 100 flips I'm going to get 5 heads in a row, I would say sure that's possible. But IF you told me the next time you flip the coin it will be 5 heads in a row, that becomes insanely more unlikely.

"cbrad wrote:

Yeah, this is another good example of where statistical analysis matters. People don't have good intuitions about what the probability of a series of events is given past data. Here's the comparable graph to that for Brady in my post above:

The p-value here is 0.2557 which is still well above 0.05 meaning it's not yet statistically significant, and that's precisely because he's only played 6 games. If that graph is exactly the same at year's end with 11 games I guarantee you that p-value is well less than 0.05 and we will definitely have to adjust our assumptions about Tannehill. Specifically, we'll know with certainty he can play at an elite level (if this continues) with proper surrounding cast over 11 games, which was not statistically likely by any stretch before.

btw.. 0.2557 means 25.57% likely Tannehill's ratings in 2019 can be explained by random variation alone (ignoring factors we don't know how to adjust for like starting for a new team midseason etc..). So you can see just how far off your intuition about probabilities is.

With Tannehill's 155.8 rating in his last game he now has his best 6 game streak in his career, but before that there were multiple 5-game streaks that were comparable, so it's not anywhere near as unlikely as you think it is. Anyway, like I've said many times before, let's let this play out. If he keeps this up, the stats will show it.

Oh.. btw, the threshold for passing attempts I use is 150 not 120 (and Tannehill does have more than 150 now).

Also, while stating that there are things we can't adjust for, but those things clearly make a difference, how can you turn around and say my probability is far off? If there is no way to capture the variance then that means there is a clear X factor, and that means that ANY statistical attempt to give a % will be wrong. In fact the only thing we can be sure of from a statistics stand point is that the answer from the math will be wrong (or incomplete if you prefer).

"Phins_to_Win wrote:

Isn't that probability of him having a 5 game stretch like this vs having that exact 5 games from a given point? If he played the next 5 years, then I assume the likelihood he could have this 5 game stretch goes up, but to have him do it immediately after switching teams with no cherry picking (you HAVE TO USE his first 5 games as a Titan) it seems like the odds would go astronomically against you.

So if you told me I was going to flip a coin 100 times and sometime in that 100 flips I'm going to get 5 heads in a row, I would say sure that's possible. But IF you told me the next time you flip the coin it will be 5 heads in a row, that becomes insanely more unlikely.

That's a good question actually.

Most statistical analysis for continuous data (ratings aren't ordinal.. they can be any number from 0 to 158.3 for passer rating), for better or for worse, ignores ordering with sets. That is, the assumptions of ANOVA (and of the t-test which I also use) include independence of observations which means that the probability of observing one rating doesn't affect the probability of observing any other rating.

In other words, that test doesn't know about the ordering of ratings within a season. It knows about "sets" but not "ordered sets" (a set where you care about the ordering of the elements in the set), which means it doesn't know about "streaks" per se. It can calculate the probability of what to you and me looks like a "streak" but to the test itself it's not a streak at all, just a set of ratings whose ordering doesn't matter. So from that test's perspective the starting point is irrelevant because all it cares about is the set of ratings observed. So ANOVA is just comparing multiple sets of ratings with no "streaks" in them (you can randomize the rating orders and get the same result).

The problem with creating a statistical test for passer rating "streaks" where starting point matters is that the ratings are not ordinal. That is, they're not restricted to a finite set of ratings such as integers from 0 to 5 (in this case only 6 possible ratings). If we restricted the set of possibilities like that, then you can start calculating the probability of different streaks starting at different points. But for data defined on a continuous axis that's really difficult and there really aren't statistical tests I know of that deal with that.

"Phins_to_Win wrote:

Also, while stating that there are things we can't adjust for, but those things clearly make a difference, how can you turn around and say my probability is far off? If there is no way to capture the variance then that means there is a clear X factor, and that means that ANY statistical attempt to give a % will be wrong. In fact the only thing we can be sure of from a statistics stand point is that the answer from the math will be wrong (or incomplete if you prefer).

We're talking about going from 25.57% to 0.001%. It's untenable to suggest probabilities will change by that much just because you change teams and conditions. This isn't a 5% or 10% change we're talking about.. it's a 25,000 fold change in probabilities lol.

"cbrad wrote:

I agree with you that a QB's surroundings are likely to change more across years than within each year, but given that the passer rating formula keeps the QB constant, any within-year variation in surrounding cast is being ignored if you just look at across-year variation. In particular, you're ignoring the effect of within-season injuries and all the variation in ability from game to game of the other players (and there are many more of them!). So ignoring game-by-game variation is artificially making the QB more important than he actually is.

I'm not sure I see that when presumably those factors would be represented in the season passer rating. Why would game-to-game variability of the surrounding cast be more important than the overall performance of the surrounding cast throughout the season?

"The Guy wrote:

I'm not sure I see that when presumably those factors would be represented in the season passer rating. Why would game-to-game variability of the surrounding cast be more important than the overall performance of the surrounding cast throughout the season?

For estimating the effect of surrounding cast? Suppose you have two different seasons A and B, and both are statistically similar. In other words, season ending passer rating washes out all game-by-game variability in both seasons. However, in season A let's suppose injuries took a toll in the first half of the season while in season B it was the second half. Season ending ratings can't see the effect of those injuries on QB performance while game-by-game ratings would.

Extreme example of course but that shows how you could underestimate the importance of surrounding cast by ignoring game-by-game ratings. In general, the more variability you remove from everything that is NOT kept constant (and the QB is the only thing required to remain constant in the formula) the more you'll bias the data towards showing that what remains constant is more influential.

Also.. just so it's clear, we're not removing the effect of different seasons here. That's still in the game-by-game ratings. So it's really a case of retaining information vs. removing it.

"Phins_to_Win wrote:

So something else to consider (or at least another way to look at it) is what was the most likely outcome from Tannehill using the assumption that he is an average QB ( or slightly less then average for a few posters on this forum).

So we have a QB that is coming in cold against teams in midseason form, who did not get an offseason with the A team or practices with the A team right up to the week he became the starter.

The highest percentage outcomes up to Cbrads 120 passes would easily be as follows:

1st Slightly below average play as he gets accustomed to the new environment.
2nd Average play across the board
3rd slightly above average play
4th bad play, clear signs of struggling
5th good play, looks competent
6th horrible play, disasterous outcome
7th Amazing play looks like a legit top 5 QB

I think its pretty clear that it becomes considerably less likely with each step down the chart. number 7 should be practically impossible, in the .001 to .0001 percentage range. In perfect conditions it would be nearly impossible to hit 7th option, and I don't think you can point to anything about Tannehills conditions that are "perfect", and yet here we are and he is clearly up to this point playing in the 7th option category.

So you really have 2 choices right now. You can stick by the notion that we just saw the lottery get won by the 1 guy we ran out of here with pitch forks.... or we have to re-evaluate our original hypothesis that he is in fact JUST an average QB.

You could say the same thing however about Andy Dalton's 16 games in 2015, where he posted a passer rating about 18 points above his career number. What was the likelihood Andy Dalton would have a passer rating of 106.2 in 2015, when his highest season passer rating before that had been 88.8, and he'd had other seasons of only 80.4, 87.4, and 83.5?

"cbrad wrote:

For estimating the effect of surrounding cast? Suppose you have two different seasons A and B, and both are statistically similar. In other words, season ending passer rating washes out all game-by-game variability in both seasons. However, in season A let's suppose injuries took a toll in the first half of the season while in season B it was the second half. Season ending ratings can't see the effect of those injuries on QB performance while game-by-game ratings would.

Extreme example of course but that shows how you could underestimate the importance of surrounding cast by ignoring game-by-game ratings. In general, the more variability you remove from everything that is NOT kept constant (and the QB is the only thing required to remain constant in the formula) the more you'll bias the data towards showing that what remains constant is more influential.

Also.. just so it's clear, we're not removing the effect of different seasons here. That's still in the game-by-game ratings. So it's really a case of retaining information vs. removing it.

So then how would we explain the fact that the passer ratings for the group of P. Manning, Rodgers, Wilson, Brees, and Brady vary at a level significantly higher than that for the group of Tannehill, Dalton, Flacco, Newton, and E. Manning, over many years? Is there an explanation for that finding that illustrates the need for a game-by-game analysis rather than a season-by-season one?

"cbrad wrote:

That's a good question actually.

Most statistical analysis for continuous data (ratings aren't ordinal.. they can be any number from 0 to 158.3 for passer rating), for better or for worse, ignores ordering with sets. That is, the assumptions of ANOVA (and of the t-test which I also use) include independence of observations which means that the probability of observing one rating doesn't affect the probability of observing any other rating.

In other words, that test doesn't know about the ordering of ratings within a season. It knows about "sets" but not "ordered sets" (a set where you care about the ordering of the elements in the set), which means it doesn't know about "streaks" per se. It can calculate the probability of what to you and me looks like a "streak" but to the test itself it's not a streak at all, just a set of ratings whose ordering doesn't matter. So from that test's perspective the starting point is irrelevant because all it cares about is the set of ratings observed. So ANOVA is just comparing multiple sets of ratings with no "streaks" in them (you can randomize the rating orders and get the same result).

The problem with creating a statistical test for passer rating "streaks" where starting point matters is that the ratings are not ordinal. That is, they're not restricted to a finite set of ratings such as integers from 0 to 5 (in this case only 6 possible ratings). If we restricted the set of possibilities like that, then you can start calculating the probability of different streaks starting at different points. But for data defined on a continuous axis that's really difficult and there really aren't statistical tests I know of that deal with that.

We're talking about going from 25.57% to 0.001%. It's untenable to suggest probabilities will change by that much just because you change teams and conditions. This isn't a 5% or 10% change we're talking about.. it's a 25,000 fold change in probabilities lol.

Ok so let me see if I can blow your mind on this. you say that its all random then from that perspective it was equally likely (actually more likely but lets get into that some other time) that he would have started out his 5 game hot streak on the 2nd game, meaning it turned into just 4 games, it was also equally likely that he would have started randomly on his 3rd game meaning it turned into only 3 games, and equally likely that he started it on the 4th game and 5th. Each of these possibilities gets 25%, so what we have is 125% that Tannehill was going to have an amazing Game for his first game??? Mathematically impossible to have any other outcome. Not bad for an average QB...

"The Guy wrote:

So then how would we explain the fact that the passer ratings for the group of P. Manning, Rodgers, Wilson, Brees, and Brady vary at a level significantly higher than that for the group of Tannehill, Dalton, Flacco, Newton, and E. Manning, over many years? Is there an explanation for that finding that illustrates the need for a game-by-game analysis rather than a season-by-season one?

In practice I don't think you'll see too many differences if your only goal is to argue the QB was a major reason for the observed differences, though sample size will naturally be larger with game-by-game ratings so that's a plus.

But that's not the question here. The question is what data would you use to try and infer whether the QB or his surroundings is more influential, and for that question you definitely don't want to remove sources of variance that matter. Regardless.. I don't think this question can be answered with the stats we have, so from that perspective it's irrelevant which data you use.. we just don't know.

"Phins_to_Win wrote:

Ok so let me see if I can blow your mind on this. you say that its all random then from that perspective it was equally likely (actually more likely but lets get into that some other time) that he would have started out his 5 game hot streak on the 2nd game, meaning it turned into just 4 games, it was also equally likely that he would have started randomly on his 3rd game meaning it turned into only 3 games, and equally likely that he started it on the 4th game and 5th. Each of these possibilities gets 25%, so what we have is 125% that Tannehill was going to have an amazing Game for his first game??? Mathematically impossible to have any other outcome. Not bad for an average QB...

Well.. you blew my mind alright. Not sure I've seen so many errors in a single post.

Like I just said, from the statistical test's point of view there is no such thing as a "streak". There is only a set of data. You can reorder the ratings in each set (season) any way you want it's the same from that test's point of view. And obviously a 5-game streak in a 6 game period can't begin from game 3. So the probability of that isn't 25% it's 0%.

Point is.. when you look at the sets of ratings from different years (regardless of ordering within the sets) what Tannehill has done so far is still consistent with random variation. As I said though, it won't be if he continues performing this way.

"cbrad wrote:

Well.. you blew my mind alright. Not sure I've seen so many errors in a single post.

Like I just said, from the statistical test's point of view there is no such thing as a "streak". There is only a set of data. You can reorder the ratings in each set (season) any way you want it's the same from that test's point of view. And obviously a 5-game streak in a 6 game period can't begin from game 3. So the probability of that isn't 25% it's 0%.

Point is.. when you look at the sets of ratings from different years (regardless of ordering within the sets) what Tannehill has done so far is still consistent with random variation. As I said though, it won't be if he continues performing this way.

If the start of a five game streak is based on randomness, then it is equally likely that the start of a 4 game 3 game 2 game and 1 game streak starts on that same game. My point was there is no way its 25% cause it doesn't stand to reason that the much easier 4 game 3 game 2 game and 1 game has a less % chance. Giving it the same % chance makes it impossible to have a failed game 1. so I don't think the 25% can be anywhere near possible in being correct for the 5 game winning streak.

"Phins_to_Win wrote:

If the start of a five game streak is based on randomness, then it is equally likely that the start of a 4 game 3 game 2 game and 1 game streak starts on that same game. My point was there is no way its 25% cause it doesn't stand to reason that the much easier 4 game 3 game 2 game and 1 game has a less % chance. Giving it the same % chance makes it impossible to have a failed game 1. so I don't think the 25% can be anywhere near possible in being correct for the 5 game winning streak.

You're not getting it. Take the ratings Tannehill has had so far in 6 games with the Titans: 78.1, 120.1, 109.8, 82.3, 133.9, 155.8.

Now.. that's the actual order of the ratings. To you that looks like the 5 game "streak" started on game #2. Now suppose you randomly reorder those SAME ratings. For example: 109.8, 82.3, 155.8, 78.1, 120.1, 133.9. What does it look like to you now? That it was a 6-game "streak"? I don't know.. doesn't matter. To the statistical test the ordering is irrelevant and there is NO streak. There is no "5 game streak", there is no "6 game streak", there is simply NO streak.

I used the word "streak" because others have used that and it usually helps with communication, but from the point of view of the statistical test there is NO streak. There are just sets of ratings (one for each season). So you can't start with the assumptions of the statistical test (the independence of observations assumption) and then say "suppose the streak started on game X". That's a meaningless assertion.

"cbrad wrote:

You're not getting it. Take the ratings Tannehill has had so far in 6 games with the Titans: 78.1, 120.1, 109.8, 82.3, 133.9, 155.8.

Now.. that's the actual order of the ratings. To you that looks like the 5 game "streak" started on game #2. Now suppose you randomly reorder those SAME ratings. For example: 109.8, 82.3, 155.8, 78.1, 120.1, 133.9. What does it look like to you now? That it was a 6-game "streak"? I don't know.. doesn't matter. To the statistical test the ordering is irrelevant and there is NO streak. There is no "5 game streak", there is no "6 game streak", there is simply NO streak.

I used the word "streak" because others have used that and it usually helps with communication, but from the point of view of the statistical test there is NO streak. There are just sets of ratings (one for each season). So you can't start with the assumptions of the statistical test (the independence of observations assumption) and then say "suppose the streak started on game X". That's a meaningless assertion.

LOL, I so don't want to have any part of this RT conversation....but I think you're completely missing his point. A "streak" is a series of games; nothing more, nothing less. What he's saying is, if RT had a hot streak in 2016, had one briefly in 2018 then another one this year, he's saying that it's likely that he'll have a similar "streak" of good games in 2020 and beyond.

Note- he's not saying RT will be an elite QB...he's saying that the past several years have shown a pattern where RT got hot for a number of games within the season. The word "streak" is meaningless here other than to represent consecutive games, and I think every 7+ year QB would have "streaks" to be found (hence, how they made it 7+ years in the league).

Anyway, I posted all of that to offer this- what if we looked at five random QB's plus Tannehill of their best 3-game stretch of a season and their worst 3-game stretch...then averaged the two? Wouldn't that give a better indication of "streakiness" in a QB?

LOL, yes, I just invented a word. Edit- crap, no I didn't =)

But cbrad, we know that the more times you do something, the harder it becomes to do. Like shooting a free throw. Making one is pretty easy. Making two in a row. Harder. 5 in a row? 100 in a row? So yes, streaks certainly matter. You aren't dealing with numbers on a page, you're dealing with human beings.

"KeyFin wrote:

LOL, I so don't want to have any part of this RT conversation....but I think you're completely missing his point. A "streak" is a series of games; nothing more, nothing less. What he's saying is, if RT had a hot streak in 2016, had one briefly in 2018 then another one this year, he's saying that it's likely that he'll have a similar "streak" of good games in 2020 and beyond.

No I think you've completely missed the point of the conversation KeyFin. Phins_to_Win asked a very good question in post #629 so you need to go back and read from there. He's not saying anything like what you're talking about. He was asking whether the probabilities calculated by the statistical test I used were biased upwards because they didn't consider when the streak started. It's a good question.. but the answer is equally important because it tells you that the assumptions of the statistical test make that question meaningless.

"KeyFin wrote:

Anyway, I posted all of that to offer this- what if we looked at five random QB's plus Tannehill of their best 3-game stretch of a season and their worst 3-game stretch...then averaged the two? Wouldn't that give a better indication of "streakiness" in a QB?

"resnor wrote:

But cbrad, we know that the more times you do something, the harder it becomes to do. Like shooting a free throw. Making one is pretty easy. Making two in a row. Harder. 5 in a row? 100 in a row? So yes, streaks certainly matter. You aren't dealing with numbers on a page, you're dealing with human beings.

What resnor just pointed out leads to the really important follow-up question to my response to Phins_to_Win: whether the implicit assumption of that statistical test – that "streaks" only exist in the minds of the human observer and not in reality – is accurate or not.

Enter the hot hand fallacy. Psychologists demonstrated decades ago that humans were perceiving "streaks" in all kinds of sports when the data was actually consistent with randomness (started with basketball players making X number of consecutive baskets). A few researchers got a Nobel Prize for showing that, among other behavioral and decision making biases they discovered. Now, it turns out that in recent years there is research showing that a "hot hand" can exist in some cases. So what's intuitive to humans may have some merit. But it's still the case that humans find patterns in randomness, and to a 1st approximation (unless proven otherwise in a specific case) the assumption that "streaks" aren't real is a good starting point.

So I wouldn't go looking for evidence of "streakiness" given the huge amount of research showing that in most cases it's more consistent with randomness. Or if one is to look for that evidence one shouldn't do so casually – the analysis has to hold up to scrutiny.

Oh and resnor.. that statistical test calculates the probability of observing any set of ratings given other sets of ratings, so whether we call it a "streak" or not, as long as Tannehill did well X out of N times, the difficulty of that is taken into account.

"cbrad wrote:

Yeah, this is another good example of where statistical analysis matters. People don't have good intuitions about what the probability of a series of events is given past data. Here's the comparable graph to that for Brady in my post above:

The p-value here is 0.2557 which is still well above 0.05 meaning it's not yet statistically significant, and that's precisely because he's only played 6 games. If that graph is exactly the same at year's end with 11 games I guarantee you that p-value is well less than 0.05 and we will definitely have to adjust our assumptions about Tannehill. Specifically, we'll know with certainty he can play at an elite level (if this continues) with proper surrounding cast over 11 games, which was not statistically likely by any stretch before.

btw.. 0.2557 means 25.57% likely Tannehill's ratings in 2019 can be explained by random variation alone (ignoring factors we don't know how to adjust for like starting for a new team midseason etc..). So you can see just how far off your intuition about probabilities is.

With Tannehill's 155.8 rating in his last game he now has his best 6 game streak in his career, but before that there were multiple 5-game streaks that were comparable, so it's not anywhere near as unlikely as you think it is. Anyway, like I've said many times before, let's let this play out. If he keeps this up, the stats will show it.

Oh.. btw, the threshold for passing attempts I use is 150 not 120 (and Tannehill does have more than 150 now).

But even this, while methodologically sound of course, illustrates the problem with the interpretation, in that there is nothing telling us definitively that the Titans' surrounding cast is any better than what Tannehill had with the Dolphins.

If Tannehill's pattern of play continues, it'll be just as plausible that Tannehill was humbled by the Dolphins' giving up on him and has redoubled his efforts or improved his mentality such that he's a better player individually.

Again the problem is that nobody is reliably measuring the effect of surrounding casts on quarterbacks' performances. Just because many people thought Tannehill would play better with a better surrounding cast doesn't mean he's actually getting that at the present time.

You certainly can't use his better play, alone, as evidence of his having a better surrounding cast and then simply say, "see there -- I told ya that would happen." First you need to actually determine whether he has a better surrounding cast in some empirically reliable manner.

"cbrad wrote:

No I think you've completely missed the point of the conversation KeyFin. Phins_to_Win asked a very good question in post #629 so you need to go back and read from there. He's not saying anything like what you're talking about. He was asking whether the probabilities calculated by the statistical test I used were biased upwards because they didn't consider when the streak started. It's a good question.. but the answer is equally important because it tells you that the assumptions of the statistical test make that question meaningless.

What resnor just pointed out leads to the really important follow-up question to my response to Phins_to_Win: whether the implicit assumption of that statistical test – that "streaks" only exist in the minds of the human observer and not in reality – is accurate or not.

Enter the hot hand fallacy. Psychologists demonstrated decades ago that humans were perceiving "streaks" in all kinds of sports when the data was actually consistent with randomness (started with basketball players making X number of consecutive baskets). A few researchers got a Nobel Prize for showing that, among other behavioral and decision making biases they discovered. Now, it turns out that in recent years there is research showing that a "hot hand" can exist in some cases. So what's intuitive to humans may have some merit. But it's still the case that humans find patterns in randomness, and to a 1st approximation (unless proven otherwise in a specific case) the assumption that "streaks" aren't real is a good starting point.

So I wouldn't go looking for evidence of "streakiness" given the huge amount of research showing that in most cases it's more consistent with randomness. Or if one is to look for that evidence one shouldn't do so casually – the analysis has to hold up to scrutiny.

Oh and resnor.. that statistical test calculates the probability of observing any set of ratings given other sets of ratings, so whether we call it a "streak" or not, as long as Tannehill did well X out of N times, the difficulty of that is taken into account.

What can't be counted or measured however is the brain and how it reacts to a "streak" good or bad. It IS random but the human element can come in and if a guy has missed 4 free throws in a row?? I think the percentages actually go down that he makes it. Our brains can make things harder or easier or some people have the ability to perform on par. Depends on a plethora of variables. And frankly the mental make-up of 'X'.

"The Guy wrote:

But even this, while methodologically sound of course, illustrates the problem with the interpretation, in that there is nothing telling us definitively that the Titans' surrounding cast is any better than what Tannehill had with the Dolphins.

If Tannehill's pattern of play continues, it'll be just as plausible that Tannehill was humbled by the Dolphins' giving up on him and has redoubled his efforts or improved his mentality such that he's a better player individually.

Again the problem is that nobody is reliably measuring the effect of surrounding casts on quarterbacks' performances. Just because many people thought Tannehill would play better with a better surrounding cast doesn't mean he's actually getting that at the present time.

You certainly can't use his better play, alone, as evidence of his having a better surrounding cast and then simply say, "see there -- I told ya that would happen." First you need to actually determine whether he has a better surrounding cast in some empirically reliable manner.

Yeah it's important to point out that no matter how much data you obtain there is technically an infinite number of hypotheses still consistent with that data – any mathematician can fit an infinite number of distinct functions through any finite amount of data. So no matter what you do you can't definitively pin down the causes of what occurred.

However, you still want to restrict your hypotheses to things that are measurable, like a change in surrounding cast. So while it's possible that "Tannehill redoubled his efforts" where's the evidence of this? At least you have evidence there was a change in surrounding cast. So I wouldn't put these hypotheses on the same level – filter out the ones for which you don't have evidence first.

"Fin-O wrote:

What can't be counted or measured however is the brain and how it reacts to a "streak" good or bad. It IS random but the human element can come in and if a guy has missed 4 free throws in a row?? I think the percentages actually go down that he makes it. Our brains can make things harder or easier or some people have the ability to perform on par. Depends on a plethora of variables. And frankly the mental make-up of 'X'.

That's exactly what was being counted though.

The initial studies looked at the conditional probability of making or missing a basket after X consecutive successes or X consecutive misses. Actually, to be technical, they sometimes conditioned on "X or more" consecutive successes or misses, and later studies pointed out that there are some subtle changes in the probability that those Nobel Prize winners weren't taking into account lol. Either way, that was precisely the condition researchers were looking at to show that the probability of making a basket or missing it after X consecutive successes or misses was essentially identical.. with some random variation (binomial probabilities).

"cbrad wrote:

No I think you've completely missed the point of the conversation KeyFin. Phins_to_Win asked a very good question in post #629 so you need to go back and read from there. He's not saying anything like what you're talking about. He was asking whether the probabilities calculated by the statistical test I used were biased upwards because they didn't consider when the streak started. It's a good question.. but the answer is equally important because it tells you that the assumptions of the statistical test make that question meaningless.

What resnor just pointed out leads to the really important follow-up question to my response to Phins_to_Win: whether the implicit assumption of that statistical test – that "streaks" only exist in the minds of the human observer and not in reality – is accurate or not.

Enter the hot hand fallacy. Psychologists demonstrated decades ago that humans were perceiving "streaks" in all kinds of sports when the data was actually consistent with randomness (started with basketball players making X number of consecutive baskets). A few researchers got a Nobel Prize for showing that, among other behavioral and decision making biases they discovered. Now, it turns out that in recent years there is research showing that a "hot hand" can exist in some cases. So what's intuitive to humans may have some merit. But it's still the case that humans find patterns in randomness, and to a 1st approximation (unless proven otherwise in a specific case) the assumption that "streaks" aren't real is a good starting point.

So I wouldn't go looking for evidence of "streakiness" given the huge amount of research showing that in most cases it's more consistent with randomness. Or if one is to look for that evidence one shouldn't do so casually – the analysis has to hold up to scrutiny.

Oh and resnor.. that statistical test calculates the probability of observing any set of ratings given other sets of ratings, so whether we call it a "streak" or not, as long as Tannehill did well X out of N times, the difficulty of that is taken into account.

You're arguing there are no streaks in statistics. You're saying they're just numbers, doesn't matter the order. I'm telling you it most certainly does. You're dealing with humans and their performance, and streaks are real. Anyone who's played any level of organized sports understands this. Playing high school basketball, for instance, some nights you just can't miss. You're "in the zone." Sometimes you can string that together over several games. Better players get "in the zone" more, and are better at stringing games together.

I don't care what statistics and statisticians say about streaks.

Another example...let's say a guy is a 65% comp rate for the season. Then one game he throws for a 90% comp rate. He then follows up with 4 more games of 85, 87, 75, 92. That's 5 games well above his average. Now, it may not be abnormal, in that he might follows that up with 5 games of 65, 55, 58, 63, 61, which could lower him back to his average (fake numbers, don't think they actually average out to 65, but you get the point I'm making). Would still make that 5 games streak significant. Of course you'd need to look at what was going on, try to figure out why he was better those 5 games.

"resnor wrote:

You're arguing there are no streaks in statistics. You're saying they're just numbers, doesn't matter the order. I'm telling you it most certainly does. You're dealing with humans and their performance, and streaks are real. Anyone who's played any level of organized sports understands this. Playing high school basketball, for instance, some nights you just can't miss. You're "in the zone." Sometimes you can string that together over several games. Better players get "in the zone" more, and are better at stringing games together.

I don't care what statistics and statisticians say about streaks.

Yeah like I said the evidence is more mixed today than decades ago. There is evidence that in some cases the "hot hand fallacy" itself was a fallacy, but in many cases it still holds. Anyway, your reaction to that was exactly the same reaction by professional athletes and coaches. Actually, that's the same kind of reaction Sabermetrics and most statistical analysis of sports that didn't fit with what coaches thought elicited. Over time though that resistance decreases as evidence mounts, and the statistics get incorporated by professional coaches.

NBA and MLB are hugely into analytics now, and NFL is slowly following suit. So I think it's better to just take the point of view that not everything that fits with intuition based on experience is actually correct. But that's up to the individual. Trend is clear though.