Rankings, The Great Myth: Quarterback Edition

In fantasy football, we accept beliefs that give us a sense of comfort, security and confidence in a course of action. Ranking players in fantasy football is a belief that is accepted as a part of the game, but it is a process rarely questioned. In the world of science, we ask questions to not only find answers, but to seek knowledge that will advance our way of life. Challenging the process is not adversarial, but a necessity to take our world to greater heights. If we don’t have the answers, we aren’t asking the right questions, or we’re not asking them at all. Fantasy football is no different. I don’t think anyone in the world of fantasy football would think rankings are infallible, but how good are they exactly? How are rankings even determined? Statistics? Instinct? Situation? Talent? Luck? It’s important to know the science behind ranking players, so we may improve our practice. I wondered how this issue could be studied, so we could improve the practice of ranking players and find variables that might help us improve efficiency.

In order to conduct this experiment, I needed projections from 2013 so I could compare this data to actual points scored by the same set of players. In this way, we could truly see how efficient projections are relative to final points at season’s end. I pulled preseason projections from four authors at a website that is well-regarded, has an outstanding reputation, and is one that I respect as leader in the field. Final points weeks 1 to 17 were obtained from the same site, using standard scoring.

Given the large amount of data collected, we will spread this across a series of five articles based on position (QB, RB, WR, TE) and will conclude with a mock draft comparing projections data versus final point totals for players included in this study.

Here is a summary of my methods:

1. Preseason projections were recorded for the 2013 season and saved on a spreadsheet.
2. Total points scored were recorded weeks 1 to 17 for all players included in this study.
3. Data for the the top 20 quarterbacks, top 40 running backs, top 40 wide receivers and top 20 tight ends were recorded.
4. Projected point totals were subtracted from actual point totals, and this ended up being a positive or negative number, depending on whether projections were over or underestimated.
5. The accuracy of each author in predicting the exact numerical ranking for each player was recorded.
6. The difference between projected and actual point totals was recorded and presented as both an average and a range of values, both positive and negative.

We will start with quarterbacks.

Table 1

2013 Top 10 Projected Point Differentials at Quarterback*
	Author #1	Author #2	Author #3	Author #4
Aaron Rodgers	+175	+186	+198	+168
Matt Schaub	+138	+142	+144	NR
Josh Freeman	NR	+247	NR	+256
Sam Bradford	+142	+172	+146	NR
Michael Vick	NR	+175	+178	+165
Eli Manning	+53	+85	+61	+58
Peyton Manning	-132	-107	-89	-122
Robert Griffin	-73	-92	-83	-100
Philip Rivers	-70	-50	NR	-50
Andy Dalton	-54	-30	-50	-46

*2013 Projected Point Differential = Projected points – Actual points

My first thought upon reviewing these data was surprise at how far off actual point totals were from initial projections. For example, authors projections were 168-198 points above the actual points scored by Rodgers. The reason is obvious, Rodgers sustained a serious injury. Rodgers injury could never be predicted, but it is still relevant in evaluating projections. After all, if injuries are a reason why projections are not accurate, that is an important variable to consider in ranking future players.

Injuries also influenced Bradford and to some extent Michael Vick. Vick was a preseason sleeper pick and a favorite to rebound as the perfect fit for the Chip Kelly offense. Vick has a long history of injuries and poor decision-making that would doom him as the starter. Kelly has no tolerance for turnovers or mistakes and Vick ended up benched, not from injuries, but Nick Foles was simply better. All signs pointed toward another disappointing season for Vick, and yet we weren’t paying attention. The subconscious desire to see Vick explode in Kelly’s offense was intoxicating, and it clouded judgement. The good news is, we can learn from it, and try not to make that mistake again.

Freeman, Schaub, Eli Manning and Griffin are examples of players who failed to meet expectations on the field. Two authors overestimated Freeman’s production by 247+ points. What were we thinking last year to believe Freeman could reach those expectations? Greg Schiano had a solid first year, but the Buccaneers offensive design was not exactly a juggernaut. Freeman has failed to meet expectations since his promising year in 2010. In fact, Freeman has averaged a paltry 224 fantasy points the past three years, has never scored more than 241 points in a season, yet two authors projected Freeman to finish 2013 with 291 and 300 points. There weren’t many logical reasons to believe Freeman would reach that level of production.

One factor with Griffin is our assumption that all players with anterior cruciate ligament (ACL) tears will return to glory in short order, much like Adrian Peterson. I do think ACL surgery and rehabilitation are improving, but not all players are like Peterson. Perhaps we need to reset our expectations for these players in the future. I don’t think ACL injuries are a career-ender, but we can’t assume a player will beat the odds in such a short period of time like Peterson. In fact, most players won’t, so players coming off serious injury should have expectations tempered when determining projections. Griffin scored 261 points in 2012, and it was reasonable to project an increase in production for 2013. Still, his projections ranged from 334 to 361 points in 2013, and that ended up being far off from his actual point production (183 points).

Eli Manning’s 2013 projections ranged from 292 to 324 points, yet he has averaged 258 points three out of the past four seasons. The Giants did have issues at offensive line, but 20 teams gave up more sacks in 2013, so it’s not all on this position. The Eagles gave up 46 sacks and yet Nick Foles and the entire Eagles offense were prolific scorers in fantasy football. Perhaps the most important factor is a decline in Eli Manning’s supporting cast. Victor Cruz and Hakeem Nicks have seen their numbers decline since 2011. The Giants rushing attack finished dead last in 2011 and middle-of-the-pack in 2012. The Eagles finished top ten in passing, rushing and receiving offense in 2013, so not surprisingly Foles finished in the top five quarterbacks in weeks three to seventeen. If the Giants can improve players around Eli, perhaps he has a chance to bounce back in 2014.

The authors range of projections for Schaub in 2013 were 281 to 287 points, yet he hasn’t reached those numbers his entire career, including his best season in 2009 (261 fantasy points). In fact, Schaub hasn’t even been worth starting in 12 team leagues the past three seasons, scoring no more than 222 points in that period of time. Injuries were an issue in 2011, but he still finished outside the top 12 fantasy QBs in 2010 and 2012 despite starting all 32 games. The numbers seem to suggest a decline in fantasy value for Schaub dating back to 2010, and perhaps identifying similar trends with other players will help improve accuracy with future projections.

Philip Rivers vastly outperformed initial projections for 2013. The range of Rivers’ projected points was 277 to 297 points, and one author didn’t even rank him as a top 20 quarterback. Rivers ended 2013 with 347 points, far above initial projections, and he turned into a strong value for dynasty owners who traded a marginal amount for him or picked him late in 2012 drafts. Improvement in Rivers’ game can be traced to a change in the Chargers’ coaching staff, with head coach Mike McCoy and former offensive coordinator Ken Whisenhunt resurrecting his career. Rivers had the highest completion percentage (69%) of his career, reduced his interceptions (11) and had the second most touchdowns of his career (32) in 2013.

Peyton Manning’s projections are interesting, in that all authors predicted a strong year from him, but nobody could have foreseen this level of production. Still, one of our faults is in visualizing a massive spike in production. Whether it’s fair or not, clearly projections for Manning fell short. Would it have been bold to predict 486 points from Manning last year? Yes. But, it would have been the right call. It’s a reasonable thought, as I recall listening to Lance Zierlein (TheSidelineView) predict Peyton would set all sorts of records on a podcast with Sigmund Bloom last year. It’s also interesting that Manning’s projections from all four authors are within 20 points of each other. Maybe those who rank players are influenced by the opinion of others, and the desire not to be seen as crazy for going against the herd. Perhaps the solution is boldness when the time is right?

Table 2

2013 Projections Finishing Top 10 and Top 20 (%)
	Author #1	Author #2	Author #3	Author #4
Top 10 QB	50%	60%	50%	60%
Top 20 QB	75%	70%	70%	75%

Table 2 presents data on the percentage of correct calls for top 10 and top 20 quarterbacks. The specific rank is not counted, but rather how many quarterbacks each author correctly projected would finish in the top 10 or top 20 of the position. This was as good as it got, in terms of positions. It’s not really surprising as turnover at quarterback is much less compared to other positions, and given there are only 32 starting quarterbacks in the NFL, it’s much easier to call the correct players in the top 10 or top 20 per year. Despite fewer numbers of players to project at quarterback, only 50-60% of projected players finished in the top 10. Top 20 projections improved to 70-75%, which again is not surprising given the number of available players to rank is much less compared to other positions.

Table 3

2013 Projected Rank Number Correct
	Author #1	Author #2	Author #3	Author #4
QB (n=20)	5% (n=1)	5% (n=1)	0	5% (n=1)

Table 3 shows the accuracy of author projections for specific rank numbers. Projected rankings from each author were compared to final rankings, and these data show the number of players who finished at the predicted rank number for each author. The results are sobering, as only three players were correctly ranked across four authors (n=80 total player rankings or 3.75% correct). One author didn’t have a single player finish at their predicted ranking. No author was able to pick more than one player ranking correctly out of 20 quarterbacks.

Table 4

2013 Average Projected Points Differential and Range (Over and Under)
	Author #1	Author #2	Author #3	Author #4
QB (n=20)	+57 (5 to 175)  -63 (-19 to -132)	+95 (29 to 247) -54 (0 to -107)	+75 (1 to 198) -47 (-1 to -89)	+73 (2 to 256) -66 (-20 to -122)

Table 4 shows how far above or below projected points were relative to final point totals for all players, presented as both an average and a range of values. These data show projected points were not accurate for most players, and in most cases far above or below final point totals. I found it surprising that projected points were not close to final point totals for most players, and these differences were remarkably similar across all authors. This suggests that authors may be influenced by their peers, intentionally or not.

Table 5

Total number of over or underestimated player projections
	Author #1	Author #2	Author #3	Author #4
Over (n=20)	13	14	13	14
Under (n=20)	7	6	7	6

Another intriguing pattern is that all authors consistently overestimated player projections, as table 5 shows. All authors overestimated projected points for 65 to 70% of the top 20 quarterbacks, and this suggests that projections tend to be overly optimistic in most cases. It may also support the notion that authors tend to stick close to each other when determining projections. It’s hard to imagine four authors overestimating at such a similar number based on pure chance.

So how can we use this data to improve projections and rankings?

1. Injuries undoubtedly influence player projections and rankings, and in some cases are impossible to avoid, such as with Aaron Rodgers. Still, it may be wise to downgrade projections for players with a history of injuries, such as Michael Vick and Robert Griffin.
2. Coaching changes can impact player production, both positively (Rivers) and negatively (Vick). Vick had other reasons to consider downgrading him, such as a history of turnovers, poor decisions and injuries. Keep an eye on players with coaching changes, and try to decide whether it will be a positive or negative impact on their final point totals.
3. There are players with a trend of declining performance that may go unnoticed, similar to Schaub. Identifying these players and adjusting projections is a wise move.
4. Avoid the herd mentality, and be bold when predicting player projections. It’s clear that author projections are remarkably similar for most players. It’s easier to be wrong amongst friends then all alone. Almost all quarterback projections were within 5-10% of predicted points from all four authors. Given projected points will inevitably be over or underestimated by the herd, why not miss making a bold choice? A bold projection or ranking could pay off, and there really isn’t much to lose in doing so.

Conclusion

Ranking players based on quarterback data is not ideal and is much more arbitrary then we might want to believe. Fantasy football owners might consider player evaluations based less on rank lists and use a more individualized approach, similar to buying and selling stocks. Traders don’t buy or sell stocks based on a rank list, but rather which stocks are under or overvalued. I would consider a similar evaluation of fantasy football players. It’s a good idea to consider expert opinions in evaluating player values, but ultimately fantasy football owners should take risks and not hesitate to go on instinct. After all, playing it safe and relying on rankings, as these data show, may not be the most reliable course of action.

Our next series will analyze projections data for running backs and it’s even more compelling.