Rankings Mythology: Value Based Drafting

Scott Peak Posted On August 16, 2014

This is the final installment of Rankings Mythology using 2013 projections data.

I’ve always wondered how truly accurate projections are in fantasy football, and this is particularly important when using a value-based drafting (VBD) approach to team-building. We hear great things about VBD and it is an interesting concept for sure. But how can VBD function as an effective tool for drafting a team when it relies heavily on projections accuracy?

I subscribe to a well-known website in the fantasy football community and I decided to check their projections data before the 2013 NFL season and compare it to the actual points scored at season’s end. The concept was very simple – how close were its authors in their projections to the actual points scored by each player? I evaluated four well-known authors in the fantasy football community. I reviewed projections for the top 24 quarterbacks (QB) and tight ends (TE), and the top 46 running backs (RB) and wide receivers (WR). I previously published detailed articles on each position and these can be found in the archives of Dynasty League Football.

Here is a summary of my data collection:

[am4show have=’p2;p3;p4;p5;p6;’ guest_error=’sub_message’ user_error=’sub_message’ ]

myth1

We take for granted that experts in the field will be accurate enough to make a difference in team-building. I don’t intend to criticize the four authors evaluated for this article, but rather to show that if four successful fantasy football authors can only predict 30% of the top 10 running backs, how useful are projections? Running backs fared the worst, with wide receivers a close second. Quarterback accuracy was decent in the top 20, but that is a reflection of the position having a more limited player pool making it easier to get it right. We can still feel more comfortable that quarterback projections are not likely to hurt our teams, as chances are reasonably good we will have at least one top 20 quarterback on our rosters. The same cannot be said for running backs and wide receivers. Tight ends did a little better than running backs and wide receivers, but not as good as quarterbacks in projections accuracy.

It’s not just accuracy in predicting top 20 finishes that is a problem, it’s the tendency to overestimate player projections that is a big problem for fantasy football players. Overestimates are a killer for fantasy teams, as those will be the busts that fail to reach expectations. In reviewing PPD, average overestimated points ranged from 40 to 75 depending on the position, and that is enough to adversely impact a fantasy football team when all roster spots are considered.

I don’t think this level of accuracy is acceptable as a benchmark for projections data, and the fantasy football community must develop better tools to improve accuracy in predicting player performance.

If projections are not at an acceptable standard, then how could this impact VBD? I’m sure DLF readers understand the concept of VBD, but briefly it predicts available player value relative to a baseline at each position. For example, in a 12-team league that starts one quarterback, VBD uses the “worst starter,” QB12, as the benchmark in determining relative value of a player in the draft. If an available quarterback has more point value relative to the worst available quarterback starter, as compared to available talent at other positions, then quarterback is the best value pick. This is a neat concept and I like it in theory. VBD as a concept will never reach its full potential given its reliance on projections data that are not accurate.

I decided to run a simple test of VBD. Using draft software that allowed me to run an automated mock using VBD principles, I was able to run a simulated draft using both projections data and actual data from 2013. The software allowed me to add in all data, projected or actual, then run a VBD mock using both data sets. If VBD truly works as a concept, mocks using projections data should be as good or better than mocks using actual points. I ran the mock 10 rounds, or 120 players, and that is considered to be the breaking point where VBD is most effective. After 120 players, the sample size becomes too large and accuracy of projections becomes an even greater problem. I ran mocks using data from author #1 and author #2 of the Rankings Mythology data set. Identical scoring formats were used for both mocks, and team points below indicate how teams would have fared at the end of the 2013 fantasy football season.

myth6

Average points scored for all teams using projections data from author #1 is 130 points below teams using actual data, a difference of eight points per game using a 16 game schedule. The best scoring team using projections data would have finished second overall in total points (team 7), but only three teams using projections data would have finished in the top 12 for all 24 teams (25%).

myth7 Average points scored for all teams using projections data from author #2 was 176 points below teams using actual data, a difference of 11 points per game using a 16 game schedule. The best scoring team using projections data would have finished third overall in total points (team 11), but four teams using projections data would have finished in the top 12 for all 24 teams (33%).

If VBD results in a team scoring 8-11 points less per game, how much will that impact an owner in the standings? I reviewed final scores for the 2013 season in two dynasty leagues that I’m in, with a sample size of 228 games. 15-17% of games were decided within eight points. That’s not a large amount, but enough to change the fate of a dynasty season.

This review has limitations. Total points will not always equate to wins and losses. There are fantasy teams with a large amount of points that don’t meet expectations, mainly from bad luck or week-to-week variability in points. I ran mocks on two authors, so again it’s a limited sampling of data, but results for both data sets were similar to support a trend. Once the 2014 has ended, I plan to run data analysis on more than one website to get a larger sample size.

The intent of this article is not to debunk VBD, but rather to support it, with hopes that improvements in projections could enhance its value. There are factors that will always adversely impact projections, such as unpredictable injuries, a change in game plans or suspensions. I still believe there is room for improvement with player projections, even with unpredictable factors clouding the picture. Until it happens, rankings and projections must be taken with a degree of skepticism and going with your instinct may be the best approach.

[/am4show]