Editor’s Note: This article is written by a new addition to Team DLF – Jeffrey Levy. We welcome Jeffrey to the team and look forward to his work in the future!
Do you exclusively use average draft position (ADP) data to fill in the gaps in your player rankings? It seems like a natural thing to do. Few of us have either the time or the expertise to properly break down game film on every player in the NFL. We have to get some help somewhere! At least I’m assuming it’s not just me.
At the same time, did you roll your eyes a little when Donte Moncrief showed up with a February 2015 ADP of 35? Latavius Murray going at #59? How about Odell Beckham, Jr. landing in the top overall spot in March? All aboard the hype train, next stop – top of the rankings! Right?
Well, you may be doing your part to perpetuate the problem. Stay with me and I’ll explain.
A simple model of the ADP problem
[am4show have=’g1;’ guest_error=’sub_message’ user_error=’sub_message’ ]
In order to illustrate the problem, we’re going to simplify things. This is a common strategy any time one is modeling a small element of an otherwise complicated system. To use a quote by Neil Gaiman I like to bring up when teaching undergraduate economics, “The more accurate the map, the more it resembles the territory. The most accurate map possible would be the territory and thus would be perfectly accurate and perfectly useless.” So rather than try to parse through an actual, messy draft to understand the problem, we’ll use some reasonable assumptions to make a simple model:
We have a 12-team, two-round draft, with only a single controversial player followed by a bunch of much less controversial ones. There are 11 owners who value the controversial player at the middle of the second round, at pick 18. There is one owner, however, who values that player at the first pick in the first round.
We can find that player’s average value amongst all the owners in the league as (11*18 + 1*1) / 12 = 16.6. However, that player’s ADP, if we calculated it from this single mock draft, will be whatever position that lone team draws in the first round. In other words, no lower than 12 and as high as 1.
In fact, as long as the proportion of the mock-drafting population that values a certain controversial player at his peak is greater than or equal to a fairly meager 1 / 12 = 0.083 = 8.3% (in a 12-team league), the odds suggest that even aggregating multiple mock drafts won’t help. That’s enough for each mock draft, on average, to have a single team valuing that player at pick one and one team is all it takes. If we assume that team is randomly assigned a draft spot in the first round, the average place the player is taken over multiple mock drafts will be right in the middle, at 6.5. That’s slightly more than ten places earlier than the average valuation of the teams in our model league.
The magnitude of the effect grows the greater the range of valuations for this one controversial player are. In our model, as long as that 8.3% threshold of high-valuing owners is met, the lower values are irrelevant. The other 11 owners could value him at pick 250 and his ADP would still be around 6.5.
Of course in a real league there are so many factors that muddy the picture, I won’t even attempt to list them. As textbooks occasionally love to state, this is left as an exercise for the reader. While these may blunt the effect, but I argue none of them remove it entirely. There is no mechanism that works through a similar channel but in reverse – a single owner with an unusually high valuation can completely drive ADP, but a single owner with an unusually low valuation can do nothing to affect ADP.
Making things even worse
There’s a final step that compounds the problem illustrated by our simple model even further – community feedback effects. A player who initially has a wide variation in how he is valued crosses some threshold where high-valuation owners start showing up frequently in mock drafts to determine ADP – this causes his ADP to rise above what the actual average value is in the community. The community then incorporates the new ADP into their valuations, raising the number of owners who value him highly. They then show up in the next round of mock drafts and the cycle repeats, continuously pushing this previously-controversial player’s value toward his peak.
What do I do?
This is not to suggest that ADP is useless. Rather, ADP tells us exactly what the name implies: where a player is likely to be drafted (the vagaries of your league notwithstanding). This is a perfectly useful thing to know. The problem arises when we adopt ADP directly as the only part of our player rankings. By doing so, we are implicitly valuing controversial players at, or at least closer, to their peak.
Deciding which player’s ADP are being driven by this effect is a little more art than science. All of the factors that muddy the story our simple model tells make approaching this question with data a dubious prospect. There are simply too many factors that drive a player’s ADP, including things that have nothing to do with them but instead raise the values of players behind them, or lower the values of players ahead of them, moving the player up or down merely by default. I mentioned three players at the start of this article, and both Murray and Moncrief are good possible recent examples. Beckham’s rise, on the other hand, seems to be driven by something more. In order for him to consistently hold the first or second spot in the ADP rankings for seven months straight, as he has done, teams that value him that highly need to very frequently be drawing the first or second pick in mocks. Over seven months’ worth of ADP, with six mocks in each round making for 42 separate drafts, the lowest he has been taken was sixth. For this to happen for so long suggests owners valuing him at that level make up a truly large part of the mock drafting population.
So, how should you approach this problem, in practical terms?
The first step is simply to be aware this may be happening. The second is to be involved in the dynasty community the ADP rankings come from, for example by browsing and posting on the forums here. This will give you some idea of which players are more controversial than others and may thus be exhibiting this effect in their ADP. You can then adjust your own rankings accordingly.
Another option may be to incorporate ADP data from multiple communities. I also recommend finding actual top-to-bottom rankings from individual experts you respect. Perhaps most importantly, do as much as you can to assemble your own rankings. Most of them will be redraft-oriented, but it can also be informative even for a dynasty player to browse player rankings on the large hosting sites such as Yahoo, ESPN, NFL and CBS. Find the players that they near consensus on and the ones who vary a lot.
You can probably come up with your own methods or use some combination of the few I mentioned. Either way, keep our simple model in mind the next time you feel a little eye roll coming on when you see a player shoot up the ADP rankings. Be wary of controversial or “hot” players as their value may be artificially inflated at the moment and while ADP information is highly valuable, don’t rely on it exclusively.
[/am4show]
- The Five Rules of Dynasty Trading - August 30, 2016
- Not So Fast There, Average - February 29, 2016
- Why Don’t Combine Metrics Predict Success? - January 21, 2016
I hope people I’m playing against use Yahoo, CBS, ESPN and NFL rankings. That means I’ve started with a decent advantage from the get go!
Good way of thinking about ADP. In MFL10s I have certainly seen this effect and worked to take advantage of it. Most notably finding players higher on my rankings and knowing that because of the ADP I can usually wait a round or two to get them.
I wouldn’t advocate actually using the rankings from main-stream hosting sites for dynasty, although I think redraft rankings can help inform win-now choices. I was referring solely to identifying volatile players, which may sometimes be reasonable across the redraft-dynasty divide. I’m actually exploring that now for a future article.
By the way, I can be reached on Twitter at @levyjeff. I’ll see about getting that added to my little tag line.
Great article. I think ADP/rankings need to be rethought for situation. Rankings can be different for team need in each league.
A rebuilding team won’t value an aging veteran the way a contending team would. Just having a resource that shows who the most valuable tradable assets would be at any given point would be great.
Rankings for a player’s ceiling, a player’s floor, and a player’s expected performance would be great too.
This is a good point, that ADP is situational and who you value for your team will change depending on where you’re at. That said, I think it’s best to consider ADP as if it were startup values, and then inform trades from there with some room for wiggle.
I have often sought to find a player’s projected ceiling and floor, as Dave mentions above. Here, on DLF, there are 8 writers ranking players in the Offense positions. I sometimes look for High and Low rankings as an indication of “possible” ceiling and floor.
For instance, take Golden Tate, current ranking on DLF is WR 24. His highest rank is 18; his lowest is 39.
3 writers have him at 23, 24 & 25 respectively. If you only had 2 contributors ranking, one at 18, the other at 39, his rank would be 19 or 20. Thankfully we have 8 to balance opinions. Would you think the ranker placing him at 18 might be influenced by mock drafts? I would think not. My initial assumption is he is more analyzing Tate’s position in DET and probably one that believes he will score that well. Perhaps he is thinking Calvin Johnson is set to decline. But wait, the same ranker has CJ at #4. What now?
Then you have the ranking of 39 to consider. That is a huge gap (21 places). He also has CJ at #4.
Throw out the high and the low and the remaining 7 rank him at 22.
Another site has 4 rankers and have Tate at 42 with six rookies above him! The same site reports his ADP (based on only 5 mocks) at 4.09, WR 23. The High of the 5 mocks was 3.05(29); the low was 5.09 (57), which would average 43, but of all positions. Perhaps, these rankers are influenced by mocks. More likely they are influenced by other substances.
One point is the high and low could be ceiling and floor, but the average of all 8 is probably a more reliable ranking of expected performance. The same applies to mock drafts. I would not rely on the ADP without looking at the high and low Draft spot and the number of Mock drafts contributing to the ADP.
The other point is stick to DLF.com.
Great article, mate – and I just have to say I’m thrilled to a fellow MSU alum hired by DLF! I think ADP is a trade/value justifier, and like you said, implicitly driven By human preference – that could be a good thing if you know how you exploit it, but I could see ten #dynastytrades answers to one question differ by “when” it’s asked (hype train, situation. Lying season, tons of reasons), so it’s good to know how to exploit adp, even if we unintentionally inflate the very players we want cheaply!
One of the great features at DLF is the year-round dynasty ADP. Also, go green!
Great article, and one of the reasons whenever possible (if I have access to the mocks themselves that make up the ADP) I simply use median. It still isn’t perfect but at least helps smooth out the outliers a bit.
I know using standard deviations and all that, but in excel I just type Median vs Average
