How Average Draft Position Inflates Controversial Players

Jeffrey Levy Posted On August 7, 2015

OBJ

Editor’s Note: This article is written by a new addition to Team DLF – Jeffrey Levy. We welcome Jeffrey to the team and look forward to his work in the future!

Do you exclusively use average draft position (ADP) data to fill in the gaps in your player rankings? It seems like a natural thing to do. Few of us have either the time or the expertise to properly break down game film on every player in the NFL. We have to get some help somewhere! At least I’m assuming it’s not just me.

At the same time, did you roll your eyes a little when Donte Moncrief showed up with a February 2015 ADP of 35? Latavius Murray going at #59? How about Odell Beckham, Jr. landing in the top overall spot in March? All aboard the hype train, next stop – top of the rankings! Right?

Well, you may be doing your part to perpetuate the problem. Stay with me and I’ll explain.

A simple model of the ADP problem

[am4show have=’g1;’ guest_error=’sub_message’ user_error=’sub_message’ ]

In order to illustrate the problem, we’re going to simplify things. This is a common strategy any time one is modeling a small element of an otherwise complicated system. To use a quote by Neil Gaiman I like to bring up when teaching undergraduate economics, “The more accurate the map, the more it resembles the territory. The most accurate map possible would be the territory and thus would be perfectly accurate and perfectly useless.” So rather than try to parse through an actual, messy draft to understand the problem, we’ll use some reasonable assumptions to make a simple model:

We have a 12-team, two-round draft, with only a single controversial player followed by a bunch of much less controversial ones. There are 11 owners who value the controversial player at the middle of the second round, at pick 18. There is one owner, however, who values that player at the first pick in the first round.

We can find that player’s average value amongst all the owners in the league as (11*18 + 1*1) / 12 = 16.6. However, that player’s ADP, if we calculated it from this single mock draft, will be whatever position that lone team draws in the first round. In other words, no lower than 12 and as high as 1.

In fact, as long as the proportion of the mock-drafting population that values a certain controversial player at his peak is greater than or equal to a fairly meager 1 / 12 = 0.083 = 8.3% (in a 12-team league), the odds suggest that even aggregating multiple mock drafts won’t help. That’s enough for each mock draft, on average, to have a single team valuing that player at pick one and one team is all it takes. If we assume that team is randomly assigned a draft spot in the first round, the average place the player is taken over multiple mock drafts will be right in the middle, at 6.5. That’s slightly more than ten places earlier than the average valuation of the teams in our model league.

The magnitude of the effect grows the greater the range of valuations for this one controversial player are. In our model, as long as that 8.3% threshold of high-valuing owners is met, the lower values are irrelevant. The other 11 owners could value him at pick 250 and his ADP would still be around 6.5.

Of course in a real league there are so many factors that muddy the picture, I won’t even attempt to list them. As textbooks occasionally love to state, this is left as an exercise for the reader. While these may blunt the effect, but I argue none of them remove it entirely. There is no mechanism that works through a similar channel but in reverse – a single owner with an unusually high valuation can completely drive ADP, but a single owner with an unusually low valuation can do nothing to affect ADP.

Making things even worse

There’s a final step that compounds the problem illustrated by our simple model even further – community feedback effects. A player who initially has a wide variation in how he is valued crosses some threshold where high-valuation owners start showing up frequently in mock drafts to determine ADP – this causes his ADP to rise above what the actual average value is in the community. The community then incorporates the new ADP into their valuations, raising the number of owners who value him highly. They then show up in the next round of mock drafts and the cycle repeats, continuously pushing this previously-controversial player’s value toward his peak.

What do I do?

This is not to suggest that ADP is useless. Rather, ADP tells us exactly what the name implies: where a player is likely to be drafted (the vagaries of your league notwithstanding). This is a perfectly useful thing to know. The problem arises when we adopt ADP directly as the only part of our player rankings. By doing so, we are implicitly valuing controversial players at, or at least closer, to their peak.

Deciding which player’s ADP are being driven by this effect is a little more art than science. All of the factors that muddy the story our simple model tells make approaching this question with data a dubious prospect. There are simply too many factors that drive a player’s ADP, including things that have nothing to do with them but instead raise the values of players behind them, or lower the values of players ahead of them, moving the player up or down merely by default. I mentioned three players at the start of this article, and both Murray and Moncrief are good possible recent examples. Beckham’s rise, on the other hand, seems to be driven by something more. In order for him to consistently hold the first or second spot in the ADP rankings for seven months straight, as he has done, teams that value him that highly need to very frequently be drawing the first or second pick in mocks. Over seven months’ worth of ADP, with six mocks in each round making for 42 separate drafts, the lowest he has been taken was sixth. For this to happen for so long suggests owners valuing him at that level make up a truly large part of the mock drafting population.

So, how should you approach this problem, in practical terms?

The first step is simply to be aware this may be happening. The second is to be involved in the dynasty community the ADP rankings come from, for example by browsing and posting on the forums here. This will give you some idea of which players are more controversial than others and may thus be exhibiting this effect in their ADP. You can then adjust your own rankings accordingly.

Another option may be to incorporate ADP data from multiple communities. I also recommend finding actual top-to-bottom rankings from individual experts you respect. Perhaps most importantly, do as much as you can to assemble your own rankings. Most of them will be redraft-oriented, but it can also be informative even for a dynasty player to browse player rankings on the large hosting sites such as Yahoo, ESPN, NFL and CBS. Find the players that they near consensus on and the ones who vary a lot.

You can probably come up with your own methods or use some combination of the few I mentioned. Either way, keep our simple model in mind the next time you feel a little eye roll coming on when you see a player shoot up the ADP rankings. Be wary of controversial or “hot” players as their value may be artificially inflated at the moment and while ADP information is highly valuable, don’t rely on it exclusively.

[/am4show]