Wisdom of the Crowd: What’s Your Window?

Brian Malone Posted On July 22, 2015

crowd

It is dangerous to make predictions, especially about the future. – Niels Bohr (or someone else)

In theory, dynasty is forever. In reality, we’re not even very good at predicting how a player will perform next season, let alone for the next decade. That’s why many owners approach dynasty from a “window” of years. As you can tell from a quick search of the DLF forums, there’s little consensus on the proper window for evaluating players.

Recently, there was a short, but fruitful thread devoted to this subject. Opinions ranged from two years (or maybe even one year) to 10+ years. Some took a more nuanced approach, applying different windows for different positions or even using an indefinite window with a discount rate.

But the most common answer was the one you’d expect: three years. And the reasons supporting this answer were all pretty similar. First, folks think that we’re not very good at predicting how a player will produce more than three years down the road. Second, winning now is better than winning later. Third, leagues fold and you don’t want to build for four years down the road if your league may not make it past year two.

I don’t have much to say about the second and third reasons. (Check out Eric Burtzlaff’s “Roughing the Kicker” series on dynasty profitability if you’re interested in those.) But I had the data to shed some light the first question, so that’s what I did.

I combined the resources of the DLF forum[1] and did a little manual data entry to compile spreadsheet including every player drafted in the first 20 rounds of dynasty startups from 2007 to 2012; his average draft position (ADP) each year he was drafted; and his fantasy production in every year he averaged more than three points/game in PPR scoring.

Using this data, I calculated each player’s seasonal production value (V) for every year through 2014. V = games played*(points/game – baseline), where baseline scoring is 11 points/game for wide receivers, 10.5 points/game for running backs, and 9.5 points/game for tight ends. Quarterbacks are a whole different issue that I may address in a later article. You can find some preliminary thoughts here.

I then plotted the Y1, Y2, and Y3 values, as well as the player’s three-year average for each draft pick in each startup from 2007 to 2012. Here are the results:

Three-Year Window

[1] In particular, Telperion’s Football Data Warehouse and cmizelle’s historical ADP were essential to this project.

[am4show have=’g1;’ guest_error=’sub_message’ user_error=’sub_message’ ]

wisdom1

And here’s a closer view of the first 80 picks:

wisdom2

As you can see, the typical pick produces more in year 1, slightly less in year 2, and less still in year 3. ADP also has less predictive power each season into the future. Note that ADP is a good predictor of a player’s three-year average, but Y1 production is typically greater than the three-year average.

These data suggests that even the three-year window may be too optimistic. Or, at least, owners should discount the second and third seasons of the window and focus on a player’s Y1 production.

Of course, these data are based on the full startup draft, including owners who are ignoring the future. What would it look like if we focused on our ability to predict younger players? It would look like this:

wisdom3

And focusing on the first 80 picks …

wisdom4

The best fit curves are closer together, indicating youngsters score almost as much in years two and three as they do in year one. But the fit of each curve remains about the same as it was using the full set of players, meaning ADP remains a better predictor of first-year performance than second- or third-year performance, even among younger players. As with the full set of players, ADP is a good predictor of a player’s three-year average, and with young players that three-year average is closer to first-year performance.

All in all, the data suggests the three-year window isn’t a bad approach (at least for non-quarterbacks). We’re pretty good at predicting a player’s three-year average, even if ADP is a better predictor of first-year performance than third-year performance.

One-Year* Window

Even though the three-year window has some support, I focus on an even shorter time frame: one year. I started using this approach after I calculated my team turnover rate this off-season. Through waivers, trades and drafting, I replace more than 60% of the players on my dynasty teams each season. In other words, of the 25 players on my typical roster in June 2015, only 1 or 2 will be on my roster in June 2018. Because of this turnover, trying to project players’ production for the next three years is wasted effort for me. But I’m also not the type to mortgage my future chasing the title each season.

Thus, the one-year* window. When valuing a player, I try to predict two factors: (1) his production value this season and (2) his trade value the following off-season. Luckily, we’re pretty good at predicting both of these. The charts above show that dynasty drafters are pretty good at predicting year-one production (and re-draft players are likely better, which is why redraft ADP is one of my favorite dynasty tools). Plus, startup ADP data shows dynasty drafters are really good at predicting next year’s dynasty ADP.

Here’s a scatter plot of players’ year 1 and year 2 ADPs for players drafted in 2007 to 2012 startups. (Actually, I used ln(ADP), which accounts for the fact that the value difference between 1.01 and 1.10 is greater than the value difference between 19.01 and 19.10, but the chart of raw ADP looks fairly similar.)

wisdom5

And here’s the same chart using players drafted 2010 to 2014 startups. This is a nice side benefit of the one-year* window: because we don’t have to look at players over a multi-year window, we can use more recent data.

wisdom6

To give you an idea of how this translates to raw ADP, here’s a conversion table based on the 2010-14 data.

wisdom7

This all shows dynasty ADP is quite stable from one year to the next. So if you can predict a player’s immediate production and you can factor in obvious things like age and contract situation, you’re well on your way to predicting next season’s trade value.

Conclusion

There’s nothing wrong with using a three-year window for player evaluation, but predicting that third year gets pretty hazy. If you’re a roster churner (turning over 50% or more of your roster each season), I recommend using the one-year* window, which is based solely on this year’s production and next year’s trade value.

[/am4show]