IDP Prediction Megabattle 2017: The Aftermath
Before the 2017 season, I was thinking quite a lot about predictions. This included how to create a model and try and build an image of what will happen, what we can predict (and what we can’t), and how we can measure our accuracy.
The last point is super interesting to me. How do we really measure how good we are? Rankings? Stats? Standard deviation? They’ve all got strengths and weaknesses, so I tried to come up with something that would be both measurable and visceral – something that would give our predictions a score but also just show how right or wrong predictions are in an easy-to-grasp manner.
The method I ended up going with was simple. I wrote up 100 statements about the 2017 season and simply asked a selection of analysts to predict whether they’d be true or false. The statements were calibrated to try and make them all roughly a 50/50 chance.
This is an account of how accurate we were. It’s also really useful to see how things we thought were obvious turned out to be untrue. And vice versa.
Just over half the analysts agreed the Cardinals would not have an effective lineman. Calais Campbell was exceptional for years but with him gone no one really stands out on that unit. Chandler Jones led the league in sacks and Haason Reddick had an extremely difficult rookie season. He played two positions and honestly didn’t look good in either. 78% of respondents thought he would play a lot but he failed to manage the target.
More than half the respondents saw Vic Beasley wouldn’t have a good season. Frankly, I’m surprised so many people thought he would hit that target. No one here thought Duke Riley would come in and take a starting job, although it was a widespread belief at the time. Unsurprisingly no defensive back managed five picks. That’s a rare number. IDP players routinely overestimate interception numbers.
Just over a fifth of respondent s thought the Ravens would have a top pass rusher. Terrell Suggs turned back time to hit 11 in the end. C.J. Mosley was a bit of a gamble in 2017. In 2016 he had an awful season (with Zach Orr piling up numbers) and we didn’t quite know what to expect. Eric Weddle was a popular pick after a very good 2016 but he shared stats with Tony Jefferson in 2017 and failed to hit the (fairly low) target.
Reggie Ragland was traded away by the Bills just before the season and failed to play a single snap for them. Woe to those respondents who thought he’d be a full-time starter in Buffalo.
[am4show have=’g1;’ guest_error=’sub_message’ user_error=’sub_message’ ]
Micah Hyde played deep more than many people expected him to in 2017. But he still scraped in with 82 total tackles. Shaq Lawson was a big disappointment and ended up only recording four sacks for the season. I still like him but it was a rough year.
As two-thirds of the entrants predicted, no Panthers lineman played big numbers. The highest was Kawann Short with just over 700. Luke Kuechly’s health was a major issue before last season. He shrugged off all concerns to play 920 snaps in the end. Shaq Thompson again teased us but didn’t quite deliver. He managed 57 total tackles, to come in just short.
Almost everyone was right in that Joey Bosa built on an excellent rookie season to become one of the top pass rushers in the NFL. Unfortunately, most of us were wrong in predicting Jatavis Brown would play a heavy load. Gus Bradley did not seem to want to give him a large snap count at all. I was not in the Melvin Ingram camp at all. He proved me (and others wrong) and managed an excellent season with double-digit sacks.
Akiem Hicks is a monster. He’s sorely undervalued as an IDP. Nick Kwiatkowski, however, is the opposite. For someone who’s not really produced much, everyone seems to think he’s a very talented player. Only 22% of respondents here fell into his trap.
Vontaze Burfict ended up missing a few games. As always. He’s a really good player but he’s not reliable. Carl Lawson actually bucked the Bengals’ normal trend of sitting rookies but managed just 480 snaps in the end.
Around half of the entrants saw that Jabrill Peppers would not be the great option he could have been. Myles Garrett missed a bunch of games but was still the Browns’ top pass rusher. Calvin Pryor was not a major player for Cleveland either. This is one of those things that seems obvious now but was a major discussion in the summer.
In Dallas, Jaylon Smith did end up playing 578 snaps but it seemed that he wasn’t 100% healthy and needed more time to recover. Jourdan Lewis ended up playing significant time at corner and logged 11 coverage plays, No-one’s talking about him but he proved an excellent draft pick. None of the respondents here thought Jeff Heath would last as a starter but he proved us all wrong with 15 starts.
Only a third of respondents had faith in Ray or Barrett to be really effective opposite Von Miller. They were wrong. Neither even managed to play more than 665 snaps so double digits was always a big ask. Aqib Talib was a long way from his PD target too with just seven for the season. And rounding out the Broncos, Jared Crick was hurt and missed the season, making almost all of us wrong about his playing volume.
Somehow Ziggy Ansah did manage double-digit sacks although nine of his 12 sacks came in just three games they all count. The majority of us thought Jarrad Davis would be a high-volume workhorse for the team. He did play 828 but played second fiddle to Tahir Whitehead towards the end of the season. Miles Killebrew managed just 25 solos tackles in the end and was used sparingly. Some analysts love his tools but for me, he’s far too incomplete to play a full-time role.
Most of the respondents thought the Packers would continue to employ a rotation approach at inside LB. Blake Martinez stopped that by seizing control of the job. Clay Matthews did not manage double-digit sacks in the end but ended up with an impressive eight and a half. Josh Jones played 735 snaps in a highly flexible role which only around half of us saw coming.
89% of us were sadly wrong about J.J. Watt. I sincerely hope his injury issues are over and we get to see a bit more of him at an elite level.
Brian Cushing had suspension issues and managed to play just 165 snaps. Andre Hal managed 47 solos and Marcus Gilchrist just failed to broach 50 solos with 47 and 45 respectively. Only 22% of respondents thought either of them would do it.
Most of us saw Clayton Geathers starting for most of the season. He ended up on IR for most of it and was only used sparingly with Matthias Fairley taking the job. Most of the respondents thought that a lineman would manage five sacks but it didn’t happen. Only Hassan Ridgeway (three), Jonathan Hankins and Henry Anderson (two each) managed more than one. Sadly, Sean Spence was cut in preseason. I still haven’t really recovered from this one.
The Jags (as predicted by about half of the respondents) were a top defense and finished way better than top half in points allowed. Myles Jack was a full-time player (albeit an inefficient one) and Dante Fowler confounded two-thirds of entrants to record eight sacks as part of the most fearsome pass-rush in the league.
I meant to write a much higher total here for Marcus Peters but wrote a typo. In the end, he managed just 145 cover plays which were way short of his previous two seasons. Justin Houston disappointed after a hot start to the season and fell slightly short, with nine and a half sacks to confound 78% of respondents. What a shame.
Tanoh Kpassagnon was one of the rawest defensive prospects in the draft last season. He only ended up playing 159 snaps. With Chris Jones managing 698, the two of them together didn’t quite equal the volume of one starting spot.
These questions were asked before Raekwon McMillan suffered a season-ending injury, but most respondents saw Lawrence Timmons being featured more anyway. As always, the Dolphins gave Ndamukong Suh a simply huge volume (881 snaps). They need to try and justify his salary somehow.
Linval Joseph did secure a Pro Bowl spot – which only a third of us predicted. But most of us also thought Danielle Hunter would rack up 11 sacks. We were way wrong there but it was accepted wisdom at the time. And, of course, no Vikings rookie played 500 snaps. Mike Zimmer hates relying on rookies.
Elandon Roberts did manage to hit the solos target in the end. In fact, he managed a creditable 43. Trey Flowers was not heavily backed by the entrants here but did establish himself as the clear top pass rusher on the team. Only a third of us thought Malcolm Butler would continue as an elite corner. And we were right, although none of us saw the Super Bowl benching coming.
Sheldon Rankins was a hot name for 2017 after his five-sack rookie season. He fell flat and disappointed 78% of us here. Four-fifths of us doubted A.J. Klein’s ability to start all year long. We were right. He managed just 664 in a pretty awful season. And defensive rookie of the year Marshon Lattimore easily hit his target. In fact only five players in the league managed more than his 23.
Virtually none of us had faith in the Giants sticking with a single LB. We were correct and the team chopped and changed all season long. Damon Harrison obliterated the tackle target set. He managed 76. Only three DTs have had more in the last five seasons and one of them was him. Landon Collins disappointed most of us by having a very good but not elite season. Volatility from season to season for safeties is higher than we tend to think it is.
Sheldon Richardson didn’t get close to being a Jet all season. He didn’t even make it to September. Jamal Adams scraped in over his target with 63 solos as most of the respondents thought he would. Darron Lee abjectly failed to record a good PFF grade. He finished slightly under 35 in fact. He’s not a good player.
About half the respondents thought Khalil Mack would manage 14 sacks. He’s a really good player but those really high numbers just seem to evade him. He managed ten and a half.
T.J. Carrie led the Raiders with nine PDs. And was promptly released by his team. No-one managed double digits let alone three of them. Almost all of us doubted the Raiders would produce a top-end LB. Signing Navorro Bowman helped but he didn’t crack the top ten.
Not only three, but four Eagles linemen managed five sacks as they rode their D line to a first ever Super Bowl. I like to think we saw it coming a little. Mychal Kendricks was needed by the team as Jordan Hicks was injured. He’s probably earned a nice new contract somewhere. But even though the team won the title, no DB was honoured individually. Allowing 500 yards passing in the title game can probably explain why.
Ryan Shazier failed to hit 13 starts because of a horrific injury. Sadly most people here saw an injury coming. We hope he recovers fully. James Harrison barely played for the Steelers and was released bizarrely. Sean Davis did manage to hit his target. He was especially effective with Shazier injured.
Aaron Donald was naturally an All Pro. He’s probably the best defensive player in the league. Almost half of us thought that one of the Rams LBs would again manage 1,000 snaps. It’s really rare that happens. LBs are just too injury prone for it to be common.
Robert Quinn rolled back time to reach eight and a half sacks. I was one of the ones who thought he wouldn’t.
Bobby Wagner shockingly did NOT lead the league in assists. The Seahawks stat crew really seemed to rein in their assist-happy nature in 2017. Earl Thomas only managed eight coverage plays in the end. Not least because oppositions bullied Shaq Griffin most of the season. Frank Clark disappointed most of the entrants here by coming in as the number two pass rusher. His efficient 2016 was a tough act to follow.
Only around 20% of us thought Reuben Foster would outsnap Bowman. Given Bowman’s release it happened even with Foster’s injury issues. Eric Reid failed to record high tackle numbers. The emergence of Jaquiski Tartt (along with injury) limited him. DeForest Buckner managed just three sacks. Given he played a huge 865 snaps he should probably have managed more.
Lavonte David just missed out LB1 status. He was injured for some games. Again it’s rare for LBs to not be hurt at some time most seasons. Noah Spence disappointed almost all of us by missing most of the season. Given how bad he was, I doubt he would have hit five sacks even if healthy. Vernon Hargreaves somehow did manage a decent PFF score. I don’t really know how, as every time I watched him he was pretty bad.
Logan Ryan just missed out here with 50 solos. Almost all of us were correct in predicting that. On the other hand, most of us did not believe in Kevin Byard, who racked up well over a thousand snaps. I always complain the Titans are boring but actually three separate players managed six or more sacks. Quick – can you name them?
Jonathan Allen was hurt early on but would never have come close to 45 tackles. He was managing a tackle on about 6% of his plays when healthy. To reach 45, he’d need to play 750 snaps. Only one Washington lineman has played that much in the last five seasons (Chris Baker managed 782 in 2016). Su’a Cravens would have been lucky to get a Pro Bowl nod given he didn’t put on a uniform all season. The two corners hit their target by dint of Breeland managing 19 PDs.
Carl Lawson scraped in to hit the rookie sack target with exactly eight of them. He did fantastically well with limited snaps. Just over half of us thought a player would manage eight interceptions and they were right as Kevin Byard sneaked in on exactly that number.
A high proportion of respondents were convinced a player would have 120 or more solos. That number has actually only been hit three times in the last five seasons and no one managed it this year. In fact no player managed over 100 solos in 2017.
The last question was about rookie passes defended. In the end a whopping EIGHT separate rookies hit the benchmark. This corner class was fantastic. It could be historically good.
So the question remains – who was the most accurate? Here’s the results:
Two people (the original hosts of the Read And React Podcast tied for first with 56 correct answers. But the majestic EC managed to win it on the tiebreaker. Congratulations, EC!
I came in third which feels OK to me, but poor Adam “StickyZ” Tzikas should be ashamed of himself with just 47 correct.
What stands out to me is how tightly we were all bunched together. Only nine answers separated top to bottom. What that says to me is that our ability to predict is actually worse than we think it is. It’s certainly possible that the questions themselves altered the rate of accuracy (maybe they were “hard” questions and everyone did really well) but Occam’s Razor says that it’s just really hard to be able to predict things here. I recently went through all my individual player projections for 2017 and thought that I was pretty accurate overall, so these two findings don’t mesh that well.
The whole exercise was a lot of fun to do and I’ll definitely be doing it again for 2018 (bigger and better of course). I’m sure a bigger sample size will tell us more about how good we are at seeing the future.
Thanks for reading everyone.
[/am4show]
- Ten IDP Fantasy Football Stats You Need to Know after Week 16 - December 29, 2023
- Ten IDP Fantasy Football Stats You Need to Know after Week 15 - December 22, 2023
- Ten IDP Fantasy Football Stats You Need to Know after Week 14 - December 14, 2023
