Grading Mock Drafts 2014

By Michael Gertz
Friday, May 9, 2014

In a couple days, the sports media will begin the inevitably fruitless task of grading the NFL Draft to determine which teams won and lost, all while having worse information on how good the players actually are than the teams had themselves. So instead of grading the draft, let's grade something that we actually do already have the results to: mock drafts.

Before the 1st round of the NFL Draft yesterday, we wrote in our Consensus Mock Draft 2014 article that mock drafts are historically very inaccurate, typically correctly picking only about 5 players in the correct draft position out of 32, with most of those usually coming in the first 5 picks of the draft. It turns out the 2014 mock drafts were almost exactly what we expected. Of our sample of 20 mock drafts from various sources, the average number of players correctly assigned to a draft pick was 4.5 (ranging from 2 to 8) out of 32. The "easy zone" of the first 5 picks was extended slightly to the first 7 picks this year, after which almost every mock draft only picked 1 or no correct locations. It should be noted that real life trades changed which team had 8 of the 32 draft slots, but mock drafts ended up picking those slots just as accurately as those that weren't traded.

Consensus vs Individual Mocks

It's not quite a perfect comparison because consensus picks are free to pick a single player to multiple draft slots or teams, but overall our consensus approach did do slightly better than individual mock drafts. The player that was most commonly mocked to each draft position was actually picked in that position 5 of 32 times, and a player actually went to the team he was most commonly mocked to 6 of 32 times. While these numbers are a slight improvement on standard mock drafts, they are still wildly inaccurate overall and most correct picks still came in the first 7 picks.

The fact is mock drafts are basically useless after the first 10 picks. Randomly shuffling players from a list of top prospects would likely produce just as accurate results. A more useful process than producing a 1st round mock draft would be to speculate a few possible scenarios for the top 5 or 7 picks, and beyond that have separate lists ranking players and listing team positional needs (which is similar to what our pre-draft article did). The truth bodes even more poorly for the bold analysts out there trying to produce 3 or 7 round mock drafts, which beyond the top 10 overall picks are probably over 95% inaccurate.

What a Consensus Mock Tells Us

Mock drafts do actually do a decent job of ranking players, which our consensus mock draft was able to summarize well. 24 players were considered 1st round picks by just about all the mock drafts, and all but WR Marqise Lee were taken in the 1st round. Most of the top 20 ranked players were indeed drafted within a few slots of where they were expected to go. The second half of the 1st round showed a bit more variation, but the order of ranking was still relatively accurate. In fact, simply using our consensus player rankings as a mock draft would have produced an above average 6 correct picks out of 32.

The ability of the mock drafts to correctly identify which positions teams would draft was mixed. Beyond the easy first 7 picks, 17 of the remaining 25 picks had at least half of the mock drafts all picking the same position at that pick. Of the 17, a slightly impressive 5 times that position was indeed the one that ended up being drafted by the team with the pick (regardless of whether they moved a few spots in a trade). However, none of the 5 ended up having the correct consensus player drafted, just position. And 11 of the 32 draft slots ended up having the teams they were associated with draft a position that not a single of the 20 mock drafts projected to them. Therefore the combined effects of accurately ranking players within positions and picking the correct positions simply makes projecting draft picks after the first 10 nearly impossible.