2014 NFL Draft: “Consensus” Top 200 Big Board From 34 Experts—”Forecasters” vs. “Evaluators”

Arif Hasan   |  

May 2, 2014

• Share on Facebook
• Share on Twitter
• Share on Reddit
• Share on WhatsApp
• Copy Link

  Link Copied

With the NFL Draft finally approaching, getting a consensus on the talents of the prospects may give us the best possible understanding of who a team selects or whether or not they did a good job (as far as we can tell). Unfortunately, not every board is constructed with the same goals in mind or with the same amount of information available to them.

NFL Draft Tracker, a great website for getting detailed reports on prospects as well as a general understanding of the theories driving the draft, made a good point not too long ago: consensus boards don’t make a lot of sense if we don’t discriminate between those who are purely evaluating player talent and those who are attempting to reflect the consensus of the league.

That’s fine—these draft resources generally answer two questions: 1) “Who’s Good?” and 2) “Who Are We Going to Pick?”

The good news is that we can easily do that. For the most part, while insiders like Rob Rang and Daniel Jeremiah do an excellent job pointing out how they differ from mainstream views, the major networks reflect often the consensus of the league, and their draft rankings do not change much from each other. At the very least, their low variance suggest that their rankings are influenced by what they hear around the league, if not driven by it.

I was initially skeptical of this approach, but when I found unusual clusters of rankings by the mainstream draft services (especially those at the same networks) but not among the third-party services, I was fairly convinced they were either stealing evaluations or working from the same inside information—it should be extremely rare to find clusters when you’re ranking around 70 or 100 (flat talent distribution means small differences of opinion cause wide ranking divergence), but among the more established services, it was not.

For what it’s worth, I don’t think they’re stealing draft evaluations from each other, I just think that when they start hearing the same things from the same people over and over again, it affects the process. The fact that many of the NFL-influenced draft boards showed much more week-to-week volatility than the film-dependent draft boards further solidifies this approach.

As a result, we can construct two consensus boards. I ended up including 34 total boards (listed at the bottom of this post), with 22 of the boards categorized as coming from “evaluators” and the other 12 coming from people categorized as “projectors.”

Some of folks (notably Bob McGinn of the Milawaukee Journal-Sentinel) have not released a Top 100 yet, but we’ll do our best regardless.

The distinction wasn’t necessarily arbitrary, but I did make some calls. A simple divergence test on the boards actually produced stark differences between the two groups—with all of the boards with similar ranks coming from those who have insider access and the others coming from a mix of those with high-level access who haven’t incorporated outside input and somewhat amateur evaluation boards (here, “amateur” is used not as a reference to evaluation talent, but recognition).

There are drawbacks and benefits to either approach. Getting information from NFL generally means getting information from a source that has more access to knowledge of injuries (they get to inspect all the top prospects at the combine and request medical information from teams, as well as do their own medical work with top doctors), off-field concerns (as Daniel Jeremiah points out, he left scouting because it has become 70 percent character background work), coaches film (invaluable for scouting safeties and quarterbacks) and years of high-pressure evaluation experience.

On the other hand, relying on information from those who have an enormous incentive to lie and those who are often stuck too far into tradition may cause serious issues in terms of accuracy and evaluation.

For these boards, I’ve used different positional names to get a clearer picture of a player’s most often projected role, so that we can compare like to like. For example, I think a rush linebacker and an edge-setting defensive end are more alike than different, and that to categorize one as an “LB” and the other as a “DE” causes too much confusion—Anthony Barr is not similar to C.J. Mosley, even though both are considered “LBs,” while Jadeveon Clowney and Stephon Tuitt hardly play the same position, despite both being “DEs.”

I’m hardly the first to come up with this concept; it’s been discussed on Twitter for quite some time, and the first time I saw it implemented in a ranking was from Josh Norris at Rotoworld. Positions below:

First, let’s look at how the Top 200 shakes out if we only look at what the mainstream sources forecast, using their access to NFL teams and what the NFL tells them (as well as their own evaluations).

After that, you can take a look at how the evaluators graded the talent of the players they saw, and there are some pretty stark differences from the table above.

In a 200-person big board, it’s difficult to really get the differences broken down at a glance, although some stand out: Teddy Bridgewater fares better with the evaluators than the projectionists. Tom Savage barely cracks the top 200 for the evaluators, while the projectionists were more optimistic. It should be noted that the forecasters had a clear split with Savage, with nearly half of them projecting a top 64 (but not top 40) appearance, and some others ranking him well outside the top 100.

At first, it might seem intuitive to simply look at the differences in rankings to see who disagrees with who the most, but it’s not particularly interesting if a player ranked 500th by one set of people gets ranked 400th by the other. The disagreements are magnified at the top—seeing a Top Five player in one board going in the second round of the other is more dramatic than 300 to 400. Talent distribution at the bottom is a little flatter, so smaller things can sway decisions than at the top, where it takes a generally big difference of opinion to matter.

So, I’ve largely corrected for that phenomenon with some dirty math and found the 42 biggest sticking points in the Top 300 of both boards, ranked in order of largest disagreement relative to their draft position:

The top disagreements by the forecasting crowd of the evaluators are all of players that are technicians with athletic or size-related deficiencies. Campanaro is an excellent and profoundly intelligent receiver with a good combine, but came in at 5’9″. Yawin Smallwood is instinctive and quick, but weighs less than 240 pounds. We all have had enough of the Teddy Bridgewater talk to be hardly surprised at the differences here, and Cody Hoffman’s prototypical size seems to impress the NFL (by proxy) but not the third-party draft services.

Just barely missing the cut were Jake Matthews and Greg Robinson, surprisingly enough, but that is largely because the differences between #2 and #5 are fairly big in this formula, and each group had one of the two tackles ranked at those spots (Matthews at #2 for the evaluators with Robinson at #5, and the reverse for the forecasters).

The other general category of differences include small-schoolers, where the evaluators were far more willing to bet on players from lesser-known foobtall factories than the forecasters.

The two groups are not constantly warring in their evals, though. They agreed far more often than disagreed (if that wasn’t the case, this would all be useless and no one knows anything. Even though we like to say that, the clear pick-by-pick performance curve tells us we know a few things), and the biggest agreements happened all over the board.

The most impressive even evaluations are usually through chance—you’re always going to end up with a number of players that happen to be in similar spots on the boards, just because over 1120 players were evaluated, and the amount of skill that the players possess is not random. Still, the amount of times they agreed by just a few positions is impressive. The biggest 44 agreements (again, through the quick math formula mentioned above) are below:

It’s hardly surprising that Jadeveon Clowney is the consensus #1 on every group of boards, and that Aaron Donald is a top ten pick to both groups, but it is sort of interesting that Dee Ford is just barely in the top 40, while Jimmie Ward and Davante Adams are just out. There’s a cluster around pick 100, as a bunch of players that are supposed to be “top 100” talents could potentially get shoehorned in by both groups, and the players at the top seem to be barely draftable.

Here, the patterns aren’t clear. There’s a decent mix of small-school and big-school players, as well as a good mix of positions. Certainly, none of the top qurterbacks made the list, but McCarron and Mettenberger solidified their 96th and 72nd overall rankings.

Some players are easy to evaluate, and it just so happens that you’ll find a lot of agreements among the different Boards—no need to incoprorate an overriding reason or pattern.

That said, there were definitely a lot of players who caused a lot of consternation among the 34 different boards. By comparing the squared difference in their rankings to the overall likelihood of disagreement for a generalized pick spot, you can see which players are the most polarizing. It was important not to take the worst ranking and the best ranking and get the difference (though displayed here), simply because one off grade will throw the whole thing out of whack when really there is general consensus on a player.

Some are obvious, but some may catch you by surprise (again, the list below looked at all 34 boards, not just an average of the two consensus boards, so it’s not the same as above):

Players well known for causing controversy didn’t necessarily score high on the list. As you’ll see in a moment, one of the most controversial players in the draft is Jadeveon Clowney, but he is the least polarizing player in the rankings. Johnny Manziel barely made the cutoff, and was about as polarizing as “safe” pick Khalil Mack—who got every rank between 10 and 2 from the 34 boards (again, because of the difference of value between #2 and #10, volatility in the rankings mean more here than they do for a player ranked 14th overall bouncing between first and sixty-second).

There are a few basic things that get a player on this list: excellent play, but poor measureables (Lamarcus Joyner), injury (Dominique Easley), an uneven career (Will Sutton), and insane bullshit (Teddy Bridgewater).

For me, a big surprise was not seeing Telvin Smith, whose on-field play has been electric, but is well outside the size demands traditionally fit for his position. He just missed the cut off here, but the general thought on him seems more settled than I thought (which is still not very settled).

While the most polarizing players list doesn’t look much like the “biggest disagreements” list, the least polarizing players list does look like the “biggest agreements” list, which makes a lot of sense. The difference between “worst” and “best” rank doesn’t mean much here, largely because they were outliers instead of a good demonstration of the volatility.

Don’t worry about the “.5” rankings—they were part of a system to assign players a rank to be averaged when a particular big board in question was too small to include them (I did not take any boards with fewer than 50 players, and even then only one). All that means is that at least one big board chose not to include them, and they therefore were penalized (the size of the penalty was directly proportional to the size of the big board).

Despite the fact that Greg Robinson was a point of contention between the two different sets of boards, there was still significant agreement that he was a top pick, and he generally floated in between two and six—unlike Khalil Mack, who was all over the top ten.

I didn’t expect to see Kelvin Benjamin, who’s movement in the first round halfway through the offseason was criticized by a number of people, given the fact that he has inconsistent hands, is a mediocre route-runner and has shown limited versatility despite incredible production at Florida State. It seems that even in a rich receiver class, people of all flavors were willing to forgive his spotty development for his size and speed.

Finally, you can look at the full consensus of picks. There are two separate Top 200 charts that are consensus, because we can either treat each category separately (the average of the “forecasters” and “evaluators”) or treat each board separately (averaging all 34 draft boards). Below is that first board, the average of the two consensus boards:

That board corrects for the fact that there were almost twice as many “evaluator” boards as “forecaster” boards, by weighting the two groups equally. On the other hand, you could avoid that and simply average out every board (functionally weighting the evaluators 67% and the forecasters 33%):

I used a lot of Big Boards for this work, naturally. The online resources used were Eric Galko’s Optimum Scouting, Dan Kadar and Matthew Fairburn’s SB Nation Mocking the Draft site, the National Football Post, Rob Rang’s Big Board at CBS, the eDraft family of boards (Vincent Frank, Luke Inman, Josh Collacchi, Matt Johnson, Derrik Klassen, Blake Meek, Vance Meek, Blake Arnold, Ryan Ratty, Patrick Conn and Richard Gatenby), Ben Natan’s board at With the First Pick, Andrew Parson’s board at Draft Mecca, Drafttek’s Big Board, James Christensen’s board at NEPatriotsDraft, Mel Kiper’s Big Board at ESPN Insider, Todd McShay’s Big Board with Scouts, Inc. at ESPN Insider, Jon Dove’s Big Board at Draft Breakdown, Tony Pauline’s board at Draft Insider, Ryan McCrystal’s Big Board at Draft Ace, Josh Norris’ Big Board at Rotoworld, Charlie Campbell’s Big Board at Walter Football, Path to the Draft’s Daniel Jeremiah’s Big Board at NFL.com, Gil Brandt’s Big Board at NFL.com, and the NFL.com official rankings (run by Nolan Nawrocki—they match his published big boards 1-for-1).

I also purchased draft guides from Bryan Perez at First Round Grade (purchase here for $4.99),  Dan Shonka at Ourlads (purchase here for $32.00), Mike Loyko at NEPatriotsDraft (purchase here for $16.00), Kyle Crabbs at NDT Scouting (purchase here for $10.00) and Dane Brugler at SportsXchange and CBS (purchase here for $4.99 – but you can find his top 100 here with a short blurb for each of the 100 here).

I also purchased two draft guides (Rookie Scouting Portfolio from Matt Waldman and Chris Trapasso’s Guide—email him for it at [email protected]), but they do not have an overall Big Board I could include (Waldman only sells work done for offensive skill players because it has a dynasty fantasy football focus).

If you’re interested in going against the grain, the most divergent ratings were from NDT Scouting’s report, the National Football Post and Ben Natan in that order. If you’d rather find a few people who better reflect the consensus of peers, Daniel Jeremiah ranks first, followed by Mike Loyko and Tony Pauline.

My advice: just print off the last table as a quick cheat sheet when you head into whatever draft party you’re hanging out at. Printable copy here.