One of the interesting things about gathering Big Boards across the country and finding the different ways that evaluators grade the players is that it gives us an ability to take a look at the draft from their perspective. There’s a big stigma against “grading the draft,” that I don’t think makes a lot of sense because we’re so willing to share our opinions on the players and teams who drafted them in every other way.
It seems we can give opinions about individual players and their teams without criticism. but as soon as we summarize it in a letter grade, we’re doing something wrong and have to wait. Instead, it may be better to wait three years to judge it.
But that’s no fun, and we want feedback. We just have to acknowledge we have a high band of uncertainty and give our impressions of the draft.
But how about instead of inserting post-hoc opinions about our favorite team, we take a look at a metric we’ve already laid the groundwork for? Let’s compare a team’s draft capital to what the Big Boards accumulated said.
It’s not perfect, especially in a deep draft, but by assigning players in their rankings an amount of points equal to the trade charts’ equivalent pick value, we can find out what players are considered to be worth. Using the NFL Trade Value Chart (put together by Jimmy Johnson way back when), we can compare the amount of draft capital a team entered the draft with to the “value” of players selected. This is perhaps the most appropriate way to gauge the number of “steals” and “reaches” a team makes and quantifying.
The question is which board to use? We have a consensus board, an evaluator board and a forecaster board from fifty different sources. After looking at the forecaster boards, I saw that they did end up—as we expected—being the most accurate at creating a Top 100 that matched the draft value of the NFL (not by counting the number of players in both top 100s, but by finding the average error for each player. I’ll write about it in more detail later, but for now just know that “predicting” where players go doesn’t tell us if a player is good).
Instead, we’ll look at the evaluator board. The two point systems I’m using are the Jimmy Johnson chart and the Football Perspective Chart based off of Approximate Value. The system is simple—the percentage difference between the amount of draft capital you have and the amount of player capital you acquired determines how well you did. I have also added and subtracted the traded future picks from their relevant teams using the traditional discount rate (50%), even though there’s good evidence that that’s a bad rate.
The scores are normalized so that 100 is average, and every 15-point difference is one standard deviation away from average. The first chart uses the traditional Jimmy Johnson chart:
|New York (J)||97.3|
|New York (G)||78.8|
The second chart uses the Approximate Value calculator from Football Perspective:
|New York (J)||91.5|
|New York (G)||72.7|
Either way, the Vikings come out ahead. Oakland’s draft was stellar, but most people didn’t catch that and Seattle may have gotten better players in UDFA than they did in the draft (this chart doesn’t account for that). If Seattle had drafted zero players with its picks, it’s score would have been 44 and 19, depending on the chart, making their draft shockingly subpar.
This chart doesn’t take into account needs, scheme fit or positional value (both concepts I think are important when evaluating drafts), simply the pre-draft judgment of talent. That might be why, when I asked on Twitter, I got wildly different responses from people when I asked them who they thought had the best draft for value. Most said San Francisco (12th and 10th) and I also got responses saying St. Louis (14th and 15th), Jacksonville (24th and 13th) and the Steelers (23rd and 14th). I would also argue that some of the responses were not driven by the amount of capital the teams entering the draft had, and both the Rams and the 49ers entered the draft in the top five of total draft capital (pick value) in the AV calculation. Because the Jimmy Johnson chart is steeper, the Rams ranked 1st in the Jimmy Johnson chart and the 49ers ranked 16th. Basically both teams had a ton of high value picks, so they should have more high-value players.
For what it’s worth, I trust the AV chart more on both accounts—determining who had the most capital and who got the most return, because it is based on historical data. One final thing we can do is look at it division by division. First, by the Jimmy Johnson TVC:
Then by the AV Chart:
Which means that the improvement teams saw relative to their draft may differ significantly by the fact that their divisional rivals did well or poorly. The Vikings’ success, for example, is going to be tamped down by the success of the rest of the division, while the New York Giants’ failure may be hidden by the weak drafting of its division.
And yes, again it is impossible to truly judge a draft until we’re three years out. But based on what we know, this is the evaluation as it stands right now—driven not by homerism but by the consensus of 50 experts and some interesting tools. If you want letter grades, they are as follows:
|New York (J)||D|
|New York (G)||F|
|Seattle||Please see the teacher after class|
Enjoy your success Vikings fans! I am sorry, Seahawks fans but you’ve had it pretty good for a bit now, anyway.