NFL draft over-analysis

Over the years, the NFL draft has been one of my favorite topics to blog about. While most of my posts have focused on the Eagles, I've done lots of overall evaluations. And by the numbers, my evaluations have been as good as most NFL teams, and in some cases better. The evidence is right here in this blog, check it out! The Eagles would have Earl Thomas and Jimmy Smith starting for them instead of Brandon Graham and Danny Watkins. The 2012 Eagles draft looked like they were drafting from my board - Cox, Kendricks, Curry, and Boykin were all guys that I was high on (I liked Alshon Jeffrey a lot too, but c'est la vie.) Also, while everyone in hindsight seems to have identified JaMarcus Russel as a flop, you can find a before-the-fact post on this blog saying I would not draft him with the first pick.

I am wrong a lot as well. But when you consider that less than half of first-round draft picks are long-term starters for their team, the bar is not very high.

Here is what an average NFL team does to evaluate players:

• Send multiple scouts live to college games. These scouts are looking to identify players that are scheme fits for the team. They talk to coaches to get a feel for the type of players that are coming out that year.
• Identify through these live evaluations about 200-300 players that will be "targets" for their team.
• Watch every play of every college game (up to 4 years) for those hundreds of players. (Other players who stand out on film may be added to the list of hundreds during this process.)
• Attend the Senior Bowl, where the coaches can work hands-on with the players throughout the week of practice.
• Attend the Scouting Combine, where the coaches and scouts can interview players (15 minutes each) as well as watch them execute various drills.
• Attend Pro Days at each college, again watching the players execute drills.
• Bring in up to 30 players for an individual workout at the team facility (usually a full day) where the team will put them through various physical and mental tests, as well as take them out to dinner.
• Interview college coaches, trainers, equipment guys, high school coaches, ex-girlfriends, family, etc.

Here's what I do to evaluate players:

• Watch a few college games during the year.
• Compare college productivity against physical measurables (height, weight, strength, speed, etc.)
• Incorporate some thoughts from outside sources I respect (Mayock, Cosell)

Given the huge discrepancy in resources and time spent, how could I possibly even compare to NFL teams' evaluations? A dozen scouts, employed full time, working ridiculous hours...coaches meeting players face-to-face...background interviews...reviewing every snap...vs. me, browsing the internet at work?

While I'd like to believe otherwise, the truth is that I am not some talent evaluation savant. There were lots of people who thought Earl Thomas was better than Brandon Graham, it was common wisdom at the time. When NFL teams make goofy, out-of-the-box picks that fly in the face of common wisdom, it does sometimes work out (ie: Bruce Irvin.) But most of the time, you end up with AJ Jenkins and Brian Quick instead of Alshon Jeffrey and Reuben Randle.

For starters, luck is a huge factor. Predicting future performance is not an exact science. You can make a list of X number of ideal qualities for a position, and even a player who has all X qualities may fail. So this levels the playing field to an extent, as even drafting by dart board would have some measure of success (in Detroit's case, more success.) I am willing to accept that there is a chance that I've simply been lucky over the years. Even if you consider me nothing more than a proxy for the wisdom of the crowds, it's possible we've all just been lucky.

But I believe there is more at stake, and NFL teams should pay attention. There is a definite Blink effect when evaluating talent. Most NFL scouts who have been watching players for years form first opinions that are as accurate, or more accurate, than months of over-analysis. A guy like Blake Bortles passes the 'eye test', ie: he looks like an NFL QB. He played some of his best football against the best competition. He has an excellent arm. Compare him to other players of similar build and college production - Big Ben comes to mind - and you can quickly see he has an excellent chance to be a quality NFL starter, with high upside. How much more analysis is required?

There is some evidence that watching slow-motion replays only reinforces pre-existing biases when attempting to derive intent. It may sound like a stretch, but I theorize that a similar effect happens when over-analyzing these draftees. Whatever your first impression of a player is, you will find select evidence confirming your impression if you dig deep enough and look at enough data. And the more you look, the more convinced you become of your thoroughness, and the more confident you become of your opinion - which was nothing more than a first impression confirmed by selection bias!

I'm not sure how else to explain the NFL's spectacular lack of success. Or even more strangely, the widely varied opinions of 32 NFL teams. If additional analysis got you closer to the 'truth', then you would expect teams would gravitate into a tight grouping of opinion as they analyzed more.

NFL teams have almost limitless resources, and scouts are often passionate about their profession. Everyone is afraid of being out-worked by another team, so over-analysis is inevitable. But maybe teams can create checks and balances to minimize the damage.