Eye test vs Analytics (1 Viewer)

[Zyphros]

When I evaluate prospects the very first step is an eye test. As a coach of a different sport I'd like to think that I know what I'm looking at a bit in terms of body movement, fluidity, footwork, that sort of thing. I use this as the base of my evaluation. Then when it comes to draft time I add in what others have brought to my attention, numbers and analytics. At this point the only real thing I might know is breakout age and overall production. But none of that really plays into how I rank them as talents.

The last 2 year's I feel like that made my rankings worse, adding in the analytics part. But it did help me in the later portion of my rankings rather than the top end. I think the obvious answer is that there needs to be some combination of both, but it's a long standing debate in the fantasy world. I rely more on my eye's which lead me to Higgins as my top WR this past year (until I added analytics), and good showings from my other top players in Shenault, Reagor and Lamb (not so much in Edwards yet). Preston Williams as my top talented WR in 2019, AJ Brown as my "real" #1, because I couldn't put Preston there for obvious reasons. 

I totally get why it's very subjective and doesn't always lead to the best results though. I'd say most people use some combination of both.

@ZWK has a great analytics guided thread for player numbers through the season's which I glance at and hope to see my players. My thread on rankings is mostly just my process of how I move people around my board, along with the subjective reasons I like player Z of player X. Who pops on screen and who doesn't is what it comes down to basically. 

I just want to get the boards perspective on which you rely on more for your own rankings, how you prefer to see the development of an evaluation, or which you think is better. Personally I think analytics is the better process. However, using the eye test has worked well for me and maybe it's more selective that people need to know what to look for. Each side has misses but I am curious what people prefer to use/see others use. 

Let's keep the arguing to a 0 and discuss. 

 

[Boston]

Great post and I definitely think like you...a combo of both (and I lean towards the eye) is the way to go...will add a few things:

*Bias/ being on the hype train...this is something I am really trying to concentrate on this year and something all of us are probably guilty of...there have been times where I just totally write off a guy (and the reason has validity behind it) and that close-minded attitude has hurt my evaluation of him...an example of this is Tee Higgins...I did not like how he looked in the playoffs last year and I completely wrote off the rest of his career and due to that I missed on what looks to be an emerging stud...on the flip-side Justice Hill is a guy I over-rated way too much and disregarded the negatives as I feel like I simply looked at his upside and was convinced he was going to contribute right away (i.e. I had tunnel vision on him)...I am really gonna keep an open-mind on players this off-season, especially if I see that others both on this board and in the draft community are maybe higher or lower on them then I am...I can already see my first test on this is Bateman as there is something about him I just don't like but I am not going to let that bias effect me like it has in the past.

*Situation...when looking at some of the high-level programs like Bama, Clemson and Ohio State you need to be real careful knocking guys for not having blow up stats or giving them a lot of credit for having blow-up stats because the amount of talent at the skill positions for these programs is just ridiculous....Josh Jacobs and Dwayne Haskins are two great examples of this...when Jacobs' value started rising pre-draft I never understood the knock on his college stats because he was sharing the backfield with a high quality veteran in Damien Harris and the #1 RB recruit in the nation in Najee Harris...there was no way anyone was gonna be a bell-cow in that situation and he should not have been dinged for that...Haskins was the opposite in that his #'s were just silly with a 4,831-50-8 stat-line but in hindsight those numbers seemed to cover-up a lot of warts with regard to how his game (and mentality) would translate to the NFL...a guy who makes me nervous of being like Haskins is Chuba Hubbard and a guy I could see being like Jacobs is Chris Olave but I will not have the bias on these two like I would in the past.

 

[Soulfly3]

80% eye test 10% analytics. 10% mancrush.

 

[Manster]

I'm an eye test/big picture kinda guy.....I hate crunching numbers, and I glaze over with details

When it comes to FF I consider...

1) eye test

2) opportunity

3) team situation...ie, good or bad offense

An example this season for me is, DK Metcalf.  I reached for him in both my leagues based on what my eyes told me last season.....Freak athlete.....Russ Wilson is his QB, and loved him as a rookie.......not a typical lockdown hawks defense anymore, so #LetRussCook was kind of inevitable.

 

[DAG]

I've typically been an eye test guy but the last few years I've tried to work in analytics content into my thinking more (I don't do my own, mainly because I don't math well). So I'm all for a holistic approach, and I'll continue to pay attention to analytics for sure.

That said, last summer I started to consume more analytics-focused content (podcasts and writers) and ended up talking myself into some guys like Taylor (jury's still out to be sure) who really didn't wow me when I watched them. So I will probably tilt the needle more heavily toward eye test , which has served me pretty well in the past. 

 

[cloppbeast]

The analytics out there for rookies that I've seen is not reliable or useful. The worst of it is cherry picking, the best is hypothesis testing seeking not to find comparable value between metrics. For example, some have found early declare and breakout age to be an important predictor for receivers. But, I have not come across anybody who has found relative importance with any other metric, specifically draft capital. So, ignoring hindsight, who was the superior prospect out of Bryan Edwards selected 91 with a boa of 18, or Brandon Aiyuk at 25 with a boa of 21.5? Unanswered.

Worse you have advanced stats like broken tackle rate, among others, where the assumed correlation to NFL success is dubius or mostly not established at all. The same goes for the underwear olympics.

The difference in schemes and talent between college programs causes difficulty in establishing statistical relationships. My opinion, nobody will find a metric from college football to build solid model. If somebody did, I'd bet it's already correlated to draft capital anyway, which nobody wants to control for.

Long story short, the accumulated eyeball tests of professionals is the best metric we have; it's still not that great really. This may not answer @Zyphros question of how to incorporate analytics into evaluating prospects. How would one rank rookies before the draft, like if one were a scout for example? It's an art I think.

 

[Boston]

One other thing and I maybe wrong whether others do it but I consider combine numbers (I.e. stuff like 40 time, size) as part of analytics...I think going too deep into the “nerdy” analytics can be very dicey.

 

[cloppbeast]

One other thing and I maybe wrong whether others do it but I consider combine numbers (I.e. stuff like 40 time, size) as part of analytics...I think going too deep into the “nerdy” analytics can be very dicey.

It's abuse of the word 'analytics' to include combine numbers. No relationship to NFL success is ever established beyond what some call thresholds, which is itself dubious.

 

[zamboni]

*Situation...when looking at some of the high-level programs like Bama, Clemson and Ohio State you need to be real careful knocking guys for not having blow up stats or giving them a lot of credit for having blow-up stats because the amount of talent at the skill positions for these programs is just ridiculous....

Situation is a great mention and perhaps as important as anything. But it’s obviously not just the college situation that is critical - it’s the NFL team/system a player lands. While supreme talent often rises to the top, even well above average talent may get lost depending on the circumstances/surrounding talent.

As for an analytic focus, the aforementioned Metcalf, he of the horrible cone drills, will likely forever be a poster child for not focusing too much on metrics.

 

[Boston]

It's abuse of the word 'analytics' to include combine numbers. No relationship to NFL success is ever established beyond what some call thresholds, which is itself dubious.

To general and definitive of a statement for me...size and speed will always have a place in evaluations.

 

[Boston]

Situation is a great mention and perhaps as important as anything. But it’s obviously not just the college situation that is critical - it’s whatever team/system a player lands. While supreme talent often rises to the top, even well above average talent may get lost depending on the circumstances/surrounding talent.

As for an analytic focus, the aforementioned Metcalf, he of the horrible cone drills, will likely forever be a poster child for not focusing too much on metrics.

Agree on going too deep on the metrics...his size and speed are elite...once you go too deep you are just looking for a reason to not like a guy or just trying to prove how smart you are.

 

[Teezee]

80% eye test / 20% analytics

No matter the position, first think I look at are feet.

 

[cloppbeast]

To general and definitive of a statement for me...size and speed will always have a place in evaluations.

Allow me to rephrase. In my wanderings I have not come across anybody establishing a causal relationship between any combine metric and NFL success, only lack thereof. If you have, do share.

 

[Boston]

Allow me to rephrase. In my wanderings I have not come across anybody establishing a causal relationship between any combine metric and NFL success, only lack thereof. If you have, do share.

I am more onboard with your wanderings stated that way...as I wrote before when I said combine I was thinking more about pure size and speed so my use of the word combine was probably not correct.

 

[Biabreakable]

The analytics out there for rookies that I've seen is not reliable or useful. The worst of it is cherry picking, the best is hypothesis testing seeking not to find comparable value between metrics. For example, some have found early declare and breakout age to be an important predictor for receivers. But, I have not come across anybody who has found relative importance with any other metric, specifically draft capital. So, ignoring hindsight, who was the superior prospect out of Bryan Edwards selected 91 with a boa of 18, or Brandon Aiyuk at 25 with a boa of 21.5? Unanswered.

Worse you have advanced stats like broken tackle rate, among others, where the assumed correlation to NFL success is dubius or mostly not established at all. The same goes for the underwear olympics.

The difference in schemes and talent between college programs causes difficulty in establishing statistical relationships. My opinion, nobody will find a metric from college football to build solid model. If somebody did, I'd bet it's already correlated to draft capital anyway, which nobody wants to control for.

Long story short, the accumulated eyeball tests of professionals is the best metric we have; it's still not that great really. This may not answer @Zyphros question of how to incorporate analytics into evaluating prospects. How would one rank rookies before the draft, like if one were a scout for example? It's an art I think.

Out of curiosity what does boa stand for?

 

[cloppbeast]

Out of curiosity what does boa stand for?

Breakout age

 

[Biabreakable]

I use both.

I think there are flaws in both methods. I think I understand both sides of this well enough to be discerning enough to identify the outliers from both methods and correct for that.

As far as ranking go. they are imperfect no matter how you do them, I think a ranking produced by data makes more sense to an observer than a ranking done through subjective method. A ranking of players based on yards per target for example is just that. I don't see it as anything else. You could try to use this ranking as part of a larger collection of data to form another ranking, but I don't know of a way to translate this data into fantasy points. 

It might be easier to make a ranking from eye test but what does the ranking actually mean? For most people it means who would you draft over who and a gun to the head test of that for each pick. We don't live in a vacuum. Whatever you know or think you know about things will go into that ranking.

I don't know if we can really separate these things. I can't anyways. Both things influence my opinions.

 

[Biabreakable]

Breakout age

Ah thanks. I figured it was something like that.

 

[Biabreakable]

I think part of the problem is that people are calling data analytics. Its not. Its data.

The analytics is what you do with the data.

 

[BoerumHill]

I find I am better served by ignoring film gurus podcasts and YT channels. Doing so caused me to discount Henry (needs a long runway, no ability to jump cut), Metcalf (look at this laughably bad route running!), McLaurin (issues tracking the ball, consistently waits on balls to hit his body instead of attacking it, plays small, loses 50/50 battles, average catch radius.) 

Its amazing how many big name FF X-spurts are mediocre at their craft. Combination of delivery, marketing, and snark drives traffic but their actual predictive ability is scarcely better than an upper echelon FBG.

Trust your own eyes. Triple check presented data (I find errors and repeated misinformation on the regular.) Trust your own projections. Recognize that aspect of the game is hot garbage, no one wants be outside a standard deviation in posting ranks. Groupthink misses value all the time.

Im not suggesting intransigent arrogance is the path to excellence. But don’t use projections and ranks as a crutch; chances are pretty good you’ll have the same 60-80% accuracy as anyone else. Nobody is perfect and missteps / bad calls are inevitable. Focus on your process and don’t let the occasional fail throw you off stride - especially when you used a good process but something unforeseen like game flow led to an unexpected outcome.

 

[cloppbeast]

I think part of the problem is that people are calling data analytics. Its not. Its data.

The analytics is what you do with the data.

Good point. Much of the analytics in this hobby is far from it, technically speaking. The moneyball guy didn't just kind of feel like OBP was an important metric and steals not so much. He built a regression model to test this theory. Otherwise it was just a hunch.

To be fair, baseball it's more copacetic to analytics. There's a lot more data available and not as much of a human element involved. In FF we should accept these limitations by putting very little confidence in anything labelled analytics. 

 

[BoerumHill]

*Situation...when looking at some of the high-level programs like Bama, Clemson and Ohio State you need to be real careful knocking guys for not having blow up stats or giving them a lot of credit for having blow-up stats because the amount of talent at the skill positions for these programs is just ridiculous....

Situation is a great mention and perhaps as important as anything. But it’s obviously not just the college situation that is critical - it’s the NFL team/system a player lands. While supreme talent often rises to the top, even well above average talent may get lost depending on the circumstances/surrounding talent.

As for an analytic focus, the aforementioned Metcalf, he of the horrible cone drills, will likely forever be a poster child for not focusing too much on metrics.

Edited 52 minutes ago by zamboni

You cannot legislate usage. Opportunity and situation are massively important. Lot of flawed guys have consistently good production due to right place, right time. I always favor talent but terrible O-line, predictable play calling, and coaches ego can all hold back good players, if only temporarily.

 

[BoerumHill]

Good point. Much of what is labelled analytics in this hobby is far from it. The moneyball guy didn't just kind of feel like OBP was an important metric and steals not so much. He built a regression model to test this theory. Otherwise it was just a hunch.

To be fair, baseball it's more copacetic to analytics. There's a lot more data available and not as much of a human element involved. In FF we should accept these limitations by putting very little confidence in anything labelled analytics. 

The athletic task is far more isolated. Pitcher versus batter, arm/fielder trying to beat legs/runner. No clock.

Whole lot more moving parts to football and even if 10 guys execute perfectly one blown assignment can mean disaster. Conversely one great athletic move (batting down a ball, beating a triple team) can overcome near perfect play by 11 guys. Harder for predictive analytics to account for either in football compared to MLB.

 

[Biabreakable]

Good point. Much of the analytics in this hobby is far from it, technically speaking. The moneyball guy didn't just kind of feel like OBP was an important metric and steals not so much. He built a regression model to test this theory. Otherwise it was just a hunch.

To be fair, baseball it's more copacetic to analytics. There's a lot more data available and not as much of a human element involved. In FF we should accept these limitations by putting very little confidence in anything labelled analytics. 

That is basically where I ended up with the charting.

The charting is trying to take the eye ball test and quantify it into data that can be compared to other players charted by the same method.

However I realized after charting about 50 or so games for players that not all of these samples for games was close to the same. Some players getting the ball twice as much as others on a per game basis and thus the data for those players more robust than the others. 

The sample sizes are too small.

This is less of a problem for baseball  where they play so many more games and the outcomes of each play are much more controlled.

I think ZWK does a terrific job of compiling data and analyzing that data to figure out which metrics are more useful than others. By doing that he is able to create a formula that is more A + B - C = points or because of this data the player falls within a certain range of outcomes. But even after all that he still has to make some judgement calls for his final rankings.

I respect ZWKs process although there are things he is still trying to figure out about this.

 

[Zyphros]

Appreciate all the responses here. 

I really don't know what I'm hoping to gain out of this conversation other than maybe a way to strengthen my process. Or insights into how you do your own.

If you're numbers driven, I feel it's easier to justify a change in process since it's essentially formula based and everything has a value. Change the value and tinker with it until the process becomes more wholesome. If you're eye test driven, then there really doesn't seem like a way to strengthen that process other than learning every nuance to every position, or just watching guys do it more often and hope you learn what's good/bad. 

Personally, as someone who uses more eye test, I can't trust someone else who used their own eye test. Especially when my eyes tell me otherwise. Hakeem Butler, probably is the most recent example of someone I wasn't sold on and there was an eye test "scout" who had him as the #1 WR in 2019. When it was all said and done, he was my WR7 and rookie #20. An opposite case was that there were eye tests on Metcalf that had him as a sure fire stud. He was my WR5 and rookie #9. Again I was skeptical but I still recognized the ceiling's for both of those players who I had as top8 WR's. Hindsight Metcalf should have been my WR3, Butler likely in the WR10-12 range. 

 

[Biabreakable]

The thing about watching the players is that you are limited to seeing what that college team does, and what that team does with that player.

The NFL will use them completely differently and show you things about the player the college team never showed you. 

You can only evaluate what you see and try to be consistent about it. I can see some imagination involved with what a player might eventually become to some extent to fill in those blanks, but thats more sketchy I think than just evaluating what is shown to you when the player plays.

 

[cloppbeast]

Appreciate all the responses here. 

I really don't know what I'm hoping to gain out of this conversation other than maybe a way to strengthen my process. Or insights into how you do your own.

If you're numbers driven, I feel it's easier to justify a change in process since it's essentially formula based and everything has a value. Change the value and tinker with it until the process becomes more wholesome. If you're eye test driven, then there really doesn't seem like a way to strengthen that process other than learning every nuance to every position, or just watching guys do it more often and hope you learn what's good/bad. 

Personally, as someone who uses more eye test, I can't trust someone else who used their own eye test. Especially when my eyes tell me otherwise. Hakeem Butler, probably is the most recent example of someone I wasn't sold on and there was an eye test "scout" who had him as the #1 WR in 2019. When it was all said and done, he was my WR7 and rookie #20. An opposite case was that there were eye tests on Metcalf that had him as a sure fire stud. He was my WR5 and rookie #9. Again I was skeptical but I still recognized the ceiling's for both of those players who I had as top8 WR's. Hindsight Metcalf should have been my WR3, Butler likely in the WR10-12 range. 

You'll never get them all correct either way. A few recent examples should not make you reconsider your approach. A batter wouldn't change his stance because he only gets a hit 30% of the time. 

I think this is a common problem, not from you specifically though. Some people don't want ever miss on a pick and won't accept it as part of the process. To avoid the next John Ross they'll never again pick a little fast receiver. This is a mistake. The play in blackjack is to hit on 13 even if you got busted the last 10 times. 

 

[Zyphros]

You'll never get them all correct either way. A few recent examples should not make you reconsider your approach. A batter wouldn't change his stance because he only gets a hit 30% of the time. 

I think this is a common problem, not from you specifically though. Some people don't want ever miss on a pick and won't accept it as part of the process. To avoid the next John Ross they'll never again pick a little fast receiver. This is a mistake. The play in blackjack is to hit on 13 even if you got busted the last 10 times. 

I'm not trying to not miss, I'm trying to improve on what I've done recently. If that requires a step backwards and NOT use data at all, then I think I'm up for it. It'll be harder to defend in some cases, like Higgins #1 last year, but that's what I'm working towards. 

 

[cloppbeast]

I'm not trying to not miss, I'm trying to improve on what I've done recently. If that requires a step backwards and NOT use data at all, then I think I'm up for it. It'll be harder to defend in some cases, like Higgins #1 last year, but that's what I'm working towards. 

So which metrics have you incorporated into your process?

 

[MAC_32]

I'm 100% information. Whether the means are eyes or data don't matter. 

 

[Zyphros]

So which metrics have you incorporated into your process?

Mostly I just use it as validation for someone I like, or sometimes it convinces me to like a guy more. Usually look at breakout age, dominator ratings, overall production, receiving production for RB's. Not a big fan of efficiency numbers usually because I think that's more indicative of their offense rather then their skill. Broken tackles being an exception to that.

I like to see multiple stats paired together to make me like a prospect, a good breakout age AND overall production for WR's (or production premiums/dominator ratings). Catching ability AND size for RB's. But I don't really eliminate a guy with/without these characteristics, so I don't know how much you can say I use metrics in my process. 

 

[Boston]

I'm 100% information. Whether the means are eyes or data don't matter. 

Great way to put it!

 

[Andy Dufresne]

It just seems to me that if analytics worked we'd have far fewer busts.

Two guys who passed the eye test yesterday were Kyle Pitts (again) and Zach Wilson (that last throw yesterday was pro level).

 

[cloppbeast]

It just seems to me that if analytics worked we'd have far fewer busts.

Two guys who passed the eye test yesterday were Kyle Pitts (again) and Zach Wilson (that last throw yesterday was pro level).

The same should apply to the eye test, am I right?

 

[Andy Dufresne]

The same should apply to the eye test, am I right?

Not really, no. It's the difference between art and science. 

Science implies reproducible results or at least a revealing of pattern. I don't see analytics doing that. 

Some people, OTOH, just have a better eye for talent than others. 

I suppose if someone doesn't have an eye for talent THEN they're better off using analytics. Or they should follow someone with a good eye.

 

[Biabreakable]

I'm not trying to not miss, I'm trying to improve on what I've done recently. If that requires a step backwards and NOT use data at all, then I think I'm up for it. It'll be harder to defend in some cases, like Higgins #1 last year, but that's what I'm working towards. 

Zyphros I think your process would be better if you could somehow filter out additional information which may influence what you are watching.

As far as ranking you just need to decide what your ranking means. If it is just I would take player X over player Y that is fine and maybe more useful than other things. We can find information on boa or whatever from other sources. If you are incorporating other information on top of that, perhaps have that be a separate ranking. I have seen you like to do tiers, and I think that can be a really good way to do it as well.

I don't know how you can filter out all of the other information that surely has some influence on all of us. I mean we have already been talking about some of the players who will be in the 2021 draft for a couple years now. 

 

[ghostguy123]

80% eye test 10% analytics. 10% mancrush.

Should be 90% drug test

 

[cloppbeast]

Mostly I just use it as validation for someone I like, or sometimes it convinces me to like a guy more. Usually look at breakout age, dominator ratings, overall production, receiving production for RB's. Not a big fan of efficiency numbers usually because I think that's more indicative of their offense rather then their skill. Broken tackles being an exception to that.

I like to see multiple stats paired together to make me like a prospect, a good breakout age AND overall production for WR's (or production premiums/dominator ratings). Catching ability AND size for RB's. But I don't really eliminate a guy with/without these characteristics, so I don't know how much you can say I use metrics in my process. 

I don't look at any number beside where guy gets drafted, so I'm curious what metric caused concern over Higgins.

 

[Zyphros]

I don't look at any number beside where guy gets drafted, so I'm curious what metric caused concern over Higgins.

It was just other risers through the process. The combine being a bit part of that. The times and such I feel are very overrated, but Higgins himself didn't participate in the drills so it was just an empty box. 

Looking at 2020 specifically, the combine separated Jeudy, Mims, Jefferson, Gibson, McFarland from Tyler Johnson, Gabriel Davis, AGG, CEH, Ke'Shawn Vaughn, and Eno Benjamin in my rankings. That's something I definitely want to continue doing. But I guess my solution is not to discredit a non-workout as much. 

 

[tangfoot]

I want to reply because I appreciate and consume every bit of analysis and data compiled by Zyphros and ZWK.  (And a lot of other sources as well)

As far as my evaluation goes, I do not watch college football.  I do not watch youtube videos of highlights.  I do not watch NFL games with an eye toward "does he have good footwork/can he run routes?".

I merely read.  I consume as much as I can from expert analysts all the way down to message board trolls.

I keep some rough Notepad documents with names of players who have been mentioned as having traits that generally lead toward NFL success.  When a player is mentioned multiple/several times, I do more reading about that player.

I enjoy reading posts that analyze both eye test and data analysis, but when it comes down to making a choice between players, I lean toward the guys who show it on the field vs workout warriors.  Sometimes those two combine into a tremendous prospect, but more often a player's evaluation is torn between people who see one thing and measure something else.

 

[Soulfly3]

Should be 90% drug test

heeeeeeeeeeee's baaaaaaaaaaack (soon)