Intelligence and Overfit

Publish Date
Principles Series
Last Edit
Apr 3, 2022 01:47 PM
Word Count
Order ID
Yesterday we spoke about intelligence and pattern recognition. As promised, today we’ll look at the downsides of excellent pattern recognition in humans.
To do that, we can draw an analog to other exceptional pattern recognition engines: computers. If intelligence were fully defined by pattern recognition, computers may already be superior to their creators. Even using basic techniques like regression, computers are often able to spot relationships that humans miss.
But the downfall of computer pattern recognition, as any good data scientist knows, is overfitting. In lay terms, overfitting is when a model uses the data to create a mathematical explanation for the data it sees, but one that won’t hold up when applied to new data.
Put practically, imagine the following. You feed a computer a bunch of people’s height, weight, hair color, shoe size and ask it to determine whether they are left-handed. The computer will draw the best relationship it can from this – one which may explain a lot or all of the data you provide. But as each of these factors is (to my knowledge) uncorrelated with left-handedness, the model will be worthless when predicting if a new person is left-handed.
notion image
It’s broadly been recognized that humans also suffer from overfitting. Superstitions can be seen as us overfitting behavior models: we onetime saw something bad happen when a guy walked under a ladder, so now no one should do it. More practically, the work of Nobel Prize winning economist Daniel Kahneman shows how humans systematically are biased in our thinking, with anchoring and loss aversion biases being two of the most common. Each of these merits its own coverage at some point.
With that all said, let’s link this back to intelligence. More intelligence leads to more pattern recognition, but it does not ensure that these patterns are real. They may be, and often are spurious overfits. This is a handicap that comes with intelligence. This is why, as Naval Ravikant put it on a recent episode of his podcast, “Judgement is the Decisive Skill,” and why he states in it that “Intellect without any experience is often worse than useless.”
Judging how much a smart person overfits with their additional pattern recognization skills is a fool’s errand, and would vary greatly from person to person. Thus I’ll bring in my personal experience: I’ve seen many objectively smart people be overconfident in “proven” ideas. My bias is to opt on the side of caution – it’s better to admit that we don’t yet know something, and that the unintended consequences floating around are both powerful and hard-to-predict.