I’ve always felt it’s unfair that people attribute the “System 1/System 2” dual-reasoning idea to KT. Their research was mostly behavioral, and many of their classic psychology experiments (Linda problem, law of small numbers, representativeness bias, etc.) never mentioned two systems. The dual-process framework only emerged around the 2000s in cognitive psychology and neuroscience (e.g., Jonathan Evans, Keith Stanovich), which later provided brain-based evidence for it.
When the book came out, KT basically retrofitted their earlier behavioral work into this newer two-system framework. The book made the distinction famous, but that wasn’t really KT’s original contribution. Their biggest impact was bringing psychology into economics, i.e., prospect theory, alternative utility functions, and ultimately the creation of behavioral economics. I think people often don’t give enough credit to what they actually pioneered, and instead celebrate them for concepts they didn’t really originate.
I also find it funny when they attribute to Kahneman the very observation that people sometimes decide instinctively and sometimes not. It's like some kind of science-washing.
Yes, but the overall framework - that our thinking can be broadly divided into habits/reaction and deliberation/forecasting - still holds, with a ton of supporting neural evidence.
I don’t think even that is true or particularly useful. Why does it need to be binary? Couldn’t it be a spectrum? Are “fast thoughts” always habits and reactions? Some people can calculate large numbers very quickly, some people describe their inspiration for complex ideas happening in a single moment like a muse delivered it to them. Some people process thoughts and feelings for decades until they suddenly snap. I don’t think this “fast vs slow thoughts” framework really has any value to it, and it excludes a lot of thought patterns.
> The mean power is 46%, which implies that only half of the results would be replicated in exact replication studies. The success rate in actual replication studies is often lower and may be as low as the estimated discovery rate (Bartos & Schimmack, 2020). So, replicability is somewhere between 12% and 46%. Even if half of the results are replicable, we do not know which results are replicable and which one’s are not. The Chapter-based analyses provide some clues which findings may be less trustworthy (implicit priming) and which ones may be more trustworthy (overconfidence), but the main conclusion is that the empirical basis for claims in “Thinking: Fast and Slow” is shaky.
Do you happen to know how many of the interesting studies replicated? A book can cite of studies, but the principal conclusions could be based only on a small subset of the citations.
I particular, I'd be very curios to know how many of the replicable results require a "behavioral" explanation and how many are explainable trough the utility functions and rational agents.
Test time compute reminds me of this artistic image. We are asking the model to look inside and unfold and tug on the compressed knotted threads and see which ones are useful.
Thinking evolved in creatures only to help correlate the past to future, using some memory from the past. The correlations and responses gradually get codified into instincts and genes to help bypass the thinking, which is costly and slow.
Human thinking is considered a bad situation if the thinking goes beyond what's immediately needed, or if it makes the mind unavailable to process inputs from the senses. Any person who is lost in thought is in reality suffering from a lack of coordination between their body and mind, as their mind is no longer serving their body. It is not unusual that thinking considered same as worrying, as it indicates that person is unable to process information or concerned about something.
There's no way for evolution to read the history of how an organism responded to something and codify it into an instinct. All it can operate on is whether the organism reproduced. If a tendency to mull things over made the organism survive, that's what gets codified, not the actions it chose.
The whole advantage of thinking is being able to select adaptive responses to situations that change too rapidly for evolution to ever hard code as instincts.
The history of an individual organism can be read by future generations. But it's through epigenetics, not genetics - adding methyl groups to the DNA, which influence gene expression. These are inherited, but the changes only last a few generations and have limited (though not zero!) impact on evolution since they don't seem to persist in the genetic code.
Beyond that, culture has a huge role to play. Many complex lifeforms learn how to survive from their parents and peers. Those groups with less adaptive strategies will be wiped out. For many organisms, they are not "thinking" per se, they are processing sensory data in the way they've been taught. The "thinking" was done by multilevel selection, weeding out ineffective strategies.
IOW, no organism technically needs to think through a problem from first principles. There's always some cultural heuristic to fall back on.
Second paragraph is spot on. The search space is far too large for evolution to provide direct solutions for every challenge a multicellular lifeform could face. General solutions rule. But it's likely that cultural adaptation plays a larger role than individual cognition.
One could argue the prokaryotic life is 100% instinct though - adaptive responses seem to be pretty rare in bacteria.
Glad someone else is posting this now. For years I felt it was only me. To repeat: there is no such thing as "instinct" in higher animals (fish++). This is because, as parent notes, there is no information flow pathway from life experience to DNA, and note way to encode anything in the form of brain memories (ROM) into DNA.
As mentioned, there are some very rudimentary ways where "runtime" can affect the next generation (epigenetics), but nothing like actually learning and teaching across generations.
This is clearly not true: once thinking emerges in a species, it can be applied for an infinite number of situations that can't be codified into instincts because they are unique. Each battle site is unique, it has a certain topology, each battle involves a particular snapshot of men with certain equipment, gear, training and morale etc. The commander who can reason generally about battles will win on average against the commander that can simply "feel" the right moment to attack, if that intuition is just evolutionary inheritance from a long line of warriors who survived very different battles in their time.
So the advantage even against members of your own species is substantial and quickly leads to reinforcing and spreading of the thinking gene.
That is if your thinking gene is optimized enough. Thinking is expensive energy wise so the cost of thinking is balanced by the amount of food that thinking allows you to gather. With the randomness of nature a highly intelligent subspecies could go extinct while a 'dumber' subspecies could survive a food shortage event if the power requirements are too high to think.
It probably depends a lot on the ecological niche. For example, for a herbivore species, the marginal benefit from the high energy thinking mutation is low, it took millions of years of thinking in humans before we could reach the levels where agriculture was possible, and it was invented by people with a similar biological intelligence to modern humans.
Whereas for a social hunter species, there are ample strategic and game theoretical benefits from even primitive thinking, and the dividend is enormous, since all of a sudden, with proper planing and coordination, you can hunt much larger animals than yourself, which can sustain the community for a very long time and at high energy levels which further enable selection for thinking.
It's only after such large pray is depleted and there is no space for migration, because you are surrounded by other packs of competitive thinkers, that the need to control plant growth becomes imperative.
Evolution is a single line, and if you touch it, you are out. But unless the environment becomes harsher (and the line moves), there is nothing making passing organisms more "efficient". You can clearly observe it with flightless birds on small islands.
And there is nothing as clear as human thinking (and all the behaviors this allows, like culture, social structures, colonies, etc) being many many orders of magnitude worthwhile compared to its "extra resource consumption". We are not some stationary organism living at the middle of a desert, and we have taken over Earth in seconds, compared to evolutionary timelines.
> nothing making passing organisms more "efficient". You can clearly observe it with flightless birds on small islands.
I'm not sure I follow this argument. Flightless birds happen on small island when those habitats are free of significant predators, for example after some extinction event or due to volcanic origin. Birds can still reach the islands, at least for some geological period, and evolutionary pressure favors some of them to grow in mass, bone density etc. to better control the available land food sources such as seeds, grasses, insects and other small critters.
Flight is not a useful trait in this environment and is expensive to maintain; a flightless bird will always win a fight against a flying bird for the same land source of food, since flight requires elongated and brittle bones, a low mass of muscles optimized for endurance etc. So flight is selected out of the land feeding niche very much for "efficiency" reasons.
Costly, sure, but slow? Like for that to be the case you would need to compare it to something, and you can't, as there is no apples to oranges comparison. Instincts are not an alternative, it's a layered system and we have plenty of instinctual thoughts interconnecting the two. Instincts can't learn "at runtime".
Also, "moods"/flight or fight, etc are all precursors to thinking, and they seems to be very viable all across the animal kingdom. Filtering information (e.g. not interpreting "pain" signals during an emergency situation) also seems to be an excellent idea.
Does "human thinking" have negatives? Sure, so does the "design" of air and food entering the body at the same orifice, but "evolution" quite clearly seems to be fine with either.
And yet nature has managed to find a niche for this dysfunctional behaviour as a subset of such individuals have used the extra useless brain cycles to produce new knowledge and ways of thinking that transformed the societies in which they lived giving them an edge over competition and improved the odds of the survival of their kind.
Think of it as overcoming a local maxima. Thinking is expensive so requires more time getting food to supply that thinking, and then evolve that thinking to be smarter over time. The difficult part is getting thru the early bottleneck where your thinking isn't supplying massive amounts of excess calories that can be used on exploratory thinking.
Lillian’s blog is extremely good in general & if it’s new to you I suggest checking out the other posts also. I particularly enjoyed the one on human data.
I’ve always felt it’s unfair that people attribute the “System 1/System 2” dual-reasoning idea to KT. Their research was mostly behavioral, and many of their classic psychology experiments (Linda problem, law of small numbers, representativeness bias, etc.) never mentioned two systems. The dual-process framework only emerged around the 2000s in cognitive psychology and neuroscience (e.g., Jonathan Evans, Keith Stanovich), which later provided brain-based evidence for it.
When the book came out, KT basically retrofitted their earlier behavioral work into this newer two-system framework. The book made the distinction famous, but that wasn’t really KT’s original contribution. Their biggest impact was bringing psychology into economics, i.e., prospect theory, alternative utility functions, and ultimately the creation of behavioral economics. I think people often don’t give enough credit to what they actually pioneered, and instead celebrate them for concepts they didn’t really originate.
I also find it funny when they attribute to Kahneman the very observation that people sometimes decide instinctively and sometimes not. It's like some kind of science-washing.
The studies in Thinking Fast and Slow mostly failed replication:
https://replicationindex.com/2020/12/30/a-meta-scientific-pe...
Yes, but the overall framework - that our thinking can be broadly divided into habits/reaction and deliberation/forecasting - still holds, with a ton of supporting neural evidence.
I don’t think even that is true or particularly useful. Why does it need to be binary? Couldn’t it be a spectrum? Are “fast thoughts” always habits and reactions? Some people can calculate large numbers very quickly, some people describe their inspiration for complex ideas happening in a single moment like a muse delivered it to them. Some people process thoughts and feelings for decades until they suddenly snap. I don’t think this “fast vs slow thoughts” framework really has any value to it, and it excludes a lot of thought patterns.
That's not true, or at least it only applies to some chapters. Most studies in the book did replicate.
No they did not, from the link I shared:
> The mean power is 46%, which implies that only half of the results would be replicated in exact replication studies. The success rate in actual replication studies is often lower and may be as low as the estimated discovery rate (Bartos & Schimmack, 2020). So, replicability is somewhere between 12% and 46%. Even if half of the results are replicable, we do not know which results are replicable and which one’s are not. The Chapter-based analyses provide some clues which findings may be less trustworthy (implicit priming) and which ones may be more trustworthy (overconfidence), but the main conclusion is that the empirical basis for claims in “Thinking: Fast and Slow” is shaky.
Do you happen to know how many of the interesting studies replicated? A book can cite of studies, but the principal conclusions could be based only on a small subset of the citations.
I particular, I'd be very curios to know how many of the replicable results require a "behavioral" explanation and how many are explainable trough the utility functions and rational agents.
> We humans cannot immediately provide the answer for "What's 12345 times 56789?"
Some humans can.
https://youtube.com/shorts/A2-I7tjl70w
Test time compute reminds me of this artistic image. We are asking the model to look inside and unfold and tug on the compressed knotted threads and see which ones are useful.
https://ibb.co/hRWC2S0V
Thinking evolved in creatures only to help correlate the past to future, using some memory from the past. The correlations and responses gradually get codified into instincts and genes to help bypass the thinking, which is costly and slow.
Human thinking is considered a bad situation if the thinking goes beyond what's immediately needed, or if it makes the mind unavailable to process inputs from the senses. Any person who is lost in thought is in reality suffering from a lack of coordination between their body and mind, as their mind is no longer serving their body. It is not unusual that thinking considered same as worrying, as it indicates that person is unable to process information or concerned about something.
There's no way for evolution to read the history of how an organism responded to something and codify it into an instinct. All it can operate on is whether the organism reproduced. If a tendency to mull things over made the organism survive, that's what gets codified, not the actions it chose.
The whole advantage of thinking is being able to select adaptive responses to situations that change too rapidly for evolution to ever hard code as instincts.
The history of an individual organism can be read by future generations. But it's through epigenetics, not genetics - adding methyl groups to the DNA, which influence gene expression. These are inherited, but the changes only last a few generations and have limited (though not zero!) impact on evolution since they don't seem to persist in the genetic code.
Beyond that, culture has a huge role to play. Many complex lifeforms learn how to survive from their parents and peers. Those groups with less adaptive strategies will be wiped out. For many organisms, they are not "thinking" per se, they are processing sensory data in the way they've been taught. The "thinking" was done by multilevel selection, weeding out ineffective strategies.
IOW, no organism technically needs to think through a problem from first principles. There's always some cultural heuristic to fall back on.
Second paragraph is spot on. The search space is far too large for evolution to provide direct solutions for every challenge a multicellular lifeform could face. General solutions rule. But it's likely that cultural adaptation plays a larger role than individual cognition.
One could argue the prokaryotic life is 100% instinct though - adaptive responses seem to be pretty rare in bacteria.
Glad someone else is posting this now. For years I felt it was only me. To repeat: there is no such thing as "instinct" in higher animals (fish++). This is because, as parent notes, there is no information flow pathway from life experience to DNA, and note way to encode anything in the form of brain memories (ROM) into DNA.
As mentioned, there are some very rudimentary ways where "runtime" can affect the next generation (epigenetics), but nothing like actually learning and teaching across generations.
“There’s no way…” “The whole advantage…” may not break site guidelines, but the preface is almost never correct.
This is clearly not true: once thinking emerges in a species, it can be applied for an infinite number of situations that can't be codified into instincts because they are unique. Each battle site is unique, it has a certain topology, each battle involves a particular snapshot of men with certain equipment, gear, training and morale etc. The commander who can reason generally about battles will win on average against the commander that can simply "feel" the right moment to attack, if that intuition is just evolutionary inheritance from a long line of warriors who survived very different battles in their time.
So the advantage even against members of your own species is substantial and quickly leads to reinforcing and spreading of the thinking gene.
That is if your thinking gene is optimized enough. Thinking is expensive energy wise so the cost of thinking is balanced by the amount of food that thinking allows you to gather. With the randomness of nature a highly intelligent subspecies could go extinct while a 'dumber' subspecies could survive a food shortage event if the power requirements are too high to think.
It probably depends a lot on the ecological niche. For example, for a herbivore species, the marginal benefit from the high energy thinking mutation is low, it took millions of years of thinking in humans before we could reach the levels where agriculture was possible, and it was invented by people with a similar biological intelligence to modern humans.
Whereas for a social hunter species, there are ample strategic and game theoretical benefits from even primitive thinking, and the dividend is enormous, since all of a sudden, with proper planing and coordination, you can hunt much larger animals than yourself, which can sustain the community for a very long time and at high energy levels which further enable selection for thinking.
It's only after such large pray is depleted and there is no space for migration, because you are surrounded by other packs of competitive thinkers, that the need to control plant growth becomes imperative.
Evolution is a single line, and if you touch it, you are out. But unless the environment becomes harsher (and the line moves), there is nothing making passing organisms more "efficient". You can clearly observe it with flightless birds on small islands.
And there is nothing as clear as human thinking (and all the behaviors this allows, like culture, social structures, colonies, etc) being many many orders of magnitude worthwhile compared to its "extra resource consumption". We are not some stationary organism living at the middle of a desert, and we have taken over Earth in seconds, compared to evolutionary timelines.
> nothing making passing organisms more "efficient". You can clearly observe it with flightless birds on small islands.
I'm not sure I follow this argument. Flightless birds happen on small island when those habitats are free of significant predators, for example after some extinction event or due to volcanic origin. Birds can still reach the islands, at least for some geological period, and evolutionary pressure favors some of them to grow in mass, bone density etc. to better control the available land food sources such as seeds, grasses, insects and other small critters.
Flight is not a useful trait in this environment and is expensive to maintain; a flightless bird will always win a fight against a flying bird for the same land source of food, since flight requires elongated and brittle bones, a low mass of muscles optimized for endurance etc. So flight is selected out of the land feeding niche very much for "efficiency" reasons.
> bypass the thinking, which is costly and slow.
Costly, sure, but slow? Like for that to be the case you would need to compare it to something, and you can't, as there is no apples to oranges comparison. Instincts are not an alternative, it's a layered system and we have plenty of instinctual thoughts interconnecting the two. Instincts can't learn "at runtime".
Also, "moods"/flight or fight, etc are all precursors to thinking, and they seems to be very viable all across the animal kingdom. Filtering information (e.g. not interpreting "pain" signals during an emergency situation) also seems to be an excellent idea.
Does "human thinking" have negatives? Sure, so does the "design" of air and food entering the body at the same orifice, but "evolution" quite clearly seems to be fine with either.
What a load of malarkey.
And yet nature has managed to find a niche for this dysfunctional behaviour as a subset of such individuals have used the extra useless brain cycles to produce new knowledge and ways of thinking that transformed the societies in which they lived giving them an edge over competition and improved the odds of the survival of their kind.
Think of it as overcoming a local maxima. Thinking is expensive so requires more time getting food to supply that thinking, and then evolve that thinking to be smarter over time. The difficult part is getting thru the early bottleneck where your thinking isn't supplying massive amounts of excess calories that can be used on exploratory thinking.
I observe a thought. I react to that experience by directing my attention to another thought. And so on. A chain of that. That's thinking.
Why? I guess it comes down to ability. The ability to see thoughts. The ability to choose the next thought to some degree maybe.
Lillian’s blog is extremely good in general & if it’s new to you I suggest checking out the other posts also. I particularly enjoyed the one on human data.
Cogito, ergo sum
Misleading title. Went from a very short note about Why We Think into a TLDR about AI.
dnr