> So let’s ask ourselves: would AI have told us this?
Why ask ourselves, when we can ask the AI? Here's the start of my conversation with Gemini:
> Me: What is known about fatty acid combustion in the brain?
> Gemini: The Brain's Surprising Use of Fat for Fuel
For a long time, the brain was thought to rely almost exclusively on glucose for its immense energy needs. However, a growing body of research reveals that fatty acid combustion, or beta-oxidation, plays a more significant and complex role in brain energy metabolism and overall neurological health than previously understood. While not the brain's primary fuel source under normal conditions, the breakdown of fatty acids is crucial for various functions, particularly within specialized brain cells and under specific physiological states....
It cites a variety of articles going back at least to the 1990s.
If you simply ask Gemini what the brain uses for fuel, it gives an entirely different answer that leaves fatty acids out completely and reinforces the glucose story.
LLMs tell you what you want to hear, sourced from a random sample of data, not what you need to, based on any professional/expert opinion.
When I ask the same question it says primarily glucose and also mentions ketone bodies. It mentions that the brain is flexible and while it normally metabolizes glucose it may sometimes need to metabolize other things. This is both at gemini.google.com and using google.com in "AI mode" in private browsing.
gemini.google.com mentions lactate and fat. But it also knows I care about science. I'm not sure how much history is used currently.
But this is kind of silly because if you're a member of the public and ask a scientist what the brain uses as fuel they'll also say glucose. If you've ever been in a conversation with someone who felt the need to tell you *every detail* of everything they know, then you'll understand that that's not how human communication typically works. So if you want something more specific you have to start the conversation in a way that elicits it.
If you ask most neuroscientists they’d say the same. Only a small subset of us would cite the literature that the brain’s caloric neuronal activity is ~10-15% unaccounted for by the amount of glucose neurons have access to. It’s a niche within a niche. And debated by the majority.
Yup. An accomplished scientist friend of mine looked up a topic in which he’s an expert and was deeply unimpressed - outdated, inaccurate, incomplete, misleading info (perhaps because much relevant research is paywalled). LLMs are amazing but not all-knowing.
You write as if this is your conclusion but it's really your premise.
> if it was only trained on papers supporting the dogma.
It's not, it's also trained on all the papers the authors of the current study read to make them think they should spend money researching fatty acid combustion in the brain.
> so that information is probably already in the training data.
Run an offline copy of Gemma with training cutoff before this study came out and it will also tell you about fatty acid combustion in the brain, with studies going back to the 60s and taking off around the 2000s or 2010s.
> So it cannot produce novel insights which would be a requirement if LLMs should "solve science".
How sure are we about this statement, and why? I have been hearing this a lot, and it might be true, but I would like to read some research into this area.
I tried it using Gemini 2.5 Pro and it cited this Hacker News thread for its first paragraph. I can't judge the other citations, other than to say they're not made up. (I see links to PubMed Central.)
>> So let’s ask ourselves: would AI have told us this?
My first thought: if it did, would you believe it?
> Yes and it did
And before today and this thread, if I asked something like it honestly, without already knowing the answer, and an LLM answered like this...
... I'd figure it's just making shit up.
Before AI will be able to "pretty much solve all our outstanding scientific problems Real Soon Now", it needs to be improved some more, but there's a second, underappreciated obstacle: we will need to learn to gradually start taking it more seriously. In particular, novel hypotheses and conclusions drawn from synthesizing existing research will, by their very nature, look like hallucinations to almost everyone, including domain experts.
The point was that LLMs are not well set up to find new insights unless they are already somehow contained in the knowledge they have been trained on.
This can mean "contained indirectly" which still makes this useful for scientific purposes.
The fact that the author maybe underestimated the knowledge about the topic of the article already contained within an LLM does not invalidate this point.
> The point was that LLMs are not well set up to find new insights unless they are already somehow contained in the knowledge they have been trained on.
The author is, to use his phrase, "deeply uninformed" on this point.
LLMs generalize very well and they have successfully pushed the frontier of at least one open problem in every scientific field I've bothered to look up.
Yes, but only after this post in science and on HN. As has been mentioned above, one of the links it offers is this very post.
So, AI will look online and synthesize the latest relevant blog posts. In Gemini's case, it will use trends to figure out what you are probably asking. And since this post caught traction, suddenly the long tail of related links are gaining traction as well.
But had Derek asked the question before writing the article, his links would not have matched. And his point that it isn't the AI that figured out that something has changed, remains relevant.
OT, I really enjoy his posts. As AI takes over, will we even read blog posts [enough for authors like him to keep writing], or just get the AI cliff notes - until there is no one writing novel stuff?
That cuts both ways: the model can be telling the truth, but because its claims look unusual, they'll get dismissed as hallucinations, and possibly even used as anti-example in training the next model generation.
It’s best to remember that ai is an extractive process, not a creative one. That’s why it seems to give you what you “want to hear”. The prompt directs the drill, the well spills out what you drilled into.
Answer to his though experiment:
Yes, I believe a sufficiently advanced AI could told us that. Scientists who have been fed with wrong information can come up with completely new ideas. Making what we know less wrong.
That being said, I don't think current token-predictors can do that.
My read of this was that AI is fundamentally limited by the lack of access to the new empirical data that drove this discovery; that it couldn't have been inferred from the existing corpus of knowledge.
Recent LLMs have larger context windows to process more data and tool use to get new data, so it would be surprising if there’s a fundamental limitation here.
I get that this is intended to be parsed "Discovering (what we think we know) is (wrong)", but it took me a while to discard the alternative "discovering (what we think (we know is wrong))".
Maybe an AI will be smart enough to realize that there's more than one explanation for a low level of triglycerides in neurons.
The RICE myth and the lactic acid myth will surely be a part of the training material so the AI will realize that there's a fair amount of unjustified conclusions in the bioworld
The RICE protocol (Rest, Ice, Compression, Elevation) for injuries has been largely debunked - inflammation is now understood as a necessary healing process. Similarly, lactic acid was wrongly blamed for muscle soreness when it's actually a fuel source during exercise, paralleling how we misunderstood neuronal fatty acid metabolism.
Is inflammation not still considered to be harmful in the long term? (Is that not why we're still expected to care about omega-6 vs omega-3 dietary fatty acids?) What is the new explanation for muscle soreness?
> So let’s ask ourselves: would AI have told us this?
Why ask ourselves, when we can ask the AI? Here's the start of my conversation with Gemini:
> Me: What is known about fatty acid combustion in the brain?
> Gemini: The Brain's Surprising Use of Fat for Fuel For a long time, the brain was thought to rely almost exclusively on glucose for its immense energy needs. However, a growing body of research reveals that fatty acid combustion, or beta-oxidation, plays a more significant and complex role in brain energy metabolism and overall neurological health than previously understood. While not the brain's primary fuel source under normal conditions, the breakdown of fatty acids is crucial for various functions, particularly within specialized brain cells and under specific physiological states....
It cites a variety of articles going back at least to the 1990s.
So
> would AI have told us this?
Yes and it did
If you simply ask Gemini what the brain uses for fuel, it gives an entirely different answer that leaves fatty acids out completely and reinforces the glucose story.
LLMs tell you what you want to hear, sourced from a random sample of data, not what you need to, based on any professional/expert opinion.
When I ask the same question it says primarily glucose and also mentions ketone bodies. It mentions that the brain is flexible and while it normally metabolizes glucose it may sometimes need to metabolize other things. This is both at gemini.google.com and using google.com in "AI mode" in private browsing.
gemini.google.com mentions lactate and fat. But it also knows I care about science. I'm not sure how much history is used currently.
But this is kind of silly because if you're a member of the public and ask a scientist what the brain uses as fuel they'll also say glucose. If you've ever been in a conversation with someone who felt the need to tell you *every detail* of everything they know, then you'll understand that that's not how human communication typically works. So if you want something more specific you have to start the conversation in a way that elicits it.
If you ask a neuroscience teacher the same question you're also told it's all glucose and maybe occasionally ketone bodies.
If you ask most neuroscientists they’d say the same. Only a small subset of us would cite the literature that the brain’s caloric neuronal activity is ~10-15% unaccounted for by the amount of glucose neurons have access to. It’s a niche within a niche. And debated by the majority.
Yup. An accomplished scientist friend of mine looked up a topic in which he’s an expert and was deeply unimpressed - outdated, inaccurate, incomplete, misleading info (perhaps because much relevant research is paywalled). LLMs are amazing but not all-knowing.
Well yeah, today we know the dogma was wrong and so that information is probably already in the training data.
I think what Lowe meant was that an LLM could not have come up with this "on its own", if it was only trained on papers supporting the dogma.
So it cannot produce novel insights which would be a requirement if LLMs should "solve science".
> So it cannot produce novel insights
You write as if this is your conclusion but it's really your premise.
> if it was only trained on papers supporting the dogma.
It's not, it's also trained on all the papers the authors of the current study read to make them think they should spend money researching fatty acid combustion in the brain.
> so that information is probably already in the training data.
Run an offline copy of Gemma with training cutoff before this study came out and it will also tell you about fatty acid combustion in the brain, with studies going back to the 60s and taking off around the 2000s or 2010s.
> So it cannot produce novel insights which would be a requirement if LLMs should "solve science".
How sure are we about this statement, and why? I have been hearing this a lot, and it might be true, but I would like to read some research into this area.
I tried it using Gemini 2.5 Pro and it cited this Hacker News thread for its first paragraph. I can't judge the other citations, other than to say they're not made up. (I see links to PubMed Central.)
>> So let’s ask ourselves: would AI have told us this?
My first thought: if it did, would you believe it?
> Yes and it did
And before today and this thread, if I asked something like it honestly, without already knowing the answer, and an LLM answered like this...
... I'd figure it's just making shit up.
Before AI will be able to "pretty much solve all our outstanding scientific problems Real Soon Now", it needs to be improved some more, but there's a second, underappreciated obstacle: we will need to learn to gradually start taking it more seriously. In particular, novel hypotheses and conclusions drawn from synthesizing existing research will, by their very nature, look like hallucinations to almost everyone, including domain experts.
I think you missed the point the article makes.
The point was that LLMs are not well set up to find new insights unless they are already somehow contained in the knowledge they have been trained on. This can mean "contained indirectly" which still makes this useful for scientific purposes.
The fact that the author maybe underestimated the knowledge about the topic of the article already contained within an LLM does not invalidate this point.
> The point was that LLMs are not well set up to find new insights unless they are already somehow contained in the knowledge they have been trained on.
The author is, to use his phrase, "deeply uninformed" on this point.
LLMs generalize very well and they have successfully pushed the frontier of at least one open problem in every scientific field I've bothered to look up.
Except it inadvertently showed that LLMs might be more flexible thinkers than most people. Everyone knows…
Yes, but only after this post in science and on HN. As has been mentioned above, one of the links it offers is this very post.
So, AI will look online and synthesize the latest relevant blog posts. In Gemini's case, it will use trends to figure out what you are probably asking. And since this post caught traction, suddenly the long tail of related links are gaining traction as well.
But had Derek asked the question before writing the article, his links would not have matched. And his point that it isn't the AI that figured out that something has changed, remains relevant.
OT, I really enjoy his posts. As AI takes over, will we even read blog posts [enough for authors like him to keep writing], or just get the AI cliff notes - until there is no one writing novel stuff?
What facts did it hallucinate and which are true?
That cuts both ways: the model can be telling the truth, but because its claims look unusual, they'll get dismissed as hallucinations, and possibly even used as anti-example in training the next model generation.
It’s best to remember that ai is an extractive process, not a creative one. That’s why it seems to give you what you “want to hear”. The prompt directs the drill, the well spills out what you drilled into.
Answer to his though experiment: Yes, I believe a sufficiently advanced AI could told us that. Scientists who have been fed with wrong information can come up with completely new ideas. Making what we know less wrong.
That being said, I don't think current token-predictors can do that.
My read of this was that AI is fundamentally limited by the lack of access to the new empirical data that drove this discovery; that it couldn't have been inferred from the existing corpus of knowledge.
Recent LLMs have larger context windows to process more data and tool use to get new data, so it would be surprising if there’s a fundamental limitation here.
I get that this is intended to be parsed "Discovering (what we think we know) is (wrong)", but it took me a while to discard the alternative "discovering (what we think (we know is wrong))".
> the constant possibility that something that Everybody Knows will turn out to be wrong
Reminds me of astronomy and also quantum mechanics
I think this could use a more informative title? The title this was posted with is actually less informative than the original title.
Derek has a little thought experiment at the end.
Maybe an AI will be smart enough to realize that there's more than one explanation for a low level of triglycerides in neurons.
The RICE myth and the lactic acid myth will surely be a part of the training material so the AI will realize that there's a fair amount of unjustified conclusions in the bioworld
The RICE protocol (Rest, Ice, Compression, Elevation) for injuries has been largely debunked - inflammation is now understood as a necessary healing process. Similarly, lactic acid was wrongly blamed for muscle soreness when it's actually a fuel source during exercise, paralleling how we misunderstood neuronal fatty acid metabolism.
Is inflammation not still considered to be harmful in the long term? (Is that not why we're still expected to care about omega-6 vs omega-3 dietary fatty acids?) What is the new explanation for muscle soreness?
There is a difference between localized inflammation that is bringing the source of healing to injury and systemic inflammation