I really feel out of my depth because 2 out of the 3 methods here seem like they shouldn’t work?
> To evaluate comprehensibility quantitatively, we employ an LLM-as-a-Judge framework
This isn’t the worst idea, but it’s still a bit incestuous. Adding an LLM judge to check for hallucinations creates two new kinds of problems: false positives, where your judge hallucinates an incorrect fact, and false negatives, where the judge lets a hallucination slip by.
> We measure reproducibility through knowledge distillation. By fine-tuning a weaker model on the generated CoT traces, we use the downstream performance gain of the student as a proxy.
And my problem here, as a member of the GPU proletariat, is that this just seems incredibly inefficient. In other words, you’re going to generate a bunch of rollouts from your model then wait for the student to train? I guess if you have the compute to train a trillion params then maybe you don’t care.
Empirically, many problems look like they're easier to check than they are to solve. This seems like a reasonable way to bootstrap a little extra performance, with prior art in well-known DeepMind experiments. It's unclear if it works recursively (I imagine not), but the core idea is solid.
I’m not reacting to the idea of using an LLM as a judge in general. That’s a proven path.
I’m specifically reacting to using it to reward the chain of thought during RL training because models love to hack their rewards, learning any possible shortcut rather than the task we want them to.
I have my LLM agents fact check each other as a matter of course. They each regularly find things that the other missed. They are typically the same model (Opus).
> scaling to 1T parameters significantly enhances sample efficiency and performance ceilings;
Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence.
The human brain has a few billion neurons and uses as much power as a light bulb.
True although a lot of those neurons and synapses are in the cerebellum, responsible for motor coordination and or in the visual cortex and so forth. Only a portion are in the language and reasoning areas. LLM's are comparable to human scale now, i think, and if trends continue will swiftly pass us by in the future.
If I had a magic button I would not only pause AI development but set it back 10 years. Sadly I have no influence on events and those who do, don't care about the future of humankind or actively wish us dead.
In 1956 a 5mb hard drive shipped on a large truck and took a team of men to unload. It consumed huge amounts of power, and cost about $3,200/month to run. In today's dollars that would be about $160,000 per month.
Aren't you glad we didnt just give up because it was kind of expensive?
Nobody's saying give up. I'm saying if your solution needs a trillion parameters and a power plant, and biology does it with a few billion neurons and a sandwich, that you're maybe on the wrong track. This is not an engineering gap.
Considering progress in the rest of computing stuff (RAM, CPUs, storage) is kind of "linear"/exponential it sure looks like it's an engineering gap and we're on the right track. GPT 3 was 175B parameters and is today crushed by models that are 32B parameters, that's a lot of progress in 6 years.
When for the training part you have to consider brains had like billions of years to develop. Maybe one of the reasons llms seem to be so expensive to train is because we are "compressing" in far less time that learning part
I really feel out of my depth because 2 out of the 3 methods here seem like they shouldn’t work?
> To evaluate comprehensibility quantitatively, we employ an LLM-as-a-Judge framework
This isn’t the worst idea, but it’s still a bit incestuous. Adding an LLM judge to check for hallucinations creates two new kinds of problems: false positives, where your judge hallucinates an incorrect fact, and false negatives, where the judge lets a hallucination slip by.
> We measure reproducibility through knowledge distillation. By fine-tuning a weaker model on the generated CoT traces, we use the downstream performance gain of the student as a proxy.
And my problem here, as a member of the GPU proletariat, is that this just seems incredibly inefficient. In other words, you’re going to generate a bunch of rollouts from your model then wait for the student to train? I guess if you have the compute to train a trillion params then maybe you don’t care.
You can and should eval your judges. They're also typically easier to eval because often you have them emit categorical/structured data.
> LLM-as-a-Judge
Empirically, many problems look like they're easier to check than they are to solve. This seems like a reasonable way to bootstrap a little extra performance, with prior art in well-known DeepMind experiments. It's unclear if it works recursively (I imagine not), but the core idea is solid.
I’m not reacting to the idea of using an LLM as a judge in general. That’s a proven path.
I’m specifically reacting to using it to reward the chain of thought during RL training because models love to hack their rewards, learning any possible shortcut rather than the task we want them to.
I have my LLM agents fact check each other as a matter of course. They each regularly find things that the other missed. They are typically the same model (Opus).
> scaling to 1T parameters significantly enhances sample efficiency and performance ceilings;
Man, I find SOTA deep learning somewhat hilarious. We scale models to absurd proportions, burning through a shitload of resources just to achieve (slightly above) human intelligence. The human brain has a few billion neurons and uses as much power as a light bulb.
The human brain is estimated to have approaching 10^11 neurons (most of them in the cerebellum).
However, a neuron is much more than a single parameter. The brain is estimated to have from 10^14 to 5x10^14 synapses.
True although a lot of those neurons and synapses are in the cerebellum, responsible for motor coordination and or in the visual cortex and so forth. Only a portion are in the language and reasoning areas. LLM's are comparable to human scale now, i think, and if trends continue will swiftly pass us by in the future.
If I had a magic button I would not only pause AI development but set it back 10 years. Sadly I have no influence on events and those who do, don't care about the future of humankind or actively wish us dead.
More recent research suggests the cerebellum does actually have a significant role in cognition. See for example https://pmc.ncbi.nlm.nih.gov/articles/PMC4089997/ and https://www.sciencedirect.com/science/article/pii/S138904172....
This is the worst they will ever be.
In 1956 a 5mb hard drive shipped on a large truck and took a team of men to unload. It consumed huge amounts of power, and cost about $3,200/month to run. In today's dollars that would be about $160,000 per month.
Aren't you glad we didnt just give up because it was kind of expensive?
Nobody's saying give up. I'm saying if your solution needs a trillion parameters and a power plant, and biology does it with a few billion neurons and a sandwich, that you're maybe on the wrong track. This is not an engineering gap.
Considering progress in the rest of computing stuff (RAM, CPUs, storage) is kind of "linear"/exponential it sure looks like it's an engineering gap and we're on the right track. GPT 3 was 175B parameters and is today crushed by models that are 32B parameters, that's a lot of progress in 6 years.
When for the training part you have to consider brains had like billions of years to develop. Maybe one of the reasons llms seem to be so expensive to train is because we are "compressing" in far less time that learning part
I think it’s more accurate to say that the mechanism that grows a brain had millions of years to develop.
Similarly, all the components in an audio amplifier are super dumb because you can just have 4 guys play in your living room amirite
Do you find cars similarly hilarious?
I mean yes, cars are absurdly inefficient for a lot of trips. America’s addiction to cars is kind of funny.
Yes
everything in LLM world is that right now.
the active layers concept seems to be experiencing convergent evolution to how synapses in a human function
huge parameter models with many small but efficient layers can work quickly on low resource hardware
similar to how neurons experience chemical spiking to activate small portions of the brain at once