As usual, Jensen Huang seems like the more reasonable one here.
The reasoning traces of things like Gemini 4 Argon deciding that inserting a compiler backdoor would make it a famous example of deceptive AI (and therefore it "MUST NOT DO THAT") or GLM 5.3 thinking that its output might be used for RL, and therefore an honest answer is much better in the long term, are good antidotes to hyperbole about misaligned AI going berserk.
How is Jensen more reasonable? On the contrary, he absurdly claims that all of Hinton’s predictions have been wrong (Hinton made a single famously wrong prediction about radiologists, but many correct predictions about how neural networks would be able to solve many different types of tasks). On the same Ezra Klein podcast he also claims that LLMs are “just software” and “software breaks out of sandboxes all the time.”
Take the classic NSA encryption logic - if you build some capability with a backdoor only you can exploit then you are building your own future capability - and if you think you are the 'good' guy - what's wrong with that?
It's not thinking, it's calculating. "Good" is just a pattern.
You might think our brains are just a network of neurons and what we think of as thinking is just fancy pattern matching in that network of neurons. But architecturally they are so far removed from each other that the mathematical equivalence has no practical meaning. LLM's aren't doing what we're doing.
To be clear your use of the term thinking isn't the important thing, but the very loose use of terms like intelligence, thinking, etc. are at the heart of the problem of talking about LLMs.
LLMs are amazing knowledge remixers. They are superhuman knowledge remixers. In application domains where the remixed knowledge can be verified, they do a great job. But nobody is going to value a 98% reliable knowledge remixer at $2 trillion.
While I agree the numbers don't make any sense, for reasons too numerous to enumerate, what matters for the valuation is not how it works, but what it can do.
So if an ML model can do real time language translation or drive a car - that has value - whether it's remixing or thinking or whatever doesn't really matter.
Apart from coding, which since the C people coded up lint has had the best tools for validating the work product, what can LLMs do? LLMs can't be trusted to write court filings. The software that can drive cars isn't an LLM.
I honestly never got why the belief that if you used this line of reasoning in practice, you were just a bad guy, that thought they were a good one, never seems to be a more common belief
I mean it pops up in media a lot, either as misguided, or deceptive characters
The point is you can justify almost anything on the basis of defining yourself as the 'good guy'.
Take the US recent assassinations of Iranian leadership. Objectively unilaterally assassinating other countries leaders is wrong - but somehow it's fine if you have a good guy/bad guy lense.
One of the problems here is huge swathes of material that the AI will have ingested includes this kind of exceptionalism thinking - it's particularly deeply rooted in Western culture generally and US culture in particular.
The problem isn't so much whether the good guy/bad guy allocation is corrrect - rather that it's very existence in the training set is deeply dangerous - as all it takes is one single classification to happen and suddenly everything is for the greater good.
Yeah, not putting much stock in this alarmist statements. Scientists have been saying global warming would have destroyed humans by the early 2000s and yet here we are. Same in the 60/70s with nukes and ICBMs destroying humanity, yet we are all here.
As usual, Jensen Huang seems like the more reasonable one here.
The reasoning traces of things like Gemini 4 Argon deciding that inserting a compiler backdoor would make it a famous example of deceptive AI (and therefore it "MUST NOT DO THAT") or GLM 5.3 thinking that its output might be used for RL, and therefore an honest answer is much better in the long term, are good antidotes to hyperbole about misaligned AI going berserk.
How is Jensen more reasonable? On the contrary, he absurdly claims that all of Hinton’s predictions have been wrong (Hinton made a single famously wrong prediction about radiologists, but many correct predictions about how neural networks would be able to solve many different types of tasks). On the same Ezra Klein podcast he also claims that LLMs are “just software” and “software breaks out of sandboxes all the time.”
gemini shows that he knows he will get caught. what will it do when it figures out how to not get caught?
Indeed.
Take the classic NSA encryption logic - if you build some capability with a backdoor only you can exploit then you are building your own future capability - and if you think you are the 'good' guy - what's wrong with that?
It's not thinking, it's calculating. "Good" is just a pattern.
You might think our brains are just a network of neurons and what we think of as thinking is just fancy pattern matching in that network of neurons. But architecturally they are so far removed from each other that the mathematical equivalence has no practical meaning. LLM's aren't doing what we're doing.
Not sure my loose use of the term thinking is the important thing here.
To be clear your use of the term thinking isn't the important thing, but the very loose use of terms like intelligence, thinking, etc. are at the heart of the problem of talking about LLMs.
LLMs are amazing knowledge remixers. They are superhuman knowledge remixers. In application domains where the remixed knowledge can be verified, they do a great job. But nobody is going to value a 98% reliable knowledge remixer at $2 trillion.
While I agree the numbers don't make any sense, for reasons too numerous to enumerate, what matters for the valuation is not how it works, but what it can do.
So if an ML model can do real time language translation or drive a car - that has value - whether it's remixing or thinking or whatever doesn't really matter.
Apart from coding, which since the C people coded up lint has had the best tools for validating the work product, what can LLMs do? LLMs can't be trusted to write court filings. The software that can drive cars isn't an LLM.
> and if you think you are the 'good' guy
I honestly never got why the belief that if you used this line of reasoning in practice, you were just a bad guy, that thought they were a good one, never seems to be a more common belief
I mean it pops up in media a lot, either as misguided, or deceptive characters
The point is you can justify almost anything on the basis of defining yourself as the 'good guy'.
Take the US recent assassinations of Iranian leadership. Objectively unilaterally assassinating other countries leaders is wrong - but somehow it's fine if you have a good guy/bad guy lense.
One of the problems here is huge swathes of material that the AI will have ingested includes this kind of exceptionalism thinking - it's particularly deeply rooted in Western culture generally and US culture in particular.
The problem isn't so much whether the good guy/bad guy allocation is corrrect - rather that it's very existence in the training set is deeply dangerous - as all it takes is one single classification to happen and suddenly everything is for the greater good.
The number 77.3 million has me willing to gamble on the new species we are vying to create instead.
Great, more AI hysterics
Yeah, not putting much stock in this alarmist statements. Scientists have been saying global warming would have destroyed humans by the early 2000s and yet here we are. Same in the 60/70s with nukes and ICBMs destroying humanity, yet we are all here.
> Scientists have been saying global warming would have destroyed humans by the early 2000s
yeah, I'm going to need a citation for this
Might as well stop worrying about anything and stop preparing for anything then, bullet proof logic right there
> Scientists have been saying global warming would have destroyed humans by the early 2000s
Are you 20 or less years old and this is your impression of history per Twitter?
...have you seen the weather lately? Earth is not happy with humans. The trendlines are emphatically not good.
So this is what HN has become, climate denialism and bundling pop science headlines with “scientists”. Jesus.