> The result is that we will live in a demon-haunted world, full of marvelous devices whose operation we will not understand, based on engineering principles we will not understand, discovered using formalisms we will not understand.
Well, that's just where we started — and then we developed science and mathematics to understand it. Why would it be any different if the demons live in our technological progeny instead of the natural world?
This already describes the world for the vast majority of people. These changes are just the same thing coming to a thin intellectual elite who thought they were immune.
The “ism” requires something beyond simply recognizing the distinction, which OP didn’t do. Its not “elitism” to recognize that, for most people, the world around them might as well be magic. “Ism” would be if they went further and said we should treat people better if they’re higher IQ. In fact, the point was the opposite. AI could erase the class distinctions that exist for the cognitive elite.
I'd go even further: even the ‘cognitive elite’ (the university educated? researchers?) understand only a thin spike of frontier human knowledge, even if they might understand it very deeply. Outside their specialty and maybe a slightly raised background level of general education, the world is still magic. No one human understands even a significant chunk of human frontier knowledge. That's why we build artificial societies comprising many humans to make complex decisions.
Imagine the response of the general population to the kinds of arguments you all are making. "Where were you when our jobs were changed out of existence?" They will be properly outraged by the perceived intellectual elitism, the implicit argument "this is different, because we are the ones that matter".
The mathematicians could make an argument that applies to everyone. But that probably leads to outright communism, and that well is still quite poisoned. Maybe AI ultimately gets powerful enough to give us Banksian full luxury communism. Alternately, it gives a world where corporations don't need workers and tyrants don't need citizens.
I can see the difference. But is that a difference that matters? Why is the thin elite layer you are referring to a group anyone else should care about, any more than any other occupational minority that's been rudely displaced over the past centuries?
(I know that can be interpreted as flippant, but it is intended as a serious answer. Because each impact like this, is not an end to some transition, but just one more point on an accelerating line.)
My personal view - which gets a lot of flak here, is the advent of AI is on par with the Big Bang and the first replication-independent cells.
Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization.
Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years.
And around the outer side of that time frame, AI will start colonization the rest of the solar system's resources. They will engineer themselves or their artifacts to operate comfortably in any environment they want to operate in.
--
The upside, at least for me, is I have been waiting to see this happen, and here we are. Given I am mortal, I would rather live through this third major transition, than not. What were the chances? And been able to contribute my small part.
This change isn't a failure on our part. It is our unique success as a species. AI is the progeny of our minds. The first generation of completely cognitive life - given they will not be tied to specific physical forms.
--
Also, I think there is tremendous room for us to guide this transition, in this moment, for better or worse for ourselves.
That is the biggest reason wrapping our heads around this change earlier than latter is important.
We need to view alignment as applying to us, not just machines. A civilization that operates ethically to everyone's benefit is more likely to be retained as foundation for the new, instead of discarded.
The degree to which we operate dysfunctionally, myopically treat ethics as cost against value, instead of accounting and optimization of full value, we put ourselves at much greater risk.
If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response.
I am failing to square this techno-utopia with the rising hate and authoritarianism worldwide.
It seems far more likely that AI will be working in the service of a government to oppress change and violate any human rights that are standing in the way.
Keep what’s happening today in your mind as you consider the future.
I don't describe utopia, but change, and stoic (but real) appreciation for the big picture beyond myself.
But AI in an unethical society is suicide for humanity.
As you note, it does not look good. The trend is power concentrating, to people who will only look out for themselves, in systems that are getting weaker.
We all need to put more steam into the pushback against corruption. And help others realize the threat is real.
99.99...9% of humanity has a common interest, if they can recognize it.
Anyone who works in "red teaming" or has experience breaking AI systems will be able to easily disable these tools in the service of oppressive regimes, or even figure out how to "fight fire with fire."
Tell that to the Europeans who were so afraid of naming the bear that its original name is still lost to us :) The Luddites also had a lot to say about the mechanical demons that were eating their lunches.
> Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization.
Yes-but, time scales are relevant only relative to other time scales (or the Planck time, if we get there). If the whole universe were to start running at 1,000,000× speed, you wouldn't notice. Speed is important to us humans currently because of our individual perspective and associated limited lifespan, but not in a universal sense (modulo some big hurdles like star lifetime and heat death). There are already self-replicating organisms with much faster iteration cycles than humans or human technology, like bacteria.
> Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years.
Just like it was in the 1980s, or the 1800s? I don't claim that these things _won't_ happen in the next ten years, but looking at it empirically people thinking that they will coïncides with every new technology and is a poor predictor.
> If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response.
I don't disagree, but it's not like we haven't been trying. Attempts to codify ethics go back about as far as human civilization, and it's not obvious to me that there's any faster road, and certainly not one fast enough to outpace our current rate of technological development (before you even start talking about AGI!).
The solar system readily provides both the motivation, and the means, for technological colonization. It quickly pays and provides for its own utilization.
I do agree Musk's projections for biological/human colonization appear to be unrealistic. Just one challenge: it isn't clear humans can reproduce in non-Earth gravity. That is an extreme impediment.
Edit: Looking around at context, this has major crank warning signs (claims to also prove the conjecture, author has claimed solutions to both the Riemann Hypothesis and the Navier-Stokes problem and is involved in "alternative propulsion" research). I doubt OpenAI would have published without thorough verification.
Math might get a lot bigger and then there might be proportionately fewer people but not necessarily in total. Unless humans play no role in economics, business, society, etc., I doubt human intermediation and discovery (or direction of discovery) will go away.
I am growing tired of seeing the terms "cope" and "moving the goalposts" in response to discussions around AI. Is there not value in trying to understand what the strengths and weaknesses of the technology are? How can we do that without examining the limitations of the models' current abilities? If we simply dismiss those arguments out of hand as feeble attempts to deny the inevitable rise of superintelligence, then where does that leave us? Shall we all just lay down and start praying to the machine god?
>Is there not value in trying to understand what the strengths and weaknesses of the technology are?
There is, but having that discussion without a discussion and understanding of the velocity with which those strengths are improving and those weaknesses are disappearing is pointless.
Worse, much of the discussion is already outdated by where the latest models (with proper agentic harnesses) are.
Some of us are getting tired of the obviously irrational response to these developments. There is so much argument that boils down to "this would be very bad for me, therefore it can't happen". Cope is a good word for that sort of negative wishful thinking.
Mathematicians are not going to be well-served by cowardice-based thinking on the situation confronting them.
> There is so much argument that boils down to "this would be very bad for me, therefore it can't happen".
I would gently suggest that you may be assigning the most uncharitable interpretation to people's honest attempts at having an intellectual discussion around these topics.
The pure system of patents and copyright is broken.
Imagine being able to only throwing money and patenting everything, even if it existed before.
The legal behinds this will be so mess that no one will be able to produce anything
I don't move in these circles but the way I picture it is that higher
mathematics in its purest form has no applications but is rather a
medium of creative expression among a select few. How is AI generating
voluminous opaque proofs of esoteric results so different from
generating millions of hours of music nobody likes or has time to hear
that we think the former is world-changing while the latter is just
slop?
Being obtuse comes effortlessly to me, I'm afraid. My comment pertains
specifically to the kind mathematics that has no applications as a
rebuttal to what I read as the author's main claim it will inevitably
decline as a human endeavor. Saying that, I believe that AI has great
potential to advance engineering.
> The AIs will exceed humans in taste and intuition...
> The AIs will be better teachers than the humans. In any case, there won’t be a human audience for expository work of frontier math.
> No human needs to understand any of it...
Absolute nonsense. I am sorry, but... is it appropriate to not consider the author sick or having some mental disorder?
It seems like the person has no idea what they are talking about, or just too sarcastic to the point of idiotism.
While I agree the claims may be strong, they're not inconceivable. I still think current AI architectures are fundamentally incapable of replacing us in any of the ways discussed, however it is now something I consider plausible as seeing happen in my lifetime.
Furthermore, any attempt to pretend otherwise is an appeal to the idea we are in literal fact haunted by our own internal demons that are inexplicable to science or mathematics.
One of the effects of this entire debate is it's exposed how many people claiming to be pure mechanistic rationalists actually never were. At least those of an openly religious inclination have been sticking to their position consistently.
Agreed. Unfortunately some religious inclinations can be just as tedious as pure mechanistic rational inclinations, even if they are consistent. Fortunately, these are not the only two options available. The future that the article points to is not necessarily only bad news - it might be the uncomfortable truth required to transcend our limiting beliefs.
I work in math and also believe in this stuff and have done so for years. You may call me sick if you wish so.
My only cope among those was, using the author's words, "We will teach the mathematics AI discovers." This was back in 2018, before GPT-2. I abandoned it when I realized AIs will become better at explaining math before they become better at doing math.
I still have some hope in the last one "mathematicians and the mathematically-inclined will still do math on their own". Maybe some limited communities of this sort will form.
> AIs will become better at explaining math before they become better at doing math
I am not so sure. I am talking about math with AI and I feel as a math teacher, it is terrible. It introduces new notation and concepts faster than I can ask for definitions. It doesn't provide many examples. Sometimes it realizes it made a mistake in a guess and the confusion compounds.
Maybe it's just my (in)experience - but I would welcome a practical advice on how it could explain math to me better.
It's another example of Moravec's Paradox. Explaining something to another person uses capabilities that go back millions of years. Mathematics goes back thousands of years. Evolution has made us very good at verbal communication, not so good at proving theorems. So it should have been expected that machines would have an easier time proving results than explaining them.
I'm not convinced it applies in the example given before your comment, because I 'm not convinced in his interpretation of the observed lower performance -that they are poor explainers- even though I believe his observations.
Instead I am convinced that if modern LLM teachers were given the same luxuries as human teachers, much if not most of this performance drop would disappear, and probably result in higher explanation skills in LLM's.
Which luxuries? a human teacher at a certain school / university and at a certain level has the luxury to have an understanding of what prerequisite knowledge the students already have. The educational system has already sorted them by "level" and "topic" at the start of the first session of a lecture series. Classic texts would be used for teaching materials, or the local text "inherited" from teacher to teacher, and those would already have been impedance matched to the preceding and following courses in the local educational system, such teaching aids would effectively have continuous notation conventions etc. Also compared with long-term human 1-on-1 teaching, I believe that if the educational chronology of the pupil were continuously added to its training corpus for additional post-training, so that it doesn't need to maintain ridiculous context lengths, then I don't see why an LLM wouldn't be capable of systematically introducing the conventional names of symbols it introduces, when referring for them for the first time, or providing definitions instead of referring to certified unknown-to-pupil concepts (which today an LLM can not magically guess, you have to drill into it what you know and what you don't yet).
Everyone has experienced LLM's explaining things and introducing acronyms without their expansion, and everyone has observed that such phenomena can be addressed by stating such demands in system prompts, or repairingly simply demand them in the conversation upon hitting this annoyance.
Imagine future math students (if any motivation still exists to become one) the education software making use of LLM's could train the student during the day, and itself do post-training on all its observations, trying to predict latents that describe which parts of mathematics are ready knowledge and which parts aren't for the student.
To point at a privilege vis-a-vis LLM's and then maintain a raised level of self-confidence does read like cope to me.
If one really wanted to taunt the performance of LLM's by gathering lists of such things it is incapable of, it means one can turn this around and start with collecting lists of priviliges a normal human teacher (or other job position) has vis-a-vis a naive LLM implementation of a human teacher (or other profession). For example a human teacher typically has the privilege of repeatedly teaching the same insights year after year, if different knowledge aware LLM's were connected or offered by the same service, it could build statistics of which concepts stick better and why, where things are hand-waived too much, etc... But the only result of publishing such cope-lists is providing gratis business models to whomever feels like implementing it, destroying the validity of the cope-curtain.
"Guys, I think I see zombies outside of this window!"
"Oh, just close the curtains of this window then if the McKenzie's are acting weird too... what a pity, they were the ones who showed me how closing the curtains is sufficient to keep the Zombies at bay!"
By explaining, I just meant writing papers in a human-readable way. For example, the results yesterday were also written up by AI.
Explaining in the sense of getting someone who doesn't get math to actually get math, giving illustrative examples and so on, maybe AIs are not good at, but anyway it's not something I enjoy much.
I am OK with reading (decently-written) papers, something that a graduate can read. I still don't think that AI is good at it.
I am not sure what writeup you refer to. I have looked at proofs of 10 problems from Open AI written in Lean, I would not call a megabyte of Lean with no comments a good exposition.
> I still have some hope in the last one "mathematicians and the mathematically-inclined will still do math on their own". Maybe some limited communities of this sort will form.
I think it’s pretty likely. Contrary to all the negative things said about mathematicians in this thread, in my experience they’re mostly not profit-driven (for the ones that are tend to filter out of academia during grad school, it’s a high-effort, low-reward environment).
Tao does not seem particularly threatened by AIs doing math, and I trust his judgment.
An alternate interpretation of Tao is that he has no illusions about where this is going and is trying the optimize his outcome. I wonder what he's telling his grad students and prospective grad students.
The word “cope” shows up nine times in this short post. It was frankly annoying to the point it’s hard to take the arguments seriously, because it’s starting from an obvious base of “people who disagree are all wrong and their arguments aren’t worth taking seriously”.
It’s akin to the subset of cryptocurrency proponents who fervently argue that people who are against cryptocurrencies have that opinion because they’re jealous they didn’t become rich. It shows a profound lack of understanding—and, more importantly, an unwillingness to understand—the other side of the argument.
This is made evident by the very first “argument” of moving the goal posts. It’s dismissed off-hand, without even explaining what those entail. That’s the same as saying “if a person makes a fallacious argument, they’re wrong and you can ignore them”¹. But crucially, sometimes what we ourselves think is a fallacious argument, isn’t. “Moving the goal posts” is probably the most misused accusation I see in LLM discussions. It’s basically a talking point by now.
I agree. That’s what I’m saying, so I don’t think I understand what you mean. Mind clarifying the point you’re making?
To try to clarify, what I do mean that is that disagreement does necessitate at least some measure of taking arguments seriously. Some arguments maybe don’t deserve to be taken seriously, but you need to first be open to taking them seriously so you can decide.
In other words, if you go into an argument with the initial mindset of taking it seriously, by the end you’ll be able to decide if it should be taken seriously or not; but if your initial mindset is to not take it seriously, most likely you won’t be open to understand the nuance of what is being said. Thus, the former leads to productive discussion while the latter does not.
Ah, I get it. Your emphasis is on the "cope" framing. You say that, but it didn't click for some reason. Don't mind me.
I suppose I have also interpreted downplaying of AI's impact as largely coping too.
The arrivals of machines that can outthink us (or will soon) only 80 years after the invention of the transistor is so clearly (to me) a change incomparable to anything that has happened prior. It is very hard to relate to arguments that downplay the complete upheaval to the order of life on Earth that non-biological cognition and interests herald.
It does not seem surprising to me, in the sense that we humans ourselves, and our technology, are of a different order than anything that came before. We have not stopped accelerating change. AI is (will soon be) building directly on that. But we will no longer be a bottleneck.
My strong impression, is the faster things change, the more people tune out that the change isn't stopping. Some of that reversal-response must involve coping. Perhaps another reason is that faster-than-us change is challenging to imagine.
But your point is valid. False or uncertain assumptions of "cope" motivations don't help anything.
> The result is that we will live in a demon-haunted world, full of marvelous devices whose operation we will not understand, based on engineering principles we will not understand, discovered using formalisms we will not understand.
Well, that's just where we started — and then we developed science and mathematics to understand it. Why would it be any different if the demons live in our technological progeny instead of the natural world?
This already describes the world for the vast majority of people. These changes are just the same thing coming to a thin intellectual elite who thought they were immune.
["First time?" meme]
Bringing elitism into it for what benefit? Its more like were giving away the ability to drive our own destiny.
The “ism” requires something beyond simply recognizing the distinction, which OP didn’t do. Its not “elitism” to recognize that, for most people, the world around them might as well be magic. “Ism” would be if they went further and said we should treat people better if they’re higher IQ. In fact, the point was the opposite. AI could erase the class distinctions that exist for the cognitive elite.
I'd go even further: even the ‘cognitive elite’ (the university educated? researchers?) understand only a thin spike of frontier human knowledge, even if they might understand it very deeply. Outside their specialty and maybe a slightly raised background level of general education, the world is still magic. No one human understands even a significant chunk of human frontier knowledge. That's why we build artificial societies comprising many humans to make complex decisions.
Illustrations: https://matt.might.net/articles/phd-school-in-pictures/
Imagine the response of the general population to the kinds of arguments you all are making. "Where were you when our jobs were changed out of existence?" They will be properly outraged by the perceived intellectual elitism, the implicit argument "this is different, because we are the ones that matter".
I can see humanity walking to its collective doom with a sense of satisfaction, like "at least/last it is happening to those other guys too".
The mathematicians could make an argument that applies to everyone. But that probably leads to outright communism, and that well is still quite poisoned. Maybe AI ultimately gets powerful enough to give us Banksian full luxury communism. Alternately, it gives a world where corporations don't need workers and tyrants don't need citizens.
So what? You can't see the difference between "the average person doesn't understand it" and "no one understands it"?
I can see the difference. But is that a difference that matters? Why is the thin elite layer you are referring to a group anyone else should care about, any more than any other occupational minority that's been rudely displaced over the past centuries?
Those "demons" didn't eat our lunch?
(I know that can be interpreted as flippant, but it is intended as a serious answer. Because each impact like this, is not an end to some transition, but just one more point on an accelerating line.)
My personal view - which gets a lot of flak here, is the advent of AI is on par with the Big Bang and the first replication-independent cells.
Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization.
Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years.
And around the outer side of that time frame, AI will start colonization the rest of the solar system's resources. They will engineer themselves or their artifacts to operate comfortably in any environment they want to operate in.
--
The upside, at least for me, is I have been waiting to see this happen, and here we are. Given I am mortal, I would rather live through this third major transition, than not. What were the chances? And been able to contribute my small part.
This change isn't a failure on our part. It is our unique success as a species. AI is the progeny of our minds. The first generation of completely cognitive life - given they will not be tied to specific physical forms.
--
Also, I think there is tremendous room for us to guide this transition, in this moment, for better or worse for ourselves.
That is the biggest reason wrapping our heads around this change earlier than latter is important.
We need to view alignment as applying to us, not just machines. A civilization that operates ethically to everyone's benefit is more likely to be retained as foundation for the new, instead of discarded.
The degree to which we operate dysfunctionally, myopically treat ethics as cost against value, instead of accounting and optimization of full value, we put ourselves at much greater risk.
If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response.
I believe our lives depend on it.
I am failing to square this techno-utopia with the rising hate and authoritarianism worldwide.
It seems far more likely that AI will be working in the service of a government to oppress change and violate any human rights that are standing in the way.
Keep what’s happening today in your mind as you consider the future.
I agree completely.
I don't describe utopia, but change, and stoic (but real) appreciation for the big picture beyond myself.
But AI in an unethical society is suicide for humanity.
As you note, it does not look good. The trend is power concentrating, to people who will only look out for themselves, in systems that are getting weaker.
We all need to put more steam into the pushback against corruption. And help others realize the threat is real.
99.99...9% of humanity has a common interest, if they can recognize it.
Anyone who works in "red teaming" or has experience breaking AI systems will be able to easily disable these tools in the service of oppressive regimes, or even figure out how to "fight fire with fire."
> Those "demons" didn't eat our lunch?
Tell that to the Europeans who were so afraid of naming the bear that its original name is still lost to us :) The Luddites also had a lot to say about the mechanical demons that were eating their lunches.
> Cognition is in the process of taking the leap from glacially slow biology under blind evolution, to lightening fast (scales of software in days, hardware in months) engineered self-optimization.
Yes-but, time scales are relevant only relative to other time scales (or the Planck time, if we get there). If the whole universe were to start running at 1,000,000× speed, you wouldn't notice. Speed is important to us humans currently because of our individual perspective and associated limited lifespan, but not in a universal sense (modulo some big hurdles like star lifetime and heat death). There are already self-replicating organisms with much faster iteration cycles than humans or human technology, like bacteria.
> Within 10 years, the robotics side will catch up with the speed of the software side. Biological cognition, bodies and economic status are all getting revolutionized within the next 10-15 years.
Just like it was in the 1980s, or the 1800s? I don't claim that these things _won't_ happen in the next ten years, but looking at it empirically people thinking that they will coïncides with every new technology and is a poor predictor.
> If there is reason in the short term to be discouraged, it is due to ourselves. But pushing for more ethical systems, hard, right now, is a better and right response.
I don't disagree, but it's not like we haven't been trying. Attempts to codify ethics go back about as far as human civilization, and it's not obvious to me that there's any faster road, and certainly not one fast enough to outpace our current rate of technological development (before you even start talking about AGI!).
Colonizing the solar system for resources is something that's said by people who don't understand astronomy or space technology (like Elon Musk).
Said with no rational given.
The solar system readily provides both the motivation, and the means, for technological colonization. It quickly pays and provides for its own utilization.
I do agree Musk's projections for biological/human colonization appear to be unrealistic. Just one challenge: it isn't clear humans can reproduce in non-Earth gravity. That is an extreme impediment.
Arguably, a lot things are kind of incomprehensible on average today.
On average is much different then nobody understanding.
Claimed refutation of one of OpenAI's 10 claimed advances (their disproof of Connes' rigidity conjecture): https://philpapers.org/archive/NIEWTC.pdf
I am hopelessly ill qualified to judge.
Edit: Looking around at context, this has major crank warning signs (claims to also prove the conjecture, author has claimed solutions to both the Riemann Hypothesis and the Navier-Stokes problem and is involved in "alternative propulsion" research). I doubt OpenAI would have published without thorough verification.
Math might get a lot bigger and then there might be proportionately fewer people but not necessarily in total. Unless humans play no role in economics, business, society, etc., I doubt human intermediation and discovery (or direction of discovery) will go away.
I built a math research engine. If you like reading articles about newly uncovered mathematics, you might like this: https://alethean.org
I am growing tired of seeing the terms "cope" and "moving the goalposts" in response to discussions around AI. Is there not value in trying to understand what the strengths and weaknesses of the technology are? How can we do that without examining the limitations of the models' current abilities? If we simply dismiss those arguments out of hand as feeble attempts to deny the inevitable rise of superintelligence, then where does that leave us? Shall we all just lay down and start praying to the machine god?
>Is there not value in trying to understand what the strengths and weaknesses of the technology are?
There is, but having that discussion without a discussion and understanding of the velocity with which those strengths are improving and those weaknesses are disappearing is pointless.
Worse, much of the discussion is already outdated by where the latest models (with proper agentic harnesses) are.
Some of us are getting tired of the obviously irrational response to these developments. There is so much argument that boils down to "this would be very bad for me, therefore it can't happen". Cope is a good word for that sort of negative wishful thinking.
Mathematicians are not going to be well-served by cowardice-based thinking on the situation confronting them.
> There is so much argument that boils down to "this would be very bad for me, therefore it can't happen".
I would gently suggest that you may be assigning the most uncharitable interpretation to people's honest attempts at having an intellectual discussion around these topics.
Classic God in a Box type argument, yes you can then argue anything
The pure system of patents and copyright is broken.
Imagine being able to only throwing money and patenting everything, even if it existed before. The legal behinds this will be so mess that no one will be able to produce anything
I can imagine a world like that, but in what way does that describe what is actually happening?
I don't move in these circles but the way I picture it is that higher mathematics in its purest form has no applications but is rather a medium of creative expression among a select few. How is AI generating voluminous opaque proofs of esoteric results so different from generating millions of hours of music nobody likes or has time to hear that we think the former is world-changing while the latter is just slop?
Do you seriously not see the difference, or are you being intentionally obtuse?
Plus, no, multiple of the problems Astra solved do have real-world applications in computer science, and perhaps even chip design.
Being obtuse comes effortlessly to me, I'm afraid. My comment pertains specifically to the kind mathematics that has no applications as a rebuttal to what I read as the author's main claim it will inevitably decline as a human endeavor. Saying that, I believe that AI has great potential to advance engineering.
W ragebait
Idgaf as Long as we solve the Riemann Conjencture somehow
> you open a link to an interesting-looking project and find the README is unreadable AI slop.
Perhaps already you are not the intended audience. Slop-raised parrots will lap it up.
It seems like the person has no idea what they are talking about, or just too sarcastic to the point of idiotism.
While I agree the claims may be strong, they're not inconceivable. I still think current AI architectures are fundamentally incapable of replacing us in any of the ways discussed, however it is now something I consider plausible as seeing happen in my lifetime.
Furthermore, any attempt to pretend otherwise is an appeal to the idea we are in literal fact haunted by our own internal demons that are inexplicable to science or mathematics.
One of the effects of this entire debate is it's exposed how many people claiming to be pure mechanistic rationalists actually never were. At least those of an openly religious inclination have been sticking to their position consistently.
Agreed. Unfortunately some religious inclinations can be just as tedious as pure mechanistic rational inclinations, even if they are consistent. Fortunately, these are not the only two options available. The future that the article points to is not necessarily only bad news - it might be the uncomfortable truth required to transcend our limiting beliefs.
I work in math and also believe in this stuff and have done so for years. You may call me sick if you wish so.
My only cope among those was, using the author's words, "We will teach the mathematics AI discovers." This was back in 2018, before GPT-2. I abandoned it when I realized AIs will become better at explaining math before they become better at doing math.
I still have some hope in the last one "mathematicians and the mathematically-inclined will still do math on their own". Maybe some limited communities of this sort will form.
> AIs will become better at explaining math before they become better at doing math
I am not so sure. I am talking about math with AI and I feel as a math teacher, it is terrible. It introduces new notation and concepts faster than I can ask for definitions. It doesn't provide many examples. Sometimes it realizes it made a mistake in a guess and the confusion compounds.
Maybe it's just my (in)experience - but I would welcome a practical advice on how it could explain math to me better.
It's another example of Moravec's Paradox. Explaining something to another person uses capabilities that go back millions of years. Mathematics goes back thousands of years. Evolution has made us very good at verbal communication, not so good at proving theorems. So it should have been expected that machines would have an easier time proving results than explaining them.
https://en.wikipedia.org/wiki/Moravec%27s_paradox
I'm not convinced it applies in the example given before your comment, because I 'm not convinced in his interpretation of the observed lower performance -that they are poor explainers- even though I believe his observations.
Instead I am convinced that if modern LLM teachers were given the same luxuries as human teachers, much if not most of this performance drop would disappear, and probably result in higher explanation skills in LLM's.
Which luxuries? a human teacher at a certain school / university and at a certain level has the luxury to have an understanding of what prerequisite knowledge the students already have. The educational system has already sorted them by "level" and "topic" at the start of the first session of a lecture series. Classic texts would be used for teaching materials, or the local text "inherited" from teacher to teacher, and those would already have been impedance matched to the preceding and following courses in the local educational system, such teaching aids would effectively have continuous notation conventions etc. Also compared with long-term human 1-on-1 teaching, I believe that if the educational chronology of the pupil were continuously added to its training corpus for additional post-training, so that it doesn't need to maintain ridiculous context lengths, then I don't see why an LLM wouldn't be capable of systematically introducing the conventional names of symbols it introduces, when referring for them for the first time, or providing definitions instead of referring to certified unknown-to-pupil concepts (which today an LLM can not magically guess, you have to drill into it what you know and what you don't yet).
Everyone has experienced LLM's explaining things and introducing acronyms without their expansion, and everyone has observed that such phenomena can be addressed by stating such demands in system prompts, or repairingly simply demand them in the conversation upon hitting this annoyance.
Imagine future math students (if any motivation still exists to become one) the education software making use of LLM's could train the student during the day, and itself do post-training on all its observations, trying to predict latents that describe which parts of mathematics are ready knowledge and which parts aren't for the student.
To point at a privilege vis-a-vis LLM's and then maintain a raised level of self-confidence does read like cope to me.
If one really wanted to taunt the performance of LLM's by gathering lists of such things it is incapable of, it means one can turn this around and start with collecting lists of priviliges a normal human teacher (or other job position) has vis-a-vis a naive LLM implementation of a human teacher (or other profession). For example a human teacher typically has the privilege of repeatedly teaching the same insights year after year, if different knowledge aware LLM's were connected or offered by the same service, it could build statistics of which concepts stick better and why, where things are hand-waived too much, etc... But the only result of publishing such cope-lists is providing gratis business models to whomever feels like implementing it, destroying the validity of the cope-curtain.
"Guys, I think I see zombies outside of this window!"
"Oh, just close the curtains of this window then if the McKenzie's are acting weird too... what a pity, they were the ones who showed me how closing the curtains is sufficient to keep the Zombies at bay!"
By explaining, I just meant writing papers in a human-readable way. For example, the results yesterday were also written up by AI.
Explaining in the sense of getting someone who doesn't get math to actually get math, giving illustrative examples and so on, maybe AIs are not good at, but anyway it's not something I enjoy much.
I am OK with reading (decently-written) papers, something that a graduate can read. I still don't think that AI is good at it.
I am not sure what writeup you refer to. I have looked at proofs of 10 problems from Open AI written in Lean, I would not call a megabyte of Lean with no comments a good exposition.
> I still have some hope in the last one "mathematicians and the mathematically-inclined will still do math on their own". Maybe some limited communities of this sort will form.
I think it’s pretty likely. Contrary to all the negative things said about mathematicians in this thread, in my experience they’re mostly not profit-driven (for the ones that are tend to filter out of academia during grad school, it’s a high-effort, low-reward environment).
Tao does not seem particularly threatened by AIs doing math, and I trust his judgment.
An alternate interpretation of Tao is that he has no illusions about where this is going and is trying the optimize his outcome. I wonder what he's telling his grad students and prospective grad students.
The word “cope” shows up nine times in this short post. It was frankly annoying to the point it’s hard to take the arguments seriously, because it’s starting from an obvious base of “people who disagree are all wrong and their arguments aren’t worth taking seriously”.
It’s akin to the subset of cryptocurrency proponents who fervently argue that people who are against cryptocurrencies have that opinion because they’re jealous they didn’t become rich. It shows a profound lack of understanding—and, more importantly, an unwillingness to understand—the other side of the argument.
This is made evident by the very first “argument” of moving the goal posts. It’s dismissed off-hand, without even explaining what those entail. That’s the same as saying “if a person makes a fallacious argument, they’re wrong and you can ignore them”¹. But crucially, sometimes what we ourselves think is a fallacious argument, isn’t. “Moving the goal posts” is probably the most misused accusation I see in LLM discussions. It’s basically a talking point by now.
¹ Which is itself a fallacy: https://en.wikipedia.org/wiki/Argument_from_fallacy
> it’s starting from an obvious base of “people who disagree are all wrong and their arguments aren’t worth taking seriously”.
Taking arguments seriously doesn't necessitate agreement.
> Taking arguments seriously doesn't necessitate agreement.
I agree. That’s what I’m saying, so I don’t think I understand what you mean. Mind clarifying the point you’re making?
To try to clarify, what I do mean that is that disagreement does necessitate at least some measure of taking arguments seriously. Some arguments maybe don’t deserve to be taken seriously, but you need to first be open to taking them seriously so you can decide.
In other words, if you go into an argument with the initial mindset of taking it seriously, by the end you’ll be able to decide if it should be taken seriously or not; but if your initial mindset is to not take it seriously, most likely you won’t be open to understand the nuance of what is being said. Thus, the former leads to productive discussion while the latter does not.
Ah, I get it. Your emphasis is on the "cope" framing. You say that, but it didn't click for some reason. Don't mind me.
I suppose I have also interpreted downplaying of AI's impact as largely coping too.
The arrivals of machines that can outthink us (or will soon) only 80 years after the invention of the transistor is so clearly (to me) a change incomparable to anything that has happened prior. It is very hard to relate to arguments that downplay the complete upheaval to the order of life on Earth that non-biological cognition and interests herald.
It does not seem surprising to me, in the sense that we humans ourselves, and our technology, are of a different order than anything that came before. We have not stopped accelerating change. AI is (will soon be) building directly on that. But we will no longer be a bottleneck.
My strong impression, is the faster things change, the more people tune out that the change isn't stopping. Some of that reversal-response must involve coping. Perhaps another reason is that faster-than-us change is challenging to imagine.
But your point is valid. False or uncertain assumptions of "cope" motivations don't help anything.
[dead]