You tell them to cancel their Astra subscriptions and get to work. Hundreds of mathematicians, including Peter Scholze, have already agreed to not use AI at all. There’s absolutely no reason to help train an AI that will just make your own life more miserable.
A friend of a friend had a grant project written up for the next 10 years, with habilitation on the horizon.
His end goal was one of the things released in the recent OpenAI publications. It took a model 3.5 hours to do.
Things are wild now on math and theoretical faculties now.
And for theoretical fields labs don’t need human data - synthetic one works well if not better.
The best thing you can do if you’re a scientist, imho - wrapup whatever grant you have now asap with ai, use ai to get more grants if possible, and spend the rest od the year learning what new science you can do with the new tools and participate in creating the new paradigm in your field.
That’s when you’re an established scientist. If you’re new in the field then this is the most exciting time you could wish for - an opportunity to make a name for yourself.
> The best thing you can do if you’re a scientist, imho - wrapup whatever grant you have now asap with ai, use ai to get more grants if possible, and spend the rest od the year learning what new science you can do with the new tools and participate in creating the new paradigm in your field.
> That’s when you’re an established scientist. If you’re new in the field then this is the most exciting time you could wish for - an opportunity to make a name for yourself.
s/scientist/mathematician/g
There are plenty of fields of research where AI is not nearly as destructive as this, currently. Most experimental science is pretty safe for a while.
and that generation was educated without AI. going to be interesting how future generations of mathematicians will fare who have been using AI from day 1. I suspect they'll be simply much worse at math.
That generation will say "How on earth people could do this without AI", just like we have said "How on earth people could do this without the internet".
You got this right, these are incredibly exciting times. Take your friend of a friend's as an example. AI's are trained with existing information, so obviously whatever he expected to discover was already out there. It took the machine 3.5 hours to find and sort out something that would take a human 10 years. Now your F of a F can dedicate the next 10 years to do what the machine cannot do: find meaning and insight.
> AI's are trained with existing information, so obviously whatever he expected to discover was already out there.
I've wondered about that myself. Does the fact that LLM's were able to solve these problems so easily indicate that most of the solution already existed, scattered in the training data, and the LLM was simply able to put together the pieces?
I think it's an interesting conjecture but nothing more without proof.
To be fair I see two separate ways CS students use LLMs, one is to just complete assignments in the same way you could by asking someone else to do it, the other way is by getting an LLM to explain certain concepts in another way, or create more examples of them to aid in understanding quicker/in more depth. I actually think in the 2nd case they can be incredibly useful and can assist in creating a deeper understanding, but in the 1st case they obviously create fundamental gaps in understanding.
In a way it's on the student to make sure whatever they're doing, they actually understand it, or come exam season the gaps in their knowledge sans Claude will become quite obvious.
There have always been students that cheat themselves out of learning. It may even be the majority. It’s best to focus on the ones who have a genuine desire to learn.
While I see your point, these people do benefit greatly from great teachers! (They’d do well in either case but do even better in the right environment).
I think a bigger problem is in programs where you have group projects. You can't really be the sole person who's against using LLMs and in favour of doing 'the right thing'. Unlike what some people think there's little difference in terms of raw intelligence or ability to get rewarded by the system in the 'cheater' group vs the 'non-cheater' group.
You ever hear the state if you owe the bank a 1000$ you have a problem, and if you owe the bank 100,000,000$ they have a problem? I think if a majority of students end up without basic skills the genius students will have problem. Will LLMs do that i don't think so but if you think its an acceptable outcome thats also not okay.
You can't just handwave the fact that we're requiring what probably amounts to two orders of magnitude more discipline and self control out of students than we've ever done before. The educational system will collapse under this.
I'm not seeing how it requires any more self control than not copying another student's work or asking them for the answer. It's basically the same thing. We're also talking about adults here. They understand if they're cheating.
The difference is huge. Put yourself in the shoes of a student.
The student you're copying from won't be sitting next to you for the rest of your career. The LLM probably will be, and you'll be probably expected to use it, so why not use it here as well?
All your peers are doing it too, and look at all the cool things they're doing while you're struggling with the basics. And what if the LLMs keep improving at a high rate? What if being able to use them efficiently turns out to be a more important skill than what you're being taught anyways?
Because you're paying a lot of money and opportunity cost to learn how to do something. No one needs the assignment. It's there for you to learn. You were given the assignment for you. Any adult should be able to understand this. We're not talking about 7 year olds asking why they need to learn to multiply when a calculator can do it better than them. These are 20 year olds. If they don't have the maturity for this, a university should not be accepting them.
Being able to use LLMs efficiently isn't a specific skill. It's a reflection of your ability to articulate what you want, which is a reflection of your understanding of the world, which is what you're in school to build. Terence Tao can get an LLM to do math better than I can. I can probably get one to build software better than he can.
And you can still use one to do cool things like your peers. Just not your assignments, the purpose of which is literally to teach you the basics that you're struggling with and that you're there to learn in the first place.
This is like why don't you copy out of the back of the book or look up proofs on the internet. It's all there, but doing that completely misses the point of why you're there.
This only makes any sense if youre happy dropping most folks from higher education.
You want adults to be very disciplined and rational. That sounds great but are most of the adults in your own life experience at this level? They certainly aren't for me (I'm not sure I am at this level).
If I recall correctly, there was a mandatory ENGR 101 class that all of the different engineering fields had to take at my school, and on the first day, the lecturer said somewhere around half (might've been more, might've been fewer, I don't remember anymore) of the students in the hall were not going to complete the degree. So yeah, most folks dropping sounds about right.
And it's not even about discipline; it's lying. If you're not going to do the work, just don't do the work. Don't turn it in. Pretty much every syllabus ever says that turning in work you did not do is grounds for failing the class and possibly expulsion.
And why would we want to pass these people through? Then when they come interview with someone like me, I'm still going to find out within like a minute of talking to them that they didn't actually learn anything, and they won't get hired. Do I want to work with somebody incompetent? No. Do I want to work with somebody dishonest? Definitely no. Those both make my job more difficult to have around. So then they end up with a bunch of debt complaining about how they don't have a job.
> To not bury your head in the sand hoping this will go away, because it won't.
Eh? As Greg K-H mentioned in that video from a few days back about how very disappointing Fable was when compared to its astronomical hype, LLMs that one can run locally are quite good enough for a great many tasks... including bug hunting in the Linux kernel. And -as LLM boosters keep saying- they're only keep getting better, right?
We absolutely should be telling people to stop using the LLMs from the major LLM manufacturers. Those manufacturers have spent so many billions of dollars on this project, made so many promises that they're going to have no way to keep, and now that they're running into resistance, [0] they're holding all of humanity hostage unless they're permitted to get intimately involved in the creation of new laws and regulations especially for them. [2]
Even if one only considers their recent threats to humanity, it's clear that these are not companies that deserve any of our hard-earned money.
[0] Some of that resistance comes from their ever-more-sharply-increasing cost to produce the next performance increment, some from folks asking the probing questions about their promises, claims, and business practices that should have been asked years ago, some from ongoing State AG's court cases in regards to their illegal conduct, and some from folks who -unsurprisingly- don't want enormous warehouses that suck up quite notable amounts of power [1] but provide dreadfully little revenue to the areas that house them in their communities, and still others who are starting to realize that benefits of cloud LLMs aren't worth the tradeoff of being unable to afford a new personal computer.
[1] Note carefully that I made no reference to water usage. The only counterargument you can make here is that 500MW -> 2GW is not a quite notable amount of power.
I am afraid that ship has already sailed, and we will have to live with the consequences.
At this point; I see no other scenario than having LLM doctors, LLM judges, etc. No matter the quality. We will get replaced. And if they do a worse job, nobody cares. If they do a better job, also nobody cares. Maybe you can pay more and get high effort model as your doctor.
These companies might not be the ones, my expectation is that they will be scrapped for their pieces, the losses distributed to the bagholders/taxpayers. But some version of them will.
This might be pure cyncisism with no positive value, but really I think we have no mechanisms left to stop going down on this trajectory.
Anyone else want their doctor to have access to LLMs? They need to verify, but to just cut them off from something with that much breadth seems insane.
I would prefer my doctor to have access to LLMs, just as I would prefer them to have access to google.
What I meant is different though. I fully expect to no longer have access to human doctors in a decade or two. Instead, LLMs will be the doctors. Likely government accredited ones. And my point is that in time, whether the LLM doctor is better or worse will not matter. Same with judges, same with the surveillance camera evaluators and private message examiners. I am saying we no longer have any mechanisms to stop these things from happening.
> They need to verify, but to just cut them off from something with that much breadth seems insane.
...homeboy here is behaving as if there aren't experts in this world who are paid to do and publish research and other experts who are paid to collate, analyze, vet, and publish that research for other domain experts to learn from. [0]
I'm fine with my doctor reading from expert-vetted documents. Hell, I'm more than fine with my doctor going through expert-vetted checklists; most of the time what's wrong with you can be easily figured by going through a few good checklists. I'm not fine with my doctor relying on a lossy-compressed database with a absentminded librarian that has a penchant for people-pleasing bolted on top.
I'm absolutely not against the use of ML systems in safety-critical domains like medicine, but I am absolutely against the use of LLMs in those domains.
[0] One might choose to retort with some variant of "But all that expert work is so expensive!". I would retort: "First, without that work the data fed into the LLM is catastrophically unreliable. Second, have you bothered to look at how much the major LLM manufacturers have spent over the past five years or so? I suspect that it's more money than has been spent on medical research 'meta analysis' over the past fifty years."
> At this point; I see no other scenario than having LLM doctors, LLM judges, etc. No matter the quality. We will get replaced. And if they do a worse job, nobody cares. If they do a better job, also nobody cares. Maybe you can pay more and get high effort model as your doctor.
Why should any of us pay more for reduced performance just so some capitalist can feel good about replacing a human with a machine? You know, the traditional premise of automation in manufacturing was that the outputs might be 80% as good as hand-crafted but 20% the price. Make that 120% the price and the whole equation changes!
Yes, but a little bit of lobbying and regulatory capture goes a long way.
At first, you provide the services for 20% the price and price out the competition. Also, human doctors also make mistakes, no? Do this long enough so that the long and costly process of educating physicians is no longer sustainable anymore. Then you regulate hard, after all, it is healthcare. Not anyone can and should just do it.
Then, once you are the only option left, jack up prices. We have seen this exact scenario play out many many times. This is just another version of it. And since trillions have already been burned, you can be sure that there will be a lot of desperation to make it happen.
Students are there to study to get the permission to use the LLM. Maybe they can use it for querying information but not to do their job (which is thinking).
One natural step of AI usage, the way I see it, is that even mediocre researchers can use AI as a harness to become prolific researchers.
So if you're part of a pure "human only" researchers that publish 1/10th of what the rest are doing, how are you going to survive?
The more prolific researchers will eat your grants for lunch.
This is exactly the same thing a lot of software devs are going through now. AI models have lifted up EVERYONES ability to produce code, so there's simply no premium anymore for those that will only code by hand. And the people paying their salary are asking why they aren't more productive, when even business analyst Joe is pumping out new products left and right.
People will pay good money for artisanal, hand-crafted clothing, or watches, or cars, or whatever, but I doubt that the market for artisanal mathematics is now or will ever be very large.
It's just stubbornness, in the end, that is manifesting as a kind of gatekeeping Luddism. I see the same thing in my Mastodon feed from folks in the software engineering world, particularly people who have a great love for writing code and hold it as a core part of their identities. I understand being irritated at a machine having been trained on (potentially, inter alia) your work with no compensation to you, and now it is perhaps better at (some of) what you do than you are; or maybe companies are now making a lot of money off of something that benefited from your contributions; but I find it extremely improbable that we will be able to refuse our way out of further progress now that so much money has been invested and so much momentum has been gathered.
I don't think I feel any discomfort about a machine being better at anything that I can do than I am at it. Maybe it's because I'm a mediocre person. I personally think that if there's something that I'm better at than a machine, then we need to build a better machine. Imagine what I'd be able to do if I used that machine!
That's going to be a very short term solution, no?
You can't really swear off looking at problems solved by AI, or keep moving away from fields whenever AI starts contributing, and have a viable career.
Today's frontier models, are everybody's local models in the near future.
If this were a change that had expected stalls or reprieves it would be different.
Doing mathematics looks to be the next chess, er, I mean protein folding challenge. The latter's computation demands have dropped significantly.
You have many fields that will be protected from ai, or even enabled by it - if I could bet, sociology will be a hot new field that will by definition still need humans always, but will gain both speed and significance.
There are an infinite number of problems that require a unique proof (you can prove this btw). Yes maybe you need to work harder to find a more obscure problem now but why give up the fight already? I believe that students should have the right to intellectually stimulating education. If you really need to use AI, go for something that isn’t trying to put you out of business or destroy your right to a decent education, like a local open weights LLM.
> There are an infinite number of problems that require a unique proof
Yes. The problem isn't that there won't be any problems. The problem will be the rate of general progress and solutions that at least some awareness is required of, to reliably identify a good new unsolved problem will just keep getting more challenging. And then an attempt needs to be make, to solve it in a very short time.
There is always another race to run too, but increasingly slow runners don't find that translates to winning.
This change is not going to slow down, it is going to speed up. Machines will be doing math systematically, checking off the meta-math theorems that ensure axiom combinations are covered, that the search does not stop where theorems are not exhausted, and does stop where it can be proven they are. Humans will never operate at that level.
It’s more like avoiding looking at the answer key while doing homework to be able to pass an exam and to understand the material better, especially for the grad students who haven’t gotten their PhDs yet. Personally I think responsible AI use is possible (not all AHM members completely avoid AI, but it’s notable that some heavy hitters do), but it’s best not to use a closed model trying to replace you. Use an open model that won’t train on your data.
I think we should apply AI into human, so we could be better and then start doing more advanced stuff required for the Universe explorations. Current AI/LLMs can be considered as first step to that. Take and look about our history. We could never achieve so many if we stopped doing great things because of <any reason>.
Why do we ever use PCs, smartphones, Internet, electricity, medicine and so on? Why do we live? For what?
I don’t have a problem with using AI in the applied sciences once a researcher has proven their skills. I do have a problem with it damaging education, the pure math pipeline, and art.
I understand but I guess the future (far future or who knows) could be like that human just download SKILL and that's it - no more learning, study ing basic things. While basic things could be advanced math, computer science, biology and just everything (or nearly everything) what human do. I know it is science fiction right now but who knows. Smartphone + the Internet were step 0. AI/LLMs are next iteration. Still knowledge is valueable and so on but I guess eventually we get to the point which I said. Then galaxy exploration - if we still be here alive :)
I don't have a problem with AI to generate code..so long as the developer understands the output and would review his own mess rather than expecting those who know how to code will save the burning house after everyone's left in panic.
So others will write their PhD thesis faster, will publish more, you will not publish so you will perish. The rules of the game in this business are brutal. One can count on inventing something entirely new that AI will not be able to figure out, but this is a really high bet.
This post in the blog comments is quite compelling. Some observations that publishing of research is drying up because results can be easily retrieved at any time. Over the long-term, I wonder whether this will result in accumulation of knowledge grinding to a halt:
> With the latest ChatGPT models, these problems are more equivalent to homework questions: the answer is `in the back of the book.’ I am not discovering new solutions. Instead, I am working on problems whose answer exists and is simply waiting to be retrieved by a user of the model. In fact, I mentioned a problem that I was interested in working on to my advisor and he informed me that he and a collaborator had completely resolved it using ChatGPT – they have no plans to write up the result, so it will sit there until another `researcher’ pulls the proof slot machine.
There is another good point that the most tenured researchers have a sense of what problems are most worth exploring and therefore are more likely to feel excitement than younger researchers:
> I have also heard the contention that math research has `gotten more exciting,’ mainly from established researchers. They have decades of open problems that they care deeply about and want to see resolved. I have no such problems.
> On the other hand, none of the proofs so far seem to contain “alien ideas,” a move-37, or completely novel arguments or new concepts that were not present in the literature in some form or another.
I have the same optimism as Prof Tao. That said, I can understand why so many mathematicians have been so upset or stressed out. It turned out much of the mathematical work is about clever combination of existing methods - this already requires enormous amount human ingenuity and years of dedicated learning. Unfortunately, or maybe fortunately, AI can be very good at knowledge transfer and finding combination of existing ideas to solve seemingly impossible problems. Even though mathematicians are extremely smart and capable, only a small number of them are capable of truly inventing "alien ideas", discovering new ground-breaking mathematical structures, or coming up with new problem-solving techniques. That is, AI can eat many mathematicians' cake. That said, I'm still hopeful. Mathematicians still understand mathematics deeply. If someone can prompt AI to solve an important problem, that person is more likely a good mathematician than an average joe like me. So, I think mathematicians do have a bright future: leverage AI, and make more and bigger math discoveries. It's still the same north star: we must know, and we shall know. It's just that with AI, we will know sooner and more.
I was recently working on a project where I had to come up with a novel solution to a problem the company was facing. The problem has been solved before, but in suboptimal ways that wouldn’t work for this specific company. I came up with a pretty interesting solution, one that combines different statistical methods, and was a little more complicated, but gave better results.
I asked Claude what it thought about the technique. It hated it. It loathed the idea that I was not sticking with traditional methods that were pointed to in papers and pre-existing open source packages. Even after talking through the idea and how it was better and showing it the results, it was weirdly hesitant to admit it was a valid method, simply because it seemed nervous that the idea was novel.
I don’t have a real moral to the story, I just thought I would add my anecdote here.
I had a similar experience with not being able to convince Claude of a different approach to a problem. Then I shared the idea with Codex and it liked it. As another commenter said, you can have the same thing happen with humans.
Interesting! My experience is that Claude has a tendency to always suggest the true and tried, but likes off-center ideas when I present them. Fable especially has a "yes, and!" attitude where it understands and plusses the idea.
It's quite amusing that this mistake gets made on every single one of these guest posts when the first words on the page are "This is a guest post by Álvaro Lozano-Robledo" and the first paragraph starts with "I would like to give Terry my heartfelt thanks for giving me the opportunity to contribute a post to his blog." It's pretty clear that everyone is skimming these at best. Maybe he should put the disclaimer in a bigger font or something.
Though it’s very funny to imagine Tao suddenly acquiring a massive ego:
> At any given time in the history of mathematics, there have been mathematicians … whose mental capacity for mathematics seems completely super human (e.g., the owner of this blog, among many others).
It's interesting, what would be Tao's views on the creativity question. Like, I don't find the empirical observation of LLMs now, the notion these proofs are mostly "just" sophisticated grinding, to be a strong indicator of what could/couldn't happen.
It's pretty much the same story with software development. There are two camps and the bigger camp got hit hard. Nowadays I feel entitled to use the "software engineer" title. Essentially, all the decisions-making and nuanced trade-offs which AI is not capable of making; that's engineering. AI can only do programming and code analysis... At least so far.
That said, in the past couple of month, AI has gained a lot of ground on the 'project management' front in terms of planning capability for a lot of simple to medium-complexity projects. It still needs a lot of help for more complex projects. Just today, it almost fooled me into making a major structural mistake but thankfully I asked the right question and saved myself a lot of future problems; it was essentially dancing around a critical point and giving me the illusion that the critical point had been addressed but only when I phrased my question in a particular way, I understood the core issue was not addressed.
Just about anyone who has played a sport has heard that they have to target where the ball is moving to, not where the ball is right now. In this regard I am consistently disappointed with the commentary that mathematicians have been producing lately.
As a student, I am actively making decisions which will shape my career for the next forty or so years. At a minimum a discussion like this should acknowledge the possibility that the current rate of AI progress continues apace. I understand the desire to be encouraging, but the best preparation for students involves the consideration of possibilities that current mathematicians negligently paper over.
Of course it will get better at exposition than humans. And it will prompt itself in due time.
Do you care to share then? Because anyone I know with any sense right now is aware that "planning for the next forty or so years" is genuinely impossible at this point, especially in mathematics, and the right approach is to remain on your toes, ride the waves, and diversify as much as possible. Heck, even planning for the next five years seems absurd. This is a time of immense uncertainty, and to think otherwise is foolish.
A few years ago I was in a pure math degree, but I switched and graduated with a combined degree in math and computer science. Once I finish my masters degree I intend to work on industry applications of formal methods, which I anticipate will be a high-growth area for the next few years. The idea is to take advantage of the growing ability to generate proofs/code and work on keeping the messy human side of the equation up to speed. So I plan on addressing the social and operational aspects which come with delegating more work to AI. I'll work on integrating these gains into the industry, on building trust in the work that AI produces, and I'll work on the interface that lets us ensure that this trust is deserved.
But nothing is certain. I am trying to avoid overspecialization and remain flexible, just as you advise. If this path fails I might become a paramedic or physiotherapist.
No one succeeds in making accurate predictions of the future over a 40 year time scale. Look back at the 20th century, and see the range of range of events, and the kind of technological change, that could occur in a span that long.
Go into things expecting you might have to change careers.
Freeman Dyson said no scientific (or, I think, technological) project should last more than about five years, because the context under which it was conceived changes too much in that time.
> No one succeeds in making accurate predictions of the future over a 40 year time scale.
The sports analogy holds; the future location of the ball may be uncertain, but that is the student's target, and not overcommitting to the present location of the ball. The commenter is expressing disappointment that the commentary seems to be reacting to the present too much.
Science was quite stable in terms of how it works over the last 100 years.
Sure you wouldn’t predict what you would work on exactly. And there were better tools, better communication and so on sure, but no disruptive innovations or seismic shifts really.
I remember talking to an admissions tutor at Cambridge and he said even there most applicants had no idea what they wanted to do for a career. They just drifted into whatever came up.
In the UK only about 30-40% of students end up working in an area related to their degree.
The people who have a realistic plan that works out for them are a minority.
It's not a good analogy because the ball is constrained inside a court with fixed rules about how it can travel. No one can predict accurately how AI is going to change things. Look at all the predictions so far. We have the whole spectrum from "the end of humanity" to "utopia".
We might not be able to precisely predict where the ball will move, but I think GP is implicitly saying "don't assume the ball doesn't move", which IMHO is kind of baseline, but seems to be over the head for some people that are now complaining.
Some of those complaining today are the ones who keep saying "LLMs are just stochastic parrots predicting the next word", "how can AI do math if they can't even count the number of 'r's in strawberry?", etc.
It doesn't take a genius to realize AI will start getting better and possibly become a "threat" even before all this mathocalypse stuff happened.
If AI continues to improve at the current rate for 40 years then there is unlikely to be such a thing as a career.
Manual labor and trades is likely to be one of the last things to go, so the whole "dropout of school and go into trades" crowd may have been more correct than ever, though their original reasoning was not.
> Manual labor and trades is likely to be one of the last things to go
Yes, this is likely true. Plumber & Electrician on existing construction will likely take a long time (relatively speaking) to automate. And even if we end up being able to do it, it could be that humans will be the cheapest option when it comes to a lot of manual jobs especially given that there will be a large supply of unemployed humans in many scenarios.
I wouldn't bet a single dime on that, especially with the advancements in robotics (have you seen the Chinese robots?)
Especially if the timeline is 40 years (as opposed to, say... 4.) I have no doubt AI-assisted robotics development would have solved the plumber and electrician jobs in a decade or two.
I’d say if we have robots that can install AC in any old european building, then we have reached both AGI and AGLabour.
But I’m not sure it will take 40 years. I’m quite sure Astra could figure it out, it’s just too slow and robots are too weak. The speed will improve, and robots may lag shortly thereafter but by not much. I give it 1-2 years for AI, and 5-10 years for robots tops (perhaps even quicker).
All a career is is an exchange of value between people. They'll certainly exist but what is valued will shift, as they have been for centuries. Basically you can't just look at what LLMs will do, but you have to also consider how that will affect the broader economy and what our response to such will be. So what happens when LLMs start to bring the value of most digital cognitive work down to something approaching $0? It will just create a large vacuum in the economy that will ultimately get filled by other things. The obvious things that will probably increase in value will be stuff done/created in the real world, creative work, and so on.
It's also quite likely that the next big era for humanity will be the space era. And that stands to create the biggest labor boom in human history. It could also happen far faster than many might expect, even moreso as there will be competition between countries, especially as those left with only terrestrial claims will see their influence relatively wane. Space jobs and industry can also, in some ways, act as a more productive Works Progress Administration [1] to compensate for any temporary economic instability that LLMs might bring along.
The challenge is not will robots fix toilets but will the economics support it if we end up with most of the world unemployed and without careers. Yes of course we can hand wave basic income and government intervention but reading the room (e.g. last 70+ years of US labor and capital trends) it’s not the most likely outcome, even if it is the most game theory optimal for all.
Realistically the outcome of highly capable advanced AI is an economic dark age more than anything else.
I read The Mountain in the Sea by Ray Nayler a couple years back. In it AIs were running ocean fishing fleets. They figured out that robots were too expensive and prone to malfunction at sea, so they would have the robots kidnap people on shore, drug them and then have them wake up on the ships too far out in the ocean to be able to escape and thenm tell them they could get back home if they work.
Robots automating everything and humans not needing to work was the stuff paradise fantasies were made of.
Sure, it would be the end of capitalism (which, I guess you could describe as "economic dark age"), but people these days have so little imagination what life outside of capitalism looks like that ideal societies sound like a horror story.
Yeah it surely is lack of imagination - not the fact that humans have evolved hundreds of thousands of years to survive in resource constrained environments.
Communism was a horror for multiple reasons, but two big ones were: the planners didn't have the data or ability to actually plan (the interactions in a market economy did that much better), and not doing the work you were assigned was a crime (that is, slavery).
AGI and AGLabor solves both these problems. If you don't need workers, you don't need to coerce them into work. And planning and massive data collection (and management in general) sounds like something AGI would work very well at. The conceit that the lower classes are disempowered before the capitalists never struck me as well founded.
It's ironic you talk about communism, because fundamentally this is a problem of "owning the means of production", i.e. who controls the AI and the robots.
If it's you and me, then hey, we can all live happily ever after pursuing hobbies and art, provided for by our mechanical friends, within the limits of resource availability.
If it's an AI trillionaire, things aren't so rosy. If they don't need workers, why would they even bother allocating outputs from the robots to keep us alive?
I don't doubt at all that some group or another will enjoy the fruits of artificial labour. The question is how we are going to get there, and whether that group is most people who are live today and their descendants, or just the families and friends of trillionaires.
Why would you pay for a robot to do it when you could just hire one of the 20 million new plumbers (ex-developers, high school graduates, etc.) who just retrained and entered the field?
Yes, all we know is that things will change, but we don't know exactly how. So - increase your tolerance to sudden life swifts and learn to enjoy uncertainty.
If it crashes, there will still be trades jobs but they won't pay enough to eat. What can you charge for electrical work when everyone is either broke or an electrician?
I don't know, a lot of SWEs are suddenly discovering the value of class consciousness now it's our jobs in the crosshairs. I've heard more talk of unionisation in the last year or so than in my entire career so far for example.
They most certainly will, for at least two reasons. First, jobs aren't just about raw productivity, they're also about responsibility / liability diffusion. A CEO managing an army of agents is possibly on the hook if anything goes wrong - for example, if one of the agents does tax fraud or hacks a competitor. A CEO paying an army of employees, even if the employees are there just there to supervise agents, usually has little to worry about.
The second reason is that we often prefer humans to do it even if a machine can do it faster / cheaper. Sometimes, it's a status thing. For example, some people will buy Ikea furniture, some will have it custom made by a local craftsman. In fact, it's sort of the hallmark of the upper middle class that you can spend more for that human touch. Cupcakes from an artisanal bakery, private banker, etc.
Now, I don't think there's enough people who want to pay extra for human-made software. From the current trends in the industry, my impression is that we don't value our craft, so it would be surprising if others did. For mathematicians, I don't know; it's a bit painful to watch.
> A CEO managing an army of agents is possibly on the hook if anything goes wrong - for example, if one of the agents does tax fraud or hacks a competitor
This is correct under present day thinking.
But imagine the accounting AI of twenty years hence. If it can reliably do the books, reduce and prevent fraud, significantly better than any human ever could, why would you still need the human in that loop?
In the present day it's a legal requirement, not to mention a practical requirement given present AI capabilities. That doesn't mean it still will be in the future.
To put it another way, if the human is on the hook for the agent, but the agent is proven by a decade+ of statistical evidence to be way less mistake/fraud prone than the human, what's the point of the human there?
I'll give you that the capabilities of LLMs are improving rapidly. Their safety, however, seems to be stuck in mid-2023. They're still easy to dupe and prone to cheating to solve problems. Prompt injection is still a thing, and it's still something we need to paper over with input and output classifiers and other hacks external to the LLM.
What we're seeing so far is consistent with the training data being the upper bound for capabilities. They get better at recall / synthesis / reasoning over the corpus, but they don't, for example, acquire trans-human ethics; they're at best as ethical as we are, except not grounded by the fear of consequences. A perfectly-behaved, perfectly-moral LLM is not a given in 20 years, not unless your position is that there's room for unbounded, exponential self-improvement without any loss of fidelity. In that case, we'll probably have problems more pressing than the outlook for accounting jobs.
I'm hearing "better" and "less" but not zero and none. The more interesting question is what happens when that rouge/hallucinating agent does commit that statistically unlikely fraud. Who is responsible then? Or is it now no low we just consider it an "accident", pay a small penalty, and move on?
Some careers that will undoubtedly exist are the 'human manipulation' trades: politicians, salespeople, con artists, religious leaders, basically anyone who convinces people to feed them.
A dark side of this is that a lot of unethical, deplorable, and amoral people who were otherwise occupied by jobs might suddenly have a lot more free time to make trouble instead of spending 1/3 of their day working for rent and food to survive.
Not needing to work is a neat future that might be ahead of us. But it could unlock some new negatives as well as positives.
Free money and universal healthcare thanks to the abundance of Ai is where "they," say we are headed. Thanks to Ai ravaging our civil society we are use to.
This got downvoted but i wasnt saying i am for it, rather pointing out what the Ai overlords have said. Ones whom we are allowing to run amok and ravage our civil society with moving in new neighbors (Ai Data Centers) that eat us alive from the inside.
> At a minimum a discussion like this should acknowledge the possibility that the current rate of AI progress continues apace.
But the post does do this: "[...] even if their capacity becomes far superior, there will always be a need for mathematicians at all levels to guide research in paths that make sense for humans to walk (not run)."
> Of course it will get better at exposition than humans.
I'm not so sure about this, because I don't know how one would train towards better exposition as it's not easy to check for good or bad exposition at scale.
We need to be working on upgrades. For us. Advice for everyone: Whatever happens, take care of your health. One of the few things that is probably not a mistake.
The trajectory of a ball can be predicted. The trajectory of the job market for mathematicians is far too uncertain at this point to make reasonable predictions. This is not a matter of intelligence.
What specific direct advice would you give then if you could talk to someone who is currently studying math?
Do you know where “the ball” is going? Do these math students? Do the professors? It’s easy to say a vague nothing platitude like “target where the ball is going to” but such empty vague platitudes do not help anyone and just waste time.
Then what? math, the kind LLMs are making "obsolete" is a pursuit unlike any other. What other field would tickle the kind of mind attracted by math the same way? I doubt they went into it for the money.
Literally what else? Either you believe the future is AI + Humans, in which case most professions are open to you or you don’t in which case almost none are.
Ironically, this very attitude could lead to AI creating one of the biggest slowdowns in progress in history.
It's this one, but different careers will diminish at different rates. Mathematics was already not financially well rewarded, and current AI is basically better than everyone in the field.
> this very attitude could lead to AI creating one of the biggest slowdowns in progress in history
I think there are enough young people with pre-AI experience that we'll probably develop superintelligence before they retire, so I don't really think a near-term lack of fresh talent will result in any significant slowdown.
I was recently trying to work out what advice I should be giving my 15 year old son, I asked various AI for their recommendations given "possible" AI time frame and it seemed pretty much study whatever you want but always add " and business". Investing in things that AI will just make better. However, in the case of AI improves robotics to the point it's better than humans... who knows. But all that may not pan out as reality. For me, AI is still making a lot of the same mistakes it was making last year, it's very marginally improved in some aspects (not quite sure how to categorize what its failing at, but its like an over eager worker who really doesn't understand what they are working on but super keen to do stuff). It's gotten a lot better at doing more from single prompts. But it feels like, for LLMs at least, they are bumping into limits. But it can be hard to see because at the same time they are actually doing more. More and more I feel like what I'm doing is getting more ambitious in what I'm asking and then a big cycle of correcting.
I think that's a really unfair characterization of GP's post. They didn't talk about "target where the ball is going" as am empty platitude - they specifically pointed out that they need to plan for a career that will last ~40 years, and on the contrary a lot of these posts by math luminaries seem to be filled with a lot of "hopium" and platitudes that won't help pay the bills if it turns out the world needs a ton fewer mathematicians.
> What specific direct advice would you give then if you could talk to someone who is currently studying math?
Do it for personal satisfaction, but find another way to pay the bills.
FWIW, I did like that the article at least acknowledged that the fundamental question is whether LLM-based approaches will eventually "max out", i.e. will they be limited to the "convex hull of ideas outlined in literature" as the article out it.
If LLMs do eventually hit a wall, then great, humans will still have a role to play. If not, though, we're all completely fucked - none of this nonsense about "humans managing agents" or providing "unique human insight" and what not, as agents will be more than capable of managing themselves and providing superior insight.
Chess has always been "just a game". Sure, it teaches valuable life lessons, and it is fun to defeat your opponents through superior training and strategy, but it is "just a game".
Mathematics was historically "just a hobby". Much theoretical work is still "just a hobby". But, for the past few hundred years, humans that better understood the theory could find very profitable applications.
That is no longer true. Computers have solved mathematics, like they did chess two decades ago. A human will never find a better application than a computer, just like they will never find a better chess move. Yet people still play chess, and people still make money tutoring chess.
That is the future of humans in mathematics. Mathematics will become mathematics competitions. It will be just another intellectual sport. Sure, a sport that teaches valuable life lessons, but still "just a game". If you are going into mathematics today, your future career is as a competition mathematics coach.
What do you think mathematics is? "Just a hobby"? Developments in mathematics made possible the industrial revolution, the information age, everything! Do you think the world will be the same when math is "solved"?
Maybe don't take words out of my mouth. I studied math when I was younger, and got to the top 0.001% in mathematics competitions. Why did I study so hard? In order:
1. I recognized that understanding math had incredibly profitable applications.
2. It was a sport.
3. It was a hobby.
Do I think the world will look the same when math is "solved"? Absolutely not! And we're seeing it transform right before our eyes. I routinely ask Claude, "here's the general idea for a math/cs/ai problem I'm working on, can you help me figure out the maths? oh and then implement it too."
Could I figure this out myself? Of course. Having the idea is halfway to formalizing it in math. But I remember even last year working on GPU kernels by hand and having a bunch of little arithmetic or indexing errors burn me, because I'm just slower and less perfect at doing the math correctly. Now I basically just tell Claude, "go optimize a GPU kernel for this problem". I still have to tell it what to look out for (architectural, algorithmic, memory improvements), but I'm sure the next model won't even need that.
> In this regard I am consistently disappointed with the commentary that mathematicians have been producing lately.
For all the arrogance the hard sciences have, in the end they are just as human as their humanities counterparts. They just thought they were above becoming worthless because science was elevated above the softer sciences.
As the article points out, though, all of this fundamentally depends on current AI approaches "hitting a wall", that is where there are some inherent limitations in LLM-based tech that leaves humans some areas where we are still superior.
It's definitely (again, to the article's credit as it points out) up in the air whether LLMs have inherent limitations. But I can't fathom how someone can say "I don’t expect a jobs apocalypse, not even in math." Because if AI does end up surpassing humans, what exactly will there be left for humans to do? And even if they don't surpass humans, there will still be a jobs apocalypse. Probably the majority of people today work in jobs where AI can surpass their performance.
Simple case in point: years ago I used TurboTax to do my taxes, but I eventually needed to hire a CPA because I had some complicated situations that TurboTax couldn't handle, and I also needed some tailored advice. I ended up finding a great CPA. Now though, AI agents can literally do 100% of the job I hired my CPA to do, including asking me the right questions and offering advice. I'm sure there may still be tasks for a CPA in the corporate world, but for personal taxes, I literally can't imagine a CPA providing value over what an AI agent can provide. And to emphasize, I would have probably considered that a ludicrous statement a year ago with all the mistakes LLMs made. But so many of those mistakes have been fixed, and you can get better-than-human performance by having agents check each other's work. And AI will only get better when tax season rolls around next year.
I'm not a mathematician. I was a college math major, got a PhD in physics, and still enjoy math.
It's not like "should I get a PhD" is a new question. It's not unheard of for unpredictable events to drastically affect the career prospects of PhDs in my field. During my lifetime:
1. Mandatory retirement of professors was ruled illegal. While good for civil rights, it created a 10+ year gap in faculty retirements.
2. End of the cold war.
3. Transition of college teaching from tenured professors to gig workers, aka "adjuncts."
The one constant during this time was the perpetual optimism of the faculty for the employment prospects of PhDs. "There will always be a need for physicists." My dad, also a PhD, confirmed that this goes back as early as the 1950s.
I would add one question to the student's letter: What are the ethics of AI and its owners?
Indeed, I left academia immediately after finishing my degree. Now in my 60s, I'm at the "senior staff" level of a F500 company that makes measurement equipment, so I've done OK even if others have done better.
One thing that helped me a lot is that both of my parents were industrial scientists, so the idea of life beyond academia wasn't foreign to me.
There's going to be a massive reshaping of how mathematics is done. If you do not like where math is going (using computers to explore fully new ideas), and you have not committed to this path, it seems totally correct to opt out of it. There probably isn't going to be a technical field that isn't reshaped by this in the immediate future though, so there is unlikely to be anything that slots in cleanly as a replacement. Maybe the philosophy department.
Not even a compiler, but a search engine. That's what LLMs are: an NLP search engine. Maybe even slightly worse than that since the only option is the "I'm feeling lucky" button.
That may be true, but mathematicians still need to eat, and for the they'll need jobs. 'Will there be more or less demand for mathematicians in the future?' and 'What will the work entail?' I think are valid concerns
the vast majority of people who pursued a maths degree, including myself here, didn't expect to compete with Terrence Tao or solve outstanding research problems.
We studied maths because it is interesting, because it teaches you how to think, you often pair it up with something that's more employable like economics or software development, or you go for a teaching career. Unless you're one of the few people who are heavily invested in cutting edge research I don't even see how new research tools change the profession.
An undergraduate maths education isn't going to change because you have people with computers churn out 400 page proofs. It's like being worried about 30 move Stockfish opening theory if you're a club level chess player.
You must be thinking of physics and other sciences. Mathematics has very little to do with the real world and can be better described as a game, similar to chess, but much more complex. It's no wonder it now went the way of chess, go and similar games
> Mathematics has very little to do with the real world
But it's a construction based on very basic properties of the real world, at least on those properties we accept and perceive as very general and we use them as building blocks.
> Mathematics has very little to do with the real world a
I don't necessarily think that that is true, as almost every social science has been turned into maths one way or another, beginning with Economics. Heck, even mundane stuff like sports (think football/soccer) has now been absorbed by data, which you can see as applied maths.
Yes, I do know that advanced maths doesn't currently explain Arteta's system at Arsenal but we're generally talking about that same semantic area of interest, i.e. if the interest on maths were to subside as a result of AI taking that interest away from us then we will have no more data/stats-focused Artetas in the future (and possible no AI-like thingie also).
Almost all sciences (at least, parts of them) have been also expressed as code, mathematics is no different. It's a tool used by sciences, but it is not itself a science and it is not necessarily rooted in anything "real". Not better or worse, just different.
I’m saying that this wouldn’t have happened had our society not become “mathematics-ised” at some point (I’m talking about said society’s technical and scientific elites, of course). I’m also saying that outsourcing our “mathematics-mind” to AI (which seems the current process we’re now part of) will not allow for similar “mathematics-ing” to take place into the future. I’ll come with an even stronger claim and say that we risk losing the “mathematics-ing” we now associate with most of our sciences.
So learn math to teach math? You see the recursion here right? What’s the end value for society? Why would anyone fund this? How is this different from a hobby?
In a future where learning math has about the same economic value as learning to play chess or the flute, you can see why many mathematicians aren’t very excited about their chosen career paths.
A lot of math is useful to society. I'm not sure the more advanced research is very useful but you could say that for a lot of advanced research in the non science subjects.
Isn't there an analogy here to what's happening in software engineering? People keep saying things like "software engineering is so much more than programming". Couldn't one say that "mathematics is so much more than writing proofs"?
At least for now someone still has to decide what to prove and why. Like why are you trying to prove that thing to begin with? Presumably it's a step along some journey, right? Maybe the journey is where you need to start deriving your satisfaction from, then.
I know most people here are already well into their careers so they might not even notice at this point, but the CS job market isn't just 'going through a tough spot' for new grads. The entire market is down in general, but entry level? Entry Level Hirings have literally collapsed. Roughly 65-75% of entry level hiring dissapeared since 2019 at major segments of the tech industry.
I’m frankly a bit disappointed that Tao published the blog post I linked to above.
Telling OpenAI they shouldn’t test frontier math on their internal models is just plain nuts and illogical. It’s surprising that the advanced math community can lack so much logic.
There are lots of similarities currently within software engineering and mathematics and a lot of analogies which apply to Software engineering apply to mathematics and vice versa.
> At least for now someone still has to decide what to prove and why. Like why are you trying to prove that thing to begin with? Presumably it's a step along some journey, right? Maybe the journey is where you need to start deriving your satisfaction from, then.
Beautifully said. I had been thinking about the same thing and the analogy b/w CS and mathematics and these were some that I had found:
1.) to prove/disprove from the proofs that OAI created, you needed an mathematician to do so and OAI had to withdraw three mathematical proofs.[0]
But it was only because an expert within the field could verify if it was true or not, I feel as if software engineering is the same as well. We are/can be paid to prove/disprove if a software is working as intended or not.
2.) for someone to be that said mathematician who disproved it, he had to learn the basics of mathematics and multiple branches of it to then perhaps specialize in one thing that he most strongly resonated with and within all this learning, there was some struggle definitely involved. They had to learn algebra etc.,
this analogy can also extend to how we teach children algebra/calculations and other things even though we have had calculators for a long time, yet, we teach children how to do calculations because it is still valuable enough and either teaching maths can help them perhaps in future make a mathematician or it can help them be less reliant on simply calculators and more confident on on the spot calculations and help them within this skill.
As knowing calculation has become the norm rather than exception, even though we have calculators. In fact knowing how to do calculation by hand can perhaps better help you write a problem to calculator. Knowing the technical aspects of CS can help you express a problem to AI with much more depth and effectiveness as well.
One thing with programming is that LLMs are actually pretty bad at it, and use of them for that purpose is driven entirely by hype. I'm not remotely qualified to judge the mathematical realm, but mathematicians seem to think that LLMs can do the math well. So there's a significant difference there. In programming, we will see a major pull back from LLMs as the abysmal quality of vibe software becomes too prominent to ignore, but the same might not happen in math if they can actually do the job.
> will math academia be big enough and accessible enough for anyone with sheer passion (despite not being the brightest mind in the field) to pursue a career in, or will it inevitably shrink such that it will only really be accessible to the brightest minds?
Could someone with a recent math PhD experience tell me if it this hasn't been so for years? Competition is incredibly high because people have to compete with every "brightest mind" from all around the world. PhDs are overproduced relative to the number of permanent positions. It's not a question that you only start to think about after you've seen LLMs do mathematics, and if that's the case the student should be made fully aware of what he's getting into. I don't want to call the original post "dishonest" but I wish this was addressed here without the LLM angle. However, it was written by a tenured professor, so he did say the only thing he realistically could.
Not from math. But I know people who did math PhD. Like you said, competition is already high in all research fields. People looking for positions are way more than Universities can hire or number of profs retiring.
I'm with you, doing a PhD was already a risky choice. Now it's even more risky.
why invoke move 37 as evidence against AI creativity? Move 37 was made by an AI. Is it so insane to think that the same selfplay training regimen that AlphaGo underwent to make that move couldn't be applied to LLMs trying to solve math problems?
Just reading the letter from the student is depressing, and then you think of all the students who are not in a first world nation and how much worse their chances are.
God, the absolute decimation that is going to come for middle income countries.
Hell, right now the people quitting frontier labs are rediscovering trust and safety isues, like language parity problems, except all the work being done is in the first world and exported back to the rest.
> However, a proof of the Riemann hypothesis, say, may need new ideas that are strictly outside of the convex hull of current mathematical ideas
[...]
> After I finished writing this blog post, and had already sent it to Terry, OpenAI released a huge treasure trove of results in mathematics [including] the resolution of the so-called quasi Riemann Hypothesis
We ought to all be careful about underestimating the speed and magnitude of the change that is coming.
> if you are a student who is passionate to learn what is new and what is left to do, then a PhD is definitely the right path for you
This is an awful lot of confidence to put behind career advice in a wildly changing world. Markets are real and tradeoffs bite. We're not in gay communist space utopia yet.
I am not sure what you are trying to imply, but quasi-Riemann hypothesis is in fact not Riemann hypothesis and resolution of the former, while truly fucking amazing (it wouldn't be an exaggeration to say it is a discovery of century), gives no hint how to resolve the later.
Late in life I sought to complete community college. The RHEL degree path included a Calculus requirement that I could not evade. I had progressed somewhat beyond Calc in the first and second iterations, somehow.
So this Calculus class was 100% online. I discovered only at finals week that the instructor was firmly based at a distant satellite campus.
The class was brutal to me, and I needed a lot of time in the tutoring hall just to barely grasp concepts as they paraded past us. I used my calculator in good faith.
I discovered that there were websites that would solve integrals and complex equations, but I recognized that as outright cheating, so I resisted those tools.
However, the online LMS was set up with frequent quizzes with unlimited attempts. Not being penalized for attempts meant that I could brute-force every answer for every quiz throughout the course and get perfect scores. Though I felt kinda evil, I did just that.
When the final exam came around, our phantom remote instructor suddenly expected everyone to show up on her campus in-person. This was absurd to someone who was (1) on FAFSA funds and (2) riding the bus, and when I protested, the solution was a proctored session in the Disability office, all alone.
I solved every last question on that thing with paper and pencil and I required every single minute of the extended 3-hour limit they granted to me. I got an A- or whatever final grade.
Perhaps I didn’t deserve it, because all those learings drained out of my skull within 3 months, but it was a textbook example of gaming the system without strictly cheating, and since I was not aspiring to a math career, who cares?
The department deleted the Calculus requirement shortly after I finished that semester.
If the mathematicians refuse to use LLMs they will be superseded by a new set of Mathematicians 2.0 willing to do so. Eventually, the flow of money and students for bespoke artisanal manual math will dry up.
I don’t think anything is going to change for actual mathematicians. The only jobs they were getting were in universities and schools as teachers and professors anyways and you’re always going to need those.
That not all humans are born equal. Equal inquisitive, equally providing of original thought and inspiration. The new, the step back and above is where AI still has weak spots and needs help.
This is a great moment to be a hassadeur, a madmen, somebody who wants to jump of the cliff with just a rope on his feet.
If you are that- great times- otherwise- good luck.
If you are one of those who try to drag down people oustanding, congrats on the team effort. Now try that with a machine..
The difference between math and all the other professions AI has been obsoleting is that math is actually not a useful pursuit. The tired old line about math someday discovering something that will be useful in an entirely unrelated field is mostly baloney, and the problems that mathematicians spend their years broadly have no real world applications at all. A modern day mathematician is much closer to a monk than a productive worker.
It reminds me of The Hitchhiker’s Guide to the Galaxy. The mice asked for the answer to everything in the universe, and it produced the answer:
“42.”
But no one could understand what the answer even meant. So they designed a computer to build the question itself again, and that was Earth. Then the story begins with Earth being destroyed because of a cosmic highway problem (I won’t write more since that would be a spoiler). In the opening background of this work, I found it interesting that after calculating for 7.5 million years, they didn’t even know what they had originally been asking. The story now feels similar to that story from back then.
The solution is obviously to make Mathematician a hereditary title and then have the European Union implement global regulations which forces AI companies to pay royalties to licensed mathematicians for any mathematical research their models perform. Care has to be taken so that initial titles are distributed in an equal and fair way between diverse groups of people based on ethnicity, sexual orientation, and other identity factors.
You tell them to cancel their Astra subscriptions and get to work. Hundreds of mathematicians, including Peter Scholze, have already agreed to not use AI at all. There’s absolutely no reason to help train an AI that will just make your own life more miserable.
https://www.ahmath.org/members
A friend of a friend had a grant project written up for the next 10 years, with habilitation on the horizon.
His end goal was one of the things released in the recent OpenAI publications. It took a model 3.5 hours to do.
Things are wild now on math and theoretical faculties now.
And for theoretical fields labs don’t need human data - synthetic one works well if not better.
The best thing you can do if you’re a scientist, imho - wrapup whatever grant you have now asap with ai, use ai to get more grants if possible, and spend the rest od the year learning what new science you can do with the new tools and participate in creating the new paradigm in your field.
That’s when you’re an established scientist. If you’re new in the field then this is the most exciting time you could wish for - an opportunity to make a name for yourself.
> The best thing you can do if you’re a scientist, imho - wrapup whatever grant you have now asap with ai, use ai to get more grants if possible, and spend the rest od the year learning what new science you can do with the new tools and participate in creating the new paradigm in your field.
> That’s when you’re an established scientist. If you’re new in the field then this is the most exciting time you could wish for - an opportunity to make a name for yourself.
s/scientist/mathematician/g
There are plenty of fields of research where AI is not nearly as destructive as this, currently. Most experimental science is pretty safe for a while.
How do you think a PhD committee should referee a dissertation where all the ideas came from AI?
It's already the case. I know a professor and he says most of his students PhD ideas are AI generated, meaning they are outsourcing the fun stuff.
and that generation was educated without AI. going to be interesting how future generations of mathematicians will fare who have been using AI from day 1. I suspect they'll be simply much worse at math.
That generation will say "How on earth people could do this without AI", just like we have said "How on earth people could do this without the internet".
You got this right, these are incredibly exciting times. Take your friend of a friend's as an example. AI's are trained with existing information, so obviously whatever he expected to discover was already out there. It took the machine 3.5 hours to find and sort out something that would take a human 10 years. Now your F of a F can dedicate the next 10 years to do what the machine cannot do: find meaning and insight.
> AI's are trained with existing information, so obviously whatever he expected to discover was already out there.
I've wondered about that myself. Does the fact that LLM's were able to solve these problems so easily indicate that most of the solution already existed, scattered in the training data, and the LLM was simply able to put together the pieces?
I think it's an interesting conjecture but nothing more without proof.
> And for theoretical fields labs don’t need human data - synthetic one works well if not better.
How could that be true? I am a mathematical rube, but surely data that works "better" than real data should be extremely suspect?
That's insane advice. That's like telling CS students to never use a cloud LLM.
Sometimes HN is crazy and out of touch.
CS students should never use an LLM
To be fair I see two separate ways CS students use LLMs, one is to just complete assignments in the same way you could by asking someone else to do it, the other way is by getting an LLM to explain certain concepts in another way, or create more examples of them to aid in understanding quicker/in more depth. I actually think in the 2nd case they can be incredibly useful and can assist in creating a deeper understanding, but in the 1st case they obviously create fundamental gaps in understanding.
In a way it's on the student to make sure whatever they're doing, they actually understand it, or come exam season the gaps in their knowledge sans Claude will become quite obvious.
They shouldn't use the LLM to do their school work. There is still and will always be value in understanding.
However, to say they "should never use an LLM" entirely is going to make them unemployable.
People who need to get places shouldn't use rockets.
Such a dumb analogy.
Maybe "people who need to be able to make decisions about rocket use should understand them deeply not just ride on them"
No, actually, it's a great analogy:
- Can get you somewhere very quickly.
- Requires technical knowledge and ability to use correctly.
- If you're not careful, you will get to the wrong place, and will probably blow up by the time you arrive (if not before).
P.S. Your analogy is dumb. (Just arguing at your level here.)
What would you tell a CS student to use an LLM for?
To learn, to study, to explore, to experiment, to build, to practice, to invent, to research, to advance.
To not bury your head in the sand hoping this will go away, because it won't.
How do you prevent them becoming like students who never get comfortable with fractions because they punched all of their homework into wolfram alpha?
There have always been students that cheat themselves out of learning. It may even be the majority. It’s best to focus on the ones who have a genuine desire to learn.
The most motivated students don't need help. They will always find ways to grow with or without their teachers.
While I see your point, these people do benefit greatly from great teachers! (They’d do well in either case but do even better in the right environment).
I think a bigger problem is in programs where you have group projects. You can't really be the sole person who's against using LLMs and in favour of doing 'the right thing'. Unlike what some people think there's little difference in terms of raw intelligence or ability to get rewarded by the system in the 'cheater' group vs the 'non-cheater' group.
You ever hear the state if you owe the bank a 1000$ you have a problem, and if you owe the bank 100,000,000$ they have a problem? I think if a majority of students end up without basic skills the genius students will have problem. Will LLMs do that i don't think so but if you think its an acceptable outcome thats also not okay.
That's on them to use the tools with discipline.
You can't just handwave the fact that we're requiring what probably amounts to two orders of magnitude more discipline and self control out of students than we've ever done before. The educational system will collapse under this.
I'm not seeing how it requires any more self control than not copying another student's work or asking them for the answer. It's basically the same thing. We're also talking about adults here. They understand if they're cheating.
The difference is huge. Put yourself in the shoes of a student.
The student you're copying from won't be sitting next to you for the rest of your career. The LLM probably will be, and you'll be probably expected to use it, so why not use it here as well?
All your peers are doing it too, and look at all the cool things they're doing while you're struggling with the basics. And what if the LLMs keep improving at a high rate? What if being able to use them efficiently turns out to be a more important skill than what you're being taught anyways?
> why not use it here as well?
Because you're paying a lot of money and opportunity cost to learn how to do something. No one needs the assignment. It's there for you to learn. You were given the assignment for you. Any adult should be able to understand this. We're not talking about 7 year olds asking why they need to learn to multiply when a calculator can do it better than them. These are 20 year olds. If they don't have the maturity for this, a university should not be accepting them.
Being able to use LLMs efficiently isn't a specific skill. It's a reflection of your ability to articulate what you want, which is a reflection of your understanding of the world, which is what you're in school to build. Terence Tao can get an LLM to do math better than I can. I can probably get one to build software better than he can.
And you can still use one to do cool things like your peers. Just not your assignments, the purpose of which is literally to teach you the basics that you're struggling with and that you're there to learn in the first place.
This is like why don't you copy out of the back of the book or look up proofs on the internet. It's all there, but doing that completely misses the point of why you're there.
This only makes any sense if youre happy dropping most folks from higher education.
You want adults to be very disciplined and rational. That sounds great but are most of the adults in your own life experience at this level? They certainly aren't for me (I'm not sure I am at this level).
If I recall correctly, there was a mandatory ENGR 101 class that all of the different engineering fields had to take at my school, and on the first day, the lecturer said somewhere around half (might've been more, might've been fewer, I don't remember anymore) of the students in the hall were not going to complete the degree. So yeah, most folks dropping sounds about right.
And it's not even about discipline; it's lying. If you're not going to do the work, just don't do the work. Don't turn it in. Pretty much every syllabus ever says that turning in work you did not do is grounds for failing the class and possibly expulsion.
And why would we want to pass these people through? Then when they come interview with someone like me, I'm still going to find out within like a minute of talking to them that they didn't actually learn anything, and they won't get hired. Do I want to work with somebody incompetent? No. Do I want to work with somebody dishonest? Definitely no. Those both make my job more difficult to have around. So then they end up with a bunch of debt complaining about how they don't have a job.
This is why assessment needs to return to taking proctored exams, and away from coursework.
Well, if they don't know how to do the work, they fail the exams
why would they do that. whats the point?
remove them from schools that test and reward students like this.
> To not bury your head in the sand hoping this will go away, because it won't.
Eh? As Greg K-H mentioned in that video from a few days back about how very disappointing Fable was when compared to its astronomical hype, LLMs that one can run locally are quite good enough for a great many tasks... including bug hunting in the Linux kernel. And -as LLM boosters keep saying- they're only keep getting better, right?
We absolutely should be telling people to stop using the LLMs from the major LLM manufacturers. Those manufacturers have spent so many billions of dollars on this project, made so many promises that they're going to have no way to keep, and now that they're running into resistance, [0] they're holding all of humanity hostage unless they're permitted to get intimately involved in the creation of new laws and regulations especially for them. [2]
Even if one only considers their recent threats to humanity, it's clear that these are not companies that deserve any of our hard-earned money.
[0] Some of that resistance comes from their ever-more-sharply-increasing cost to produce the next performance increment, some from folks asking the probing questions about their promises, claims, and business practices that should have been asked years ago, some from ongoing State AG's court cases in regards to their illegal conduct, and some from folks who -unsurprisingly- don't want enormous warehouses that suck up quite notable amounts of power [1] but provide dreadfully little revenue to the areas that house them in their communities, and still others who are starting to realize that benefits of cloud LLMs aren't worth the tradeoff of being unable to afford a new personal computer.
[1] Note carefully that I made no reference to water usage. The only counterargument you can make here is that 500MW -> 2GW is not a quite notable amount of power.
[2] <https://news.ycombinator.com/item?id=49951653>
I am afraid that ship has already sailed, and we will have to live with the consequences.
At this point; I see no other scenario than having LLM doctors, LLM judges, etc. No matter the quality. We will get replaced. And if they do a worse job, nobody cares. If they do a better job, also nobody cares. Maybe you can pay more and get high effort model as your doctor.
These companies might not be the ones, my expectation is that they will be scrapped for their pieces, the losses distributed to the bagholders/taxpayers. But some version of them will.
This might be pure cyncisism with no positive value, but really I think we have no mechanisms left to stop going down on this trajectory.
Anyone else want their doctor to have access to LLMs? They need to verify, but to just cut them off from something with that much breadth seems insane.
I would prefer my doctor to have access to LLMs, just as I would prefer them to have access to google.
What I meant is different though. I fully expect to no longer have access to human doctors in a decade or two. Instead, LLMs will be the doctors. Likely government accredited ones. And my point is that in time, whether the LLM doctor is better or worse will not matter. Same with judges, same with the surveillance camera evaluators and private message examiners. I am saying we no longer have any mechanisms to stop these things from happening.
> They need to verify, but to just cut them off from something with that much breadth seems insane.
...homeboy here is behaving as if there aren't experts in this world who are paid to do and publish research and other experts who are paid to collate, analyze, vet, and publish that research for other domain experts to learn from. [0]
I'm fine with my doctor reading from expert-vetted documents. Hell, I'm more than fine with my doctor going through expert-vetted checklists; most of the time what's wrong with you can be easily figured by going through a few good checklists. I'm not fine with my doctor relying on a lossy-compressed database with a absentminded librarian that has a penchant for people-pleasing bolted on top.
I'm absolutely not against the use of ML systems in safety-critical domains like medicine, but I am absolutely against the use of LLMs in those domains.
[0] One might choose to retort with some variant of "But all that expert work is so expensive!". I would retort: "First, without that work the data fed into the LLM is catastrophically unreliable. Second, have you bothered to look at how much the major LLM manufacturers have spent over the past five years or so? I suspect that it's more money than has been spent on medical research 'meta analysis' over the past fifty years."
> At this point; I see no other scenario than having LLM doctors, LLM judges, etc. No matter the quality. We will get replaced. And if they do a worse job, nobody cares. If they do a better job, also nobody cares. Maybe you can pay more and get high effort model as your doctor.
Why should any of us pay more for reduced performance just so some capitalist can feel good about replacing a human with a machine? You know, the traditional premise of automation in manufacturing was that the outputs might be 80% as good as hand-crafted but 20% the price. Make that 120% the price and the whole equation changes!
Yes, but a little bit of lobbying and regulatory capture goes a long way.
At first, you provide the services for 20% the price and price out the competition. Also, human doctors also make mistakes, no? Do this long enough so that the long and costly process of educating physicians is no longer sustainable anymore. Then you regulate hard, after all, it is healthcare. Not anyone can and should just do it.
Then, once you are the only option left, jack up prices. We have seen this exact scenario play out many many times. This is just another version of it. And since trillions have already been burned, you can be sure that there will be a lot of desperation to make it happen.
You:
Me: I'm left wondering how much of my comment you read, and how much you understood from the parts you managed to read.Students are there to study to get the permission to use the LLM. Maybe they can use it for querying information but not to do their job (which is thinking).
You think a top 2 greatest mathematician alive is out of touch?
Lol - I'll say I do agree with you on this, but...
Being the greatest [anything] alive probably puts you at an extreme risk of being "out of touch". Is LeBron James out of touch?
He's probably out of touch on most things, but not on basketball.
Students should absolutely not use cloud LLMs.
Maybe there should be a class on how how to use LLMs too.
But learning computer science using LLMs all the time will be like learning to ride a bike using training wheels, and never taking them off.
> But leaning computer science using LLMs all the time will be like learning to ride a bike using training wheels, and never taking them off.
Or like learning to add fractions by looking up the answer key instead of actually struggling to figure it out.
Between your first and last statement there is an entirely reasonable middle ground that you've chosen to ignore
Yes, I addressed it. It is the logical conclusion of what I wrote.
I expect people that read things here to be literate. i choose optimism.
> Sometimes HN is crazy and out of touch.
Most vote-based forums like Reddit end up like that
But how do they expect to sustain on that?
One natural step of AI usage, the way I see it, is that even mediocre researchers can use AI as a harness to become prolific researchers.
So if you're part of a pure "human only" researchers that publish 1/10th of what the rest are doing, how are you going to survive?
The more prolific researchers will eat your grants for lunch.
This is exactly the same thing a lot of software devs are going through now. AI models have lifted up EVERYONES ability to produce code, so there's simply no premium anymore for those that will only code by hand. And the people paying their salary are asking why they aren't more productive, when even business analyst Joe is pumping out new products left and right.
People will pay good money for artisanal, hand-crafted clothing, or watches, or cars, or whatever, but I doubt that the market for artisanal mathematics is now or will ever be very large.
It's just stubbornness, in the end, that is manifesting as a kind of gatekeeping Luddism. I see the same thing in my Mastodon feed from folks in the software engineering world, particularly people who have a great love for writing code and hold it as a core part of their identities. I understand being irritated at a machine having been trained on (potentially, inter alia) your work with no compensation to you, and now it is perhaps better at (some of) what you do than you are; or maybe companies are now making a lot of money off of something that benefited from your contributions; but I find it extremely improbable that we will be able to refuse our way out of further progress now that so much money has been invested and so much momentum has been gathered.
I don't think I feel any discomfort about a machine being better at anything that I can do than I am at it. Maybe it's because I'm a mediocre person. I personally think that if there's something that I'm better at than a machine, then we need to build a better machine. Imagine what I'd be able to do if I used that machine!
That's going to be a very short term solution, no?
You can't really swear off looking at problems solved by AI, or keep moving away from fields whenever AI starts contributing, and have a viable career.
Today's frontier models, are everybody's local models in the near future.
If this were a change that had expected stalls or reprieves it would be different.
Doing mathematics looks to be the next chess, er, I mean protein folding challenge. The latter's computation demands have dropped significantly.
You have many fields that will be protected from ai, or even enabled by it - if I could bet, sociology will be a hot new field that will by definition still need humans always, but will gain both speed and significance.
If there is a field that desperately needs practitioners, it is net-sum political system design and dysfunction mitigation and elimination.
When things really start going downhill, if we are still in the current mess, it will grease the the wheels, and the brakes.
There are an infinite number of problems that require a unique proof (you can prove this btw). Yes maybe you need to work harder to find a more obscure problem now but why give up the fight already? I believe that students should have the right to intellectually stimulating education. If you really need to use AI, go for something that isn’t trying to put you out of business or destroy your right to a decent education, like a local open weights LLM.
> There are an infinite number of problems that require a unique proof
Yes. The problem isn't that there won't be any problems. The problem will be the rate of general progress and solutions that at least some awareness is required of, to reliably identify a good new unsolved problem will just keep getting more challenging. And then an attempt needs to be make, to solve it in a very short time.
There is always another race to run too, but increasingly slow runners don't find that translates to winning.
This change is not going to slow down, it is going to speed up. Machines will be doing math systematically, checking off the meta-math theorems that ensure axiom combinations are covered, that the search does not stop where theorems are not exhausted, and does stop where it can be proven they are. Humans will never operate at that level.
That might be like telling early biologists to throw away their microscopes because they are unfair competition to human eyes.
It’s more like avoiding looking at the answer key while doing homework to be able to pass an exam and to understand the material better, especially for the grad students who haven’t gotten their PhDs yet. Personally I think responsible AI use is possible (not all AHM members completely avoid AI, but it’s notable that some heavy hitters do), but it’s best not to use a closed model trying to replace you. Use an open model that won’t train on your data.
I think we should apply AI into human, so we could be better and then start doing more advanced stuff required for the Universe explorations. Current AI/LLMs can be considered as first step to that. Take and look about our history. We could never achieve so many if we stopped doing great things because of <any reason>.
Why do we ever use PCs, smartphones, Internet, electricity, medicine and so on? Why do we live? For what?
I don’t have a problem with using AI in the applied sciences once a researcher has proven their skills. I do have a problem with it damaging education, the pure math pipeline, and art.
I understand but I guess the future (far future or who knows) could be like that human just download SKILL and that's it - no more learning, study ing basic things. While basic things could be advanced math, computer science, biology and just everything (or nearly everything) what human do. I know it is science fiction right now but who knows. Smartphone + the Internet were step 0. AI/LLMs are next iteration. Still knowledge is valueable and so on but I guess eventually we get to the point which I said. Then galaxy exploration - if we still be here alive :)
I don't have a problem with AI to generate code..so long as the developer understands the output and would review his own mess rather than expecting those who know how to code will save the burning house after everyone's left in panic.
Well, if they can afford in first place
With all the slop being generated I would also like to see the same initiative in the software industry.
So others will write their PhD thesis faster, will publish more, you will not publish so you will perish. The rules of the game in this business are brutal. One can count on inventing something entirely new that AI will not be able to figure out, but this is a really high bet.
ah yes, the ol' bury your head in the sand approach
This post in the blog comments is quite compelling. Some observations that publishing of research is drying up because results can be easily retrieved at any time. Over the long-term, I wonder whether this will result in accumulation of knowledge grinding to a halt:
> With the latest ChatGPT models, these problems are more equivalent to homework questions: the answer is `in the back of the book.’ I am not discovering new solutions. Instead, I am working on problems whose answer exists and is simply waiting to be retrieved by a user of the model. In fact, I mentioned a problem that I was interested in working on to my advisor and he informed me that he and a collaborator had completely resolved it using ChatGPT – they have no plans to write up the result, so it will sit there until another `researcher’ pulls the proof slot machine.
There is another good point that the most tenured researchers have a sense of what problems are most worth exploring and therefore are more likely to feel excitement than younger researchers:
> I have also heard the contention that math research has `gotten more exciting,’ mainly from established researchers. They have decades of open problems that they care deeply about and want to see resolved. I have no such problems.
> On the other hand, none of the proofs so far seem to contain “alien ideas,” a move-37, or completely novel arguments or new concepts that were not present in the literature in some form or another.
I have the same optimism as Prof Tao. That said, I can understand why so many mathematicians have been so upset or stressed out. It turned out much of the mathematical work is about clever combination of existing methods - this already requires enormous amount human ingenuity and years of dedicated learning. Unfortunately, or maybe fortunately, AI can be very good at knowledge transfer and finding combination of existing ideas to solve seemingly impossible problems. Even though mathematicians are extremely smart and capable, only a small number of them are capable of truly inventing "alien ideas", discovering new ground-breaking mathematical structures, or coming up with new problem-solving techniques. That is, AI can eat many mathematicians' cake. That said, I'm still hopeful. Mathematicians still understand mathematics deeply. If someone can prompt AI to solve an important problem, that person is more likely a good mathematician than an average joe like me. So, I think mathematicians do have a bright future: leverage AI, and make more and bigger math discoveries. It's still the same north star: we must know, and we shall know. It's just that with AI, we will know sooner and more.
I was recently working on a project where I had to come up with a novel solution to a problem the company was facing. The problem has been solved before, but in suboptimal ways that wouldn’t work for this specific company. I came up with a pretty interesting solution, one that combines different statistical methods, and was a little more complicated, but gave better results.
I asked Claude what it thought about the technique. It hated it. It loathed the idea that I was not sticking with traditional methods that were pointed to in papers and pre-existing open source packages. Even after talking through the idea and how it was better and showing it the results, it was weirdly hesitant to admit it was a valid method, simply because it seemed nervous that the idea was novel.
I don’t have a real moral to the story, I just thought I would add my anecdote here.
I had a similar experience with not being able to convince Claude of a different approach to a problem. Then I shared the idea with Codex and it liked it. As another commenter said, you can have the same thing happen with humans.
Interesting! My experience is that Claude has a tendency to always suggest the true and tried, but likes off-center ideas when I present them. Fable especially has a "yes, and!" attitude where it understands and plusses the idea.
Real humans react the same way. The best argument you can pull to do something at work is to show that someone else already does it.
> I have the same optimism as Prof Tao.
This is not written by Tao, but by Álvaro Lozano-Robledo.
It's quite amusing that this mistake gets made on every single one of these guest posts when the first words on the page are "This is a guest post by Álvaro Lozano-Robledo" and the first paragraph starts with "I would like to give Terry my heartfelt thanks for giving me the opportunity to contribute a post to his blog." It's pretty clear that everyone is skimming these at best. Maybe he should put the disclaimer in a bigger font or something.
Though it’s very funny to imagine Tao suddenly acquiring a massive ego:
> At any given time in the history of mathematics, there have been mathematicians … whose mental capacity for mathematics seems completely super human (e.g., the owner of this blog, among many others).
Ouch! I don't know how I missed that. In fact, I thought it was the opposite: a guest post by Tao for Álvaro Lozano-Robledo
It's interesting, what would be Tao's views on the creativity question. Like, I don't find the empirical observation of LLMs now, the notion these proofs are mostly "just" sophisticated grinding, to be a strong indicator of what could/couldn't happen.
It's pretty much the same story with software development. There are two camps and the bigger camp got hit hard. Nowadays I feel entitled to use the "software engineer" title. Essentially, all the decisions-making and nuanced trade-offs which AI is not capable of making; that's engineering. AI can only do programming and code analysis... At least so far.
That said, in the past couple of month, AI has gained a lot of ground on the 'project management' front in terms of planning capability for a lot of simple to medium-complexity projects. It still needs a lot of help for more complex projects. Just today, it almost fooled me into making a major structural mistake but thankfully I asked the right question and saved myself a lot of future problems; it was essentially dancing around a critical point and giving me the illusion that the critical point had been addressed but only when I phrased my question in a particular way, I understood the core issue was not addressed.
Just about anyone who has played a sport has heard that they have to target where the ball is moving to, not where the ball is right now. In this regard I am consistently disappointed with the commentary that mathematicians have been producing lately.
As a student, I am actively making decisions which will shape my career for the next forty or so years. At a minimum a discussion like this should acknowledge the possibility that the current rate of AI progress continues apace. I understand the desire to be encouraging, but the best preparation for students involves the consideration of possibilities that current mathematicians negligently paper over.
Of course it will get better at exposition than humans. And it will prompt itself in due time.
Do you care to share then? Because anyone I know with any sense right now is aware that "planning for the next forty or so years" is genuinely impossible at this point, especially in mathematics, and the right approach is to remain on your toes, ride the waves, and diversify as much as possible. Heck, even planning for the next five years seems absurd. This is a time of immense uncertainty, and to think otherwise is foolish.
A few years ago I was in a pure math degree, but I switched and graduated with a combined degree in math and computer science. Once I finish my masters degree I intend to work on industry applications of formal methods, which I anticipate will be a high-growth area for the next few years. The idea is to take advantage of the growing ability to generate proofs/code and work on keeping the messy human side of the equation up to speed. So I plan on addressing the social and operational aspects which come with delegating more work to AI. I'll work on integrating these gains into the industry, on building trust in the work that AI produces, and I'll work on the interface that lets us ensure that this trust is deserved.
But nothing is certain. I am trying to avoid overspecialization and remain flexible, just as you advise. If this path fails I might become a paramedic or physiotherapist.
No one succeeds in making accurate predictions of the future over a 40 year time scale. Look back at the 20th century, and see the range of range of events, and the kind of technological change, that could occur in a span that long.
Go into things expecting you might have to change careers.
Freeman Dyson said no scientific (or, I think, technological) project should last more than about five years, because the context under which it was conceived changes too much in that time.
> No one succeeds in making accurate predictions of the future over a 40 year time scale.
The sports analogy holds; the future location of the ball may be uncertain, but that is the student's target, and not overcommitting to the present location of the ball. The commenter is expressing disappointment that the commentary seems to be reacting to the present too much.
Science was quite stable in terms of how it works over the last 100 years.
Sure you wouldn’t predict what you would work on exactly. And there were better tools, better communication and so on sure, but no disruptive innovations or seismic shifts really.
> accurate predictions of the future over a 40 year time scale
I think the problem is the reverse. We no longer can make good predictions for four years ahead, certainly not eight.
And yet, people have to make choices - so they need advice and some kind of hope of a guess.
> you might have to change careers.
I hope this is some dust dry humor!
I remember talking to an admissions tutor at Cambridge and he said even there most applicants had no idea what they wanted to do for a career. They just drifted into whatever came up.
In the UK only about 30-40% of students end up working in an area related to their degree.
The people who have a realistic plan that works out for them are a minority.
It's not a good analogy because the ball is constrained inside a court with fixed rules about how it can travel. No one can predict accurately how AI is going to change things. Look at all the predictions so far. We have the whole spectrum from "the end of humanity" to "utopia".
We might not be able to precisely predict where the ball will move, but I think GP is implicitly saying "don't assume the ball doesn't move", which IMHO is kind of baseline, but seems to be over the head for some people that are now complaining.
Some of those complaining today are the ones who keep saying "LLMs are just stochastic parrots predicting the next word", "how can AI do math if they can't even count the number of 'r's in strawberry?", etc.
It doesn't take a genius to realize AI will start getting better and possibly become a "threat" even before all this mathocalypse stuff happened.
If it gets better and better for the next 40 or so years, will your career exist?
If AI continues to improve at the current rate for 40 years then there is unlikely to be such a thing as a career.
Manual labor and trades is likely to be one of the last things to go, so the whole "dropout of school and go into trades" crowd may have been more correct than ever, though their original reasoning was not.
> Manual labor and trades is likely to be one of the last things to go
Yes, this is likely true. Plumber & Electrician on existing construction will likely take a long time (relatively speaking) to automate. And even if we end up being able to do it, it could be that humans will be the cheapest option when it comes to a lot of manual jobs especially given that there will be a large supply of unemployed humans in many scenarios.
I wouldn't bet a single dime on that, especially with the advancements in robotics (have you seen the Chinese robots?)
Especially if the timeline is 40 years (as opposed to, say... 4.) I have no doubt AI-assisted robotics development would have solved the plumber and electrician jobs in a decade or two.
I’d say if we have robots that can install AC in any old european building, then we have reached both AGI and AGLabour.
But I’m not sure it will take 40 years. I’m quite sure Astra could figure it out, it’s just too slow and robots are too weak. The speed will improve, and robots may lag shortly thereafter but by not much. I give it 1-2 years for AI, and 5-10 years for robots tops (perhaps even quicker).
This may well be offset by the flood of people entering those trades. Likely the days of tradespeople earning high salaries will be over as well.
All a career is is an exchange of value between people. They'll certainly exist but what is valued will shift, as they have been for centuries. Basically you can't just look at what LLMs will do, but you have to also consider how that will affect the broader economy and what our response to such will be. So what happens when LLMs start to bring the value of most digital cognitive work down to something approaching $0? It will just create a large vacuum in the economy that will ultimately get filled by other things. The obvious things that will probably increase in value will be stuff done/created in the real world, creative work, and so on.
It's also quite likely that the next big era for humanity will be the space era. And that stands to create the biggest labor boom in human history. It could also happen far faster than many might expect, even moreso as there will be competition between countries, especially as those left with only terrestrial claims will see their influence relatively wane. Space jobs and industry can also, in some ways, act as a more productive Works Progress Administration [1] to compensate for any temporary economic instability that LLMs might bring along.
[1] - https://en.wikipedia.org/wiki/Works_Progress_Administration
Don't you think a robot can fix your toilet as well?
Why are meta glasses such a big deal? Because they need training data for manual labor.
The challenge is not will robots fix toilets but will the economics support it if we end up with most of the world unemployed and without careers. Yes of course we can hand wave basic income and government intervention but reading the room (e.g. last 70+ years of US labor and capital trends) it’s not the most likely outcome, even if it is the most game theory optimal for all.
Realistically the outcome of highly capable advanced AI is an economic dark age more than anything else.
I read The Mountain in the Sea by Ray Nayler a couple years back. In it AIs were running ocean fishing fleets. They figured out that robots were too expensive and prone to malfunction at sea, so they would have the robots kidnap people on shore, drug them and then have them wake up on the ships too far out in the ocean to be able to escape and thenm tell them they could get back home if they work.
Humans were doing a hundred years ago in reality. https://en.wikipedia.org/wiki/Shanghaiing
We will always have jobs that require humans, they just don’t pay enough right now. Teachers, therapists, childcare, doctors, and whatnot.
And jobs that require humans’ responsibility.
People can do people stuff, machines can do machine stuff. And market will react accordingly.
It's hilarious to read stuff like this.
Robots automating everything and humans not needing to work was the stuff paradise fantasies were made of.
Sure, it would be the end of capitalism (which, I guess you could describe as "economic dark age"), but people these days have so little imagination what life outside of capitalism looks like that ideal societies sound like a horror story.
Yeah it surely is lack of imagination - not the fact that humans have evolved hundreds of thousands of years to survive in resource constrained environments.
I know, right?
Communism was a horror for multiple reasons, but two big ones were: the planners didn't have the data or ability to actually plan (the interactions in a market economy did that much better), and not doing the work you were assigned was a crime (that is, slavery).
AGI and AGLabor solves both these problems. If you don't need workers, you don't need to coerce them into work. And planning and massive data collection (and management in general) sounds like something AGI would work very well at. The conceit that the lower classes are disempowered before the capitalists never struck me as well founded.
It's ironic you talk about communism, because fundamentally this is a problem of "owning the means of production", i.e. who controls the AI and the robots.
If it's you and me, then hey, we can all live happily ever after pursuing hobbies and art, provided for by our mechanical friends, within the limits of resource availability.
If it's an AI trillionaire, things aren't so rosy. If they don't need workers, why would they even bother allocating outputs from the robots to keep us alive?
I don't doubt at all that some group or another will enjoy the fruits of artificial labour. The question is how we are going to get there, and whether that group is most people who are live today and their descendants, or just the families and friends of trillionaires.
The AI overlords will eventually consider the majority of human lives to be worthless. Let's face it, one day we will need to fight for mere survival.
> Why are meta glasses such a big deal? Because they need training data for manual labor.
Meta glasses are a terrible example for the case you're making a point about: 90+% of whatever video they'll capture will be trash data.
It'd be much more efficient to set up dedicated sites just for producing the very same training data you're claiming they harvest.
Unlike code, with a toilet it can be very unpleasant and/or expensive if AI gets it wrong three times before fixing it correctly.
Why would you pay for a robot to do it when you could just hire one of the 20 million new plumbers (ex-developers, high school graduates, etc.) who just retrained and entered the field?
Better to train on our own adaptability, which to an extend includes an advanced understanding of personal mental health
Yes, all we know is that things will change, but we don't know exactly how. So - increase your tolerance to sudden life swifts and learn to enjoy uncertainty.
If it crashes, there will still be trades jobs but they won't pay enough to eat. What can you charge for electrical work when everyone is either broke or an electrician?
Will any careers exist? How do you even plan for that
>How do you even plan for that
The honest answer is via politics not education, but this debate is often had in "apolitical" circles that try their best to ignore this.
Hacker News is largely libertarian/conservative and believes the free hand of the market will fix all.
I don't know, a lot of SWEs are suddenly discovering the value of class consciousness now it's our jobs in the crosshairs. I've heard more talk of unionisation in the last year or so than in my entire career so far for example.
They most certainly will, for at least two reasons. First, jobs aren't just about raw productivity, they're also about responsibility / liability diffusion. A CEO managing an army of agents is possibly on the hook if anything goes wrong - for example, if one of the agents does tax fraud or hacks a competitor. A CEO paying an army of employees, even if the employees are there just there to supervise agents, usually has little to worry about.
The second reason is that we often prefer humans to do it even if a machine can do it faster / cheaper. Sometimes, it's a status thing. For example, some people will buy Ikea furniture, some will have it custom made by a local craftsman. In fact, it's sort of the hallmark of the upper middle class that you can spend more for that human touch. Cupcakes from an artisanal bakery, private banker, etc.
Now, I don't think there's enough people who want to pay extra for human-made software. From the current trends in the industry, my impression is that we don't value our craft, so it would be surprising if others did. For mathematicians, I don't know; it's a bit painful to watch.
> A CEO managing an army of agents is possibly on the hook if anything goes wrong - for example, if one of the agents does tax fraud or hacks a competitor
This is correct under present day thinking.
But imagine the accounting AI of twenty years hence. If it can reliably do the books, reduce and prevent fraud, significantly better than any human ever could, why would you still need the human in that loop?
In the present day it's a legal requirement, not to mention a practical requirement given present AI capabilities. That doesn't mean it still will be in the future.
To put it another way, if the human is on the hook for the agent, but the agent is proven by a decade+ of statistical evidence to be way less mistake/fraud prone than the human, what's the point of the human there?
I'll give you that the capabilities of LLMs are improving rapidly. Their safety, however, seems to be stuck in mid-2023. They're still easy to dupe and prone to cheating to solve problems. Prompt injection is still a thing, and it's still something we need to paper over with input and output classifiers and other hacks external to the LLM.
What we're seeing so far is consistent with the training data being the upper bound for capabilities. They get better at recall / synthesis / reasoning over the corpus, but they don't, for example, acquire trans-human ethics; they're at best as ethical as we are, except not grounded by the fear of consequences. A perfectly-behaved, perfectly-moral LLM is not a given in 20 years, not unless your position is that there's room for unbounded, exponential self-improvement without any loss of fidelity. In that case, we'll probably have problems more pressing than the outlook for accounting jobs.
I'm hearing "better" and "less" but not zero and none. The more interesting question is what happens when that rouge/hallucinating agent does commit that statistically unlikely fraud. Who is responsible then? Or is it now no low we just consider it an "accident", pay a small penalty, and move on?
Some careers that will undoubtedly exist are the 'human manipulation' trades: politicians, salespeople, con artists, religious leaders, basically anyone who convinces people to feed them.
Universal Basic Income from all the Ai abundance. Is what Musk touts and since 2016, adding working will be optional.
Since AI abundance is taken for granted, why not just just offer every single person in the world full health insurance?
yes with UBI they say they would be offered too. I am only pointing out what the Ai overlords are saying.
A dark side of this is that a lot of unethical, deplorable, and amoral people who were otherwise occupied by jobs might suddenly have a lot more free time to make trouble instead of spending 1/3 of their day working for rent and food to survive.
Not needing to work is a neat future that might be ahead of us. But it could unlock some new negatives as well as positives.
That's a rather strange concern to have.
Yeah especially when the Terminator(s) are keeping us in check ... sounds crazy today but the world is quickly changing.
Give me one reason why Musk would honour this.
Looks like he and the other Ai overlords have Trump on this train too.
https://youtube.com/shorts/CaJd88zh1h0?feature=shared
Free money and universal healthcare thanks to the abundance of Ai is where "they," say we are headed. Thanks to Ai ravaging our civil society we are use to.
UBI is not going to happen. You're talking about ballooning the federal budget by trillions of dollars a year.
I'm not talking about it as fan, rather just pointing out Elon Musk, Sam Altman and etc have been for decades and still are.
I believe Sam Altman no longer supports UBI. But that doesn't matter because nothing he says can be trusted.
This got downvoted but i wasnt saying i am for it, rather pointing out what the Ai overlords have said. Ones whom we are allowing to run amok and ravage our civil society with moving in new neighbors (Ai Data Centers) that eat us alive from the inside.
UBI is politically dead on arrival in the U.S.
A lot of people put a lot of their self-worth into their work and that’s a good thing that won’t change.
UBI is investing in dividend ETFs.
> At a minimum a discussion like this should acknowledge the possibility that the current rate of AI progress continues apace.
But the post does do this: "[...] even if their capacity becomes far superior, there will always be a need for mathematicians at all levels to guide research in paths that make sense for humans to walk (not run)."
> Of course it will get better at exposition than humans.
I'm not so sure about this, because I don't know how one would train towards better exposition as it's not easy to check for good or bad exposition at scale.
> Just about anyone who has played a sport has heard that they have to target where the ball is moving to, not where the ball is right now.
But the ball is flying out of the court and into the abyss.
We need to be working on upgrades. For us. Advice for everyone: Whatever happens, take care of your health. One of the few things that is probably not a mistake.
Don't go classically into that qubit night.
Yes, looking at mathematicians, It's remarkable how such smart people can behave so stupidly. Looks like intelligence is overrated.
The trajectory of a ball can be predicted. The trajectory of the job market for mathematicians is far too uncertain at this point to make reasonable predictions. This is not a matter of intelligence.
“It is difficult to get a man to understand something, when his salary depends upon his not understanding it”
What specific direct advice would you give then if you could talk to someone who is currently studying math?
Do you know where “the ball” is going? Do these math students? Do the professors? It’s easy to say a vague nothing platitude like “target where the ball is going to” but such empty vague platitudes do not help anyone and just waste time.
I would probably tell them to not study math.
Then what? math, the kind LLMs are making "obsolete" is a pursuit unlike any other. What other field would tickle the kind of mind attracted by math the same way? I doubt they went into it for the money.
Robot monks aren't replacing humans yet, even if their traditional crafts (brewing etc.) could be automated.
Human-crafted religious items and artwork may be more appealing than machine-crafted replicas.
> I doubt they went into it for the money.
I assumed the student ultimately wants a career that is financially viable.
> Then what?
Literally anything else?
It's like asking a budding pro basketball player to.. go study machine learning
Worked for budding pro football player John Urschel
Literally what else? Either you believe the future is AI + Humans, in which case most professions are open to you or you don’t in which case almost none are.
Ironically, this very attitude could lead to AI creating one of the biggest slowdowns in progress in history.
> in which case almost none are
It's this one, but different careers will diminish at different rates. Mathematics was already not financially well rewarded, and current AI is basically better than everyone in the field.
> this very attitude could lead to AI creating one of the biggest slowdowns in progress in history
I think there are enough young people with pre-AI experience that we'll probably develop superintelligence before they retire, so I don't really think a near-term lack of fresh talent will result in any significant slowdown.
What would you study? Current AI is basically better than everyone in every field. Have you tried the latest models?
> we'll probably develop superintelligence before they retire.
That ship has sailed:
https://www.whitehouse.gov/presidential-actions/2026/09/inau...
> Mathematics was already not financially well rewarded,
Then why are frontier labs ploughing millions into racing human math researchers when they're already cash-strapped? What's the ROI for that?
It’s good marketing because people perceive mathematicians as the smartest of the smart.
I was recently trying to work out what advice I should be giving my 15 year old son, I asked various AI for their recommendations given "possible" AI time frame and it seemed pretty much study whatever you want but always add " and business". Investing in things that AI will just make better. However, in the case of AI improves robotics to the point it's better than humans... who knows. But all that may not pan out as reality. For me, AI is still making a lot of the same mistakes it was making last year, it's very marginally improved in some aspects (not quite sure how to categorize what its failing at, but its like an over eager worker who really doesn't understand what they are working on but super keen to do stuff). It's gotten a lot better at doing more from single prompts. But it feels like, for LLMs at least, they are bumping into limits. But it can be hard to see because at the same time they are actually doing more. More and more I feel like what I'm doing is getting more ambitious in what I'm asking and then a big cycle of correcting.
I think that's a really unfair characterization of GP's post. They didn't talk about "target where the ball is going" as am empty platitude - they specifically pointed out that they need to plan for a career that will last ~40 years, and on the contrary a lot of these posts by math luminaries seem to be filled with a lot of "hopium" and platitudes that won't help pay the bills if it turns out the world needs a ton fewer mathematicians.
> What specific direct advice would you give then if you could talk to someone who is currently studying math?
Do it for personal satisfaction, but find another way to pay the bills.
FWIW, I did like that the article at least acknowledged that the fundamental question is whether LLM-based approaches will eventually "max out", i.e. will they be limited to the "convex hull of ideas outlined in literature" as the article out it.
If LLMs do eventually hit a wall, then great, humans will still have a role to play. If not, though, we're all completely fucked - none of this nonsense about "humans managing agents" or providing "unique human insight" and what not, as agents will be more than capable of managing themselves and providing superior insight.
Chess has always been "just a game". Sure, it teaches valuable life lessons, and it is fun to defeat your opponents through superior training and strategy, but it is "just a game".
Mathematics was historically "just a hobby". Much theoretical work is still "just a hobby". But, for the past few hundred years, humans that better understood the theory could find very profitable applications.
That is no longer true. Computers have solved mathematics, like they did chess two decades ago. A human will never find a better application than a computer, just like they will never find a better chess move. Yet people still play chess, and people still make money tutoring chess.
That is the future of humans in mathematics. Mathematics will become mathematics competitions. It will be just another intellectual sport. Sure, a sport that teaches valuable life lessons, but still "just a game". If you are going into mathematics today, your future career is as a competition mathematics coach.
What do you think mathematics is? "Just a hobby"? Developments in mathematics made possible the industrial revolution, the information age, everything! Do you think the world will be the same when math is "solved"?
Maybe don't take words out of my mouth. I studied math when I was younger, and got to the top 0.001% in mathematics competitions. Why did I study so hard? In order:
1. I recognized that understanding math had incredibly profitable applications.
2. It was a sport.
3. It was a hobby.
Do I think the world will look the same when math is "solved"? Absolutely not! And we're seeing it transform right before our eyes. I routinely ask Claude, "here's the general idea for a math/cs/ai problem I'm working on, can you help me figure out the maths? oh and then implement it too."
Could I figure this out myself? Of course. Having the idea is halfway to formalizing it in math. But I remember even last year working on GPU kernels by hand and having a bunch of little arithmetic or indexing errors burn me, because I'm just slower and less perfect at doing the math correctly. Now I basically just tell Claude, "go optimize a GPU kernel for this problem". I still have to tell it what to look out for (architectural, algorithmic, memory improvements), but I'm sure the next model won't even need that.
> In this regard I am consistently disappointed with the commentary that mathematicians have been producing lately.
For all the arrogance the hard sciences have, in the end they are just as human as their humanities counterparts. They just thought they were above becoming worthless because science was elevated above the softer sciences.
You could being playing best jai alai player always knowing where the ball goes. Problem is, nobody cares.
[flagged]
Please don't cross into personal attack. It's against HN's rules, and it's never necessary.
https://news.ycombinator.com/newsguidelines.html
Nice piece. I think the fundamental message that there is still demand for mathematicians and mathematics done by humans is fundamentally correct.
I expect it will remain true into the future (10+ years). I have moderately high confidence (75% or higher) in this prediction.
I use frontier AI models in my work all the time. I think they accelerate my work by helping me understand faster and prompt better.
The models are most useful and most productivity-enhancing in the hands of experts and in the area of their expertise.
I don’t expect a jobs apocalypse, not even in math.
As the article points out, though, all of this fundamentally depends on current AI approaches "hitting a wall", that is where there are some inherent limitations in LLM-based tech that leaves humans some areas where we are still superior.
It's definitely (again, to the article's credit as it points out) up in the air whether LLMs have inherent limitations. But I can't fathom how someone can say "I don’t expect a jobs apocalypse, not even in math." Because if AI does end up surpassing humans, what exactly will there be left for humans to do? And even if they don't surpass humans, there will still be a jobs apocalypse. Probably the majority of people today work in jobs where AI can surpass their performance.
Simple case in point: years ago I used TurboTax to do my taxes, but I eventually needed to hire a CPA because I had some complicated situations that TurboTax couldn't handle, and I also needed some tailored advice. I ended up finding a great CPA. Now though, AI agents can literally do 100% of the job I hired my CPA to do, including asking me the right questions and offering advice. I'm sure there may still be tasks for a CPA in the corporate world, but for personal taxes, I literally can't imagine a CPA providing value over what an AI agent can provide. And to emphasize, I would have probably considered that a ludicrous statement a year ago with all the mistakes LLMs made. But so many of those mistakes have been fixed, and you can get better-than-human performance by having agents check each other's work. And AI will only get better when tax season rolls around next year.
I'm not a mathematician. I was a college math major, got a PhD in physics, and still enjoy math.
It's not like "should I get a PhD" is a new question. It's not unheard of for unpredictable events to drastically affect the career prospects of PhDs in my field. During my lifetime:
1. Mandatory retirement of professors was ruled illegal. While good for civil rights, it created a 10+ year gap in faculty retirements.
2. End of the cold war.
3. Transition of college teaching from tenured professors to gig workers, aka "adjuncts."
The one constant during this time was the perpetual optimism of the faculty for the employment prospects of PhDs. "There will always be a need for physicists." My dad, also a PhD, confirmed that this goes back as early as the 1950s.
I would add one question to the student's letter: What are the ethics of AI and its owners?
> Transition of college teaching from tenured professors to gig workers
This is a huge, and likely permanent, change.
Philip Greenspun has a 2-decade old essay that is relevant:
https://philip.greenspun.com/careers/women-in-science
Basically, it's arguing that science (really academia) is generally an awful career path for most Americans.
Indeed, I left academia immediately after finishing my degree. Now in my 60s, I'm at the "senior staff" level of a F500 company that makes measurement equipment, so I've done OK even if others have done better.
One thing that helped me a lot is that both of my parents were industrial scientists, so the idea of life beyond academia wasn't foreign to me.
There's going to be a massive reshaping of how mathematics is done. If you do not like where math is going (using computers to explore fully new ideas), and you have not committed to this path, it seems totally correct to opt out of it. There probably isn't going to be a technical field that isn't reshaped by this in the immediate future though, so there is unlikely to be anything that slots in cleanly as a replacement. Maybe the philosophy department.
The “convex hull of ideas” is a very interesting framing.
My own suspicion/guess is that AI can break out of the bounds of the convex hull in math, and that’s going to become apparent soon.
Treat the model as a compiler, not an oracle. You still must know how to specify the problem correctly, or you will only generate garbage faster.
Not even a compiler, but a search engine. That's what LLMs are: an NLP search engine. Maybe even slightly worse than that since the only option is the "I'm feeling lucky" button.
Use of AI actively reduces brain capacity, especially when learning.
AI is never to take the place of understanding, knowledge, and education.
It's best that AI sycophants smarten up.
Learning mathematics is about understanding the world, not proving things.
That may be true, but mathematicians still need to eat, and for the they'll need jobs. 'Will there be more or less demand for mathematicians in the future?' and 'What will the work entail?' I think are valid concerns
the vast majority of people who pursued a maths degree, including myself here, didn't expect to compete with Terrence Tao or solve outstanding research problems.
We studied maths because it is interesting, because it teaches you how to think, you often pair it up with something that's more employable like economics or software development, or you go for a teaching career. Unless you're one of the few people who are heavily invested in cutting edge research I don't even see how new research tools change the profession.
An undergraduate maths education isn't going to change because you have people with computers churn out 400 page proofs. It's like being worried about 30 move Stockfish opening theory if you're a club level chess player.
That's very fair. I enjoyed undergrad maths because it kind of blew my mind but I never considered pursuing graduate studies
You must be thinking of physics and other sciences. Mathematics has very little to do with the real world and can be better described as a game, similar to chess, but much more complex. It's no wonder it now went the way of chess, go and similar games
> Mathematics has very little to do with the real world
But it's a construction based on very basic properties of the real world, at least on those properties we accept and perceive as very general and we use them as building blocks.
or maybe not https://en.wikipedia.org/wiki/Mathematical_universe_hypothes...
> Mathematics has very little to do with the real world a
I don't necessarily think that that is true, as almost every social science has been turned into maths one way or another, beginning with Economics. Heck, even mundane stuff like sports (think football/soccer) has now been absorbed by data, which you can see as applied maths.
Yes, I do know that advanced maths doesn't currently explain Arteta's system at Arsenal but we're generally talking about that same semantic area of interest, i.e. if the interest on maths were to subside as a result of AI taking that interest away from us then we will have no more data/stats-focused Artetas in the future (and possible no AI-like thingie also).
Almost all sciences (at least, parts of them) have been also expressed as code, mathematics is no different. It's a tool used by sciences, but it is not itself a science and it is not necessarily rooted in anything "real". Not better or worse, just different.
> ) have been also expressed as code
I’m saying that this wouldn’t have happened had our society not become “mathematics-ised” at some point (I’m talking about said society’s technical and scientific elites, of course). I’m also saying that outsourcing our “mathematics-mind” to AI (which seems the current process we’re now part of) will not allow for similar “mathematics-ing” to take place into the future. I’ll come with an even stronger claim and say that we risk losing the “mathematics-ing” we now associate with most of our sciences.
What economic value is there in a small group of people understanding something, if they provide no added value by proving things?
They can provide value by teaching a larger group as is traditional. Probably 99% of math teachers haven't proved ground breaking theorems.
So learn math to teach math? You see the recursion here right? What’s the end value for society? Why would anyone fund this? How is this different from a hobby?
In a future where learning math has about the same economic value as learning to play chess or the flute, you can see why many mathematicians aren’t very excited about their chosen career paths.
A lot of math is useful to society. I'm not sure the more advanced research is very useful but you could say that for a lot of advanced research in the non science subjects.
Isn't there an analogy here to what's happening in software engineering? People keep saying things like "software engineering is so much more than programming". Couldn't one say that "mathematics is so much more than writing proofs"?
At least for now someone still has to decide what to prove and why. Like why are you trying to prove that thing to begin with? Presumably it's a step along some journey, right? Maybe the journey is where you need to start deriving your satisfaction from, then.
I know most people here are already well into their careers so they might not even notice at this point, but the CS job market isn't just 'going through a tough spot' for new grads. The entire market is down in general, but entry level? Entry Level Hirings have literally collapsed. Roughly 65-75% of entry level hiring dissapeared since 2019 at major segments of the tech industry.
It's difficult to pick that apart though. How much of it was down to the over hiring and bounce back in the early twenties and how much is down to AI?
I don't think it's that difficult.
1. There was no overhiring in 2019.
2. It's much worse at the entry level than the overall industry.
3. We're years out of COVID. It's not relevant anymore.
Source?
https://www.signalfire.com/blog/signalfire-state-of-talent-r...
https://digitaleconomy.stanford.edu/news/canariesaug26/?trk=...
Thank you.
Yes. I wrote about it here in the other post: https://news.ycombinator.com/item?id=50000421#50000660
I’m frankly a bit disappointed that Tao published the blog post I linked to above.
Telling OpenAI they shouldn’t test frontier math on their internal models is just plain nuts and illogical. It’s surprising that the advanced math community can lack so much logic.
There are lots of similarities currently within software engineering and mathematics and a lot of analogies which apply to Software engineering apply to mathematics and vice versa.
> At least for now someone still has to decide what to prove and why. Like why are you trying to prove that thing to begin with? Presumably it's a step along some journey, right? Maybe the journey is where you need to start deriving your satisfaction from, then.
Beautifully said. I had been thinking about the same thing and the analogy b/w CS and mathematics and these were some that I had found:
1.) to prove/disprove from the proofs that OAI created, you needed an mathematician to do so and OAI had to withdraw three mathematical proofs.[0]
But it was only because an expert within the field could verify if it was true or not, I feel as if software engineering is the same as well. We are/can be paid to prove/disprove if a software is working as intended or not.
2.) for someone to be that said mathematician who disproved it, he had to learn the basics of mathematics and multiple branches of it to then perhaps specialize in one thing that he most strongly resonated with and within all this learning, there was some struggle definitely involved. They had to learn algebra etc.,
this analogy can also extend to how we teach children algebra/calculations and other things even though we have had calculators for a long time, yet, we teach children how to do calculations because it is still valuable enough and either teaching maths can help them perhaps in future make a mathematician or it can help them be less reliant on simply calculators and more confident on on the spot calculations and help them within this skill.
As knowing calculation has become the norm rather than exception, even though we have calculators. In fact knowing how to do calculation by hand can perhaps better help you write a problem to calculator. Knowing the technical aspects of CS can help you express a problem to AI with much more depth and effectiveness as well.
[0]: https://news.ycombinator.com/item?id=50002650
One thing with programming is that LLMs are actually pretty bad at it, and use of them for that purpose is driven entirely by hype. I'm not remotely qualified to judge the mathematical realm, but mathematicians seem to think that LLMs can do the math well. So there's a significant difference there. In programming, we will see a major pull back from LLMs as the abysmal quality of vibe software becomes too prominent to ignore, but the same might not happen in math if they can actually do the job.
> One thing with programming is that LLMs are actually pretty bad at it, and use of them for that purpose is driven entirely by hype
You haven't tried the latest gen of frontier models, I take it? These things aren't just hype.
Entry Level CS Hirings have literally collapsed. There is no difference.
What the hell are you talking about? In a lot of companies and I mean a lot, developers haven’t hand written their code in months.
With all the talk about great level of problem solving. I wonder if their LLM could come with unique research problems to solve?
A student's letter from the post says
> will math academia be big enough and accessible enough for anyone with sheer passion (despite not being the brightest mind in the field) to pursue a career in, or will it inevitably shrink such that it will only really be accessible to the brightest minds?
Could someone with a recent math PhD experience tell me if it this hasn't been so for years? Competition is incredibly high because people have to compete with every "brightest mind" from all around the world. PhDs are overproduced relative to the number of permanent positions. It's not a question that you only start to think about after you've seen LLMs do mathematics, and if that's the case the student should be made fully aware of what he's getting into. I don't want to call the original post "dishonest" but I wish this was addressed here without the LLM angle. However, it was written by a tenured professor, so he did say the only thing he realistically could.
Not from math. But I know people who did math PhD. Like you said, competition is already high in all research fields. People looking for positions are way more than Universities can hire or number of profs retiring.
I'm with you, doing a PhD was already a risky choice. Now it's even more risky.
why invoke move 37 as evidence against AI creativity? Move 37 was made by an AI. Is it so insane to think that the same selfplay training regimen that AlphaGo underwent to make that move couldn't be applied to LLMs trying to solve math problems?
Reading all the AI progress on math I wonder if current SOTA LLMs could have come up with the Incompleteness theorems.
Just reading the letter from the student is depressing, and then you think of all the students who are not in a first world nation and how much worse their chances are.
God, the absolute decimation that is going to come for middle income countries.
Hell, right now the people quitting frontier labs are rediscovering trust and safety isues, like language parity problems, except all the work being done is in the first world and exported back to the rest.
that is kinda true, AI (LLM) can't think right now but maybe soon enough it is gonna change
> However, a proof of the Riemann hypothesis, say, may need new ideas that are strictly outside of the convex hull of current mathematical ideas
[...]
> After I finished writing this blog post, and had already sent it to Terry, OpenAI released a huge treasure trove of results in mathematics [including] the resolution of the so-called quasi Riemann Hypothesis
We ought to all be careful about underestimating the speed and magnitude of the change that is coming.
> if you are a student who is passionate to learn what is new and what is left to do, then a PhD is definitely the right path for you
This is an awful lot of confidence to put behind career advice in a wildly changing world. Markets are real and tradeoffs bite. We're not in gay communist space utopia yet.
I am not sure what you are trying to imply, but quasi-Riemann hypothesis is in fact not Riemann hypothesis and resolution of the former, while truly fucking amazing (it wouldn't be an exaggeration to say it is a discovery of century), gives no hint how to resolve the later.
From Siegel Zero to Siegel Hero.
> This is a guest post by Álvaro Lozano-Robledo.
Confusing since it's on a terrytao's wordpress.com blog. I wonder if he's aware of the drama at Automattic.
Late in life I sought to complete community college. The RHEL degree path included a Calculus requirement that I could not evade. I had progressed somewhat beyond Calc in the first and second iterations, somehow.
So this Calculus class was 100% online. I discovered only at finals week that the instructor was firmly based at a distant satellite campus.
The class was brutal to me, and I needed a lot of time in the tutoring hall just to barely grasp concepts as they paraded past us. I used my calculator in good faith.
I discovered that there were websites that would solve integrals and complex equations, but I recognized that as outright cheating, so I resisted those tools.
However, the online LMS was set up with frequent quizzes with unlimited attempts. Not being penalized for attempts meant that I could brute-force every answer for every quiz throughout the course and get perfect scores. Though I felt kinda evil, I did just that.
When the final exam came around, our phantom remote instructor suddenly expected everyone to show up on her campus in-person. This was absurd to someone who was (1) on FAFSA funds and (2) riding the bus, and when I protested, the solution was a proctored session in the Disability office, all alone.
I solved every last question on that thing with paper and pencil and I required every single minute of the extended 3-hour limit they granted to me. I got an A- or whatever final grade.
Perhaps I didn’t deserve it, because all those learings drained out of my skull within 3 months, but it was a textbook example of gaming the system without strictly cheating, and since I was not aspiring to a math career, who cares?
The department deleted the Calculus requirement shortly after I finished that semester.
As much as I respect Dr. T (and by proxy, the guest author on his blog), it seems that Math Bros are having the meltdown that SWEs had last year.
SWE is a capitalistic game and Academia seems to be a game of status ?
Regardless, they are not going to make it if they remove LLMs.
Someone, somewhere will use SOL/Opus and mog the big boys.
If the mathematicians refuse to use LLMs they will be superseded by a new set of Mathematicians 2.0 willing to do so. Eventually, the flow of money and students for bespoke artisanal manual math will dry up.
"Solve the Riemann hypothesis, make no mistakes"
What actual industries where you could get a phd in, died?
If I was to ever suggest one, it would be Philosphers.
Yet they have found ways to get tenure and/or other jobs for as long as the field exists.
I don’t think anything is going to change for actual mathematicians. The only jobs they were getting were in universities and schools as teachers and professors anyways and you’re always going to need those.
Most of my fellow math students work now in software engineering or consulting. There aren't a lot of professor positions available.
Sounds like a pyramid scheme
That not all humans are born equal. Equal inquisitive, equally providing of original thought and inspiration. The new, the step back and above is where AI still has weak spots and needs help.
This is a great moment to be a hassadeur, a madmen, somebody who wants to jump of the cliff with just a rope on his feet.
If you are that- great times- otherwise- good luck.
If you are one of those who try to drag down people oustanding, congrats on the team effort. Now try that with a machine..
The difference between math and all the other professions AI has been obsoleting is that math is actually not a useful pursuit. The tired old line about math someday discovering something that will be useful in an entirely unrelated field is mostly baloney, and the problems that mathematicians spend their years broadly have no real world applications at all. A modern day mathematician is much closer to a monk than a productive worker.
It's like the spinoff argument for NASA, which is also nonsense.
Funny because monks discovered a lot. Genetics and the Bayes theorem come to mind.
Stop the nonsense. 99% of monks are not scientists.
It reminds me of The Hitchhiker’s Guide to the Galaxy. The mice asked for the answer to everything in the universe, and it produced the answer:
“42.”
But no one could understand what the answer even meant. So they designed a computer to build the question itself again, and that was Earth. Then the story begins with Earth being destroyed because of a cosmic highway problem (I won’t write more since that would be a spoiler). In the opening background of this work, I found it interesting that after calculating for 7.5 million years, they didn’t even know what they had originally been asking. The story now feels similar to that story from back then.
The solution is obviously to make Mathematician a hereditary title and then have the European Union implement global regulations which forces AI companies to pay royalties to licensed mathematicians for any mathematical research their models perform. Care has to be taken so that initial titles are distributed in an equal and fair way between diverse groups of people based on ethnicity, sexual orientation, and other identity factors.
just work hard as much as they can
You should tell them that they were lied to as most of the “advice” no longer applied (or never did) and were completely setup to be displaced.
You should also tell them “better luck next time” as they have to create and find their own luck unfortunately.
This game was meant to be unfair because the ones who have “won” want to keep it unfair for others, and cannot stand losing due to complete greed.
The truth is they (students) have to find a way to outsmart the incumbents. The best advice is to find your own strategy and listen to no-one.