> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.
METR reran the study early this year and, while they caveat it, this time they found a speedup, which is consistent with subjective estimates of productivity also having increased -- the simplest explanation is that subjective estimates exaggerate, but there's still a speedup with current models: https://metr.org/blog/2026-02-24-uplift-update/#wider-adopti...
(Nobody seems to cite the followup since it's not such a fun counterintuitive finding.)
Have you looked at the raw data of the follow-up? It doesn't paint a pretty picture, many tasks attempted but not completed, refusal to attempt without AI, higher overall estimates and completion times than the previous study (both for AI and Non AI).
There's no definitive conclusion about productivity that can be drawn from the followup. However I think it's reasonable to argue that AI use overtime has made devs worse without AI.
"Unfortunately, given participant feedback and surveys, we believe that the data from our new experiment gives us an unreliable signal of the current productivity effect of AI tools."
That's more than a caveat. That says, this study is unreliable. And why? "The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI."
So, maybe people are fooling themselves, just like this article suggests (I almost wrote asserts, but saying, "I think..." isn't really an assertion)
The people who chose to use AI would be the ones who would be fooling themselves, right? But they measured actually positive productivity there.
METR's concern makes most sense if it's about the inability to measure a consistent trend over time, not about the validity of the study for the population that participated.
> The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI.
Why do you think this is? Who wants to spend an afternoon gluing API calls together when the robot can do it in 5 minutes?
Don't let the downvotes get you down. They don't know who funds or constitutes METR. If you actually read these papers, you'll find they cite themselves 4-5 times each publication. They also consider "hosting an HTTP server using Python" to be a "long time horizon" task.
The OP cited a METR study... if you don't think they're credible, fine, but it's hard for me to see how it could be intellectually honest to cite them when their research is helpful to your argument and discredit them when it's not. One might even call this approach "fooling yourself".
The OP is not citing their claims, but their commentary. Do you understand why I find that to be different than citing their findings? We're not talking percentages here.
By now, anyone writing articles in 2026 but ignoring the 2026 study is either intentionally trying to mislead people, or so uninformed on the topic that they're just regurgitating old anti-AI content.
If you want to talk about studies, we need to be talking about current studies with current tools. Not studies about how the situation was in the past with different tools.
I think there's a simple dichotomy based on use cases. For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear. But for things you aren't good at, they're zomg amazing. So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey what's all the hype about?' But if you're doing things outside your domain then it's a revolutionary game-changer.
This also explains why an independent dev can proclaim a 10x productivity boost or whatever, and actually seem to be showing that - while major companies dumping obscene amounts of $$$ on tokens don't seem to be have much to show for it.
> So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey what's all the hype about?' But if you're doing things outside your domain then it's a revolutionary game-changer.
They may also be seen as a revolutionary game-changer by people who actually kinda suck at their job.
And a lot of software engineers actually kinda suck at their jobs.
> For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear.
Preposterous. I can tell 5.6 Sol to do something like "optimize this entire subsystem to have no ongoing memory allocations, fix these 5 bugs, implement these 2 features" and go work on other code or do something else entirely, and it does it all flawlessly. All things I could have done myself but nowhere near as fast and right the first time with such thorough test cases. We've reached the point where the AI is usually faster and always less effort overall compared to the old days, even if you have to send a few followup prompts.
Validation is really hard and requires a great understanding of the entire relevant code.
So to answer your question, they don’t know or asked the AI to do that too which has very obvious problems.
I find it hard to believe they would spend the effort and money making an AI do tasks like that, but are ALSO willing to spend the effort and time to properly validate the changes. I’m sure many do… but probably not a large percentage.
In my case it's usually a video game in question, not a life support system, so I'm not poring over every line of code.
I mainly look at:
- The overall architecture to make sure it's in line with what I want
- The points where it interacts with certain other systems I'm concerned about
And I test the feature myself in addition to the test cases. Sol has proven to be outstanding at good test cases and having a coherent view on overall architecture integration. I basically never think to myself "this entire implementation is slop, I need to rip it out and refactor it" anymore.
I suspect the model's capacity to translate requirements into contextually appropriate test cases, and insistence on thoroughly covering functionality in tests, is a large part of its strength.
This is going to sound accusatory but I promise it isn’t.
Did you feel the same way about 5.5? The same general sentiment keeps being expressed with every model release. The prior generation is immediately cast aside with a vague “well yes, actually we didn’t mean it last time” attitude.
I think I even heard Theo the T3 guy suggesting that his work comes to halt if he can’t use the latest generation model, despite heaping significant praise on each generation previous and suggesting it does all of his programming tasks.
The boundless energy with which each model generation is said to be revolutionary and leaving everyone behind is tiresome even when incremental progress is real and measurable.
Prior to 5.5 I thought the models were very useful but not trustworthy for certain tiers of tasks and needed more in depth verification. Frequent hand edits and times where it felt more productive to just finish something myself than keep rolling the dice on the model not subtly messing it up. But still great for targeted things.
5.5 on Extra High I found to be a huge leap in what it could tackle and how effectively but only up to a point. Still had common failure modes here and there but a big improvement
When I tried 5.6 I wasn't expecting more than an incremental upgrade, but it blew me away with how "smart" it seemed, how it completely sidestepped failure modes I'd been used to dealing with in 5.5, its success rate on the first try for wide-sweeping architectural tasks, and how I've basically never seen a bad hallucination make it through to the final output. It's the first time I've felt like if I left it with a task on the highest setting and it had any way to verify its output, and told it to keep going, it could solve almost anything. This could all be incremental upgrades in many different areas of the model+harness, but the cumulative effect feels like crossing that "escape velocity" threshold. The point where there are no more rough edges and it "just works".
I can't speak to others, but my feelings on all of these have been consistent since I tried them in the first place. I've also heard model providers draw people in at each new release with the model at full performance and then degrade it over time hoping users won't notice, which has seemed plausible anecdotally but I have no hard proof.
Which ones? That there is no evidence (not personal anecdotes but the numbers) of productivity increase? Or that our subscriptions are heavy subsidized (then try to compare produced value against real price of tokens)? Or that AI companies are burning tonn of cash without clear plan for monetization?
Article has good points which are worth discussing rather than discarding. We need to bring arguments from both sides of the fense.
> That there is no evidence (not personal anecdotes but the numbers) of productivity increase?
The blog tries to cite a 2025 METR study as evidence of no productivity increase, but they ignored the newer 2026 study by the same group that did show a productivity increase with the newer tools.
I also think it's funny that there have become these hard demands for studies and proof of increased productivity. We never saw the same standard of evidence applied to previous advancements like different programming languages or using git for version control. We don't argue with people when they say they're personally more productive in emacs than in VS Code or vice versa. It's only when the topic of AI comes up that the bar for evidence gets raised to the sky
Because those tools were adopted organically by users because they were obviously good. AI is pushed from leaders onto users despite the users not thinking they are good. If my boss decides the whole company is switching to emacs I think he should have some evidence to support that decision, yes.
That phenomenon is more rare than what I mentioned. Every* tech company is forcing AI. A few AI enthusiasts want more of it. Those are two unrelated situations. I can find developers pushing companies to use F#; that does not mean anything, and is very different from my boss switching the whole stack to F# suddenly. The former is someone wanting to use something he likes, the latter is forcing everyone to use something they may not like and that probably has big effects on the company, so should require evidence that the change is beneficial.
The ones that shout about that one METR study, use it to imply that millions of developers are deceiving themselves in believing that tools they use every day are useful, then admit that they don't like and don't use the tools themselves.
> This is why I'm not using AI. I'm not using AI on moral, ethical grounds.
The funny thing is - they are using AI and they will be using even more AI as it soaks into world economy even more; they just won‘t be using it directly. Well, not for a while at least, until practicality overcomes subjective morals.
There is no longer, in my opinion, any scope for reasonable debate that AI models like Fable 5 are net productivity increases for many, if not most, if not ALL, programming tasks. Anyone can, or should be able to, prove this to themselves in about 5 minutes.
And so if someone like the author is trying to argue the contrary, that leads me to believe they either: (1) have not bothered to spend even 5 minutes validating their belief or (2) do not care to test their beliefs against reality, for whatever reason. Either one is a bit of a red flag.
I just attempted to verify this myself, but they want my hard earned dollars first. I didn't see any satisfaction guarantee or anything. That already feels like a scam.
You know, if I held an important opinion about something, and there was a fairly easy way to conclusively validate or falsify it one way or the other, and all it was going to take was a couple of bucks, I'd just pay it. Having good info about the world I live in is important to me.
Why do you think so many people who have used the same models you do, who are otherwise intelligent, and who should be able to prove to themselves your position in five minutes, don't?
I'm somewhat agnostic about the whole thing, and I find the utter confidence you display strange and arrogant. Some really good devs say AI is not that great. Some other really good devs say it is great. Why is one side of them automatically delusional? Could they know something you don't?
> Why is one side of them automatically delusional? Could they know something you don't?
I wrote and deleted a couple of answers to this - they didn't satisfy, so I'll just be honest.
No, they don't know anything I don't know. There's now only one reason for denying that AI is helpful in software engineering, and that is emotional, be it fear of change or cultural point-scoring.
The only honest answer to the question, "Is modern frontier AI helpful in most software engineering?" is "Yes, of course it is." Anyone answering differently than that is, by now, simply wrong. Why they are wrong may vary, but they are wrong.
I disagree and I think the distinction is what we mean by "helpful in software engineering".
Is Fable helpful in that it can write some code that does what I told it to do? Yes, obviously. I don't think anyone is arguing that, because it's trivial to see that if you tell a frontier model to do a coding task, it usually does it well enough.
Is Fable helpful in that over the span of six months of using it, the codebase and my brain's relation to it is in a better state than if I had not used it? Well, I don't know. In this case I understand the code less, and the code is probably in a poor state, but it is further along in the feature list. Whether that tradeoff is a net benefit to whatever metric you care about is debatable.
I think most will agree that the extreme case in which you understand zero and the codebase is spaghetti (true vibecoding), but it has a bunch of features, is not good for most professional contexts, because we have seen this results in a high bugrate and eventual loss of trust in the tool and inability for the AI to maintain invariants due to context size.
So somewhere on the spectrum between no-AI and vibecoding is probably ideal. Where on the spectrum is the argument, and everyone is going to have their own opinions based on their own domains. I don't see this as anyone being wrong or emotional.
> it's trivial to see that if you tell a frontier model to do a coding task, it usually does it well enough
So you do not disagree? You agree 100%? Yes, it is trivial to see. Hence my near-anaphylactic reaction to articles like this, which specifically claim that it is NOT useful. Do you just mentally filter that out or something?
> Is [..] the codebase and my brain's relation to it is in a better state than if I had not used it?
But now this is a different question. It will not be the same, that's for sure. With new tools we must learn new patterns, new best practises. I have come around to the opinion that we must collectively move to a higher level of abstraction and accept that we will understand the code less - but I don't agree it must necessarily be worse. That will all depend on the new techniques we discover and apply.
> So somewhere on the spectrum between no-AI and vibecoding is probably ideal. Where on the spectrum is the argument, and everyone is going to have their own opinions based on their own domains. I don't see this as anyone being wrong or emotional.
I think that anyone planting a big flag at 0 (no AI) in 2026, with what we have available, is indeed doing it for emotional reasons, and when they claim there's no benefit being anywhere else, they're wrong. Which is what the article we're talking about is doing - hence my, and other's, scathing reaction.
"do not fool yourself" proceeds to fool himself... if you are honest about your bias that's still fooling yourself if your conclusions are just demonstrably wrong.
Using LLMs in a similar way to already existing IDE features like refactor, find/replace, search is unambiguously a productivity increase. Whether using them for actual task completion purposes is good or not is unknown.
But imagine that Ctrl+F was just introduced as a feature and you had a bunch of people saying "All these people saying Ctrl+F is increasing their productivity are fooling themselves", it's just a silly thing to say.
The entire thing is an AI skeptic saying “I’m skeptical I found some outdated anecdata that supports my point”. Feynman’s quote is meant for the author of this post, not people using AI to execute better day to day. It’s not reasonable to point at a chatgpt subscription and say “look the entire industry isn’t sustainable”. We all know those are subsidized just like uber was. And there are plenty of people paying API rates for tokens and getting along just fine. Further, if there was a single point in history where an environmental concern killed an entire industry please point me to it. What happens in reality is that humans compromise and innovate under new sets of constraints.
It’s not a consideration. I am more productive with AI. Period. Maybe that’s not the reality for you, but it is for me. I don’t need a study to prove it.
Ah - near the bottom where he is disclosing his own bias. Got it.
So do you disagree that AI companies are in very large amounts of debt? And do you also disagree that debt will have real consequences (and one of those consequences might be either the lights going out or a buyout or merger)?
Not at all. And to be honest, I don't hear the original author asserting that all AI will go away.
But if you have a ChatGPT plan and OpenAI is bought by another company (because they are insolvent) ... do you honestly think you will be able to continue using your service without a significant price increase?
I suppose that would be up to the new owner? How would I know what happens to their services? You shouldn't conflate AI-the-technology with OpenAI-the-company. I'm super bullish about the tech, couldn't care less about a specific company. I mean, I hate Oracle like any other sane person, but that doesn't mean I hate databases.
I'm very confident that AI will continue to be available no matter what happens, at reasonable prices. We don't have to guess - we know what the open source models take to run, the hardware, the electricity, everything. They are served at a profit today - no-one is subsidizing them. Anyone can run GLM5.2 or (soon) K3 etc - you have capex and opex, you can serve X tokens per second, you sell them at Y price, it's just maths.
AI is a technology, not a company or group of companies.
Zitron has been predicting an imminent bubble pop since 2023 (although in less and less falsifiable terms), as OpenAI and Anthropic revenues have steadily grown 3-10x per year, but people keep listening. Zitron has done a lot to misinform the public but at some point you have to conclude it's demand-driven, and people won't re-evaluate his credibility because he tells them what they want to hear.
Right, I didn't mention anything about profit or debt.
The question is whether this is a bubble that's going away - whether "AI will just be turned off". A revenue trajectory indicating a vast increase in actual consumer/enterprise usage every year indicates otherwise. This is true regardless of whether or not forward-looking investment becomes too much at some point. If CSPs or AI labs have bought more compute than they can sell at current prices, maybe they'll go bust, but at that point supply and demand mean compute and therefore AI inference becomes much cheaper, which is unlikely to make it go away.
I've been seeing a lot of this in online discourse where people are like "the bubble's gonna pop and then this will all be over!"
it's a combination of wishlful thinking and straightup delusion to think that AI will just poof and disappear (maybe the specific LLM approach eventually, but not the general idea of AI/the industry/product as a whole)
I had the same reaction. It is the same kind of argument people use to claim that computer science is useless because, “What would you do if the electricity went out?”
- a lot of folks use ai poorly and innudate you with slop, reducing your own efficiency
- because you can just prompt it doesn't mean you should. many let the machine crank out tokens until it works, rather than thinking about the problem first.
of course, if AI solves my second point automatically, it'll solve the first as well, and we'll call that AGI. But we don't have that yet.
I feel sorry for anyone who isn’t experiencing the gains I’m experiencing. I can see why AI could suck in massive teams, or software with very messy and inconsistent architecture. As a 2 man developer team working on a mono repo with clear and simple architecture, consistent schema conventions, and heavy context available, it’s becoming ludicrous how fast we can pump out stable features. We must be 3-4x faster at this point. I had estimated that I would have to take on at least 2 or 3 more developers at X customers, now it seems at even double that quantity of customers we’ll be absolutely fine with just 2 developers. It’s been a tremendous boon for my small business, beyond any possible imagination. And then all the time I hear the claim that there’s no productivity gain to be had. I guess me and my customers are living some kind of mass delusion.
Just the other day there was data corruption discovered and a relatively urgent fix required. Just a few years ago I would have been working deep into the night to make sure everything was fixed for the morning. Within an hour Fable had it all wrapped up neatly and the fix deployed…
Are you claiming you’ve never been responsible for a bug that caused data corruption? Perhaps not, but it’s hardly a novel post-LLM bug. It’s regular stuff and obviously you know that.
As a 2 man developer team working on a mono repo with clear and simple architecture, no I have never caused urgent data corruption or seen it happen. That is more something that happens in codebases that are too big and complex to fully grasp. That it has already happened to you in this scenario does not bode well for the future stability of your codebase, in my opinion.
I find it fascinating how humans search for meaning in the unlikeliest of places, and how ancient tribal instincts will turn anything, anything at all, into an excuse for an in-group, an out-group, and a search for kinship on the most ridiculous little hill.
This "Anti AI Social Club" is just the latest example, and reality has nothing to do with it. Denying reality might just be the whole point.
I find it fascinating how humans search for meaning in the unlikeliest of places, and how ancient tribal instincts will turn anything, anything at all, into an excuse for an in-group, an out-group, and a search for kinship on the most ridiculous little hill.
This "Pro AI Social Club" is just the latest example, and reality has nothing to do with it. Denying reality might just be the whole point.
It's a very human thing really.
Zitron might be right about the economics of it, maybe it really does come crashing down. But we won't be forgetting about it. Even if AI stopped progressing today, it will still be massively, undeniably, insanely useful.
I could use current models forever. LLM does things for me that couldn't be done in 2023. I don't want to go back to a world without it, but I'm also ok with progress stalling right here.
As for the productivity benefits, I find there is a kind of skeptic that just has his eyes closed. Open an AI page, ask it to write you an app that renames all your scientific papers so they have their title as the filename instead of the weird long number.
Without AI you could easily waste an hour looking for the right PDF library and working out how to use it. Or fiddling with the file search mechanism so that you only get the articles and not every file in your downloads. Now you will have the files renamed in a few minutes, and a tool that keeps them correct every time you have download some more.
How is this not a productivity win? It clearly is. Do we really need a double blind test to check this kind of thing?
Whether it gets squandered in modern orgs is another matter, but the core win is clear for everyone to see.
Taking your anecdotes at face value for a moment (which I should not), how does this benefit of AI you describe justify the nearly 1 trillion dollar investment?
Do you think that your perceived increased productivity offsets the true, unsubsidised cost of AI?
Zitron is a hack who sells anger to people who don't know any better. Just because he's right about some things doesn't mean you should give him the time of day.
I think it really depends on your knowledge and skill, and the kind of work you are using it for. It's hard to measure because the more you use it the more you lose some skills. But I have to admit that I can do much more parallel work when using the best models. It's messy, because sometimes I feel that I get lazier and just prompt them and go do some random stuff and sometimes even skip the part of verifying what they actually did meanwhile. But you can command AI to do massive stuff, you can get smart non developers to do useful work with them, it's really a new paradigm. One funny thing that I think people should consider more is pride in the code. If I code something by hand, like fully developing a service, I will get proud of seeing it work, I will attach to it like if it's my creation. That hardly happens if I use AI a lot.
Yes! Everything else aside, whatever you think of its output and value, shipping largely AI-coded features now doesn't really give me the satisfaction that putting things together myself used to.
I'll just have to move up a level of abstraction to get that same feeling, I guess.
I think it's very productive in areas that aren't economically impactful. Reverse engineering old windows games for example. Doing absurd ports between languages.
Now imagine if this was written about the first steam engines being used inefficiently to pull coal up out the ground and pump out the water. It's so expensive -- is this really more effective than just hiring 50 men? We all know how that ended up...
That kills all arguments from the article instantly.
December 2025 through today GenAI has become massively better. I’ve built things with Claude in contracted spans of time that would have been man-years in the before times.
>> although participating developers felt they completed tasks faster using AI, they where around 19% slower
> Maybe that was true in 2025, but it's clearly not true any more - I feel so productive now!
Even if you are right, this is a completely baseless argument. The people in the study felt the exact same way! And "it's 6 months old, so I won't listen to it" is an insanely high standard. You're eliminating all the evidence!
That study…I would have agreed with the group at that time. Up until December, I was vocally skeptical that GenAI could be used for anything more complex than a “departmental” app.
Opus and similar models completely invalidated that skepticism, especially when context windows increased.
In the last two weeks I designed and deployed a new programming language that compiles in a browser. Impossible in the before times.
I think you might be fooling yourself. Personal-plan subscription subsidies do not equate to "it's not viable, and all of it is about to be shut off." If that were true, then Kilo Code, Claude Code, Cline, Pi, and Command Code would not be occupying 5 of the top 10 apps using OpenRouter. People have purchased 20 trillion tokens this month already for coding on OpenRouter alone at full-price market rates.
Would you commission a study about whether cars are really a faster form of locomotion before deciding to purchase one?
There is empirical data right in front of your face. You're just choosing to exclude it because of where you already stand.
We've been debating how to measure software engineering productivity since long before LLMs. It depends on so many different factors. Scrum is sort of the only tool we have that comes close, but we all hate it. Aside from that, all we really have are personal anecdotes.
Here are my personal anecdotes for the week. I occasionally ask AI to write some SystemVerilog test bench code. This is an area where I have deep and long experience. AI has never once saved me time with that. Just yesterday, however, I was writing a tricky Makefile, something I rarely do, and AI helped a lot.
Aside from productivity, this article mentions ethical and financial issues with AI. That doesn't stick with the subject that the title of the essay implies, but it's a more concrete thing to discuss and I wish we'd spend more time on that. I think that's where people really are fooling themselves.
This is the perfect archetype of an intellectually lazy anti-AI post. The author ends with his admission of bias, but he'd be better off starting with it, as it's clear that his reasoning flows entirely from it. AI is bad for productivity, says this single study from 2025. AI is unsustainable, says this article I found. AI will all just disappear one day... because I hope so?
Ironically, the author would have been better asking an LLM to write the article. At least it would have found better sources and developed more convincing arguments.
The study itself from late 2025 is also written to be _highly_ favorable to non-ai usage. Developers on a codebase they're already experienced with, largely using these tools for the first time. Of course they aren't faster! They've already done the hard part! They're also open-source maintainers, which, ime, means they're already a cut above the average dev.
I'd be much more curious on a study that used regular developers on brand new-to-them codebases for significant features or migrations. I'm personally encountering what I'd describe as a 200%+ productivity boost.
I read this headline and thought "METR study"... and sure enough, a few paragraphs in:
> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.
I think you might be fooling yourself if you build your entire worldview concerning the productivity benefits of AI-assisted programming around that one study from one organization that confirms your priors.
>I'm quite influenced by Ed Zitron's reporting. And I'm also standing by my own idea that AI will run out of money and just be turned off due to the huge operational cost.
I wish this were how things are going but Uber is still surviving. I figure at least one AI company is going to come out of this alive and kicking.
1. I am not yet convinced that I am more productive with AI for complex tasks. But, I am fairly certain for trivial tasks, or bureaucratic tasks AI has lessened my cognitive load (data point of one).
2. Does AI run on money, or does it run on energy? Money is the means by which we facilitate exchange. But in reality, the energy in the universe is ginormous. The fundamental thing to keep AI going is available in excess, all other things are negotiation.
3. Is AI bad for the environment or is it the way we obtain energy bad for the environment? The trend seems to be that energy is getting cheaper and cleaner. The world is not static, but will move to accommodate a world with AI.
I agree with the point about we don't have clear/perceivable increase in value I.e. companies are not 10x profitable due AI.
But, I don't understand the position of that has no value or that is going to be "turned off", that is just doomsday prophecy at this point.
We focus so much in code, but AI have changed many if not most of the work. It made possible to take a chance in challenges that would once be dismissed of the bat.
I do think we are still get used to the new ways of work, which IMO is a moving target. Once that we overcome that, results may come clear, but that may not be in companies making 10x more revenue.
From TFA>>I'm open to suggestions how this is going to play out. Close to a trillion dollars invested in AI. Where will the money come from to recuperate all these investments?
If investors are looking for a 50% return, they need the investment to be worth about $1.5 trillion. If they're looking for a 100% return, it needs to be worth about $2 trillion. That's still an enormous hurdle, but it's much smaller than suggesting AI needs to somehow generate trillions of dollars in cash simply to recoup the original investment.
AI costs aren’t going to stay this high. They will go down. Nobody is going to turn off AI, but they will care about making sure that we are reasonably efficient when using it. Compute that can happen locally will. Communities that responsibly want to build data centers are. Climate implications will be addressed by policy. People said the same about the internet 30 years ago… It’s not about raw productivity. It’s about perceived effort. If a job feels easier with AI assistance we’ll demand it. If AI checking a human’s work is more reliable than a human checking AI’s work (it is) that’s what we’ll deploy.
> AI costs aren’t going to stay this high. They will go down.
People said this a year ago, then all the companies admitted they were incinerating cash and had to increase to token pricing. Consumers basically mutinied and managed to push back a bit, but that is the struggle right now - to keep prices from increasing 25+ times, not to see them decrease.
If you're premising this on some future technology like ASICs, even if that works out then we probably shouldn't be spending a trillion dollars on datacenters that it will make obsolete.
Why do you feel this way? Are you talking about frontier models from US companies or smaller weight open source models?
From the perspective of US companies, it seems pretty clear they're severely under water and will need some way to become profitable or risk collapse, so how else will they do that or they than jacking up prices or some unknown future miracle advancement?
I could see the argument applied to self-hosted/OSS models tho.
> AI costs aren’t going to stay this high. They will go down.
I hear this a lot and it seems to be far too optimistic. Call me skeptical but plan pricing continues to increase with tighter usage constraints. We’re still at the point in time that companies like Anthropic, OpenAI, and GitHub charge seemingly reasonable prices in order to get people hooked onto AI before raising prices to get their return on investment.
> And I'm also standing by my own idea that AI will run out of money and just be turned off due to the huge operational cost.
Thats a pretty strong outcome. It implies that not only are GPUs that power AI too expensive long term, but they cost too much to operate even if they were free.
Seems to me the more likely outcome is a wave of dotCom style bankruptcies wiping out equity holders for companies who contracted to buy chips and datacenters at MSRP, and a second wave for the groups that step in after to operate whats left without the absurd financing charges and lower capex.
They are free. You can download glm-5.2 and run it on your own hardware. Even you got hardware for free, electricity would cost you more than the sub, in most places.
Many improvements have come to AI that have reduced costs of inference and training, like sparse attention, and many more will arise in the future. To talk about the cost of AI right now versus the future is to talk about the first IBM computers versus the personal computing that appeared years later. The cost will go down. In fact that is what the major labs are definitely banking on.
Author does not draw full enough picture (by linearly projecting current trends). But still has good points. I liked the one where perceived "productivity" is compared against unsubsisdized token prices.
Eh, I feel the productivity argument kinda depends what sort of project you're using it for. If it's a local tool that only needs to solve a very specific problem you personally have, or you're treading well trodden ground represented in the data set (like WordPress plugin or React component development), you could probably just let an LLM handle almost everything and get a perfectly adequate solution in minutes.
And if you don't know anything about programming, or don't care that the solution it'll come up with is potentially flawed, then it's a lot quicker to vibe code up some sort of hack job answer than learn how to do everything manually. I'm pretty sure that my non-technical parents or grandparents could ask Claude or ChatGPT to build a simple website for their small business or charity, and have something workable within an hour or two.
For a software engineer, AI might not make them any more productive. For a non-engineer, it's basically magic.
The unsustainability thing is more of a concern though, and the external effects of AI are a huge problem. It's just that pure productivity wise, a lot of people will be significantly more productive with these systems than without them.
There are definitely many open questions around things like what tasks is AI appropriate for, how involved should you be in the process (fully automated agent swarms, or planning steps back and forth with human input on every decision), at what level of usage might you hit diminishing or negative returns, will we be able to continue to use it at this rate / price, will it get cheaper or will energy use continue to rise, how much will AI improve in the future, how will AI use affect people's development or temperament, will skills atrophy, will people no longer learn critical skills, how will we deal with AI spam or a loss of "proof of work" in producing a lengthy written artifact.... in sum there is no shortage of AI related topics worth exploring and thinking deeply about.
Some people still suggest that AI flat out isn't useful though, and that angle to me doesn't hold much water. I was a slow AI adopter, but I have found plenty of benefit from using AI in various applications recently. I haven't come close to using on the level of some people who orchestrate agents and rarely read or touch the code they are writing, but there have been clear times when I popped into Claude and had it help me very successfully with something that would have taken longer to do without that help. So increasingly I am skeptical of anyone positing that there is no potential benefit to appropriately using AI assistance. I think the author could rewrite around the angle of their perceived negative externalities outweighing potential advantages, but to deny or ignore those potential benefits ultimately distracts or actively takes away from a serious discussion of said negative externalities.
I don't put much stock in the very very frequently cited study about AI productivity included in this blog. It's a single study. The study itself cites multiple other studies that did find a productivity boost in writing software with AI assistance. I don't see the methodology as being very sound, the sample size isn't very big, some of those developers had likely never used AI before so of course they were slower as they explored using a new tool for the first time (which the study more or less writes off as a non-factor). They also used "Claude 3.5/3.7 Sonnet", which is not the strongest model. I personally find the Sonnet family of models not that useful and much more inaccurate than something like Opus or Fable.
Sure - it was light on the meat, But it's an opinion blog post, not a full article. the author ventures to cite their refs, acknowledges potential bias.. way more than you get with a tonne of front page articles (many of which probably should have been short opinion pieces like this)
Sure - it's not particularly notable or high entropy enough to save, but it's far from offensive or bothersome
...yet all I'm seeing (at time of posting) is defensive dismissals, ad hominems, and echo chamber pablum
Quotes study from early 2025. Same group redid study now shows opposite result. “Fooling yourself” indeed.
Anybody actually using AI in anger (to get paid) will tell you AI was pretty bad early 2025. It wasn’t usable till June. By December it was writing most of my code - with significant reviews by me. Now it’s basically getting it right. YMMV.
I've noticed that you can either move very fast with AI, or very slow. Moving fast, you place a lot of trust in the agent and let it proceed and create something very quickly that works at a surface level and that you don't understand at all. Moving slow you take the time to ask questions and understand every part of the application and learn more about how to code it yourself and make changes as necessary.
One thing I've noticed is, that coding manually is exponentially more stressful than the slow AI approach. I'm not going any faster than I normally would, but the burnout doesn't show up on the fifth day of the week. The hyper-focus is kind of fun but utterly exhausting.
With the fast approach I can create impressive demos and POCs that are nowhere near ready for production. But sometimes that's exactly what I need.
> One thing I've noticed is, that coding manually is exponentially more stressful than the slow AI approach.
I find the exact opposite. Typically the AI will make too many mistakes (even with latest advanced models), they usually aren't major/program breaking mistakes, but they still require fixing if I want to keep the code quality good enough, and make future changes possible without a falling house of cards. In order to describe the correct approach I have to write more English to describe the problem than the amount of code I could write myself. I could maybe see it working by writing rough pseudo code and having the AI translate, but then I loose out on any code analysis or tab completion as I write. It's also just innately frustrating to have to babysit and correct somethings work all the time. This all may be in part caused by the nature of my current work for the last couple years, which has mostly been on library code rather than application code.
I suspect I'd have a very different experience if I was working on CRUD related tasks like making new API endpoints, interfacing with a DB, or writing FE components. Not to make light of those tasks, I enjoy that work, though it is much more repetitive.
I agree with the parts about using AI to ask questions and gain better understanding though. I do that constantly for APIs and languages that I don't have a clear mental model of. It's like an interactive search.
Also agree on POCs, sometimes there are small tools, demos, or prototype features that I don't need to care deeply about.
If I'm working on a non-coercive environment (No deadlines, requirements, etc..) hand coding in hyper-focus mode is extremely zen. But working on a deadline it really hurts. My brain can't help but ratchet up the stress.
AI has been a big boost for my data science productivity.
I can have an agent put together a shitty prototype of something just to get a general idea of what I'm looking at, and save probably half a day of work and avoid physically typing potentially hundreds of lines of code. And I can rubber duck with it, and very quickly search through software packages and literature, produce summary reports, dig through my own company source code and wiki and Slack messages to figure out how some specific detail works, etc. These things are important but take my time and energy away from focusing on the hard parts of what I'm supposed to be doing.
My new favorite thing with AI is that my pile of random helper functions is now a thoughtfully-designed library with an extensive test suite for every single thing. I'm not auditing every line of code but I'm reading the plans carefully and I do take a glance at what the agent wrote before I start using it. Most of the time it's not perfect or it's ugly in some way I wouldn't have done, but it's good enough to do research with, and it's way better than what I could have put together by hand, and in much less time. I trust my own research results more because I can put together these kinds of helper libraries much faster, and if I have to make changes I have a big test suite to catch me when I make a mistake.
The only hard part is managing the context switching where instead of putting my work on pause to go write a helper routine, I now have to account for periodically interrupting my work in order to keep an eye on what the agent is doing. For now my approach has been to just ignore it for a while and then take a break to cycle through AI work sessions/chats.
I agree, the speed at which the agents work is a problem. It creates an interrupt workflow instead of a flow workflow. This works for some people but I notice that I don't do well that way. Typically I'll have an agent working on a task while I use a separate one in Ask mode just kind of satisfying my curiosity while I wait. This keeps me engaged but makes the interrupts less jarring.
I just spent multiple evenings talking through a small data model. With the massive number of misses the model had, I can’t imagine how bad it would have been if I had let it yolo the first design of its choice, to build on.
The requirements didn’t really change, opus just couldn’t see all the real world potential issues with its various suggestions.
But it was helpful for me to talk it out. So I guess it fell in the (very) slow category of productive use.
The rubber ducking potential is the first thing that got me interested in AI as a useful tool.
You do have to be very careful of when the AI might be confidently wrong, or leading you down some kind of weird narrow path.
When searching the web to gather info on a topic, all have a tendency to find 2-4 search results and fixate on those as if they represented the entire scope of the topic. No surprise about the fixation because that's kind of how LLMs work (X, Y, and Z are in context -> more tokens about X, Y, and Z). But you have to be aware of it, and I often have to prompt it to do another search and keep turning up more options.
I've found that my requirements and acceptance criteria are much less clear than I think they are. AI reveals when you are being too vague, almost instantly sometimes.
They always are. When you think you’ve got it all… ya don’t.
Unfortunately this recent experience wasn’t that as much as it producing rapid-fire misses when it did have the constraints. It regularly just violated X to kinda-solve Y. Or made up a solution to both that solves neither. And this wasn’t some wildly complicated scenario.
Sure made me solidify some opinions (we ended up pretty close to where I had started without it), but woo-boy… it was a painful journey.
Yeah, sometimes you just have to throw out the current solution and start over/rollback. AI will go down some weird rabbit holes to solve a problem with the wrong requirements. Sometimes I feel like I'm just coding in English instead of Python or TypeScript.
This, I'm working on a game engine as a hobby project. I can do one "big" feature over a weekend, most of it including designing and manual testing that it's working as expected, but then it takes me 2 weekends to break up the changes into smaller commits using jj, which also gives me the chance to review and understand the code myself. So it's a weird mix of fast and slow, but I think I've grown to prefer reviewing over writing so it works for a hobby project.
> …but then it takes me 2 weekends to break up the changes into smaller commits using jj, which also gives me the chance to review and understand the code myself.
I just tell Claude Claude code, "commit all, grouping logically" after a tranche of work.
I love that you can mix these approaches. It lets me very quickly get a working prototype and then I can focus on optimizing the areas I care about / are having issues.
I think almost anything you can build autonomously with a fully independent AI agent, you can build far better with the slow AI approach. It is a good trade off between the super fast agentic approach and the full manual approach, and leaves you with a codebase you can still reason about and find ways to improve.
Also, once you build something with slow AI, it becomes more practical to actually rewrite it by hand quickly and refactor it in a way that makes more sense for humans. It’s “humanwashing” the code.
tired of this one dimensional articles that paint a wide brush, there are wide variety of AI uses and it absolutely has a productivity multiplying effect.
These AI sceptics always omit the possibility that running LLMs could become much cheaper in the future, which it almost certainly will.
This is largely an infrastructure problem, and it will be solved just as similar problems were solved with storage, network capacity, and computing power.
There will be bankruptcies, but the technology will prevail, just as the web did after the dot com bubble. Personally, I see the loss of craft and practical skill, which can only be built through doing, as a much greater danger. I still practise LeetCode from time to time so I do not lose the ability to write code myself.
> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.
METR reran the study early this year and, while they caveat it, this time they found a speedup, which is consistent with subjective estimates of productivity also having increased -- the simplest explanation is that subjective estimates exaggerate, but there's still a speedup with current models: https://metr.org/blog/2026-02-24-uplift-update/#wider-adopti...
(Nobody seems to cite the followup since it's not such a fun counterintuitive finding.)
Have you looked at the raw data of the follow-up? It doesn't paint a pretty picture, many tasks attempted but not completed, refusal to attempt without AI, higher overall estimates and completion times than the previous study (both for AI and Non AI).
There's no definitive conclusion about productivity that can be drawn from the followup. However I think it's reasonable to argue that AI use overtime has made devs worse without AI.
"Unfortunately, given participant feedback and surveys, we believe that the data from our new experiment gives us an unreliable signal of the current productivity effect of AI tools."
That's more than a caveat. That says, this study is unreliable. And why? "The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI."
So, maybe people are fooling themselves, just like this article suggests (I almost wrote asserts, but saying, "I think..." isn't really an assertion)
The people who chose to use AI would be the ones who would be fooling themselves, right? But they measured actually positive productivity there.
METR's concern makes most sense if it's about the inability to measure a consistent trend over time, not about the validity of the study for the population that participated.
> The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI.
Why do you think this is? Who wants to spend an afternoon gluing API calls together when the robot can do it in 5 minutes?
Don't let the downvotes get you down. They don't know who funds or constitutes METR. If you actually read these papers, you'll find they cite themselves 4-5 times each publication. They also consider "hosting an HTTP server using Python" to be a "long time horizon" task.
The OP cited a METR study... if you don't think they're credible, fine, but it's hard for me to see how it could be intellectually honest to cite them when their research is helpful to your argument and discredit them when it's not. One might even call this approach "fooling yourself".
The OP is not citing their claims, but their commentary. Do you understand why I find that to be different than citing their findings? We're not talking percentages here.
By now, anyone writing articles in 2026 but ignoring the 2026 study is either intentionally trying to mislead people, or so uninformed on the topic that they're just regurgitating old anti-AI content.
If you want to talk about studies, we need to be talking about current studies with current tools. Not studies about how the situation was in the past with different tools.
These articles are so off from reality to the point that it can't be taken seriously at all. I keep wondering how they keep popping at HN.
At least the author admit he is biased..
I think there's a simple dichotomy based on use cases. For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear. But for things you aren't good at, they're zomg amazing. So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey what's all the hype about?' But if you're doing things outside your domain then it's a revolutionary game-changer.
This also explains why an independent dev can proclaim a 10x productivity boost or whatever, and actually seem to be showing that - while major companies dumping obscene amounts of $$$ on tokens don't seem to be have much to show for it.
> So for somebody evaluating things based on what they do at work (where they're probably quite competent) or on personal projects well within their own domain, then it's 'hey what's all the hype about?' But if you're doing things outside your domain then it's a revolutionary game-changer.
They may also be seen as a revolutionary game-changer by people who actually kinda suck at their job.
And a lot of software engineers actually kinda suck at their jobs.
> For things you're already highly skilled at LLMs can be handy but the overall gains, after all is accounted for, are not so clear.
Preposterous. I can tell 5.6 Sol to do something like "optimize this entire subsystem to have no ongoing memory allocations, fix these 5 bugs, implement these 2 features" and go work on other code or do something else entirely, and it does it all flawlessly. All things I could have done myself but nowhere near as fast and right the first time with such thorough test cases. We've reached the point where the AI is usually faster and always less effort overall compared to the old days, even if you have to send a few followup prompts.
> it does it all flawlessly
Serious question: How do you assess this? It seems to me that it would take very significant time to establish that conclusion.
Validation is really hard and requires a great understanding of the entire relevant code. So to answer your question, they don’t know or asked the AI to do that too which has very obvious problems.
I find it hard to believe they would spend the effort and money making an AI do tasks like that, but are ALSO willing to spend the effort and time to properly validate the changes. I’m sure many do… but probably not a large percentage.
In my case it's usually a video game in question, not a life support system, so I'm not poring over every line of code.
I mainly look at:
- The overall architecture to make sure it's in line with what I want
- The points where it interacts with certain other systems I'm concerned about
And I test the feature myself in addition to the test cases. Sol has proven to be outstanding at good test cases and having a coherent view on overall architecture integration. I basically never think to myself "this entire implementation is slop, I need to rip it out and refactor it" anymore.
I suspect the model's capacity to translate requirements into contextually appropriate test cases, and insistence on thoroughly covering functionality in tests, is a large part of its strength.
This is going to sound accusatory but I promise it isn’t.
Did you feel the same way about 5.5? The same general sentiment keeps being expressed with every model release. The prior generation is immediately cast aside with a vague “well yes, actually we didn’t mean it last time” attitude.
I think I even heard Theo the T3 guy suggesting that his work comes to halt if he can’t use the latest generation model, despite heaping significant praise on each generation previous and suggesting it does all of his programming tasks.
The boundless energy with which each model generation is said to be revolutionary and leaving everyone behind is tiresome even when incremental progress is real and measurable.
Prior to 5.5 I thought the models were very useful but not trustworthy for certain tiers of tasks and needed more in depth verification. Frequent hand edits and times where it felt more productive to just finish something myself than keep rolling the dice on the model not subtly messing it up. But still great for targeted things.
5.5 on Extra High I found to be a huge leap in what it could tackle and how effectively but only up to a point. Still had common failure modes here and there but a big improvement
When I tried 5.6 I wasn't expecting more than an incremental upgrade, but it blew me away with how "smart" it seemed, how it completely sidestepped failure modes I'd been used to dealing with in 5.5, its success rate on the first try for wide-sweeping architectural tasks, and how I've basically never seen a bad hallucination make it through to the final output. It's the first time I've felt like if I left it with a task on the highest setting and it had any way to verify its output, and told it to keep going, it could solve almost anything. This could all be incremental upgrades in many different areas of the model+harness, but the cumulative effect feels like crossing that "escape velocity" threshold. The point where there are no more rough edges and it "just works".
I can't speak to others, but my feelings on all of these have been consistent since I tried them in the first place. I've also heard model providers draw people in at each new release with the model at full performance and then degrade it over time hoping users won't notice, which has seemed plausible anecdotally but I have no hard proof.
It has taken me a few months to get productive with them. I did not pick it up easily.
> These articles are so off from reality
Which ones? That there is no evidence (not personal anecdotes but the numbers) of productivity increase? Or that our subscriptions are heavy subsidized (then try to compare produced value against real price of tokens)? Or that AI companies are burning tonn of cash without clear plan for monetization?
Article has good points which are worth discussing rather than discarding. We need to bring arguments from both sides of the fense.
> That there is no evidence (not personal anecdotes but the numbers) of productivity increase?
The blog tries to cite a 2025 METR study as evidence of no productivity increase, but they ignored the newer 2026 study by the same group that did show a productivity increase with the newer tools.
I also think it's funny that there have become these hard demands for studies and proof of increased productivity. We never saw the same standard of evidence applied to previous advancements like different programming languages or using git for version control. We don't argue with people when they say they're personally more productive in emacs than in VS Code or vice versa. It's only when the topic of AI comes up that the bar for evidence gets raised to the sky
Because those tools were adopted organically by users because they were obviously good. AI is pushed from leaders onto users despite the users not thinking they are good. If my boss decides the whole company is switching to emacs I think he should have some evidence to support that decision, yes.
> AI is pushed from leaders onto users despite the users not thinking they are good.
Developers are pushing companies to give them access to AI tools and increase their token budgets.
That phenomenon is more rare than what I mentioned. Every* tech company is forcing AI. A few AI enthusiasts want more of it. Those are two unrelated situations. I can find developers pushing companies to use F#; that does not mean anything, and is very different from my boss switching the whole stack to F# suddenly. The former is someone wanting to use something he likes, the latter is forcing everyone to use something they may not like and that probably has big effects on the company, so should require evidence that the change is beneficial.
> Which ones?
The ones that shout about that one METR study, use it to imply that millions of developers are deceiving themselves in believing that tools they use every day are useful, then admit that they don't like and don't use the tools themselves.
I appreciate how the author is explicit about their bias.
This opinion piece is more what he wants to happen rather than what's actually happening on the ground
From the article:
> This is why I'm not using AI. I'm not using AI on moral, ethical grounds.
The funny thing is - they are using AI and they will be using even more AI as it soaks into world economy even more; they just won‘t be using it directly. Well, not for a while at least, until practicality overcomes subjective morals.
"These articles are so off from reality to the point that it can't be taken seriously at all"
So you disagree with the quote from Feynman? (the quote that forms the basis of the article)
I'm genuinely curious to know what part of the article you feel is "so far off from reality"?
There is no longer, in my opinion, any scope for reasonable debate that AI models like Fable 5 are net productivity increases for many, if not most, if not ALL, programming tasks. Anyone can, or should be able to, prove this to themselves in about 5 minutes.
And so if someone like the author is trying to argue the contrary, that leads me to believe they either: (1) have not bothered to spend even 5 minutes validating their belief or (2) do not care to test their beliefs against reality, for whatever reason. Either one is a bit of a red flag.
I just attempted to verify this myself, but they want my hard earned dollars first. I didn't see any satisfaction guarantee or anything. That already feels like a scam.
You know, if I held an important opinion about something, and there was a fairly easy way to conclusively validate or falsify it one way or the other, and all it was going to take was a couple of bucks, I'd just pay it. Having good info about the world I live in is important to me.
Just saying.
So you'll buy the supplements I'm selling? They'll cure everything that ails you, I promise ;-)
Let's try another tack. What would it take to change your mind?
It's pre-Fable 5 but they did site a real study.
Why do you think so many people who have used the same models you do, who are otherwise intelligent, and who should be able to prove to themselves your position in five minutes, don't?
I'm somewhat agnostic about the whole thing, and I find the utter confidence you display strange and arrogant. Some really good devs say AI is not that great. Some other really good devs say it is great. Why is one side of them automatically delusional? Could they know something you don't?
> Why is one side of them automatically delusional? Could they know something you don't?
I wrote and deleted a couple of answers to this - they didn't satisfy, so I'll just be honest.
No, they don't know anything I don't know. There's now only one reason for denying that AI is helpful in software engineering, and that is emotional, be it fear of change or cultural point-scoring.
The only honest answer to the question, "Is modern frontier AI helpful in most software engineering?" is "Yes, of course it is." Anyone answering differently than that is, by now, simply wrong. Why they are wrong may vary, but they are wrong.
I disagree and I think the distinction is what we mean by "helpful in software engineering".
Is Fable helpful in that it can write some code that does what I told it to do? Yes, obviously. I don't think anyone is arguing that, because it's trivial to see that if you tell a frontier model to do a coding task, it usually does it well enough.
Is Fable helpful in that over the span of six months of using it, the codebase and my brain's relation to it is in a better state than if I had not used it? Well, I don't know. In this case I understand the code less, and the code is probably in a poor state, but it is further along in the feature list. Whether that tradeoff is a net benefit to whatever metric you care about is debatable.
I think most will agree that the extreme case in which you understand zero and the codebase is spaghetti (true vibecoding), but it has a bunch of features, is not good for most professional contexts, because we have seen this results in a high bugrate and eventual loss of trust in the tool and inability for the AI to maintain invariants due to context size.
So somewhere on the spectrum between no-AI and vibecoding is probably ideal. Where on the spectrum is the argument, and everyone is going to have their own opinions based on their own domains. I don't see this as anyone being wrong or emotional.
> it's trivial to see that if you tell a frontier model to do a coding task, it usually does it well enough
So you do not disagree? You agree 100%? Yes, it is trivial to see. Hence my near-anaphylactic reaction to articles like this, which specifically claim that it is NOT useful. Do you just mentally filter that out or something?
> Is [..] the codebase and my brain's relation to it is in a better state than if I had not used it?
But now this is a different question. It will not be the same, that's for sure. With new tools we must learn new patterns, new best practises. I have come around to the opinion that we must collectively move to a higher level of abstraction and accept that we will understand the code less - but I don't agree it must necessarily be worse. That will all depend on the new techniques we discover and apply.
> So somewhere on the spectrum between no-AI and vibecoding is probably ideal. Where on the spectrum is the argument, and everyone is going to have their own opinions based on their own domains. I don't see this as anyone being wrong or emotional.
I think that anyone planting a big flag at 0 (no AI) in 2026, with what we have available, is indeed doing it for emotional reasons, and when they claim there's no benefit being anywhere else, they're wrong. Which is what the article we're talking about is doing - hence my, and other's, scathing reaction.
"do not fool yourself" proceeds to fool himself... if you are honest about your bias that's still fooling yourself if your conclusions are just demonstrably wrong.
Using LLMs in a similar way to already existing IDE features like refactor, find/replace, search is unambiguously a productivity increase. Whether using them for actual task completion purposes is good or not is unknown.
But imagine that Ctrl+F was just introduced as a feature and you had a bunch of people saying "All these people saying Ctrl+F is increasing their productivity are fooling themselves", it's just a silly thing to say.
The entire thing is an AI skeptic saying “I’m skeptical I found some outdated anecdata that supports my point”. Feynman’s quote is meant for the author of this post, not people using AI to execute better day to day. It’s not reasonable to point at a chatgpt subscription and say “look the entire industry isn’t sustainable”. We all know those are subsidized just like uber was. And there are plenty of people paying API rates for tokens and getting along just fine. Further, if there was a single point in history where an environmental concern killed an entire industry please point me to it. What happens in reality is that humans compromise and innovate under new sets of constraints.
"Feynman’s quote is meant for the author of this post, not people using AI to execute better day to day."
Lol! Feynman's quote is in fact for everyone. Refusing to consider that it might apply to yourself is proof of that!
It’s not a consideration. I am more productive with AI. Period. Maybe that’s not the reality for you, but it is for me. I don’t need a study to prove it.
The part that says
> AI will just be turned off.
Ah - near the bottom where he is disclosing his own bias. Got it.
So do you disagree that AI companies are in very large amounts of debt? And do you also disagree that debt will have real consequences (and one of those consequences might be either the lights going out or a buyout or merger)?
Tell me, when the dot com bubble burst, and all those debt-laden companies collapsed and had buyouts and mergers, was the internet turned off?
Not at all. And to be honest, I don't hear the original author asserting that all AI will go away.
But if you have a ChatGPT plan and OpenAI is bought by another company (because they are insolvent) ... do you honestly think you will be able to continue using your service without a significant price increase?
I suppose that would be up to the new owner? How would I know what happens to their services? You shouldn't conflate AI-the-technology with OpenAI-the-company. I'm super bullish about the tech, couldn't care less about a specific company. I mean, I hate Oracle like any other sane person, but that doesn't mean I hate databases.
I'm very confident that AI will continue to be available no matter what happens, at reasonable prices. We don't have to guess - we know what the open source models take to run, the hardware, the electricity, everything. They are served at a profit today - no-one is subsidizing them. Anyone can run GLM5.2 or (soon) K3 etc - you have capex and opex, you can serve X tokens per second, you sell them at Y price, it's just maths.
AI is a technology, not a company or group of companies.
Internet existed before dot com bubble.
> I'm quite influenced by Ed Zitron's reporting
Zitron has been predicting an imminent bubble pop since 2023 (although in less and less falsifiable terms), as OpenAI and Anthropic revenues have steadily grown 3-10x per year, but people keep listening. Zitron has done a lot to misinform the public but at some point you have to conclude it's demand-driven, and people won't re-evaluate his credibility because he tells them what they want to hear.
Useful: https://www.theargumentmag.com/p/ais-biggest-critic-has-lost...
Revenue and profit are not the same.
And you didn't even mention the debt these companies have accrued in the last 3 years.
References:
https://www.tomshardware.com/tech-industry/big-tech/ai-tech-...
https://finance.yahoo.com/technology/article/techs-ai-debt-b...
https://www.forbes.com/sites/robertszczerba/2026/07/17/bond-...
Right, I didn't mention anything about profit or debt.
The question is whether this is a bubble that's going away - whether "AI will just be turned off". A revenue trajectory indicating a vast increase in actual consumer/enterprise usage every year indicates otherwise. This is true regardless of whether or not forward-looking investment becomes too much at some point. If CSPs or AI labs have bought more compute than they can sell at current prices, maybe they'll go bust, but at that point supply and demand mean compute and therefore AI inference becomes much cheaper, which is unlikely to make it go away.
I've been seeing a lot of this in online discourse where people are like "the bubble's gonna pop and then this will all be over!"
it's a combination of wishlful thinking and straightup delusion to think that AI will just poof and disappear (maybe the specific LLM approach eventually, but not the general idea of AI/the industry/product as a whole)
I had the same reaction. It is the same kind of argument people use to claim that computer science is useless because, “What would you do if the electricity went out?”
The same thing as you, man. Die in the chaos.
id say they're not 100% wrong:
- a lot of folks use ai poorly and innudate you with slop, reducing your own efficiency
- because you can just prompt it doesn't mean you should. many let the machine crank out tokens until it works, rather than thinking about the problem first.
of course, if AI solves my second point automatically, it'll solve the first as well, and we'll call that AGI. But we don't have that yet.
I stopped reading at the productivity comment, it's just deeply unserious stuff.
Citing a study published in '25, and thus conducted before that, basically before AI got good.
Yeah this article is safe to ignore
I feel sorry for anyone who isn’t experiencing the gains I’m experiencing. I can see why AI could suck in massive teams, or software with very messy and inconsistent architecture. As a 2 man developer team working on a mono repo with clear and simple architecture, consistent schema conventions, and heavy context available, it’s becoming ludicrous how fast we can pump out stable features. We must be 3-4x faster at this point. I had estimated that I would have to take on at least 2 or 3 more developers at X customers, now it seems at even double that quantity of customers we’ll be absolutely fine with just 2 developers. It’s been a tremendous boon for my small business, beyond any possible imagination. And then all the time I hear the claim that there’s no productivity gain to be had. I guess me and my customers are living some kind of mass delusion.
Just the other day there was data corruption discovered and a relatively urgent fix required. Just a few years ago I would have been working deep into the night to make sure everything was fixed for the morning. Within an hour Fable had it all wrapped up neatly and the fix deployed…
>pump out stable features
>data corruption...urgent fix required
Do you people hear yourselves?
Are you claiming you’ve never been responsible for a bug that caused data corruption? Perhaps not, but it’s hardly a novel post-LLM bug. It’s regular stuff and obviously you know that.
As a 2 man developer team working on a mono repo with clear and simple architecture, no I have never caused urgent data corruption or seen it happen. That is more something that happens in codebases that are too big and complex to fully grasp. That it has already happened to you in this scenario does not bode well for the future stability of your codebase, in my opinion.
I find it fascinating how humans search for meaning in the unlikeliest of places, and how ancient tribal instincts will turn anything, anything at all, into an excuse for an in-group, an out-group, and a search for kinship on the most ridiculous little hill.
This "Anti AI Social Club" is just the latest example, and reality has nothing to do with it. Denying reality might just be the whole point.
It's a very human thing really.
I find it fascinating how humans search for meaning in the unlikeliest of places, and how ancient tribal instincts will turn anything, anything at all, into an excuse for an in-group, an out-group, and a search for kinship on the most ridiculous little hill. This "Pro AI Social Club" is just the latest example, and reality has nothing to do with it. Denying reality might just be the whole point. It's a very human thing really.
Cute, but I'm not the one who wrote the blog post.
Zitron might be right about the economics of it, maybe it really does come crashing down. But we won't be forgetting about it. Even if AI stopped progressing today, it will still be massively, undeniably, insanely useful.
I could use current models forever. LLM does things for me that couldn't be done in 2023. I don't want to go back to a world without it, but I'm also ok with progress stalling right here.
As for the productivity benefits, I find there is a kind of skeptic that just has his eyes closed. Open an AI page, ask it to write you an app that renames all your scientific papers so they have their title as the filename instead of the weird long number.
Without AI you could easily waste an hour looking for the right PDF library and working out how to use it. Or fiddling with the file search mechanism so that you only get the articles and not every file in your downloads. Now you will have the files renamed in a few minutes, and a tool that keeps them correct every time you have download some more.
How is this not a productivity win? It clearly is. Do we really need a double blind test to check this kind of thing?
Whether it gets squandered in modern orgs is another matter, but the core win is clear for everyone to see.
Taking your anecdotes at face value for a moment (which I should not), how does this benefit of AI you describe justify the nearly 1 trillion dollar investment?
Do you think that your perceived increased productivity offsets the true, unsubsidised cost of AI?
Zitron is a hack who sells anger to people who don't know any better. Just because he's right about some things doesn't mean you should give him the time of day.
I think it really depends on your knowledge and skill, and the kind of work you are using it for. It's hard to measure because the more you use it the more you lose some skills. But I have to admit that I can do much more parallel work when using the best models. It's messy, because sometimes I feel that I get lazier and just prompt them and go do some random stuff and sometimes even skip the part of verifying what they actually did meanwhile. But you can command AI to do massive stuff, you can get smart non developers to do useful work with them, it's really a new paradigm. One funny thing that I think people should consider more is pride in the code. If I code something by hand, like fully developing a service, I will get proud of seeing it work, I will attach to it like if it's my creation. That hardly happens if I use AI a lot.
Yes! Everything else aside, whatever you think of its output and value, shipping largely AI-coded features now doesn't really give me the satisfaction that putting things together myself used to.
I'll just have to move up a level of abstraction to get that same feeling, I guess.
I think it's very productive in areas that aren't economically impactful. Reverse engineering old windows games for example. Doing absurd ports between languages.
> very productive in areas that aren't economically impactful
Writing emails and summarizing other emails...
Now imagine if this was written about the first steam engines being used inefficiently to pull coal up out the ground and pump out the water. It's so expensive -- is this really more effective than just hiring 50 men? We all know how that ended up...
> We all know how that ended up...
It ended up with investors losing their shirts time and time again - https://en.wikipedia.org/wiki/Railway_Mania and https://en.wikipedia.org/wiki/Panic_of_1893 for example - but we did get some very useful infrastructure out of it.
That's not really how it "ended up"? Yes, there are always losers when investing in new technologies.
But the steam engine was an amazing invention, and ended up propelling humanity forward.
It ended with global warming from burning too much coal. I'm not sure that's the comparison you are trying to make though
These people keep citing that survey from 2025.
That kills all arguments from the article instantly.
December 2025 through today GenAI has become massively better. I’ve built things with Claude in contracted spans of time that would have been man-years in the before times.
>> although participating developers felt they completed tasks faster using AI, they where around 19% slower
> Maybe that was true in 2025, but it's clearly not true any more - I feel so productive now!
Even if you are right, this is a completely baseless argument. The people in the study felt the exact same way! And "it's 6 months old, so I won't listen to it" is an insanely high standard. You're eliminating all the evidence!
That study…I would have agreed with the group at that time. Up until December, I was vocally skeptical that GenAI could be used for anything more complex than a “departmental” app.
Opus and similar models completely invalidated that skepticism, especially when context windows increased.
In the last two weeks I designed and deployed a new programming language that compiles in a browser. Impossible in the before times.
what evidence? more than half of social science studies fail to replicate, why would I believe that this one is in the good half?
I think you might be fooling yourself. Personal-plan subscription subsidies do not equate to "it's not viable, and all of it is about to be shut off." If that were true, then Kilo Code, Claude Code, Cline, Pi, and Command Code would not be occupying 5 of the top 10 apps using OpenRouter. People have purchased 20 trillion tokens this month already for coding on OpenRouter alone at full-price market rates.
Would you commission a study about whether cars are really a faster form of locomotion before deciding to purchase one?
There is empirical data right in front of your face. You're just choosing to exclude it because of where you already stand.
That's exactly what Feynman was warning about.
It's like looking at the popularity of sports betting and concluding that it's a viable way to make money.
And it is? Look at kindred group for example, they made >700 million GBP last year. seems viable enough to me!
To clarify, I mean for the bettors.
We've been debating how to measure software engineering productivity since long before LLMs. It depends on so many different factors. Scrum is sort of the only tool we have that comes close, but we all hate it. Aside from that, all we really have are personal anecdotes.
Here are my personal anecdotes for the week. I occasionally ask AI to write some SystemVerilog test bench code. This is an area where I have deep and long experience. AI has never once saved me time with that. Just yesterday, however, I was writing a tricky Makefile, something I rarely do, and AI helped a lot.
Aside from productivity, this article mentions ethical and financial issues with AI. That doesn't stick with the subject that the title of the essay implies, but it's a more concrete thing to discuss and I wish we'd spend more time on that. I think that's where people really are fooling themselves.
This is the perfect archetype of an intellectually lazy anti-AI post. The author ends with his admission of bias, but he'd be better off starting with it, as it's clear that his reasoning flows entirely from it. AI is bad for productivity, says this single study from 2025. AI is unsustainable, says this article I found. AI will all just disappear one day... because I hope so?
Ironically, the author would have been better asking an LLM to write the article. At least it would have found better sources and developed more convincing arguments.
The study itself from late 2025 is also written to be _highly_ favorable to non-ai usage. Developers on a codebase they're already experienced with, largely using these tools for the first time. Of course they aren't faster! They've already done the hard part! They're also open-source maintainers, which, ime, means they're already a cut above the average dev.
I'd be much more curious on a study that used regular developers on brand new-to-them codebases for significant features or migrations. I'm personally encountering what I'd describe as a 200%+ productivity boost.
I read this headline and thought "METR study"... and sure enough, a few paragraphs in:
> Meanwhile, a study (late 2025) seems to report that although participating developers felt they completed tasks faster using AI, they where around 19% slower.
I think you might be fooling yourself if you build your entire worldview concerning the productivity benefits of AI-assisted programming around that one study from one organization that confirms your priors.
What study do you base your worldview on?
Four years of personal exploration of the technology.
(I had an argument on Lobste.rs the other day about why only believing aspects of software engineering that have survived peer review doesn't make sense to me: https://lobste.rs/s/5kgenk/reviewing_ai_code_is_not_viable_a... )
>I'm quite influenced by Ed Zitron's reporting. And I'm also standing by my own idea that AI will run out of money and just be turned off due to the huge operational cost.
I wish this were how things are going but Uber is still surviving. I figure at least one AI company is going to come out of this alive and kicking.
This is where I find it lacking:
1. I am not yet convinced that I am more productive with AI for complex tasks. But, I am fairly certain for trivial tasks, or bureaucratic tasks AI has lessened my cognitive load (data point of one).
2. Does AI run on money, or does it run on energy? Money is the means by which we facilitate exchange. But in reality, the energy in the universe is ginormous. The fundamental thing to keep AI going is available in excess, all other things are negotiation.
3. Is AI bad for the environment or is it the way we obtain energy bad for the environment? The trend seems to be that energy is getting cheaper and cleaner. The world is not static, but will move to accommodate a world with AI.
I agree with the point about we don't have clear/perceivable increase in value I.e. companies are not 10x profitable due AI.
But, I don't understand the position of that has no value or that is going to be "turned off", that is just doomsday prophecy at this point.
We focus so much in code, but AI have changed many if not most of the work. It made possible to take a chance in challenges that would once be dismissed of the bat.
I do think we are still get used to the new ways of work, which IMO is a moving target. Once that we overcome that, results may come clear, but that may not be in companies making 10x more revenue.
From TFA>>I'm open to suggestions how this is going to play out. Close to a trillion dollars invested in AI. Where will the money come from to recuperate all these investments?
If investors are looking for a 50% return, they need the investment to be worth about $1.5 trillion. If they're looking for a 100% return, it needs to be worth about $2 trillion. That's still an enormous hurdle, but it's much smaller than suggesting AI needs to somehow generate trillions of dollars in cash simply to recoup the original investment.
AI costs aren’t going to stay this high. They will go down. Nobody is going to turn off AI, but they will care about making sure that we are reasonably efficient when using it. Compute that can happen locally will. Communities that responsibly want to build data centers are. Climate implications will be addressed by policy. People said the same about the internet 30 years ago… It’s not about raw productivity. It’s about perceived effort. If a job feels easier with AI assistance we’ll demand it. If AI checking a human’s work is more reliable than a human checking AI’s work (it is) that’s what we’ll deploy.
> AI costs aren’t going to stay this high. They will go down.
People said this a year ago, then all the companies admitted they were incinerating cash and had to increase to token pricing. Consumers basically mutinied and managed to push back a bit, but that is the struggle right now - to keep prices from increasing 25+ times, not to see them decrease.
If you're premising this on some future technology like ASICs, even if that works out then we probably shouldn't be spending a trillion dollars on datacenters that it will make obsolete.
People said this a year ago
A year ago Opus 4.1 was $75 per million tokens, and now Opus 4.8 is $25 and Fable is $50.
> AI costs aren't going to stay this high
Why do you feel this way? Are you talking about frontier models from US companies or smaller weight open source models?
From the perspective of US companies, it seems pretty clear they're severely under water and will need some way to become profitable or risk collapse, so how else will they do that or they than jacking up prices or some unknown future miracle advancement?
I could see the argument applied to self-hosted/OSS models tho.
so how else will they do that
By selling more tokens, which by all reasonable accounts are profitable on the margin.
> AI costs aren’t going to stay this high. They will go down.
I hear this a lot and it seems to be far too optimistic. Call me skeptical but plan pricing continues to increase with tighter usage constraints. We’re still at the point in time that companies like Anthropic, OpenAI, and GitHub charge seemingly reasonable prices in order to get people hooked onto AI before raising prices to get their return on investment.
> And I'm also standing by my own idea that AI will run out of money and just be turned off due to the huge operational cost.
Thats a pretty strong outcome. It implies that not only are GPUs that power AI too expensive long term, but they cost too much to operate even if they were free.
Seems to me the more likely outcome is a wave of dotCom style bankruptcies wiping out equity holders for companies who contracted to buy chips and datacenters at MSRP, and a second wave for the groups that step in after to operate whats left without the absurd financing charges and lower capex.
> even if they were free.
They are free. You can download glm-5.2 and run it on your own hardware. Even you got hardware for free, electricity would cost you more than the sub, in most places.
Many improvements have come to AI that have reduced costs of inference and training, like sparse attention, and many more will arise in the future. To talk about the cost of AI right now versus the future is to talk about the first IBM computers versus the personal computing that appeared years later. The cost will go down. In fact that is what the major labs are definitely banking on.
Author does not draw full enough picture (by linearly projecting current trends). But still has good points. I liked the one where perceived "productivity" is compared against unsubsisdized token prices.
Eh, I feel the productivity argument kinda depends what sort of project you're using it for. If it's a local tool that only needs to solve a very specific problem you personally have, or you're treading well trodden ground represented in the data set (like WordPress plugin or React component development), you could probably just let an LLM handle almost everything and get a perfectly adequate solution in minutes.
And if you don't know anything about programming, or don't care that the solution it'll come up with is potentially flawed, then it's a lot quicker to vibe code up some sort of hack job answer than learn how to do everything manually. I'm pretty sure that my non-technical parents or grandparents could ask Claude or ChatGPT to build a simple website for their small business or charity, and have something workable within an hour or two.
For a software engineer, AI might not make them any more productive. For a non-engineer, it's basically magic.
The unsustainability thing is more of a concern though, and the external effects of AI are a huge problem. It's just that pure productivity wise, a lot of people will be significantly more productive with these systems than without them.
I think you might be fooling yourself with stale data.
There are definitely many open questions around things like what tasks is AI appropriate for, how involved should you be in the process (fully automated agent swarms, or planning steps back and forth with human input on every decision), at what level of usage might you hit diminishing or negative returns, will we be able to continue to use it at this rate / price, will it get cheaper or will energy use continue to rise, how much will AI improve in the future, how will AI use affect people's development or temperament, will skills atrophy, will people no longer learn critical skills, how will we deal with AI spam or a loss of "proof of work" in producing a lengthy written artifact.... in sum there is no shortage of AI related topics worth exploring and thinking deeply about.
Some people still suggest that AI flat out isn't useful though, and that angle to me doesn't hold much water. I was a slow AI adopter, but I have found plenty of benefit from using AI in various applications recently. I haven't come close to using on the level of some people who orchestrate agents and rarely read or touch the code they are writing, but there have been clear times when I popped into Claude and had it help me very successfully with something that would have taken longer to do without that help. So increasingly I am skeptical of anyone positing that there is no potential benefit to appropriately using AI assistance. I think the author could rewrite around the angle of their perceived negative externalities outweighing potential advantages, but to deny or ignore those potential benefits ultimately distracts or actively takes away from a serious discussion of said negative externalities.
I don't put much stock in the very very frequently cited study about AI productivity included in this blog. It's a single study. The study itself cites multiple other studies that did find a productivity boost in writing software with AI assistance. I don't see the methodology as being very sound, the sample size isn't very big, some of those developers had likely never used AI before so of course they were slower as they explored using a new tool for the first time (which the study more or less writes off as a non-factor). They also used "Claude 3.5/3.7 Sonnet", which is not the strongest model. I personally find the Sonnet family of models not that useful and much more inaccurate than something like Opus or Fable.
The responses to this post made me really sad.
Sure - it was light on the meat, But it's an opinion blog post, not a full article. the author ventures to cite their refs, acknowledges potential bias.. way more than you get with a tonne of front page articles (many of which probably should have been short opinion pieces like this)
Sure - it's not particularly notable or high entropy enough to save, but it's far from offensive or bothersome ...yet all I'm seeing (at time of posting) is defensive dismissals, ad hominems, and echo chamber pablum
Dig down and I think you’ll find that AI boosters are really quite insecure.
It certainly seems that way from this - admittedly tiny - sample
Thank you.
Both things can be true.
Quotes study from early 2025. Same group redid study now shows opposite result. “Fooling yourself” indeed.
Anybody actually using AI in anger (to get paid) will tell you AI was pretty bad early 2025. It wasn’t usable till June. By December it was writing most of my code - with significant reviews by me. Now it’s basically getting it right. YMMV.
I've noticed that you can either move very fast with AI, or very slow. Moving fast, you place a lot of trust in the agent and let it proceed and create something very quickly that works at a surface level and that you don't understand at all. Moving slow you take the time to ask questions and understand every part of the application and learn more about how to code it yourself and make changes as necessary.
One thing I've noticed is, that coding manually is exponentially more stressful than the slow AI approach. I'm not going any faster than I normally would, but the burnout doesn't show up on the fifth day of the week. The hyper-focus is kind of fun but utterly exhausting.
With the fast approach I can create impressive demos and POCs that are nowhere near ready for production. But sometimes that's exactly what I need.
> One thing I've noticed is, that coding manually is exponentially more stressful than the slow AI approach.
I find the exact opposite. Typically the AI will make too many mistakes (even with latest advanced models), they usually aren't major/program breaking mistakes, but they still require fixing if I want to keep the code quality good enough, and make future changes possible without a falling house of cards. In order to describe the correct approach I have to write more English to describe the problem than the amount of code I could write myself. I could maybe see it working by writing rough pseudo code and having the AI translate, but then I loose out on any code analysis or tab completion as I write. It's also just innately frustrating to have to babysit and correct somethings work all the time. This all may be in part caused by the nature of my current work for the last couple years, which has mostly been on library code rather than application code.
I suspect I'd have a very different experience if I was working on CRUD related tasks like making new API endpoints, interfacing with a DB, or writing FE components. Not to make light of those tasks, I enjoy that work, though it is much more repetitive.
I agree with the parts about using AI to ask questions and gain better understanding though. I do that constantly for APIs and languages that I don't have a clear mental model of. It's like an interactive search.
Also agree on POCs, sometimes there are small tools, demos, or prototype features that I don't need to care deeply about.
If I'm working on a non-coercive environment (No deadlines, requirements, etc..) hand coding in hyper-focus mode is extremely zen. But working on a deadline it really hurts. My brain can't help but ratchet up the stress.
AI has been a big boost for my data science productivity.
I can have an agent put together a shitty prototype of something just to get a general idea of what I'm looking at, and save probably half a day of work and avoid physically typing potentially hundreds of lines of code. And I can rubber duck with it, and very quickly search through software packages and literature, produce summary reports, dig through my own company source code and wiki and Slack messages to figure out how some specific detail works, etc. These things are important but take my time and energy away from focusing on the hard parts of what I'm supposed to be doing.
My new favorite thing with AI is that my pile of random helper functions is now a thoughtfully-designed library with an extensive test suite for every single thing. I'm not auditing every line of code but I'm reading the plans carefully and I do take a glance at what the agent wrote before I start using it. Most of the time it's not perfect or it's ugly in some way I wouldn't have done, but it's good enough to do research with, and it's way better than what I could have put together by hand, and in much less time. I trust my own research results more because I can put together these kinds of helper libraries much faster, and if I have to make changes I have a big test suite to catch me when I make a mistake.
The only hard part is managing the context switching where instead of putting my work on pause to go write a helper routine, I now have to account for periodically interrupting my work in order to keep an eye on what the agent is doing. For now my approach has been to just ignore it for a while and then take a break to cycle through AI work sessions/chats.
I agree, the speed at which the agents work is a problem. It creates an interrupt workflow instead of a flow workflow. This works for some people but I notice that I don't do well that way. Typically I'll have an agent working on a task while I use a separate one in Ask mode just kind of satisfying my curiosity while I wait. This keeps me engaged but makes the interrupts less jarring.
I just spent multiple evenings talking through a small data model. With the massive number of misses the model had, I can’t imagine how bad it would have been if I had let it yolo the first design of its choice, to build on.
The requirements didn’t really change, opus just couldn’t see all the real world potential issues with its various suggestions.
But it was helpful for me to talk it out. So I guess it fell in the (very) slow category of productive use.
The rubber ducking potential is the first thing that got me interested in AI as a useful tool.
You do have to be very careful of when the AI might be confidently wrong, or leading you down some kind of weird narrow path.
When searching the web to gather info on a topic, all have a tendency to find 2-4 search results and fixate on those as if they represented the entire scope of the topic. No surprise about the fixation because that's kind of how LLMs work (X, Y, and Z are in context -> more tokens about X, Y, and Z). But you have to be aware of it, and I often have to prompt it to do another search and keep turning up more options.
I've found that my requirements and acceptance criteria are much less clear than I think they are. AI reveals when you are being too vague, almost instantly sometimes.
They always are. When you think you’ve got it all… ya don’t.
Unfortunately this recent experience wasn’t that as much as it producing rapid-fire misses when it did have the constraints. It regularly just violated X to kinda-solve Y. Or made up a solution to both that solves neither. And this wasn’t some wildly complicated scenario.
Sure made me solidify some opinions (we ended up pretty close to where I had started without it), but woo-boy… it was a painful journey.
Yeah, sometimes you just have to throw out the current solution and start over/rollback. AI will go down some weird rabbit holes to solve a problem with the wrong requirements. Sometimes I feel like I'm just coding in English instead of Python or TypeScript.
This, I'm working on a game engine as a hobby project. I can do one "big" feature over a weekend, most of it including designing and manual testing that it's working as expected, but then it takes me 2 weekends to break up the changes into smaller commits using jj, which also gives me the chance to review and understand the code myself. So it's a weird mix of fast and slow, but I think I've grown to prefer reviewing over writing so it works for a hobby project.
> …but then it takes me 2 weekends to break up the changes into smaller commits using jj, which also gives me the chance to review and understand the code myself.
I just tell Claude Claude code, "commit all, grouping logically" after a tranche of work.
Genuine question a why are you using AI for a hobby project like that? It seems like the worst of both worlds.
I love that you can mix these approaches. It lets me very quickly get a working prototype and then I can focus on optimizing the areas I care about / are having issues.
I think almost anything you can build autonomously with a fully independent AI agent, you can build far better with the slow AI approach. It is a good trade off between the super fast agentic approach and the full manual approach, and leaves you with a codebase you can still reason about and find ways to improve.
Also, once you build something with slow AI, it becomes more practical to actually rewrite it by hand quickly and refactor it in a way that makes more sense for humans. It’s “humanwashing” the code.
tired of this one dimensional articles that paint a wide brush, there are wide variety of AI uses and it absolutely has a productivity multiplying effect.
These AI sceptics always omit the possibility that running LLMs could become much cheaper in the future, which it almost certainly will.
This is largely an infrastructure problem, and it will be solved just as similar problems were solved with storage, network capacity, and computing power.
There will be bankruptcies, but the technology will prevail, just as the web did after the dot com bubble. Personally, I see the loss of craft and practical skill, which can only be built through doing, as a much greater danger. I still practise LeetCode from time to time so I do not lose the ability to write code myself.
This mostly reads like wishful thinking about the future, that's not a basis to make decisions on right now.