We're reaching a point currently where output quality is very much determined by input quality. Previously output quality was hampered fundamentally by model knowledge, hallucinations, and model quality.
Now, we have better knowledge of prompting as people have learnt what to say, models are better, models make use of memory from other conversations, they have skills written by humans or even themselves on how to do things, access to the internet to get live info, access to project files to check info, and the built in 'thinking' to challenge their own assumptions and loop on outputs until its refined.
You're right that output is always off still, but a lot of people have reached a point where it's only 'off' by an amount that is less than the effort required to do the task themselves, and considerably so.
My example today is prompting Claude to do a technical audit of a new client site.
It has skills for UX and SEO audits. Connects to an SEO tool. Pulls client info from OneDrive. Outputs to Word from a template for our agency. I even had it drive a remote pagespeed testing tool in Chrome because they don't have an MCP server currently.
Doing that report myself is 3.5-7 hours depending on what's found. Claude did it in 0.5 hours. Now I'm sorting out the oddities and anything that feels 'off'. I know and understand the full content of the report and can get on with actioning the recommendations or prioritising them for others. I've got maybe 1 hour of review and writing to do. It's not a 10x improvement but I'm happy with it.
Although, whilst Claude did it's bit I was doing other work. So, perhaps the multiplier is higher than I give it credit for.
The way AI is able to interact with outside resources is pretty impressive, but the quality of code it produces to me is still questionable, more so in the larger scope, and the errors it produces are sometimes hard to catch because they're not normal human errors.
Recently I tried to get Claude to write a script that produces large amounts of code so I could profile a compiler. The script ended up outputing code that uses variables outside of their scope, didn't utilize like 90% of the features of the language, and basically ended up being something that I could make by spamming copy paste.
The script itself was also written in really weird way, utilizing recursion for pretty much everything when most of what it did could be done in simple loops. It ended up being a bit of a nightmare to fix and the entire time I was asking myself "why didn't I just write this in 30 minutes instead of going through all of this".
I can't speak to coding as it's not my area but certainly the pattern I've spotted is that it's best at grunt work. That's where the time savings kick in.
Browsing sites, linking up data, spotting anomalies, writing documentation, formatting documents, etc.
If a task isn't repetitive or doesn't involve ingesting data, then I think the time savings shrink rapidly and the need for oversight increases massively. I think some people are managing to set up enough automated oversight to get round that, but it's adding a layer that multiplies your token usage to do so and still has no guarantee. But certainly all these layers being added are increasing success rates.
Andrei Karpathy is speaking about barely coding now. He has a bias, a comment from him like that is marketing for Anthropic, but I believe he's found some groove with his setup to achieve that.
I think the current status quo this month in 2026 we're at a point where the best tips and tricks to get usable answers out of ChatGPT a year ago have been consolidated into what we know call memory and skills in Claude and other agent harness type systems. You might need to explore those more, in fact I think for Claude Code/Cursor there are even more layers for checking outputs that I've not even seen in Claude Desktop.
And I think your exact issue, and the experience of the vast volumes of people who share it with you, are an audience that the app makers want to better convince. The free tiers and marketing sites are going to step up their game gradually and there will be new features that lower failure rates even more.
I am shocked how much my experience is different from yours. I wrote Claudine, my own version of Claude Code, almost 2 years ago. This experience gave me the understanding of how the technology works. Since then I've produced maybe 300k lines of open source code, and all of it meaningful to the bones. What kind of projects are you working on, maybe it's the specificity of your domain?
When you join a new company, is it faster to fix a bug rewriting everything from scratch or to modify what's there? Seriously, get your head out of your ass.
We're reaching a point currently where output quality is very much determined by input quality. Previously output quality was hampered fundamentally by model knowledge, hallucinations, and model quality.
Now, we have better knowledge of prompting as people have learnt what to say, models are better, models make use of memory from other conversations, they have skills written by humans or even themselves on how to do things, access to the internet to get live info, access to project files to check info, and the built in 'thinking' to challenge their own assumptions and loop on outputs until its refined.
You're right that output is always off still, but a lot of people have reached a point where it's only 'off' by an amount that is less than the effort required to do the task themselves, and considerably so.
My example today is prompting Claude to do a technical audit of a new client site.
It has skills for UX and SEO audits. Connects to an SEO tool. Pulls client info from OneDrive. Outputs to Word from a template for our agency. I even had it drive a remote pagespeed testing tool in Chrome because they don't have an MCP server currently.
Doing that report myself is 3.5-7 hours depending on what's found. Claude did it in 0.5 hours. Now I'm sorting out the oddities and anything that feels 'off'. I know and understand the full content of the report and can get on with actioning the recommendations or prioritising them for others. I've got maybe 1 hour of review and writing to do. It's not a 10x improvement but I'm happy with it.
Although, whilst Claude did it's bit I was doing other work. So, perhaps the multiplier is higher than I give it credit for.
The way AI is able to interact with outside resources is pretty impressive, but the quality of code it produces to me is still questionable, more so in the larger scope, and the errors it produces are sometimes hard to catch because they're not normal human errors.
Recently I tried to get Claude to write a script that produces large amounts of code so I could profile a compiler. The script ended up outputing code that uses variables outside of their scope, didn't utilize like 90% of the features of the language, and basically ended up being something that I could make by spamming copy paste.
The script itself was also written in really weird way, utilizing recursion for pretty much everything when most of what it did could be done in simple loops. It ended up being a bit of a nightmare to fix and the entire time I was asking myself "why didn't I just write this in 30 minutes instead of going through all of this".
I can't speak to coding as it's not my area but certainly the pattern I've spotted is that it's best at grunt work. That's where the time savings kick in.
Browsing sites, linking up data, spotting anomalies, writing documentation, formatting documents, etc.
If a task isn't repetitive or doesn't involve ingesting data, then I think the time savings shrink rapidly and the need for oversight increases massively. I think some people are managing to set up enough automated oversight to get round that, but it's adding a layer that multiplies your token usage to do so and still has no guarantee. But certainly all these layers being added are increasing success rates.
Andrei Karpathy is speaking about barely coding now. He has a bias, a comment from him like that is marketing for Anthropic, but I believe he's found some groove with his setup to achieve that.
I think the current status quo this month in 2026 we're at a point where the best tips and tricks to get usable answers out of ChatGPT a year ago have been consolidated into what we know call memory and skills in Claude and other agent harness type systems. You might need to explore those more, in fact I think for Claude Code/Cursor there are even more layers for checking outputs that I've not even seen in Claude Desktop.
And I think your exact issue, and the experience of the vast volumes of people who share it with you, are an audience that the app makers want to better convince. The free tiers and marketing sites are going to step up their game gradually and there will be new features that lower failure rates even more.
>> Now, we have better knowledge of prompting as people have learnt what to say
Can you back up this claim? what do you mean exactly by "better knowledge" ?
Depends heavily on the models you use. SOTA models (Fable 5, Opus 4.8, GPT 5.5) are quite good in their native harness.
I am shocked how much my experience is different from yours. I wrote Claudine, my own version of Claude Code, almost 2 years ago. This experience gave me the understanding of how the technology works. Since then I've produced maybe 300k lines of open source code, and all of it meaningful to the bones. What kind of projects are you working on, maybe it's the specificity of your domain?
You might be even more shocked to learn that the author's experience isn't rare.
I think AI is good for creating a foundation, then branching out and adding features, you shouldn't overdo it with AI.
Are you treating it like a genie to build huge things in one shot or working on small incremental changes?
I’ve found the latter works way better
When you join a new company, is it faster to fix a bug rewriting everything from scratch or to modify what's there? Seriously, get your head out of your ass.