It’s important to remember these people you mention who work for Anthropic have a vested interest in selling Claude Code to the world. They are not an impartial third-party, so I would take anything they say with a grain of salt.
> Less code because code equals responsibility.
This is true. The problem with AI is that while someone may personally write less code, they are still responsible for it and have to answer questions about the minutiae of what it does. One of my least favorite things is being responsible for, or having to answer for, work that isn’t mine. I’m not sure why I’d willing make that my whole job.
When I first started coding, I knew how my code worked down to assembly language because that was the only way I could get anything to run at a sufficient speed on a 1Mhz computer, I then graduated to C and C++ with some VB and then C#, JavaScript and Python
Back in 2000 I knew every server and network switch in our office and eventually our self hosted server room with a SAN and a whopping 3TB of RAM before I left. Now I just submit a yaml file to AWS
Code is becoming no different, I treat Claude/Codex as junior developers, I specify my architecture carefully, verify it after it’s written and I test the code that AI writes for functionality and scalability to the requirements. But I haven’t looked at the actually code for the project I’m working on.
I’ve had code that I did write a year ago that I forgot what I did and just asked Codex questions about it.
Although I write very little code myself anymore, I don't trust AI code at all. My default assumption: every line is the most mid possible implementation, every important architecture constraint violated wantonly. Your typical junior programmer.
So I run specialized compliance agents regularly. I watch the AI code and interrupt frequently to put it back on track. I occasionally write snippets as few-shot examples. Verification without reading every line, but not "vibe checking" either.
I like this. The few-shot example snippet method is something I’d like to incorporate in my workflow, to better align generated code with my preferences.
I have written a research paper on another interesting prompting technique that I call axiomatic prompting. On objectively measurable tasks, when an AI scores below 70%, including clear axioms in the prompt systematically increases success.
In coding this would convert to: when trying to impose a pattern or architecture that is different enough from the "mid" programming approach that the AI is compelled to use, including axioms about the approach (in a IF this THEN than style, as opposed to few shot examples) will improve success.
The key is the 70% threshold: if the model already has enough training data, axioms hurt. If the model is underperforming because the training set did -not- have enough examples (for example hyperscript), axioms helps.
Compilers have a finite set of inputs and outputs that should generate reproducible results. There's a larger amount of possible outputs for the same question with AI and very little reproducbility.
Yes but once the code is written it’s not going to magically change. I am going to test the code just like I would test something I wrote - again like I’ve been doing for 40 years when writing my code by hand.
I input x and I expect y behavior and check for corner cases - just like I have checked for correctness for 40 years. Why do I care how the code was generated as long as it has the correct behavior?
Of course multithreaded code is the exception unless the LLM is putting a bunch of rnd() calls in the code to make it behave differently.
The metaphor I'd use is, can you understand the a story if you don't read it in the original language? Code is a language that describes the function.
I want to say, I've lived through the time (briefly) where folks felt if you didn't understand the memory management or, even assembly, level ops of code, you're not going to be able to make it great.
High level languages, obviously, are a counter-argument that demonstrate that you don't necessarily need to understand all the details to deliver an differentiable experience.
Personally, I can get pretty far with a high-level mental model and deeper model of key high-throughput areas in the system. Most individuals aren't optimizing a system, they're building on top of a core innovation.
At the core you need to understand the system.
Code is A language that describes it but there's others and arguably, in a lot of cases, a nice visual language goes much further for our minds to operate on.
Yes, and I like the points you are making. I feel like the mental models we make are exercises in a purer form of knowledge building than the code artifacts we produce. A kind of understanding that is liberated from the confines of languages.
If anything, you have to understand code more now.
Before you (or your devs) could write code a couple different ways and understand it. Now you have to look a code generated by an agent that is not necessary writing code in the same way as the culture at your company. There might be a thousand different ways a feature gets written. You have to spend more time reviewing and thinking it about it in my opinion.
When juniors use LLM, because they don't have experience, it becomes a nightmare for tenured engineers, and we just end up "mopping the slop", as I tend to say.
Just as with an LLM, a detailed style and format guide helps an incredible amount both for the LLMs and juniors. If you have standards and they’re not written down, you either require everyone to go teach them to anyone new, or you don’t have standards.
Not very often, and most of the time it shouldn't be generating those but rather formatting code to test that. If you accept the non-determinism and use some of the more recent models, you'll find it can do 99% of it very fast, and with some guardrails and testing it can fairly reliably produce working solutions.
This does not match my experience, have been working with LLM since 2023. We presently use the latest models, I assure you. We can definitely afford it.
I am not saying LLM is worthless, but being able to check its outputs is still necessary at this stage, because as you said, it is non-deterministic.
We have had multiple customer impacting events from code juniors committed without understanding it. Please read my top level comment in this post for context.
I genuinely hope you do not encounter issues due to your confidence in LLM, but again, my experience does not match yours.
Edit: Would also add that LLM is not good at determining line numbers in a code file, another flaw that causes a lot of confusion.
Two layers vibe coding can't touch: architecture decisions (where the constraints live) and cleanup when the junior-dev-quality code accumulates enough debt. Someone has to hold the mental model.
I've used Claude Code a lot over the last year and I've generally been very happy with it. I have a lot of experience writing code both professionally and for personal projects. I've found that for things like basic APIs and websites and database operations, I don't have to pay attention to the code being produced much at all anymore. It Just Works for the most part as long as you adequately describe what you're trying to build. There are only so many ways you can write a CRUD app after all, and generally the implementation isn't "special" just necessary.
But my experience on 3d game dev in particular has been quite different. I've been able to get good results for basic 2d games and basic features in 3d worlds, but have been struggling to build more complicated scenarios with Claude Code without laying out every specific detail. I have to tell it to use quaternions for a particular rotation because I know about issues with gimble lock. I have to suggest a ray traced solution in another area because relative mouse position isn't good enough when accounting for resolution and aspect ratio. If I didn't know about ray tracing or quaternions and how they are used and fit into game development I wouldn't have been able to interrupt Claude Code and guide it down a better path. I think claude code is particularly weak in spatial reasoning and I suspect the context required for some GPU operations are pushing other parts of instructions out of context. It's forgetting "the basics" far more than I've experienced in any other project. Building a 3d world simulation featuring a bastardization of plate tectonics and weather systems is the first thing I've tried to do with Claude Code that I could have probably written myself faster. If it wasn't for the crippling adhd.
```
- code I don’t need to model in my head (low risk, follows established conventions, predictable, easy to verify), and
- code I can’t help modelling in my head (business-critical, novel, experimental, or introduces new patterns).
I feel like there’s actually one or two more shades in between.
```
Sometimes I think something belongs in the second category, but then it turns out it’s really more like the first.
And sometimes something is second-category, but for the sake of getting things done, it makes more sense to treat it like the first.
If vibe coding keeps evolving, this is probably the path it needs to explore.
I just wonder what we’ll end up discovering along the way.
If the AI provides 0-1 nines of reliability and you refuse to provide the rest of nines required by the customer, then who will provide these, and what is your role and claim to margin here?
If you don't understand code, you're asking for a whole heap of trouble.
Why? You can't validate the LLM outputs properly, and commit bugs and maybe even blatantly non-functional code.
My company is pressuring juniors to use LLM when coding, and I'm finding none of them fully understand the LLM outputs because they don't have enough engineering experience to find code smells, bugs, regressions, and antipatterns.
In particular, none of them have developed strong unit testing skills, and they let the LLM mock everything because they don't know any better, when they should generally only mock API dependencies. Sometimes LLM will even mock integration tests, which to me isn't generally a super good idea.
So the tests that are supposed to validate the code are completely worthless.
It has led to multiple customer impacting issues, and we spend more time mopping the slop than we do engineering as tenured engineers.
Have CC users been raving about rock-solid stability improvements, more insightful spending analytics, and overall quantum improvements in customer experience?
No, most of the chatter I’ve heard here has been the opposite. Changes have been poorly communicated, surprising, and expensive.
If he’s been vibe-coding all this and feeling impressed with himself, he’s smelling his own farts. The performance thus far has been ascientific, tone-deaf and piss-poor.
The "good riddance" attitude surprises me also. On one hand, it can be unpleasant to sort through obscure syntactical gobbledegook, like tracing around multiple levels of pointer indirection, but then again, I have found a certain enjoyable satisfaction in such things. It can be tough, but a good tough.
It does seem to me that the people who consistently get the best results from AI coding aren't that far away from the code. Maybe they aren't literally writing code any more, but still communicating with the LLM in terms that come from software development experience.
I think there will still be value in learning how to code, not unlike learning arithmetic and trigonometry, even if you ultimately use a calculator in real life.
But I think there will also still be value in being able to code even in real life. If you have to fix a bug in a software product, you might be able to fix it with more precise focus than an LLM would, if you know where to look and what to do, resulting in potentially less re-testing.
Personally, I balk at the idea of taking responsibility for shipping real software product that I (or, in a team environment, other humans on my team) don't understand. Perhaps that is my aerospace software background speaking -- and I realize most software is not safety-critical -- but I would be so much more confident shipping something that I understood how it worked.
I don't know. Maybe in time that notion will fade. As some are quick to point out, well, do you understand the compiled/assembled machine code? I do not. But I also trust the compilation process more than I trust LLMs. In aerospace, we even formally qualify tools like compilers to establish that they function as expected. LLM output, especially well-guided by good prompts and well-tested, may well be high quality, but I still lack trust in it.
Don’t buy into self promotion bullshit. AI can be helpful. It’s another form of automation. It is not creative and will not make you a better programmer. The only thing that will make you a better programmer is time spent programming, just like with anything else.
That seems like exactly the wrong lesson to learn from LLM "AI". Under no circumstances does such an "AI" understand anything, much less important semantics, so human understanding becomes that much more important.
I realize that director level managers may not get this because they've always lived and worked in the domain of "vibes" but that doesn't mean it's not true
It’s important to remember these people you mention who work for Anthropic have a vested interest in selling Claude Code to the world. They are not an impartial third-party, so I would take anything they say with a grain of salt.
> Less code because code equals responsibility.
This is true. The problem with AI is that while someone may personally write less code, they are still responsible for it and have to answer questions about the minutiae of what it does. One of my least favorite things is being responsible for, or having to answer for, work that isn’t mine. I’m not sure why I’d willing make that my whole job.
When I first started coding, I knew how my code worked down to assembly language because that was the only way I could get anything to run at a sufficient speed on a 1Mhz computer, I then graduated to C and C++ with some VB and then C#, JavaScript and Python
Back in 2000 I knew every server and network switch in our office and eventually our self hosted server room with a SAN and a whopping 3TB of RAM before I left. Now I just submit a yaml file to AWS
Code is becoming no different, I treat Claude/Codex as junior developers, I specify my architecture carefully, verify it after it’s written and I test the code that AI writes for functionality and scalability to the requirements. But I haven’t looked at the actually code for the project I’m working on.
I’ve had code that I did write a year ago that I forgot what I did and just asked Codex questions about it.
How do you verify the code without actually looking at it?
Although I write very little code myself anymore, I don't trust AI code at all. My default assumption: every line is the most mid possible implementation, every important architecture constraint violated wantonly. Your typical junior programmer.
So I run specialized compliance agents regularly. I watch the AI code and interrupt frequently to put it back on track. I occasionally write snippets as few-shot examples. Verification without reading every line, but not "vibe checking" either.
I like this. The few-shot example snippet method is something I’d like to incorporate in my workflow, to better align generated code with my preferences.
I have written a research paper on another interesting prompting technique that I call axiomatic prompting. On objectively measurable tasks, when an AI scores below 70%, including clear axioms in the prompt systematically increases success.
In coding this would convert to: when trying to impose a pattern or architecture that is different enough from the "mid" programming approach that the AI is compelled to use, including axioms about the approach (in a IF this THEN than style, as opposed to few shot examples) will improve success.
The key is the 70% threshold: if the model already has enough training data, axioms hurt. If the model is underperforming because the training set did -not- have enough examples (for example hyperscript), axioms helps.
How do you verify the compiler without looking at the assembled code? How do you verify code that links against binary libraries?
You run it and check for your desired behavior.
Compilers have a finite set of inputs and outputs that should generate reproducible results. There's a larger amount of possible outputs for the same question with AI and very little reproducbility.
Yes but once the code is written it’s not going to magically change. I am going to test the code just like I would test something I wrote - again like I’ve been doing for 40 years when writing my code by hand.
(Those are hardly analogous comparisons to LLM generated code, are they?)
So you do a vibe check?
What’s “vibe checking”?
I input x and I expect y behavior and check for corner cases - just like I have checked for correctness for 40 years. Why do I care how the code was generated as long as it has the correct behavior?
Of course multithreaded code is the exception unless the LLM is putting a bunch of rnd() calls in the code to make it behave differently.
"Let's check that we can do X, Y, Z"
"Create documentation and then write tests"
a few moments later...
"There's a bug where we cannot do Y. Investigate the code and then let's discuss the best fix"
"Update the documentation and tests"
The metaphor I'd use is, can you understand the a story if you don't read it in the original language? Code is a language that describes the function.
I want to say, I've lived through the time (briefly) where folks felt if you didn't understand the memory management or, even assembly, level ops of code, you're not going to be able to make it great.
High level languages, obviously, are a counter-argument that demonstrate that you don't necessarily need to understand all the details to deliver an differentiable experience.
Personally, I can get pretty far with a high-level mental model and deeper model of key high-throughput areas in the system. Most individuals aren't optimizing a system, they're building on top of a core innovation.
At the core you need to understand the system.
Code is A language that describes it but there's others and arguably, in a lot of cases, a nice visual language goes much further for our minds to operate on.
Yes, and I like the points you are making. I feel like the mental models we make are exercises in a purer form of knowledge building than the code artifacts we produce. A kind of understanding that is liberated from the confines of languages.
If anything, you have to understand code more now.
Before you (or your devs) could write code a couple different ways and understand it. Now you have to look a code generated by an agent that is not necessary writing code in the same way as the culture at your company. There might be a thousand different ways a feature gets written. You have to spend more time reviewing and thinking it about it in my opinion.
Made a similar comment.
It's great for tenured engineers, when we use it.
When juniors use LLM, because they don't have experience, it becomes a nightmare for tenured engineers, and we just end up "mopping the slop", as I tend to say.
I also have issue with how LLM do testing.
Just as with an LLM, a detailed style and format guide helps an incredible amount both for the LLMs and juniors. If you have standards and they’re not written down, you either require everyone to go teach them to anyone new, or you don’t have standards.
> you don’t have standards.
The problem is that LLM mess up things as basic as math and dates, and that's before the context gets too large and it starts making other mistakes.
Edit: Also LLM over mock tests and juniors trust that...
Not very often, and most of the time it shouldn't be generating those but rather formatting code to test that. If you accept the non-determinism and use some of the more recent models, you'll find it can do 99% of it very fast, and with some guardrails and testing it can fairly reliably produce working solutions.
> Not very often
> testing
This does not match my experience, have been working with LLM since 2023. We presently use the latest models, I assure you. We can definitely afford it.
I am not saying LLM is worthless, but being able to check its outputs is still necessary at this stage, because as you said, it is non-deterministic.
We have had multiple customer impacting events from code juniors committed without understanding it. Please read my top level comment in this post for context.
I genuinely hope you do not encounter issues due to your confidence in LLM, but again, my experience does not match yours.
Edit: Would also add that LLM is not good at determining line numbers in a code file, another flaw that causes a lot of confusion.
I haven’t run into that problem but I do also hold agents on a tight leash!
Job security for those of us who think like this.
Two layers vibe coding can't touch: architecture decisions (where the constraints live) and cleanup when the junior-dev-quality code accumulates enough debt. Someone has to hold the mental model.
I've used Claude Code a lot over the last year and I've generally been very happy with it. I have a lot of experience writing code both professionally and for personal projects. I've found that for things like basic APIs and websites and database operations, I don't have to pay attention to the code being produced much at all anymore. It Just Works for the most part as long as you adequately describe what you're trying to build. There are only so many ways you can write a CRUD app after all, and generally the implementation isn't "special" just necessary.
But my experience on 3d game dev in particular has been quite different. I've been able to get good results for basic 2d games and basic features in 3d worlds, but have been struggling to build more complicated scenarios with Claude Code without laying out every specific detail. I have to tell it to use quaternions for a particular rotation because I know about issues with gimble lock. I have to suggest a ray traced solution in another area because relative mouse position isn't good enough when accounting for resolution and aspect ratio. If I didn't know about ray tracing or quaternions and how they are used and fit into game development I wouldn't have been able to interrupt Claude Code and guide it down a better path. I think claude code is particularly weak in spatial reasoning and I suspect the context required for some GPU operations are pushing other parts of instructions out of context. It's forgetting "the basics" far more than I've experienced in any other project. Building a 3d world simulation featuring a bastardization of plate tectonics and weather systems is the first thing I've tried to do with Claude Code that I could have probably written myself faster. If it wasn't for the crippling adhd.
``` - code I don’t need to model in my head (low risk, follows established conventions, predictable, easy to verify), and
- code I can’t help modelling in my head (business-critical, novel, experimental, or introduces new patterns). I feel like there’s actually one or two more shades in between. ```
Sometimes I think something belongs in the second category, but then it turns out it’s really more like the first. And sometimes something is second-category, but for the sake of getting things done, it makes more sense to treat it like the first.
If vibe coding keeps evolving, this is probably the path it needs to explore. I just wonder what we’ll end up discovering along the way.
If it’s in the second category, I struggle not to mentally model it. How do you stop yourself? And should you?
If the AI provides 0-1 nines of reliability and you refuse to provide the rest of nines required by the customer, then who will provide these, and what is your role and claim to margin here?
Creating work for the clean-up crew and leaving good money on the table for them (because it ain't gonna be cheap).
If you don't understand code, you're asking for a whole heap of trouble.
Why? You can't validate the LLM outputs properly, and commit bugs and maybe even blatantly non-functional code.
My company is pressuring juniors to use LLM when coding, and I'm finding none of them fully understand the LLM outputs because they don't have enough engineering experience to find code smells, bugs, regressions, and antipatterns.
In particular, none of them have developed strong unit testing skills, and they let the LLM mock everything because they don't know any better, when they should generally only mock API dependencies. Sometimes LLM will even mock integration tests, which to me isn't generally a super good idea.
So the tests that are supposed to validate the code are completely worthless.
It has led to multiple customer impacting issues, and we spend more time mopping the slop than we do engineering as tenured engineers.
Have CC users been raving about rock-solid stability improvements, more insightful spending analytics, and overall quantum improvements in customer experience?
No, most of the chatter I’ve heard here has been the opposite. Changes have been poorly communicated, surprising, and expensive.
If he’s been vibe-coding all this and feeling impressed with himself, he’s smelling his own farts. The performance thus far has been ascientific, tone-deaf and piss-poor.
Maybe vibe-coding is not for him.
The "good riddance" attitude surprises me also. On one hand, it can be unpleasant to sort through obscure syntactical gobbledegook, like tracing around multiple levels of pointer indirection, but then again, I have found a certain enjoyable satisfaction in such things. It can be tough, but a good tough.
It does seem to me that the people who consistently get the best results from AI coding aren't that far away from the code. Maybe they aren't literally writing code any more, but still communicating with the LLM in terms that come from software development experience.
I think there will still be value in learning how to code, not unlike learning arithmetic and trigonometry, even if you ultimately use a calculator in real life.
But I think there will also still be value in being able to code even in real life. If you have to fix a bug in a software product, you might be able to fix it with more precise focus than an LLM would, if you know where to look and what to do, resulting in potentially less re-testing.
Personally, I balk at the idea of taking responsibility for shipping real software product that I (or, in a team environment, other humans on my team) don't understand. Perhaps that is my aerospace software background speaking -- and I realize most software is not safety-critical -- but I would be so much more confident shipping something that I understood how it worked.
I don't know. Maybe in time that notion will fade. As some are quick to point out, well, do you understand the compiled/assembled machine code? I do not. But I also trust the compilation process more than I trust LLMs. In aerospace, we even formally qualify tools like compilers to establish that they function as expected. LLM output, especially well-guided by good prompts and well-tested, may well be high quality, but I still lack trust in it.
Many irrelevant difference between programming languages are now exposed for what they are.
Thinking clearly is just as relevant or encumbering as it always was.
Great question, but not specific to LLMs. Same applies to importing a C library.
Answer: no. Just harder.
Don’t buy into self promotion bullshit. AI can be helpful. It’s another form of automation. It is not creative and will not make you a better programmer. The only thing that will make you a better programmer is time spent programming, just like with anything else.
That seems like exactly the wrong lesson to learn from LLM "AI". Under no circumstances does such an "AI" understand anything, much less important semantics, so human understanding becomes that much more important.
I realize that director level managers may not get this because they've always lived and worked in the domain of "vibes" but that doesn't mean it's not true