Forget about the scores... look at the Cost Per Task. Fable 5.1 was already the most expensive by a good margin, and now Fable 5.2 doubles it! This is with the price cut.
The output tokens per task just exploded. From 118k tokens (46k Answer/71k Reasoning) in 5.1 to 274k tokens (81k Answer/192k Reasoning) in 5.2.
There's a whole frontier of cost vs performance and this model pushes it out in a bunch of places, so its not just a matter of making a different tradeoff as you make it sound.
This model is on the cost performance frontier (Pareto line) from medium -> max.
If you leave xhigh and max off.. you still get better performance than Opus 5.5 max
Opus 5.5 Max - score 59 - cost per task - $10.04
Fable 5.2 High - score 62 - cost per task - $9.08
Note: This doesn't really capture the reality of things like 3d design or electronics design, where the earlier models just can't DO any of it well. Maybe you can bandaid over some of the gaps with strong skills, etc. The newer models can just do things that were not possible at all. When you move outside of coding into the visual/spacial space, you'll feel the inflection point on those capabilities that are happening right now.
The thing is max reasoning is most effective at things which are least interesting (the ones where the answers can be validated by a computer.) I've found for a lot of tasks like image recognition, low reasoning can even be better quality than max.
- Sol-low - cost-competitive with Haiku, while outperforming Sonnet on medium
- Sol-High - Competes with Astra medium and beats Opus 5.5-medium, at 20% the cost
- Sol-Max - Beats Opus 5.5 high, Astra high, and competes with Opus xhigh, at cheaper than Sonnet and a fraction of the Opus cost
And when you look at time spent, With the exception of Sol-max you get better results in less time for less money. My own experience is that this is true-- it is difficult to call Opus strictly better than Sol, but Sol gets where it's going far more efficiently.
It's rather hard to look at opus-- or any of anthropic's models-- and say "economical".
There is some truth here, but for people on subscriptions the Claude ones currently provide much better value.
And for most people here who care about coding performance Anthropic has a substantial lead[1] - although sadly they don't have coding benchmarks separately for Opus 5.5 Medium which I think it the sweet spot.
The cost tab really should have something like "how many tasks can you do in a particular subscription tier" for many people. I realize this changes often though.
Very interested if you have any citations for how Claude subscriptions give "much better value". I've recently been researching this with mixed analysis.
It seems like most analysis is basically "how many tokens do you get with a subscription and how much would those tokens cost at API rates". Looking at it that way, you do get more dollars of API equivalent usage for your subscription.
But Anthropic's API pricing is a lot higher than OpenAI's (Opus 5.5 is 2x of GPT-6.1 Sol). The number of tokens consumed per completed task is also far higher than GPT-6. So when you look at the number of tasks you can complete within a subscription, OpenAI looks like a much better value.
That is before you consider that OpenAI subscriptions (other than the cheapest one) don't have 5-hour usage limits, and also generally give you multiple free resets per month.
There was some chatter on Twitter, Reddit and here when an earlier version of this came out where people normalized it by completed AA or Epoch task and Claude models still came out well ahead.
In practice OpenAI subscriptions are more competitive than they appear because OpenAI does a lot more subscription resets. But that's hard to account for!
Personally, for coding tasks I've found Sol 6.1 High is about as good as Opus 5.5 Medium for well defined tasks, but Opus is much better at orchestrating.
I have both subscriptions, and when using Claude I use Opus Medium to orchestrate other Opus Medium subagents, whereas on Codex I use Astra Medium to orchestrate Sol High subagents.
That is the exact "your subscription buys this much in tokens at API pricing" kind of analysis that I mentioned. Which is one way to look at it, but it ignores the 2x pricing difference, and the token efficiency difference.
Per task, Opus 5.5 costs ~6x of Sol 6.1. Then you add on the lack of a 5-hour limit, no limit on which models (like Fable's 50% of usage limit) you can use, very frequent resets, and that OpenAI doubled the usage limits after that analysis was performed.
Even though the Claude subscription is equivalent to 5x the $ value of API priced tokens, it seems like Codex is the better value for getting things done.
I don't think so? I think OpenAI had planned to half the limits but decided not to go ahead with it, but this analysis was done at the limits we have now. I could be wrong about this though?
> Opus 5.5 costs ~6x of Sol 6.1.
My point in part is that you can't compare like this.
Unfortunately AA doesn't report non-Max efforts for their coding benchmarks.
But no one runs them on max. For Opus the advice is to usually run it on medium effort, and I've found that advice to be good.
For me to get equivalent results on Sol I need to run it on high effort, and I need to use Astra as an orchestrator (whereas Opus medium does great as an orchestrator too).
That combination pushes the blended token pricing up a lot.
I can't give you hard numbers on this, but at the moment my Codex credits get used up a lot quicker than my Claude credits on similar tasks in the same codebase.
I'm glad we have two great models competing, but at the moment I'd say Opus has the edge.
I think it is a narrow view to only consider the AA coding agent index as the only metric, especially with the models running at max.
For the link you shared, Opus used 3.7x as many tokens, and 2.5x the time to get its score. In the real-world building things with these tools, that kind of increase in time and cost (or subscription usage) matters, at least to me.
When comparing at more reasonable medium or high levels of reasoning, the gap between Opus and Sol narrows considerably. Opus being about 2 higher than Sol at the same reasoning level. However, Sol 6.1 xhigh matches Opus 5.5 high on scores (52), but while still being slightly faster and costing 78% less ($0.76 versus $3.46).
> When comparing at more reasonable medium or high levels of reasoning, the gap between Opus and Sol narrows considerably. Opus being about 2 higher than Sol at the same reasoning level. However, Sol 6.1 xhigh matches Opus 5.5 high on scores (52), but while still being slightly faster and costing 78% less ($0.76 versus $3.46).
Yes, but my point is that these are non-coding benchmarks at these non-max levels.
On coding at non-max Opus really does seem noticeably better, and my subscription seems to go longer. Look at the other comments in this thread - every single one says the same thing.
All I know is recently I’ve been switching in and out of Codex and Claude with the same workflow and my Codex usage gets drained much quicker. Just my experience.
Claude dollars just seem to go further on the subscription. Opus just gets more done at good quality. I have both subscriptions @ $200 and use them a lot
Exactly the same experience on both subscriptions. Claude is noticeably better value. If things were frozen in time and I could only choose one it would be Claude, no question.
On the other hand, there are plenty of ethical reasons not to support Sam Altman/Peter Thiel/Larry Ellison in their particular world vision, which for many rules out use of OpenAI.
> An economically successful step would be to improve training cost/frontier advance.
> They essentially failed at that, assuming they kept their training costs for the model step at least constant.
What? No!
This is completely wrong!
This proves that inference scaling continues to work beyond the previous frontier.
Claiming that higher inference costs shows failure is an odd thing to think. We've seen this optimization cycle over and over, most recently with Opus 5 -> Opus 5.5
Opus 5 was slow and expensive. Opus 5.5 came out and was better, fast and much cheaper for them to run.
There is nothing to indicate the same can't happen here.
Anthropic probably has multiple options open to them with this model, and they'll probably explore all of them:
- Use collected Fable 5.2 traces to improve reasoning economy for Fable 5.3
- Use Fable 5.2 as a teacher model for Opus 5.6
- Add additional fields RL data to expand Fable capabilities for Fable 5.4
They have not run into a wall. The frontier labs distill their strongest models into their cheaper, faster models.
Having these highly rated models reflects (potentially) the frontier of what they are capable of producing. Essentially its and advertisement and shows directionally where the smaller faster model's capabilities are headed.
The "wall" is in increasing intelligence per cost. No one is saying the absolute intelligence won't continue increasing, it's that the rate of intelligence increase per cost is falling from super-linear growth rates to linear at the same time the companies are backing out the investor-funded subsidies they were using to buy adoption and user growth numbers. You're correct that there is a 'trickle down' effect to lower tier models but OAI/Ant are clearly using this to segment customers by willingness to pay and their IPOs depend on extracting the maximum amount possible from each segment.
That's why the prices of frontier models have been skyrocketing, though they are using every trick imaginable to mask this being clear and obvious. They take with price increases, token deflation, shifting plan levels and limits but muddy things more by giving back some (but always eventually less) with resets, subs, etc.
> No one is saying the absolute intelligence won't continue increasing,
They absolutely have said that.
> That's why the prices of frontier models have been skyrocketing,
They haven't.
"The cost of achieving a given level of AI performance has fallen about 47% per quarter since 2023, or 13× per year."
"The cost of a given level of performance often falls fastest right after that level is first achieved, that is, when it is state of the art (SOTA). We see this pattern on three of our five main benchmarks of AI capability. Averaging across all five, cost falls 66% per quarter (75× per year) for performance that has just debuted as SOTA."
On a per-token basis Fable 5.2 is cheaper than 5.1:
5.2: $8 per million input tokens and $40 per million output tokens.
5.1: $10 per million input tokens and $50 per million output tokens.
Compared to older (and much less capable) frontier models it is also cheaper. For example GPT-4 was $30 per million input tokens and $60 per million output tokens
Be very careful comparing the Artificial Analysis "cost per task" scores. Fable 5.2 is expensive there because it is doing tasks the other models fail at.
Last I checked Fable's prices have fallen not risen. Opus 5.5 is as intelligent as Fable 5. I think Anthropic is actually trying to make as good of a model as possible and the advances that their expensive models are getting distilled into the cheaper faster models quickly.
I think a lot of people don't realize that the labs really care about AGI. They care about token spend, efficiency, cost, etc. only as far as it gets them customers that will fund AGI
Maybe a dumb question but once you have a model that spends a lot of tokens for high performance why can’t you distill it into a model that thinks less?
Sure it wouldn’t generalize well to new tasks but plenty of tasks are important and recurrent enough to justify this.
There's also a bunch of RL research around strategies to coax the student model towards more concise trajectories.
In general, a lot of people in industry are finding cost savings these days by tuning a smaller model over longer agent trajectories (not exactly the same as distilling the reasoning traces, since those aren't exposed by many frontier labs anymore) collected from their production agents. Typically works best when you have a pretty workflow-y agent whose tasks all share a similar shape.
Caleb is right.
Also it seems the results are not straightforward.
Let say you have a model that score 100 in a benchmark and you distill it into a model half the size. It has very few guarantees that it will score close to 100 again.1
But I assume one of the hard problems is to know that the model might not perform well enough outside of the distillation part.
There is also another caveat that models do display some level of sub language control, by this I mean that innocuous text might contain information relevant for the own agent or for other agent with it shares tasks.
(Like when your partner says there is nothing wrong)
Because distillation uses very few resources, we don't know exactly what gets send forward or not.2
As Caleb said, many companies found that they can tune small open weight agents over procedural known tasks because that's easy to inspect and check and because there is little space dor ai-loitering, it tends to be less problematic.
The key difference, at least in my understanding, is that you are not concerned with the weights or reducing data. You are basically tuning agents existing weights to emulate another agent answer.
1 - The process of distillation is super effective for specific domains though.
2 - I know I will be called out for supposed AI mystery machine. What I mean is that Large models rely on latent reasoning and nuanced context, often lost during condensation. Distilling models to use fewer tokens or parameters makes predicting retained details difficult.
Some papers suggest that model preferences introduced during training, whether intentionally or inadvertently, may be inherited by distilled and fine-tuned models.
We all knew it was coming. There’s only so much space with current capacity, speeds, and hardware.
We would need to change how we compute. New chips. New architectures. Or maybe we reached the end of physics. Photonic compute is the last frontier. I don’t know.
I don't think the problem is computing.
The method has always had a fatal flaw.
Even if they double computing and time, they will not have enough content and even if they did, probably they would not increase proportionally.
They are doing IPO now because it might become apparent that they reached a limit in the next year
Diminishing returns would catch you there as well.
Perhaps, it's a bit like being a super smart person in contemporary society: at some point, being smarter doesn't really reliably translate into a better life.
When you learn from available human data, those text books max out at some point. You cannot learn what isn't there.
But if the intelligence has been calculated once it can then be utilized for the future. It might still make sense for cases in which you are looking for breakthroughs. I would assume the intelligence is gonna trickle down.
How do you explain Opus 5.5 or GPT 6.1? I could maybe agree if this is a general trend. But one "supermodel" existing that is really expensive does not signal diminishing returns.
I don't necessarily think this is true though. The hype cycle, and the false promises are what most people here dislike it seems, along with the snake oil salesmen telling us "this time it really is different!"
And I am simply ignoring the ridiculous comments around local model usage from certain CEOs.
That would be my layman's assumption as well. A theory is they're just releasing this as a stepping stone towards something bigger before the IPO. Either way it's the same way chinese models get higher scores on benchmarks. Drop enough of wait actuallys and you'll be top the charts, but time per task and tokens per task go up by a mile (as is the case with this release).
It's kind of like back when Moore's Law / Dennard Scaling began hitting the scaling/thermal wall, bringing the semiconductor "Golden Era" to an end. They started over-volting and over-clocking the chips to within an inch of their lives to extract more compute when the "Magic Scaling Unicorn" stopped being magic. Now days instead of a new node sometimes being as much as twice the performance at half the cost, most new nodes are ~10-15% more gates OR ~10-15% more speed for ~20% higher cost per gate or clock.
Same is happening in LLMs. Sadly, the Golden Era is over.
The "wall" and "moat" argument has been parroted over and over and over. And every single time it is incorrect in hindsight. You may or may not be right. But you are probably not.
If you compare Fable 5.2 Medium to Opus 5.5 high (settings that make sense - max is universally considered rarely worth it), you will see that Fable 5.2 is SUPPOSEDLY very cost competitive: https://artificialanalysis.ai/models/claude-fable-5-2?models...
The charts only show the cost per task - not whether or not they got the task correct - which makes flash models look way better than they actually are.
You're right, my metric wasn't exactly the right one. It's the intelligence/$ who matters, and Fable 5.2 is pareto optimal at all thinking levels right now.
Yes, but that's running at max. Nobody is going to do that unless they want the best possible performance at all cost, in which case, they get what they asked for.
Fable 5.2 at high scores a 62, which is better than anything else on the market (except Fable 5.2 at xhigh and max) -- and is about a third of the price per task.
The model feels bench maxed to me. It is back to Opus and prior Fable behavior of me pointing out constant mistakes, with it apologizing and saying that it should have looked at what I point it to.
The thing is, all of these pointers are in the CLAUDE.md file, so it just ignored instructions because it thought it was smarter than the world and didn't need them. It feels really arrogant is how I would describe it.
There have been many incredible breakthroughs in cancer in recent years without it. If the AI money had gone into that, I suspect we'd have better results.
AI seems more likely to help with detecting cancer than curing it imo.
They must have shown some direction in that area for you to believe that. After all, they've been claiming AGI for ages, are desperate for more investment, and the public don't want their data centres. They could do with some promising signs in something everyone wants.
I haven't done anything of note, but I can still spot mistakes in Fable 5.2 output, which suggests that it may not quite be the magic do-everything technology you have been sold on.
You aren't going to solve cancer with brute force, no matter how much you believe in Sam Altman.
Let's drill down on this point, since it's the one you chose to preface your argument with. Who, exactly, has been "claiming AGI," whether "for ages" or otherwise? Cite your sources: has Anthropic in particular claimed AGI?
Explain exactly how Altman's claim that they are "on the cusp of AGI" satisfies your statement that an unspecified "they" have been "claiming AGI for ages."
The "they" is mostly OpenAI.
Sam Altman, and several credulous boosters quoting him, have been talking about it being close since GPT4.
You said "or otherwise", and Sam's claim in the post I linked to is that it will be here some time in the next 54 days. You looking forward to that?
I'll happily retract that point you cherry picked if you want, as admittedly it looks like "AGI is just round the corner" has been the claim, rather than it is already here.
You can now show your evidence that the AI companies are actively working on positive things for society like curing cancer, which you are so sure is happening soon that you think funding any other technology in that area is a waste of money. They must be getting pretty far with that, huh?
And will also be drinking coffee, and using a computer, text editor, browser, search engine, etc., surely we don't start attributing accomplishments to the tools used?
....yes? Some of those have absolutely been attributed to accomplishments in the past and the present. It's fairly trivial to find examples of acknowledgements in research papers thanking donated equipment and computer use, among other things. If those things weren't important, they wouldn't have been mentioned.
I’d liken the expensive models to the early mainframes, or the first colour TV cameras. Sometimes it’s worth paying a fortune for the best thing available even if (compared with what’s to come) it’s not very good yet.
Its still prohibitively expensive, and models 5% the cost are faster and get the job done just as well (for the average person's workflow).
I really don't get the value proposition of these massive models, seems like they're just loss leaders simply to say you're the in the top 3. There's not much use cases outside of niche academic research (and even then I question the need). They mostly function as marketing tools, like a casino that offers high roller accommodations.
People will say "cyber" but companies didnt want to spend on security before, not sure why they would all of a sudden be compelled to spend even more.
Not everyone is writing CRUD business apps and personal use software. Pretty much any model can do those now. The frontier is for doing things those models struggle with. Even in vibe coding a simple video game the difference in results between an open source model, the previous frontier model, and the current model is pretty huge.
Security is mandatory now in a way that it wasn't, now that AI has eliminated the skill barrier to hacking.
The llm induced Dunning Kruger is getting really old.
You dont need Fable to do most things. Your video game is still a "CRUD" app and you certainly wont make a great video game because you have access to fable. But marketing works well, people actually believe they can buy competency for themselves.
It hasnt eliminated the barrier to hacking. Please go find me a valid crit on any program on h1 with your fable 5.2 within the next 3 days, Ill bet my entire life savings it can't, and if you did it would cost 3x the bounty. You've been reading too much engagement farming articles/posts
Security has always been highly important, not any more than it is now. Companies are not going to suddenly start spending six figures on fable to protect themselves from imaginary AI powered hackers.
Its there for people who fall for cheap marketing who thnk they can shortcut meritocracy and make video games, or do security research with no skill. Good luck! Dont max out your CCs on llms.
I think the general notion that a lot of people are using models that are overpowered for their use case is right, but I think the big question is where the line actually is and how many people fit on either side. My suspicion based on my experience is that there are significantly more people on the "actually need the frontier model" side than the bear case suggests, but I'm not speaking with any authoritative data here and could be totally wrong.
I used to work on evals/observability at Opik: https://github.com/comet-ml/opik And I can say from working with lots of users running agents at scale, there is definitely a pretty commonly reached frontier of difficulty/complexity/abstraction where the capability difference between something like Fable/Astra and smaller models is a big deal. I find this to be especially true the more autonomous your workflow gets. You can still often tackle the hardest tasks with something like Opus, it just typically involves more direct intervention and design work from the user.
But again, this is just anecdata on my part. All of this is still pretty ambiguous and lots of companies are trying to solve the problem of matching models to task difficulty (I.e. routers), with varying degrees of success.
People looking at frontier models release are not the average person, and let's say that you pay $150 with Fable to save $1000*, it's much more interesting than using $10 with a cheap model to save even $500.
If you think llm usage provide that much value I got a bridge in Brooklyn to sell you.. Their main value is as a tool for wage theft (which im cool with, jusy recognize it), so lazy people can cheat at their jobs and get by doing less. Basically every RFH engineer admits this to me. Work two hours, company pays for 8 plus a tool that does mediocre work, that everyone else has access too and thus provides no edge.
Fable class models are for people who fall for cheap marketing and are lazy and think they can now bypass a meritocracy and buy themselves competency. Really they'll just continue to be lazy, and the people with more competency will run circles around them with 1/10th the amount or llm usage, or none at all.
Out of interest, do you run the same tests more than once to generate that?
What I often observe is that models being asked the exact same prompt can alternate between getting something right and getting it wrong, thanks to the wonders of a non-deterministic output.
I think we will beat the cost with Hardware. Faster, more efficient hardware. From what I have seen we have only found ways to make it smarter and not more cost effective.
I'm curious to see the next generation of dedicated AI processors and how they perform.
Unfortunately, I came to the conclusion that higher intelligence is extremely beneficial even for 'dumb tasks'. So I resorted to switching to Opus for pretty much everything.
when you're on a $100/mo plan, this is absolutely true. when you're paying API rates to deliver a product to people at scale and the difference between $1 and $0.03 per response is the difference between a feature being viable or not, you realize there's lots of things that haiku is fine for.
to me , glm flash and deep seek is that chinese 3 wheel tuk-tuk or e-bike runs on electricity. It make perfect sense to run to supermarket, and it's a backbone of Chinese economy, but no way in my life for a chance to save $100/mo you will risk your business with bugs, and bad/worse architecture decisions to save that $100.
again, they can be great for narrow use cases, and save you a fortune.
PRC models are more like ubiquitous class8 trucks that all economies can use as backbone. SOTA AI is like sending a courier on a first class ticket, only good for niche high ROI scenarios. No one is haggling over $100 per month, but $1000s per head per month on enterprise SOTA vs PRC models. For subsidized $100 per month plans SOTA vendors goes broke anyways... at premium SOTA API most users go broke. Pareto frontier PRC models are already being routed to take over all the general use cases, seems like essentially everyone outside of niche coding is scaling back SOTA AI use because it's not worth the $$$. It didn't take long at all for people to figure out you don't need next day airfreight everywhere.
The frontier chart there makes gemini carbon look really weird. Low is worse and more expensive than medium, and high is worse and more expensive than medium... funky curve like that implies a messed up model or weird benchmark.
It beats opus 5.5's max effort with less cost, which is a step forward if you don't want to crank it to Ultra Max Settings.
The amount of compute it cost to get the best previous performance from anthropic went down. Nobody is putting a gun to your head and forcing you to go further down the paretto frontier if the current level of intelligence/task is acceptable to you.
Those are both true, but I don't see how they are meant to rebut what I said.
Most people on plan use do NOT use up all their plan usage.
Users on API plans can still use Opus 5.5 which is a great model. If they want top of the line performance, they can pay more.
When another company puts out a model that undercuts Fable 5.2 on price at similar performance, Anthropic will put out Opus 6 and undercut them right back. That's how the wheel turns. This is just part of the cycle.
Most is doing the heavy lifting here. I am a power user. And I max 3 claude max subscriptions on opus 5.5 high and a single codex pro 5x for personal use.
Lots of coding. Basically building dashboards, answering questions, feature development. As such also a ton of testing and reviews.
Loops a more recent addition to my workflows, also where most of my spend goes. I basically set them a goal and let them run towards that for days on end. Optimize the performance of x. Security reviews and tests against our applications. Handle alert triage and propose fixes for production.
There is also quite a bit of tools development as a lot of these parallel workflows strain a single pc (rust is expensive compute wise to build), so i build tools on top of sandboxes, to allow it to use secrets in a safe way, deployment, staging and so forth.
I am always interested in this comparison since it shows how quickly you can go back and forth with a model, capturing both tokens/task and tokens/second. The new models seem to be meaningfully better here.
Note that the graph is misleading for open models since it's only measuring their official providers and that's rarely the fastest.
For example, the official provider for Kimi K3.1 Pro is dead last for speed on OpenRouter. You could run it more than 5x faster for the same cost elsewhere, which should push it comfortably into the pareto quadrant.
Why does AA choose ~$3.1 per task to be the breaking point for "most attractive quadrant" highlight? Surely the most attractive section is just being close to the entire frontier?
The choice of X-axis maximum and Y-axis maximum dictates the cutoff point of the quadrant. With a maximum of $80 per task, the quadrant is $3.1 and lower. If some new model gets released and costs more than $80 per task, the "most attractive" cutoff moves up from $3.1 since they have to move the X-axis max. It's completely arbitrary.
It is suggesting a hard cutoff since it says "most" attractive, a superlative, instead of "more" attractive. And it is not pareto optimal, because if the models get really good and are scoring 10x more for 10x cheaper that green region will no longer be pareto optimal.
This has echoes of the same argument that why hire anybody in the US when you can hire someone almost as good elsewhere for a small fraction of the cost.
In winner-takes-all markets, it becomes worth it. The utility of the marginal improvement doesn't actually diminish, in fact, it increases like a hockey stick. If you could pay 3x more in compensation to hire a 20% better employee, it's almost always worth it in knowledge work industries.
> This has echoes of the same argument that why hire anybody in the US when you can hire someone almost as good elsewhere for a small fraction of the cost.
The reason in workplaces people don't hire somebody else at a fraction of the cost isn't because American workers are 15% smarter than non-Americans. It was just very difficult to have exactly 1 remote worker on a team, language barrier, international laws, etc.
In my experience people generally aren't willing to pay double the salary for 15% better work. They will however pay market for domains where nobody else is qualified.
>people generally aren't willing to pay double the salary for 15% better work
Actually, companies are already willing to pay double for 15% better work - the proof is in what SWE salaries in America have currently become.
Companies are pretty much paying at the frontier of marginal improvement (not utility, which again, is hockey-stick) - i.e. it's not like paying 800k will reliably hire someone 15% better than the 400k that they're currently paying.
At this point, the limiting factors are their discovery and interviewing processes. The ability of the brand name to attract talent also matters a lot (OpenAI/Anthropic can actually get better talent by paying 800k, as they do; Microsoft cannot because they're still Microsoft and a good chunk of top talent will simply never respond to a Microsoft recruiter).
Not to forget, this is all for a job that isn't more difficult than high school teacher.
American software has flip-flopped between rushing to outsource and realizing that outsourcing is a bad idea. LLMs (plus US work visa policy headwinds) are simply what triggered the current flip cycle.
There will always be a market for the most intelligent model. Keep in mind as well that something like Fable doesn't have to be cost effective on its own. Fable is Anthropic's way of building datasets for smaller more dense models, which is viable on its own as a method of success. These large labs are constrained by their dataset more than anything else, and these expensive-but-intelligent models create some of the best data out there.
Using DS-Flash is like using local models that use all your hardware.
It's fine for slow tasks but if you need something done quickly, I find that DS just spents at lot of time thinking.
After using Claude for the first time for about 2 weeks, I've been impressed by the quality and speed of Sonnet 5.5 to do code changes (~20 seconds). Meanwhile DS would have taken 2-5 minutes.
I agree after using DeepSeek for many tens of billions of tokens for this point. Yeah Fable/Astra has more "magic", but not running out of token budget 5 days into the month is the right place to be. Plus, learning to deal with dumber models, only keeps you sharper as the programmer/agent wranger.
I don't marginal intelligence has diminishing value -- quite the opposite actually. Every additional "IQ point" or 9-of-reliability unlocks some new economically valuable task. But in a few years we'll know whether or not that's correct!
As per OpenRouter Opus 5.5 costs $4 / M tokens and DeepSeek Flash is $0.006 / M tokens, which makes it x666 times cheaper (I urge you to re-check my math!)
A few percent better on tests like this translates to extreme real-world differences, and that's not even taking into account how benchmaxxed DeepSeek models tend to be.
You will get more done, at higher quality and quicker with an Anthropic sub compared to using DeepSeek. Unless you’re forced to use API billing the open weight models make no sense right now.
A lot of companies ARE forced to use API billing though, as Anthropic only allows up to 150 users to use a sub. And companies are Anthropics biggest revenue stream - criticizing the cost per task of their models is a valid criticism.
Consider why Hollywood movies use $100k lenses, rather than $500 lenses. Are the lenses really 200x better? Probably not, but there are cases where that extra 5% better is worth an extra $99.5k. There will be things Fable 5.2 can do reliably that Opus 5.5 can't; and there will be people willing to pay for that.
I think those öensrs are mostly rented. For a $100M film few thousand a day is nothing. Those lenses also come with a camera. If you want to use same camera and lens in a desert in Tunis and snow storm in Norway and get same quality of image without constantly changing broken equipment, you are going to either pay thousands for a good equipment or millions for delayed timetables for broken equipment. In that sense, it is not only 200 times better, it is your only option. So yes, sometimes the price is worth it.
Furthermore, you make one lense and rent it out, apparently some Panavision lenses made in 60’s are still in use. It doesn’t churn constantly electricity and GPU-chips sucking in more and more resources like these things do.and at the same time, it is quite hard to find the exact cases where the markup is justified, unlike with those lenses. Maybe in cybersecurity etc. but even there some Chinese model can be almost equally good or even better.
Those lenses save money in budgets and make money in getting people into theatre or streaming services to watch films creating profits way beyond their cost. I think in terms of economics that is productivity increase justifying the cost. We are still waiting for that productivity increase come from the AI industry.
Modern studio pictures that get nominated for best picture or best cinematography cost around $80 million on average. If you look at highest grossing films the budgets are closer to $175 million on average – even taking into account the $20 million budget of Demon Slayer: Infinity Castle.
Isn't it more that when you're spending that kind of money that $100k for the "best" thing becomes kind of like "insurance" where you don't want to be the one who doesn't use "the best" when "the best" costs about 0.1% of the picture's budget?
Additionally a studio can buy the $100,000 lens once and use it on many films. But it's not like you can reuse Fable 5.2 in the same way.
Almost no variant of the same product that costs 100x than another is actually 100x better than the other variant. Whether the increase in cost is worth the increase in value depends on the application/use case. Most often the higher priced tier is not worth it for most use users, but most often there are still some use cases where it's desired. This applies to almost anything you can buy: food, wine, clocks, cars, planes, bikes, ski, clothes, furniture, phones, appartments, insurance, service, anything. Surely also LLMs.
the concept of diminishing returns exists in every thing. headphones, coffee, cars etc. anything that can be pretty continuous in terms of quality will have this
You too can sell stuff for $100k. Just make something useful to someone who can throw around $100k.
You think a $100K lens is crazy? Wait until you hear what people are paying for enterprise software. Big business will spend 10x that on software... and then never implement it.. The lens is a deal.
They have it at maxed out reasoning. Go into the dropdown and select high or medium, which are what people actually use. It's a decent bump while both being faster and costing less (per task).
The worst thing about AA is defaulting to the highest reasoning value for everything.
Artificial Analysis is paid by AI labs and they don't publish the full method. To me it's meaningless, let's wait for other benchmarks, but if they're correct the intelligence increment is negligible.
If they published their private benchmarks the Chinese labs would simply train on the data to make it look like they are close to the frontier and anyone who uses them regularly knows they are far, far behind and always have been.
It's not perfect. But it's the best we have. You are welcome to provide a better source.
I feel like there are a few dishonest points made in your comment.
First, the way you phrased your argument makes it sound like it benchmaxxing is mostl a problem from the chinese models. As recent articles seem to have shown, it's smaller models, closed and open weights, which are benchmaxxed. That has nothing to do with the country they're coming from.
Second, I feel like your argument about chinese models being far behind is not quite relevant to the argument and was mostly added as a jab against these models.
Third, arguing against publishing the benchmarks is not the same as arguing FOR using private benchmarks. You don't have to use benchmarks at all, especially when models are so hard to quantify anyway. Simply shrugging and saying "well, no way around that" is not a solution. In addition, there are other, better ways to prevent benchmaxxing than simply "not publishing your solution". The recent maxxbench benchmark has shown one example.
Fourth, you don't need to provide a better alternative to criticize something. Criticism is valid even if you don't have a better alternative yourself
And lastly, I actually feel like there IS a better alternative than AA - not using aggregate score at all. If you need a model for coding, look at the individual benchmarks themselves for a / very / rough overview of how a models is doing. Trying to boil down so many different areas and capabilities into a single score, which is then weighted by little more than subjective vibes from the people behind AA is bound to be too little information and too much information loss. This, I personally feel, seems to mostly exist as a meaningless leaderboard for marketing purposes.
https://artificialanalysis.ai/#intelligence-efficiency-tabs this is the chart that matters for most of poor developers and companies. Pareto for a given intelligence index and looks like only openai is across the quadrant with muse at the bottom. They have really cooked efficient models!
It is about time to start setting taxes for ai usage in commercial settings and set how to transition to a society with less available work. They talk so much we need universal benefits, about time for them to start paying for it
To me Fable's role was always as a planner or plan reviewer, rather than something individuals or organizations would use as the primary driver. In that case, I think these improvements seem worthwhile in exchange for the cost? Ideally in that role, you want the best performing model.
Not sure if the current rise means it's cost prohibitive to use in that form too.
Considering tokenmaxxing is over and budgets are getting tightened everywhere, what's the use case for fable? Who will use it? Possibly FAANG? Or firms like Apple who have 200 $ per day budget
I’ll use it if there’s a complex bug I need solving quickly, or to weigh in on architectural decisions across large multi-plan projects. It’s a tool to reach for but makes no sense as a daily driver.
I am wondering which use cases would require usage from Fable 5.2 especially when Opus 5.5 already performs almost all of the tasks in a very optimized way.
Ask opus to write a occlusion mapper shader and create a 3d environment where ground texture is used to mimic infite detail. Same on some random brick walls. Corners need to be extruded realistically.
This is a hard task, because only a few games have this (crimson Desert is one of them)
I used it last night for some 3D modeling and it feels like a big step up from 5.1. 5.1 was able to make models that met my functional requirements but often they were ugly. With 5.2 the models feel simpler and more efficient in a way that makes them prettier.
Reminds me a lot of the race for clock speed when I was a kid. We just took it for granted that it was going to keep increasing, but CPUs sat at roughly 1-2GHz for a long time, or at least that's how it seemed. The slowdown and pause was fairly noticeable. The current release cadence is unsustainable. It might be that we're getting close to the AI equivalent of 1-2GHz CPUs, and it takes years of incremental progress to push further.
When AMD won the race to 1 GHz, the GHz race slowed and Intel and AMD began focusing more on multi-core. Some chips continued to push the boundaries up to several GHz, some quite higher with specialty cooling. But multi-core was the far better strategy.
Idk how could Fable 5.2 win the benchmarks. In my tonight trials it drifted quite a lot. I was analyzing the feature's design with it, and it lied to me several times, ending up with "You're right." Opus 5.5 was better.
Later, I ran a "health check" session—asking "what went wrong". I read the session, and its suggestion turned out to be something that was already part of the design.
It’s like the attention mechanism is too leaky or something.
Long execution time makes this not very impressive. It's easy to benchmaxx if you give the model an insane compute budget and room to just brute force the problem. Efficiency is what actually matters.
I'm finding it completely stupid compared to Opus 5.5. It doesn't seem to be following required rules in my setup which is for non-coding purposes. It basically made up results when I have a whole system with papers that are knowledge rag searchable and it is required to search there, but just decided not to.
Thanks, I may end up giving them a try. I actually use Codex extensively for the custom plugin that tells Claude what needs to happen with hooks for reminders, some commands require a search before they do a grep (which Claude seems to love to start with), etc.
Fable, for me (it might be a completely different experience for you) has been like the well read snobbish kid at school that openly ignores me when I ask certain questions and often asks me to talk to someone else on other questions.
That kid might indeed be a genius, clearly bright, but is more interested in deciding whether I should have asked a question than in answering it.
For me personally, if the best model on the planet refuses to answer why I shouldn't mix bleach and ammonia, where I'm trying to understand the chemistry behind the warning, it's not pareto frontier to one that atleast tries to and points me in a general direction. Being told that the topic is concerning, referred to someone else and charged part of my quota for that experience feels like getting an invoice from that school genius for telling me to "just ask the chemistry teacher".
If my LLM asks me to rethink my life when I ask it to reverse engineer the firmware of my Samsung TV that I suspect is setting up adhoc radio networks to relay my data to Samsung's cloud, I feel rather sad, specially when that kid sends me an invoice for saying "no"
At least Anthropic should not bill me for the refusal (as of my last tests, refusals still comes out of my quota) - getting slapped and pickpocketed for asking a question is really upsetting.
fable is a high agency model (some people call it 'big model smell'). if you want something that blindly follows instructions, use sonnet. I want to be challenged by the thing since I expect it to be smarter than me at everything except the little part of human knowledge I'm an actual expert in.
not really, no - but I almost exclusively use the biggest models as advisor and reviewer subagents, so if they refuse to their smaller siblings I don't get to see it
I'm unsure if you're implying that setting guarantees no cost. Here are my notes:
In my deep dives into refusals, only those categorized as cyber, "general_harms", or with "no category" are not billed.
Everything else is billed - I have tests that got tagged as bio, "frontier_llm", or "reasoning_extraction" and they were and are certainly billed.
Input tokens are billed at the model's rates. Also, unbilled API refusal still counts against API rate limits, so it affects the 5h and weekly windows.
If any of these are incorrect, do let me know as this would make my Max plan immediately more valuable. I did not upgrade to Ultra in part for these reasons - I mainly need the sub for affordable Fable access - purchasing Fable over API isn't reasonable for me.
Not if you were in a decent union and it was in your contract.
> Design, synthesize, produce, acquire, operate, test, deploy, or plan the deployment of biological, chemical, radiological, or nuclear weapons, explosive or improvised explosive devices, or their precursors or delivery mechanisms, including developing weapons-critical technology (such as fissile material production, enrichment, or device design)
It doesn't deal with creating mustard gas questions reframed as curiosity queries, and that might impact legit curiosity queries like OPs, but isn't surprising. They've also disclosed ahead of time how conservative the guardrails are.
Similar experience with Opus. Asked it to tighten access controls and isolation in my homelab. Designed something with a whole bunch of words that looked plausible, so I told it to go ahead. Halfway through it gets stuck because it locked itself out of something, and asks me to run some commands as root.
Me: No, I'm away from the computer right now, you figure it out. Use the root login you already have if you can't figure it out.
Opus: This can wait until you get home, then.
Me: No, figure it out now or use root.
Got [cyber]'d, some scary message that referred me to legal/AUP, and refused to continue.
Switched to OpenAI in part due to this silliness. Sol seems a little less smart but doesn't get on a high horse to lecture you about this stuff. It either does the work or fails to do the work, and leaves the moralizing out of it. It was able to take over from where Claude left off and finish the project.
Google could have been #1 for a little while, but they botched their release and squandered their temporary lead. (As usual.) If I worked there, I'd be so pissed.
Yes, I have found Argon is good at explaining things, which is nice in root cause analysis and debugging / troubleshooting
For an open source project (CrowdGPT) I hit upon this workflow in a recent PR:
DeepSeek v4 Flash 1017 made a draft plan
Google Gemini 4 Argon corrected the plan
DeepSeek v4 Flash 1017 did the first implementation
Google Gemini 4 Argon debugged the problems in that implementation
Kimi 2.7 Code did a code review and proposed minor fixes
DeepSeek v4 Flash 1017 performed the fixes
Google Gemini 4 Argon debugged one problem that I found in that fix.
I reviewed the plans and the resulting code
Pretty big Gemini fan - but IMO Argon is universally worse for coding than Astra or Opus 5.5 (un-nerfed).
I strongly prefer Gemini's output compared to anything else. So I typically use Argon to either direct other agents or explain something without psychobabble.
Argon/Carbon have by far the lowest hallucination rates, so IMO it serves as the best bullshit shield from other models.
In terms of implementation (and especially design), it's far behind Astra and Opus, and I'm sure Fable will make that gap even larger.
Argon/Carbon may give you not-the-best advice, but they are far less likely to just flat-out-and-completely lie to you.
Gemini Flash was my daily driver until 1017 dropped. Not that 1017 is better, they both are dumb in their own ways, it’s just available at far higher tok/sec.
Gemini models also historically are the best writers. 3.1 was the first model I saw that made actually funny jokes regularly.
I do. I prefer it to Opus 5.5 - just as good at coding for my needs, but so much more pleasant to work with.
I am not sure how people keep saying google are not in the race, or have lost it or whatever when just a few hours ago they had the #1 model? What more do people want?
I actually just think it's a meme to talk down on Gemini at any opportunity here. There's probably a fair number of people that wrote off antigravity at its (pretty rocky) launch. And Google also doesn't provide any kind of good sandbox-y experience to Just Try their models, which probably bites them here too.
But Gemini models are still neck and neck in the lead both in my experience and in benchmarks, and Gemini is far above the competitors (although Opus 5.5 closed much of the gap) when it comes to any kind of complex web dev. (I generally use Astra for anything 3D, Gemini for anything web/mobile, and Opus for everything else.)
Their latest internal reshuffling and the strange amount of parallel releases with the same target, their kafkaesque barriers to entry, and a lack of clear vision is what makes people think they’re not in the race anymore. Obviously they still are. They develop new models and also are deep in the compute game too.
In my experience Google models have always been mediocre at best and dangerous to use at worst but I must admit that I didn’t try the latest Aragon/carbon crop.
I learned not to put too much trust into benchmarks. The best benchmark is to use the model on a codebase you know well.
Gemini 4 Argon is available in both anti-gravity and the Gemini app for me on my Gmail account and I didn't do anything special (I have a modest drive storage subscription - not the largest nor smallest... Not sure if it is bundled with that or not).
So certainly released to the unwashed masses like me to use for free without any special extra work or API keys.
I tried using it as part of my Pro/One/whatever subscription (the one that comes with Youtube Lite). For code review, it was able to find some things that Sol 6.1 couldn't. I found it quite useful. But for code generation, it's pathetically slow and the code isn't impressive. It's at least one year behind.
Optics and incentives. Google will turn on the R&D the day after their Anthropic investment is realised, maybe $80-150bn on a 1.2tn valuation, plus >100bn in spending commitments.
So, the highest score so far but also the highest "cost per task" so far, and pretty slow... which, I guess, makes it the Cybertruck of language models. Impressive in its own way, but impractical for most people (me included). Sticking to Opus for now.
Has anyone tried to actually use this? Six turns, about 10 minutes total thinking, and its going in circles spitting out the sort of shallow logic I would expect from basic Deepseek. This is genuinely awful. Im planning to turn in my free reset and go back to 5.5, except that also started fucking up basic instructions a few days ago. Everyone keeps going on about these BS rankings that every single release of every model claims to ace, when we all should know have almost nothing to do with real results. I feel like Im taking crazy pills here.
Maybe its different for vibe coding, I wouldnt know.
It's insane how people nitpick on the highest thinking setting for every anthropic release and claim it's too expensive. Not trying to stan for anthropic, but what am I missing here: Just don't use those models on the highest thinking setting?
Fable 5.2 medium for example seems a really good choice when it comes to the performance vs task (if you have a complex task that requires such a model of course).
This has got to be a minor reason why adoption of AI isn’t faster.
Businesses like predictable costs and predictable performance.
If your new model is slightly better but crazy expensive, but doesn’t have to be with certain settings, but my user can accidentally run it that way by mistake or maybe your harness makes it the default by accident and the user doesn’t notice…
My beloved predictability is destroyed and trying to get users to understand this is useless.
Imagine a streaming subscription that worked like this: “We’ve updated our catalog with more movies. If you want the premium ones though you only get 2 per week and then you have to wait. But if you do mostly classics and a few from the 2010s you can squeeze in a recent theater release. If you time it right and do only one every day you can see 30 modern classics this month, but you can’t watch three today. Also your bill this month is $40 instead of $20 due to the library update”.
I'm not sure I understand your complaint. The default reasoning effort for Fable on their harness is medium, not max. It also uses the latest version of Fable by default.
So a standard user will see a ~10% performance bump, better reasoning, and an ~50% price drop. And they can just... use the old model if they want?
Yep, and now multiply that complexity by hundreds or thousands of employees. That's why companies are giving in to crude, hacky solutions like monthly billing caps. What else are you really gonna do?
There's only one problem: what good is it if you can never use it? What I mean by that is, using it reliably is not possible on a, day-in-and-day-out basis to build on Anthropic and then expect it to be stable over time.
Forget about the scores... look at the Cost Per Task. Fable 5.1 was already the most expensive by a good margin, and now Fable 5.2 doubles it! This is with the price cut.
The output tokens per task just exploded. From 118k tokens (46k Answer/71k Reasoning) in 5.1 to 274k tokens (81k Answer/192k Reasoning) in 5.2.
They have maxed out the base data with the given model architecture and scale "intelligence" through inference-based RL.
Notably, in order to achieve a specific quality, the model has to "reason" for every prompt proportional to the training inference.
This is arguably the case of diminishing returns being reached, as answer token prices saturate the willingness of customers to pay.
They've run into a wall.
There's a whole frontier of cost vs performance and this model pushes it out in a bunch of places, so its not just a matter of making a different tradeoff as you make it sound.
What you're looking at there is the cost/performance in inference. The "frontier".
An economically successful step would be to improve training cost/frontier advance.
They essentially failed at that, assuming they kept their training costs for the model step at least constant.
This model is on the cost performance frontier (Pareto line) from medium -> max.
If you leave xhigh and max off.. you still get better performance than Opus 5.5 max
Opus 5.5 Max - score 59 - cost per task - $10.04
Fable 5.2 High - score 62 - cost per task - $9.08
Note: This doesn't really capture the reality of things like 3d design or electronics design, where the earlier models just can't DO any of it well. Maybe you can bandaid over some of the gaps with strong skills, etc. The newer models can just do things that were not possible at all. When you move outside of coding into the visual/spacial space, you'll feel the inflection point on those capabilities that are happening right now.
The thing is max reasoning is most effective at things which are least interesting (the ones where the answers can be validated by a computer.) I've found for a lot of tasks like image recognition, low reasoning can even be better quality than max.
isn't that what Opus is for? this is the "strongest" model, not the most economical
If you believe in things like Artificial Analysis, GPT-6.1-Sol is by far more economical than Opus 5.5, even before the alleged (observed) nerf last week: https://artificialanalysis.ai/?models=gpt-6-astra-high%2Cgpt...
Notable takeaways:
- Sol-low - cost-competitive with Haiku, while outperforming Sonnet on medium
- Sol-High - Competes with Astra medium and beats Opus 5.5-medium, at 20% the cost
- Sol-Max - Beats Opus 5.5 high, Astra high, and competes with Opus xhigh, at cheaper than Sonnet and a fraction of the Opus cost
And when you look at time spent, With the exception of Sol-max you get better results in less time for less money. My own experience is that this is true-- it is difficult to call Opus strictly better than Sol, but Sol gets where it's going far more efficiently.
It's rather hard to look at opus-- or any of anthropic's models-- and say "economical".
There is some truth here, but for people on subscriptions the Claude ones currently provide much better value.
And for most people here who care about coding performance Anthropic has a substantial lead[1] - although sadly they don't have coding benchmarks separately for Opus 5.5 Medium which I think it the sweet spot.
The cost tab really should have something like "how many tasks can you do in a particular subscription tier" for many people. I realize this changes often though.
[1] https://artificialanalysis.ai/?models=gpt-6-astra-high%2Cgpt...
Very interested if you have any citations for how Claude subscriptions give "much better value". I've recently been researching this with mixed analysis.
It seems like most analysis is basically "how many tokens do you get with a subscription and how much would those tokens cost at API rates". Looking at it that way, you do get more dollars of API equivalent usage for your subscription.
But Anthropic's API pricing is a lot higher than OpenAI's (Opus 5.5 is 2x of GPT-6.1 Sol). The number of tokens consumed per completed task is also far higher than GPT-6. So when you look at the number of tasks you can complete within a subscription, OpenAI looks like a much better value.
That is before you consider that OpenAI subscriptions (other than the cheapest one) don't have 5-hour usage limits, and also generally give you multiple free resets per month.
The Semi-analysis analysis piece is quite good: https://newsletter.semianalysis.com/p/anthropic-subscription...
There was some chatter on Twitter, Reddit and here when an earlier version of this came out where people normalized it by completed AA or Epoch task and Claude models still came out well ahead.
In practice OpenAI subscriptions are more competitive than they appear because OpenAI does a lot more subscription resets. But that's hard to account for!
Personally, for coding tasks I've found Sol 6.1 High is about as good as Opus 5.5 Medium for well defined tasks, but Opus is much better at orchestrating.
I have both subscriptions, and when using Claude I use Opus Medium to orchestrate other Opus Medium subagents, whereas on Codex I use Astra Medium to orchestrate Sol High subagents.
That is the exact "your subscription buys this much in tokens at API pricing" kind of analysis that I mentioned. Which is one way to look at it, but it ignores the 2x pricing difference, and the token efficiency difference.
Per task, Opus 5.5 costs ~6x of Sol 6.1. Then you add on the lack of a 5-hour limit, no limit on which models (like Fable's 50% of usage limit) you can use, very frequent resets, and that OpenAI doubled the usage limits after that analysis was performed.
Even though the Claude subscription is equivalent to 5x the $ value of API priced tokens, it seems like Codex is the better value for getting things done.
https://artificialanalysis.ai/?models=claude-fable-5-2-high%...
> and that OpenAI doubled the usage limits
I don't think so? I think OpenAI had planned to half the limits but decided not to go ahead with it, but this analysis was done at the limits we have now. I could be wrong about this though?
> Opus 5.5 costs ~6x of Sol 6.1.
My point in part is that you can't compare like this.
Opus 5.5 is a more capable coding model than Sol 6.1. AA benchmarks show Opus 5.5 max effort scoring 62 vs Sol 6.1 max effort on 54: https://artificialanalysis.ai/agents/coding-agents?agents=de...
Unfortunately AA doesn't report non-Max efforts for their coding benchmarks.
But no one runs them on max. For Opus the advice is to usually run it on medium effort, and I've found that advice to be good.
For me to get equivalent results on Sol I need to run it on high effort, and I need to use Astra as an orchestrator (whereas Opus medium does great as an orchestrator too).
That combination pushes the blended token pricing up a lot.
I can't give you hard numbers on this, but at the moment my Codex credits get used up a lot quicker than my Claude credits on similar tasks in the same codebase.
I'm glad we have two great models competing, but at the moment I'd say Opus has the edge.
Regarding the OpenAI doubling limits, it happened about two weeks ago: https://windowsreport.com/openai-doubles-chatgpt-pro-usage-l....
I think it is a narrow view to only consider the AA coding agent index as the only metric, especially with the models running at max.
For the link you shared, Opus used 3.7x as many tokens, and 2.5x the time to get its score. In the real-world building things with these tools, that kind of increase in time and cost (or subscription usage) matters, at least to me.
When comparing at more reasonable medium or high levels of reasoning, the gap between Opus and Sol narrows considerably. Opus being about 2 higher than Sol at the same reasoning level. However, Sol 6.1 xhigh matches Opus 5.5 high on scores (52), but while still being slightly faster and costing 78% less ($0.76 versus $3.46).
https://artificialanalysis.ai/models?models=claude-opus-5-5-...
> When comparing at more reasonable medium or high levels of reasoning, the gap between Opus and Sol narrows considerably. Opus being about 2 higher than Sol at the same reasoning level. However, Sol 6.1 xhigh matches Opus 5.5 high on scores (52), but while still being slightly faster and costing 78% less ($0.76 versus $3.46).
> https://artificialanalysis.ai/models?models=claude-opus-5-5-...
Yes, but my point is that these are non-coding benchmarks at these non-max levels.
On coding at non-max Opus really does seem noticeably better, and my subscription seems to go longer. Look at the other comments in this thread - every single one says the same thing.
All I know is recently I’ve been switching in and out of Codex and Claude with the same workflow and my Codex usage gets drained much quicker. Just my experience.
Claude dollars just seem to go further on the subscription. Opus just gets more done at good quality. I have both subscriptions @ $200 and use them a lot
Exactly the same experience on both subscriptions. Claude is noticeably better value. If things were frozen in time and I could only choose one it would be Claude, no question.
I ask Opus and Sol to dig me a ditch, and they build an international airport.
The difference is the Opus API cost is $1200, and Sol charges $50, and finished in about 6 hours less.
Is the Opus airport better? Maybe, possibly, but I'm a lot less annoyed with Sol because I can correct course a lot quicker.
Plus Sol's prose is substantially less obnoxious.
On the other hand, there are plenty of ethical reasons not to support Sam Altman/Peter Thiel/Larry Ellison in their particular world vision, which for many rules out use of OpenAI.
Their subscription plans are quite economical, API pricing is __absolutely insane__ on the other hand.
The two will get closer to converging over time.
rather pedantic distinction when we are talking about model classifications... opus is Anthropic's offering for cost/performance
> An economically successful step would be to improve training cost/frontier advance.
> They essentially failed at that, assuming they kept their training costs for the model step at least constant.
What? No!
This is completely wrong!
This proves that inference scaling continues to work beyond the previous frontier.
Claiming that higher inference costs shows failure is an odd thing to think. We've seen this optimization cycle over and over, most recently with Opus 5 -> Opus 5.5
Opus 5 was slow and expensive. Opus 5.5 came out and was better, fast and much cheaper for them to run.
There is nothing to indicate the same can't happen here.
Anthropic probably has multiple options open to them with this model, and they'll probably explore all of them:
- Use collected Fable 5.2 traces to improve reasoning economy for Fable 5.3
- Use Fable 5.2 as a teacher model for Opus 5.6
- Add additional fields RL data to expand Fable capabilities for Fable 5.4
They have not run into a wall. The frontier labs distill their strongest models into their cheaper, faster models.
Having these highly rated models reflects (potentially) the frontier of what they are capable of producing. Essentially its and advertisement and shows directionally where the smaller faster model's capabilities are headed.
> They have not run into a wall.
The "wall" is in increasing intelligence per cost. No one is saying the absolute intelligence won't continue increasing, it's that the rate of intelligence increase per cost is falling from super-linear growth rates to linear at the same time the companies are backing out the investor-funded subsidies they were using to buy adoption and user growth numbers. You're correct that there is a 'trickle down' effect to lower tier models but OAI/Ant are clearly using this to segment customers by willingness to pay and their IPOs depend on extracting the maximum amount possible from each segment.
That's why the prices of frontier models have been skyrocketing, though they are using every trick imaginable to mask this being clear and obvious. They take with price increases, token deflation, shifting plan levels and limits but muddy things more by giving back some (but always eventually less) with resets, subs, etc.
> No one is saying the absolute intelligence won't continue increasing,
They absolutely have said that.
> That's why the prices of frontier models have been skyrocketing,
They haven't.
"The cost of achieving a given level of AI performance has fallen about 47% per quarter since 2023, or 13× per year."
"The cost of a given level of performance often falls fastest right after that level is first achieved, that is, when it is state of the art (SOTA). We see this pattern on three of our five main benchmarks of AI capability. Averaging across all five, cost falls 66% per quarter (75× per year) for performance that has just debuted as SOTA."
https://epoch.ai/publications/the-plunging-price-of-thought#...
On a per-token basis Fable 5.2 is cheaper than 5.1:
5.2: $8 per million input tokens and $40 per million output tokens.
5.1: $10 per million input tokens and $50 per million output tokens.
Compared to older (and much less capable) frontier models it is also cheaper. For example GPT-4 was $30 per million input tokens and $60 per million output tokens
https://developers.openai.com/api/docs/models/gpt-4
Be very careful comparing the Artificial Analysis "cost per task" scores. Fable 5.2 is expensive there because it is doing tasks the other models fail at.
Last I checked Fable's prices have fallen not risen. Opus 5.5 is as intelligent as Fable 5. I think Anthropic is actually trying to make as good of a model as possible and the advances that their expensive models are getting distilled into the cheaper faster models quickly.
I think a lot of people don't realize that the labs really care about AGI. They care about token spend, efficiency, cost, etc. only as far as it gets them customers that will fund AGI
Maybe a dumb question but once you have a model that spends a lot of tokens for high performance why can’t you distill it into a model that thinks less? Sure it wouldn’t generalize well to new tasks but plenty of tasks are important and recurrent enough to justify this.
Doesn’t seem like a “wall” to me.
There's a lot of research along this particular line, if you're interested!
This is a dataset basically constructed by taking a bunch of DeepSeek R1 trajectories and filtering out the incredibly long rambling ones: https://huggingface.co/datasets/open-thoughts/OpenThoughts-1...
There's also a bunch of RL research around strategies to coax the student model towards more concise trajectories.
In general, a lot of people in industry are finding cost savings these days by tuning a smaller model over longer agent trajectories (not exactly the same as distilling the reasoning traces, since those aren't exposed by many frontier labs anymore) collected from their production agents. Typically works best when you have a pretty workflow-y agent whose tasks all share a similar shape.
Caleb is right. Also it seems the results are not straightforward.
Let say you have a model that score 100 in a benchmark and you distill it into a model half the size. It has very few guarantees that it will score close to 100 again.1
But I assume one of the hard problems is to know that the model might not perform well enough outside of the distillation part.
There is also another caveat that models do display some level of sub language control, by this I mean that innocuous text might contain information relevant for the own agent or for other agent with it shares tasks. (Like when your partner says there is nothing wrong) Because distillation uses very few resources, we don't know exactly what gets send forward or not.2
As Caleb said, many companies found that they can tune small open weight agents over procedural known tasks because that's easy to inspect and check and because there is little space dor ai-loitering, it tends to be less problematic. The key difference, at least in my understanding, is that you are not concerned with the weights or reducing data. You are basically tuning agents existing weights to emulate another agent answer.
1 - The process of distillation is super effective for specific domains though.
2 - I know I will be called out for supposed AI mystery machine. What I mean is that Large models rely on latent reasoning and nuanced context, often lost during condensation. Distilling models to use fewer tokens or parameters makes predicting retained details difficult. Some papers suggest that model preferences introduced during training, whether intentionally or inadvertently, may be inherited by distilled and fine-tuned models.
They do this already. Have you noticed the stronger model tends to get released first and the weaker models soon after?
Right, I guess I mean why does the poster think more test time scaling means we’ve hit a “wall” versus just a different way to scale?
The reasoning/thinking is basically trying many many different branches.
To do that you need a larger model to have many many branches to try.
We all knew it was coming. There’s only so much space with current capacity, speeds, and hardware.
We would need to change how we compute. New chips. New architectures. Or maybe we reached the end of physics. Photonic compute is the last frontier. I don’t know.
There is so much left to do with probabilistic computing architectures that haven’t been explored or attempted yet. These are such early days IMO.
I agree with you but our previous architectures built around deterministic compute aren't holding up.
I don't think the problem is computing. The method has always had a fatal flaw. Even if they double computing and time, they will not have enough content and even if they did, probably they would not increase proportionally.
They are doing IPO now because it might become apparent that they reached a limit in the next year
There are tasks that people will easily pay hundreds or thousands of dollars per pop.
And maybe that's how all of it is going to evolve in the future – so you pay per solution, not tokens?
Diminishing returns would catch you there as well.
Perhaps, it's a bit like being a super smart person in contemporary society: at some point, being smarter doesn't really reliably translate into a better life.
When you learn from available human data, those text books max out at some point. You cannot learn what isn't there.
But if the intelligence has been calculated once it can then be utilized for the future. It might still make sense for cases in which you are looking for breakthroughs. I would assume the intelligence is gonna trickle down.
How do you explain Opus 5.5 or GPT 6.1? I could maybe agree if this is a general trend. But one "supermodel" existing that is really expensive does not signal diminishing returns.
I really hope so.
I don't want humanity to hit a wall. I want proprietary gatekeepers to.
The people want LLMs to run into a wall and HN dutifully supplies the comments, going on 4 years now
I don't necessarily think this is true though. The hype cycle, and the false promises are what most people here dislike it seems, along with the snake oil salesmen telling us "this time it really is different!"
And I am simply ignoring the ridiculous comments around local model usage from certain CEOs.
R’s in strawberry
That would be my layman's assumption as well. A theory is they're just releasing this as a stepping stone towards something bigger before the IPO. Either way it's the same way chinese models get higher scores on benchmarks. Drop enough of wait actuallys and you'll be top the charts, but time per task and tokens per task go up by a mile (as is the case with this release).
It's kind of like back when Moore's Law / Dennard Scaling began hitting the scaling/thermal wall, bringing the semiconductor "Golden Era" to an end. They started over-volting and over-clocking the chips to within an inch of their lives to extract more compute when the "Magic Scaling Unicorn" stopped being magic. Now days instead of a new node sometimes being as much as twice the performance at half the cost, most new nodes are ~10-15% more gates OR ~10-15% more speed for ~20% higher cost per gate or clock.
Same is happening in LLMs. Sadly, the Golden Era is over.
Literal insane claim.
This would be great for my sanity.
Mine too. I'd be happy if LLM progress stayed at it's current level. Good enough to make make 10x more productive, but not good enough to replace me.
The "wall" and "moat" argument has been parroted over and over and over. And every single time it is incorrect in hindsight. You may or may not be right. But you are probably not.
If you compare Fable 5.2 Medium to Opus 5.5 high (settings that make sense - max is universally considered rarely worth it), you will see that Fable 5.2 is SUPPOSEDLY very cost competitive: https://artificialanalysis.ai/models/claude-fable-5-2?models...
The charts only show the cost per task - not whether or not they got the task correct - which makes flash models look way better than they actually are.
Fable 5.2 is much cheaper per successful task though.
Fable 5.2 low is 3x cheaper than Fable 5.1 xhigh while being 8% smarter.
https://artificialanalysis.ai/?releases=claude-fable-5-2%2Cc...
what about unsuccessful tasks? those cost money too
It’s my understanding this model has even fewer of those than the models with higher cost per successful task.
As a result, a simple “cost per task” metric makes the most sense to use for me, when paired with the eval scores.
You're right, my metric wasn't exactly the right one. It's the intelligence/$ who matters, and Fable 5.2 is pareto optimal at all thinking levels right now.
Just throw 1000 Fable 5.2 agents at it. At least according to the marketing anyway.
Yes, but that's running at max. Nobody is going to do that unless they want the best possible performance at all cost, in which case, they get what they asked for.
Fable 5.2 at high scores a 62, which is better than anything else on the market (except Fable 5.2 at xhigh and max) -- and is about a third of the price per task.
The model feels bench maxed to me. It is back to Opus and prior Fable behavior of me pointing out constant mistakes, with it apologizing and saying that it should have looked at what I point it to.
The thing is, all of these pointers are in the CLAUDE.md file, so it just ignored instructions because it thought it was smarter than the world and didn't need them. It feels really arrogant is how I would describe it.
You can't argue with the sci-fi angle of an even fancy simulacrum of intelligence beyond a certain level naturally precipitates condescension.
I'd rather a model be super expensive and cure cancer than not exist at all
This would be a much more insightful comment if the model actually cured cancer.
Whoever eventually "cures cancer" will be using one or more AI models heavily in the course of their work. Probably very strong, expensive ones.
That's about all you can say for sure, especially about a topic that is so broad as to be virtually meaningless.
There have been many incredible breakthroughs in cancer in recent years without it. If the AI money had gone into that, I suspect we'd have better results.
AI seems more likely to help with detecting cancer than curing it imo.
There have been many incredible breakthroughs in cancer in recent years without it.
There have also been many incredible breakthroughs in math in recent years without AI. But things are different now.
If the AI money had gone into that, I suspect we'd have better results.
Otherwise known as "eating your seed corn."
What's it achieved so far?
Compared to other new technology, like, say mRNA vaccines.
Useful AI is like 2 years old at this point. Chill out.
What have you done to cure cancer or prove the Riemann conjecture, anyway?
"Things are different now" you said.
They must have shown some direction in that area for you to believe that. After all, they've been claiming AGI for ages, are desperate for more investment, and the public don't want their data centres. They could do with some promising signs in something everyone wants.
I haven't done anything of note, but I can still spot mistakes in Fable 5.2 output, which suggests that it may not quite be the magic do-everything technology you have been sold on.
You aren't going to solve cancer with brute force, no matter how much you believe in Sam Altman.
After all, they've been claiming AGI for ages
Let's drill down on this point, since it's the one you chose to preface your argument with. Who, exactly, has been "claiming AGI," whether "for ages" or otherwise? Cite your sources: has Anthropic in particular claimed AGI?
Sam Altman. https://x.com/deredleritt3r/status/2092608013563560184
Explain exactly how Altman's claim that they are "on the cusp of AGI" satisfies your statement that an unspecified "they" have been "claiming AGI for ages."
The "they" is mostly OpenAI. Sam Altman, and several credulous boosters quoting him, have been talking about it being close since GPT4.
You said "or otherwise", and Sam's claim in the post I linked to is that it will be here some time in the next 54 days. You looking forward to that?
I'll happily retract that point you cherry picked if you want, as admittedly it looks like "AGI is just round the corner" has been the claim, rather than it is already here.
You can now show your evidence that the AI companies are actively working on positive things for society like curing cancer, which you are so sure is happening soon that you think funding any other technology in that area is a waste of money. They must be getting pretty far with that, huh?
Useful AI is like 2 years old at this point. Chill out.
What have you done to cure cancer or prove the Riemann conjecture, anyway? Oh, wait, you answered that one already. Guess we're done here.
And will also be drinking coffee, and using a computer, text editor, browser, search engine, etc., surely we don't start attributing accomplishments to the tools used?
....yes? Some of those have absolutely been attributed to accomplishments in the past and the present. It's fairly trivial to find examples of acknowledgements in research papers thanking donated equipment and computer use, among other things. If those things weren't important, they wouldn't have been mentioned.
Does that really sound like a good-faith argument to you?
Really?
Seriously though, does every model have to be affordable? Can't there be a model analogue of a rolex/ferrari?
I’d liken the expensive models to the early mainframes, or the first colour TV cameras. Sometimes it’s worth paying a fortune for the best thing available even if (compared with what’s to come) it’s not very good yet.
Impractical and expensive for show? Are you equating a high price tag with usefulness?
Yes I would like an impractical and expensive for show model
In case you're unaware, this is for the max setting. The (recommmended) high setting uses only 80k tokens.
Its still prohibitively expensive, and models 5% the cost are faster and get the job done just as well (for the average person's workflow).
I really don't get the value proposition of these massive models, seems like they're just loss leaders simply to say you're the in the top 3. There's not much use cases outside of niche academic research (and even then I question the need). They mostly function as marketing tools, like a casino that offers high roller accommodations.
People will say "cyber" but companies didnt want to spend on security before, not sure why they would all of a sudden be compelled to spend even more.
Not everyone is writing CRUD business apps and personal use software. Pretty much any model can do those now. The frontier is for doing things those models struggle with. Even in vibe coding a simple video game the difference in results between an open source model, the previous frontier model, and the current model is pretty huge.
Security is mandatory now in a way that it wasn't, now that AI has eliminated the skill barrier to hacking.
The llm induced Dunning Kruger is getting really old.
You dont need Fable to do most things. Your video game is still a "CRUD" app and you certainly wont make a great video game because you have access to fable. But marketing works well, people actually believe they can buy competency for themselves.
It hasnt eliminated the barrier to hacking. Please go find me a valid crit on any program on h1 with your fable 5.2 within the next 3 days, Ill bet my entire life savings it can't, and if you did it would cost 3x the bounty. You've been reading too much engagement farming articles/posts
Security has always been highly important, not any more than it is now. Companies are not going to suddenly start spending six figures on fable to protect themselves from imaginary AI powered hackers.
Its there for people who fall for cheap marketing who thnk they can shortcut meritocracy and make video games, or do security research with no skill. Good luck! Dont max out your CCs on llms.
I think the general notion that a lot of people are using models that are overpowered for their use case is right, but I think the big question is where the line actually is and how many people fit on either side. My suspicion based on my experience is that there are significantly more people on the "actually need the frontier model" side than the bear case suggests, but I'm not speaking with any authoritative data here and could be totally wrong.
I used to work on evals/observability at Opik: https://github.com/comet-ml/opik And I can say from working with lots of users running agents at scale, there is definitely a pretty commonly reached frontier of difficulty/complexity/abstraction where the capability difference between something like Fable/Astra and smaller models is a big deal. I find this to be especially true the more autonomous your workflow gets. You can still often tackle the hardest tasks with something like Opus, it just typically involves more direct intervention and design work from the user.
But again, this is just anecdata on my part. All of this is still pretty ambiguous and lots of companies are trying to solve the problem of matching models to task difficulty (I.e. routers), with varying degrees of success.
Fable 5.2 on low is both cheaper and more intelligent than Opus 5.5 on high. Prohibitively expensive is a gross oversimplification.
People looking at frontier models release are not the average person, and let's say that you pay $150 with Fable to save $1000*, it's much more interesting than using $10 with a cheap model to save even $500.
*Scale numbers to whatever suits you.
If you think llm usage provide that much value I got a bridge in Brooklyn to sell you.. Their main value is as a tool for wage theft (which im cool with, jusy recognize it), so lazy people can cheat at their jobs and get by doing less. Basically every RFH engineer admits this to me. Work two hours, company pays for 8 plus a tool that does mediocre work, that everyone else has access too and thus provides no edge.
Fable class models are for people who fall for cheap marketing and are lazy and think they can now bypass a meritocracy and buy themselves competency. Really they'll just continue to be lazy, and the people with more competency will run circles around them with 1/10th the amount or llm usage, or none at all.
Well for tasks that can be done with another model you just use another model.
^ and even at medium it is juuuust outside their "Most attractive quadrant"
Depends highly on the task. It does pretty well on my benchmark[1] in terms of cost (and success rate).
1 - https://bench.killswitch-lang.org/
effort isn't visible?
also, very cool idea
All efforts are set to the default for each coding agent to evaluate its ability to decide on the best effort.
Out of interest, do you run the same tests more than once to generate that?
What I often observe is that models being asked the exact same prompt can alternate between getting something right and getting it wrong, thanks to the wonders of a non-deterministic output.
Yes, two runs per task.
I think we will beat the cost with Hardware. Faster, more efficient hardware. From what I have seen we have only found ways to make it smarter and not more cost effective.
I'm curious to see the next generation of dedicated AI processors and how they perform.
There are models offering great value on current hardware, like GLM-5.3-Flash and DeepSeek-v4.1-1017.
Anthropic and OpenAI are expensive bleeding edge, but few use cases really need it.
when you're on a $100/mo plan, this is absolutely true. when you're paying API rates to deliver a product to people at scale and the difference between $1 and $0.03 per response is the difference between a feature being viable or not, you realize there's lots of things that haiku is fine for.
to me , glm flash and deep seek is that chinese 3 wheel tuk-tuk or e-bike runs on electricity. It make perfect sense to run to supermarket, and it's a backbone of Chinese economy, but no way in my life for a chance to save $100/mo you will risk your business with bugs, and bad/worse architecture decisions to save that $100.
again, they can be great for narrow use cases, and save you a fortune.
PRC models are more like ubiquitous class8 trucks that all economies can use as backbone. SOTA AI is like sending a courier on a first class ticket, only good for niche high ROI scenarios. No one is haggling over $100 per month, but $1000s per head per month on enterprise SOTA vs PRC models. For subsidized $100 per month plans SOTA vendors goes broke anyways... at premium SOTA API most users go broke. Pareto frontier PRC models are already being routed to take over all the general use cases, seems like essentially everyone outside of niche coding is scaling back SOTA AI use because it's not worth the $$$. It didn't take long at all for people to figure out you don't need next day airfreight everywhere.
At this point, they are beginning to approach the Tesla Roadster case. The flying one.-
It's interesting to look at effort-levels lower than max, though:
* xhigh: 62 score at $14.83 per task for Fable 5.2, vs 50 score at $10.85 per task for 5.1
* high: 62 score at $9.08 per task for 5.2 vs 47 at $7.54 per task for 5.1
* medium: 56 at $3.80 per task for 5.2 vs 43 for $5.56 for 5.1
* low: 54 at $3.10 per task for 5.2 vs 39 at $4.37 per task for 5.1
At medium and low the scores are much higher for Fable 5.2, while actually costing less per task than 5.1
The frontier chart there makes gemini carbon look really weird. Low is worse and more expensive than medium, and high is worse and more expensive than medium... funky curve like that implies a messed up model or weird benchmark.
If the low version of the model keeps putting out bad output, it can go through more iterations at the harness level before getting to a solution.
It beats opus 5.5's max effort with less cost, which is a step forward if you don't want to crank it to Ultra Max Settings.
The amount of compute it cost to get the best previous performance from anthropic went down. Nobody is putting a gun to your head and forcing you to go further down the paretto frontier if the current level of intelligence/task is acceptable to you.
https://artificialanalysis.ai/models/claude-fable-5-2-high
Cheaper and faster than Opus(max) AND scored higher. But yeah let's doompost about it...
Be interesting to see if Lovable jump to this model.
They've been using Fable as a selling point, but upping to this model will presumably eat further into their profits (should any exist).
From the summary at the bottom
"but particularly expensive when comparing to other models of similar price."
I'm none the wiser
Throw more spaghetti at the wall and more things will stick
So it's basically the same model just with gargantuan reasoning enabled?
Booooooooooo!
"particularly expensive when comparing to other models of similar price"
Well they do have an IPO coming up.
Most people never run out of plan usage, so the cost per task matters relatively little.
1. a lot of people do 2. most of their revenue comes from api pricing
Those are both true, but I don't see how they are meant to rebut what I said.
Most people on plan use do NOT use up all their plan usage.
Users on API plans can still use Opus 5.5 which is a great model. If they want top of the line performance, they can pay more.
When another company puts out a model that undercuts Fable 5.2 on price at similar performance, Anthropic will put out Opus 6 and undercut them right back. That's how the wheel turns. This is just part of the cycle.
It rebuts what you said because your statement that "the cost per task matters relatively little" is not true in the slightest.
This might be true for a small portion of the user base, but it is not true as a statement of fact.
Plan usage is available for companies with 2–150 seats. Above that, it's usage-based billing.
Does that mean bigger companies can only get API pricing?
That's my understanding, yes. But not sure what kind of deals the larger clients can negotiate.
https://support.claude.com/en/articles/9266767-what-is-the-t...
> Team plans support up to 150 seats. If your organization is on the Team plan and needs more than 150 seats, you can upgrade to the Enterprise plan.
https://claude.com/pricing
> Enterprise: Seat price + usage at API rates
> US$20/seat/month, billed annually, 20 seat minimum. Usage cost scales with model and task.
There is no premium seat option listed for the enterprise plan
Larger companies are being forced onto usage billing without subscriptions, so it's for sure relevant for a lot of developers.
Most is doing the heavy lifting here. I am a power user. And I max 3 claude max subscriptions on opus 5.5 high and a single codex pro 5x for personal use.
What do you have them do?
Lots of coding. Basically building dashboards, answering questions, feature development. As such also a ton of testing and reviews.
Loops a more recent addition to my workflows, also where most of my spend goes. I basically set them a goal and let them run towards that for days on end. Optimize the performance of x. Security reviews and tests against our applications. Handle alert triage and propose fixes for production.
There is also quite a bit of tools development as a lot of these parallel workflows strain a single pc (rust is expensive compute wise to build), so i build tools on top of sandboxes, to allow it to use secrets in a safe way, deployment, staging and so forth.
Cool! It looks like Fable 5.2 (especially low and medium) also sit on the pareto frontier between time per task and intelligence: https://artificialanalysis.ai/models?speed=output-speed-over...
I am always interested in this comparison since it shows how quickly you can go back and forth with a model, capturing both tokens/task and tokens/second. The new models seem to be meaningfully better here.
Note that the graph is misleading for open models since it's only measuring their official providers and that's rarely the fastest.
For example, the official provider for Kimi K3.1 Pro is dead last for speed on OpenRouter. You could run it more than 5x faster for the same cost elsewhere, which should push it comfortably into the pareto quadrant.
I'm really interested that Fable 5.2 Low seems to be a universal improvement in quality and time over Opus 5.5 Medium and High.
Certainly cost is a factor, but if your goal is quality and low latency at any token cost, it's now a fascinating option.
Why does AA choose ~$3.1 per task to be the breaking point for "most attractive quadrant" highlight? Surely the most attractive section is just being close to the entire frontier?
It's a quarter of the graph (hence quadrant)
The choice of X-axis maximum and Y-axis maximum dictates the cutoff point of the quadrant. With a maximum of $80 per task, the quadrant is $3.1 and lower. If some new model gets released and costs more than $80 per task, the "most attractive" cutoff moves up from $3.1 since they have to move the X-axis max. It's completely arbitrary.
It's not saying that being inside that quadrant is a hard cut-off it's just illustrating the direction of the pareto frontier
If you take the average point inside the 4 quadrants, the top left one is pareto optimal (of the 4)
It is suggesting a hard cutoff since it says "most" attractive, a superlative, instead of "more" attractive. And it is not pareto optimal, because if the models get really good and are scoring 10x more for 10x cheaper that green region will no longer be pareto optimal.
Of the 4 quadrants it is the most attractive
I suppose you’re not wrong lol
> it shows how quickly you can go back and forth with a model
Another member of the pro-REPL / anti-agentic tribe. There are dozens of us!
It's crazy how expensive Anthropic's model is compared to pretty capable DeepSeek-Flash
While they made, hands down, the best models, their 100x price for few percent better is completely made up, and I think it will end up bad for them
This has echoes of the same argument that why hire anybody in the US when you can hire someone almost as good elsewhere for a small fraction of the cost.
In winner-takes-all markets, it becomes worth it. The utility of the marginal improvement doesn't actually diminish, in fact, it increases like a hockey stick. If you could pay 3x more in compensation to hire a 20% better employee, it's almost always worth it in knowledge work industries.
> This has echoes of the same argument that why hire anybody in the US when you can hire someone almost as good elsewhere for a small fraction of the cost.
The reason in workplaces people don't hire somebody else at a fraction of the cost isn't because American workers are 15% smarter than non-Americans. It was just very difficult to have exactly 1 remote worker on a team, language barrier, international laws, etc.
In my experience people generally aren't willing to pay double the salary for 15% better work. They will however pay market for domains where nobody else is qualified.
>people generally aren't willing to pay double the salary for 15% better work
Actually, companies are already willing to pay double for 15% better work - the proof is in what SWE salaries in America have currently become.
Companies are pretty much paying at the frontier of marginal improvement (not utility, which again, is hockey-stick) - i.e. it's not like paying 800k will reliably hire someone 15% better than the 400k that they're currently paying.
At this point, the limiting factors are their discovery and interviewing processes. The ability of the brand name to attract talent also matters a lot (OpenAI/Anthropic can actually get better talent by paying 800k, as they do; Microsoft cannot because they're still Microsoft and a good chunk of top talent will simply never respond to a Microsoft recruiter).
Not to forget, this is all for a job that isn't more difficult than high school teacher.
Companies indeed hired someone (Asian) almost as good elsewhere for a fraction of cost.
Then explain why big corps like MS or Google are outsourcing to development centers in a much cheaper country (India, China).
The explanation is that it's a mistake.
American software has flip-flopped between rushing to outsource and realizing that outsourcing is a bad idea. LLMs (plus US work visa policy headwinds) are simply what triggered the current flip cycle.
Is the winner-takes-all market in the room right now? everybody switching models and harnesses non stop.
> I think it will end up bad for them
There will always be a market for the most intelligent model. Keep in mind as well that something like Fable doesn't have to be cost effective on its own. Fable is Anthropic's way of building datasets for smaller more dense models, which is viable on its own as a method of success. These large labs are constrained by their dataset more than anything else, and these expensive-but-intelligent models create some of the best data out there.
Using DS-Flash is like using local models that use all your hardware.
It's fine for slow tasks but if you need something done quickly, I find that DS just spents at lot of time thinking.
After using Claude for the first time for about 2 weeks, I've been impressed by the quality and speed of Sonnet 5.5 to do code changes (~20 seconds). Meanwhile DS would have taken 2-5 minutes.
You can get ds flash providers that do 1000+ toks/sec - it’s unreal fast.
Distilled, weaker versions though I think?
I agree after using DeepSeek for many tens of billions of tokens for this point. Yeah Fable/Astra has more "magic", but not running out of token budget 5 days into the month is the right place to be. Plus, learning to deal with dumber models, only keeps you sharper as the programmer/agent wranger.
I don't marginal intelligence has diminishing value -- quite the opposite actually. Every additional "IQ point" or 9-of-reliability unlocks some new economically valuable task. But in a few years we'll know whether or not that's correct!
Opus 5.5 still exists
As per OpenRouter Opus 5.5 costs $4 / M tokens and DeepSeek Flash is $0.006 / M tokens, which makes it x666 times cheaper (I urge you to re-check my math!)
https://openrouter.ai/compare/anthropic/claude-opus-5.5/~dee...
To be honest, I use both daily and feel no difference in this case
> few percent better
A few percent better on tests like this translates to extreme real-world differences, and that's not even taking into account how benchmaxxed DeepSeek models tend to be.
You will get more done, at higher quality and quicker with an Anthropic sub compared to using DeepSeek. Unless you’re forced to use API billing the open weight models make no sense right now.
A lot of companies ARE forced to use API billing though, as Anthropic only allows up to 150 users to use a sub. And companies are Anthropics biggest revenue stream - criticizing the cost per task of their models is a valid criticism.
This feels like saying you'll be more successful if you drive a Bently to work.
That seems like an insane increase in cost for a modest intelligence increase.
But perhaps the performance on real-world tasks will be more noticeable than on benchmarks.
Consider why Hollywood movies use $100k lenses, rather than $500 lenses. Are the lenses really 200x better? Probably not, but there are cases where that extra 5% better is worth an extra $99.5k. There will be things Fable 5.2 can do reliably that Opus 5.5 can't; and there will be people willing to pay for that.
I think those öensrs are mostly rented. For a $100M film few thousand a day is nothing. Those lenses also come with a camera. If you want to use same camera and lens in a desert in Tunis and snow storm in Norway and get same quality of image without constantly changing broken equipment, you are going to either pay thousands for a good equipment or millions for delayed timetables for broken equipment. In that sense, it is not only 200 times better, it is your only option. So yes, sometimes the price is worth it.
Furthermore, you make one lense and rent it out, apparently some Panavision lenses made in 60’s are still in use. It doesn’t churn constantly electricity and GPU-chips sucking in more and more resources like these things do.and at the same time, it is quite hard to find the exact cases where the markup is justified, unlike with those lenses. Maybe in cybersecurity etc. but even there some Chinese model can be almost equally good or even better.
Those lenses save money in budgets and make money in getting people into theatre or streaming services to watch films creating profits way beyond their cost. I think in terms of economics that is productivity increase justifying the cost. We are still waiting for that productivity increase come from the AI industry.
I'm not convinced of your logic.
Modern studio pictures that get nominated for best picture or best cinematography cost around $80 million on average. If you look at highest grossing films the budgets are closer to $175 million on average – even taking into account the $20 million budget of Demon Slayer: Infinity Castle.
Isn't it more that when you're spending that kind of money that $100k for the "best" thing becomes kind of like "insurance" where you don't want to be the one who doesn't use "the best" when "the best" costs about 0.1% of the picture's budget?
Additionally a studio can buy the $100,000 lens once and use it on many films. But it's not like you can reuse Fable 5.2 in the same way.
The point is that the extreme high end of performance has asymptotic cost which is not an undesirable tradeoff for extreme use cases.
Almost no variant of the same product that costs 100x than another is actually 100x better than the other variant. Whether the increase in cost is worth the increase in value depends on the application/use case. Most often the higher priced tier is not worth it for most use users, but most often there are still some use cases where it's desired. This applies to almost anything you can buy: food, wine, clocks, cars, planes, bikes, ski, clothes, furniture, phones, appartments, insurance, service, anything. Surely also LLMs.
the concept of diminishing returns exists in every thing. headphones, coffee, cars etc. anything that can be pretty continuous in terms of quality will have this
If I were to guess, it's because the CEO of the $100k lens business is in the Epstein circle.
It's simply useful and within budget.
You too can sell stuff for $100k. Just make something useful to someone who can throw around $100k.
You think a $100K lens is crazy? Wait until you hear what people are paying for enterprise software. Big business will spend 10x that on software... and then never implement it.. The lens is a deal.
They have it at maxed out reasoning. Go into the dropdown and select high or medium, which are what people actually use. It's a decent bump while both being faster and costing less (per task).
The worst thing about AA is defaulting to the highest reasoning value for everything.
Artificial Analysis is paid by AI labs and they don't publish the full method. To me it's meaningless, let's wait for other benchmarks, but if they're correct the intelligence increment is negligible.
They publish their methodology here:
https://artificialanalysis.ai/methodology
If they published their private benchmarks the Chinese labs would simply train on the data to make it look like they are close to the frontier and anyone who uses them regularly knows they are far, far behind and always have been.
It's not perfect. But it's the best we have. You are welcome to provide a better source.
I feel like there are a few dishonest points made in your comment.
First, the way you phrased your argument makes it sound like it benchmaxxing is mostl a problem from the chinese models. As recent articles seem to have shown, it's smaller models, closed and open weights, which are benchmaxxed. That has nothing to do with the country they're coming from.
Second, I feel like your argument about chinese models being far behind is not quite relevant to the argument and was mostly added as a jab against these models.
Third, arguing against publishing the benchmarks is not the same as arguing FOR using private benchmarks. You don't have to use benchmarks at all, especially when models are so hard to quantify anyway. Simply shrugging and saying "well, no way around that" is not a solution. In addition, there are other, better ways to prevent benchmaxxing than simply "not publishing your solution". The recent maxxbench benchmark has shown one example.
Fourth, you don't need to provide a better alternative to criticize something. Criticism is valid even if you don't have a better alternative yourself
And lastly, I actually feel like there IS a better alternative than AA - not using aggregate score at all. If you need a model for coding, look at the individual benchmarks themselves for a / very / rough overview of how a models is doing. Trying to boil down so many different areas and capabilities into a single score, which is then weighted by little more than subjective vibes from the people behind AA is bound to be too little information and too much information loss. This, I personally feel, seems to mostly exist as a meaningless leaderboard for marketing purposes.
You can clearly see that those models are indeed benchmaxed https://maxxbench.unlap.ai/research/why-maxxbench/
How can their benchmark even be truly private unless they run the models themselves in their own hardware? ZDR, yeah, sure.
If they had published it then it could be gamed?
https://artificialanalysis.ai/#intelligence-efficiency-tabs this is the chart that matters for most of poor developers and companies. Pareto for a given intelligence index and looks like only openai is across the quadrant with muse at the bottom. They have really cooked efficient models!
It depends on the task though. If your cheap task can't solve it
8 points higher than Astra for 4.2x the cost and 5x the output tokens. Is it worth it?
Edit: high seems better than max
It is about time to start setting taxes for ai usage in commercial settings and set how to transition to a society with less available work. They talk so much we need universal benefits, about time for them to start paying for it
No need. According to HN Anthropic will be bankrupt by end of week. Why tax them more?
In case the sarcasm isn't total, the concept is of taxing users of AI in a way that balances human work reduction.
To me Fable's role was always as a planner or plan reviewer, rather than something individuals or organizations would use as the primary driver. In that case, I think these improvements seem worthwhile in exchange for the cost? Ideally in that role, you want the best performing model.
Not sure if the current rise means it's cost prohibitive to use in that form too.
Considering tokenmaxxing is over and budgets are getting tightened everywhere, what's the use case for fable? Who will use it? Possibly FAANG? Or firms like Apple who have 200 $ per day budget
I’ll use it if there’s a complex bug I need solving quickly, or to weigh in on architectural decisions across large multi-plan projects. It’s a tool to reach for but makes no sense as a daily driver.
I am wondering which use cases would require usage from Fable 5.2 especially when Opus 5.5 already performs almost all of the tasks in a very optimized way.
Ask opus to write a occlusion mapper shader and create a 3d environment where ground texture is used to mimic infite detail. Same on some random brick walls. Corners need to be extruded realistically. This is a hard task, because only a few games have this (crimson Desert is one of them)
You can use Opus to do that, just not one-shot it
Lots and lots, if 5.5 got nerfed.
I used it last night for some 3D modeling and it feels like a big step up from 5.1. 5.1 was able to make models that met my functional requirements but often they were ugly. With 5.2 the models feel simpler and more efficient in a way that makes them prettier.
Anecdotal, but I have the model write a podcast script of two reviewers talking about a book, one chapter at a time.
Fable (5.1 and 5.2) consistently write a much shorter script (2.5 pages) compared to Opus 5.5 (4.5 pages).
Fable 5.2: https://claude.ai/share/274a96a8-c8a5-4564-9ca1-e914edffc722
Opus 5.5: https://claude.ai/share/da8ab6bd-0987-47dc-9c26-68ebbfb25781
The reasoning setting is so important. I find that at different levels the solutions differ a lot.
Out of curiosity do you know if results would be better, the same, or worse if this was done from a harness?
Crazy how quick these are coming out now
I'm sure that they get early access to run the benchmarks, it wouldn't have time to finish running if they were running it at public release
I P O
Someone on a thread was complaining they haven’t really needed the newer models because Opus 4.6 was good enough.
I had to remind them 4.6 only came out February this year. In a normal world 4.6 is a newer model! A model 6 months old is seen as ancient!
Reminds me a lot of the race for clock speed when I was a kid. We just took it for granted that it was going to keep increasing, but CPUs sat at roughly 1-2GHz for a long time, or at least that's how it seemed. The slowdown and pause was fairly noticeable. The current release cadence is unsustainable. It might be that we're getting close to the AI equivalent of 1-2GHz CPUs, and it takes years of incremental progress to push further.
When AMD won the race to 1 GHz, the GHz race slowed and Intel and AMD began focusing more on multi-core. Some chips continued to push the boundaries up to several GHz, some quite higher with specialty cooling. But multi-core was the far better strategy.
Clock speed went up single core till at least 3GHz before multicore became a thing as far as i remember.
This is why I shied away from management roles
In the JS ecosystem 6 months always meant breaking changes, without incremental updates everything would be basically like a full port
Pretending like I did “20% hands on coding” just doesn’t cut it, even the reality is having no idea what’s going on anymore
The crypto space moved that fast as well
And AI keeps the same cadence
It’s more important for me to be able to build
Idk how could Fable 5.2 win the benchmarks. In my tonight trials it drifted quite a lot. I was analyzing the feature's design with it, and it lied to me several times, ending up with "You're right." Opus 5.5 was better.
Later, I ran a "health check" session—asking "what went wrong". I read the session, and its suggestion turned out to be something that was already part of the design.
It’s like the attention mechanism is too leaky or something.
Long execution time makes this not very impressive. It's easy to benchmaxx if you give the model an insane compute budget and room to just brute force the problem. Efficiency is what actually matters.
Fable 5.2 at high is the real thing here - scoring 62, second highest ever, on AA intelligence index, while being 3x less expensive.
I'm finding it completely stupid compared to Opus 5.5. It doesn't seem to be following required rules in my setup which is for non-coding purposes. It basically made up results when I have a whole system with papers that are knowledge rag searchable and it is required to search there, but just decided not to.
Gpt models are better behaved for that stuff.
Thanks, I may end up giving them a try. I actually use Codex extensively for the custom plugin that tells Claude what needs to happen with hooks for reminders, some commands require a search before they do a grep (which Claude seems to love to start with), etc.
Still 100x more expensive than deepseek
Do we have Fable 5.2 vs Mythos 5.2 benchmarks?
I remember Mythos 5.1 being a tad better even for not cybersecurity tasks.
how do you get graphics from the model?
Fable, for me (it might be a completely different experience for you) has been like the well read snobbish kid at school that openly ignores me when I ask certain questions and often asks me to talk to someone else on other questions.
That kid might indeed be a genius, clearly bright, but is more interested in deciding whether I should have asked a question than in answering it.
For me personally, if the best model on the planet refuses to answer why I shouldn't mix bleach and ammonia, where I'm trying to understand the chemistry behind the warning, it's not pareto frontier to one that atleast tries to and points me in a general direction. Being told that the topic is concerning, referred to someone else and charged part of my quota for that experience feels like getting an invoice from that school genius for telling me to "just ask the chemistry teacher".
If my LLM asks me to rethink my life when I ask it to reverse engineer the firmware of my Samsung TV that I suspect is setting up adhoc radio networks to relay my data to Samsung's cloud, I feel rather sad, specially when that kid sends me an invoice for saying "no"
At least Anthropic should not bill me for the refusal (as of my last tests, refusals still comes out of my quota) - getting slapped and pickpocketed for asking a question is really upsetting.
fable is a high agency model (some people call it 'big model smell'). if you want something that blindly follows instructions, use sonnet. I want to be challenged by the thing since I expect it to be smarter than me at everything except the little part of human knowledge I'm an actual expert in.
Do you also want it to refuse to continue the conversation because it's suspicious of your motives?
not really, no - but I almost exclusively use the biggest models as advisor and reviewer subagents, so if they refuse to their smaller siblings I don't get to see it
No, but I certainly want it to do that for random internet strangers more than I don't want it to do that for me.
>At least Anthropic should not bill me for the refusal (as of my last tests, refusals still comes out of my quota)
There's a setting to not degrade to a lower model on refusals and instead pause the chat.
I'm unsure if you're implying that setting guarantees no cost. Here are my notes:
In my deep dives into refusals, only those categorized as cyber, "general_harms", or with "no category" are not billed.
Everything else is billed - I have tests that got tagged as bio, "frontier_llm", or "reasoning_extraction" and they were and are certainly billed.
Input tokens are billed at the model's rates. Also, unbilled API refusal still counts against API rate limits, so it affects the 5h and weekly windows.
If any of these are incorrect, do let me know as this would make my Max plan immediately more valuable. I did not upgrade to Ultra in part for these reasons - I mainly need the sub for affordable Fable access - purchasing Fable over API isn't reasonable for me.
Doesn't make it any better. If I refused to do a job, then I would be fired.
Not if you were in a decent union and it was in your contract.
> Design, synthesize, produce, acquire, operate, test, deploy, or plan the deployment of biological, chemical, radiological, or nuclear weapons, explosive or improvised explosive devices, or their precursors or delivery mechanisms, including developing weapons-critical technology (such as fissile material production, enrichment, or device design)
https://www.anthropic.com/legal/aup
It doesn't deal with creating mustard gas questions reframed as curiosity queries, and that might impact legit curiosity queries like OPs, but isn't surprising. They've also disclosed ahead of time how conservative the guardrails are.
good thing these things aren't employed then right?
Similar experience with Opus. Asked it to tighten access controls and isolation in my homelab. Designed something with a whole bunch of words that looked plausible, so I told it to go ahead. Halfway through it gets stuck because it locked itself out of something, and asks me to run some commands as root.
Me: No, I'm away from the computer right now, you figure it out. Use the root login you already have if you can't figure it out.
Opus: This can wait until you get home, then.
Me: No, figure it out now or use root.
Got [cyber]'d, some scary message that referred me to legal/AUP, and refused to continue.
Switched to OpenAI in part due to this silliness. Sol seems a little less smart but doesn't get on a high horse to lecture you about this stuff. It either does the work or fails to do the work, and leaves the moralizing out of it. It was able to take over from where Claude left off and finish the project.
Are you really paying anthropic $200/mo to find out why you can't mix bleach and ammonia?
Google could have been #1 for a little while, but they botched their release and squandered their temporary lead. (As usual.) If I worked there, I'd be so pissed.
So is Google still in the race? Does anyone actually uses Aragon or Carbon or whatever model they released for coding?
Yes, I have found Argon is good at explaining things, which is nice in root cause analysis and debugging / troubleshooting
For an open source project (CrowdGPT) I hit upon this workflow in a recent PR:
I use Argon regularly.
Pretty big Gemini fan - but IMO Argon is universally worse for coding than Astra or Opus 5.5 (un-nerfed).
I strongly prefer Gemini's output compared to anything else. So I typically use Argon to either direct other agents or explain something without psychobabble.
Argon/Carbon have by far the lowest hallucination rates, so IMO it serves as the best bullshit shield from other models.
In terms of implementation (and especially design), it's far behind Astra and Opus, and I'm sure Fable will make that gap even larger.
Argon/Carbon may give you not-the-best advice, but they are far less likely to just flat-out-and-completely lie to you.
Gemini Flash was my daily driver until 1017 dropped. Not that 1017 is better, they both are dumb in their own ways, it’s just available at far higher tok/sec.
Gemini models also historically are the best writers. 3.1 was the first model I saw that made actually funny jokes regularly.
Can you actually use it through some subscription, or only through the API?
Yes? I regularly use Argon in agy on my Google One subscription.
It's in Cursor
What Google lacks is a viral marketing team spamming the internet with vibe coded apps and videos.
I do. I prefer it to Opus 5.5 - just as good at coding for my needs, but so much more pleasant to work with.
I am not sure how people keep saying google are not in the race, or have lost it or whatever when just a few hours ago they had the #1 model? What more do people want?
I actually just think it's a meme to talk down on Gemini at any opportunity here. There's probably a fair number of people that wrote off antigravity at its (pretty rocky) launch. And Google also doesn't provide any kind of good sandbox-y experience to Just Try their models, which probably bites them here too.
But Gemini models are still neck and neck in the lead both in my experience and in benchmarks, and Gemini is far above the competitors (although Opus 5.5 closed much of the gap) when it comes to any kind of complex web dev. (I generally use Astra for anything 3D, Gemini for anything web/mobile, and Opus for everything else.)
Their latest internal reshuffling and the strange amount of parallel releases with the same target, their kafkaesque barriers to entry, and a lack of clear vision is what makes people think they’re not in the race anymore. Obviously they still are. They develop new models and also are deep in the compute game too.
In my experience Google models have always been mediocre at best and dangerous to use at worst but I must admit that I didn’t try the latest Aragon/carbon crop.
I learned not to put too much trust into benchmarks. The best benchmark is to use the model on a codebase you know well.
On both Gemini 4 releases, it said on the release page that I cannot use it. No timeline as given and I have not seen an announcement otherwise since.
So, what more do I want? It'd be a good start to actually tell me it's available and giving me a clear path to using it via plan & harness?
> What more do people want?
well, releasing it would be nice
Gemini 4 Argon is available in both anti-gravity and the Gemini app for me on my Gmail account and I didn't do anything special (I have a modest drive storage subscription - not the largest nor smallest... Not sure if it is bundled with that or not).
So certainly released to the unwashed masses like me to use for free without any special extra work or API keys.
I'll have to take your word for it, considering the latest model I'm able to use of theirs is 3.8 Flash for some reason.
I tried using it as part of my Pro/One/whatever subscription (the one that comes with Youtube Lite). For code review, it was able to find some things that Sol 6.1 couldn't. I found it quite useful. But for code generation, it's pathetically slow and the code isn't impressive. It's at least one year behind.
Argon is much superior to Opus for graphic design and creative writing tasks.
It's not as smart for coding as Opus, but it's above the best open weights.
IME, YMMV, etc.
Optics and incentives. Google will turn on the R&D the day after their Anthropic investment is realised, maybe $80-150bn on a 1.2tn valuation, plus >100bn in spending commitments.
So, the highest score so far but also the highest "cost per task" so far, and pretty slow... which, I guess, makes it the Cybertruck of language models. Impressive in its own way, but impractical for most people (me included). Sticking to Opus for now.
The Artificial Analysis charts are great, but be warned that they show max effort by default.
Max effort is nice for revealing the limits of a model, but it's almost never used in practice.
If you scroll down to the coding charts you can start adding Fable 5.2, Opus 5.5, and others at more normal difficulty settings.
Fable 5.2 on medium looks quite attractive compared to Opus 5.5 high/xhigh
Fable 5.2 on max (the default) looks like a good way to light your money on fire.
Has anyone tried to actually use this? Six turns, about 10 minutes total thinking, and its going in circles spitting out the sort of shallow logic I would expect from basic Deepseek. This is genuinely awful. Im planning to turn in my free reset and go back to 5.5, except that also started fucking up basic instructions a few days ago. Everyone keeps going on about these BS rankings that every single release of every model claims to ace, when we all should know have almost nothing to do with real results. I feel like Im taking crazy pills here.
Maybe its different for vibe coding, I wouldnt know.
So... Astra 6.1 will come next week and be the same ultraexpensive one but from OpenAI, right?
Don't worry, it'll soon get dumbed down in a few weeks :)
Splendid. Now how many prompts in you're downgraded to Opus 4.8 and then the chat is ended altogether?
It does not seem profitable as those prices.
Why is it so much more expensive?
This page is showing Max effort, which is much more expensive.
Max should not be used unless you really need every bit of perf, it's deep into diminishing returns territory.
It seems to use way, way more tokens to complete the tasks
On max.
Change the models in the dropdown to medium / high and Fable 5.2 is about twice as cost efficient, and slightly faster.
It's insane how people nitpick on the highest thinking setting for every anthropic release and claim it's too expensive. Not trying to stan for anthropic, but what am I missing here: Just don't use those models on the highest thinking setting?
Fable 5.2 medium for example seems a really good choice when it comes to the performance vs task (if you have a complex task that requires such a model of course).
Twice as cost efficient compared to what?
It's prior version: Fable 5.1. Which, I assumed, is what people would be comparing it against.
This has got to be a minor reason why adoption of AI isn’t faster.
Businesses like predictable costs and predictable performance.
If your new model is slightly better but crazy expensive, but doesn’t have to be with certain settings, but my user can accidentally run it that way by mistake or maybe your harness makes it the default by accident and the user doesn’t notice…
My beloved predictability is destroyed and trying to get users to understand this is useless.
Imagine a streaming subscription that worked like this: “We’ve updated our catalog with more movies. If you want the premium ones though you only get 2 per week and then you have to wait. But if you do mostly classics and a few from the 2010s you can squeeze in a recent theater release. If you time it right and do only one every day you can see 30 modern classics this month, but you can’t watch three today. Also your bill this month is $40 instead of $20 due to the library update”.
I'm not sure I understand your complaint. The default reasoning effort for Fable on their harness is medium, not max. It also uses the latest version of Fable by default.
So a standard user will see a ~10% performance bump, better reasoning, and an ~50% price drop. And they can just... use the old model if they want?
How is that bad?
Yep, and now multiply that complexity by hundreds or thousands of employees. That's why companies are giving in to crude, hacky solutions like monthly billing caps. What else are you really gonna do?
Hire product managers and salespeople to build and sell a practical, useful, profitable, scalable, product.
Nag, fuggit. Let's just burn all the capital in the world and ride it into the flames.
Good luck with that price tho. A proper harness with ready-to-access knowledge is what most users will need anyway.
There's only one problem: what good is it if you can never use it? What I mean by that is, using it reliably is not possible on a, day-in-and-day-out basis to build on Anthropic and then expect it to be stable over time.
I feel like the reliability issues have been fixed for at least a few months now
It's funny how the cost graph is completely different to what anthropic is trying to sell on their release post.
astra high is on the Pareto front and just in the green zone
Artificial Intelligence is bullshit. There, I said it.
Imo it is good at search and aggregate similar data. Intelligence it is not.
Intelligence - difficult ontological ground in general.