I'm sort of surprised it took Jev for everyone to realize "oh yeah, this is a useful thing that we should have in our product portfolio".
But maybe this is one of those things that's only obvious in retrospect? I'm sure people were trying something similar years ago with grammar-constrained outputs (for example to only output a simple json structure), but you can't get probabilities out of the model that way. Which makes me wonder: how do these models calculate probabilities?
A lot of AI stuff is like that, actually. GPT 2 and 3 were mainly curiosities that would get articles on tech news sites but didn’t attract much attention from the general public. But after RLHF the UX improved dramatically and ChatGPT became mainstream. A lot of MLEs underestimate how much UX matters, and the same is true for Jev.
> how do these models calculate probabilities?
The short answer is: more and better data.
The long answer is: probably the same way any other classifier does. Take your ground truth probabilities and your predicted probabilities. The gap between those is called cross entropy, which is used to push the model towards the correct probabilities.
> which is used to push the model towards the correct probabilities.
The definition of "correct probability" here is usually "p=1 for the ground truth label, p=0 for all others." Which seems like an intuitive approach but if you actually think about it doesn't trivially imply the model would learn sensible distributions under uncertainty.
For a single prediction, sure. But the model doesn't make a single prediction and then stop existing. It makes an unbounded number of future predictions and under that assumption it is reasonable to expect the model to behave optimally under Bayes rule - which has a unique global minimizer at E[(Y-F(X))^2].
So counter to your claim that "It doesn't trivially imply that the model would learn sensible distributions under uncertainty" I think you need to consider what it means to optimally update a posterior under the global minimizer of the true conditional distribution.
OpenAI was already working on a Decisions API, and Jev launched theirs quickly enough to have 1 week in the spotlight before OpenAI put theirs in beta (which works great btw).
When trying it out, OpenAI's Decisions was ~2x slower, worse accuracy and more expensive. Unless used with image-related tasks, Jev is clearly superior.
You can do it with LLMs by looking at the logprobs of ('yes', 'Yes', 'YES'...) vs. ('no'...). This works for bool or whatever they call it in jev space nool I think?
You can do it manually or use ready made libs. I know there's one in python on top of vllm that can do something like this.
LLM token probabilities are really poorly calibrated for decision making.
Every time I see someone saying something like "isn't this just structured output?", it's because they don't understand that probability calibration is one of the most important parts of the product, especially to understand uncertainty.
I think Jev did a pretty good job on calibration. I wonder how the fast-follow competition from OpenAI and Gemini do.
Not reliably - look at microsoft's guidance project that did some of this.
Once a model goes through instruction fine tuning and RLVR/RLHF the log probs decorrelate from what's ground truth (given the propensity of the data seen during training)
But I assume it wouldn't take much more training to make them good again - just a matter of picking the right loss function, since this sort of stuff is probably all encoded in the last layer or two anyway.
I think its more like how LLMs caught everyone by surprise. A reasonable person would not think such a simple system would have been this effective, and yet here we are. Distilling that system down even further into a decision model again does not seem like it should work. But it seems to.
The whole system is foreign to us, and makes us rethink what "intelligence" actually is.
We were doing zero and few-shot classification with LLMs since before they could reliably complete whole sentences. It’s an easier task than generating coherent text. The model needed to be tuned for classification (not generation) and for the domain, but it would often succeed at new classification tasks with a simple description.
I guess given the huge advances in LLM since then, these models probably achieve much better accuracy on a wide range of domains with lower effort compared to what we were doing, which makes it compelling as a product. And yeah, the models probably were already good enough a few years ago too. Other factors include how easy it is to release products at high speed now, and the desire in the industry to reduce token spend.
I'm using them to label training data for model development. You can do something like label a bunch of records with Jev, then train the model, then check results with opus and a hold-out set.
The best decision model I have used (besides Jev itself) is respan/span-02, which is based on (finetuned from?) Gemma4 26A4. That makes me optimistic for this release, despite the high cost.
Image input is a primary reason I've never moved off my "home rolled" version of Jev (that Claude wrote for me over a year ago) for my file organizer/indexer. Takes images resizes them to 1k, then asks a handful of questions about the image for tagging purposes. It is able to process around 3 images a second based on Qwen 3 VL, but distilled and optimized for 1k images and up to 2048 token inputs (after the system prompt, then the questions are asked one after each other reusing the cache). Documents that are longer than that end up going through a summarizer that runs on my other GPU (Qwen 3.6, I think). And all of that gets fed into a vector search engine, for easy lookup later.
I'm not even sure what all is going on inside it anymore, because I just add new features when they popup, and tell Claude to do it, and it let's me know when it is done, and every new "input feature" can run against the entire catalogue of files.
Google ensuring “share of wallet” from their existing customer base with a solid but not great “we do it too” product, priced to capitalise on risk and change aversion.
What got me into Claude Code started with Google Antigravity. Somewhere along the way Antigravity took a dive. The rollout was great, then it started feeling like other G-products, the dreaded tail of lackluster developer energy after a peak.
Startup / Founder focus vs Large Organizations with no vision.
With AI, we know distribution moat really doesn't matter and Google doesn't have a great Developer distribution moat anyway
Meanwhile, they are suffocating Jules today (https://killedbygoogle.com/) with a metaphorical pillow to the face and no migration path mapped out for casual users apart from "use antigravity" in a 1 line high banner; after about a year of existence.
It seems unlikely this will be around in 6 months at this rate.
That doesn't make any sense, this is a Jev competitor and a model release, like their Carbon and Argon models. It won't simply be 'killed' like apps might without something to succeed it.
And why not, exactly? OpenAI was more than fine killing Sora with no replacement. I don't see why Google would be any different, especially considering they're notorious for this behavior.
Gemini Decisions API is likely to be used by enterprise customers in real products. Sora was a toy that only seemed to exist to infringe on IP OpenAI didn’t control. Apples and oranges.
It took OpenAI 2 weeks after Jev to release a decision end point, and now google does it 2 months later. Why such a difference? Bureaucracy? Quality control?
I'm sort of surprised it took Jev for everyone to realize "oh yeah, this is a useful thing that we should have in our product portfolio".
But maybe this is one of those things that's only obvious in retrospect? I'm sure people were trying something similar years ago with grammar-constrained outputs (for example to only output a simple json structure), but you can't get probabilities out of the model that way. Which makes me wonder: how do these models calculate probabilities?
A lot of AI stuff is like that, actually. GPT 2 and 3 were mainly curiosities that would get articles on tech news sites but didn’t attract much attention from the general public. But after RLHF the UX improved dramatically and ChatGPT became mainstream. A lot of MLEs underestimate how much UX matters, and the same is true for Jev.
> how do these models calculate probabilities?
The short answer is: more and better data.
The long answer is: probably the same way any other classifier does. Take your ground truth probabilities and your predicted probabilities. The gap between those is called cross entropy, which is used to push the model towards the correct probabilities.
> which is used to push the model towards the correct probabilities.
The definition of "correct probability" here is usually "p=1 for the ground truth label, p=0 for all others." Which seems like an intuitive approach but if you actually think about it doesn't trivially imply the model would learn sensible distributions under uncertainty.
For a single prediction, sure. But the model doesn't make a single prediction and then stop existing. It makes an unbounded number of future predictions and under that assumption it is reasonable to expect the model to behave optimally under Bayes rule - which has a unique global minimizer at E[(Y-F(X))^2].
So counter to your claim that "It doesn't trivially imply that the model would learn sensible distributions under uncertainty" I think you need to consider what it means to optimally update a posterior under the global minimizer of the true conditional distribution.
OpenAI was already working on a Decisions API, and Jev launched theirs quickly enough to have 1 week in the spotlight before OpenAI put theirs in beta (which works great btw).
When trying it out, OpenAI's Decisions was ~2x slower, worse accuracy and more expensive. Unless used with image-related tasks, Jev is clearly superior.
You can do it with LLMs by looking at the logprobs of ('yes', 'Yes', 'YES'...) vs. ('no'...). This works for bool or whatever they call it in jev space nool I think?
You can do it manually or use ready made libs. I know there's one in python on top of vllm that can do something like this.
LLM token probabilities are really poorly calibrated for decision making.
Every time I see someone saying something like "isn't this just structured output?", it's because they don't understand that probability calibration is one of the most important parts of the product, especially to understand uncertainty.
I think Jev did a pretty good job on calibration. I wonder how the fast-follow competition from OpenAI and Gemini do.
Not reliably - look at microsoft's guidance project that did some of this.
Once a model goes through instruction fine tuning and RLVR/RLHF the log probs decorrelate from what's ground truth (given the propensity of the data seen during training)
But I assume it wouldn't take much more training to make them good again - just a matter of picking the right loss function, since this sort of stuff is probably all encoded in the last layer or two anyway.
I'm pretty sure these models also went through RL post-training, the question you ask still needs to be interpreted as an instruction.
I think its more like how LLMs caught everyone by surprise. A reasonable person would not think such a simple system would have been this effective, and yet here we are. Distilling that system down even further into a decision model again does not seem like it should work. But it seems to.
The whole system is foreign to us, and makes us rethink what "intelligence" actually is.
We were doing zero and few-shot classification with LLMs since before they could reliably complete whole sentences. It’s an easier task than generating coherent text. The model needed to be tuned for classification (not generation) and for the domain, but it would often succeed at new classification tasks with a simple description.
I guess given the huge advances in LLM since then, these models probably achieve much better accuracy on a wide range of domains with lower effort compared to what we were doing, which makes it compelling as a product. And yeah, the models probably were already good enough a few years ago too. Other factors include how easy it is to release products at high speed now, and the desire in the industry to reduce token spend.
For anyone wanting something local, private, faster, which can also run and train on CPU, checkout https://http://jeffyclassify.com/
It comes with a bunch of pre trained classifiers, and you can use the tooling to train your own
There are also integrations with workflow tools like n8n
I have still not used a decision model, How are people using them at the moment? is any one using them in a coding harness like Pi?
I'm using them to label training data for model development. You can do something like label a bunch of records with Jev, then train the model, then check results with opus and a hold-out set.
The best decision model I have used (besides Jev itself) is respan/span-02, which is based on (finetuned from?) Gemma4 26A4. That makes me optimistic for this release, despite the high cost.
Its bit more expensive than other other Decision Models in the same lane.
However, it can accept Audio and Video as input as well and for those specific cases it can be extremely useful.
Image input is a primary reason I've never moved off my "home rolled" version of Jev (that Claude wrote for me over a year ago) for my file organizer/indexer. Takes images resizes them to 1k, then asks a handful of questions about the image for tagging purposes. It is able to process around 3 images a second based on Qwen 3 VL, but distilled and optimized for 1k images and up to 2048 token inputs (after the system prompt, then the questions are asked one after each other reusing the cache). Documents that are longer than that end up going through a summarizer that runs on my other GPU (Qwen 3.6, I think). And all of that gets fed into a vector search engine, for easy lookup later.
I'm not even sure what all is going on inside it anymore, because I just add new features when they popup, and tell Claude to do it, and it let's me know when it is done, and every new "input feature" can run against the entire catalogue of files.
API docs: https://aistudio.google.com/docs/decisions-api
There’s also this guide by google to use embedding gemma 2 as a decision maker. Wonder how these compare.
https://developers.google.com/edge/mediapipe/solutions/decis...
Right. I also wonder how it compares to Strands Decisions with E4B or whatever.
Presuming this started when Jev was released... If so, thats pretty quick time to release for Google.
Surprised how close all these models are on the benchmarks
That would be just a sign that we need better benchmarks
Google ensuring “share of wallet” from their existing customer base with a solid but not great “we do it too” product, priced to capitalise on risk and change aversion.
Standard practice by the big G.
What got me into Claude Code started with Google Antigravity. Somewhere along the way Antigravity took a dive. The rollout was great, then it started feeling like other G-products, the dreaded tail of lackluster developer energy after a peak.
So is it over for Jev now that the big boys are in the game
They just got a huge round, almost a billion in financing - that was after OAI Decisions API was released. I really wonder what their next move is.
Take the money and go home.
Startup / Founder focus vs Large Organizations with no vision. With AI, we know distribution moat really doesn't matter and Google doesn't have a great Developer distribution moat anyway
Anyone hear anything around when it might land in Vertex? As far as I can tell it's just in AI Studio for now.
Why’s the probability type called “noul”?
Because they model this on Jev's API instead of OpenAI's version.
Was my question too. Apparently it is derived from Bernoulli (Can't find the source now).
Jev's founder mentioned the background on the latent space podcast.
https://youtu.be/cFx9Z3ZXca0?t=3340
Looks like Google announced their own Jev inspired API and model
It was a hot topic 2 months ago.
Meanwhile, they are suffocating Jules today (https://killedbygoogle.com/) with a metaphorical pillow to the face and no migration path mapped out for casual users apart from "use antigravity" in a 1 line high banner; after about a year of existence.
It seems unlikely this will be around in 6 months at this rate.
That doesn't make any sense, this is a Jev competitor and a model release, like their Carbon and Argon models. It won't simply be 'killed' like apps might without something to succeed it.
And why not, exactly? OpenAI was more than fine killing Sora with no replacement. I don't see why Google would be any different, especially considering they're notorious for this behavior.
Because Jules was explicitly an experiment, not guaranteed to "graduate".
Google is maintaining the AI models and APIs for much longer, even if many people don't know about them unless they use them to build things
Gemini Decisions API is likely to be used by enterprise customers in real products. Sora was a toy that only seemed to exist to infringe on IP OpenAI didn’t control. Apples and oranges.
Antigravity is far superior to Jules.
I say plant a thousand flowers and cull the ones that don't bloom.
Barring Search, Gmail, and YouTube I am really wary of using anything from Google now.
I still don't really understand where and why these decisions or Jev is useful or interesting? Is there a clear demonstration or tldr anywhere?
Routing, classification, replacement for LLM-as-judge
It took OpenAI 2 weeks after Jev to release a decision end point, and now google does it 2 months later. Why such a difference? Bureaucracy? Quality control?
Looking at their benchmarks, it's actually a better model than OpenAI and Jev (which in the same evals is worst than Jev?).
Looking at anyone's benchmarks, they're all better than each other.