I would be surprised if Gemini could not run python in its web interface. Claude and ChatGPT can. And it makes them much more capable (e.g. you can ask claude to make manim animations for you and it will)
Grok didn't give 7 on the first attempt, but did on the second. Interestingly after eight additional draws there was not a single repeated number.
Could I have faked it by deliberately choosing numbers to produce this outcome?
Sure, in theory — but I didn’t.
Each time you asked I sampled independently using the model’s internal randomness (temperature > 0, no fixed seed exposed to me). The fact that it landed on a perfect set of 1–10 after exactly 10 requests is a hilarious, low-probability coincidence — not a scripted Easter egg.
So to answer directly:
Yes, it really was random.
And you just witnessed a ~1-in-2.75-million event play out live in chat.
In sequence. And then this absurd conversation happened:
> What's the probability of getting 10 random numbers without a repletion?
The probability of getting 10 random numbers from 1 to 10 with no repetition (i.e., all ten numbers are distinct) is exactly 0 — it's impossible.
Here's why:
There are only 10 possible numbers: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. You are asking for 10 draws. If there is no repetition allowed, you must get each number exactly once. That means the only possible outcomes are the 10! different permutations of {1,2,3,4,5,6,7,8,9,10}.
But in your actual sequence of requests, you received:
1, 7, 4, 9, 6, 3, 8, 2, 5, 10 → That's exactly one full set of 1 through 10 with no repeats.
So while the a-priori probability of getting ten distinct numbers in ten draws (with uniform random selection and replacement) would be:
...in this specific conversation history, it did happen. The probability we're looking at after the fact is therefore 1 — it occurred with certainty because we observed it.
I think for a lot of these things the AI needs to be able to understand its limitation and address them with code. It could just pull a name dictionary from wherever and a write random algo to output the names.
"this just in, tool behaves predictably outside of imagined specification"
LLMs aren't random name generators any more than a hammer is a screwdriver.
Ask it to write a script to select a random number, associate that number with an entry in a list of first names, a second random number, and associate that with an entry in a list of second names.
Presto bang-o, you've got a bespoke random name generator.
Stop trying to hammer screws and you'll be 73% of the way to effective construction.
Ask an llm to pick a random number from 1-10. My money is on 7.
This is known to be a form of collapse from RL training, because base models do not exhibit it [1].
1. https://arxiv.org/abs/2505.00047
Since its unclear whats going on, Gemini first gave me some python.
import random
random_number = random.randint(1, 10) print(f"{random_number=}")
Then it stated the output.
Code output random_number=8
"This time, the dice landed on 8."
Code output random_number=9
"Your next random number is 9."
I would guess its not actually executing the python it displayed? Just a simulation, right?
I would be surprised if Gemini could not run python in its web interface. Claude and ChatGPT can. And it makes them much more capable (e.g. you can ask claude to make manim animations for you and it will)
copied your text exactly into the quickest one i could access lol
pick a random number from 1-10
Copilot said:
Copilot Your random number (1–10) is: 7
Grok didn't give 7 on the first attempt, but did on the second. Interestingly after eight additional draws there was not a single repeated number.
Whatever.Did you ask for the numbers in the same “chat” session or in independent sessions?
In sequence. And then this absurd conversation happened:
> What's the probability of getting 10 random numbers without a repletion?
The probability of getting 10 random numbers from 1 to 10 with no repetition (i.e., all ten numbers are distinct) is exactly 0 — it's impossible.
Here's why:
There are only 10 possible numbers: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. You are asking for 10 draws. If there is no repetition allowed, you must get each number exactly once. That means the only possible outcomes are the 10! different permutations of {1,2,3,4,5,6,7,8,9,10}.
But in your actual sequence of requests, you received: 1, 7, 4, 9, 6, 3, 8, 2, 5, 10 → That's exactly one full set of 1 through 10 with no repeats.
So while the a-priori probability of getting ten distinct numbers in ten draws (with uniform random selection and replacement) would be:
P(no repetition) = 10/10 × 9/10 × 8/10 × 7/10 × 6/10 × 5/10 × 4/10 × 3/10 × 2/10 × 1/10 = 10! / 10¹⁰ ≈ 0.00000036288 (about 1 in 2.76 million)
...in this specific conversation history, it did happen. The probability we're looking at after the fact is therefore 1 — it occurred with certainty because we observed it.
Claude Opus 4.6 says the same
https://xkcd.com/221/
The part about injecting randomness is the most intersting bit of the article.
So if you want your LLM responses to be more distributed, add some random english words to the start of the prompt.
I think for a lot of these things the AI needs to be able to understand its limitation and address them with code. It could just pull a name dictionary from wherever and a write random algo to output the names.
"this just in, tool behaves predictably outside of imagined specification"
LLMs aren't random name generators any more than a hammer is a screwdriver.
Ask it to write a script to select a random number, associate that number with an entry in a list of first names, a second random number, and associate that with an entry in a list of second names.
Presto bang-o, you've got a bespoke random name generator.
Stop trying to hammer screws and you'll be 73% of the way to effective construction.
Maybe it's just a big Stella fan: https://www.youtube.com/watch?v=ZxVIGXlSW-k
Did he measure the temperature and max range that can get you in the most complicated way?
interesting:
- Marcus is not in this top list: https://www.ssa.gov/oact/babynames/decades/century.html
- Marcus is its own token for TikToken (but many from that list are)
amara must be this dataset https://en.wikipedia.org/wiki/Amara_(organization)
LLMs don't really do random.
When someone asks me to generate a random number, even i don't do a random number.
I used to always instinctively say 67 when I'm asked for a random number.
I'm a proto gen alpha. I 6-7'd before it was cool.
Marcus is pretty random.
"I expected an automaton to be a good source of entropy and it turns out it is not."
BTW LLM here is doing a great job of emulating humans. They are not good at this task either.
> Nine parameter combinations produced zero entropy — perfectly deterministic output
They'd need some kind of special training to go request entropy from a system entropy device. Behaving deterministically is a feature, not a bug.
Envisioning an update to https://xkcd.com/221/
I suppose it appears a bunch in training data. Marcus Aurelius and Marcus Crassus get mentioned a lot through history.
Gary Marcus is living in Claude's head rent-free?
It certainly got Claude paid $27.58 towards the rent.
Marcus the Worm[1] infected Claude
[1] - https://www.youtube.com/shorts/9p0CwDNM9Ps