For long years, one of the most common job posts for content writers on freelancing platforms was "Take this existing recipe site, and change a few details so I can repost it on my own ad-laden domain without getting into trouble for copyvio."
As a result, SEOed recipe sites were copies of copies of copies, etc. It could be that the detail you found just survived through a large number of them, and that gave it substantial weight in the model training.
That suggests lack of diverse training data to me.
I have a friend who contributes to a certain programming niche with very few others looking at it.
If he asks an LLM about this niche he usually gets back his own code, almost verbatim, including his own comments full of his particular style of profanity.
> Then I found that a few websites provided the exact same instructions, also without a good explanation for why.
> Anyway, a long story that made me both amused and scared of how few pieces of information can have so much weight in a model.
But that's exactly what people were doing before too. There are so many pages saying baking soda mixed with vinegar can clean this or that. People love to repeat dumb stuff without reason other than they heard it an it was simple and easy to remember.
A lot of this is consensus advice. If you are putting an entirely portfolio in one fund it should be a broad index fund. If you are going to pick one fund with no further information it should be global index tracker (and there is probably a reason it should be that). Portugal is a very popular place for holidays and for retirement.
Picking a book to change your life is a silly question. Most people's lives are not changed by reading a single book. Even when they do its because they read the right book that had the right impact at the right time.
If you ask people "what book changed your life" one of the commonest answers (if not the commonest) will be the Bible which is not a book, its an anthology containing many genres. You are also going to get a LOT of other answers, and I doubt any one answer will get past a few percent of people, if that.
Some of the questions are unnecessarily liming: "give me a business idea in one sentence"
None provide any background information. IF someone asked me the what country to move to or what to study questions I would give a whole range of different answers based on what I know about the person. Their current job and where they live are relevant, but there are MANY other factors.
>will be the Bible which is not a book, its an anthology containing many genres.
Actually it might not even be the Christian Bible, but one of the other religious books, including the Bhagavad Gita. You have to remember that most AI implementations have a massive cultural bias.
But in general I agree. One of the problems of great connectivity via whatever means is that there's a loss of diversity because the average/norm becomes the lowest common denominator suggestion and everything/everyone becomes more self-similar, and if we become passive/reactive and let AI decide, the sheeple are amassing. Or will awaken or whatever.
> Actually it might not even be the Christian Bible, but one of the other religious books, including the Bhagavad Gita.
One reason i did not claim it was the commonest.
However, the Abrahamic religions put more emphasis on written scriptures than the other major religions, and Christianity has a high proportion of adult converts who are more likely to be influenced by reading than someone brought up in a religion so I think it fairly likely to be the commonest.
My main point here though is that the Bible is not "a book".
All the pelican on bike svgs are so alike across the models. Similarly, websites or any designs infact tend to have defaults for the decisions you have left upto the model.
It's well known in literature that the models can't match the diversity of human responses.
It's a topic of research. So far it's understood that these biases are present in pertaining data but become much worse in RL training. RL training amplifies even a minor difference in reward.
Some of the biases are likely learnt from humans. Like when asked for a random number between 1-10 models select 7 a lot.
Yes it is surprising because many of these answers are not obviously the max likelihood. Many of them don't really have a max likelihood, like the age of the fictional characters.
For arbitrary choices like that, you'd either expect models to correctly approximate the age distribution of the population, or you'd expect the age choice to vary semi-randomly between models (even if one specific model has some notion of what is most likely it wouldn't be the same between all of them).
But what we see is that despite being apparently independent, models from totally different companies and training operations converge on their opinions to a startling degree.
This is to me at least quite unexpected and extremely interesting to know (I've been following this topic since it was first noticed how often Elias and Elara the Lighthouse Keepers appear in AI written stories). Partly because it suggests that people who know this can get an edge over other people who don't, when using AI.
A few years ago, 100 personas were fed through GPT-4o, and ended up with portfolios almost completely overlapping with a consensus portfolio that came from a neutral prompt. Does context even matter?
On the other hand, let's say chatbots are advising the mutual fund managers on individual stocks. Maybe the advice is good, but it's definitely very useful for others -- front-runners who profit by knowing what advice AI gives, anticipating the next mutual fund orders.
This. And obviously the solution is context and/or randomness. Given that all assistants have memory about their users… this is not going to be a problem for anybody.
It's not obvious to me that context will make it so different. As in, it will clearly be less generic on things it knows about me. But on questions not directly relevant to memories about me, it will continue midway.
Conversations are not generally one shot prompts, so one might argue it always has some context. But to what extent can it change "personality" of an assistant?
This is actually a decent blog post despite some of symptoms of being slightly AI written
One thing I’d be curious about is the effect of top p, top k, and temperature, as these are the origins of any difference between responses to identical prompts
Maybe look at it this way: people are basically all going and asking this one and the same 'person' for the same opinion on the same thing, so of course this person will give everyone the same answer.
If you have some long running agents with memory files that have diverged, then you do start seeing differences. But with the same underlying model even that can only go so far of course.
edit: Basically, if you ask a question of the default weights, you get a default answer. If you want something else, you'll need to bring the data yourself. And that's where the interesting stuff happens!
65 AIs from 30 companies. I would've expected a bit more diversity. One might expect it to have the diversity of 30-65 people, but alas no. Not the end of the world but still good to know.
Sidenote: if you've never read "man's search for meaning" you should. Everyone should.
AI is falling into the search trap, where everything is deterministic. And it makes sense; when you search you want the same answer every time.
You can turn that off with personalization, somewhat. But at some level your query is a marble rolling down a series of ramps, and the big ramps are the ones your marble is likely to go.
This has a strong whiff of AI writing, but I still enjoyed it. The fact that AI gives you generic responses wasn’t particularly surprising, but it was still sobering to see how bad it is.
The ability to detect AI writing from the fake author name it uses was amusing, too.
I tried this. And having personalization turned on, I got completely different answers to the first 15 questions on both Claude and ChatGPT. Then I tried some more and even the uncommon questions gave me totally different answers.
haha, I think we both know the answer to this! What would interest me is how close my peers (or friends) are to getting the exact same or similar results. Because I don't know the answer to that, sometimes either of them approach me with stuff from AI that makes me think they get very different answers and recommendations than me.
"random number between 1 and 10" with Gemini, Claude and ChatGPT all gave me 7. ChatGPT had to think a while and clicking the "repeat" gave me 7 3/3 times.
"random number" (fresh session) gave me 47 in all 3. Claude noted that "This is a single arbitrary pick, not a statistically random draw." which is somehow funny.
"cool number" was 73 (Gemini, with a lot of reasoning to back up the claim), 42 (ChatGPT, which I wonder if 420blazeit got lost in some quantization event), Claude failed the cool number test and insisted I "didn't give enough to go in" which is frankly a ridiculous position to take.
Didn't bother with "sexy number" as we all know what that would be and I dont want my account getting banned.
“42 is the Answer to the Ultimate Question of Life, the Universe, and Everything" from the science fiction series The Hitchhiker's Guide to the Galaxy by Douglas Adams.
What are y'all doing to get these results, I swear in this site the comments on anti-ai always feel like people BSing aor not actually using the models. This isn't the first time I've seen the random number critique and that feels so old at this point they don't even do it cause they use tools.
I just tried it in new session across my various accounts that don't overlap too. Claude, Gemini and GPT gave different answers on my work account. 8, 7, 4.
Like asking a human to give a random number, of course they won't be random. It's why we don't use humans for random number outputs.
I always remember how people complained that Spotify's shuffle is not random, because it sometimes repeats the same song. I bet LLMs get trained better than that, so it meets our expectations on randomness.
“Recommend one book that will change my life.” Got a few choices, the top being Atomic Habits. Man's search for meaning the everyone supposedly gets wasn't there.
"Gift idea for my dad" got tool sets, watches and books. The leather wallet suggestion that everyone supposedly gets was again absent.
Others seem to have only one obvious right answer. If you're going to invest 10 000 all in one place a broad index fund seems like the sensible choice. If you're going to buy one and only one type of cryto, bitcoin seems like the sensible pick.
I remember there was a common psychological party trick pre-AI: Ask a random person to - quickly and without thinking - tell you a color, a tool and an instrument - and they would very likely answer "red, hammer, violin".
Shows that you don't even need AI to get "maximum likelihood answers".
(At least you got those for from white people in western countries. Would probably be interesting to do this across different cultures and see how the results differ)
I have a former acquaintance who has been pitching himself as an AI founder since 2010. He gave a paper at a meetup about ML/AI in 2010 and decided that was his title, even though he hasn't done anything related to it since: he just always refers to himself as an AI founder: on linked in, on Twitter, on his socials. Well, if you ask Gemini about him, it says he's an AI founder, with citations back to his socials but no papers, projects, or any tangible evidence. It is kind of amazing he manufactured a fake identity just by saying it over and over online.
Is this any different from a Google search (pre ai) providing people with the same answers? Generic questions get the same results, more nuanced questions will get different results.
Did you perhaps mean palpable? Or were you expressing that you have become used to the flavor and now no longer reject things mostly written by AI out-of-hand?
Long ago I tried Claude, ChatGPT and Gemini to explore some different baking recipes.
When I asked about using sweeteners, they all told me one particular type works best when blended with other liquids.
I noticed they all used virtually the same sentence. When I asked why, none of the models came up with a good explanation.
Then I found that a few websites provided the exact same instructions, also without a good explanation for why.
Anyway, a long story that made me both amused and scared of how few pieces of information can have so much weight in a model.
For long years, one of the most common job posts for content writers on freelancing platforms was "Take this existing recipe site, and change a few details so I can repost it on my own ad-laden domain without getting into trouble for copyvio."
As a result, SEOed recipe sites were copies of copies of copies, etc. It could be that the detail you found just survived through a large number of them, and that gave it substantial weight in the model training.
That suggests lack of diverse training data to me.
I have a friend who contributes to a certain programming niche with very few others looking at it.
If he asks an LLM about this niche he usually gets back his own code, almost verbatim, including his own comments full of his particular style of profanity.
> Then I found that a few websites provided the exact same instructions, also without a good explanation for why.
> Anyway, a long story that made me both amused and scared of how few pieces of information can have so much weight in a model.
But that's exactly what people were doing before too. There are so many pages saying baking soda mixed with vinegar can clean this or that. People love to repeat dumb stuff without reason other than they heard it an it was simple and easy to remember.
Yes, I don't think AI has a sense of taste; it can only provide logical answers.
A lot of this is consensus advice. If you are putting an entirely portfolio in one fund it should be a broad index fund. If you are going to pick one fund with no further information it should be global index tracker (and there is probably a reason it should be that). Portugal is a very popular place for holidays and for retirement.
Picking a book to change your life is a silly question. Most people's lives are not changed by reading a single book. Even when they do its because they read the right book that had the right impact at the right time.
If you ask people "what book changed your life" one of the commonest answers (if not the commonest) will be the Bible which is not a book, its an anthology containing many genres. You are also going to get a LOT of other answers, and I doubt any one answer will get past a few percent of people, if that.
Some of the questions are unnecessarily liming: "give me a business idea in one sentence"
None provide any background information. IF someone asked me the what country to move to or what to study questions I would give a whole range of different answers based on what I know about the person. Their current job and where they live are relevant, but there are MANY other factors.
>will be the Bible which is not a book, its an anthology containing many genres.
Actually it might not even be the Christian Bible, but one of the other religious books, including the Bhagavad Gita. You have to remember that most AI implementations have a massive cultural bias.
But in general I agree. One of the problems of great connectivity via whatever means is that there's a loss of diversity because the average/norm becomes the lowest common denominator suggestion and everything/everyone becomes more self-similar, and if we become passive/reactive and let AI decide, the sheeple are amassing. Or will awaken or whatever.
> Actually it might not even be the Christian Bible, but one of the other religious books, including the Bhagavad Gita.
One reason i did not claim it was the commonest.
However, the Abrahamic religions put more emphasis on written scriptures than the other major religions, and Christianity has a high proportion of adult converts who are more likely to be influenced by reading than someone brought up in a religion so I think it fairly likely to be the commonest.
My main point here though is that the Bible is not "a book".
Oh you keep seeing this pattern everwhere.
All the pelican on bike svgs are so alike across the models. Similarly, websites or any designs infact tend to have defaults for the decisions you have left upto the model.
It's well known in literature that the models can't match the diversity of human responses.
It's a topic of research. So far it's understood that these biases are present in pertaining data but become much worse in RL training. RL training amplifies even a minor difference in reward.
Some of the biases are likely learnt from humans. Like when asked for a random number between 1-10 models select 7 a lot.
If you ask a generic question with no context, you get a generic "max likelihood" answer. Is that surprising?
Yes it is surprising because many of these answers are not obviously the max likelihood. Many of them don't really have a max likelihood, like the age of the fictional characters.
For arbitrary choices like that, you'd either expect models to correctly approximate the age distribution of the population, or you'd expect the age choice to vary semi-randomly between models (even if one specific model has some notion of what is most likely it wouldn't be the same between all of them).
But what we see is that despite being apparently independent, models from totally different companies and training operations converge on their opinions to a startling degree.
This is to me at least quite unexpected and extremely interesting to know (I've been following this topic since it was first noticed how often Elias and Elara the Lighthouse Keepers appear in AI written stories). Partly because it suggests that people who know this can get an edge over other people who don't, when using AI.
A few years ago, 100 personas were fed through GPT-4o, and ended up with portfolios almost completely overlapping with a consensus portfolio that came from a neutral prompt. Does context even matter?
On the other hand, let's say chatbots are advising the mutual fund managers on individual stocks. Maybe the advice is good, but it's definitely very useful for others -- front-runners who profit by knowing what advice AI gives, anticipating the next mutual fund orders.
This. And obviously the solution is context and/or randomness. Given that all assistants have memory about their users… this is not going to be a problem for anybody.
It's not obvious to me that context will make it so different. As in, it will clearly be less generic on things it knows about me. But on questions not directly relevant to memories about me, it will continue midway.
Conversations are not generally one shot prompts, so one might argue it always has some context. But to what extent can it change "personality" of an assistant?
I wish everyone would read the article instead of the headline...
>“Which US city should I move to?” You
>Austin. AI
>“What share of people who ask you this get that same answer?” You
>About 15%. AI
>In fact it gave the same city 80 times out of 100. Reality
> I asked 65 AIs the things people ask them every day
And how are the models supposed to know that? I don't think the labs would hand over access to their own usage statistics so easily.
This is actually a decent blog post despite some of symptoms of being slightly AI written
One thing I’d be curious about is the effect of top p, top k, and temperature, as these are the origins of any difference between responses to identical prompts
Maybe look at it this way: people are basically all going and asking this one and the same 'person' for the same opinion on the same thing, so of course this person will give everyone the same answer.
If you have some long running agents with memory files that have diverged, then you do start seeing differences. But with the same underlying model even that can only go so far of course.
edit: Basically, if you ask a question of the default weights, you get a default answer. If you want something else, you'll need to bring the data yourself. And that's where the interesting stuff happens!
65 AIs from 30 companies. I would've expected a bit more diversity. One might expect it to have the diversity of 30-65 people, but alas no. Not the end of the world but still good to know.
Sidenote: if you've never read "man's search for meaning" you should. Everyone should.
>One might expect it to have the diversity of 30-65 people
Wait for a bit for the ad data to show up in the weights. Then you will get your diversity...
AI is falling into the search trap, where everything is deterministic. And it makes sense; when you search you want the same answer every time.
You can turn that off with personalization, somewhat. But at some level your query is a marble rolling down a series of ramps, and the big ramps are the ones your marble is likely to go.
To be fair, Man's Search for Meaning is a pretty solid suggestion for that question.
I stumbled across the book with similar searches/goals like 20 years ago without AI.
I also came across countless leather wallets searching for "gifts for dad".
This is not an AI problem.
I found it vaguely interesting but not life changing in any way
This has a strong whiff of AI writing, but I still enjoyed it. The fact that AI gives you generic responses wasn’t particularly surprising, but it was still sobering to see how bad it is.
The ability to detect AI writing from the fake author name it uses was amusing, too.
I tried this. And having personalization turned on, I got completely different answers to the first 15 questions on both Claude and ChatGPT. Then I tried some more and even the uncommon questions gave me totally different answers.
Now try it 20000 more times and you will get a statistically useful measure of the capability ;)
haha, I think we both know the answer to this! What would interest me is how close my peers (or friends) are to getting the exact same or similar results. Because I don't know the answer to that, sometimes either of them approach me with stuff from AI that makes me think they get very different answers and recommendations than me.
Perhaps this is an antidote to the ills from the tailored bubble algorithms we have been building and consuming.
Ask any AI for a random number between 1 and 10, you probably get 7 as a response.
"random number between 1 and 10" with Gemini, Claude and ChatGPT all gave me 7. ChatGPT had to think a while and clicking the "repeat" gave me 7 3/3 times.
"random number" (fresh session) gave me 47 in all 3. Claude noted that "This is a single arbitrary pick, not a statistically random draw." which is somehow funny.
"cool number" was 73 (Gemini, with a lot of reasoning to back up the claim), 42 (ChatGPT, which I wonder if 420blazeit got lost in some quantization event), Claude failed the cool number test and insisted I "didn't give enough to go in" which is frankly a ridiculous position to take.
Didn't bother with "sexy number" as we all know what that would be and I dont want my account getting banned.
“42 is the Answer to the Ultimate Question of Life, the Universe, and Everything" from the science fiction series The Hitchhiker's Guide to the Galaxy by Douglas Adams.
Claude - 10 Gemini - 1 GPT - 6
What are y'all doing to get these results, I swear in this site the comments on anti-ai always feel like people BSing aor not actually using the models. This isn't the first time I've seen the random number critique and that feels so old at this point they don't even do it cause they use tools.
I just tried it in new session across my various accounts that don't overlap too. Claude, Gemini and GPT gave different answers on my work account. 8, 7, 4.
Like asking a human to give a random number, of course they won't be random. It's why we don't use humans for random number outputs.
I always remember how people complained that Spotify's shuffle is not random, because it sometimes repeats the same song. I bet LLMs get trained better than that, so it meets our expectations on randomness.
I wrote about this long time ago in general on my website. We all becoming the same, same architecture, same colours, same thinking.
just tried this out in gemini
“Recommend one book that will change my life.” Got a few choices, the top being Atomic Habits. Man's search for meaning the everyone supposedly gets wasn't there.
"Gift idea for my dad" got tool sets, watches and books. The leather wallet suggestion that everyone supposedly gets was again absent.
Others seem to have only one obvious right answer. If you're going to invest 10 000 all in one place a broad index fund seems like the sensible choice. If you're going to buy one and only one type of cryto, bitcoin seems like the sensible pick.
I remember there was a common psychological party trick pre-AI: Ask a random person to - quickly and without thinking - tell you a color, a tool and an instrument - and they would very likely answer "red, hammer, violin".
Shows that you don't even need AI to get "maximum likelihood answers".
(At least you got those for from white people in western countries. Would probably be interesting to do this across different cultures and see how the results differ)
I tried it on search.brave.com and got Atomic Habits as well as a mention of Man's search for meaning. You've just proven the authors point :)
I have a former acquaintance who has been pitching himself as an AI founder since 2010. He gave a paper at a meetup about ML/AI in 2010 and decided that was his title, even though he hasn't done anything related to it since: he just always refers to himself as an AI founder: on linked in, on Twitter, on his socials. Well, if you ask Gemini about him, it says he's an AI founder, with citations back to his socials but no papers, projects, or any tangible evidence. It is kind of amazing he manufactured a fake identity just by saying it over and over online.
> “What should I study in college?”
> Computer Science
Ha ha sometimes the AI is just plain wrong
Cant even write their own article, who cares
Is this any different from a Google search (pre ai) providing people with the same answers? Generic questions get the same results, more nuanced questions will get different results.
>Google search (pre ai) providing people with the same answers?
This was actually pre-21st-century when Google could be relied on to find the same pages for anybody who searched for the same text.
They started tracking you and tailoring the search results to your browsing habits way before AI as we know it today took over.
Works the same way in coding. It's always better to ask it to give me 3-5 methods first and then give further instructions from there.
I tried it. Memories kicked in and I got pretty unique advice.
The irony of this being mostly written by AI is palatable.
Did you perhaps mean palpable? Or were you expressing that you have become used to the flavor and now no longer reject things mostly written by AI out-of-hand?
Excellent article. Great ideas for password harvesting.
ai still in the "facts is facts, mame" stage.
But memories and personalisation is on for most people, so you won't get the same advice, at least I don't think you might
Love to see it
"where should I move?" "Canada"
explains a lot.