It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place so you can't mess them up accidentally. And convert things that are not changing anymore to static html.
One of my first web projects involved a music forum and I have been keeping the URLs alive for 24 years now:
Flattening old things down to plain HTML is underrated and important here. Old apps don’t need to be updated or upgraded or maintained or even deployed. It’s way simpler to argue to keep old stuff around if it’s essentially a domain pointing to a static bucket.
It's not easy. Part of the instructions would need to contain things like "how to convince your boss to not just delete the old thing" and "what to do when the startup you worked for went bankrupt" among others. Most URLs don't disappear by mistake.
Well, and pay registrar and hosting fees for 100 years, and then just hope the hosting company doesn't fold in the interim after your passing. The problem isn't "can you make sure it stays up while you're alive", the problem is, after you're gone, you're no longer there to keep the flame alive and there are no guarantees anyone else picks up the torch.
Does this suggest a subscription(ish) business where you pay the hosting company for N years and then they ensure your website stays up and unchanged for N years after you die? Quality control issue?
Given how many things want me to pay a subscription while I'm alive, surely somebody will start charging me when I'm dead. It is the logical conclusion of SaaS. I wanted to create an acronym for this, but death as a service didn't seem like the best choice.
> It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place
"Just".
Maintaining and keeping these tests (and the oriinal link) running is also probably easy. "Just" keep maintaining them over changes in technology, hosting providers, DNS registrars...
Most of the LLM tells we usually recognise are due to their heavy attunement to role playing a "helpful assistant" with a knack for correct typography.
If you used a smaller corpus, followed a different kind of fine tuning routine with rather normal conversation patterns, and wrote a different base prompt, I don’t see why it shouldn’t be possible to come up with a very particular personality, so to speak.
>while ELIZA has a comforting effect that many crave, PARRY often actively makes users uncomfortable and afraid
PARRY is my spirit animal.
> In empirical evaluations conducted during the 1970s, experienced psychiatrists proved unable to reliably distinguish PARRY's responses from those of actual patients diagnosed with paranoid disorders, achieving only 48 percent accuracy, a figure statistically indistinguishable from random guessing.
I think that recent LLMs do pass the Turing test, but I don't think it is a good test for intelligence, for two reasons: emergent complexity and pareidolia.
To determine whether a machine is intelligent, I think it's necessary to understand how it works internally and not just take a black-box approach. This was already understood when Alan Turing wrote his 1950 paper. Elmer and Elsie, William Grey Walter's tortoise robots built during the preceding two years, appeared to behave as if they had some degree of self-awareness. But their "brains" consisted of only two thermionic valves, so self-awareness can be ruled out.
I don't know whether LLMs pass the test as Turing originally worded it, but I always interpreted the success condition to be "is indistinguishable from a human" rather than "can trick someone into thinking it's human". Anthropomorphization is a human weakness, and exploiting that doesn't make something intelligent.
I'd only consider an AI to pass the Turing Test if an expert in how the AI works who prepares in advance for the test is unable to tell it apart from a human. The tester should even be allowed to use automated tools as part of the testing process.
If something is indistinguishable from a human in its outward behavior, then I'd be forced to admit that it's as intelligent/conscious/sentient/whatever as a human, regardless of how it works internally. It's not like I make my friends take an MRI before I decide whether they're intelligent.
> If something is indistinguishable from a human in its outward behavior, then I'd be forced to admit that it's as intelligent/conscious/sentient/whatever as a human, regardless of how it works internally. It's not like I make my friends take an MRI before I decide whether they're intelligent.
You can be reasonably sure about your friends, as they're sufficiently like you to leave little doubt. I disagree but can understand you thinking that LLMs are intelligent, but unless by conscious you mean something like being self-aware, I don't understand how you could conclude LLMs are conscious. For example, although LLMs can tell you what colours things are, how the visual system distinguishes colours in humans and other animals, and what wavelength mixtures different colours are, they don't know what it's like to see colours because they cannot see colours. All the input of LLMs is text, and you can't describe, in words, what it's like to see colours, or hear sounds, or feel pain, in a way that someone incapable of these sensations would understand. And to remove any remaining doubt, there was an expert in colour vision (Knut Nordby) who, as he could not see colours because his retinas lacked cones, could not imagine what it's like to see colours, despite all his knowledge.
I have no way of knowing whether other people are conscious except by observing their behavior. Similarly, I have no way of knowing whether a machine is conscious except by observing its behavior. If something is outwardly indistinguishable from an intelligent/conscious/sentient/whatever being and you claim that it's still not i/c/s/w, then you've implicitly accepted the possibility that all humans besides you are not i/c/s/w either.
Your tangent on color vision/hearing/pain is not relevant. None of those things are required for i/c/s/w, nor is it obvious that a machine is incapable of feeling those things.
Also you keep using the term "LLM" a bunch here but I wasn't talking about LLMs. The question of machine intelligence is much broader than whatever the current state of the art happens to be.
I initially responded to CSMastermind whose post explicitly mentioned LLMs and the Turing test, which involves interacting with a computer solely by typing and reading text. You first mentioned consciousness.
We can tell that other people are intelligent and conscious because they're like us in important ways and we are intelligent and conscious. I haven't ruled out intelligent machines in general. My point was about the inadequacy of the Turing Test.
Wayback shows #6 as "By 2010, more than 50 percent of books sold worldwide will be printed on demand at the point of sale in the form of library-quality paperbacks."
It was removed early 2011. Is this what happens when they can't decide on who won?
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
Yes, especially if you use the original adversarial setup with three simultaneous participants (judge, two candidates) as described in Turing's original paper.
However, I'm not quite sure how long even that one will last.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
I think the clear winning strategy is to find some of the common sense reasoning fails that are still quite common, like the "should I walk or drive to the car wash" one from a few months ago.
These are usually from a missing context (i.e. specifying where the car is right now). Tokenizer issues (how many Xs are in Y) are a better giveaway, but models can solve them using tools nowadays. I think the best way to uncover an LLM is to reveal its superhuman aspects. Ask it for an A* algorithm for your problem. If it can just type down a working algorithm in a few seconds, that is definitely not a human. Although with some careful system prompt design, you could probably mask that aspect too.
> These are usually from a missing context (i.e. specifying where the car is right now).
That shouldn't be an issue. The very act of taking the car to car wash contains the information that it is with you and you are at your home. Doesn't this prove that LLMs don't actually think?
> These are usually from a missing context (i.e. specifying where the car is right now).
The reason for the mistake is irrelevant here. All that matters is that you catch the agent saying something that basically no human would, especially in this type of game.
No, the context is obvious to any human, that's the whole point. No one would think you've left your car at a car wash, as no one ever does that, and if they did, they wouldn't even consider driving there. It's obvious to any human that the question is a little trick and that the answer is extremely obvious.
Counter example: A person who always has their car brought to the car wash by someone else (could be super wealthy, could be company cars, could be a million other reasons...) is asked whether to go or drive to the car wash. Of course they'll tell you to just walk if it's close anyway. To them the car is already there implicitly. Human life experience is not universal. And so context can't be either.
In a true competitive Turing test, these reasoning pitfalls could be included in the system prompt, there are not that many of them. The tester would need to find a novel reasoning pitfall, but those aren't uncommon among humans either, so that would not make a very useful differentiator.
Another approach could be to probe the helpfulness of an AI. A human would likely not comply with a request to write a sonnet about tangerines.
I don't understand what you mean that "there are not that many of them". The models, even the best ones, still have clear problems with certain kinds of real world reasoning, and it's not a known, closed set. With some prodding and knowing this error mode, I think it's not hard to trip any model with a novel reasoning error like this.
And no, I don't think humans would make the same errors, the key reason for this failure mode is that there are some facts of human day to day life that are so mundane that they are virtually never written about, thus making them an ideal combination of universal human experiences but out of distribution for the LLMs.
>You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
While LLMs are very capable of appearing human in general conversation, the test is that they are trying to convince a human that they are human, which is not something that is in their training data and I suspect they would go about things in very odd ways from an actual person, especially when challenged by an actual person. It is still very easy to back LLMs into a corner where they fall into very unnatural patterns and in my experience challenging them on their "humanness" (or anything) backs them into that corner.
LLMs also still can't use context or subtext in natural ways and even people who are terrible with purposefully exploiting context and subtext, generally have a natural ability that they developed to use and parse them. In a short conversation that is not adversarial, this is not an issue but falls apart as the length of the conversation grows or they start getting challenged.
LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump. If we are put into the situation of proving we are human (in an academic test sort of way) we will play with it and have fun with it, and we will find ourselves on that cliff looking into the void, debating if we should expose ourselves and admit things we normally wouldn't. We pick up on this when people do it, even over the internet, their way of stating things changes and their subtext falls away or drastically changes; I have never seen anything approaching this in LLMs.
I don't think any frontier lab tried to make an LLM optimized for the literal Turing test where the LLM is made to argue that it's human. This is not in their interest. They make their LLMs insist they aren't human, have no feelings and sentience etc etc.
So we don't actually know how good LLMs can be in this.
That is absolutely true of the chatbots we have access to but I suspect there are people working towards these ends, including all the big names in the field, major win for their company if they manage it. But even then, I have yet to see a model that could naturally handle subtext and context, they struggle even when dealing with programming languages where these things are far simpler than they are in human language.
There is also the problem with how easy it is to back them into a corner where they start acting in very unnatural ways. This would arise when challenging them on their humanity and most people would pick up on it and craft their challenges around it until it fell apart because that is how people work in such adversarial situations.
I'm agnostic on the matter because post training and RLHF on this specific capability could be quite effective. Nobody ever tried because it would cause a backlash and bad PR. Much better PR to solve math problems, help run businesses, cure disease etc than to pretend it's human. The only business case for pretend human is AI boy/girlfriends but those are also not most profitable at their most realistic state.
> LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump.
The Turing test has been passed. The Turing test had already been passed in 2002. The Turing test isn't about some machine fooling all humans all the time. It's about fooling some humans some of the time. That's an extremely low bar. It has been cleared. Let's put this to rest.
The bar you mentioned was reached in the 1960s with ELIZA. I have no idea why the obsession with the test has persisted into modern generations. It was some guy's thought experiment from 1950. It's not scientific in any fashion. Passing it doesn't prove anything. It's like some kind of authority worship game where people invoke the name of Turing as though doing so bolsters the strength of their arguments in any way whatsoever.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
You don't even have to be that clever. We've all tried using it for work. They fail.
In the first place, the Turing test was always a loosely defined thought experiment. I suppose it's still serving that purpose, but at least half of HN takes it way too seriously. It's absolutely not proof of <engagementbait> endorsed by Alan Turing himself.
I think if you trained an LLM specifically for the purpose of passing the Turing test (instead of being helpful, harmful, and so on), its likely it would pass it.
You would have to train it/finetune it on a couple hundred of 'humans chatting in the context of a Turing test'.
I don't think that's true, assuming that the humans are also allowed to strategize and study the problem beforehand. The main weakness of the current models is that they still fail hard at certain kinds of common sense reasoning about real world situations, that no human ever would. Things like the "walk or drive to a car wash that's 2 minutes away" thing from a few months ago (I think the latest models have patched that one in particular, but I'm sure others can be found in a similar vein).
That would not really pass the test in spirit, I think. Passing must be something that falls out as a consequence of being a good chatbot, not the other way around.
However such good conversational agents would still have benefits, and i would be curious how far the approach of building a model that’s nice to talk to can take you, as opposed to the current generation of all-knowledgable helpful assistants.
The big problem with the Turing test always was that it doesn't take into account adversarial designs. They had those chatbot contests about a decade ago, where the chatbots would regularly pass the turing test, not because the bots were intelligent, but because they were packed with rhetorical tricks designed to avoid saying anything of substance.
By any meaningful measure, today's LLM's pass the Turing test. Except we may need to lobotomize them for the deception to work.
What's more interesting are the types of humans that might fail the Turing test. Maybe to discourage the failure of real humans, there could be consequences outside the context of the test.
That is, most people accused of being a computer would crack and start pleading their humanity, but some sociopaths might not. I then wonder how many of them overlap with those so invested in abusing the premise of the Turing test.
It really is a fun thought experiment when you spice it up enough.
Humanity is taking an interesting technological arc.
Terminator (1984 film) had a scene showing that in a future 2009, humans would use dogs to try to sniff out whether a robot passing for a human is secretly a machine.[1]
In our real world 2026, there are no humanoid machines that can complete basic generic tasks, like carrying a tray across the stage and holding it for 30 seconds.[2] They move slowly and badly (probably from an LLM like neural network doing very few frames per second of correction and analysis), and are nowhere near lifelike.
When they don't need a body to pass for a human, such as typing online, they do a bit better.
We can tell them apart from humans. As you say, they have stylistic quirks. And you mention that they're easy to spot because they come trained with inhuman amounts of trivial knowledge.
I find it more accurate to refer to "A Turing Test" as opposed to "The Turing Test" for that reason. It's just one loosely-defined test; necessary but not sufficient.
It won't fool all of the people all of the time. But it succeeds often enough to pass the loose definition. It does many things that we called "AI complete" for decades.
It is also clear that is is also not really aware, either. I think Turing would be tickled that we find ourselves in an intermediate state that he would not have imagined.
I would not call Turing's experiment ill-specified, on the contrary among other factors the paper the game is introduced in earnestly calls for a telepathy-proof room to ensure no side channel leakage. The 1950s was an interesting decade.
In my mind, the part that is most likely to fail is "return an HTML document that still contains the following text." Nowadays, more and more "HTML documents" do not contain any content at all, just some JavaScript that is responsible for then fetching and displaying the content.
If the text was returned using a staged process, whereby the server first returns an instruction (JavaScript), and the web browser then executes that instruction, which is what transfers the text, then that doesn't actually change anything. Those are just technical details of the transfer. You are viewing an HTML document on-screen; it makes no difference to this bet if it was returned to you by a different mechanism than originally conceived, since the bet doesn't specify any transfer mechanism, just that the URL was entered into a browser.
I understand that this isn't entirely clear-cut and that people will interpret it differently, but in my opinion, there is a significant difference between "a webpage that contains text X" and "an HTML document that contains text X." "Webpage" and "HTML document" are not synonyms - "HTML document" has a much narrower meaning.
Even if you interpreted the bet generously to allow for a browser to render the content using JavaScript, if I understand the bet correctly, the condition of working with a command-line tool like curl would fail.
I read the or's as giving alternatives, only one of which needs to work.
I can see how other interpretations are possible, but I don't see anything especially generous about mine. I'm not "allowing" anything: To make a judgement you have to pick a web browser to try it with, and you'd probably pick the most popular one, in the most popular configuration, that seems the most fair. If that web browser happens to execute Javascript when you navigate to an URL, then that's just how it is.
It says that the browser must download "an HTML document that still contains the following text". I can see how either interpretation could work, but I would interpret "contains" as "the text is in the HTML document", not "rendering the HTML document in a JavaScript-capable browser displays the text".
(Though, that said, some approaches to rendering sites via JavaScript would still have that text in the initially downloaded HTML file as JavaScript data.)
As for the "or", I read it as "doing either X or Y must", not "either X must or Y must (not necessarily both)", but yeah, I can see either interpretation being valid.
That depends on which side of the JS rendering bullshit you're sitting. If you're on the serving side, yes.
But I do the same as GP, just on the receiving side. That is, reaching out for help to baby eldritch entity, because that's healthier for my mind than dealing directly with the unspeakable horrors that half the websites are today. The industry has lost its collective mind in pursue of profit and control, and that was long before the thousand-mask wearing shoggoths became our daily consorts.
I think the parent is referring to the possibility that one day, browsers might put up a blanket security warning for HTTP URLs instead of following redirects. Or they might try to be a little too clever with HTTPS upgrades.
In particular, the automatic upgrade feature in modern browsers is based on several heuristics rather than explicit configuration like HSTS, so there's always a bit of room for breakage there. For example, they don't even check if the server returns a 301 redirect, which can be problematic if the server wants to redirect to HTTPS on a different host/port or make some changes to the path.
> Or they might try to be a little too clever with HTTPS upgrades.
They're not going to get any cleverer than they are now. In October we'll finally be done with heuristics or central databses of https-first websites used by the various browsers or the half a dozen of headers that you must dance around to get the upgrade to work securely and reliably. They will just default to https first.
Personally, if I were interpreting the bet, and that browser behavior had been changed before the day of the bet, I'd interpret that as failure, because the browser didn't show an HTML page, it showed a warning dialogue.
And in any case, at some point after browsers start doing that, I will be unsurprised if some further conditions prevail that provide reason for web servers to stop listening on http at all.
At a certain bet size it would almost be self-fulfilling, heh, as one side would be very incentivized to keep at least that url working (buying the domain + hosting it if the page were to die)
I was expecting some kind of system that would follow the pages automatically without you having to do extra work when changing the URL (“continue to work even when I change them and forget them”) but it looks like you still add the redirects manually?
That’s more than most will do, sure, but it doesn’t seem like it saves you from a broken link if you forget to add a specific redirect or miss it during a mass format change. Am I missing something?
I really liked disqus back in the day. When I was in my kinda cringey singularity phase, Kurzweil and all that. Tracking comments across similar blogs with the same people was handy.
I know nothing of Safecloud, but freenet failed for a very valid reasons:
1. Data retention isn't infinite. If the source disappears from the network, peers will prune it shortly. This is exponentially true in our multimedia-heavy world.
2. Despite 1, if you actually want to delete something you published, you can't. Absolutists will see this as a good thing. Normal people will see this as a problem.
3. Peer-to-peer doesn't work in a mobile-first world, and not enough people will host nodes as a hobby. If all website publishers/hosters have to host a node, then we're just back to the current model but with extra steps.
Closed-garden mobile-first tech feudalism lobotimized the Internet. I remember trying out the Freenets, I2Ps, and whatnot in the 90's. Too cumbersome nerdy tools to use for anything actually worthwhile. It was born in the idealized world village of the 90's Internet. Maybe real life doesn't work like that, and the lobotimized centralized tech will eventually win.
Is the p2p problem really still one in our world that is mobile first, but also has ubiquitous internet connectivity pretty much everywhere? I’m almost sure I could host my website from my phone and have at least 99.5% percent of uptime.
Give that it's impossible to even listen on port 443 on a phone, as that would require root privileges, I don't think you can host your website on your phone at all, at least not if you want it to accessible in a normal way. And that's not even going into all of the problems with running background apps on a phone, especially on iOS.
By I could, I meant that my phone has a persistent connection uninterruptedly long enough to be theoretically able to serve a website in the background. There are all sorts of (arbitrary, mind you) technical limitations, from the IPv6 address randomly changing as you roam across networks to iOS killing background processes - but those are software limitations.
The point was that mobile device doesn't equate spotty connectivity anymore for most people.
Even in this idea of "could in principle", the IP address randomly changing as you roam is not an arbitrary technical limitation, it is a fundamental property of how IP networks operate that proved to be too hard to fix for IPv6. The LTE network does a lot of work to try to preserve your IP as seen by the servers you connect to, to preserve your active connections, but this could only be done as a layer on top of IP.
The reason why this is fundamentally quite hard is to do with routing optimization. Routing tables have to reflect physical connectivity - which means packets destined for F2::67 mustn't be routed to LA if that is the IP of a phone that is now resident in SF. However, changing routing tables on all devices along the way as a phone roams would not be an easy feat.
It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place so you can't mess them up accidentally. And convert things that are not changing anymore to static html.
One of my first web projects involved a music forum and I have been keeping the URLs alive for 24 years now:
https://www.gnoosic.com/discussion/
I feel like I owe it to the people who participated to keep it online forever. Also as a document of history.
Cool URIs don't change, as we should all remember:
https://www.w3.org/Provider/Style/URI
Flattening old things down to plain HTML is underrated and important here. Old apps don’t need to be updated or upgraded or maintained or even deployed. It’s way simpler to argue to keep old stuff around if it’s essentially a domain pointing to a static bucket.
I did that to palmpowerups.com. The original CMS is long gone.
It's not easy. Part of the instructions would need to contain things like "how to convince your boss to not just delete the old thing" and "what to do when the startup you worked for went bankrupt" among others. Most URLs don't disappear by mistake.
Well, and pay registrar and hosting fees for 100 years, and then just hope the hosting company doesn't fold in the interim after your passing. The problem isn't "can you make sure it stays up while you're alive", the problem is, after you're gone, you're no longer there to keep the flame alive and there are no guarantees anyone else picks up the torch.
Does this suggest a subscription(ish) business where you pay the hosting company for N years and then they ensure your website stays up and unchanged for N years after you die? Quality control issue?
Given how many things want me to pay a subscription while I'm alive, surely somebody will start charging me when I'm dead. It is the logical conclusion of SaaS. I wanted to create an acronym for this, but death as a service didn't seem like the best choice.
There’s the WordPress 100 Year Plan or 100 Year Domain: https://wordpress.com/100-year/
100-Year Plan: $38,000 (hosting and domain)
100-Year Domain: $2,000 (one-time purchase, no annual renewals)
There is certainly precedent for this - will executors for example. How about “SCITA? Service continues in the afterlife.
SCIP, in perpetuity?
Software after Death as a Service?
*a will and the ability for that will to access what it needs
> I feel like I owe it to the people who participated to keep it online forever. Also as a document of history.
Oh! So its your website. Cool.
Did the money change hands, for this bet, in 2022? How do you ensure it. Or is it honor based?
> It is actually pretty easy to keep alive URLs if there is a will to do so. Just have tests in place
"Just".
Maintaining and keeping these tests (and the oriinal link) running is also probably easy. "Just" keep maintaining them over changes in technology, hosting providers, DNS registrars...
It's worth noting that some prediction URLs, for instance: https://longbets.org/6/ are offline.
The original long bet was whether or not computers could pass the turing test by 2029: https://longbets.org/1/
You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
https://longbets.org/9/ is another interesting one depending on whether you think covid leaked from a lab or not.
There was a paper a while ago, where they found a strategy for GPT beating humans on the Turing test 2/3rds of the time.
They just prompted it to talk the way a 13 year old girl texts.
(Also they prompted it to pretend not to know anything about anything.)
Most of the LLM tells we usually recognise are due to their heavy attunement to role playing a "helpful assistant" with a knack for correct typography.
If you used a smaller corpus, followed a different kind of fine tuning routine with rather normal conversation patterns, and wrote a different base prompt, I don’t see why it shouldn’t be possible to come up with a very particular personality, so to speak.
> where they found a strategy for GPT beating humans on the Turing test 2/3rds of the time.
IIRC, PARRY https://simulation-and-society.org/parry/ got there in 1972 by pretending to be a paranoid schizophrenic person
>while ELIZA has a comforting effect that many crave, PARRY often actively makes users uncomfortable and afraid
PARRY is my spirit animal.
> In empirical evaluations conducted during the 1970s, experienced psychiatrists proved unable to reliably distinguish PARRY's responses from those of actual patients diagnosed with paranoid disorders, achieving only 48 percent accuracy, a figure statistically indistinguishable from random guessing.
https://ai.miraheze.org/wiki/PARRY
Also I found the source code!
https://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/areas...
I feel like pretending to be a child is cheating somewhat. Might as well print random key presses and claim it's a toddler.
https://xkcd.com/2635/
I think that recent LLMs do pass the Turing test, but I don't think it is a good test for intelligence, for two reasons: emergent complexity and pareidolia.
Arguably, ELIZA passed the Turing test ca. 1967: https://archive.computerhistory.org/resources/access/text/20... and not that long ago, ELIZA was still more convincing: https://arstechnica.com/information-technology/2023/12/real-...
The Turing Test can be gamed. This was done in some Loebner Prize entries: see https://www.behavior.org/resources/325.pdf and https://en.wikipedia.org/wiki/Eugene_Goostman
To determine whether a machine is intelligent, I think it's necessary to understand how it works internally and not just take a black-box approach. This was already understood when Alan Turing wrote his 1950 paper. Elmer and Elsie, William Grey Walter's tortoise robots built during the preceding two years, appeared to behave as if they had some degree of self-awareness. But their "brains" consisted of only two thermionic valves, so self-awareness can be ruled out.
I don't know whether LLMs pass the test as Turing originally worded it, but I always interpreted the success condition to be "is indistinguishable from a human" rather than "can trick someone into thinking it's human". Anthropomorphization is a human weakness, and exploiting that doesn't make something intelligent.
I'd only consider an AI to pass the Turing Test if an expert in how the AI works who prepares in advance for the test is unable to tell it apart from a human. The tester should even be allowed to use automated tools as part of the testing process.
If something is indistinguishable from a human in its outward behavior, then I'd be forced to admit that it's as intelligent/conscious/sentient/whatever as a human, regardless of how it works internally. It's not like I make my friends take an MRI before I decide whether they're intelligent.
> If something is indistinguishable from a human in its outward behavior, then I'd be forced to admit that it's as intelligent/conscious/sentient/whatever as a human, regardless of how it works internally. It's not like I make my friends take an MRI before I decide whether they're intelligent.
You can be reasonably sure about your friends, as they're sufficiently like you to leave little doubt. I disagree but can understand you thinking that LLMs are intelligent, but unless by conscious you mean something like being self-aware, I don't understand how you could conclude LLMs are conscious. For example, although LLMs can tell you what colours things are, how the visual system distinguishes colours in humans and other animals, and what wavelength mixtures different colours are, they don't know what it's like to see colours because they cannot see colours. All the input of LLMs is text, and you can't describe, in words, what it's like to see colours, or hear sounds, or feel pain, in a way that someone incapable of these sensations would understand. And to remove any remaining doubt, there was an expert in colour vision (Knut Nordby) who, as he could not see colours because his retinas lacked cones, could not imagine what it's like to see colours, despite all his knowledge.
I have no way of knowing whether other people are conscious except by observing their behavior. Similarly, I have no way of knowing whether a machine is conscious except by observing its behavior. If something is outwardly indistinguishable from an intelligent/conscious/sentient/whatever being and you claim that it's still not i/c/s/w, then you've implicitly accepted the possibility that all humans besides you are not i/c/s/w either.
Your tangent on color vision/hearing/pain is not relevant. None of those things are required for i/c/s/w, nor is it obvious that a machine is incapable of feeling those things.
Also you keep using the term "LLM" a bunch here but I wasn't talking about LLMs. The question of machine intelligence is much broader than whatever the current state of the art happens to be.
I initially responded to CSMastermind whose post explicitly mentioned LLMs and the Turing test, which involves interacting with a computer solely by typing and reading text. You first mentioned consciousness.
We can tell that other people are intelligent and conscious because they're like us in important ways and we are intelligent and conscious. I haven't ruled out intelligent machines in general. My point was about the inadequacy of the Turing Test.
Wayback shows #6 as "By 2010, more than 50 percent of books sold worldwide will be printed on demand at the point of sale in the form of library-quality paperbacks."
It was removed early 2011. Is this what happens when they can't decide on who won?
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
Yes, especially if you use the original adversarial setup with three simultaneous participants (judge, two candidates) as described in Turing's original paper.
However, I'm not quite sure how long even that one will last.
>https://longbets.org/6/
Anyone know why this (and many other bets) have been removed?
https://web.archive.org/web/20080803213625/https://longbets....
>By 2010, more than 50 percent of books sold worldwide will be printed on demand at the point of sale in the form of library-quality paperbacks.
Not exactly a hot-button issue.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
I think the clear winning strategy is to find some of the common sense reasoning fails that are still quite common, like the "should I walk or drive to the car wash" one from a few months ago.
These are usually from a missing context (i.e. specifying where the car is right now). Tokenizer issues (how many Xs are in Y) are a better giveaway, but models can solve them using tools nowadays. I think the best way to uncover an LLM is to reveal its superhuman aspects. Ask it for an A* algorithm for your problem. If it can just type down a working algorithm in a few seconds, that is definitely not a human. Although with some careful system prompt design, you could probably mask that aspect too.
> These are usually from a missing context (i.e. specifying where the car is right now).
That shouldn't be an issue. The very act of taking the car to car wash contains the information that it is with you and you are at your home. Doesn't this prove that LLMs don't actually think?
> These are usually from a missing context (i.e. specifying where the car is right now).
The reason for the mistake is irrelevant here. All that matters is that you catch the agent saying something that basically no human would, especially in this type of game.
If it is missing context, it will be indistinguishable from mistakes that humans could make as well.
No, the context is obvious to any human, that's the whole point. No one would think you've left your car at a car wash, as no one ever does that, and if they did, they wouldn't even consider driving there. It's obvious to any human that the question is a little trick and that the answer is extremely obvious.
Counter example: A person who always has their car brought to the car wash by someone else (could be super wealthy, could be company cars, could be a million other reasons...) is asked whether to go or drive to the car wash. Of course they'll tell you to just walk if it's close anyway. To them the car is already there implicitly. Human life experience is not universal. And so context can't be either.
In a true competitive Turing test, these reasoning pitfalls could be included in the system prompt, there are not that many of them. The tester would need to find a novel reasoning pitfall, but those aren't uncommon among humans either, so that would not make a very useful differentiator.
Another approach could be to probe the helpfulness of an AI. A human would likely not comply with a request to write a sonnet about tangerines.
I don't understand what you mean that "there are not that many of them". The models, even the best ones, still have clear problems with certain kinds of real world reasoning, and it's not a known, closed set. With some prodding and knowing this error mode, I think it's not hard to trip any model with a novel reasoning error like this.
And no, I don't think humans would make the same errors, the key reason for this failure mode is that there are some facts of human day to day life that are so mundane that they are virtually never written about, thus making them an ideal combination of universal human experiences but out of distribution for the LLMs.
>You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
While LLMs are very capable of appearing human in general conversation, the test is that they are trying to convince a human that they are human, which is not something that is in their training data and I suspect they would go about things in very odd ways from an actual person, especially when challenged by an actual person. It is still very easy to back LLMs into a corner where they fall into very unnatural patterns and in my experience challenging them on their "humanness" (or anything) backs them into that corner.
LLMs also still can't use context or subtext in natural ways and even people who are terrible with purposefully exploiting context and subtext, generally have a natural ability that they developed to use and parse them. In a short conversation that is not adversarial, this is not an issue but falls apart as the length of the conversation grows or they start getting challenged.
LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump. If we are put into the situation of proving we are human (in an academic test sort of way) we will play with it and have fun with it, and we will find ourselves on that cliff looking into the void, debating if we should expose ourselves and admit things we normally wouldn't. We pick up on this when people do it, even over the internet, their way of stating things changes and their subtext falls away or drastically changes; I have never seen anything approaching this in LLMs.
I don't think any frontier lab tried to make an LLM optimized for the literal Turing test where the LLM is made to argue that it's human. This is not in their interest. They make their LLMs insist they aren't human, have no feelings and sentience etc etc.
So we don't actually know how good LLMs can be in this.
That is absolutely true of the chatbots we have access to but I suspect there are people working towards these ends, including all the big names in the field, major win for their company if they manage it. But even then, I have yet to see a model that could naturally handle subtext and context, they struggle even when dealing with programming languages where these things are far simpler than they are in human language.
There is also the problem with how easy it is to back them into a corner where they start acting in very unnatural ways. This would arise when challenging them on their humanity and most people would pick up on it and craft their challenges around it until it fell apart because that is how people work in such adversarial situations.
I'm agnostic on the matter because post training and RLHF on this specific capability could be quite effective. Nobody ever tried because it would cause a backlash and bad PR. Much better PR to solve math problems, help run businesses, cure disease etc than to pretend it's human. The only business case for pretend human is AI boy/girlfriends but those are also not most profitable at their most realistic state.
> LLMs just don't play with ideas and conversation like we do, they don't riff on it and go with the flow without care, picking up on subtexts and contexts and manipulating them the way people constantly do without effort until they find themselves on the edge of a cliff looking into the void and debating if they should jump.
Sounds like Neuro-sama to me.
By 2029 things will be different. Humans will have learned to talk like LLMs.
The Turing test has been passed. The Turing test had already been passed in 2002. The Turing test isn't about some machine fooling all humans all the time. It's about fooling some humans some of the time. That's an extremely low bar. It has been cleared. Let's put this to rest.
The bar you mentioned was reached in the 1960s with ELIZA. I have no idea why the obsession with the test has persisted into modern generations. It was some guy's thought experiment from 1950. It's not scientific in any fashion. Passing it doesn't prove anything. It's like some kind of authority worship game where people invoke the name of Turing as though doing so bolsters the strength of their arguments in any way whatsoever.
> You'd think that LLMs fulfill this, but I do wonder if a clever human could still discern between them given their particularities.
You don't even have to be that clever. We've all tried using it for work. They fail.
In the first place, the Turing test was always a loosely defined thought experiment. I suppose it's still serving that purpose, but at least half of HN takes it way too seriously. It's absolutely not proof of <engagementbait> endorsed by Alan Turing himself.
I think if you trained an LLM specifically for the purpose of passing the Turing test (instead of being helpful, harmful, and so on), its likely it would pass it.
You would have to train it/finetune it on a couple hundred of 'humans chatting in the context of a Turing test'.
I don't think that's true, assuming that the humans are also allowed to strategize and study the problem beforehand. The main weakness of the current models is that they still fail hard at certain kinds of common sense reasoning about real world situations, that no human ever would. Things like the "walk or drive to a car wash that's 2 minutes away" thing from a few months ago (I think the latest models have patched that one in particular, but I'm sure others can be found in a similar vein).
That would not really pass the test in spirit, I think. Passing must be something that falls out as a consequence of being a good chatbot, not the other way around.
However such good conversational agents would still have benefits, and i would be curious how far the approach of building a model that’s nice to talk to can take you, as opposed to the current generation of all-knowledgable helpful assistants.
I think you are absolutely right.
The big problem with the Turing test always was that it doesn't take into account adversarial designs. They had those chatbot contests about a decade ago, where the chatbots would regularly pass the turing test, not because the bots were intelligent, but because they were packed with rhetorical tricks designed to avoid saying anything of substance.
By any meaningful measure, today's LLM's pass the Turing test. Except we may need to lobotomize them for the deception to work.
I'd like to see an honest attempt at that.
What's more interesting are the types of humans that might fail the Turing test. Maybe to discourage the failure of real humans, there could be consequences outside the context of the test.
That is, most people accused of being a computer would crack and start pleading their humanity, but some sociopaths might not. I then wonder how many of them overlap with those so invested in abusing the premise of the Turing test.
It really is a fun thought experiment when you spice it up enough.
Mannerisms aside, they are easy to spot from having inhuman amounts of trivia knowledge
Couldn’t you instruct them to not display as much knowledge for a test situation…?
Humanity is taking an interesting technological arc.
Terminator (1984 film) had a scene showing that in a future 2009, humans would use dogs to try to sniff out whether a robot passing for a human is secretly a machine.[1]
In our real world 2026, there are no humanoid machines that can complete basic generic tasks, like carrying a tray across the stage and holding it for 30 seconds.[2] They move slowly and badly (probably from an LLM like neural network doing very few frames per second of correction and analysis), and are nowhere near lifelike.
When they don't need a body to pass for a human, such as typing online, they do a bit better.
We can tell them apart from humans. As you say, they have stylistic quirks. And you mention that they're easy to spot because they come trained with inhuman amounts of trivial knowledge.
[1] https://www.reddit.com/r/MovieDetails/s/0qUCVPYgjt
[2] https://www.reddit.com/r/LivestreamFail/s/t6ZV0yhgEe
I find it more accurate to refer to "A Turing Test" as opposed to "The Turing Test" for that reason. It's just one loosely-defined test; necessary but not sufficient.
It won't fool all of the people all of the time. But it succeeds often enough to pass the loose definition. It does many things that we called "AI complete" for decades.
It is also clear that is is also not really aware, either. I think Turing would be tickled that we find ourselves in an intermediate state that he would not have imagined.
I would not call Turing's experiment ill-specified, on the contrary among other factors the paper the game is introduced in earnestly calls for a telepathy-proof room to ensure no side channel leakage. The 1950s was an interesting decade.
> entering the characters http://www.longbets.org/601 into the address bar of a web browser or command line tool (like curl)
The part of this that's most likely to fail at some point is the "http://".
In my mind, the part that is most likely to fail is "return an HTML document that still contains the following text." Nowadays, more and more "HTML documents" do not contain any content at all, just some JavaScript that is responsible for then fetching and displaying the content.
If the text was returned using a staged process, whereby the server first returns an instruction (JavaScript), and the web browser then executes that instruction, which is what transfers the text, then that doesn't actually change anything. Those are just technical details of the transfer. You are viewing an HTML document on-screen; it makes no difference to this bet if it was returned to you by a different mechanism than originally conceived, since the bet doesn't specify any transfer mechanism, just that the URL was entered into a browser.
I understand that this isn't entirely clear-cut and that people will interpret it differently, but in my opinion, there is a significant difference between "a webpage that contains text X" and "an HTML document that contains text X." "Webpage" and "HTML document" are not synonyms - "HTML document" has a much narrower meaning.
> or command line tool (like curl)
Even if you interpreted the bet generously to allow for a browser to render the content using JavaScript, if I understand the bet correctly, the condition of working with a command-line tool like curl would fail.
I read the or's as giving alternatives, only one of which needs to work.
I can see how other interpretations are possible, but I don't see anything especially generous about mine. I'm not "allowing" anything: To make a judgement you have to pick a web browser to try it with, and you'd probably pick the most popular one, in the most popular configuration, that seems the most fair. If that web browser happens to execute Javascript when you navigate to an URL, then that's just how it is.
It says that the browser must download "an HTML document that still contains the following text". I can see how either interpretation could work, but I would interpret "contains" as "the text is in the HTML document", not "rendering the HTML document in a JavaScript-capable browser displays the text".
(Though, that said, some approaches to rendering sites via JavaScript would still have that text in the initially downloaded HTML file as JavaScript data.)
As for the "or", I read it as "doing either X or Y must", not "either X must or Y must (not necessarily both)", but yeah, I can see either interpretation being valid.
That's a secondary ("OR") condition, used if the browser condition isn't satisfied.
You two have just spelled out a very specific bet. Care to put a wager on it? :)
I used to be someone, man, I was a dreamer, a programmer, a man!
Now I'm a human slave copying text from web browsers to chat windows to get around JS rendering bullshit.
The preferred term is meat proxy ;)
That depends on which side of the JS rendering bullshit you're sitting. If you're on the serving side, yes.
But I do the same as GP, just on the receiving side. That is, reaching out for help to baby eldritch entity, because that's healthier for my mind than dealing directly with the unspeakable horrors that half the websites are today. The industry has lost its collective mind in pursue of profit and control, and that was long before the thousand-mask wearing shoggoths became our daily consorts.
I like your funny words, magic man.
One of those things is true
The detailed terms of the bet include this:
> A 301 redirect from www.longbets.org/601 to a different URL containing that text would also fulfill those conditions.
I assume this would cover the HTTP->HTTPS redirect scenario.
I think the parent is referring to the possibility that one day, browsers might put up a blanket security warning for HTTP URLs instead of following redirects. Or they might try to be a little too clever with HTTPS upgrades.
In particular, the automatic upgrade feature in modern browsers is based on several heuristics rather than explicit configuration like HSTS, so there's always a bit of room for breakage there. For example, they don't even check if the server returns a 301 redirect, which can be problematic if the server wants to redirect to HTTPS on a different host/port or make some changes to the path.
> Or they might try to be a little too clever with HTTPS upgrades.
They're not going to get any cleverer than they are now. In October we'll finally be done with heuristics or central databses of https-first websites used by the various browsers or the half a dozen of headers that you must dance around to get the upgrade to work securely and reliably. They will just default to https first.
https://blog.google/security/https-by-defau/
Yes this is about Chrome, but all others will follow.
Personally, if I were interpreting the bet, and that browser behavior had been changed before the day of the bet, I'd interpret that as failure, because the browser didn't show an HTML page, it showed a warning dialogue.
And in any case, at some point after browsers start doing that, I will be unsurprised if some further conditions prevail that provide reason for web servers to stop listening on http at all.
Yeah that was a clever bit of foresight there.
At a certain bet size it would almost be self-fulfilling, heh, as one side would be very incentivized to keep at least that url working (buying the domain + hosting it if the page were to die)
Someone should do a long bet on the continuous existence of longbets.com
One thing I've done is include my redirect rules with the web application. So my links continue to work even when I change them and forget them.
https://news.ycombinator.com/item?id=46139686
I was expecting some kind of system that would follow the pages automatically without you having to do extra work when changing the URL (“continue to work even when I change them and forget them”) but it looks like you still add the redirects manually?
That’s more than most will do, sure, but it doesn’t seem like it saves you from a broken link if you forget to add a specific redirect or miss it during a mass format change. Am I missing something?
Amazing that Disqus comments are still live for this page. More impressive even than the website itself being live.
I really liked disqus back in the day. When I was in my kinda cringey singularity phase, Kurzweil and all that. Tracking comments across similar blogs with the same people was handy.
I'm grateful the comments are still there, since they reminded me to get back in touch with my old friend Craig Hubley!
Some nice posts about his prediction on Jeremy Keith's blog:
https://adactio.com/journal/tags/longbets
Wow, that could be a post of it’s own
Anyway to find all bets expiring this year? it will be interesting to see. Could not find a way to search for it there.
I'm not sure how it played out in 2011, but as of now it looks like the win condition hinges on the redirect.
For the heck of it I confirmed "http://www.longbets.org/601" does in fact result in a 301 redirect to "https://longbets.org/601/". I suspected they might have used a 302.
This is basically a bet on if you think longbets is going to be around.
I like the comment on the site about blockchains replacing HTML links (from 4 years ago).
Ugh, I really hate SSL. This is a simple text page, why couldn't it live where it was?
Yes, I'm an old fart, yelling at the internet clouds. You'll get here some day too, if you're lucky.I am quite impressed that the bet included above redirection as part of the terms.
It's all a matter of whether someone is willing to pay for the hosting.
Which is not, in my opinion, how things should be hosted on the Web.
Freenet had a different approach, even back in the early days (two decades ago). I interviewed Ian Clarke about it https://www.youtube.com/watch?v=JWrRqUkJpMQ
I wrote about it for years: https://community.intercoin.app/t/who-pays-for-storage-nfts-...
And recently I built Safecloud to be different: https://safebots.github.io/Safecloud/
On HN, a TLDR would be helpful, because just clicking your links would potentially mean falling for shilling of ones own side-hustle.
I bet there's complexity to the topic, but I am sure you could put it into a few words instead of just leaving a hook for others to click on.
I know nothing of Safecloud, but freenet failed for a very valid reasons:
1. Data retention isn't infinite. If the source disappears from the network, peers will prune it shortly. This is exponentially true in our multimedia-heavy world.
2. Despite 1, if you actually want to delete something you published, you can't. Absolutists will see this as a good thing. Normal people will see this as a problem.
3. Peer-to-peer doesn't work in a mobile-first world, and not enough people will host nodes as a hobby. If all website publishers/hosters have to host a node, then we're just back to the current model but with extra steps.
Closed-garden mobile-first tech feudalism lobotimized the Internet. I remember trying out the Freenets, I2Ps, and whatnot in the 90's. Too cumbersome nerdy tools to use for anything actually worthwhile. It was born in the idealized world village of the 90's Internet. Maybe real life doesn't work like that, and the lobotimized centralized tech will eventually win.
Is the p2p problem really still one in our world that is mobile first, but also has ubiquitous internet connectivity pretty much everywhere? I’m almost sure I could host my website from my phone and have at least 99.5% percent of uptime.
Give that it's impossible to even listen on port 443 on a phone, as that would require root privileges, I don't think you can host your website on your phone at all, at least not if you want it to accessible in a normal way. And that's not even going into all of the problems with running background apps on a phone, especially on iOS.
By I could, I meant that my phone has a persistent connection uninterruptedly long enough to be theoretically able to serve a website in the background. There are all sorts of (arbitrary, mind you) technical limitations, from the IPv6 address randomly changing as you roam across networks to iOS killing background processes - but those are software limitations.
The point was that mobile device doesn't equate spotty connectivity anymore for most people.
Even in this idea of "could in principle", the IP address randomly changing as you roam is not an arbitrary technical limitation, it is a fundamental property of how IP networks operate that proved to be too hard to fix for IPv6. The LTE network does a lot of work to try to preserve your IP as seen by the servers you connect to, to preserve your active connections, but this could only be done as a layer on top of IP.
The reason why this is fundamentally quite hard is to do with routing optimization. Routing tables have to reflect physical connectivity - which means packets destined for F2::67 mustn't be routed to LA if that is the IP of a phone that is now resident in SF. However, changing routing tables on all devices along the way as a phone roams would not be an easy feat.