One of those research areas where nothing they currently do is unethical, but also nothing they currently do is useful, and right about the point where it might ever become useful is also the point where it becomes hugely unethical. So we'd do well to outlaw it right now, but seems like we don't really have a moral framework for cases like this.
I don't see why we should outlaw this. There's so many potential insights to be gleaned from research like this that you just can't predict.
And making something like this illegal often has unintended side effects. What if brain organoid research is what ends up curing alzheimer's?
Advocating for outlawing parts of research just on the basis that in the future they could potentially turn unethical is very shortsighted in my opinion.
Sure, and you could say the same thing about unit 731. The argument is not that it is inconceivable that there are benefits, but that they should be weighed against the drawbacks, which you don't mention at all.
If anything, my argument would be more "just because unit 731 ended up doing live vivisections of humans, that doesn't mean we should have outlawed dissection of dead humans centuries earlier".
I don't think it's a good idea at all to try to outlaw totally reasonable research just because we can imagine a future where people end up doing a much more extreme much more problematic similar thing.
It would be the end of science. (and the end of any culture really... we're very good at imagining future dystopias and armageddons)
It's just an extension of Dario's brand of brainrot where consciousness has an almost religious existence that just pops out if you have a large enough system.
No proof is required beyond psuedoscientific tests and the "saints" of this belief system must be given absolute authority over the system to ensure no one commits any sins.
Is it really so wacky to suggest we might want to be careful with bootstrapping complex computation from human neurons? It's quite different from an LLM and the fact that it's not made out of neurons seems to be one of the core arguments for opponents of the possibility of LLM consciousness...
>that just pops out if you have a large enough system.
certainly not a claim I would make.
>No proof is required
Isn't it legitimate to reason on a more abstract level and try to make inferences that way?
>psuedoscientific tests
no clue what you're referring to here or in the rest of your comments about saints and authorities and sins. Another parallel to religion could be drawn w.r.t. the idea that only human souls could ever feel suffering but I don't think it's such a productive level to have this conversation on.
I may have overreacted, but jumping from "we should probably ban this research preemptively because it might pose difficult questions later" to comparisons to Unit 731 when someone disagreed, reminded me of the way Anthropic pushes their belief that Claude is conscious onto everyone with "evidence" of questionable value and even when there are many questions they haven't bothered to answer about such a claim. Plus, they do this despite various features we generally associate with conscious experience that current gen LLMs lack.
Given the complexity of behavior observed in nature that doesn't require conscious experience, yet is more advanced than what we can achieve in circuitry, I think it would be too early to close off this area of research.
> you must acknowledge consciousness can arise in an agglomeration of human brain cells
That doesn't follow. You could, for example, be a substance dualist. Alternatively you could suspect that there exists some special as yet undocumented property of the human CNS that is required as a prerequisite. Or etc.
Consciousness can arise from an agglomeration of molecules. How sure are you that empty vats (which are made of molecules) are not aware and suffering?
What's going on here? Answer: Fallacy of affirmation of the consequent.
Certainly it wouldn't be the end of science, we could still do all the other science that doesn't involve trying to massage human neurons into doing complex computations.
I want to be clear: I hate this. I hate everything about it. HOWEVER...
If you want to argue against wetware then we need a better argument than this. The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives. Unit 731 isn't an apt comparison because cells are not inherently sentient to our knowledge. If they were it would absolutely shatter the profession of medical ethics as we know it.
I honestly don't know if better arguments are needed. I think most people simply have no idea that this research is happening right now, and if they did the vast majority would be against it. Feels fundamentally different from stem cell or IVF.
We know that human brain cells in sufficiently large and complex assemblages are sentient (I don't know if the word "inherently" has any purpose here). We know this research aims at making the assemblages larger and more complex. It's a fundamentally different kind of argument IMO compared to those against e.g. stem cells or IVF.
(Slight aside but did you know that they're also doing tests on whole human brains taken from corpses, and they give them tranquilizers "just to be sure"? https://news.ycombinator.com/item?id=48212992)
> We know that human brain cells in sufficiently large and complex assemblages are sentient
No, we know that human brain cells in specific configurations honed by billions of years of evolution and in specific states are sentient. (The same collection of cells in a slightly different state or a different arrangement might be permanently comatose).
This argument is no different in kind than the claim that Mamdani and El-Sayed are terrorists: it's a fallacy of affirmation of the consequent.
How do you know your fingernails aren't sentient? Better stop clipping them immediately!
> i want to know about proof
The epistemology of empirical science is inference to the best explanation, not proof. As pointed out by Hume centuries ago, no empirical generalization can be proven.
> The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives.
You write as if objections to (certain kinds of) stem cell research, IVF, etc. have been roundly defeated in some way simply because they've "saved and helped lives". This is naked utilitarianism and I hope I don't have to rehearse the reasons why utilitarianism fails utterly as moral theory.
If an experiment could theoretically cure a disease, how unethical can it be before we outlaw it? Maybe it's better to simply discuss the line in the sand, rather than to pretend this is one one side of something not-yet-defined.
You could do that if you knew enough about how it works in detail. In practice they're just gonna make as big a blob as they can muster and apply RL on it or something, and nobody will be any the wiser as to what the thing's feeling.
So many questions! Since we still don’t understand consciousness how can we draw a line to stay on the right side of? Is consciousness correlated with intelligence or independent of it? What if we keep growing these organoids until they are near-human-brain sized. We know that such things can host a consciousness yet not always eg. “brain dead”.
What if the only reason we don’t believe they are conscious is because they have no mouth to scream with?
Also, it occurs to me that a right-to-lifer would have to be against such biotech to remain consistent.
I find it all incredibly fascinating but ethically highly fraught.
Terrifying. I just don't see how this doesn't inevitably lead to brain-scale organoids. The ethics of it seem impossible. Given how we treat non-human animals this reminds me of the torment nexus meme.
If a device made of human brain cells were designed to function like a traditional, deterministic CPU, that would raise similar questions as LLMs raise today.
No, not at all. For one thing, they said "made of brain cells", not brains--those are two very different things. (Yes, brains are made of brain cells, but that's a fallacy of affirmation of the consequent.)
Computers made of transistors were designed to function like computers made of vacuum tubes without negating the benefits of transistors.
I get the sense that there is a school of thought I’d call “biological exceptionalism”, a belief that somehow constructs made up of many functionally simple units, is fundamentally different and special compared to a similar construct composed of silicon units. I can’t see how someone could hold this belief and be a materialist (I’m not implying you are btw). Biological exceptionalism surely implies the existence of a “soul” or “spirit” (which raises a whole bunch more questions).
I don't think it'd have to imply a soul - one of these "functionally simple units" is much less well-understood than the other, and you can insert whatever belief you want to fill the gaps.
Maybe a materialist biological exceptionalism position could be that biological substrates are capable of sending kinds of signals that silicon substrates are not, in the sense that brain matter especially might give rise to far greater complex processing in ways that we don't know about or appreciate yet, and which we may discover can't be efficiently recreated in silicon?
I don't know if I buy that argument or even if it makes sense. But yeah this all smacks inextricably of soul talk.
I agree that as per current understanding, the functional biological base units have more degrees of freedom and a greater functional diversity as a result, but that’s merely a quantitative difference not a fundamental distinction. One could imagine equally more complex silicon base units beyond what we have now. Whether they need to follow the same pattern/modes as the biological units remains to be seen (but it’s pretty hard to argue against evolution as an efficiency maximiser).
I think it goes the opposite way, starting with a soul and arriving at biological exceptionalism: You start by saying "I have a soul / spirit". Then you extrapolate "things like me probably have a soul / spirit". But since there is no theory of souls, the extrapolation is not based on principles but rather on vibes and guesswork, ending up somewhere like "there must be something weird going on with neurons that somehow create a soul, connect with an existing one or something like that.. maybe.."
Which is an interesting argument because: do neurons somehow create a soul (as you mentioned), or are do neurons somehow attract souls, or do souls choose what to inhabit. In the first case it posits that a “soul” is purely a function of biology. In the others that there’s some sort of inextricable link between the two (which in the absence of any theory of soul or how these are linked to biology, seems rather far fetched, unless of course souls could choose to inhabit anything with sufficient degrees of freedom: a brain, a mega computer, a planet, a galaxy)
It would do no such thing. LLMs are neither living nor sentient nor intelligent. The fallacy of computationalism is a failure to read a coarse metaphor as merely analogical. Any correspondence is superficial and limited, and even then, merely correspondence, not identity.
So, the use of human brain cells is different territory. Bioethically speaking, it is illicit to press human body parts (which includes cells) into service for technological ends. They can only licitly enter into service of the integral human whole (such a life-saving transplants). Using them in technology also commoditizes the human body. (The reverse is not the case: it can be licit to use cybernetic devices to restore native function within a human body.)
I wondered if anyone had written about the ethics of this type of research, they have. It's super long so I've not finished it yet, but it generally seems the standard scientific assessment is that current organoids very likely lack the organization needed for "sentience", but, that methods for detecting subjective experience in organoids are not at all well established.
"The researchers can’t yet tell why three organoids performed better than just one; it could be because individual variation between the organoids means that they complement each other."
i know ofc that for them 'knowing' means measurements and proofs etc. but i found this excerpt tickle me a bit and wanted to highlight it :').
my first thought was a bit tongue in cheek:
"who knew, working together with others that think alike is better than working alone"
Cant wait for biosofts and the ability to jack-in instead of jack-off :D
However, when engineers scale these chips toward cortex-level complexity, projected power climbs into the kilowatt-megawatt bracket, whereas the human brain performs comparable computation on roughly 20 watts (Mehonic and Kenyon, 2022). Biocomputing has been proposed as one route to address energy limitations in silicon-based systems by using brain organoids grown on multielectrode arrays.
All our experiments were conducted using FinalSpark's NeuroPlatform (Jordan et al., 2024), providing access to organoids over a cloud-based system, as shown in Figure 1. In this setup, forebrain organoids are cultured to approximately 500 μm in diameter (Govindan et al., 2021) and contain mature neurons, astrocytes, and oligodendrocytes displaying robust network activity. Maintained under continuous perfusion, they can remain electrophysiologically active for up to approximately 100 days under favorable conditions
This is absolutely fucked. The original argument for these human brain organoids were they're only 10k neurons. Now five minutes later they're talking about energy savings in scaling to cortex-level human brain cloud API.
"Head cheese" from Watts' "Starfish", the beginning.
One of those research areas where nothing they currently do is unethical, but also nothing they currently do is useful, and right about the point where it might ever become useful is also the point where it becomes hugely unethical. So we'd do well to outlaw it right now, but seems like we don't really have a moral framework for cases like this.
I don't see why we should outlaw this. There's so many potential insights to be gleaned from research like this that you just can't predict.
And making something like this illegal often has unintended side effects. What if brain organoid research is what ends up curing alzheimer's?
Advocating for outlawing parts of research just on the basis that in the future they could potentially turn unethical is very shortsighted in my opinion.
Sure, and you could say the same thing about unit 731. The argument is not that it is inconceivable that there are benefits, but that they should be weighed against the drawbacks, which you don't mention at all.
How is unit 731 an apt comparison here?
Those were unethical experiments from the onset.
If anything, my argument would be more "just because unit 731 ended up doing live vivisections of humans, that doesn't mean we should have outlawed dissection of dead humans centuries earlier".
I don't think it's a good idea at all to try to outlaw totally reasonable research just because we can imagine a future where people end up doing a much more extreme much more problematic similar thing.
It would be the end of science. (and the end of any culture really... we're very good at imagining future dystopias and armageddons)
It's just an extension of Dario's brand of brainrot where consciousness has an almost religious existence that just pops out if you have a large enough system.
No proof is required beyond psuedoscientific tests and the "saints" of this belief system must be given absolute authority over the system to ensure no one commits any sins.
Is it really so wacky to suggest we might want to be careful with bootstrapping complex computation from human neurons? It's quite different from an LLM and the fact that it's not made out of neurons seems to be one of the core arguments for opponents of the possibility of LLM consciousness...
>that just pops out if you have a large enough system.
certainly not a claim I would make.
>No proof is required
Isn't it legitimate to reason on a more abstract level and try to make inferences that way?
>psuedoscientific tests
no clue what you're referring to here or in the rest of your comments about saints and authorities and sins. Another parallel to religion could be drawn w.r.t. the idea that only human souls could ever feel suffering but I don't think it's such a productive level to have this conversation on.
I may have overreacted, but jumping from "we should probably ban this research preemptively because it might pose difficult questions later" to comparisons to Unit 731 when someone disagreed, reminded me of the way Anthropic pushes their belief that Claude is conscious onto everyone with "evidence" of questionable value and even when there are many questions they haven't bothered to answer about such a claim. Plus, they do this despite various features we generally associate with conscious experience that current gen LLMs lack.
Given the complexity of behavior observed in nature that doesn't require conscious experience, yet is more advanced than what we can achieve in circuitry, I think it would be too early to close off this area of research.
If you believe you are conscious, you must acknowledge consciousness can arise in an agglomeration of human brain cells.
How sure are you that the human brain cells in the vat are not aware and suffering?
> you must acknowledge consciousness can arise in an agglomeration of human brain cells
That doesn't follow. You could, for example, be a substance dualist. Alternatively you could suspect that there exists some special as yet undocumented property of the human CNS that is required as a prerequisite. Or etc.
Consciousness can arise from an agglomeration of molecules. How sure are you that empty vats (which are made of molecules) are not aware and suffering?
What's going on here? Answer: Fallacy of affirmation of the consequent.
Certainly it wouldn't be the end of science, we could still do all the other science that doesn't involve trying to massage human neurons into doing complex computations.
I want to be clear: I hate this. I hate everything about it. HOWEVER...
If you want to argue against wetware then we need a better argument than this. The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives. Unit 731 isn't an apt comparison because cells are not inherently sentient to our knowledge. If they were it would absolutely shatter the profession of medical ethics as we know it.
I honestly don't know if better arguments are needed. I think most people simply have no idea that this research is happening right now, and if they did the vast majority would be against it. Feels fundamentally different from stem cell or IVF.
We know that human brain cells in sufficiently large and complex assemblages are sentient (I don't know if the word "inherently" has any purpose here). We know this research aims at making the assemblages larger and more complex. It's a fundamentally different kind of argument IMO compared to those against e.g. stem cells or IVF.
(Slight aside but did you know that they're also doing tests on whole human brains taken from corpses, and they give them tranquilizers "just to be sure"? https://news.ycombinator.com/item?id=48212992)
> We know that human brain cells in sufficiently large and complex assemblages are sentient
No, we know that human brain cells in specific configurations honed by billions of years of evolution and in specific states are sentient. (The same collection of cells in a slightly different state or a different arrangement might be permanently comatose).
This argument is no different in kind than the claim that Mamdani and El-Sayed are terrorists: it's a fallacy of affirmation of the consequent.
how do u know cells are not sentient to our knowledge? (i am serious, not pulling ur strings. i want to know about proof)
How do you know your fingernails aren't sentient? Better stop clipping them immediately!
> i want to know about proof
The epistemology of empirical science is inference to the best explanation, not proof. As pointed out by Hume centuries ago, no empirical generalization can be proven.
> The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives.
You write as if objections to (certain kinds of) stem cell research, IVF, etc. have been roundly defeated in some way simply because they've "saved and helped lives". This is naked utilitarianism and I hope I don't have to rehearse the reasons why utilitarianism fails utterly as moral theory.
If an experiment could theoretically cure a disease, how unethical can it be before we outlaw it? Maybe it's better to simply discuss the line in the sand, rather than to pretend this is one one side of something not-yet-defined.
You can probably do a lot of interesting stuff without giving the blob any capacity of having a consciousness or higher intelligence.
You could do that if you knew enough about how it works in detail. In practice they're just gonna make as big a blob as they can muster and apply RL on it or something, and nobody will be any the wiser as to what the thing's feeling.
Neuroscientists don't share your (and other people here like phys318u below) level of ignorance about how brains work.
So many questions! Since we still don’t understand consciousness how can we draw a line to stay on the right side of? Is consciousness correlated with intelligence or independent of it? What if we keep growing these organoids until they are near-human-brain sized. We know that such things can host a consciousness yet not always eg. “brain dead”.
What if the only reason we don’t believe they are conscious is because they have no mouth to scream with?
Also, it occurs to me that a right-to-lifer would have to be against such biotech to remain consistent.
I find it all incredibly fascinating but ethically highly fraught.
Radiolab had a good episode on brain organoids - https://radiolab.org/podcast/brain-balls
Yeah folks are commenting without understanding organioids. This episode is really educational and helps explain it. Really exciting development
Terrifying. I just don't see how this doesn't inevitably lead to brain-scale organoids. The ethics of it seem impossible. Given how we treat non-human animals this reminds me of the torment nexus meme.
If a device made of human brain cells were designed to function like a traditional, deterministic CPU, that would raise similar questions as LLMs raise today.
This seems reductionist with false equivalence. Wouldn't making a brain a "deterministic CPU" negate all the benefits of using organic tissue?
Probably, but the questions I was thinking of are whether it's the design or the substrate that we consider significant.
The question comes up a lot here when there are posts about whether to consider LLMs conscious.
"We're going to condemn conscious entities to slavery"
vs
"They'll always just be machines and it's dehumanizing to consider them anything else"
No, not at all. For one thing, they said "made of brain cells", not brains--those are two very different things. (Yes, brains are made of brain cells, but that's a fallacy of affirmation of the consequent.)
Computers made of transistors were designed to function like computers made of vacuum tubes without negating the benefits of transistors.
Not if the organic tissue is faster or more energy efficient.
> that would raise similar questions as LLMs raise today
I think the questions would be very much different. LLMs are not made of human brain cells.
I get the sense that there is a school of thought I’d call “biological exceptionalism”, a belief that somehow constructs made up of many functionally simple units, is fundamentally different and special compared to a similar construct composed of silicon units. I can’t see how someone could hold this belief and be a materialist (I’m not implying you are btw). Biological exceptionalism surely implies the existence of a “soul” or “spirit” (which raises a whole bunch more questions).
I don't think it'd have to imply a soul - one of these "functionally simple units" is much less well-understood than the other, and you can insert whatever belief you want to fill the gaps.
Maybe a materialist biological exceptionalism position could be that biological substrates are capable of sending kinds of signals that silicon substrates are not, in the sense that brain matter especially might give rise to far greater complex processing in ways that we don't know about or appreciate yet, and which we may discover can't be efficiently recreated in silicon?
I don't know if I buy that argument or even if it makes sense. But yeah this all smacks inextricably of soul talk.
I agree that as per current understanding, the functional biological base units have more degrees of freedom and a greater functional diversity as a result, but that’s merely a quantitative difference not a fundamental distinction. One could imagine equally more complex silicon base units beyond what we have now. Whether they need to follow the same pattern/modes as the biological units remains to be seen (but it’s pretty hard to argue against evolution as an efficiency maximiser).
I think it goes the opposite way, starting with a soul and arriving at biological exceptionalism: You start by saying "I have a soul / spirit". Then you extrapolate "things like me probably have a soul / spirit". But since there is no theory of souls, the extrapolation is not based on principles but rather on vibes and guesswork, ending up somewhere like "there must be something weird going on with neurons that somehow create a soul, connect with an existing one or something like that.. maybe.."
Which is an interesting argument because: do neurons somehow create a soul (as you mentioned), or are do neurons somehow attract souls, or do souls choose what to inhabit. In the first case it posits that a “soul” is purely a function of biology. In the others that there’s some sort of inextricable link between the two (which in the absence of any theory of soul or how these are linked to biology, seems rather far fetched, unless of course souls could choose to inhabit anything with sufficient degrees of freedom: a brain, a mega computer, a planet, a galaxy)
It would do no such thing. LLMs are neither living nor sentient nor intelligent. The fallacy of computationalism is a failure to read a coarse metaphor as merely analogical. Any correspondence is superficial and limited, and even then, merely correspondence, not identity.
So, the use of human brain cells is different territory. Bioethically speaking, it is illicit to press human body parts (which includes cells) into service for technological ends. They can only licitly enter into service of the integral human whole (such a life-saving transplants). Using them in technology also commoditizes the human body. (The reverse is not the case: it can be licit to use cybernetic devices to restore native function within a human body.)
Maybe the real monsters were the friends we made along the way.
Ethics and Regulation of Human Brain Organoid Research: Recommendations from the Asia Pacific Neuroethics Working Group - https://link.springer.com/article/10.1007/s41649-025-00398-6
I wondered if anyone had written about the ethics of this type of research, they have. It's super long so I've not finished it yet, but it generally seems the standard scientific assessment is that current organoids very likely lack the organization needed for "sentience", but, that methods for detecting subjective experience in organoids are not at all well established.
"The researchers can’t yet tell why three organoids performed better than just one; it could be because individual variation between the organoids means that they complement each other."
i know ofc that for them 'knowing' means measurements and proofs etc. but i found this excerpt tickle me a bit and wanted to highlight it :').
my first thought was a bit tongue in cheek: "who knew, working together with others that think alike is better than working alone"
Cant wait for biosofts and the ability to jack-in instead of jack-off :D
This is terrifying. Also it isn't new: https://research.ufl.edu/publications/explore/v10n1/extract2...
From 2004.
I can't help but think of "I have no mouth, and I must scream"
Do human brain organoids provide any advantage over animal brain organoids in these types of use cases?
I have no fingers and I must read Braille. (But I really want to scream).
Someone call Stuart Hameroff and Anirban.
who is asking for biohybrid computing? this is how we get on people's bad list, can we just get quantum
However, when engineers scale these chips toward cortex-level complexity, projected power climbs into the kilowatt-megawatt bracket, whereas the human brain performs comparable computation on roughly 20 watts (Mehonic and Kenyon, 2022). Biocomputing has been proposed as one route to address energy limitations in silicon-based systems by using brain organoids grown on multielectrode arrays.
All our experiments were conducted using FinalSpark's NeuroPlatform (Jordan et al., 2024), providing access to organoids over a cloud-based system, as shown in Figure 1. In this setup, forebrain organoids are cultured to approximately 500 μm in diameter (Govindan et al., 2021) and contain mature neurons, astrocytes, and oligodendrocytes displaying robust network activity. Maintained under continuous perfusion, they can remain electrophysiologically active for up to approximately 100 days under favorable conditions
This is absolutely fucked. The original argument for these human brain organoids were they're only 10k neurons. Now five minutes later they're talking about energy savings in scaling to cortex-level human brain cloud API.