AI Eroded Doctors' Ability to Spot Cancer Within Months in Study

(bloomberg.com)

59 points | by zzzeek 16 hours ago ago

65 comments

  • frankc 15 hours ago ago

    But it sounds like with AI doctors did better overall, or that is how I read the first couple of lines. If that is true, I don't really see a problem here. Compilers have eroded my ability to write assembly, that is true. If compilers went away, I would get back up to speed in a few weeks.

    • jackvalentine 15 hours ago ago

      The compiler example is very helpful, thanks for posting it.

      My follow up question is now “if junior doctors are exposed to AI through their training is doctor + AI still better overall?” e.g. do doctors need to train their ‘eye’ without using AI tools to benefit from them.

      • benterix 8 hours ago ago

        Yeah everybody compares traditional programming to using assembler now. This analogy would be great except that the current generation of LLMs is still quite unreliable and a large part of what it generates is a mess.

        In a perfect world, there would be no problem - those of us who enjoy the experience would vibe-code whatever they need, and the rest of us would develop our skills the way we want. Unfortunately, there is a group of CXOs, such as the ones at Github, who know better and force the only right way down their employees throats.

        • jackvalentine 7 hours ago ago

          These radiography AIs aren’t LLMs AFAIK.

      • frankc 15 hours ago ago

        I think that is a good question and we don't really know yet. I think we are going to have to overhaul a lot of how we educate people. Even if all AI progress stops today, there is still a massive shift in how many professions operate that is incoming.

        • jackvalentine 14 hours ago ago

          > I think we are going to have to overhaul a lot of how we educate people.

          I agree.

          I work in healthcare and if you take a tech view of all the data there are a lot of really low hanging fruit to pick to make things more standardised and efficient. One example is extracting data from patient records for clinical registries.

          We are trying to automate that as much as possible but I have the nagging sense that we’re now depriving junior doctors of the opportunity to look over hundreds of records about patients treated for X to find the data and ‘get a feel’ for it. Do we now have to make sure we’re explicitly teaching something since it’s not implicitly being done anymore? Or was it a valueless exercise.

          The assumptions that we make about training on the job are all very chesterton’s fence really.

        • benterix 8 hours ago ago

          I think this is has already started a while ago with data science. While it might not be as fashionable as before, it's really at the core of many of these jobs where various form of machine learning or generative AI are being used.

    • dweinus 15 hours ago ago

      Except the paper doesn't say that the doctors + AI performed better than doctors pre-AI. It is well documented that people will trust and depend on AI, even if it is not better. It is not clear in the paper, but possible that this is just lose-lose.

      Paper link: https://www.thelancet.com/journals/langas/article/PIIS2468-1...

      • mcbrit 14 hours ago ago

        Line: The ADR of standard colonoscopy decreased significantly from 28·4% (226 of 795) before to 22·4% (145 of 648) after exposure to AI

        Supprt: Statistically speaking, on average, for each 1% increase in ADR, there is a 3% decrease in the risk of CRC. (colorectal cancer)

        My objection is all the decimal points without error bars. Freshman physics majors are beat on for not including reasonable error estimates during labs, which massively overstates how certain they should be; sophomores and juniors are beat on for propogating errors in dumb ways that massively understates how certain they should be.

        This article is up strolls rando doctor (granted: with more certs than I will ever have) with a bunch of decimal points. One decimal point, but that still looks dumb to me. What is the precision of your measuring device? Do you have a model for your measuring device? Are you quite sure that your study, given error bars, which you don't even acknowledge the existence of, don't cancel out the study?

        • dweinus an hour ago ago

          Either way, to be clear: the 28.4% -> 22.4% is human performance vs human performance (before and after "exposure to AI"). There are no numbers provided on accuracy with the use of AI.

      • itsdrewmiller 13 hours ago ago

        That is paywalled. From another article on the topic (https://bioengineer.org/study-suggests-routine-ai-use-in-col...)

        > While short-term data from randomized controlled trials consistently demonstrate that AI assistance improves detection outcomes...

    • lurking_swe 15 hours ago ago

      if the hospital IT system is temporarily down, i certainly expect my doctors to still be able to do their job. So it is a (small) problem that needs addressing.

      Perhaps a monthly session to practice their skills would be useful? So they don’t atrophe…

      • sothatsit 15 hours ago ago

        If the AI systems allow my doctor to make more accurate diagnoses, then I wouldn't want a diagnosis done without them.

        Instead, I would hope that we can engineer around the downtime. Diagnosis is not as time-critical as other medical procedures, so if a system is down temporarily they can probably wait a short amount of time. And if a system is down for longer than that, then patients could be sent to another hospital.

        That said, there might be other benefits to keeping doctors' skills up. For example, I think glitches in the diagnosis system could be picked up by doctors if they are double-checking the results. But if they are relying on the AI system exclusively, then unusual cases or glitches could result in bad diagnoses that could otherwise have been avoided.

      • randcraw 15 hours ago ago

        Or, each time the software is updated or replaced, the human in the loop will be unable to... be in the loop. Patients will be entirely dependent on software vendor corporations who will claim that their app is always correct and trustworthy. But what human (or other external unbiased trustworthy authority) will be available to check its work?

      • yread 9 hours ago ago

        If the hospital IT system is down, hospital stops. That's why hospitals spend resources so that it doesn't happen

      • oliwarner 5 hours ago ago

        A lot of imaging and note-taking is digital now, so AI aside, they probably won't meet your expectations.

      • stavros 15 hours ago ago

        Do you expect doctors to be able to image your insides if the X-ray machine is down too?

        • lurking_swe 14 hours ago ago

          An x ray machine can be used “locally” without uploading the images into the IT system. So i don’t understand the question. If it was designed to be cloud only then that would be horrendous design (IMO).

          The x ray machine would still work, it’s connected directly to a PC. A doctor can look at the image on the computer without asking some fancy cloud AI.

          A power outage on the other hand is a true worst case scenario but that’s different.

          • stavros 14 hours ago ago

            I'm not talking about the IT system, I'm talking about when the X-ray machine breaks, same as how we're talking about when the colonoscopy diagnosis machine breaks.

            • fzeroracer 14 hours ago ago

              How often do you think the x-ray machine breaks vs how often software shits the bed?

              Like one of the biggest complaints I've heard around hospital IT systems is how brittle they are because there are a million different vendors tied to each component. Every new system we add makes it more brittle.

              • d3vnull 6 hours ago ago

                Seems like a pretty easy fix in treating the system that runs the cancer detection algorithm as an hospital machine and not as part of the IT system. It can be an airgapped system that runs just the needed software in a a controlled configuration.

                This is not new, lots of mission critical software systems are run like this.

      • accoil 15 hours ago ago

        Does cancer progress that fast?

      • akoboldfrying 15 hours ago ago

        > if the hospital IT system is temporarily down

        I think we have to treat the algorithm as a medical tool here, whose maintenance will be prioritised as such. So your premise is similar to "If all the scalpels break...".

        • tylerrobinson 15 hours ago ago

          I’m think you might agree, though, that the likelihood of one premise is significantly greater than the other!

          • akoboldfrying 15 hours ago ago

            Sure. "MRI machine" would have been a better metaphor than "scalpel".

        • azemetre 15 hours ago ago

          Which is easier to build resilient systems for: the one where you have a few dozen extra scalpels in a storage closet or the one that requirements offsite backups, separate generators, constant maintenance?

          The premise is absolutely not the same.

          • akoboldfrying 15 hours ago ago

            s/scalpel/MRI machine/. How about now?

            • azemetre 4 hours ago ago

              Sounds like a great system that benefits from having lots of money. IDK how such a thing can last in rural areas where there may be one single MRI machine to use in a 100 mile radius.

            • thunderfork 14 hours ago ago

              [dead]

      • add-sub-mul-div 15 hours ago ago

        "People will practice their skills" is the new "drivers will keep their attention on the road and remain ready to quickly take the wheel from the autonomous driver in an emergency."

        • xorbax 15 hours ago ago

          It's like research. People had encyclopedias. If they wanted to know real, deep information about subjects they'd have to specifically spend effort seeking and finding books or papers about that specific subject (which are typically just distilled papers in a far wider range and number than an encyclopedia would be)

          Then we could just go Google it, and/or skim the Wikipedia page. If you wanted more details you could follow references - which just made it easier to do the first point.

          Now skills themselves will be subject to the same generalizing phenomenon as finding information.

          We have not seen information-finding become better as technology has advanced. More people are able to become barely-capable regarding many topic, and this has caused a lot of fragmentation, and a general lowering of general knowledge with regard to information.

          The overall degradation that happened with politics and public information will now be generalized to anything that AI can be applied to.

          You race your MG? Hey my exoskeleton has a circuit racer blob we should go this weekend. You like to paint? I got this Bougereau app I'll paint some stuff for you. You're a physicist? The font for chalk writing just released so maybe we can work on the grand unified theory sometime, you say you part and I can query the LLM and correct your mistakes

          • salawat 6 hours ago ago

            >Then we could just go Google it, and/or skim the Wikipedia page. If you wanted more details you could follow references - which just made it easier to do the first point.

            Except at this point, market forces and going whole hog on neural networking and such instead of sticking with just reflective, impartial indexing of the digital medium has made it nigh impossible for technological aid to actually improve your ability to find niche things. Search Engine Optimization, plus the interests in shaping narratives, have made searchability take a plunge. Right now the unpolluted index may as well be a WMD for how hard it is to find/keep operating one.

        • seanmcdirmid 14 hours ago ago

          This already happened in aviation a long time ago, they have to do things to keep the pilots paying attention and not falling asleep on a long haul where the auto pilot is doing most of the work. It isn't clear at what point it will just be safer to not have pilots if automated systems are able to tackle exceptions as well as take offs and landings well enough.

    • WhyOhWhyQ 15 hours ago ago

      You would get back up to speed in a few weeks. The guy who comes after you and never had formative years writing assembly would never get to the level you were at.

      • frankc 15 hours ago ago

        Perhaps, but I don't think we should optimize for scenario of going back before these tools existed. Of course you need the medical equivalent of BCP, but it's understood that BCP doesn't imply you must maintain the same capacities, just that you can function until you get your systems back online.

        To continue to torture analogies, and be borderline flippant, almost no one can work an abacus like old the masters. And I don't think it's worth worrying about. There is an opportunity cost in maintaining those abilities.

    • nfnriri8 15 hours ago ago

      Back up to speed.

      Who gets the next generation "up to speed" if the teachers are always forgetting?

    • BolexNOLA 15 hours ago ago

      The stakes of a colonoscopy are typically way, way higher than your typical assembly projects.

      • akoboldfrying 15 hours ago ago

        Think of it as a medical device, like an MRI machine. Should we have workarounds for when the MRI machine is down? I think we are better off allocating budget to keeping the MRI machine maintained and running, and assuming that as the normal state -- and likewise for this.

        • jghn 15 hours ago ago

          In many cases these software tools are literally classified as medical devices by the FDA with all of the regulatory compliance that comes with it

        • BolexNOLA 5 hours ago ago

          As the other person said - an MRI has way more oversight/rules surrounding it to insure it’s functioning properly and that people are held accountable when it isn’t. The same can’t be said here.

          They aren’t parallel situations and you can’t cleanly graft these requirements over.

        • xorbax 15 hours ago ago

          I think the same thing about meals.

  • mrmincent 15 hours ago ago

    Doctors’ ability to taste diabetes in urine has also probably eroded since more effective methods have come on the market. If they’re more accurate with the use of AI, why would you continue without it?

    • 15 hours ago ago
      [deleted]
  • throwup238 15 hours ago ago
    • smitty1e 15 hours ago ago

      Thanks for the usable archive link. AI erodes human skill like advertising erodes site utility.

  • stavros 15 hours ago ago

    Every time I see "but your skills will atrophy" arguments like this, they always leave an implied "and you'll need them!" lingering, which is a neat trick because then you never need to explain.

    However, I would like someone to explain this to me: If I haven't needed these skills in enough time for then to atrophy, what catastrophic event has suddenly happened that means I now urgently need them?

    This just sounds very much like the old "we've forgotten how to shoe our own horses!" argument to me, and exactly as relevant.

    • zzzeek 14 hours ago ago

      I think it's a problem when decisions about disease regimens are turned over to software which then becomes the sole arbiter of these decisions, because humans no longer know how to verify the results and have no choice but to trust the machines entirely.

      The scenario we want to avoid is:

      "sorry, your claim was denied, the AI said your condition did not need that treatment. You'll have to sell your house."

      • stavros 9 hours ago ago

        That's a very different situation, though, and already illegal in the EU.

    • 14 hours ago ago
      [deleted]
  • shaldengeki 15 hours ago ago

    Should be in the first, not seventh paragraph: this was a survey of 19 doctors, who performed ~1400 colonoscopies.

  • bn-l 15 hours ago ago

    I know it’s lowering my programming ability. I’m forgetting a lot syntax.

    My solution is increase the amount I write purely by hand.

    • phire 10 hours ago ago

      I mainly use GitHub copilot as nothing more than a fancy autocomplete. I just program as normal and if it happens to suggest more or less what I was about to type, I accept it.

      The completions are usually no more than a few lines.

      This speeds up my typing (not that my 60-70wpm is “slow”) and allows me to get to the next bit of thinking, without getting too much in the way, or decreasing my syntax knowledge, or requiring me to put brainpower into checking the code it generates (since it was what I was about to type). And hopefully avoids copyright issues, how can “what I was about to type” be infringing?

      Typing speed isn’t usually considered to major bottleneck for programming, but I do think using a LLM this way does actually increase my productivity somewhat. It’s not the typing speed increase itself (hell, I’m dubious there even is a real typing speed increase, reading possible completions takes time. But it does feel faster).

      It’s more that my adhd-ass brain had a tendency to get bored while typing and has a tendency to get distracted, either with irrelevant tasks, or I go crazy with Don’t-Repeat-Yourself, wasting way more time creating complex unneeded layers of abstractions.

      Using an LLM as a fancy autocomplete helps me short circuit these bad tendencies. The resulting code is less DRY and way more KISS.

      https://salmonmode.github.io/2020/08/14/the-harmful-obsessio...

    • SethMurphy 15 hours ago ago

      Avoiding copy and paste is the key for me to keeping my syntax memory.

    • frankc 15 hours ago ago

      I think it's doing the same for me but tbh, I am ok with that and not trying to fix it. I do not want to go back to the world before claude could knock out all of the tedious parts of programming.

    • stavros 15 hours ago ago

      Good riddance to my syntax memory. When am I going to ever need it again? The skill I need now is reviewing and architecture.

      • AstroBen 14 hours ago ago

        Being able to spend more of my time thinking about architecture has been amazing

    • seanmcdirmid 14 hours ago ago

      I'm forgetting small nuanced details about programming systems that I only occasionally have to access.

    • throwaway984393 15 hours ago ago

      [dead]

  • alberth 15 hours ago ago

    Another perspective…

    I’m sure similar things have been said with:

    - calculators & impact on math skills

    - sewing machines & people’s stitching skills

    - power tools & impacts on craftsmanship.

    And for all of the above, there’s both pros and cons that result.

    • ViscountPenguin 15 hours ago ago

      My concern is that people seemingly lack the ability to be discerning about when and where to use new technologies. A world in which more deep thought was put into where to apply AI almost certainly wouldn't feature things like AI image generation, as an example.

      If we accidentally put ourselves in a position where humans fundamental skills are being eroded away, we could potentially lose our ability to make deep progress in any non-AI field and get stuck in a suboptimal and potentially dangerous trajectory.

      • alberth 14 hours ago ago

        I completely agree — it’s a tricky human challenge.

        For example, (a) we’ve lost the knowledge of how the Egyptian pyramids were built. Maybe that’s okay, maybe it’s not. (b) On a smaller scale, we’ve also forgotten how to build quality horse-and-buggies, and that’s probably fine since we now live in a world of cars. (c) We almost forgot how to send someone to the moon, and that was just in the last 50-years (and that’s very bad).

  • aspenmayer 7 hours ago ago

    The paper, previously on HN:

    https://news.ycombinator.com/item?id=44883350

    https://www.thelancet.com/journals/langas/article/PIIS2468-1...

    https://doi.org/10.1016/S2468-1253(25)00133-5

    If someone finds a link to a pre-print or other open access, please post it in the thread, as this is just the abstract.

  • dzhiurgis 10 hours ago ago

    Moral panic as doctors' value will deflate as it's easier to get into field.

  • llm_nerd 3 hours ago ago

    I know nothing about this field, and the actual paper is behind a paywall, but it says that after the "exposure to AI", the adenoma detection rate (ADR) dropped from 28.4% to 22.4%.

    As a layman, does ADR simply mean suspicion, or does it mean they correctly and accurately saw adenomas in 28.4% of patients before and now the rate is only 22.4%. Or just that they suspected it 6% more before? Does the actual paper detail if they simply stopped seeing illusions, or did they actually stop seeing meaningful things they used to see?

    I'm sure the paper goes into more detail, but I'm more interested in the false positive vs false negatives than just overall %.