Health Care AI, Intended To Save Money, Turns Out To Require a Lot of Expensive Humans.

Oh hey, this same quote is relevant yet again:

In other words, an AI-supported radiologist should spend exactly the same amount of time considering your X-ray, and then see if the AI agrees with their judgment, and, if not, they should take a closer look. AI should make radiology *more* expensive, in order to make it more accurate.

But that’s not the AI business model. AI pitchmen are explicit on this score: The purpose of AI, the source of its value, is its capacity to increase productivity, which is to say, it should allow workers to do *more*, which will allow their bosses to fire some of them, or get each one to do more work in the same time, or both. The entire investor case for AI is “companies will buy our products so they can do more with less.” It’s *not* “business custom­ers will buy our products so their products will cost more to make, but will be of higher quality.”

Cory Doctorow: What Kind of Bubble is AI?

You know it's funny, this keeps happening with every single fucking AI thing they produce. It always still needs humans fixing its mistakes because its just not reliable enough.

I think it's doing a lot less reducing headcount and a lot more making people specialize in what I would call "bullshit," to be able to fix mistakes made by an AI quickly and efficiently.

Maybe, just maybe, if they have to pay people to fix the AIs work that they can cut out the middleman and just pay the people to do the fucking job to begin with. No, what am I saying, that's just ridiculous! /s

😱

“Even in the best case, the models had a 35% error rate,” said Stanford’s Shah

So, when the AI makes a critical error and you die, who do you sue for malpractice?

The doctor for not catching the error? The hospital for selecting the AI that made a mistake? The AI company that made the buggy slop?

(Kidding, I know the real answer is that you're already dead and your family will get a coupon good for $3.00 off a sandwich at the hospital cafeteria.)

Guess what!

When accuracy matters, the labor cost of babysitting the LLM's output is the same as doing the work yourself. That's before you even consider having to unfuck it after it paints itself into a corner and ruins its own model.

I hope those employees work really really slowly.

I feel like any AI tool that's being sold as saving you money just won't do that. *Some* of the ones that sell improved detection rates might.

AI that works as a tool designed to be used by an existing or new professional to augment their abilities works as well as any other tool. An ultrasound doesn't save you money except in the abstract of being more freely usable than x-ray allowing for more checks with less equipment.
A tool that highlights concerning areas on a mammogram isn't replacing a person anymore than the existing tools that highlight concerning heart rhythms.

Trying to get llms to replace people, particularly when it comes to trying to explain the content of a potentially technical medical discussion is just not going to be reliable.