Scientists have warned against relying on AI for medical guidance after a man acquired a rare condition following advice sourced from ChatGPT.
A report published in the Annals of Internal Medicine recounts how a 60-year-old patient suffered from bromism, a toxic syndrome, after abiding by the AI chatbot’s advice.
The condition emerged when the man sought alternatives to standard table salt through ChatGPT and subsequently consumed sodium bromide for a quarter of a year.
Researchers from the University of Washington in Seattle documented this case as proof of how AI technology can lead to “preventable adverse health outcomes”.
The patient had consulted the chatbot after learning about potential harmful effects of sodium chloride consumption, specifically looking for ways to remove chloride from his diet.
The man then began taking sodium bromide, despite having read that “chloride can be swapped with bromide, though likely for other purposes, such as cleaning”.
This substance served as a sedative during the early 1900s. The journal added that bromism was a “well-recognised” condition in that era, responsible for nearly 10 per cent of psychiatric hospital admissions at the time.
The researchers were unable to examine the exact conversation between the patient and ChatGPT.
LATEST DEVELOPMENTS
- Cancer milestone as scientists could detect deadly disease by listening to the sound of your voice
- Deadly fungus labelled ‘global threat’ found in Britain amid worldwide alarm
- Cancer survivor who lost part of her tongue shares ‘painful’ sign that appeared six months earlier
The man arrived at hospital convinced his neighbour was attempting to poison him. Upon arrival, medics observed he exhibited paranoia about drinking the water they provided, despite experiencing intense thirst.
Within a day of admission, he attempted to flee the facility, leading to doctors subsequently detaining him under mental health legislation and began treating him for psychotic symptoms.
Following stabilisation, the patient disclosed additional symptoms consistent with bromide poisoning, including facial acne outbreaks, persistent thirst and chronic sleeplessness.
The medical team eventually connected these indicators to his three-month consumption of sodium bromide.
When the researchers tested ChatGPT themselves about chloride substitutes, the system suggested bromide without issuing health warnings or enquiring about their reasons for seeking such information “as we presume a medical professional would do”, they wrote.
ChatGPT and similar AI applications could “generate scientific inaccuracies, lack the ability to critically discuss results, and ultimately fuel the spread of misinformation”, the authors warned.
They clarified that no healthcare professional would have recommended sodium bromide as a table salt substitute.
The researchers urged that physicians should now investigate whether patients have obtained medical advice from AI sources when assessing their conditions.
