AI Chatbots Misdiagnose Health Issues Two‑Thirds of the Time in Real‑World Use

TL;DR **AI chatbots diagnose correctly just over one‑third of the time in everyday use, even though they excel when given complete medical information.**

Context People increasingly turn to AI chatbots for health advice because they are instantly available and feel personal. England’s Chief Medical Officer, Prof Sir Chris Whitty, warned that the responses are often confident yet wrong, raising safety concerns.

Key Facts In a controlled experiment where chatbots received full medical case details, they answered correctly 95% of the time. This high performance shows the models can process complete information accurately.

When 1,300 participants interacted naturally with chatbots to seek a diagnosis, accuracy fell to 35%. Two‑thirds of the users received incorrect advice, indicating a substantial drop when information is incomplete or ambiguous.

What It Means The gap between 95% and 35% accuracy suggests that missing or unclear user input drives most errors, not a flaw in the model’s core knowledge. Correlation does not prove causation, but the pattern points to the need for better user guidance or clarification prompts.

Practical takeaway: treat chatbot suggestions as a starting point, not a definitive diagnosis, and verify urgent symptoms with a healthcare professional.

What to watch next Regulators may issue guidance on AI‑based triage tools, and researchers will likely test whether structured prompts improve real‑world accuracy.