Study Evaluates AI Chatbots on Parkinson's Disease Levodopa Therapy
A recent cross-sectional analysis assessed five major artificial intelligence chatbots on their ability to provide public patient education regarding levodopa therapy for Parkinson's disease, highlighting significant variations in accuracy, safety, and readability.
🏃FitnessPhoto: Nature Health Sciences
Key points
- Five major AI chatbots were evaluated on levodopa therapy education for Parkinson's disease.
- Copilot and DeepSeek achieved the highest median accuracy scores among the tested models.
- Safety gaps, inconsistent information quality, and challenging readability remain key limitations.
As individuals increasingly turn to artificial intelligence for medical inquiries, a new study published in Nature Health Sciences evaluated how well five prominent large language model chatbots—ChatGPT, Gemini, Copilot, DeepSeek, and Doubao—answer questions about levodopa therapy for Parkinson's disease. Five blinded raters assessed 265 responses using standardized benchmarks including DISCERN, EQIP, the JAMA criteria, and the Global Quality Score, alongside six readability indices. An additional independent trial of 265 responses tested the stability of the output over repeat generations.
The evaluation found that safe-response rates across the tested models spanned from 86.8% to 94.3%, with high stability observed in repeat-generation testing at 97.0% exact safety agreement. However, performance varied notably across other metrics. Copilot and DeepSeek demonstrated the highest median accuracy scores, while Doubao achieved the highest scores on the DISCERN and EQIP information quality assessments. Conversely, ChatGPT recorded the lowest information-quality scores, and Gemini offered the most favorable readability profile, whereas Doubao presented the poorest readability.
What to know: While current language models generally generate informative responses regarding Parkinson's disease medication, the study emphasizes that notable safety gaps, variable information quality, and poor text readability persist. Experts caution that these findings reflect expert-rated output quality rather than proven patient education effectiveness, meaning patient-centered validation is still necessary. Readers should consult a qualified healthcare professional before making any personal medical decisions regarding medication regimens.
Source Nature Health Sciences — nature.com/articles/s41598-026-74929-y ↗
Read next
🏥Public HealthPreventive Services Task Force Resumes Meetings Under New Leadership
The U.S. Preventive Services Task Force is set to reconvene this week for the first time in over a year following a significant membership overhaul by Health Secretary Ro…
❤️Conditions & CareBlood-Based Genetic Signature May Predict CAR T-Cell Therapy Success in Lymphoma
Researchers have identified a specific RNA signature in blood plasma that may help predict how well lymphoma patients respond to CAR T-cell therapy.
🧠AI AnalysisUltraprocessed Diets May Impact Brain Reward Systems in Younger Adults
A Virginia Tech study suggests that individuals aged 18 to 21 may experience changes in brain reward processing after consuming ultraprocessed foods, a response not obser…
🧠Mind & Mental HealthAddressing Body Image and the Reality of Gluteal Anatomy
A personal reflection on the social and psychological impact of body shape, specifically regarding the gluteal region, and the limitations of consumer solutions for perce…
❤️Conditions & CareMigraine and Smoking Linked to Increased Early Stroke Risk in Women
A large-scale study suggests that people who have given birth and experience migraines while also smoking face a significantly higher risk of stroke before age 60.
🔬ResearchCBD Oil Found Ineffective for Managing Behavioral Issues in Youths with Intellectual Disabilities
A double-blind clinical trial published in JAMA Pediatrics indicates that CBD oil provides no significant behavioral improvement for children and teenagers with intellect…