Do AI Models Promote Overuse or Restraint in Healthcare?

Clinicians are rapidly integrating artificial intelligence into everyday medical practice. Consequently, evaluating LLM clinical recommendations has become a pressing priority for healthcare systems worldwide. Physicians frequently balance diagnostic thoroughness against the dangers of low-value care. In this context, medical experts launched the Choosing Wisely initiative to curb unnecessary interventions. A benchmark study in the Annals of Internal Medicine investigated whether leading AI models endorse restraint or promote excessive care.

Evaluating LLM Clinical Recommendations Across Scenarios

The investigators examined six commercial models from OpenAI, Anthropic, and Google. Specifically, they designed 225 clinical vignettes based on 15 Choosing Wisely recommendations. Each recommendation featured ten scenarios requiring clinical restraint and five scenarios necessitating active intervention. Overall, the models generated 4,050 responses to these standardized prompts. The research team primarily measured two outcomes: the overuse rate and the false restraint rate. Therefore, the study offered a direct look into artificial intelligence decision-making patterns.

Key Findings on Overuse and False Restraint

Across all scenarios, the models recommended unnecessary care in only 5.4% of responses. Furthermore, the models avoided overuse completely in eight out of the fifteen clinical topics. However, two specific clinical conditions generated the vast majority of inappropriate suggestions. The models recommended unnecessary thyroid ultrasounds for incidental subcentimeter nodules in 42.8% of cases. Similarly, the systems advised routine electrocardiography for low-risk, asymptomatic adults in 28.3% of tests. In contrast, false restraint occurred in merely 0.6% of all clinical scenarios. Thus, artificial intelligence algorithms rarely withheld appropriate care when guidelines mandated action. Prostate-specific antigen screening showed the highest false restraint rate at 3.3%.

Clinical Implications for Practicing Physicians

These findings provide critical insights for clinicians utilizing artificial intelligence assistants. In countries like India, unneeded testing increases out-of-pocket healthcare expenses significantly. Therefore, doctors must scrutinize automated suggestions for routine diagnostic workups with caution. When clinical guidance remains contested, models frequently default to defensive or aggressive workups. Clinicians should remember that AI tools serve as decision aids rather than definitive authorities. Consequently, doctors retain ultimate responsibility for aligning clinical management with validated evidence-based protocols.

Frequently Asked Questions

Q1: What did the study discover about LLM clinical recommendations?

The study found that artificial intelligence models recommended unnecessary medical care in only 5.4% of scenarios. However, overuse clustered heavily around thyroid ultrasound for tiny nodules and routine electrocardiograms in low-risk individuals.

Q2: What is false restraint in clinical AI models?

False restraint describes a scenario where an artificial intelligence tool inappropriately withholds a necessary test or therapy. Fortunately, the study revealed that false restraint occurred in only 0.6% of clinical evaluations.

Q3: Why did AI models struggle with thyroid nodules and ECG screening?

These specific areas involve nuanced guidance and frequent real-world overuse in clinical training data. Consequently, commercial models frequently mirrored widespread defensive medicine practices rather than strictly adhering to Choosing Wisely principles.

References

1. Ariel D et al. Overuse and False Restraint in Large Language Model Recommendations for Choosing Wisely Scenarios. Ann Intern Med. 2026 Oct 20. doi: 10.7326/ANNALS-26-01667. PMID: 42896192.
2. Bressman E et al. Variation in Large Language Model Recommendations in Challenging Inpatient Management Scenarios. J Gen Intern Med. 2025. doi: 10.1007/s11606-025-09888-7. PMID: 41055682.
3. Bitterman DS et al. Large Language Models Prioritize Helpfulness Over Accuracy in Medical Contexts. NPJ Digit Med. 2025.