How Cognitive Bias Impacts AI in Radiology Exams

How Cognitive Bias Impacts AI in Radiology Exams

Large language models are rapidly entering the medical field. However, LLM cognitive bias poses a hidden threat to diagnostic accuracy. A recent study reveals how specific prompts exploit these biases to degrade performance in radiology board-style examinations. Consequently, Indian medical professionals must understand these limitations before relying on AI tools for clinical decisions.

The Impact of LLM Cognitive Bias on Accuracy

Researchers tested ten contemporary models using 400 radiology questions. Initially, models performed well on text-based queries with an 84.8% baseline accuracy. However, performance plummeted when the prompts introduced cognitive biases like authority or complexity bias. For instance, authority bias caused a significant 21.1% drop in text question accuracy. Furthermore, multimodal questions involving images showed even lower baseline scores. This suggests that AI models remain vulnerable to subtle psychological manipulation in clinical workflows.

Mitigation Strategies for Radiology AI

Despite these vulnerabilities, specific techniques can improve AI reliability. The study evaluated prompt bias audits and one-shot mitigation strategies. These methods helped recover some lost accuracy. Nevertheless, the risk of anchoring bias remains a challenge. Therefore, doctors should use AI as a secondary tool rather than a primary decision-maker. Additionally, continuous education on AI literacy is vital for radiology departments across India.

Frequently Asked Questions

Q1: What is authority bias in the context of LLMs?

Authority bias occurs when the AI is prompted with a statement from a supposed expert, leading it to favor that opinion even if it is incorrect.

Q2: Can multimodal AI handle radiology images effectively?

Currently, multimodal AI shows lower accuracy compared to text-only models, particularly when faced with complex visual data and biased instructions.

References

1. Dietrich NT et al. Cognitively Biased Prompt Effects on Large Language Model Accuracy for Radiology Board-Style Examination Questions. Radiol Artif Intell. 2026 Apr 15. doi: 10.1148/ryai.250585. PMID: 41983923.
2. Ahuja AS. The impact of artificial intelligence in medicine on the future role of the physician. PeerJ. 2019;7:e7702.
3. Kundu S. AI in radiology: Current state of the art and future challenges. Indian J Radiol Imaging. 2023;33(2):125-128.