How we read the signal

Analysis frame

Evidence level

Peer-reviewed research

Analytical lens

Compare observed query purposes with the diagnosis-heavy evaluation agenda, while keeping platform use separate from answer quality and patient outcomes.

Affected groups
  • Patients whose treatment choices may be informed by AI-synthesized evidence
  • Clinicians and health systems choosing, teaching and auditing decision-support tools
What remains unknown
  • The study did not measure accuracy of answers, actual treatment decisions or patient outcomes
  • A single vendor's users and model-classified queries may not represent all clinical AI use
Second-order effects to watch
  • Treatment-focused evaluations may become more important than diagnostic quiz scores
  • Early-career clinicians may need explicit training on when and how to challenge AI evidence summaries

What the 271 million questions show

The study recorded use on one NPI-verified platform from 2024 through June 2026. It found 33.1% of physician queries concerned treatment and management and 13.9% diagnosis. That is a distribution of questions, not a measured treatment benefit.

What the next test must ask

Independent evaluations should test medication choices, changing evidence and patient-specific contraindications in realistic workflows. They should track whether clinicians catch wrong answers and whether patients do better, not merely whether a model performs on vignettes.

Primary trail

Go to the source

Read the evidence behind this analysis. External links open in a new tab.

JAMA — clinician use of an AI-enabled clinical information platform JAMA — accompanying clinical evidence editorial American Medical Association — AI evaluation guide