AIImpactLab
9 impact clustersWork & marketsCognition & learningTechnical failuresPrivacySecurityEnvironmentLaw & informationSocial good & healthSystemic risk
Search the evidence

Find the signal.

Search titles, impact clusters, countries, organizations and the full text of every analysis.

5 stories found

A wall of 1,357 medical-device approval tiles narrows to three illuminated patient-outcome records beside an empty hospital evidence chart.
Social good & healthUnited States · Global implications+3 clusters01

Only three of 1,357 FDA-authorized AI medical devices were evaluated on patient outcomes

A PLOS Digital Health evidence census linked the FDA's 1,357 authorized AI and machine-learning medical devices through December 5, 2025 to prospective trials and publications. Thirty-four devices were linked to registered prospective trials, 12 had posted results, 12 had peer-reviewed publications, and only three evaluated patient-centered outcomes such as mortality, morbidity, or readmission. The review does not show that the remaining devices are ineffective; it shows that authorization and benchmark performance rarely answer the outcome question patients care about most. With 78 percent of the devices concentrated in radiology and vulnerable populations often excluded from studies, the validation gap can travel through hospitals and across countries long before durable benefit or equitable performance is known.

5 min
Cognition & learningGlobal+2 clusters02

Churpek et al., “Early Nephrology Consultation and Acute Kidney Injury in Hospitalized Patients”

University of Chicago and University of Wisconsin researchers randomized 180 hospitalized patients identified by a real-time machine-learning score as being at elevated risk of acute kidney injury. Triggering an early structured nephrology consultation did not significantly reduce peak creatinine changes, acute kidney injury, mortality, readmission, or other major outcomes; many specialist recommendations were not followed by the treating teams.

2 min
A medical AI system faces an unfinished clinical evaluation maze as a benchmark score floats above real patient-care tasks.
Technical failuresGlobal+3 clusters03

Medicine lacks a credible test for AI superintelligence

A Nature Medicine commentary argues that medical AI urgently needs a rigorous, task-based framework for defining and measuring “superintelligence.” Existing benchmarks can reward narrow performance without showing that a system can improve care across real clinical work, making headline claims potentially misleading. The proposal shifts attention from whether a model beats a score to which medical tasks are tested, against which human comparison, under what conditions, and with what evidence of patient benefit and safety.

3 min
A clinical waveform and reinforcement-learning decision tree ending at an evidence gap.
Cognition & learningGlobal+2 clusters04

Tang et al., “Reinforcement learning for treatment decision-making in sepsis: a scoping review”

Reviewing 72 studies of reinforcement-learning systems for sepsis treatment, the authors found that every study was retrospective, 58 studies—80.6%—relied on the same MIMIC critical-care database, and only 10 used private datasets. Although many papers claimed that AI-derived treatment policies outperformed clinicians, variation in how patient states, treatment actions, rewards, and counterfactual outcomes were defined made those comparisons difficult to validate.

2 min
Work & marketsGlobal+3 clusters05

Strong et al., “Human-AI Collaboration in Healthcare: A Scoping Review”

This Oxford-led npj Digital Medicine review screened 17,463 records and included 140 empirical studies of human-AI collaboration in healthcare from January 2015 through October 2025. It finds that the evidence base is concentrated in diagnostic interpretation, while triage, therapeutic, administrative, and system-level workflows remain thinner; it also notes that AI benefits depend heavily on task fit, workflow integration, training, and calibrated trust.

2 min