Explainable AI improved oncologists’ lung-cancer predictions, but external validation exposed the limits
A multi-country study in Nature Medicine evaluated explainable AI support for treatment decisions in advanced non-small-cell lung cancer. The retrospective I3LUNG cohort included 2,396 patients treated with immunotherapy-based regimens across six centers in six countries. Models using routine clinical and blood data achieved test performance up to an area under the curve of 0.77 and outperformed traditional single biomarkers and clinical scores in the independent test set. In a separate usability study, twenty oncologists reviewed one hundred cases first without and then with model predictions and SHAP-based explanations. Sensitivity for predicting disease control increased from 0.72 to 0.87, with gains in accuracy and F1 performance; overall-survival prediction improved more modestly. The paper is valuable because it reports the limits alongside the gains. External-validation performance fell to an AUC range of 0.55 to 0.72, the complete multimodal sample was small, and added imaging, pathology, and genomic data did not produce a reliable benefit across test and external cohorts. Differences between patient populations may explain some decline, which is exactly why local calibration and prospective evaluation matter. The authors describe silent prospective validation in more than two thousand patients, another usability study, and a planned pragmatic randomized trial before deployment. The result is promising decision support, not autonomous clinical authority.