
Who gets left out of an AI test? Two studies count the gaps
A million images sounds like a generous foundation for skin AI. What if the patient is a child? Researchers who examined 29 public dermatology image collections started with more than 1.1 million images. After quality filtering and removal of near duplicates, they estimated that 2.3 percent of images were pediatric and 11.0 percent fell in the two darkest Fitzpatrick categories. Some missing demographic labels were estimated, not recorded at the bedside. These are shares of images, not percentages of patients misdiagnosed. A separate study asked what happens when safety benchmarks look beyond a familiar set of obvious requests. Researchers tested ten language models on 5,747 instructions derived from real legal cases. The collection spanned 269 offense types and 50 intents under Chinese and US law. Scores describe how those models responded under that evaluation. They do not measure how often users act on an answer, and they are not a judicial determination of any particular response. The connection between the studies is a practical one for anyone buying an AI system: ask to see what is in the test. An image library can grow while scarcely broadening the ages or skin tones it contains. A new set of cases can change what a safety score means even if the model stays the same. Neither audit alone can say whether a particular clinic or service is safe. Both tell its evaluators where to look next.











