Analysis frame
Peer-reviewed research
Comparing how two independent teams define coverage reveals how aggregate numbers can hide absent cases without implying a shared population.
- Clinics evaluating dermatology software for children and adults
- Teams evaluating language models across legal contexts
- Error rates for any specific clinic or patient group
- How often the evaluated responses would cause a real-world harm
- Purchasers can request subgroup evidence before a deployment decision.
- Benchmark designers may prioritize new situations over additional near-duplicate examples.
How to request a better evaluation
Ask for sample counts and how missing labels were handled before interpreting a subgroup score. In a legal-safety test, ask who chose the cases, which jurisdictions were covered and exactly what counted as a safe response.
Go to the source
Read the evidence behind this analysis. External links open in a new tab.
Nature Communications — case-grounded legal safety evaluation Nature Communications — public dermatology image collections


