AI completed the research engineering. Scientists rejected both results
A Nature report and the underlying arXiv preprint test whether frontier AI agents can conduct open-ended AI research, not merely execute a benchmark. In two shadow evaluations, an agent received the central question from a high-quality unpublished NeurIPS 2026 submission, six days, and thousands of dollars in compute. The systems completed the engineering without human help, including coding and experiments, but the original researchers judged that neither made substantial progress on the scientific question and rejected both results. A robustness check using another model and scaffold reproduced the broad failure pattern. The paper identifies recurring weaknesses in judging the publishable bar, responding creatively to design shortcomings, backtracking from dead ends, managing resources, and maintaining the research objective. This is early evidence from two case studies, not proof that AI cannot improve at research. It does show that completing a research workflow is not the same as exercising scientific judgment.
