MirrorCode benchmark reveals rapid progress in long-horizon software synthesis

Evaluation data released by Epoch and METR through the MirrorCode benchmark suggests a shift in the scale of software projects that artificial intelligence can autonomously navigate. In one instance, the Claude Opus 4.7 model successfully reimplemented a program in 14 hours at a cost of $251 in inference—a task that human researchers estimate would typically require between two and seventeen weeks of engineering time. The benchmark requires models to rebuild programs such as Apple’s pkl configuration language and the gotree phylogenetic tree parser without access to source code or the internet.

Of the 25 target programs tested, 17 were solved with a perfect score. The results indicate that models are developing the ability to self-orient within alien software environments, deriving architecture and logic purely through functional interaction. While success rates were high, certain targets remained out of reach; models struggled significantly with the ruff Python linter and the giac_subset mathematics package, marking the current ceiling of automated code reconstruction.

Parallel research from Anthropic suggests these gains in reasoning are translating to physical environments. In a recent robotics trial, the company’s latest models completed a series of quadruped tasks in under 10 minutes—a significant decrease from the 181 minutes required by human-assisted models just a year prior. The improvement appears to be a direct consequence of general model scaling rather than specific robotics-focused training, supporting the theory that broader intelligence yields immediate dividends in narrow industrial applications.