Projects, with context.
Existing scenes, assets, scripts, and settings. Agents modify connected project state while preserving what already works.
COMPLETE UNITY PROJECTSAI Hub
Unity ChinaUnityBench-Mini3D evaluates AI agents on real Unity development workflows. It pairs 77 complete projects with natural-language requirements across seven development skills and three difficulty tiers. A total of 1,178 held-out, human-authored milestone tests check scene structure, component configuration, and runtime behavior directly in Unity.
Research overviewIn game development, correctness lives across code, scene hierarchies, component settings, and runtime behavior. A project can compile and still fail to work.
UnityBench-Mini3D puts agents inside complete Unity projects, then checks their final state with human-authored tests running in the engine.
Existing scenes, assets, scripts, and settings. Agents modify connected project state while preserving what already works.
COMPLETE UNITY PROJECTSSeven expert-defined categories measure the development capabilities behind a game, across three difficulty tiers.
EXPERT-CURATED TASKSHeld-out EditMode and PlayMode tests verify structure and behavior, with partial credit for independently satisfied milestones.
DETERMINISTIC EVALUATIONGame development stresses coding agents in ways general software engineering does not: correctness depends on how engine APIs, serialized scene state, and runtime behavior interact, not on source code alone. Existing benchmarks mainly target platforms such as Godot and heavily rely on LLM judges or coarse task-level outcomes. We present UnityBench-Mini3D, a benchmark of 77 project-level Unity tasks organized around seven expert-defined skill categories rather than tutorials or genres, with three difficulty tiers. Each task pairs a complete Unity project with natural-language requirements and is evaluated using 1,178 held-out, human-authored milestone tests covering scene structure, component configuration, and runtime behavior. This in-engine evaluation provides deterministic partial credit, localizes failures, and distinguishes partial implementations from complete solutions without relying on subjective judges. Across 21 models and 9 coding harnesses, the average pass rate reaches 80.9% on easy tasks but falls to 35.2% on hard tasks, with just 57.7% overall. UnityBench-Mini3D provides a reproducible, diagnostic testbed distinguishing systems across difficulty levels.
Explore the projects behind the scores. Open a task to compare its
initial scene and human-authored golden reference.
Loading benchmark tasks…
Screenshots show initial and golden reference projects. They are not agent-generated outputs.
No prescribed action sequence.
No LLM judge.
Just the final project state.
An initial Unity project and a specification: goal, context, constraints, and resources.
The agent uses its harness tools to edit code, assets, and engine state within a one-hour budget.
Clean reimport and compilation, followed by injected EditMode and PlayMode milestone tests.
Compare milestone pass rates across skills
and difficulty levels, alongside cost
and wall-clock time.
Overall across coding configurations.
Performance drops as skills must work together.
EditMode tests inspect scene hierarchies, object transforms, and asset references to verify that required objects are present, correctly arranged, and connected.
Tests check attached components, serialized properties, and resource bindings against task requirements, including settings for physics, animation, rendering, and user interfaces.
PlayMode tests run the project in Unity to verify dynamic behavior, including physical interactions, animation, and interface responses.
Paper v1.0 · Table 3
| # | Model / Harness |
|---|
Loading results…
Download dataScores (%) are mean per-task milestone pass rates, not the percentage of fully solved projects. Cost is estimated API cost in USD; time is average wall-clock minutes. “—” indicates an unreported value. Computer use agents are evaluated separately from the 32 coding configurations.
* Equal contribution. † Corresponding authors: Minghao Shao, Heng Chang, Jie Feng
If you use UnityBench-Mini3D in your research,
please cite the paper.
@misc{zhou2026unitybench,
title = {UnityBench-Mini3D: Evaluating LLM Agents
for 3D Game Development in Unity},
author = {Zhou, Yijun and Shao, Minghao and
Zhang, Ke and Yuan, Renkai and Yuan, Hongbo and
Hu, Bin and Qian, Bozhao and Shen, Hongxia and
Liu, Yanhong and Chen, Yuting and Shi, Yuntao and
Xu, Zhongxuan and Mao, Shuo and Zhou, Haijia and
Li, Zhendong and Zhang, Tao and Li, Chenggang and
Wang, Yuxuan and Zheng, Zinan and Zhang, Calvin and
Li, Jia and Chang, Heng and Feng, Jie},
year = {2026},
eprint = {2610.35980},
archivePrefix = {arXiv},
primaryClass = {cs.SE},
url = {https://arxiv.org/abs/2610.35980}
}