ChipAgents introduced Build-A-Chip-Bench, a benchmark built in collaboration with NVIDIA designed to test whether AI agents can carry an end-to-end digital chip-design task from specification through physical implementation and verified GDS; thus, measuring true progress toward semiconductor design superintelligence.
AI agents are increasingly capable of generating RTL and completing individual semiconductor engineering tasks. But designing a chip requires far more than producing code that passes functional verification. Architectural decisions must ultimately be validated through synthesis, timing closure, placement and routing, power and area constraints, and physical verification before a design is ready for tapeout.
“As chip designs get more complex and time to market pressures increase, we see the need for chip superintelligence that can fully engineer the next generation of silicon,” said William Wang, Founder and CEO of ChipAgents. “The real challenge is carrying engineering intent through a long sequence of interconnected decisions until the design satisfies both functional and physical requirements. Build-A-Chip-Bench gives the industry a way to measure that gap and track progress toward that level of autonomous semiconductor engineering.”
Existing hardware-AI benchmarks have given researchers shared ways to evaluate RTL generation and verification. But those tests largely stop before downstream physical implementation. An RTL design that simulates correctly can still miss timing, exceed its area or power budget, fail to route, or produce physical-design violations.
Build-A-Chip-Bench extends evaluation across the complete chip design flow, including architecture, microarchitecture, RTL, functional verification, logic synthesis, floor planning, placement, clock-tree synthesis, routing, timing and power signoff, and physical verification.
“Chip design is a defining test of AI’s ability to make decisions that hold up under the demands of the physical world,” said Tim Costa, vice president and general manager of industrial and computational engineering at NVIDIA. “ChipAgents’ Build-A-Chip-Bench tests whether agents’ decisions hold up from design intent through physical implementation, giving developers a clear view of where reasoning and tool use must improve to deliver impactful autonomous engineering.”
Initial Results Show How Far Today’s Agents Still Have to Go
Initial evaluations found that tested agents passed functional verification on ~40% of tasks on average, while only approximately 10% met the required physical performance, power and area (PPA) targets. The results illustrate an important distinction in AI-driven semiconductor design: generating functionally correct RTL is only one part of the engineering challenge. These results highlight the challenge of carrying functionally correct designs through physical implementation under the benchmark’s test conditions. Autonomous systems must also reason across abstraction levels, preserve validated behavior through revisions, operate specialized engineering tools and plan around expensive real-world physical feedback.
Build-A-Chip-Bench task details, methodology and initial results are available at https://chipagents.ai/research/build-a-chip-bench. The benchmark was developed by Surya Gunukula, Jerry Yicheng Wang and Kevin Zhang, with technical expertise from NVIDIA.
