As AI coding agents accelerate software development, testing is no longer a separate step after coding. It’s the feedback that helps agents reach the right result, and that only works if it’s fast. CircleCI is expanding its Smarter Testing suite with new capabilities designed to help organizations shorten feedback cycles, reduce unnecessary testing, and control compute costs as development activity grows.
In this VMblog Q&A, Rob Zuber, CTO of CircleCI, discusses how AI-driven development is changing software testing, why treating coding and testing as separate steps no longer works, and how new capabilities such as Dynamic Parallelism can help engineering teams deliver software faster without sacrificing quality or control.
VMblog: AI coding agents are accelerating software development. What new challenges does this create for engineering teams?
Rob Zuber: Agents have made it much easier to produce code, which is exciting, but the challenge is that we can’t think of writing code and validating it as two separate steps anymore. Testing isn’t just a gate at the end. It’s the feedback that tells the agent whether it’s on the right track, so it can correct course and get to the right result faster.
That only works if the feedback is very fast. An agent can iterate far faster than a person, so a slow test run wastes most of that advantage, and the more code being produced, the more every slow loop costs. The tools used to generate code can change, but the need to independently validate the result doesn’t go away. The agent shouldn’t be grading its own work, but it needs that independent answer back fast enough to act on it.
VMblog: Why does traditional software testing struggle to keep up with AI-driven development?
Rob Zuber: A lot of our testing infrastructure was built around testing as a separate stage that happens after the code is written: someone pushes, and we run the same set of checks every time. That made sense when changes happened at a human pace. But agents are generating and iterating constantly, and they need an answer while they’re still working, not after the fact. Treating coding and testing as two separate steps is the thing that doesn’t hold up.
If I’m making a small change to one part of an application, I probably don’t need to run every test across the entire codebase to understand whether that change works. And I shouldn’t need the same amount of compute as someone making a much larger change.
Getting that feedback to be very fast means being much more intentional about what we’re validating and when, and that’s where the real opportunity is.
VMblog: What is CircleCI announcing with its expanded Smarter Testing suite?
Rob Zuber: The idea behind Smarter Testing is to make that feedback loop very fast for developers and coding agents, without doing a bunch of unnecessary work along the way.
That starts with running affected tests locally before code reaches CI. Intelligent Test Selection identifies which tests need to run, so that check stays quick. In CI, Dynamic Test Splitting and Dynamic Parallelism help distribute that work efficiently across machines. We can also automatically rerun failed tests without starting the entire suite over.
Together, these capabilities help teams validate changes earlier and make better use of their existing resources.
VMblog: What makes Dynamic Parallelism particularly important as development activity increases?
Rob Zuber: Dynamic Parallelism adjusts the number of CI machines based on the tests selected for each run. So a small change that only needs a fraction of the test suite can run on fewer machines, while a larger change gets the resources it needs.
As agents make it easier to produce more changes, those efficiency gains become increasingly important. Teams need to handle that growth without having their infrastructure costs increase at the same rate.
VMblog: Are organizations already seeing measurable benefits from Smarter Testing?
Rob Zuber: We’re seeing some encouraging results from customers.
Medallion is an interesting example because their pull request volume has grown about 10x and their test suite has grown 2.5x. Despite that growth, their internal measurements show that build times and cost per pull request have come down. That’s a pretty meaningful result when you consider how much more work is moving through the system.
Kustomer is another example. They enabled Intelligent Test Selection, so they’re no longer running the full test suite for every change. Instead, they run the tests affected by that change, which has helped reduce the compute required and get feedback to developers sooner. They’ve reported cutting the costs of running their tests by a quarter.
What I find interesting about both examples is that the improvements came from being more intentional about how testing happens, rather than adding more infrastructure.
VMblog: What should engineering leaders rethink about their software delivery processes as AI adoption grows?
Rob Zuber: We need to get better at measuring the work it actually takes to get a change into production.
It’s easy to focus on how fast an agent can generate code or how quickly an individual pipeline runs. But a change might go through several rounds of testing, feedback, and revision before it’s ready to ship. All of those cycles have a cost, both in compute and in the time developers spend waiting or working through failures.
I think engineering leaders need to look at that entire process, specifically, where the work is waiting. Where feedback could happen earlier so that they can make validation more efficient?
That’s really the opportunity with Smarter Testing. When validation is very fast, it becomes part of the loop instead of a step after it, and developers and agents get to the right result sooner. We can improve the way developers and agents work within their existing workflows, while reducing the time and resources it takes to get changes ready for production.
##






