HAProxy Technologies, provider of the world's fastest and most widely used software load balancer, announced that HAProxy 2.3 tested on a single Arm Neoverse-based AWS Graviton2 instance scales and achieves 2.04 million requests per second. While the tests demonstrate that the load balancer can scale to an incredible level if run on very powerful servers with significant network bandwidth, it's also possible to run on much more modest hardware while still achieving excellent results.

To learn more, read the HAProxy blog post: HAProxy Forwards Over 2 Million HTTP Requests per Second on a Single Arm-based AWS Graviton2 Instance

"This proves that the radical move to multithreading engaged four years ago and the determination to progressively attack all the remaining bottlenecks was the right strategic move for the project at an era dominated by multi-core CPUs: it now performs better than ever without all the functional limitations that once were imposed by a multi-process architecture. I'm even happier to completely remove support for this obsolete model in 2.5," said Willy Tarreau, Community Project Lead & CTO at HAProxy Technologies.

"Infrastructure power and efficiency demands are changing rapidly with increasing and diverse workloads," said Bhumik Patel, Director of Ecosystem Software Development, Infrastructure Line of Business, Arm. "The performance and scale numbers from HAProxy are a testament to the benefits solution providers can achieve when combining best-in-class performance of Arm Neoverse-based cloud instances with one of the fastest and widely-used software load-balancers in the market."

High Performance Arm-based AWS Environment

Wanting to see how HAProxy would scale in a high performance environment, Willy Tarreau, Community Project Lead & CTO at HAProxy Technologies, accessed some of AWS's new Arm Neoverse-based Graviton2-powered instances, which provide up to 64 cores. Each core uses its own L2 cache and there's a single L3 cache shared by all cores. In addition, Tarreau switched to the new Armv8.1-A Large System Extensions (LSE) atomic instructions, which totally unlocked the true power of these machines. This environment gave him the opportunity to push the HAProxy benchmarks to the next level.

Results

HAProxy's performance peaked at 2.04 to 2.05 million requests per second. The direct communication from the client to the server shows 160 microseconds response time while adding HAProxy increases it to 560. So, HAProxy adds at most 400 microseconds on average-less than half a millisecond.

Caption: HAProxy reaches over 2 Million requests per second on Arm-based AWS cloud instances.

At that point, however, the machine is totally saturated and due to this, HAProxy is measuring the effect of queuing at every stage in the chain (network buffers and HAProxy). Preliminary tests on the forthcoming HAProxy 2.4 release show very promising results with even higher rates and lower traversal times, indicating that 2.4 will be an excellent release.