By Keith Pitt, Founder and CEO at Buildkite
Software development’s environmental consequences are an emerging problem.
Picture the scene. Your phone wakes you up, news headlines appear on your screen and your thermostat adjusts to your perfect morning temperature. Before work, you begin a workout on your smartwatch as you go for your morning run, followed by a calming meditation session, tracked by an app. When you finally head to work, real-time traffic maps inform you of how to avoid delays, and your calendar fills up with all the meeting invitations you’ve just received, and you’re set for the day. Software isn’t just an important tool in our lives… it’s in everything we do.
This is the new ‘environment’ we live in – phone apps, cryptocurrency, social media and artificial intelligence are some of the central components of our digitized, software-driven existence. But this ‘environment’ has a polluting downside just like our physical one. By 2040, the IT industry is expected to account for 14% of the world’s carbon footprint – up from about 1.5% in 2007.
The impact of software use on our planet
Consider the world of cryptocurrency, which has only been around for just over 10 years. Now crypto mining accounts for 0.5 percent of worldwide consumption of electricity a year, as much as a country like Sweden or Malaysia.
Next consider artificial intelligence (AI), which is touted as one of the most transformative technologies of ouhr time. When you apply an AI model to assess the “iris flowers dataset”, a publicly available corpus of data, you see some interesting results energy wise when you try to dial up accuracy levels. An AI model can achieve an accuracy of 96.17% in classifying flowers’ different species using just 964 joules of energy. The next 1.74%-point increase in accuracy requires 2,815 joules of energy consumption, and the last 0.08% incremental increase in accuracy took nearly 400% more energy than the first stage.
That might not seem like much in an isolated incident, but if you consider that training AI models happens on a global scale now, you start to see the environmental impact AI is having overall. Training a single neural network model today can emit as much carbon as five cars in their lifetimes. And the amount of computational power required to run large AI training models has been increasing exponentially, with a 3.4-month doubling time.
Software testing only makes things worse – but we need it
The environmental issues don’t just exist for software use; they exist for software creation as well. When developers write software, the compound debt of related energy usage of running it and ensuring it runs safely can be considerable.
Software building and testing can seem harmless at first – especially the testing part. Commonly, the testing software is developed alongside the core programming. Engineers code a set of instructions that will be performed on the system or software as a way of ensuring that the system functions as expected. There are various means for executing these tests, and thousands of new testing scripts are added every year, all of which contribute significantly to compute resources, a company’s costs, and resulting carbon emissions.
But testing is an integral part of of software development, and getting rid of it won’t solve these problems – much like how Rachel Carson’s book The Silent Spring reveals that eliminating the world of insects by bombing them with poisons is a very bad idea. As elementary as it sounds, each time you think you understand the impact of testing, it gets more confusing.
Let’s start with software itself, the language that tells your computing device what to do. All that language is written by people, the people who write the code that runs the commands that make computers compute. Those people, programmers, developers and engineers are the people designing, building, refining and testing volumes of code that drive the software we use in almost every aspect of our lives, from your simple SMS to an elderly grandparent to controlling the trajectory of satellites flying over us. Every release, every update, every time something is launched or needs troubleshooting, it takes the people who test and develop the code to make it happen.
As the pace of business increases, the increasing demands for software productivity, efficiency, and on-time delivery follows. With these pressures, the margin for wasteful practices increases the complexity of software solutions for ensuring operational, launch-ready code.
Bad-to-worse testing practices
Nowadays, testing software is a time-consuming task not in terms of writing the test – that can be almost mechanical and straightforward. The true cost is in the thousands of times it will be executed for years to come. Every time you try to mitigate a problem and hit the test button you’re burning time and money. It can be never-ending as developers usually want to add more tests. It’s easy to add tests, but it’s hard to remove them, and if, in all of this madness, you remove the wrong test, people can get fired.
The trouble is in reality, no-one really looks after these tests. This is a precipice that engineers stand on – can they remove a testing script? Is it important? Consider working with a small 25-person team of engineers. You can have 100,000 lines of code and over 1 million lines of test code to check it. Some of those scripts work, some don’t, some people leave, new people join, some code is overlooked… and so on. People don’t know what they don’t know, and so for fear of breaking a system, they leave things alone.
But even if everything goes well, by appearances testing itself seems almost free of any consequences. Yet, that’s not really true.
Software testing is amassing a debt that has yet to be paid, and more and more tests are written at every company making software for every release, every day. If something doesn’t work, they’ll just keep trying until it works. At hundreds of companies, there’s just zero visibility into how much time and energy is wasted. So here we have a very much a silent but costly (both financially and environmentally) protagonist.
And therein lies the reality – that inefficiency of software testing wastes a lot of time and energy (in both an environmental and effort sense). Thousands of companies worldwide are mindlessly automating their software by running tens of thousands of servers, testing millions of lines of code, every minute of every day, and much of it is unnecessary.
Whatever approach companies choose to take today and in the future, they can’t fall back on the old wasteful strategies of ensuring code safety and continue to call themselves a ‘planet conscious’ enterprise any longer.
In the end, we need to wake up and realize, keystrokes are polluting the world, and it’s time to do something about it.
Earlier this year, Buildkite launched a product to help reduce the inefficiencies in test suites for companies of all sizes. To date, we’ve ingested billions of test executions and have helped teams improve their test reliability by an average of 20%, allowing them to remove or repurpose their most inefficient tests. You can try out Test Analytics for free at https://buildkite.com/test-analytics.
##
To hear more about cloud native topics, join the Cloud Native Computing Foundation and the cloud native community at KubeCon + CloudNativeCon North America 2022 in Detroit (and virtual) from October 24-28.
ABOUT THE AUTHOR
Keith Pitt is founder and CEO of Buildkite, the CI/CD and testing platform that pioneered the hybrid model (part hosted, part self-hosted) for running CI pipelines on your own infrastructure.
