Industry executives and experts share their predictions for 2022.  Read them in this 14th annual VMblog.com series exclusive.

Bring on the next wave of DevOps

By Nati Shalom, CTO and founder of Cloudify.io

Earlier this year I had an interesting discussion with Sean Tucker, managing director at AGC Partners. Sean shared his insights on the state of DevOps and the new wave of DevOps that will be mostly the result of Enterprise digital transformation. This wave accelerated due to Covid, and funding thrown at the opportunity turned the wave into a perfect storm.

What we *didn't* see is a similar wave of new DevOps automation tools that address the special needs of transforming enterprises-and indirectly, the needs of the SaaS vendors that needed to scale fast in response to rising demand. The result is a growing complexity of the DevOps environment; it changed many DevOps teams from being an enabler of agility into a bottleneck.

Why did things get so complex so quickly? Consider the "rise of the -Ops"-DevSecOps, MLOps, FInOps, GitOps. All these new DevOps automation concepts have complicated our need to shift to the highly distributed architectures of microservices, serverless, machine learning, and multi-cloud. As a result, DevOps teams are constantly dealing with integration tasks and writing custom glue code to close the gaps. Of course, this also means they're spending less time automating their development environments and-let's not forget-actually writing code to improve the product. Throwing "more DevOps" at the problem clearly doesn't scale, especially since finding skilled DevOps engineers is borderline impossible.

Velocity and productivity will drive new DevOps approaches and tools

As a result, in 2022, we'll see a new class of DevOps solutions to provide better productivity for managing these new, highly distributed environments. And as IDC predicted, they'll have greater out-of-the-box integrations. What's more interesting, however, are four implications that arise from this trend:

1. Reduced IaaC development complexity for developers
2. Reduced barriers to entry for IaaC to non-developers through low or no code
3. Faster release cycles by "democratizing" the development environment from production
4. Streamlined continuous change management between IT and DevOps
   

We'll look briefly at at each of these:

1. Reduced IaaC development complexity for developers: IaC v IaS

The IaaC language war  - Infrastructure as code ( Terraform) vs Infrastructure as software (Pulumi)

What's the right approach to improving developer productivity? Consider the Hacker News discussion comparing Pulumi (IaS, or Infrastructure as Software) and Terraform (IaC, or Infrastructure as Code). HCL was designed by HashiCorp as a domain-specific language for Terraform, but it misses attributes you'd expect from a formal language. This leads to frustration among developers who can't express more complex business logic in HCL.

Pulumi responded by moving IaC into native code. This seemed logical at first, but things get messy at scale because none of those native languages were really designed to handle infrastructure management domains, which involve lots of parallel, distributed, and asynchronous tasks as well as handling partial failure scenarios.

Infrastructure automation is a specific domain, and it requires a domain-specific language. Pulumi founder Joe Duffy seems to agree in the Hacker News thread:

A "default language for IaC" may emerge with time, but I suspect that is more likely to be Python than, say, HCL. ...I also won't discount the idea of us introducing a cloud infrastructure-specific language ;-)

I believe that HCL is here to stay. It has a critical mass of adoption so the right approach is to augment HCL with tools like Terragrunt and structura.io. In 2022 we'll see a growing set of frameworks (Cloudify included) that augment Terraform rather than attempting to replace it.

In short, there won't be one language to rule them all, at least not any time soon. This will drive demand for better interoperability and integration among orchestration domains. I expect we'll see new DevOps frameworks geared specifically to saving all the custom glue code in today's IaC environments.

Having said that, HCL will live side by side with AWS CloudFormation and Azure ARM which comes with an inherent benefit of supporting the entire cloud resources. This is in contrast to HCL, where you are always dependent until an external provider implementation catches up with their API. I also believe that Ansible will continue to play an important role in managing the configuration of services, and that Helm will continue to act as a packaging template for Kubernetes services. The reality of having more than one orchestration language to manage IaaC environments isn't going to change either, as each one of those languages does a better job of addressing the needs of its own domain.

2. Reduced barriers to entry for IaaC to non-developers through low or no code

Another approach to simplifying IaC complexity is to allow non-developers to take part in infrastructure automation. We've seen this trend before with workflow automation, which also started as a developer tool. Later, tools like Monday, ServiceNow flow designer, and IFTT simplified writing workflow automation for non-developers through canned integration and graphical interfaces. Structura.io Is taking a similar approach, and so is Cloudify.

In 2022 we're likely to see new Low/No code tools to simplify DevOps automation development. Those tools will combine the No Code and IaC approaches, allowing users to move from one model to the other interchangeably during development cycles. In addition, we'll also see closer integration of the IaC tools into IT and business workflow management.

3. Faster release cycles by "democratizing" the development environment from production

The time it takes to create a development and testing environment has a major impact on agility, as it affects the number of tests that can be run on a given day. This in turn affects the time required to release a new feature. It also impacts software quality, as development environments are very different from production. Development environments usually run for a limited amount of time, they need to be optimized for quick troubleshooting, and they don't need rigorous SLAs and security.

Having said that, it is quite common to see that most development teams use production tools to run their development teams. The main reason is that it's quite complex to maintain more than one flavor of the same environment, as it multiplies maintenance overhead.

Kubernetes provides a degree of abstraction from infrastructure that opens new ways to create different flavors of the same environment to meet different development and production requirements. For example, we can use a Kubernetes cluster with Minikube for development and a managed Kubernetes service like EKS for production just by changing the Kubernetes endpoint in our pipeline. Having said that, a development environment often consists of a more complex stack that includes database, storage, network, and more. This still makes the overhead of maintaining multiple flavors of the same environment fairly substantial. In 2022, we'll see tools emerge that provide a simpler way to create different flavors of the same environment.

4. Streamlined continuous change management between IT and DevOps

The IT and DevOps automation change management process has evolved in almost opposite directions. IT change management is driven through ITSM such as ServiceNow, and it tends to be more human-driven through ticketing and approval systems. DevOps change management, by contrast, was designed to allow full automation (continuous deployment) with minimal human intervention and fit into the current software development cycle. The most notable trend here is GitOps, which aims to streamline the release cycle process by integrating it with the software development cycles through Git.

In 2022, we'll see ITSM take a major shift left towards better alignment with DevOps automation processes like GitOps. And as with the rest of the Shift Left movement, I believe that 2022 will bring new tools for closer integration between ITSM and DevOps automation to allow more streamlined change management processes between the two worlds. This will address IT needs for better visibility and governance automation processes but without adding friction to the development and release cycles. This in turn will speed up delivery of certified environments. Shay Naeh has a great example of how this works.

Final notes - Different time calls for different measures

When we look at the current DevOps automation landscape over the past couple of years we can see that not much has really changed since we started the automation and cloud transformation journey. Our toolchain is mostly centered around CI/CD, IaaC, Ansible, and Kubernetes as the main layers of our stack. Those tools were not built to deal with many of the new challenges of cloud-native and distributed architecture, simply since those challenges didn't exist at the time they were written. It is time to think beyond the current toolchain and start to move up the stack-literally speaking-in order that we may address the next set of challenges. Cloudify's approach to Environment as a Service (EaaS) is one such solution, and we plan to be in the thick of it as organizations see the new wave of DevOps automation tools.

 ##

ABOUT THE AUTHOR

Nati Shalom is the CTO and founder of Cloudify.io. He is a serial entrepreneur and widely published thought leader and speaker in open source, multicloud orchestration, network virtualization, DevOps, and edge computing. Nati has received multiple recognitions including YCombinator and is one of the leaders of the Cloud Native and DevOps Israel Meetup groups.