Virtualization Technology News and Information
Operators: A Journey from Helm to Golang to Deliver on Cloud-native Applications' Day-2 Operations
By Joe Leslie, Senior Product Manager at NuoDB

For several years now, containers and containerization have dominated application development and software delivery conversations, along with process enablers such as CI/CD pipelines. While installing a containerized application can be fairly easy, when it comes time to configure access permissions, make backups, apply security patches, or apply updates, everything becomes more challenging. When you add in the operational knowledge needed to make all these changes and the business knowledge necessary for your application to function as intended, compounded with actually scaling your applications in the cloud, it becomes much more difficult to deploy everything as intended. This is why Kubernetes Operators have become such a hot topic for the last year or so. Operators build in that operational knowledge and business logic into the application package in Kubernetes, significantly simplifying provisioning, deploying, managing, and scaling applications in the cloud.

Kubernetes has been able to logically scale container workloads fairly easily when it came to stateless applications that didn't require much context from the services around them, but stateful applications and services remained a challenge. Using Kubernetes, stateless containers can start and stop at any time and run on any node in the cluster. Stateful containers just aren't as flexible. This is partly because state information, which is typically stored on disk, needs to persist and be accessible on any node in the event the container is restarted or moved to another node, and that's been another challenge.

As Kubernetes continues to mature, features such as container-native storage solutions continue to improve. The use of Kubernetes persistent volumes also allows storage vendors such as OpenEBS, LinBit, Portworx, and many others to easily integrate their solution into Kubernetes via a Kubernetes storage class definition. These new storage options are feature-rich and game-changers for storing data - but when deploying enterprise SQL applications, you still need a relational database to access and organize the data. To work in containers and the Kubernetes orchestration environment, you'll need a database that deploys a fault-tolerant architecture by running multiple processes across physical locations simultaneously without any data loss or consistency issues. In other words, you need a cloud-native database.

Using Kubernetes Operators, it's much simpler to manage the complexities of provisioning, configuring, managing, and autoscaling a custom configurable database. Because of the considerable functionality offered by Operators, the team at NuoDB decided to start building Operators to deploy our distributed SQL database. 

Helm to Kickoff Our Certified Operator

Helm is an ideal tool to quickly package and deploy an Operator without having to write Go code. To get started, you can use the Helm Operator kit from the Operator Framework to package, deploy, and manage your Kubernetes application. Using the Helm Operator, we were able to rapidly automate and simplify Day-1 application provisioning tasks that are common when standing up applications in Kubernetes orchestrated environments. Helm also made it simple for us to customize the deployment options for our database and then start up a sample SQL workload in order to demonstrate and visualize how the database performs and scales under various changing client SQL workloads. As this Red Hat Operator Maturity Model shows, Phase I doesn't require a custom Operator but delivers the initial functionality you need to create a stable, auditable, and repeatable deployment process.


Once we put the final polish on our Helm Operator, we published it to Operatorhub earlier this year, which is accessible from within Red Hat's OpenShift Container Platform 4. Next, it was time to move beyond provisioning and take full advantage of the functionalities offered by creating a custom Operator in Golang. In short, we were ready to implement day-2 operations - such as patching and upgrading the database, managing the database lifecycle events, delivering backup and recovery functionality, providing metrics and visual analysis, and being able to let the application run itself. In our case, that meant starting with the ability to deploy on-premises visual monitoring.

Ready for the Next Phase

Building a Golang Operator offered dramatically more user control and task automation when running a stateful app such as a distributed SQL database in the Kubernetes environment, including easing the management effort associated with common Day-2 operational tasks for our database, such as:

●      deploying the NuoDB Insights visual monitor

●      backing up and recovering a database

●      managing and reporting on rolling upgrades

●      autoscaling transaction engines

With a lot more functionality available to deliver, we found that there were many more choices to make, and a lot more configurations and knobs necessary for the new Operator to control. For starters, in a cloud-native world, we know that failure events will happen. In fact, failure events are common. We expect software, hardware, or network failures to occur, and so we must design for failure events.

Building Out with Go

Once we knew we were ready to move from Helm to Go, we started to build out some of the essential parts needed. We began by automating the deployment tasks of certain Kubernetes functions our product relies on that differ slightly between various Kubernetes distributions, which allows our single Operator to work seamlessly in a variety of Kubernetes distros. This level of customization wasn't available before. Next, we were able to write the programming necessary to deploy NuoDB Insights visual monitoring "on-cluster" (which is a Phase IV capability in the maturity model above).

All of the Operator phase II and beyond capabilities require interactions with other pieces, so using Go, we were now able to install Elasticsearch to store performance metrics and enable easy-of search capability for our database. We also added a connection with Logstash to ingest and transform the data, and Grafana for flexible query of real-time and historical performance and application workload data - combining them into a single easy-to-use visual performance monitoring and analytics dashboard. This allows NuoDB application and database admins to easily track NuoDB memory, CPU, and storage resource usage and consumption as well as the SQL application workload that it's processing in SQL transactions per second (TPS).

Our new Operator also allows us to check on the status of previous steps to make sure that everything started correctly before moving on. This enables and clears the path for us to perform more complex tasks reliably, like seamless rolling upgrade and fully managing backup and recovery events automatically.

Get Started with Operators

For any organization trying to decide how to tackle Kubernetes, start with a Helm Operator to get your feet under you, and see how easy it is to automate. Next, get ready to build out your Golang Operator to really take full advantage of all the capabilities that an Operator can offer. While many companies haven't yet moved to Kubernetes, or even containers, the evolutionary technology of Operators automates many of the challenging tasks organizations have struggled with. These new capabilities may well speed your decision to deploy Kubernetes sooner than later.

To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon NA, November 18-21 in San Diego.


About the Author

Joe Leslie, Senior Product Manager at NuoDB

Joe Leslie 

As Senior Product Manager at NuoDB, Joe Leslie drives the NuoDB product strategy and roadmap for deploying NuoDB in container-native computing environments such as Kubernetes. Joe works closely with the NuoDB Engineering and Marketing teams to ensure NuoDB's database leadership position in the distributed SQL database marketplace, where NuoDB focuses on delivering horizontal scale out and continuous availability for hybrid cloud applications. Joe has over 20 years of experience delivering database products and management tools in the transactional and analytical database marketplace.

Published Monday, November 04, 2019 7:33 AM by David Marshall
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<November 2019>