For modern organizations needing to build and operate digital platforms at scale, the proliferation of stand-alone Kubernetes clusters has significantly increased operational complexity, making it difficult to deliver desired efficiency gains. To understand how virtual Kubernetes clusters empower platform teams, I spoke with Lukas Gentele, co-founder and CEO of Loft Labs. Loft Labs recently announced the availability of its vCluster integration with Rancher, enabling self-service virtual Kubernetes cluster creation and management for teams using Rancher for Kubernetes management.
VMblog: Kubernetes is widely adopted in enterprises, but building on it remains difficult. What are some of the key challenges faced by organizations today as they attempt to deliver efficient, reliable platforms?
Lukas Gentele: Despite its widespread adoption, there are certainly significant challenges to build on Kubernetes. It has a steep learning curve due to its complex architecture, and managing resources requires incredibly careful planning and optimization. The sheer number of stand-alone Kubernetes clusters that enterprises are managing has become a huge operational obstacle for enterprises seeking efficiency. Trying to manage these clusters, all of which are configured differently, is a nightmare for platform teams. While Kubernetes adoption has enabled many businesses to optimize resource usage, I will zero in on the challenges in implementing secure Kubernetes multi-tenancy.
One major burden is that Kubernetes does not offer multi-tenancy by default. When operating a Kubernetes multi-tenant architecture, achieving cluster isolation and optimal resource allocation is onerous. By implementing multi-tenancy, you automatically impose limitations on the tenants. While the tenants still have access to many resources within their namespaces, they do not retain the complete control and capabilities they had with single-tenant clusters.
Another potential issue is the “noisy neighbor” problem. This happens when a tenant takes up excessive shared resources, leading to less-than-optimal performance for other tenants. This problem can slow down workloads, and sometimes even prevent them from running altogether.
With virtual clusters, on the other hand, isolation is strong and it is straightforward to build a production-grade Kubernetes platform. Platform operators only need to manage the small number of underlying Kubernetes clusters, while developers can spin up as many virtual clusters as they need.
VMblog: Can you talk more about virtual clusters? How does vCluster answer the demands of developers and enterprises?
Gentele: We like to think about virtual clusters as the next generation virtualization layer for the cloud. In the same way that a number of successful companies virtualized physical hardware to make “the cloud” possible, we are virtualizing Kubernetes to make clusters so lightweight, fast to spin up, and cost-effective to operate that any organization can create the Kubernetes clusters they need with ease.
Basically, virtual clusters are fully working Kubernetes clusters that run on top of other Kubernetes clusters. Compared to fully separate “real” clusters, virtual clusters reuse worker nodes and networking of the host cluster. They have their own control plane and schedule all workloads into a single namespace of the host cluster. Like virtual machines, virtual clusters partition a single physical cluster into multiple separate ones.
The virtual cluster itself only consists of the core Kubernetes components: API server, controller manager, storage backend and optionally a scheduler. To reduce virtual cluster overhead, vCluster builds by default on k3s, which is a fully working, certified, lightweight Kubernetes distribution that compiles the Kubernetes components into a single binary and disables by default all unneeded Kubernetes features, such as the pod scheduler or certain controllers. Virtual clusters offer many benefits, including more stability than Kubernetes namespaces, because the virtual cluster creates its own Kubernetes resource objects, which are stored in its own data store. The host cluster has no knowledge of these resources, and this isolation is great for resiliency.
Because you can have many virtual clusters within a single cluster, they are also much cheaper than the traditional Kubernetes clusters, and require less management and maintenance. This makes them ideal for running experiments, continuous integration, and setting up sandbox environments. I would also highlight that virtual clusters can be configured independently of the physical cluster. This is great for multi-tenancy, like giving your customers the ability to spin up a new environment or quickly setting up demo applications for your sales team.
VMblog: You announced this week the availability of vCluster for Rancher. What benefits does this integration provide for platform teams?
Gentele: This integration with Rancher offers users a number of key benefits. First, it provides simplified multi-cluster management for Rancher users. vCluster as a whole allows teams to create and manage isolated virtual Kubernetes clusters within a single host cluster. vCluster for Rancher simplifies managing multiple Kubernetes environments, as Rancher users can now provision, configure, and manage these virtual clusters directly through the familiar Rancher interface.
Next, Rancher users will see improved resource efficiency. vCluster utilizes the resources of the underlying host cluster for users’ virtual clusters, which eliminates the need for separate physical infrastructure for each Kubernetes environment. This leads to better resource utilization and significant cost savings.
I also want to highlight the enhanced isolation and security benefits of vCluster for Rancher. vCluster provides isolation between virtual clusters, ensuring workloads in one environment don’t impact another. This aligns with multi-tenant deployments or separating development, testing, and production environments.
And finally, this integration includes some important expanded platform capabilities. By integrating vCluster, Rancher users can broaden the capabilities of the Rancher platform with role-based access control and project management for virtual clusters – all from within the platform.
VMblog: Organizations using Rancher have already sought out a solution to the operational and security challenges of managing multiple Kubernetes clusters. What additional benefits can they expect from shifting to Loft virtual clusters?
Gentele: I believe that organizations employing virtual clusters will immediately see massive gains in efficiency and reduced complexity, no matter what solution they currently employ to manage traditional Kubernetes clusters. In particular, vCluster unlocks game-changing benefits for Kubernetes multi-tenancy. Teams can seamlessly build a production-grade multi-tenant Kubernetes platform with virtual clusters, as they provide a strong isolation for tenants on top of a shared Kubernetes cluster.
Using virtual clusters to partition a single physical cluster into multiple clusters allows teams to leverage all the benefits of Kubernetes, such as optimal resource distribution and workload management, with none of the downsides. While Kubernetes does provide namespaces for multiple environments, they are limited in terms of cluster-scoped resources and control-plane usage. Certain resources live globally in the cluster, and users cannot isolate them using namespaces. Engineers who use namespace-based isolation often still need access to cluster-scoped resources like cluster roles, shared CRDs or persistent volumes. If an engineer breaks something in one of these shared resources, it will likely fail for all the teams that rely on it. In addition, the API server, etcd, scheduler, and controller-manager are shared in a single Kubernetes cluster across all namespaces. Request or storage rate-limiting based on a namespace is very hard to enforce, and faulty configuration might bring down the whole cluster.
vCluster solves all of these issues, providing immense benefits and stability for large-scale Kubernetes deployments and multi-tenancy. Users can expect significant cost reductions as virtual clusters share the underlying host cluster; it is trivial to reuse and share services across multiple virtual clusters. It is also very simple to transition to vCluster. For example, virtual clusters run with the API server from any Kubernetes distribution that an organization is using, ensuring complete Kubernetes API server compliance.
VMblog: Is there anything else VMblog readers should know? What can VMblog readers expect from Loft Labs in the future?
Gentele: If you are at KubeCon EU this week, stop by and see us at Booth J13! We will be giving demonstrations of the new vCluster capabilities delivered in this release, and we would love to talk to you about all things virtual clusters and Kubernetes management.
Moving forward, we will have more exciting updates to Loft’s platform this year as we continue to make it easier to spin up and operate Kubernetes clusters. We see a future where anyone in any organization can create Kubernetes clusters on-demand, whenever they need them and on top of any underlying cloud infrastructure. Advances like this Rancher integration empower platform teams to realize the benefits of self-service environment provisioning and virtual clusters, and we are always adding more exciting integrations and capabilities. I invite everyone to join our Slack User Community to stay up to date with Loft’s progress and be part of an inspiring network of innovators.
##
Bio:
Lukas Gentele is the Co-Founder and CEO of Loft Labs, the leading provider of platform engineering building blocks. Loft Labs delivers Kubernetes-native tools, functionality and frameworks purpose-built for platform engineers to manage, activate and optimize their platform stack. Gentele is an experienced platform engineering innovator, committed to delivering solutions that empower platform teams to build seamlessly on Kubernetes. Prior to Loft Labs, Gentele served as the Co-Founder and CEO at covexo GmbH and Webmans. He holds a Bachelor of Science in Computer Science and Information Systems, and a Master of Science, Computer Science & Management of Enterprise Information Systems, both from the University of Mannheim.
