What is the problem that Nodeless is trying to solve?
Kubernetes capacity planning on public cloud
is hard - average utilization of Kubernetes compute on public cloud is 20%.
Cost of over and under provisioning compute capacity is high, and operational
complexity of plethora of pet worker node options (on-demand, spot, CaaS)
across various cloud providers is overwhelming.
What is Nodeless?
Instead of hand curating pet worker nodes,
Nodeless enables you to manifest cloud compute for your pods in lock-step with
pod lifecycle. Vanilla kubernetes bin
packs large worker nodes with pods of varying sizes (resource
requirements). Nodeless bin selects optimal bin for each pod. A pod's bin
can be optimized along various axes: resource requirements, pod SLA, compute
cost, cloud vendor preference, etc.
What is the catch?
Nodeless eco-system has the following big
ticket items that are not yet supported (work in progress).
Persistent
volumes: Pending implementation.
Daemon
sets: Daemonsets come with the implication that you
are bin packing nodes with pods. They represent products that were designed in
the private datacenter era where you had large static server blades that needed
one agent (for logging/monitoring/security/etc) per server blade.
In a nodeless world, since there is one
compute launch type per pod, the intention of daemonset translates to the
functionality of a sidecar. One way to execute the intent is to have an
admission controller that converts daemonset to sidecar. The caveat of this
approach is that some commercial offerings that are implemented via daemonset
charge per node (ex: Datadog, Instana). Charging per node translates to
charging per pod in a nodeless world which is cost prohibitive. Nodeless
community aims to work with such daemonset-based products to arrive at a
pod-based pricing model in a nodeless world.
Affinity
and Anti-affinity rules: Similar to daemon sets,
affinity and anti-affinity rules assume bin packed nodes. In a nodeless world,
you get anti-affinity out of the box resulting in stronger multi-tenant
security by design! Intent behind affinity rules can be manifested by having an
admission controller that creates a single "nodeless pod" consisting of all the
pods belonging to a single affinity rule.
What kind of workloads are best
suited to run in a Nodeless environment?
CICD workloads like Jenkins and Buildkite jobs
are best first fit for Nodeless. Since these workloads are time bound, bursty,
stateless, and (relatively) low risk, they are ideally suited for trying out
Nodeless.
What are my options for trying
Nodeless?
Your best nodeless fit depends on your
business and technical needs. If you do not have special needs for a bespoke
stack wrt {CNI, CSI, security, logging, monitoring}, cloud provider options
like EKS with Fargate on AWS and AKS with virtual nodes on Azure are a good
start.
If you have technical and/or business needs
that are unmet by bespoke stack (ex: custom networking/storage/security
requirements), or, if you want to step away from cloud vendor lock in, nodeless
setup using virtual-kubelet
with a cloud vendor neutral provider is a better option.
How does Nodeless interop with my
preferred {storage, networking, security, logging, monitoring} stack?
Nodeless interop with your custom stack
depends on which Nodeless model you pick. For example, virtual-kubelet with
Kloud Instance Provider gets you {CNI, Istio, logging, Prometheus} support out
of the box, but is missing CSI support as of KubeConEU 2020.
Where can I learn more?
Blog Posts: https://medium.com/elotl-blog
KubeCon talk: Virtual Kubelet KubeCon 2019 talk
I have a Nodeless question. Where
do I ask?
Please share your questions/comments in
Nodeless community channel #virtual-kubelet in the Kubernetes slack!
##
About the Author
Madhuri Yechuri, CEO, Elotl
Madhuri Yechuri is a systems engineer with
20 years experience in database server technologies (Oracle), virtualization
(VMware), and container technologies (ClusterHQ) before founding Elotl. Madhuri
received her Masters in Computer Science from Indiana University Bloomington,
and Bachelors in Computer Science from Indian Institute of Technology
Kharagpur. https://www.linkedin.com/in/madhuri-yechuri-4111461/