Written by Damien DUPORTAL
KubeCon Seattle 2018 was a place of great exchanges with the community,
providing a great opportunity to meet people face to face and discuss without
timezone or sound issues.
With the opportunity of the various
SIGs meetings and the "booth crawling", we had a great exchange with Puja
Abbassi (Giant Swarm) and Ross Kukulinski (Heptio) during the KubeCon US 2019
at Seattle, about the Kubernetes "Ingress" objects.
1 - Challenges of the Kubernetes Ingresses
An "Ingress" is a Kubernetes object
describing HTTP and HTTPS routes to guide the requests from outside to the
services inside the Kubernetes cluster. It also allows virtual-name hosting,
TLS termination and much more (Official Reference: https://kubernetes.io/docs/concepts/services-networking/ingress/#what-is-ingress).
An "Ingress Controller" is a
software appliance, deployed and managed in Kubernetes, responsible for
implementing the "Ingress" rules. There are plenty of these controllers, and they are non-mutually exclusive.
This is the challenge #1: which
Ingress to use on your Kubernetes cluster? How to make sure to select the best
one at 1st try?
The "Ingress" object provides
functionalities like "forward incoming https request for the hostname
‘company.org' and path ‘/backend' to the service named ‘backend' on the port
8080", that are implemented by each controller. Without any doubt, your choice
of Ingress Controller is based on differentiating features: it means you'll
have to configure these features, outside the default model of the "Ingress
object".
As underlined in the documentation
of Contour (https://github.com/heptio/contour/blob/master/docs/ingressroute.md), these "additional" configurations are done through Kubernetes Annotations:
a collection of key/values providing configuration items for the specific or
advanced features of your controller. The result is a bloat of annotations,
hard to manage and validate: 87 annotations for Nginx
or 62 annotations for Traefik v1x ...
Here comes the Challenge #2:
switching from an ingress controller to another one is not only changing the
value of the annotation "ingress.class".
It's also listing exhaustively all additional features and try to convert to
the new ingress configuration format. If you need 2 or 3 different ingress
controllers for different requirements, your developers will have a hard time
switching when their needs are changing.
2 - Solving Challenges with Kubernetes Custom Resources (C.R.D.)
Kubernetes Ingress have challenges
in their current form. OK.
What are the next steps?
Ross
Kukulinski from Heptio pointed us to the documentation of Contour as a support
to the discussion. The goal was to underline these challenges, and explain how
Heptio is proposing to solve it using a Kubernetes object named "Custom Resource Definition" ("CRD").
As
the Traefik community are currently working (hard!) on the v2.0 of Traefik, the
timing of this discussion was perfect. It was obvious that, if we want to bring
new features without swarming our Kubernetes users with a bunch of annotations,
we had to follow the "CRD" path.
At
its core, a "Custom Resource" is an extension to the Kubernetes API to store
and retrieve structured data in the Kubernetes API. They are different ways to
extend the API, as underlined in the Kubernetes documentation, but you can see that as a way to define your own objects
to be manipulated with `kubectl` for example: `kubectl get coffee-cups`.
The
power of the "Custom Resources Definition" resides in the "Definition":
Kubernetes users can declaratively define their own configuration objects as
well as how they already do with Deployment, Services, DaemonSet, etc.
Once the new Traefik v2 is installed as Ingress
Controller in your Kubernetes cluster, you can configure a new kind of object
named "IngressRoute":
However,
moving away from annotations to CRD is adding another challenge: what about the
end user who only requires the current Ingress functionalities? Maybe one day
their requirements will shift and imply a CRD. But for simple use cases,
Ingress in its current form is still simple.
This
is why the "normal" ingress should be maintained, without CRD involvement, for
simple use cases, but with no annotation support to draw the line.
Enough
about Traefik: you can read more about the "Middleware" CRD in the documentation
or the announcement blog post.Time
to speak "community challenges".
3 - Kubecon Is a Community Event Triggering Discussions
To
summarize:
- A market competitor, Heptio, started
a discussion with the community about Kubernetes pros and cons in our market
area, the "ingresses".
- We started to go on the same
direction to solve the same issue, but we went on a slightly different path,
based on our specificities (different CRD objects, standard Ingress support,
Traefik specific configuration)
This is the beauty of the community:
Market competitors helping each other on common features, providing additional
value to the end user.
What would the next steps be?
As
the KubeCon EU 2019 is coming in a few weeks, our responsibility is to give back to the
community what we learn: starting another discussion to catch the (high-speed)
train and challenge the others about the future of Ingress.
Such
discussion should underline a subset of features shared by all ingress
controllers, that should be put on the Kubernetes ingress "new generation". But
it also should build the foundations on "how to extend Ingress", where the
challenge is to provide enough space for every competitor to provide innovative
features, while keeping the end user happy and not locked-in to one Ingress
controller.
So let's meet during the KubeCon, during the booth crawling,
on the Kubernetes network SIG
or on any location of the conference to share tea, coffee, cupcakes!
To learn more about containerized infrastructure
and cloud native technologies, consider coming to KubeCon + CloudNativeCon Barcelona, May 20-23 in Barcelona.
##
About the Author
Damien DUPORTAL, Traefik's Developer Advocate at Containous and freelancer. Former Training Engineer at CloudBees. Docker mentor. Human stack focused. Rock climber. Passionate. Talk to me!
Contact: damien@containo.us | https://twitter.com/DamienDuportal | https://github.com/dduportal