Virtualization Technology News and Information
The Road to a Better Kubernetes Ingress

Written by Damien DUPORTAL

KubeCon Seattle 2018 was a place of great exchanges with the community, providing a great opportunity to meet people face to face and discuss without timezone or sound issues.

With the opportunity of the various SIGs meetings and the "booth crawling", we had a great exchange with Puja Abbassi (Giant Swarm) and Ross Kukulinski (Heptio) during the KubeCon US 2019 at Seattle, about the Kubernetes "Ingress" objects.

1 - Challenges of the Kubernetes Ingresses

An "Ingress" is a Kubernetes object describing HTTP and HTTPS routes to guide the requests from outside to the services inside the Kubernetes cluster. It also allows virtual-name hosting, TLS termination and much more (Official Reference:

An "Ingress Controller" is a software appliance, deployed and managed in Kubernetes, responsible for implementing the "Ingress" rules. There are plenty of these controllers, and they are non-mutually exclusive.

This is the challenge #1: which Ingress to use on your Kubernetes cluster? How to make sure to select the best one at 1st try?

The "Ingress" object provides functionalities like "forward incoming https request for the hostname ‘' and path ‘/backend' to the service named ‘backend' on the port 8080", that are implemented by each controller. Without any doubt, your choice of Ingress Controller is based on differentiating features: it means you'll have to configure these features, outside the default model of the "Ingress object".

As underlined in the documentation of Contour (, these "additional" configurations are done through Kubernetes Annotations: a collection of key/values providing configuration items for the specific or advanced features of your controller. The result is a bloat of annotations, hard to manage and validate: 87 annotations for Nginx or 62 annotations for Traefik v1x ...

Here comes the Challenge #2: switching from an ingress controller to another one is not only changing the value of the annotation "ingress.class". It's also listing exhaustively all additional features and try to convert to the new ingress configuration format. If you need 2 or 3 different ingress controllers for different requirements, your developers will have a hard time switching when their needs are changing.

2 - Solving Challenges with Kubernetes Custom Resources (C.R.D.)

Kubernetes Ingress have challenges in their current form. OK.

What are the next steps?

Ross Kukulinski from Heptio pointed us to the documentation of Contour as a support to the discussion. The goal was to underline these challenges, and explain how Heptio is proposing to solve it using a Kubernetes object named "Custom Resource Definition" ("CRD").

As the Traefik community are currently working (hard!) on the v2.0 of Traefik, the timing of this discussion was perfect. It was obvious that, if we want to bring new features without swarming our Kubernetes users with a bunch of annotations, we had to follow the "CRD" path.

At its core, a "Custom Resource" is an extension to the Kubernetes API to store and retrieve structured data in the Kubernetes API. They are different ways to extend the API, as underlined in the Kubernetes documentation, but you can see that as a way to define your own objects to be manipulated with `kubectl` for example: `kubectl get coffee-cups`.

The power of the "Custom Resources Definition" resides in the "Definition": Kubernetes users can declaratively define their own configuration objects as well as how they already do with Deployment, Services, DaemonSet, etc.

Once the new Traefik v2 is installed as Ingress Controller in your Kubernetes cluster, you can configure a new kind of object named "IngressRoute":

Traefik IngressRoute 

However, moving away from annotations to CRD is adding another challenge: what about the end user who only requires the current Ingress functionalities? Maybe one day their requirements will shift and imply a CRD. But for simple use cases, Ingress in its current form is still simple.

This is why the "normal" ingress should be maintained, without CRD involvement, for simple use cases, but with no annotation support to draw the line.

Enough about Traefik: you can read more about the "Middleware" CRD in the documentation or the announcement blog post.Time to speak "community challenges".

3 - Kubecon Is a Community Event Triggering Discussions

To summarize:

  • A market competitor, Heptio, started a discussion with the community about Kubernetes pros and cons in our market area, the "ingresses".
  • We started to go on the same direction to solve the same issue, but we went on a slightly different path, based on our specificities (different CRD objects, standard Ingress support, Traefik specific configuration)

This is the beauty of the community: Market competitors helping each other on common features, providing additional value to the end user.

What would the next steps be?

As the KubeCon EU 2019 is coming in a few weeks, our responsibility is to give back to the community what we learn: starting another discussion to catch the (high-speed) train and challenge the others about the future of Ingress.

Such discussion should underline a subset of features shared by all ingress controllers, that should be put on the Kubernetes ingress "new generation". But it also should build the foundations on "how to extend Ingress", where the challenge is to provide enough space for every competitor to provide innovative features, while keeping the end user happy and not locked-in to one Ingress controller.

So let's meet during the KubeCon, during the booth crawling, on the Kubernetes network SIG or on any location of the conference to share tea, coffee, cupcakes!

To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon Barcelona, May 20-23 in Barcelona.


About the Author

Damien DUPORTAL, Traefik's Developer Advocate at Containous and freelancer. Former Training Engineer at CloudBees. Docker mentor. Human stack focused. Rock climber. Passionate. Talk to me!

Contact: | |

Published Tuesday, May 07, 2019 10:03 AM by David Marshall
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<May 2019>