Compatible with Static Users, LDAP/AD and External IdPs
Multi-tenancy is a central feature of Kubeflow and providing
a flexible authentication scheme is important for enterprise use-cases.
In this article, we will explore how we leveraged the power
of Istio and open-source components to create a flexible, robust and clean
authentication solution.
This solution is especially powerful because:
-
It leverages Istio features and doesn't add
extra components.
-
It can work on any Kubernetes cluster and
doesn't rely on having access to the API-Server.
-
It is a general solution and can be applied to
other projects besides Kubeflow.
With that said, let's dive into the solution.
The
end-goal
We will focus on using Istio to perform end-user
authentication, meaning that our apps won't contain any authentication logic!
Authentication, for user access to an application, will be
done at the Istio Gateway: the one point where all traffic enters the cluster.
Authentication is a major area that developers may choose to
leave up to Istio. The idea is simple:
-
Incoming traffic includes a JSON Web Token (JWT)
for authentication.
-
The JWT is verified by the Istio Gateway.
-
Apps inside the cluster trust the JWT because it
has been verified by the Gateway.

Figure 1:
Istio Gateway enforces Auth for the Kubeflow apps
This way, our apps contain no authentication logic at all!
Unfortunately, it's not that simple.
Authentication
in Istio
At first glance, Istio seems to support end-user
authentication.
However, notice how Istio can only perform the last part,
token verification (i.e. verify the JWT and allow the request).
The question is: how are we going to get that token in the
first place?
Enter OpenID Connect
(OIDC): a way to authenticate a user using a standardized OAuth2 flow.
The OIDC
Flow
Because a picture is worth a thousand words, let's take a
look at what an OIDC flow looks like.
Figure 2:
The OIDC Flow - Istio Gateway only supports JWT verification
Notice how Istio can only perform the last part, token
verification.
Because of this, we need a new entity that will act as the
OIDC client and execute the flow.
Envoy - The
power behind Istio
As many of you will already know, Istio is mainly in the control
path.
Under the hood, the data is handled by Envoy, a very efficient and versatile
proxy.
Why is this relevant?
Istio lets you configure its underlying Envoy Proxies using
an EnvoyFilter
object.
The EnvoyFilter object enables us to insert Envoy
Filters in the data path of certain proxies.
In our case, we want to configure the Envoy Proxy of the
Istio Gateway.
But what filter should we use?
The
ext_authz HTTP Filter
The ext_authz
HTTP filter is an extremely powerful and versatile capability of Envoy.
Figure 3:
Example EnvoyFilter using ext_authz HTTP Filter
Its function is very simple:
Every request is forwarded to an Authorization Service
(AuthService). Depending on the answer, that response can be:
-
Accepted
-
Rejected
-
Redirected
Figure 4:
AuthService different responses
The OIDC
AuthService
We just need to implement the component that will handle
those actions.
For this purpose, we have built our own oidc-authservice
based on open source solutions.
Putting it
all Together
Our Istio Gateway can now act as an OIDC client and execute
the whole flow to authenticate a user.
Let's test it out using Dex, a popular OIDC provider.
Dex supports many authentication backends, including static
users, LDAP and external Identity Providers, so you can have the power of
choice.
The final architecture is the following:
Figure 5:
Overall architecture
Demo
All of the aforementioned functionality is available for
Kubeflow v0.6 and above.
At this time, the way to install this reference architecture
is to use the "kfctl_istio_dex" configuration file, to install on an existing
Kubernetes Cluster.
https://www.kubeflow.org/docs/started/k8s/kfctl-istio-dex/
Simply follow the instructions from the Kubeflow docs and
you should have a Kubernetes Cluster with Kubeflow, Istio and Dex for
Authentication.
Figure 6:
You should be greeted by the Dex login screen
Final
Thoughts
In this article, we unlocked the powerful feature of the
Envoy Proxy and used Istio along with Dex and the OIDC AuthService to form a
complete Authentication architecture. This enables applications to offload all
authentication logic to Istio and focus on the business logic, which works
great for Kubeflow's microservice-oriented architecture.
The ext_authz feature of Envoy is not very well-known within
the Istio userbase, however it is an incredibly powerful feature that enables
us to integrate with virtually any authentication scheme.
##
To
learn more about containerized infrastructure and cloud native technologies,
consider coming to KubeCon + CloudNativeCon EU, March 30-April 2 in
Amsterdam.
About the Author
Yannis Zarkadas is a software
engineer at Arrikto, working with Kubeflow. He loves contributing to open
source projects and has authored the Cassandra Operator in Rook and the
official Scylla Operator, which he is currently maintaining. https://twitter.com/yanniszark