Network infrastructure is radically different in the world of containers and cloud-native than it was in the prior decade of virtual machine infrastructure. And there’s a big leap when you go from deploying Kubernetes to a single cluster, to trying to network containers and microservices across multiple clusters. VMblog was excited to connect with Nico Vibert from Isovalent to learn more about his new “Kubernetes Networking and Cilium for the Network Engineer” eBook, designed to help enterprises navigate these new distributed computing networking concepts.
VMblog: How has networking changed in cloud-native, compared to the old world of virtual machines?
Nico Vibert: There are actually a lot of similarities – and I am sure we’ll come to them when we start talking about Cilium – but one of the primary differences between networking for virtual machines and networking for Kubernetes is that IP addresses have become absolutely irrelevant in the cloud native space.
Looking at the time when I used to manage physical networks, you’d remember the IP addresses of some of your core servers. You could also tell, from their IP address, in which rack the server was located and from which business group it belonged to. And firewall rules were built on IP addresses – in other words, IP addresses were also identities.
When we moved to virtual machines, you would still massively rely on IP addresses. To vMotion your machine from one host to another seamlessly required you to preserve the IP address of the machine as it moved across the data center.
In Kubernetes – IP addresses are meaningless. You have no say over which IP address is allocated. Containers are spawned and destroyed and pick up random IP addresses. Pods in different namespaces – used in Kubernetes to logically isolate resources – can be part of the same network. This means we need to be smarter about network security – creating firewalls rules based on IP addresses would be pointless.
VMblog: What are the hardest new skills to learn with Kubernetes, from a networking perspective?
Vibert: Outside the terminology (it’s vastly different from traditional networking), visualizing how microservices communicate is one of the toughest challenges. Pre-virtualization, you could literally walk around the data center and follow the cable from one physical server’s network interface to another. Of course, virtualization and then containerization introduced new layers of abstraction. If two Kubernetes workloads on different hosts are having trouble communicating, the issue could be related to any of the multiple underlying networking layers, including within the Linux networking stack.
It is one of the reasons Cilium’s built-in network observability tool Hubble has become so popular: providing a map of how pods communicate, even when located in different clusters or regions, helps operators tremendously.
VMblog: What can you tell VMblog readers about the general opportunities around Cilium for cloud-native networking? Why is the technology so interesting?
Vibert: In many ways, the trends your readers would have seen in virtual machine networking still apply in Kubernetes and the cloud native space.
Let’s rewind: initially, virtual networking was simply about connectivity for virtual machines. Quickly, we realized the need to secure and scale our applications so we started deploying virtual firewalls, virtual load balancers, virtual VPNs, etc… We ended up having to manage multiple tools from various vendors. Eventually, network virtualization technologies like VMware NSX were introduced to support all virtual networking needs and eliminate the need for 3rd party tools.
Kubernetes followed the same pattern: initially, the focus of container network interface plugins (also known as CNIs) was to provide connectivity for Kubernetes pods and containers. It became quickly evident that additional tools were needed for intra-cluster, inter-cluster, and multi-cluster communications. Ask any Kubernetes engineers and they probably have 5 to 10 networking and security tools running in their cluster. That’s one of the reasons why Cilium is fascinating – it’s become a universal cloud native networking platform.
Anecdotally, when I joined Isovalent – the company behind Cilium – the creator of NSX and the godfather of software defined networking Martin Casado told me how Cilium would become something even bigger than NSX. Time will tell if he was right!
VMblog: Who is your eBook aimed at – what will it help them learn?
Vibert: Despite my 20-odd years working in the networking industry, I found the Kubernetes networking terminology baffling. It vastly differed from the terms we used in traditional data center networks.
I found that all the material and content around it had not been created with network engineers like me in mind.
Ultimately, even though Kubernetes was clearly taking off, I ended up exploring other networking areas – like cloud networking and networking automation – as I found them more approachable.
My aim for this eBook was to provide network engineers with the book I wish I had five years ago.
The idea was to provide parallels between concepts they are already familiar with and the Kubernetes networking model. For example – is there a DHCP server for my Kubernetes pods? Where is my web proxy? How do I interact with the Kubernetes network – is there a Cisco IOS-like CLI I can use?
By explaining Kubernetes networking in terms network engineers will be familiar with, I hope it will help them avoid some of the challenges I faced.
VMblog: Were there any other interesting gotchas or surprises learned when you wrote the eBook?
Vibert: The truth is it wasn’t meant to be an eBook – initially, we wanted to only write a 5-pager brief explainer of Kubernetes networking concepts. But the topic was too complex to stay succinct. The first draft was 75 pages long! We eventually cut a couple of chapters out, which we’ll probably publish in a subsequent volume.
Overall, what I came to realize as I was writing this is that all networking models are converging.
It wasn’t hard to come up with parallels because all modern networks follow an API-driven GitOps-based, declarative model with a centralized control plane.
Whether you’re deploying cloud networks in AWS with tools like Terraform/OpenTofu, or embracing a NetDevOps approach in your own data center by using tools like Ansible or indeed using Kubernetes networking, you are pushing a network configuration expressed in code – primarily YAML – to a central API and you then expect the network to implement your intent.
##
