Virtualization Technology News and Information
Kubernetes security best practices: Learn how a managed Kubernetes offering on cloud can help

This paper provides an overview of best practices for hardening your Kubernetes clusters and how you can easily achieve them using a managed Kubernetes offering. We've included our learnings from operating large-scale hardened Kubernetes clusters across Oracle, working with our customers, analysts, and based on industry best practices, such as the NSA and CISA Kubernetes Hardening Guidance.

We cover best practices across the following topics and how you can achieve them:

  • Isolate the network access.
  • Control access.
  • Encrypt all data and rotate keys.
  • Keep underlying software secure.
  • Validate the deployed software and scan for vulnerabilities.
  • Keep calm and observe.

Isolate the network access

Access to your cluster components must be restricted to individuals or systems that require access to the resources. The network design must be tailored to your requirements but include the appropriate level of isolation of the Kubernetes cluster infrastructure, the APIs to access the control planes, and the individual pods.

Lock down access to cluster control planes and worker nodes

Design the Kubernetes clusters so that only the individuals or systems that need to access the cluster control planes or worker nodes through the network can access it. You can achieve this restriction by implementing firewall security rules to authorize only the minimum network traffic necessary to operate.

Cloud providers automate the provisioning of Kubernetes clusters with fully isolated infrastructure, often fully managing the cluster's control plane. This automation makes it easy for you to use isolated infrastructure without networking administrators and expertise.

Restrict access to the Kubernetes API

The cluster Kubernetes API endpoint is the main entry point to deploy applications, check state, or perform other cluster operations. We often consider authorization policies, but restricting access to this endpoint starts with restricting the network traffic to the endpoint. You can configure the Kubernetes API endpoint as part of a restricted subnet in your network.

Cloud providers usually offer the ability to configure the Kubernetes API endpoint as part of a virtual network that peers with your company's network or restrict it to a private subnet that can only be accessed through a bastion host. You can also configure the Kubernetes API endpoint on a public subnet, enabling cloud services, such as a continuous integration-continuous delivery (CI/CD) service, to access it. When using a public subnet, we recommend restricting access to certain known network IPs. Most cloud providers have managed services that enable these scenarios without the need to understand networking concepts.

Control the pods communication

Network policies allow you to control the traffic between pods or network endpoints within your cluster. They act as firewalls within your cluster. Application deployments can use network policies to isolate pods from other applications and namespaces.

Configure your Kubernetes clusters with a network policy CNI to apply network policy rules, which are otherwise ignored. Many cloud providers support network policies CNI as part of the Kubernetes cluster that they provision.

Control access

Kubernetes clusters support anonymous login by default. We recommend disabling this feature, so the cluster is configured with strong user authentication. Start with the principle of least privilege, and authorize access to each component, user, or program only the necessary information and resource to accomplish the task.

Most cloud providers integrate the Kubernetes cluster with an Identity and Access Management (IAM) service that lets you control who has access to the cluster and integrates with Kubernetes role-based access control (RBAC) to control what users can do in the cluster. With this control, you can easily configure administrator user roles and apply them across multiple clusters. As an extra layer of security, some cloud providers let you enable multi-factor authentication (MFA) as part of their IAM service.

Encrypt all data and rotate the keys

Keep your secrets secret

Avoid confidential data in your application code or configuration files by using Kubernetes secrets. Kubernetes secrets are stored in the cluster control plane etcd. The encryption key for these secrets is still stored in plain text, so we recommend storing that in a separate secrets store.

Most cloud providers allow you to encrypt your encryption key, offering a second-level encryption key that can be stored in a managed key management service. Key management services offer secret storage and rotation in highly available and durable hardware security modules (HSM) that meet Federal Information Processing Standards (FIPS) 140-2 Security Level 3 security certification.

Is it paranoia to encrypt all your data?

We don't think so. Encrypt the data of your stateful applications at rest and in-transit. Cloud providers offer managed block and file storage services with Kubernetes and encrypt data at rest by default. You can also choose to encrypt this data using encryption keys that you manage and apply cloud security services to manage and rotate these keys.

Keep the underlying software secure

Keeping Kubernetes and the operating system up to date enables you to stay ahead of security vulnerabilities. A good practice includes patching your clusters at least once a quarter. We recommend treating your cluster worker nodes as non-mutable entities that are regularly replaced with the latest images.

Cloud providers make this process easy by providing automatic updates and an automated cordon-drain procedure.

Validate the deployed software and scan for vulnerabilities

Start with signed and verified images and scan regularly

An automated continuous integration and continuous delivery (CI/CD) pipeline to build container images, push to a registry, and deploy to Kubernetes clusters is a good start. You can implement checks in this process to ensure that the container images have the following characteristics:

  • They're free of known critical vulnerabilities that can cause an accidental system failure or result in malicious activity.
  • They haven't been modified since they were published to maintain their integrity.
  • They're only deployed to a Kubernetes cluster and come from a trusted source.

Cloud providers provide container image scanning, signing, and verification. Vulnerability checks are done against the NIST National Vulnerability Database (NVD) and MITRE CVE databases. You can sign images and verify that the image signatures have not changed, ensuring that the integrity of the image has not been compromised. Finally, you can enable the cluster admission controller to accept deployments that use only signed images.

It's a good practice to periodically run vulnerability scans and penetration tests on your overall infrastructure. Many cloud providers offer the ability to monitor the containers for unusual behavior as a cloud service.

Run the applications with the least privilege

We recommend that you run applications in your Kubernetes cluster with non-root user privileges on immutable filesystems, reducing the blast radius if your container gets compromised. As an extra layer of security, we recommend enabling SELinux or AppArmor on the cluster worker nodes, which enables more security context settings to apply to your pods. Pod security policy (PSP) is a built-in admission controller that allows cluster administrators to enforce that pods are configured with a minimal set of security settings (set with the pod's security context). PSP is being deprecated in Kubernetes 1.21. The Kubernetes guideline says to keep PSP simple while waiting for the PSP successor next year.

Keep calm and observe

A key component of your security hygiene is ensuring that you have the right mechanisms to detect the ongoing behavior and acting to remediate any breaches. You can start with turning on audit logs that offer visibility into the cluster operations, turning on monitoring to help you understand the health and performance of your nodes, pods, and services, and logging to troubleshoot failures and performance issues. For monitoring and alerting, Prometheus with Grafana is a popular choice, but alternatives exist. Send the standard output, standard error output of each container, and the worker nodes logs to the logging system of your choice.

With logging and monitoring, proactive, ongoing analysis of the information is necessary. For these processes, you can use Security Information and Event Management (SIEM) or security posture services offered by the cloud providers.


Running a Kubernetes cluster in a secure manner requires a significant amount of work and knowledge. Using a managed Kubernetes service helps you focus on running your applications effectively. Repetitive tasks, such as node provisioning and upgrades involving cordon-drain, are automated. The services that are peripheral to your Kubernetes cluster can be fully managed, such as IAM, monitoring, logging, and registry. Managed Kubernetes cloud services bring a real value to help you focus on what matters, your applications.

For more information on Kubernetes security and hardening, see the following resources:


To hear more about cloud native topics, join the Cloud Native Computing Foundation and cloud native community at KubeCon+CloudNativeCon North America 2021 - October 11-15, 2021       


Greg Verstraeten, Senior Principal Product Manager, Oracle

Greg Verstraeten 

Greg is a Product Manager for the managed Kubernetes service (OKE) within Oracle Cloud Infrastructure. He focuses on delivering features that makes Kubernetes in OCI secure and easy to use. He has 20 years of experience building infrastructure and applications, and he has been working with Kubernetes for 4 years. Greg is French and lives in the San Francisco Bay Area with his family.

Published Monday, October 11, 2021 7:34 AM by David Marshall
Filed under: ,
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<October 2021>