Security incidents in production environments lead to dramatic
headlines, but prevention of large-scale attacks starts with the smallest
components of the software supply chain. Cloud native applications and AI
workloads can be vulnerable to sophisticated supply chain attacks executed by
advanced persistent threats (APTs) and nation-state actors, so proactive steps
are essential. The Cloud Native Computing Foundation's Security Technical
Advisory Group published the Software
Supply Chain Best Practices in 2021, and in the
intervening years the topic has become even more critical. Azure's Containers
Secure Supply Chain framework offers guidance in this space,
and, in this article, we'll look specifically at open-source approaches.
While the landscape can seem vast and at times overwhelming,
it's possible to choose iterative improvement in areas best suited to your
needs. Numerous tools are available to ensure the integrity and security of
every component, package, and model weight for each stage of the software
supply chain; the task at hand is figuring out the highest-impact places to
start.
It is worth noting that, at the time of this publication
(late 2024), most open-source security efforts and available tools are
currently concentrated in the build and deploy stages of the software supply
chain. For reference, we look at the software
supply chain in five different stages as shown in the picture below.
1: Stages of the
Containers Secure Supply Chain
With that in mind, we will explore three specific areas where
open-source tools can improve your security posture, with details about the
problem space and current solutions available:
- Authenticity and integrity (with the
tools Ratify, Gatekeeper, Notary, and ORAS)
- Vulnerability management (with Copa)
- Dependency management (with Dependabot
and GUAC)
Let's talk about how these tools apply to today's needs, and end
with a look at what's coming next.
Authenticity & Integrity
Supply-chain Levels for Software Artifacts (SLSA)
provenance and in-toto
attestations are produced at the Build stage, stored with the software
artifacts, and verified before deployment.
Ratify/Gatekeeper
Admission controllers like Ratify/
Gatekeeper
are able to verify these attestations and check their integrity. The admission
controllers can also verify the integrity of the artifacts to be deployed.
Ratify's integration with Gatekeeper ensures that only
trusted and compliant container images run on Kubernetes clusters based on user
defined policy. Once set up, policies can be assigned based on Open
Containers Initiative (OCI) artifacts stored in the container registry. Here's
how the validation process works:
-
When a pod request reaches the Gatekeeper
validation webhook, it sends the tag or digest of the image to Ratify.
-
Ratify downloads the image signature metadata,
analyzes the attached artifacts, and runs a verifier using Notary Project or Cosign to produce
a verification report that aligns with the custom trust policy the user has
assigned. This can be done using an inline certificate or one sourced from a
key management service.
-
The result is relayed back to Gatekeeper, which
then issues either an ‘admit' or ‘deny' response based on the verification
tasks.
This lets you verify the authenticity and integrity of
software artifacts and their corresponding attestations before they reach
production.
ORAS
ORAS
lets you manage OCI artifacts such as container images, WASM modules, AI
weights, attestations, and SBOMs. When using ORAS, each file is pushed to the
registry as a blob, and you can use the 'oras attach' command to link artifacts
together. Linked artifacts are known as "referrer artifacts" and can be
discovered with the ‘oras discover' command. Referrer artifacts are
cryptographically linked to the primary artifact and can be distributed
together.
Syft
Syft can be
used to generate executive
order-compliant software bill of materials (SBOMs) that can easily be
attached with ORAS to cloud-native artifacts
stored in OCI registries.
Vulnerability
Management
Copacetic (Copa)
Copacetic, or Copa, is a CNCF sandbox project
that facilitates an efficient container image patching experience. Copa is a
CLI tool that seamlessly integrates into existing build infrastructure and
leverages reports from vulnerability scanners such as Trivy and Grype to reduce the vulnerability management burden on your
engineers.
Copa patches container images by upgrading
packages that are outdated or executing targeted updates by parsing
vulnerability scanning reports. It detects critical OS-level package updates
and applies them to the target image using Docker's default builder, Buildkit, to create a new patch layer on top of the
existing original image.
Check out this blog to get started with Copa today.
Dependency
Management
Dependabot / GUAC
Copacetic patching, combined with Dependabot for source
code dependency management, enables you to not only stay on top of package
dependencies but also base images and cloud-native deployment dependencies.
Using Dependabot is just the first step in the process of
improving dependency management. Tools for software composition analysis like Graph for Understanding Artifact
Composition (GUAC) will prove invaluable for understanding how cloud-native
artifacts are built. Extending that to the deployment and runtime stage will
help us get an up-to-date picture of deployed and running workloads. This
enables complete, end-to-end observability of the supply chain.
Creating purpose-built AI models that understand the supply
chain data and can act on the dependency graph to trigger automation with
Dependabot will become a requirement to enable security at the current and
future scale of cloud-native workloads.
Future Tooling Needs
While verifying source code processes has existed for a
while, verification of binaries has relied solely on cryptographic signatures.
There are more and more examples where this is not enough. Verifying the
signatures and the claims attesting how the artifacts are produced is becoming
a necessity to avoid sophisticated supply chain attacks. Transparency logs like
SCITT and Rekor will play a
significant role in the software supply chain security in the future.
Such transparency logs will not only keep a record of the
identities producing the artifacts but verifiable receipts for claims produced
at each step of the software supply chain.
Today's tools used for signing, storing, and attesting
will serve the initial needs of AI workloads, but new tooling is needed to
ensure those workloads are created using trusted and factual data, and do not
contain malicious prompts and instructions.
And on August 13th, 2024, NIST released the
first three finalized post-quantum encryption standards; the
industry needs to work on migrating all signing and counter-signing tools to
use those new and enhanced algorithms.
In Summary
As we've seen in this overview, engineering teams have plenty
of tools to start tightening the security of their software supply chain.
However, there are still numerous opportunities for innovation and improvement.
Today's software supply chain to-do list in summary:
-
Integrate the existing software supply
chain tools into our processes
-
Automate the generation of supply chain
artifacts
-
Implement strong authenticity and
integrity across our software supply chain
-
Ingest and index supply chain artifacts
to create an end-to-end observability layer
-
Lay groundwork for future automation
We live in revolutionary times for the software supply
chain; let's work together to shape the secure cloud native and AI future for
our society!
To learn more about Kubernetes and the cloud native
ecosystem, join us at KubeCon +
CloudNativeCon North America, in Salt Lake City, Utah, on November
12-15, 2024.
##
ABOUT THE AUTHOR
Toddy Mladenov, Principal Product Manager, Cloud-Native
Security & Registries Team at Microsoft
Toddy Mladenov has over 25 years of experience in
software engineering and design, consulting, and product management for
companies like Microsoft, T-Mobile, and SAP. He started his cloud journey 14
years ago as part of the Azure team. Since then, Toddy worked on large-scale
cloud implementations using Azure and AWS by utilizing cloud-native
technologies. Now, he is part of the Azure Cloud Native and Ecosystem team and
is responsible for container supply chain security for Azure services and
customers.
++
Payal Mahesh, Product Manager, Cloud-Native Security
& Registries Team at Microsoft
Payal is a Product Manager at Microsoft working on the
Containers' Secure Supply Chain in the Azure Core division. She has prior
experience in analytics, product management, and strategy for companies
including Microsoft, iRobot and Schneider Electric. In her current role, Payal
has developed a deep understanding of the need for security and reliability
when it comes to protecting user data and vulnerability management.