Opens in a new tab
vmblog logo 2024 wht (updated)

The Data Lake Hidden Inside Every Kubernetes Backup

Share: 

Michael Haigh | Published: October 21, 2026
data lake hidden kubernetes

By Michael Haigh, Staff Product Manager, Veeam Kasten

Backups are the largest line item in IT that nobody expects to generate value. They exist for one purpose: To restore when disaster strikes. However, every Kubernetes backup captures a rich record of your applications, including database contents, file trees, storage statistics, and the manifests that define how they’re configured. A fleet of applications with backup schedules, all of it versioned, and none of it queryable. In a world where AI agents are beginning to act on structured data autonomously, this is a missed opportunity.

Today, the only way to answer a question about what’s inside a backup is to run a full restore, execute your query, and tear it all down. That’s 30 to 90 minutes with associated infrastructure cost for a single SQL query. According to a recent industry survey, 81% of respondents see value in transforming backups into a queryable data lake. So, the gap between what backup repositories contain and what we can do with that data is becoming its own category.

Backup data reuse is now a category

Gartner added “Backup Data Reuse” to the Hype Cycle for Data Management as an Innovation Trigger in 2025 and predicts that 30% of enterprises will integrate backup copies as analytics sources by 2029 (up from less than 5% today), and the analytics industry is responding. The Databricks Storage Ecosystem, announced in June 2026, explicitly targets the “hundreds of exabytes” of backup and secondary data across enterprises as untapped analytics assets. We’re seeing movement from both ends, too: Backup vendors are being drawn into the analytics world, and analytics platforms are reaching into backup repositories.

At the protocol layer, OpenSharing (a new Linux Foundation project and the successor to Delta Sharing) provides the first open, vendor-neutral protocol for sharing data and AI assets across platforms and organizations. OpenSharing natively supports Apache Iceberg, Delta Lake, and Parquet, and enables zero-copy sharing from wherever your data lives, on-premises, in the private cloud, or at the edge. The Iceberg REST Catalog is the universal interface for table discovery and access, with native support across DuckDB, Spark, Trino, Databricks, Snowflake, and BigQuery.

The question is no longer should backup data be queryable. It’s how.

What makes Kubernetes backup data unique

Unlike traditional VM or file-level backups, Kubernetes backups capture two distinct layers of structure that make them particularly well-suited for open table formats.

The first layer is application data. Depending on the backup approach, this may include logical database dumps (mongodump, pg_dump, mysqldump) in parseable formats, or storage-level volume snapshots that capture filesystem artifacts like log and configuration files.

The second layer is what makes Kubernetes backups truly distinctive: Manifest metadata. A backup captures the complete application construct, deployments, StatefulSets, Services, ConfigMaps, PersistentVolumeClaims, and the security-relevant resources like Roles, RoleBindings, ServiceAccounts, and admission webhooks. No other backup domain captures the full infrastructure-as-code state alongside application data, versioned together at each backup point.

image

Each snapshot becomes a point-in-time record of both what your application contained and how it was configured. When you expose this data through an open table format like Apache Iceberg (where each row carries a snapshot_id), you unlock time-travel SQL across your entire backup history.

The translation from backup repository to Iceberg table is more straightforward than it might seem. Open-source backup tools like Kopia already store structured metadata internally: Snapshot manifests, file tree indexes, and content statistics. This metadata is small (KB to MB per snapshot), well-organized, and versioned. The manifests and application data are already in parseable formats too (JSON, YAML, logical database dumps). The engineering work is in materializing these internal structures as Parquet files with Iceberg metadata, so external engines can query them without understanding the backup format.

Three use cases that change when backup data is queryable

Compliance without restore. A GDPR data subject access request requires proving what personal data your organization held on a specific date. With backup data exposed as Iceberg tables, you filter by the snapshot closest to that date and export the result. What used to require a full restore becomes a SQL query that returns in seconds. The same pattern applies to SOC 2 audit evidence, retention policy verification, and any scenario where you need to demonstrate what existed (or didn’t exist) at a given point in time.

image

Security forensics through manifest diff. By comparing the complete application construct across consecutive backup snapshots using standard SQL, security teams can detect role based access control (RBAC) escalation (with new RoleBindings granting elevated permissions), sidecar injection (with new containers in Pod specs), image swaps, new ServiceAccounts, and admission webhook changes. Analysts can reconstruct exactly when each change was introduced and build an attack timeline, all without accessing the live cluster or performing a restore.

Agentic AI on backup data. OpenSharing doesn’t just share tables. It also shares agent skills, including portable instruction packages that tell AI agents how to use specific data. Imagine that an organization publishes Iceberg tables containing Kubernetes manifest metadata alongside a discoverable agent skill called “backup-anomaly-detection.” That skill describes which tables to query and what SQL patterns indicate ransomware like file change rate spikes, mass file extension changes, unusual size distribution shifts. A security team’s AI agent discovers both the data and the skill through OpenSharing, loads the instructions, queries the Iceberg tables, and produces an alert. The backup data becomes not just queryable by humans, but actionable by agents, in a governed, audited, and zero-copy manner.

image

The building blocks exist today

The cloud native ecosystem already has everything needed to make backup-data-as-analytics a reality:

Wire these together and backup metadata becomes queryable from any engine through a standard REST Catalog.

Looking ahead

The backup-data-as-analytics-source category is forming now, and Kubernetes has a structural advantage: Its backups carry richer metadata than any other backup domain. The tools are open source and the protocols are standardized. The question is whether the industry will expose this data through open formats, or keep it locked behind proprietary restore workflows. The building blocks are here. It’s time to start building!

KubeCon + CloudNativeCon is taking place in Salt Lake City from November 9 to 12. Register Now!

##

ABOUT THE AUTHOR

michael haigh

Michael Haigh is a Staff Product Manager at Veeam Kasten, where he helps enterprises protect their Kubernetes workloads at scale. He has spent his career spanning product management, technical marketing, and systems engineering, with a focus on making complex infrastructure accessible. He brings over 15 years of experience from Cisco, Nutanix, and NetApp.