Astronomer announced
the private preview of
Astro Observe.
Astro Observe allows any team using Airflow to now have access to an
actionable view of the data supply chain and extends Astronomer beyond
data orchestration into a platform for data operations. This means
proactively identifying opportunities for optimization, zeroing in on
problems areas for rapid interventions, and benefitting any team's
Airflow projects - regardless of if they are on Astro, open source
Airflow or another managed service. By adding observability to the
existing data orchestration capabilities of Astro, businesses can now
easily build, run, and observe data pipelines as code for seamless data
operations. Astro Observe means organizations can now support the
development, expansion and upkeep of modern data products better than
ever.
"Data products have become just as business-critical as software
products, and any failure can have an outsized impact on revenue and
reputation," said Julian LaNeve, CTO,
Astronomer. "Astro Observe unifies orchestration and observability
across the full data stack on a single platform, meaning the reliability
and trust of data products are improved, development velocity is
increased, costs are lowered, mission-critical SLAs are respected and
critical data assets are better secured. We want to give data teams the
same operational rigor as software teams."
Delivering reliable data products is imperative, but complex
Reliance on data has evolved over the past decade from powering basic
internal dashboards to data becoming a product itself, with source data
packaged alongside artifacts like metadata, transformation logic, docs
and tests, access policies-all exposed as APIs to drive a diverse set of
mission-critical use cases that depend on data orchestration to work.
Whether it's retail recommendations with dynamic pricing, automated
customer support, financial trading strategy, regulatory reporting, or
AI-full stack data orchestration has become the linchpin of modern
software solutions.
Observability and Orchestration on a single platform provides connective tissue between the entire data stack
Observability has historically already been a differentiator for
Astronomer compared to OSS Airflow and other managed services, with logs
and metrics to provide operational visibility across all of a
customer's Airflow deployments and workspaces. Astro Observe introduces
data observability as an additional capability of our platform. Central
to Astronomer's data observability capabilities is the use of
OpenLineage as the framework to make data pipelines observable by
automatically collecting and correlating detailed information about
their behavior and data movement. In the last few years, Astronomer has
invested heavily in the development of OpenLineage and it has emerged as
an industry standard for solving problems around data observability.
Key features and benefits of Astro Observe
- SLA Dashboard: Enables data product owners to be assigned, to
provide a clear picture of accountability and ownership. Owners of data
products can set policies and monitor thresholds for data freshness and
delivery times to better align with business outcomes.
- Dependency Graph: Visibility into upstream and downstream
relationships across data products, providing critical context for issue
resolution and ensuring compliance.
- Recommendation Engine: Leverages best practices from Airflow
alongside a holistic view of deployments to surface actionable
recommendations for optimizing trust and reliability of data products.
- Predictive Alerting: Allows teams to pinpoint and understand
risk before the delivery of data products are impacted; providing time
and context necessary to remediate and make adjustments across the stack
to ensure the on-time delivery of fresh and reliable data.
- Single Destination for Orchestration and Observability:
Astronomer provides a single destination for teams to orchestrate and
manage pipelines along with the visibility they need to monitor
performance and make adjustments as needed to solve problems or make
proactive optimizations; minimizing context switching, and allowing for
one interface to understand and take action. As modern data stacks
become more complex with dependencies that sprawl across multiple
platforms, teams, and AIrflow deployments, Astronomer's centralized view
helps teams to quickly assess and make decisions about the management
of their projects.
To learn more, visit the resources below to get started with Astro today.