Dremio, the data lake engine company, today introduced a new offering, purpose-built for Amazon Web Services (AWS), with two new technologies to support on-demand data lake insights and reduce cloud infrastructure costs. In a related announcement, Dremio also announced the availability of Dremio AWS Edition in the AWS Marketplace. 

Available in the new Dremio AWS Edition, elastic engines and parallel projects technologies deliver deep automation, resource efficiency and elastic scale enhancements. The combination of these new capabilities delivers tremendous performance gains and deep infrastructure cost savings.

"Dremio AWS Edition will make it even easier and more cost-efficient to run business intelligence tools such as Tableau on AWS S3 data lake storage while accelerating queries for our predictive analytics models," said Adrian Daniel, head of data platforms, NewDay, a financial services company. "The low-latency SQL interface, highly elastic compute engines, and self-service semantic layer will dramatically lower our cloud infrastructure costs while empowering our data analysts to explore data and derive new virtual datasets with minimal dependency on engineering."

Independent and Resource-Efficient Compute Engines

Elastic engines address two critical challenges for data teams that are tightly coupled; performance and cloud infrastructure costs. Cloud software and services aren't typically architected to take advantage of the inherent elasticity of AWS and thus incur ongoing infrastructure costs for idle compute resources. At the same time, traditional scale-out query engines are built around a single execution cluster architecture that supports multiple, dynamic query workloads. As a result, the cluster is either under-provisioned, leading to workload contention and inconsistent, degraded performance, or more commonly it is heavily over-provisioned to cover peak demand, leading to low efficiency and increased infrastructure costs.

"Data teams are struggling to process, query and extract value from the flood of data landing in Amazon S3," said Tomer Shiran, chief product officer, Dremio. "Direct, on-demand querying of that data remains too slow-causing data engineers to copy the data into proprietary data warehouses. And once there, performance is still too slow, as additional complex and time consuming external acceleration technologies are required such as BI extracts, OLAP cubes, and aggregation tables. With Dremio AWS Edition, data teams can query the data in place in S3 with lightning-fast interactive performance while reducing their cloud infrastructure costs by over 90 percent compared to traditional SQL engines."

Elastic engines enable data teams to configure any number of compute engines, each sized and tailored to the workload it supports and running inside customers' own AWS accounts. Elastic engines therefore provide workload isolation which eliminates both under- and over-provisioning of compute resources, maximizing concurrency and performance while at the same time minimizing the required compute infrastructure. Elastic engines are also dynamic, spinning up automatically only when needed to service queries and elastically spinning back down when queries stop. This elasticity eliminates any infrastructure costs associated with idle compute resources.

Multi-Tenant Dremio Environments With Deep Lifecycle Automation

Cloud software and services often require complex and manual deployment, configuration and upgrade processes that create a fragile, error-prone environment and delay time to value. To address these challenges, Dremio AWS Edition enables multi-tenant instances via parallel projects with deep lifecycle automation. Each instance contains all associated configuration, metadata, and data reflection details allowing for complete isolation and enabling business units to operate fully independently while also facilitating compliance.

Parallel projects also provide end-to-end lifecycle automation across deployment, configuration with best practices, and upgrades, all running in customers' own AWS accounts. This automation delivers a streamlined experience where data engineers and data analysts can deploy an optimized Dremio AWS Edition instance from scratch, start querying data in minutes, and effortlessly stay current with the latest Dremio features.

"A data lake has become a critical component of a modern data architecture but extracting value from it requires high-performance, self-service tools along with ample governance," said Wayne Eckerson, founder and principal consultant, Eckerson Group. "By addressing these issues head on, Dremio is at the forefront of helping organizations harness the full potential of their data lakes."

Availability

The new Dremio AWS Edition is immediately available in the AWS Marketplace here.