Virtualization Technology News and Information
At Subsurface LIVE 2023 Dremio Launches New Data Lakehouse Features and Sees Exponential Community Growth

At Subsurface LIVE 2023, Dremio announced the rollout of key new features. These features enable customers to more easily create their data lakehouses by performantly loading data into Apache Iceberg tables, query and federate across more data sources with Dremio Sonar, automatically format SQL queries in the Dremio SQL Runner, and securely connect Microsoft PowerBI using single sign-on (SSO).

Dremio also has added roll back and data optimization features for Apache Iceberg tables, making it even easier to manage data lakehouses using the open table format standard. Furthermore, customers can now use several new SQL functions for an even better SQL experience.

Dremio's expanding functionality with Apache Iceberg now includes:

  • Copying data into Apache Iceberg tables - Dremio's new COPY INTO SQL command makes it even easier and faster to load data into Apache Iceberg tables, which are a foundational component of data lakehouses. With one command, customers can now copy data from CSV and JSON file formats stored in Amazon S3, Azure Data Lake Storage (ADLS), HDFS, and other supported data sources into Apache Iceberg tables using the columnar Parquet file format for performance. Dremio efficiently distributes the copy operation across the entire engine to load data more quickly.
  • Optimizing Apache Iceberg tables - When using Dremio's data manipulation (DML) commands to insert, update, and delete data from an Apache Iceberg table, additional files are created to represent these mutations to the table. Often, customers will have many small files as a result of these operations, which can impact read and write performance on that table and utilize excess storage. To improve the performance of Apache Iceberg tables, customers can now use the OPTIMIZE command in Dremio Sonar to consolidate these files into an optimal size. Customers running frequent DML operations can use OPTIMIZE at a regular interval to keep their Apache Iceberg tables efficient.
  • Table roll back for Apache Iceberg - Customers can now restore their Apache Iceberg tables to a specific time or snapshot ID with Dremio's new ROLLBACK command. This makes it easy to revert a table back to a previous state with a single command. When rolling back a table, Dremio will create a new Apache Iceberg snapshot from the prior state and use it as the new current table state.

Dremio's new functionality also includes new connectors for Microsoft PowerBI, Snowflake, and IBM Db2. Customers using Dremio and PowerBI can now use single sign-on (SSO) to access their Dremio Cloud and Dremio Software engines from PowerBI, simplifying access control and user management across their data architecture. The Snowflake and IBM Db2 connectors give customers the ability to quickly add Snowflake data warehouses and IBM Db2 databases as data sources for Dremio. This makes it easy to include data in these systems as part of the Dremio semantic layer, enabling customers to explore this data in their Dremio queries and views.

Additionally, customers can now add Dremio clusters as data sources, enabling query federation across these clusters. This feature set enables connectivity across Dremio environments, including hybrid environments where you have Dremio clusters running in a public cloud and on-premises. This makes it easy to extend Dremio semantic layers across clusters, giving analysts and BI engineers access to more datasets and insights. Along with popular connectors already operational like Tableau, ThoughtSpot, dbt, and Alteryx, these new connectors add more flexibility for customers.

"It's been great to see the incredible growth of our product driving value for our customers," said Tomer Shiran, co-founder and CPO of Dremio. "Every company we speak to is struggling to help their business move faster and self-serve while maintaining security and governance. Data meshes solve for these competing priorities, and we've been innovating to make it easier and easier for companies to create and operate data mesh architectures."

Dremio's growth and momentum continue to be recognized across the industry with key awards in 2022 and already in 2023. These include: CNBC's 25 Top Startups for the Enterprise, Deloitte's 2022 Technology Fast 500, CRN's The 10 Hottest Big Data Tools Of 2022, DBTA's Trend-Setting Products in Data and Information Management for 2023, and Tech Ascension's 2022 Big Data Award Winners. In addition to these awards, Dremio has also received 11 badges from G2, including "Easiest to Use," "High Performer," and "Momentum Leader."

"We are delighted to be recognized by trusted organizations like Deloitte and CNBC and are excited to see the growth of our community. I am proud of Team Dremio for their hard work last year and am looking forward to more growth in 2023 as we see a continued interest in open data lakehouse architectures," said Read Maloney, CMO of Dremio.

Published Wednesday, March 01, 2023 12:40 PM by David Marshall
Filed under:
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<March 2023>