Industry executives and experts share their predictions for 2024.  Read them in this 16th annual VMblog.com series exclusive.

Data mesh and data contracts face off with data quality

By Sonny Rivera, Senior Analytics Evangelist, ThoughtSpot

Without a good data foundation, AI is not possible. The immense value of quality data to Gen AI cannot be overstated, nor can the impact of bad data on an organization. Don't just take my word for it-here's a snippet of what the data has to say.

Recent research by Experian makes a startling claim: companies typically lose between 15% to 25% of their revenue due to poor data quality.

• These losses manifest in various forms, including:
• Time and resources spent in correcting errors
• Verifying data through alternative sources
• Adhering to compliance
• Mitigating consequences of ensuing inaccuracies
  

Gartner's analysis underscores that poor data quality significantly impairs an organization's competitive edge, impeding crucial business goals. Similarly, a Harvard Business Review report citing IBM reveals a broader economic impact, where the U.S. economy annually incurs a staggering $3.1 trillion loss owing to data-related inefficiencies. These losses are attributed to reduced productivity, frequent system outages, and increased maintenance expenses.

The research clearly demonstrates that prioritizing high- quality, reliable data is not only a technical necessity but also a strategic imperative for Gen AI to achieve its maximum potential. Enter data mesh and data contracts.

Zhamak Dehghani, CEO of NextData and author of "Data Mesh," states that "Trusted data still isn't accessible enough to support the AI revolution. Decentralized data is the future."

Chad Sanderson, CEO and co-founder of Gable.ai, is creating a data collaboration platform based on data contracts to meet the growing needs of BI and AI. The emergence of startups focused on data contracts and data mesh indicates a growing interest in these trends, but data quality issues have plagued the industry for decades. Leaving many to question: Is the solution data contracts or data mesh?

Both sociotechnical approaches strive to enhance the agility, efficiency, and value derived from data in large enterprises. Data mesh focuses on a decentralized, domain-oriented data management and architecture approach; data as a product is a key concept as opposed to a centralized data warehouse.

Data contracts focus more on greater collaboration to decentralize data ownership, manage schema evolution, provide data quality measures, and enforce SLA, enabling data to reach its fullest value.

Sounds amazing, right? Yet, the controversy still remains. Here's why:

• Aren't data contracts just a sub-component of a data quality tool?
• What are the standards, tooling, and processes for data mesh and data contracts?
• Aren't data contracts just a practical application of the underlying data mesh concepts, not a product or practice unto itself?
• Will they become a standard or die a slow and silent death like other approaches?
  

Even in the face of these questions, NextData and Gable.ai raised funds in the Fall of 2023. Zhamak led NextData to acquire $12 million in funding, and Chad Sanderson raised $7 million to start Gable.ai. Clearly, the investors see value in both data mesh and data contracts.

The demand for quality data at scale from AI and BI coupled with the industry-wide data quality issues suggest that there is potential for data mesh and data contracts to revolutionize data management in BI and AI. However, the future remains uncertain, with their widespread acceptance and efficacy yet to be determined.

Below are some resolutions to consider as we embark on 2024:

• Analyze the costs of bad data to your organization. Use this discovery to determine if data mesh or data contracts could help you. Decide on your key objective(s) for implementing data contracts in your organization. What problems do you want to solve first, and why are they essential to the business?
• Use your leadership platform to promote data quality within. Ensure the conversation focuses on value realization and creation, and highlight the critical role of data quality in delivering value.
• Evaluate your 1st-party data to determine if there is hidden value. No data is too small to add value. Learn about smart-sizing your data for Gen AI from Andrew Ng.
• Recognize that data mesh is not something you can buy. It requires modern cloud technology but, more so, cultural readiness to shift power to business units.

##

ABOUT THE AUTHOR

Sonny Rivera is a Sr. Analytics Evangelist at ThoughtSpot, a modern data stack thought leader and expert with over 25 years of experience delivering data solutions that drive business value and increase speed to insights. He has delivered multiple embedded analytics products, resulting in over $28MM in annual recurring revenue (ARR). Sonny advises customers on their journey to the cloud, data literacy and fluency, FinOps, and embedded analytics. He also works with ThoughtSpot's product team and provides thought leadership to the data and developer communities.
 
As one of Snowflake's original Data SuperHeros, he has a deep knowledge of data platforms, data modeling, and analytics technologies such as dbt Cloud, Snowflake, AWS Redshift, Sisense, and Looker. He has led multiple migrations to cloud data platforms from on-premise platforms such as Cloudera and Oracle. Before joining ThoughtSpot, he provided data delivery leadership for Ally Financial, AAA, Randall-Reilly Digital Media, and Hartford.