The appetite for data is only increasing, as enterprises invest in the various evolutions of data repositories, such as data warehouses and data lakes, used to generate business insights. To adequately scale it as a business grows and changes, a paradigm shift is required to view the data as a domain rather than a platform.

To better understand data mesh and explore the challenges being faced, VMblog reached out to industry expert Billy Bosworth, Chief Executive Officer at Dremio.

VMblog:  What do people mean today when they use the term data mesh? Can you define it?

Billy Bosworth:  Data mesh is evolving to mean treating data as a product within various business groups (or lines of business) across a company, overlaid by governance, security, and accessibility provided by a centralized data team. This allows individual business groups to move faster, under certain requirements, with their data assets connected to the bigger mesh owned by the centralized team, without being dependent on that team for building and maintaining their departmental data repositories.

VMblog:  What role does open architecture play in a data mesh?

Bosworth:  A key tenet of data mesh is not being locked into any particular vendor format. Open architecture becomes critical in a data mesh because each business group can choose their own technology. That means, at bottom, the data ultimately must be accessible in open formats to make the mesh work effectively. Limiting the mesh to any particular vendor would defeat the whole point of connecting the disparate data assets in a coherent way. 

VMblog:  What's the alternative? If customers weren't going to do it, what would they do and how would they run their business?

Bosworth:  The danger for companies today is to trend to one of two extremes: flexibility vs. security/governance. When flexibility is the prevailing extreme, data silos operate independently of each other. This creates chaos for data governance and security and leads to multiple copies of data floating around the company, without teams understanding which ones are correct and up to date. When security/governance is the extreme, business speed grinds to a halt as the backlog of requests to the central teams becomes untenable, and there's no flexibility for businesses to move fast and compete in their markets. Data mesh tries to strike the right balance between these two. 

VMblog:  What do you see as current biggest challenges in implementing a data mesh?

Bosworth:  The main problems are interrelated issues: (1) Lack of a corporate, top-down mandate that requires the teams to work together; (2) Improper organizational design; and (3) Infighting and territorialism among the various teams due to each group's desire to protect their people, resources and investments. Note that "technology" is not on this list because, in general, technology is not the reason most of these large analytics projects historically fail. The ‘people' aspect of these projects, combined with the commensurate change management, are often not given the attention they deserve.  

VMblog:  How do data mesh owners act like vendors internally? 

Bosworth:  In a very real way, the centralized team views the various lines of business as its customers. That team often creates service-level agreements and is held accountable for the end users' success when it comes to getting the data they need in the timeframe that they need it. In this sense, the centralized team operates like a vendor itself - one who is passionate about "customers" and very much wants to see them successfully use their technology. 

## 

Relevant Articles:

https://www.dremio.com/blog/enabling-a-data-mesh-with-an-open-lakehouse/#:~:text=More%20specifically%2C%20 Dremio%20provides%20four,%2C%20fine%2Dgrained%20access%20policies.