Virtualization Technology News and Information
Databricks Survey Gets to the Heart of the AI Dilemma: Nearly 90% of Organizations Investing in AI, Very Few Succeeding

Databricks today announced the results of a commissioned survey looking at the AI dilemma. Today, only one in three AI projects are succeeding, and, perhaps more importantly, it is taking businesses more than six months to go from concept to production. The primary reasons behind these challenges are that 96 percent of organizations face data-related problems like silos and inconsistent datasets, and 80 percent cite significant organizational friction like lack of collaboration between data scientists and data engineers. IT executives point to unified analytics as a solution for these challenges with 90 percent of respondents saying the approach of unifying data science and data engineering across the machine learning lifecycle will conquer the AI dilemma.

"Data is the fuel that powers AI. Large amounts of reliable data that data scientists can iterate on is the key to AI success. But organizational silos between data science and engineering cripples the iterative model development process. And to make matters worse the divide between today's data and AI technologies increases complexity throughout the lifecycle further slowing down AI," said Bharath Gowda, vice president of product marketing at Databricks. "Unified analytics addresses this AI dilemma by providing an end-to-end analytics platform that unifies big data and AI, while fostering better collaboration between data science and engineering teams."

View the complete survey results within the "Conquer the AI Dilemma by Unifying Data Science and Engineering" report found here:

The survey, commissioned by Databricks through IDG's CIO Research Services, surveyed 200 IT executives at larger companies (1000+ employees) across the U.S. and Europe. The results speak to the complexity and organizational confusion being creating as companies pursue AI initiatives:

  • 98 percent of those surveyed believe preparation and aggregation of large datasets in a timely fashion is a major challenge;
  • 96 percent of respondents found data exploration and iterative model training challenging;
  • 90 percent cited the deployment of models to production quickly and reliably as a significant challenge
  • 87 percent of organizations invest in an average of seven different machine learning tools, adding to the organizational complexity

So, what will help these organizations conquer the AI dilemma? The surveyed executives said they need end-to-end solutions that combine data processing with machine learning capabilities. These streamlined solutions would simplify workflows, improve efficiency and ultimately accelerate business value.

In fact, nearly 80 percent of executives surveyed said they highly valued the notion of a unified analytics platform. Unified analytics makes AI more achievable for enterprise organizations by unifying data processing and AI technologies. Unified analytics solutions provide collaboration capabilities for data scientists and data engineers to work effectively across the entire AI development-to-production lifecycle. With more than 90 percent of large companies facing data-related challenges and increasing complexity driven by an explosion of machine learning tools, the need for platforms and processes that can remove technology and organizational silos is more pronounced than ever. Unified analytics provides an ideal approach for companies facing modern AI implementation barriers.

Databricks accelerates innovation by unifying data science, engineering, and business. Through a fully managed, cloud-based service built by the original creators of Apache Spark, the Databricks Unified Analytics Platform lowers the barrier for enterprises to innovate with AI and accelerates their innovation.

Published Tuesday, July 24, 2018 11:47 AM by David Marshall
Filed under: ,
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<July 2018>