Industry executives and experts share their predictions for 2018.  Read them in this 10th annual VMblog.com series exclusive.

Contributed by Kunal Agarwal, CEO and Co-Founder of Unravel Data

2018 Prediction: Applications are the Future of Big Data

Big Data is evolving in a way that resembles the growth of the cloud. At the beginning of 2017, people were still navigating the basic Big Data ecosystem, technologies like Hadoop or Spark or Kafka. As 2017 ends, those conversations have finally been settled, and the industry is ready to move to a new era where the focus shifts to use cases from systems.

Big data applications will rise as a clear solution to address and solve challenges as we move toward this next step. I predict that in 2018, enterprises will shift their focus from traditional big data technologies more toward mission-critical big data applications as they begin to see their true value.

In the past, people were focused on learning the various big data technologies: Hadoop, Spark, Kafka, Cassandra, etc. It took time for users to understand, differentiate, and ultimately deploy them. There was a lot of debate and plenty of hype. Now that organizations have cut through the noise and figured all that out, they're concerned about putting their data to use - this is where the applications come in.

Take the recommendation engine, for example, a critical app for most all web companies. Consider Netflix: Their recommendation engine isn't just a nice add-on that enhances the user experience, it's fundamental to the experience and to Netflix's bottom line. Their platform depends on the ability to accurately suggest relevant movies and TV shows to people - otherwise, it'd be almost impossible for viewers to dig through their enormous library.

Netflix/the enterprise doesn't really care about the technology being used. It's not important which distribution or database or analytics they're using, what matters is the result. The enterprise has realized this and we can expect to see an increased adoption of an application-centric approach to big data in the coming year.

In 2018, with the use of these big data applications, enterprises will learn how to apply their systems and tools to get the most out of them. This next phase is about driving value from big data, a next step that will give companies a considerable edge as we look toward the year ahead.

About the Author

Kunal Agarwal is the CEO and Co-Founder of Unravel Data. He also founded Yuuze.com in 2010, a pioneer in personalized shopping recommendations. Prior to this, Kunal he was a business development manager for Oracle products. He has managed enterprise data for companies like United Technologies and Express Scripts. Kunal helped Sun Microsystems evaluate Big Data infrastructure like Sun's Grid Computing Engine. Kunal holds a Bachelors in Computer Engineering from Valparaiso University, Indiana and M.B.A from The Fuqua School of Business, Duke University, NC.