
Virtualization and Cloud executives share their predictions for 2016. Read them in this 8th Annual VMblog.com series exclusive.
Contributed by Sean Ma, Director of Product Management at Trifacta
Big data makes it to the front office and transforms the value and framework of the analytics industry in 2016
The "big data" space is still maturing - continuing to
define its vision while simultaneously converting aspects of that vision into
reality. There is a natural progression to this development that amounts to
answering the following questions in order: Can we build tools that make
working with data easier, faster, etc...? Can we make these tools reliable and
robust? Can we build processes around the use of these tools that safeguard
their use and that provide consistent value?
By drawing analogies from other mature product spaces, these
questions are answered in three phases. First, vendors build early stage
products demonstrating that viable solutions exist. As early customers adopt these
products, we move into the second phase where vendors make these products
robust and reliable, and customers start experiencing their value. The third
phase wraps these products into workflows and organizational structures that
consistently deliver value and can scale to multiple teams or departments.
In 2015, vendors were largely focused on hardening their tools
as they moved beyond early stage products, and big data projects were still
tightly controlled and managed by IT. In 2016, the front office (business
analysts, managers and executives) will adopt technology to solve specific
business challenges or opportunities, the scale of which will force vendors to
extend their offerings to manage end-to-end solutions. This forces vendors to develop
or integrate with existing tools and provide best practices for how customers
should orchestrate their use and coexistence. Based on that framework, we think
2016 will show:
Industry specific
solutions will significantly increase:
-
Vendors, like Cloudera, will work with ecosystem
partners to develop and leverage solution "blueprints" to demonstrate actual
value to their customers in specific verticals. These solution blueprints based
on actual, working, successful customers will provide both IT and LOB users
with the confidence to invest or expand upon their big data initiatives. We
will see most of this value surface in exploratory or innovation projects. The difference
is that companies will not require a back-office black-ops team of data
scientists to achieve this value. As a greater number of analysts have access to the appropriate tooling to
make use of big data and gain a better understanding of how to apply these
tools, the next step will be to customize early prototypes for specific business
initiatives.
Increasing the amount
of deployments will force vendors to focus their efforts on building and
marketing management tools:
-
For projects to operationalize at big data scale,
vendors can no longer hold customers' hands throughout the duration of a big
data project. Once customers experience initial value, vendors will take the
next step by developing tools and best practices for mass productionization of valuable
big data solutions in a consistent and repeatable fashion.
Much of the functionality in these
tools as they gain broader adoption will need to replicate functionality in analogous tools
from the enterprise data warehouse space, specifically in the metadata
management and workflow orchestration. There will be several key differences
with these new big data solutions compared to their traditional counterparts:
o
Ability to handle a wider variety of data both
in terms of content and structure / semi-structured data.
o
The agility to handle frequent changes in that
data, which is something that traditional enterprise data warehouses struggle
with.
o
Exposing metadata, traditionally locked away in
technical tools, to non-IT users in ways that improve data analysts usage and
understanding of their data.
Front-office business
stakeholders will demand self-service access to big data platforms which will
drive organizations to adopt a hub-and-spoke IT model to support a wider
variety of data-driven initiatives:
-
Rather than having IT in one large centralized
team, companies will complement a smaller centralized IT team with individual technical
teams to collaborate with each line-of-business group. This allows for a
two-tier support structure with universal needs serviced centrally and
specialized needs serviced in-situ for speed, goodness-of-fit, consistency and
efficiency. This will initially complicate the go-to-market strategies of big data
vendors who need to develop marketing and sales mechanisms to target newly
formed LOB-IT groups.
Beyond 2016? With new organizational structures and workflow
management tools in place, many companies will be in a place to democratize
access to their data and to reap the benefits of a data innovation funnel that
starts with broad, lightweight exploration and continues to data curation and
data pipeline productionalization.
The next challenge? Data governance for Big Data in this new
self-service world. 2016 will have built out all the building blocks necessary,
but balancing top-down and bottom-up data governance processes within
organizations will be critical to driving continued innovation from big data?
##
Author the
Author
Sean Ma, Director of Product Management at
Trifacta.
With
over 10+ years of experience in enterprise data management software, Sean has
spent the last 5 years leading Big Data products at companies such as
Informatica and Trifacta. He holds a Bachelor of Science degree in Electrical
Engineering and Computer Science from the University of California Berkeley.