Industry executives and experts share their predictions for 2019. Read them in this 11th annual VMblog.com series exclusive.
Contributed by Amnon Drori, Co-Founder and CEO of Octopai
2019 Will be a 'Breakthrough Year' for Machine Learning and Metadata Management
If data is the "new
oil," then metadata is the barrel that the
oil needs in order to be useable. And more executives are beginning to
understand that - which is why automation and machine learning in metadata
management will, we believe, be a focus of many more business users in the
coming year.
Metadata - the "information about data" that, among other
things, classifies it by name, tag, date, type, etc. - makes it possible to
search for data by category, relationships, etc. Without metadata, information
in databases would be of very limited use. With more data than ever to manage,
organizations are turning to automation to discover, classify, and safely store
the data they have, enabling them to more easily build a metadata framework
around it.
The move to automation in managing metadata isn't just a matter of
convenience. Privacy and security regulations of the GDPR, California's CCPR,
HIPAA, etc. - require organizations to have full control over the data they
have, and full knowledge over how it is used, and. In addition, the huge amount
of data that is being collected today, thanks to IoT and advanced devices,
guarantees that without automated metadata management organizations will be
unable to build data catalogs, maintain business intelligence best practices,
or even make sense of the huge reams of data they are collecting.
For example: What does the metadata label "account type" refer to
- the size of the organization? the industry? non-profit or commercial status?
Regulations require that organizations be able to answer that question without
hesitation - and failure to comply could entail massive fines and other
penalties. And in these hyper-competitive times, being able to quickly extract
meaning and context from data becomes more important than ever. Unfortunately,
the traditional BI infrastructure, including ETL, DB, analysis, and reporting
tools. that have been used to manage metadata are not up to the task. But now
there are new is big data-based tools based on machine learning, developed
algorithms, and automation to enable BI teams - to work with metadata more
efficiently and effectively. Just in time; until now, much of that BI work was
done manually, requiring weeks or even months to answer questions that the
regulations require immediate responses for.
Hence the growth of
automation and machine learning in metadata management. Organizations don't
want to risk violating the new - and growing - number of regulations on data
quality and accuracy. To ensure compliance, organizations will focus on
incorporating new technologies, with machine learning chief among them. ML will
continue to play a bigger and bigger role in the fields of metadata and
metadata management in 2019 and beyond.
##
About the Author
Amnon has over 20
years of leadership experience in technology companies. Before co-founding Octopai he led sales efforts at companies like
Panaya (Acquired by Infosys), Zend Technologies (Acquired by Rogue Wave
Software), ModusNovo and Alvarion, and also served as the Chief Revenue Officer
at CoolaData, a big data behavioral analytics platform. Amnon studied
Management and Computer Science at the Open University of Tel Aviv.