Virtualization Technology News and Information
Article
RSS
Iterative 2022 Predictions: MLOps tools in focus

vmblog predictions 2022 

Industry executives and experts share their predictions for 2022.  Read them in this 14th annual VMblog.com series exclusive.

MLOps tools in focus

By Dmitry Petrov, Iterative

Based on the hundreds of conversations we've had with MLOps teams this year, we've compiled some trends that we've seen as a commonality across our discussions. Machine learning (ML) teams have been maturing their process and tooling in the past few years and we see this trend rising as organizations cope with larger ML teams and faster-paced ML modeling to maintain a competitive advantage.

More tools to manage unstructured data in ML projects

AI projects and models around computer vision and deep learning have matured as evidenced by self-driving technology, health tech advancements, and more. This means that working with unstructured data becomes a bigger priority - video, images, etc. - as data science teams work more with this data versus traditional structured data. "Data-centric AI" as a term has come up because data management, and versioning specifically, is critical in improving models. In fact, data is the most critical component to building and optimizing ML models. More MLOps tools will arise to fit this need around structured data, including data labeling and management as well as downstream technologies around capturing this type of data.

MLOps solutions focus on collaboration and enterprise-grade features

Organizations have been shifting workstyles to allow more remote workers and COVID-19 hastened this trend. Compounding this is the maturity of ML departments. It's no longer one or two people working on models - organizations small and large have entire teams dedicated to AI and ML model development. Many of today's toolsets for ML teams are focused on one person - but there is a strong need to collaborate across large teams and locations.

The need to enable teams of people also means that a new set of governance capabilities focused on ML needs will arise. Data security, management, and access controls are all important - but in the context of ML this becomes more complex. It's not just about managing access based on roles to data directories. Data sets will become the new layer of governance as team members build models based on these data sets and versions of them versus managing access to data on the s3 level. Data security for ML is complicated because of working with versioned data across multiple models, necessitating ML solutions to come up with features customized for ML.

Specialization over full-stack MLOps

With maturing ML teams at organizations, data scientists and ML engineers are being enabled to work on their core jobs versus having to do it all. This means data scientists can focus on model development and don't have to be experts in infrastructure or Kubernetes when they want to train their models (or anything else not related to model development). Data engineers will work on data management and DevOps teams will abstract compute for data scientists. Connected to this will be new toolsets to enable this trend. These new tools will specialize on these specific tasks around data management, infrastructure management, compute orchestration, model development, and more. Existing MLOps solutions are more general purpose and not customized for the increasingly complex world of AI. New tools will also be customized for monitoring of data and models as ML programs mature.

##

ABOUT THE AUTHOR

Dmitry Petrov 

Dmitry Petrov is an ex-Data Scientist at Microsoft with Ph.D. in Computer Science and active open source contributor. He has written and open sourced the first version of DVC.org - machine learning workflow management tool. Also he implemented Wavelet-based image hashing algorithm (wHash) in open source library ImageHash for Python. Now Dmitry is working on tools for machine learning and ML workflow management as a co-founder and CEO of Iterative in San Francisco. 
Published Wednesday, December 01, 2021 7:29 AM by David Marshall
Comments
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
Calendar
<December 2021>
SuMoTuWeThFrSa
2829301234
567891011
12131415161718
19202122232425
2627282930311
2345678