Virtualization Technology News and Information
Iguazio Expands Serverless To Scale-out Machine Learning and Analytics Workloads

Iguazio, the Data Science Platform for automating machine learning pipelines, today announced Nuclio ML Functions, broadening the serverless capabilities of Iguazio's Data Science Platform for scalable machine learning training and data preparation. Nuclio is the only open source serverless framework that extends beyond event driven workloads to long lasting, parallel, and data-intensive jobs. This brings the benefits of serverless - allocating resources on demand, auto-scaling, and dev ops automation - to machine learning, data preparation and analytics workloads.

Serverless frameworks have so far only addressed challenges posed by the beginning and end of ML pipelines, namely data ingestion and model serving. Iguazio's open source Nuclio has stood out with its unmatched performance and parallelism, its support of GPUs at scale, multi-cloud deployment and native integration into the data science stack.

With Nuclio ML Functions, Iguazio now provides a layer of automation and monitoring for the widely used ML and analytics frameworks on top of Kubernetes, relying on its high-speed shared data layer for seamless scaling. ML function activities and data artifacts are automatically logged, allowing users to trace data and experiment results and re-run older jobs when needed. Nuclio ML Functions also leverages Kubeflow to speed up the running of ML pipelines.

With the introduction of these new serverless features, Iguazio enables full automation and CI/CD for ML workloads, cutting infrastructure costs, tedious development and operations tasks. The parallelism and auto-scaling capabilities are enabling Iguazio's customers to process the same tasks at a fraction of the time, consuming server and GPU resources upon demand.

"Data preparation, experiments and devops are the most time-consuming tasks in data science. Our goal is to minimize the overhead so data science teams can focus on innovation and building new applications," said Iguazio's CTO, Yaron Haviv. "Iguazio is democratizing data science, enabling deployment either in the cloud or on prem with familiar open source tools."

More than half of data science projects are not fully deployed, according to Gartner: "Many organizations struggle when it comes to systematically productizing machine learning results, as the production process is either overlooked or left solely to the DevOps team."

Nuclio ML Functions supports the following workloads:

  1. Real-time ingestion and APIs
  2. Analytics and data preparation - using Spark and Dask engines
  3. Machine learning - using Dask, XGBoost and Scikit
  4. Deep learning - using TensorFlow, PyTourch and Horovod
  5. Model Serving - using Nuclio Serving
Published Wednesday, September 25, 2019 11:23 AM by David Marshall
Filed under: ,
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<September 2019>