Virtualization Technology News and Information
VMblog Expert Interview: A Chat with WekaIO on the Release of Weka AI

interview weka manjrekar 

VMblog speaks to Shailesh Manjrekar, Head of AI and Strategic Alliances of WekaIO, on the company's release of Weka AI, a storage solution framework designed to enable Accelerated DataOps. He expands on the architecture behind Weka AI and why this type of framework has become essential for the success and sustainability of the digital organization.  He also outlines how AI is changing the landscape and the value that it can offer if data pipelines and capabilities are managed correctly.

VMblog:  What's driven the development of Weka AI?

Shailesh Manjrekar:  Artificial intelligence (AI) data pipelines are introducing challenges to traditional storage solutions. The latter lacks the abilities and capabilities required to meet the performance and scale that AI requires. AI data pipelines are inherently different from traditional file-based I/O applications, as each stage within these pipelines has distinct I/O requirements. These include massive bandwidth for ingesting and training, mixed read/write handling for extract, transform, load (ETL), ultra-low latency for inference, a single namespace for entire data pipeline visibility, and the need for edge to core cloud data pipelines. We recognized that any solution needs to meet all these varied requirements, and deliver timely insights at scale, bypassing the complexities that are inherent in traditional solutions. Traditional file-based I/O applications are unable to meet expectations in terms of performance and shareability across personas and data mobility requirements in the DataOps AI era. DataOps can provide actionable intelligence by breaking silos, delivering operational agility, and managing governance within these data pipelines. Still, it can only become a critical weapon in the business arsenal when supported by the right frameworks.

VMblog:  What comprises the Weka AI architecture?

Manjrekar:  Weka AI is a transformative storage solution framework designed to enable accelerated DataOps and, to this end, incorporates multiple elements to ensure it's capable of scaling easily from small to medium and large integrated solutions. The framework consists of customizable reference architectures (RAs) and software development kits (SDKs) with leading technology alliances like NVIDIA, Mellanox, and HPE, alongside other companies in the Weka Innovation Network (WIN). The framework is underpinned by the Weka File System (WekaFS) and is architected to accelerate DataOps by solving the storage challenges common to I/O-intensive workloads. NVIDIA and Weka have collaborated on critical projects that address I/O bottlenecks for data analytics and deep learning, and Weka AI takes these learnings to bootstrap the customer's AI journey in the deployment of data pipelines. The Mellanox InfiniBand utilizes GPUDirect and GPUDirect storage with the Weka AI framework to provide a robust platform for AI applications. HPE is also currently collaborating with Weka AI for multiple use cases such as semantic segmentation for ADAS and NVIDIA Clara Train SDK for healthcare. Implementing GPUDirect storage in the Weka AI framework eliminates the I/O bottlenecks, and dramatically reduces latency to deliver full bandwidth to data-hungry applications and high performance at scale for data-intensive applications. 

VMblog:  What sectors would benefit from this solution?

Manjrekar:  From medical imaging to accelerated genomics in the healthcare industry to financial services solutions and advanced driver assistance systems, Weka AI gives chief data officers, data scientists and data engineers the capabilities they need to get results. Weka AI can also be used in deep learning pipelines, providing the data collection and workspaces required for deep neural network (DNN) training. It offers simulation, inference, and lifecycle management across the entire data pipeline, which helps any organization and sector to meet the requirements of modern AI applications. 

VMblog:  What value does Weka AI deliver to the business?

Manjrekar:  Weka AI is designed to help businesses move seamlessly into their data transformation journey, providing lines of business and IT with the tools they need to implement AI 2.0 and cognitive computing workflows that deliver actionable insights and accelerated DataOps. It solves the storage challenges associated with I/O-intensive workloads and delivering production-ready solutions. It also provides the business with operational agility with versioning, explainability, and reproducibility. Another advantage is that the system also ticks the boxes of governance and compliance thanks to its in-line encryption and data protection. Businesses can also benefit from Weka AI solutions that are being engineered with partners in the WIN program, giving them access to a variety of capabilities and functionalities across the entire data pipeline. The flexibility and scalability of the framework help drive innovation through customized solutions aimed at solving particular business and industry challenges to deliver improved business outcomes. 

VMblog:  What's driving the evolution of frameworks such as Weka AI?

Manjrekar:  AI is becoming entrenched in traditional high-performance computing, and high-performance data analytics markets and data has become the most important strategic asset for the business. The rise and rise of accelerated DataOps is being driven by the need for actionable intelligence, the operationalization of pipelines, and the growing demand for governance and trust. These factors determine the success of any digital organization, and this means that companies have to reconsider how they architect their storage stacks and make purchasing decisions. They need to enable accelerated DataOps at scale and on time.


Published Thursday, April 23, 2020 7:34 AM by David Marshall
Filed under: , ,
There are no comments for this post.
To post a comment, you must be a registered user. Registration is free and easy! Sign up now!
<April 2020>