In this latest VMblog interview, Jemiah Sius, Sr Director of Developer Relations at New Relic, discusses the company’s groundbreaking integration with NVIDIA NIM. As the first full-stack observability platform to support NVIDIA NIM, New Relic is positioning itself at the forefront of enterprise AI adoption. This partnership aims to simplify AI application deployment, reduce costs, and optimize performance through comprehensive observability tools.
With Gartner predicting that over 80% of enterprises will use or deploy GenAI apps by 2026, New Relic’s integration comes at a crucial time for organizations seeking to gain a competitive edge in the rapidly evolving AI landscape.
In this Q&A, Sius explores the key features, benefits, and potential industry impacts of this significant collaboration between New Relic and NVIDIA.
VMblog: Can you walk us through New Relic’s recent integration with NVIDIA?
Jemiah Sius: New Relic is the first full-stack observability platform to support NVIDIA NIM. The extension of our observability platform to support AI apps built with NVIDIA NIM is a crucial addition to our expanding ecosystem of over sixty AI integrations, including NVIDIA GPUs and NVIDIA Triton Inference Server software. For organizations pursuing enterprise AI adoption with NVIDIA’s AI technology, New Relic’s observability toolkit provides DEV and IT teams with the expertise to fine-tune their AI app deployments with detailed visibility, performance metrics, and operational insights. This announcement follows New Relic’s recent integration with NVIDIA’s AIOps partner ecosystem. By leveraging NVIDIA AI’s accelerated computing, New Relic combines the power of observability and AI to streamline IT operations through its machine learning, and generative AI assistant, New Relic AI.
A central focus of this partnership is reducing the complexity and costs associated with developing, deploying, and monitoring AI applications. Together, this approach can address the high costs and performance issues companies face when deploying LLMs. Our goal is to ultimately accelerate AI adoption and support organizations in achieving ROI on their AI investments and deployments.
VMblog: What role does observability play in adopting AI, and how does this integration support businesses looking to deploy AI?
Sius: Gartner predicts that over 80% of enterprises will use GenAI or deploy GenAI apps by 2026. Quick deployment and faster ROI are crucial for organizations to gain or maintain market advantage in a competitive AI landscape. Observability provides the essential cost and performance data that is key to optimizing AI applications. It supplies a holistic, real-time view of the AI application stack across services, NVIDIA GPU-based infrastructure, and the AI layer along with comprehensive metrics on throughput and latency. This in-depth monitoring also traces the request flows across services and models to understand the inner workings of AI apps. Pursuing enterprise AI adoption without observability is akin to ‘flying blind’.
VMblog: What are some key features and use cases of the integration?
Sius: One important feature is the capability of generating deep trace insights for each response, which addresses crucial performance issues like bias and hallucinations. In addition, model inventory tracks key metrics across the NIM microservices to isolate model-related performance easily. Finally, deep GPU insights analyze critical accelerated computing metrics such as utilization and temperature.
Overall, these use cases involve managing the costs of your stack, effectively deploying AI models, understanding model performance, and improving customer experience by providing higher-quality service.
VMblog: What are the benefits of the integration of New Relic AI monitoring with NVIDIA NIM?
Sius: First and foremost, it provides New Relic AI monitoring to customers with applications built with NVIDIA NIM. Organizations that are rapidly adopting AI to enhance digital experiences, boost productivity, and drive revenue using NVIDIA’s AI technology, NVIDIA NIM, or any of New Relic’s other AI partners will have full visibility to improve their AI applications. For most enterprises, it will take months, if not years, before they start seeing the returns from their AI investments. This integration is to help our partners and our customers increase adoption and to the market quicker. Starting with our simplified NIM setup, enterprises can ensure that their data is safe and secure. For partners, the same benefit if it was a large-scale enterprise company with their internal engineers developing, let’s say, an LLM or a copilot. There’s going to be benefits on both sides.
VMblog: Which AI models are supported by New Relic’s extended monitoring for NVIDIA NIM?
Sius: New Relic’s extended monitoring for NVIDIA NIM supports a wide range of AI models, including Databrick’s DBRX, Google’s Gemma, Meta’s Llama 3, Microsoft’s Phi-3, Mistral Large and Mixtral 8x22B, and Snowflake’s Arctic.
VMblog: As more organizations pursue AI, what changes do you expect to see across industries?
Sius: Concerns about GPUs, chips, infrastructure, and LLMs have understandably dominated the AI adoption conversation. Now, as more organizations and industries are assembling all of these puzzle pieces together so they can offer new AI services to their customers, there will be an increased focus on the assembly and corresponding harmony of these components that only observability can offer. Organizations that can reap the benefits of faster AI ROI, improved customer experiences, and enhanced operational insights will garner the attention of the tech industry more broadly. We anticipate that the accelerated integration of advanced observability tools will be a part of the recipe from notable, first-to-market successful deployment ‘recipes’. In turn, this will drive more widespread and effective use of AI, leading to competitive advantages and the transformation of business processes and services across various sectors.
##
