New Relic, the all-in-one observability platform for every engineer, has announced it is integrating its platform with NVIDIA NIM inference microservices to reduce the complexity and costs of developing, deploying, and monitoring Generative Artificial Intelligence (GenAI) apps.
Now, customers can use New Relic AI monitoring to gain comprehensive visibility across the AI stack for applications built with NVIDIA NIM, all with a simplified setup while ensuring data security. This complements the robust security features and ease of use of NVIDIA NIM’s self-hosted models, which accelerates generative AI application delivery. Together, New Relic integrated with NVIDIA NIM can help customers adopt AI faster and achieve quicker ROI.
Observability Is Mission Critical to Deploying Cost-Effective, High-Performance Models
Organisations are rapidly adopting Generative AI to enhance digital experiences, boost productivity, and drive revenue. Gartner predicts that over 80% of enterprises will use GenAI or deploy GenAI apps by 2026. Quick deployment and faster ROI are crucial for organisations to gain market advantage and Observability is the key. It offers a holistic, real-time view of the AI application stack—across services, infrastructure, and the AI layer—to ensure efficient, reliable, and cost-effective operation.
“In today’s hyper-competitive market, organisations cannot afford to wait years for AI ROI,” said New Relic CEO Ashan Willy. “Observability solves this by providing visibility across the AI stack. We are pioneering AI observability by extending our platform to include AI apps built with NVIDIA NIM. Combining NVIDIA’s AI technology with our expertise in observability and APM gives enterprises a competitive edge in the AI race.”
“As Asia Pacific organisations brace for huge growth in generative AI users within the next five years, it is crucial for organisations to ensure they have visibility into their AI apps,” said New Relic Chief Architect, APJ Peter Marelas. “Our support for NVIDIA NIM allows organisations to understand how their NVIDIA NIM deployments are performing in real-time.”
“As enterprises race to adopt generative AI, NVIDIA NIM can help businesses quickly deploy applications in production,” said NVIDIA Director of AI Software Amanda Saunders. “New Relic’s integration with NVIDIA NIM enables IT and development teams to optimise their AI applications by rapidly observing and responding to operational insights.”
New Relic Speeds Faster ROI for AI Applications Built with NVIDIA NIM
AI applications can complicate tech stacks, increase security concerns, and be cost prohibitive. New Relic AI monitoring provides a comprehensive view of the AI stack, along with key metrics on throughput, latency, and costs while ensuring data privacy.
It also traces the request flows across services and models to understand the inner workings of AI apps. New Relic extends its in-depth monitoring to NVIDIA NIM, supporting a wide range of AI models including—Databricks DBRX, Google’s Gemma, Meta’s Llama 3, Microsoft’s Phi-3, Mistral Large and Mixtral 8x22B, and Snowflake’s Arctic. This helps organisations deploy AI applications built with NVIDIA NIM confidently, accelerate time-to-market, and improve ROI.
Key features and use cases for AI monitoring include:
- Full AI stack visibility. Spot issues faster with a holistic view across apps, NVIDIA GPU-based infrastructure, AI layer, response quality, token count, and APM golden signals.
- Deep trace insights for every response. Fix performance and quality issues like bias, toxicity, and hallucinations by tracing the entire lifecycle of AI responses
- Model inventory.Easily isolate model-related performance, error, and cost issues by tracking key metrics across all NVIDIA NIM inference microservices in one place
- Model comparison. Compare the performance of NVIDIA NIM inference microservices running in production in a single view to optimise model choice based on infrastructure and user needs.
- Deep GPU insights. Analyse critical accelerated computing metrics such as GPU utilisation, temperature, and performance states; understand context and resolve problems faster.
- Enhanced data security. In addition to NVIDIA’s self-hosted model’s security advantage, New Relic allows you to exclude monitoring of sensitive data (PII) in your AI requests and responses.
New Relic Deepens 60+ AI Integration Ecosystem with NVIDIA
This integration follows New Relic’s recent addition to NVIDIA’s AIOps partner ecosystem. Leveraging NVIDIA AI’s accelerated computing, New Relic combines observability and AI to streamline IT operations and accelerate innovation through its machine learning, and generative AI assistant, New Relic AI. New Relic offers the most comprehensive observability solution with 60+ AI integrations including NVIDIA GPUs and NVIDIA Triton Inference Server software.
New Relic AI monitoring is available as part of its all-in-one observability platform and offered via its usage-based pricing model. Get started by contacting your New Relic account representative or signing up for a free account.
Archive
- October 2024(44)
- September 2024(94)
- August 2024(100)
- July 2024(99)
- June 2024(126)
- May 2024(155)
- April 2024(123)
- March 2024(112)
- February 2024(109)
- January 2024(95)
- December 2023(56)
- November 2023(86)
- October 2023(97)
- September 2023(89)
- August 2023(101)
- July 2023(104)
- June 2023(113)
- May 2023(103)
- April 2023(93)
- March 2023(129)
- February 2023(77)
- January 2023(91)
- December 2022(90)
- November 2022(125)
- October 2022(117)
- September 2022(137)
- August 2022(119)
- July 2022(99)
- June 2022(128)
- May 2022(112)
- April 2022(108)
- March 2022(121)
- February 2022(93)
- January 2022(110)
- December 2021(92)
- November 2021(107)
- October 2021(101)
- September 2021(81)
- August 2021(74)
- July 2021(78)
- June 2021(92)
- May 2021(67)
- April 2021(79)
- March 2021(79)
- February 2021(58)
- January 2021(55)
- December 2020(56)
- November 2020(59)
- October 2020(78)
- September 2020(72)
- August 2020(64)
- July 2020(71)
- June 2020(74)
- May 2020(50)
- April 2020(71)
- March 2020(71)
- February 2020(58)
- January 2020(62)
- December 2019(57)
- November 2019(64)
- October 2019(25)
- September 2019(24)
- August 2019(14)
- July 2019(23)
- June 2019(54)
- May 2019(82)
- April 2019(76)
- March 2019(71)
- February 2019(67)
- January 2019(75)
- December 2018(44)
- November 2018(47)
- October 2018(74)
- September 2018(54)
- August 2018(61)
- July 2018(72)
- June 2018(62)
- May 2018(62)
- April 2018(73)
- March 2018(76)
- February 2018(8)
- January 2018(7)
- December 2017(6)
- November 2017(8)
- October 2017(3)
- September 2017(4)
- August 2017(4)
- July 2017(2)
- June 2017(5)
- May 2017(6)
- April 2017(11)
- March 2017(8)
- February 2017(16)
- January 2017(10)
- December 2016(12)
- November 2016(20)
- October 2016(7)
- September 2016(102)
- August 2016(168)
- July 2016(141)
- June 2016(149)
- May 2016(117)
- April 2016(59)
- March 2016(85)
- February 2016(153)
- December 2015(150)