New Relic Introduces the First APM for AI in the Industry

New Relic has introduced New Relic AI Monitoring (AIM), marking the industry's inaugural APM solution tailored for AI-powered applications. With AI observability, AIM from New Relic grants engineers unparalleled visibility and insights across the AI application stack. This facilitates streamlined troubleshooting and optimization of AI applications, addressing aspects such as performance, quality, cost, responsible AI use, and compliance with upcoming regulations and standards. Boasting over 50 integrations and features like LLM response tracing and model comparison, AIM empowers teams to develop and operate Large Language Model (LLM)-based applications with confidence.

AI-driven applications introduce heightened complexity to organizational processes. Elements such as large language models (LLMs) and vector databases within AI are often opaque to engineers, carrying the risk of delivering inaccurate or biased results, introducing security vulnerabilities, and generating substantial telemetry data requiring tracking and analysis.

New Relic's AIM addresses these challenges by offering comprehensive observability for all AI-powered applications. Engineers gain access to a unified interface for troubleshooting, comparing, and optimizing various LLM prompts and responses across dimensions like performance, cost, security, and quality. This includes addressing issues such as hallucinations, bias, toxicity, and fairness.

AIM ensures engineers have complete visibility into every facet of the AI stack, including services and infrastructure, equipping them with the necessary data for the responsible use of AI and to demonstrate compliance with emerging AI regulations.

“Just about every organization is integrating AI applications into their tech stacks to provide better customer experiences and improve efficiency, with the hope of improving their bottom line. The trade-off is that AI creates complexity in their tech stack and must be adopted responsibly with security, quality, compliance, and cost top of mind,” said IDC Group Vice President Stephen Elliot. “Applying observability to AI applications is a smart and efficient way to address these complexities so that companies can scale and drive innovation. Any company that provides these solutions is ultimately enabling organizations to deliver better products and customer experiences.”

“With every organization integrating AI into their products and processes, AI workloads are now part of modern organisations’ application architectures,” New Relic Chief Product Officer Manav Khurana. “With AI monitoring, we have applied our deep expertise from inventing cloud APM to providing end-to-end visibility into AI-powered applications to help businesses manage performance and costs while complying with emerging AI regulations and standards.”

Key features and use cases include:

– Auto instrumentation: New Relic agents come fully equipped with all AIM functionalities, ensuring swift and effortless setup, encompassing comprehensive AI stack visibility, response tracing, model comparison, and more.

– Full AI Stack Visibility: Comprehensive perspective spanning the application, infrastructure, and the AI layer, encompassing AI metrics such as response quality and tokens, in addition to APM golden signals.

– Deep trace insights for every LLM response: Track the entire lifecycle of intricate Large Language Model (LLM) responses crafted using tools like LangChain to address performance challenges and quality issues, including bias, toxicity, and hallucination.

– Compare performance and costs: Monitor usage, performance, quality, and cost for all models through a unified interface; enhance efficiency with insights into frequently asked prompts, chains of thought, and prompt templates and caches.

– Enable responsible use of AI: Guarantee the safe and responsible use of AI by confirming that responses are properly tagged as AI-generated and free from bias, toxicity, and hallucinations, leveraging insights from response trace analysis.

– Instantly monitor your AI ecosystem: The most extensive solution for monitoring the complete stack of any AI ecosystem, featuring over 50 integrations and quickstarts, encompassing:

• Orchestration framework: LangChain
• LLM: OpenAI, PaLM2, HuggingFace
• Machine learning libraries: Pytorch, TensorFlow
• Model serving: Amazon SageMaker, AzureML
• Vector databases: Pinecone, Weaviate, Milvus, FAISS
• AI infrastructure: Azure, AWS, GCP

“By seamlessly integrating the Pinecone vector database with New Relic AI monitoring, we are providing transparency and actionable insights to help developers build better search and Generative AI applications in the enterprise,” said Pinecone Founder and CEO Edo Liberty. “We have seen incredible demand for vector databases as companies build and deploy AI applications since it is a core part of the AI stack. Now developers can build better search and Generative AI solutions by ensuring relevant and fast responses alongside their AI observability practice.”