/dqc/media/media_files/ODqIXaI3IcjkrLqFQDgd.png)
BT Group Launches GenAI Gateway for Large Language Model Integration
BT Group’s Digital Unit has introduced the GenAI Gateway, a platform designed to integrate large language models (LLMs) from providers such as Anthropic, Meta, Cohere, and Amazon. Built-in collaboration with AWS, the platform leverages services like Amazon Bedrock, Amazon SageMaker, and AWS Professional Services to provide secure access to natural-language processing and LLM capabilities. This platform is a key part of BT Group's strategy to embed AI across its business operations.
Challenges of Large-Scale LLM Adoption
While ad-hoc LLM usage is suitable for test environments, it is less practical for large-scale deployment due to concerns around cost control, security, and privacy. Monitoring LLM performance is critical to address issues such as unexpected errors and model degradation over time. The GenAI Gateway mitigates risks, including potential vendor lock-in, and ensures BT Group engineers can select the most appropriate model for each use case, with built-in budget tracking for cost management.
Centralized Management for Efficiency and Security
The GenAI Gateway platform consolidates efforts across BT Group, reducing duplication and simplifying management. It enables centralized control of APIs, security configurations, and infrastructure, lowering the cost and risk of maintaining separate LLMs for different use cases. The platform is deployed on AWS and accessed through secure APIs, in line with BT Group’s modular digital architecture.
Key Features of the GenAI Gateway
The platform uses Amazon Bedrock to provide a selection of foundation models from AI providers like AI21 Labs, Cohere, Stability AI, and Amazon. It also incorporates Amazon SageMaker to deliver high-performance, cost-efficient machine learning tools. The GenAI Gateway supports prompt security, chat history, FinOps billing, enterprise search, and integration with corporate data sources. It also includes privacy controls, such as tenant isolation for each use case, data residency within the UK, and filters for Personal Identifiable Information (PII). Built-in guardrails help prevent misuse and ensure ethical compliance.
Initial Use Cases and Future Plans
The GenAI Gateway is already in beta with initial use cases. In Openreach, the platform is used to summarize engineering notes for Ethernet and full fibre jobs, improving process efficiency and productivity. Another use case supports contract analysis for BT Group’s Business, Legal, and Procurement teams. BT Group also plans to integrate the GenAI Gateway with its “data fabric” platform to enforce data governance policies and manage access control and data sovereignty.
The GenAI Gateway marks a significant step in BT Group’s efforts to become an AI-enabled enterprise.
Fabio Cerone, GM EMEA Telco at AWS, said: “The BT Group GenAI Gateway is showcasing how enterprises can effectively deploy generative AI at scale and speed. It’s been a brilliant, pioneering opportunity to collaborate and work backward from the customer to provide a way to accelerate the deployment of generative AI use cases into production with embedded security and compliance. The GenAI Gateway will trigger the flywheel effect in the adoption of generative AI, delivering quicker results for BT Group and its customers.”
Deepika Adusumilli, Managing Director, Data & AI, BT Group’s Digital Unit said: “AI is helping us reimagine the future of our company. We believe that where our data is a constant, we need flexibility with our LLMs. GenAI Gateway allows us to tap into this powerful new set of technologies at scale, in a way that is safe, responsible, flexible, and scalable, delivering the ambition we have for AI to unlock the human potential within BT Group, today and in the future.”
Read More: