Databricks Expands NVIDIA Integration for Enhanced Data Workloads

Databricks announced an expanded collaboration with NVIDIA at the Data+ AI Summit, focusing on optimizing data and AI workloads by integrating NVIDIA CUDA accelerated computing into Databricks’ Data Intelligence Platform. This collaboration aims to improve the efficiency, accuracy, and performance of AI development pipelines crucial for modern AI applications. Databricks is introducing native support for NVIDIA GPU acceleration across its platform, enhancing enterprise experiences in training ML models, deploying generative AI applications, and optimizing digital twins.

The partnership underscores Databricks and NVIDIA’s commitment to delivering enhanced solutions for enterprises.

“We’re thrilled to continue growing our partnership with NVIDIA to deliver on the promise of data intelligence for our customers from analytics use cases to AI,” said Ali Ghodsi, Co-founder and CEO at Databricks. “Together with NVIDIA, we’re to assisting every organization build their own AI factories on their private data.”

“Data is the fuel for the generative AI industrial revolution, so reducing data processing energy demands with accelerated computing is essential to sustainable AI platforms,” said Jensen Huang, founder and CEO of NVIDIA. “Databricks is the pioneer of large-scale data processing. By bringing NVIDIA CUDA acceleration to Databricks’ core computing stack, we’re laying the foundation for customers everywhere to use their data to power enterprise generative AI.”

Integrating Native Support with Photon and NVIDIA

Databricks plans to integrate native support for NVIDIA-accelerated computing into its next-generation vectorized query engine, Photon. This enhancement aims to significantly improve speed and efficiency for customers’ data warehousing and analytics workloads. Photon powers Databricks SQL, the company’s serverless data warehouse known for its industry-leading price performance and total cost of ownership (TCO). Databricks and NVIDIA anticipate that this collaboration will redefine price-performance standards in data processing.

At COMPUTEX, Databricks introduced DBRX as an NVIDIA NIM microservice. NVIDIA NIM inference microservices offer fully optimized, pre-built containers for deploying generative AI models anywhere, dramatically enhancing enterprise developer productivity. Launched in March 2024, DBRX utilizes Databricks’ tools and NVIDIA DGX Cloud, a scalable AI platform, for model training.

Organizations can customize DBRX with enterprise data to create high-quality, organization-specific models or use it as a reference architecture for building a custom mixture of expert (MoE) models.

Databricks’ Data Intelligence Platform provides a comprehensive solution for building, evaluating, deploying, securing, and monitoring end-to-end generative AI applications. With Databricks Mosaic AI’s data-centric approach, customers can scale generative AI applications on their unique data securely, accurately, and in compliance with governance requirements.

This announcement follows Databricks' acquisition of Tabular, a data management startup, and its commitment to enhancing the Delta Sharing open ecosystem. Databricks continues to innovate in response to increasing demand for data and AI capabilities, achieving over $1.6 billion in revenue for its fiscal year ending January 31, 2024, marking more than 50% year-over-year growth.

Read more from Bharti Trehan..

Read IT Product News here..