NVIDIA and AWS pair Blackwell GPUs with OpenSearch Serverless to push AI into production

NVIDIA and Amazon Web Services are rolling out a tighter stack for deploying AI systems at scale, focusing on the parts that tend to break first in production: inference latency, vector search speed, and infrastructure that can scale up or down without adding new operational work for engineering teams.

In an NVIDIA blog post published June 24, the companies say they are integrating NVIDIA’s AI infrastructure across AWS services including Amazon EC2 and Amazon OpenSearch. The goal, as described by NVIDIA, is to help enterprises stand up GPU-accelerated workloads for both compute-heavy inference and retrieval-based AI applications with fewer custom components.

On the compute side, NVIDIA introduced a new AWS instance line, called Amazon EC2 G7, powered by NVIDIA RTX PRO 4500 Blackwell Server Edition GPUs. NVIDIA positioned G7 as an instance type designed for production workloads across AI inference, graphics, video and GPU-accelerated data analytics. The company said G7 is available in one-, two-, four- and eight-GPU configurations and that bare metal options are “coming soon.”

NVIDIA also gave performance and capacity figures to distinguish G7 from the prior generation, G6. The company said G7 delivers up to 4.6x AI inference performance versus G6, up to 2.1x graphics performance, and significantly faster GPU-accelerated data analytics on Amazon EMR for Apache Spark using the NVIDIA cuDF library. It also described the platform’s right-sizing posture, saying customers can scale to the number of GPUs and memory their workloads actually need rather than over-provisioning a dedicated GPU platform.

For teams that rely on large-scale retrieval, NVIDIA said AWS’s next-generation Amazon OpenSearch Serverless is shifting to GPU-accelerated vector indexing using NVIDIA cuVS. cuVS is NVIDIA’s GPU-accelerated library for vector indexing, aimed at speeding up similarity search against embeddings, which are numeric representations of text, images or other data. NVIDIA said cuVS becomes the default compute choice for all vector collections in OpenSearch Serverless, and that this matters for retrieval-augmented generation (RAG), semantic search, recommendation systems and agentic AI applications.

NVIDIA quantified the potential retrieval improvement, saying vector indexing can be up to 10x faster at a quarter of the cost compared with CPU-only vector database builds. The company tied this to a practical deployment milestone, saying billion-scale vector databases could be built in under an hour, framing the update as a way to make production-grade retrieval pipelines more attainable for teams without specialized performance tuning.

Beyond runtime inference and search, the collaboration includes assurances for training workloads. NVIDIA said AWS has achieved NVIDIA Exemplar Cloud status for NVIDIA GB300 on training performance. NVIDIA described Exemplar Clouds as a performance evaluation and benchmark framework, where AWS meets thresholds used to compare cloud performance against NVIDIA’s reference architecture through co-engineering between the companies.

NVIDIA said Exemplar Clouds is meant to give developers more confidence that their training workloads will run on cloud infrastructure with consistent, high performance, while also helping teams improve total cost of ownership and move from planning to production more efficiently. For enterprises, that is often the harder part of the AI procurement cycle, where teams must validate training throughput and cost before they commit to a long-lived platform.

The release also detailed how customers can access the new building blocks through common AWS software paths. NVIDIA said G7 instances can be reached via AWS Deep Learning Amazon Machine Images (AMIs), Amazon Deep Learning Containers, Amazon EMR, Amazon EKS, Amazon ECS, and graphics AMIs, with support coming soon to Amazon SageMaker AI. On the storage and networking side, NVIDIA cited support for up to eight GPUs, total GPU memory up to 256GB, up to 700 Gbps of EFA-enabled networking (Enhanced Fabric Adapter), and up to 7.6TB of local NVMe SSD storage across single- to eight-GPU configurations.

It remains unclear from the blog post exactly when every component is broadly available, which regions it will roll out to first, and what specific application benchmarks NVIDIA expects customers to reproduce in their own environments. NVIDIA also did not provide concrete pricing for EC2 G7 or for OpenSearch Serverless vector collections; it only cited relative performance and cost comparisons and the operational promise of “serverless scaling” when workloads are idle.

For the next phase, the main thing to watch is whether customers can translate these infrastructure changes into measurable deployment outcomes, such as lower latency for inference-driven applications and faster time-to-production for RAG and vector search workloads. If the promised retrieval speedups and the Exemplar Cloud training assurances hold up in customer testing, the pairing of G7 compute plus cuVS-backed OpenSearch Serverless could become a more standard reference architecture for production AI systems built on AWS. However, teams will likely still need to validate performance, security controls, and cost under their own data distributions and workload patterns.