IBM and Together AI agree to scale open-source AI inference using NVIDIA infrastructure on IBM Cloud

IBM has announced a collaboration with Together AI aimed at scaling the use of open-source artificial intelligence models for inference, the process of running a trained model to generate outputs for users and applications. The company said the effort will be delivered with NVIDIA AI infrastructure and run on IBM Cloud.

In a separate announcement reported by Yahoo Finance, IBM and Together AI outlined a multi-year agreement valued at $240 million between IBM and Together AI. The arrangement is structured to position IBM to deploy a large cluster supporting the inference workloads associated with the deployment of open-source AI models.

The announcement ties the operational build-out to NVIDIA’s data center AI stack, positioning NVIDIA as the underlying infrastructure provider while IBM and Together AI focus on deploying and serving models. For NVIDIA, deals that expand inference capacity are part of a broader pattern in which companies pursue ways to put trained models into reliable, low-latency production environments.

Inference has become a major part of the AI spending conversation as firms move beyond training large models and toward running them at scale. Companies typically need substantial compute, optimized networking, and software tooling to manage throughput and cost per request, which is why infrastructure partnerships remain central to rollout plans for production AI services.

IBM did not, in the report summarized by Yahoo Finance, provide additional specifics such as target model families, performance benchmarks, or timelines for when the cluster capacity would be available to customers. The company also did not publicly disclose whether the agreement includes exclusive supply terms, specific service-level commitments, or particular vertical deployments within the scope of the partnership.

A key element of the collaboration is the “open-source inference” focus, which generally refers to serving models whose code and weights are made available under open licensing. For enterprises, open-source approaches can offer flexibility in model choice and potential cost control, but they still require enterprise-grade infrastructure to operate reliably at scale.

For sector context, the move reflects continued competition across the AI platform layer, where cloud providers and AI infrastructure companies try to pull workloads onto their stacks. These partnerships can also strengthen customer lock-in, as enterprises may prefer to keep model serving on a consistent platform that integrates compute, orchestration, and support.

What remains unclear is how IBM and Together AI will measure success beyond general scaling goals. The announcement does not spell out expected utilization, customer adoption targets, or any public roadmap for expanding the cluster beyond the initial deployment described in the report.