AWS CEO Matt Garman: AI “inference” is only a small fraction of where enterprise AI is headed

AWS CEO Matt Garman said that AI inference is still “even 1%” of what it will become, arguing that the technology is only beginning to scale beyond early use cases and prototypes. In comments carried by Yahoo Finance through a market-news report, Garman framed inference as the portion of the AI pipeline that runs once models are trained, when applications make predictions in production.

The remarks come as cloud providers and chip makers race to build the infrastructure needed for AI workloads, not just model development. Inference, in particular, is increasingly tied to data center capacity, specialized accelerators, and the software stacks that schedule and optimize workloads across large fleets of servers.

Garman also suggested that the AI infrastructure opportunity is already being reflected across the supply chain, with different companies capturing value in different ways. The report points to Amazon, Advanced Micro Devices (AMD), and Broadcom as firms positioned to benefit from inference-driven demand, though it characterizes their benefit mechanisms as distinct rather than identical.

For Amazon, the implication is straightforward: AWS is the primary route by which many enterprises consume AI compute, including inference at scale. AWS sells cloud capacity that customers can provision for running models behind applications, which makes inference demand directly tied to cloud usage.

For AMD and Broadcom, the report frames the relationship as more indirect but still consequential. AMD is known for designing data center processors and AI-focused accelerators used to run workloads, while Broadcom provides infrastructure and networking products that help move and manage data inside and between data centers. The report’s takeaway is that inference is expanding across multiple layers of the technology stack, not just at the model level.

The broader industry context is that enterprise AI spending is increasingly bifurcated between experimentation and production deployment. While training and experimentation garner most of the attention, inference volumes can become the dominant driver of ongoing compute consumption once applications go live. That dynamic can raise the importance of performance-per-watt, scheduling efficiency, and the ability to scale workloads reliably.

What the market-news report does not provide are specific figures tied to Garman’s “1%” characterization, such as how AWS measures inference progress, what time horizon he had in mind, or whether the company sees a particular vertical or workload type as the biggest driver. It also does not break down which customers or contracts are contributing most to inference demand, nor does it quantify expected infrastructure spend by AWS.

Still, the comments point to a consistent message in the cloud-and-chips ecosystem: today’s inference deployments may be only an early sample of the long-run demand curve. Investors and customers are likely to keep watching indicators such as cloud AI service uptake, accelerator supply and pricing, and networking capacity upgrades as the industry moves from limited rollouts toward larger production rollovers. -

Amazon’s public newsroom offers company updates on AWS and broader operations, but it does not appear to contain details matching the specific inference comment attributed to Garman in the cited market-news post. As a result, the most defensible elements for now are the qualitative characterization of inference’s early stage and the high-level idea that multiple infrastructure players can benefit as production AI scales.