Jensen Huang says NVIDIA wants AI “token” costs to keep dropping, aiming for 30 to 40x per year

NVIDIA CEO Jensen Huang said he wants the cost of an AI “token” to keep falling at a rapid pace, describing an objective of cost declines on the order of 30 to 40 times per year. In modern AI systems, a token is a unit of text (and sometimes code) that models read and generate, so the “cost per token” is a shorthand for the computing expense required to process and produce language output at scale.

Huang’s comments, reported in a market news write-up linked to Yahoo Finance coverage, frame cheaper tokens as central to NVIDIA’s business outlook. The logic is that AI usage can expand quickly when token costs fall, even if per-unit economics are pressured downward. If demand for inference (running a model to answer requests) grows faster than prices decline, NVIDIA can still see robust revenue tied to the hardware and software ecosystem that supports those workloads.

The company’s strategy, as characterized in the reporting, is not simply to reduce costs once. Huang’s emphasis on token costs falling repeatedly through the year suggests a continuing engineering and supply-chain effort aimed at reducing the compute and infrastructure cost required for each unit of AI output.

NVIDIA is already closely associated with the underlying infrastructure for AI inference and training, and the market routinely treats its roadmap as a proxy for how quickly large-scale model deployments can become cheaper. Token-cost reductions typically depend on multiple factors, including improvements in GPUs and accelerated compute, efficiency gains in model architecture and compilation, and better utilization of data center resources, though the cited reporting does not break out which levers NVIDIA highlighted.

While the token-cost goal described by Huang is stark, the disclosed details in the coverage are limited. The reporting does not provide a specific time window beyond “per year,” does not define whether the “30, 40 times” language refers to percentage changes, absolute price changes, or another measurement framework, and does not specify the exact components included in the token cost calculation (for example, whether it covers only GPU inference cost or also adds networking, memory, and datacenter overhead).

The economics matter across the AI industry because most enterprise and consumer use cases are sensitive to per-request costs. If token costs drop quickly enough, product teams can expand features that require more model output, such as longer responses, additional tool use, or higher-quality generation, which can increase total compute consumption.

For NVIDIA, the broader implication is that sustaining rapid cost-down trajectories could help keep demand for AI infrastructure growing through successive hardware and software cycles. Cheaper tokens can also encourage higher throughput deployments, where more requests are processed per second, which can support the broader data center utilization that underpins the company’s compute platform.

Investors and customers will likely look next for tighter quantification and operational context, such as any follow-on statements from NVIDIA on how it measures token costs and what specific technical or product milestones support the target pace of decline. The company’s blog and investor communications may offer additional clarity on the initiatives behind token-level efficiency, if and when those details are released.