OpenAI and Broadcom (AVGO) introduce “Jalapeño,” a custom accelerator aimed at speeding up large language model inference

OpenAI and Broadcom said they have introduced “Jalapeño,” a custom AI accelerator intended to improve performance for large language model inference, the phase of an AI workflow where an already-trained model generates output in response to prompts.

The announcement, reported by Yahoo Finance, frames Jalapeño as a hardware solution designed specifically for inference rather than training. That distinction matters because inference workloads are often the dominant cost driver for deployed AI systems, where models are queried repeatedly across many users and products.

Broadcom is identified in the report as a participant in developing the accelerator. The piece also indicates the effort involved significant engineering work, describing Jalapeño as something developed over roughly nine months, though it did not provide additional technical specifications in the information available for this editorial draft.

In practical terms, accelerators are typically deployed to reduce time-to-response and lower the compute cost per request. For AI providers and enterprise customers, even modest efficiency gains can translate into lower data-center power use, more capacity per server, or the ability to serve more simultaneous users without expanding infrastructure.

Broadcom, whose shares trade under the ticker AVGO, has been active in the infrastructure buildout around AI compute, including networking and custom silicon efforts that tie into accelerator ecosystems. The Jalapeño announcement fits that broader pattern by aiming at the “last mile” of inference deployment: running models after they are trained.

OpenAI and Broadcom did not, in the information surfaced here, disclose key performance details such as throughput numbers, power consumption targets, batch sizing, or comparative results versus other accelerator platforms. The report also does not clarify where Jalapeño will first be deployed, such as which cloud environments, system vendors, or model families.

That leaves open several questions that typically matter to customers evaluating accelerator rollouts: whether Jalapeño is intended for a narrow set of inference runtimes, how it integrates with existing inference software stacks, and what pricing or availability timeline the companies plan to follow for partners and enterprise users.

Investors and industry watchers will likely focus next on whether the companies publish benchmark data, production deployment milestones, and partner announcements that show Jalapeño’s role in real inference workloads, rather than pilots or internal testing.