Salesforce details how it is cutting AI inference costs by redesigning the way its agents use models

Salesforce has laid out a new blueprint for controlling the cost of running large language models, arguing that the real problem is not just token pricing but the architecture of an AI agent itself. In a July 8 post, the company said enterprises often get squeezed by inference spend, the charges incurred each time an AI model processes requests, sometimes moving faster than companies can adjust their own pricing. Salesforce’s response is to route different parts of an agent’s workflow to different models, while building a harness that keeps those steps observable and governable.

The company said it has seen the symptoms in the market, including the way usage-based pricing can trigger backlash and the rapid ramp-up of AI costs when a single model handles too much. It also pointed to its own experience with Agentforce, an agent platform it has been rolling out, where it initially ran “entirely on a single rented model” and the token bill grew linearly with traffic. Salesforce said it could have passed rising costs to customers only for a limited period, because the bill would keep increasing as long as it rented the full “engine.”

Instead of repricing, Salesforce said it rebuilt the system. The company describes moving from a one-model-per-request design to a multi-model stack where targeted open-source models take on specific jobs. It says the architecture separates tasks such as intent detection, safety screening, evaluation, and retrieval ranking, while a frontier model remains responsible for core multi-step reasoning. Salesforce framed the shift as “precision over power,” arguing that general-purpose models handle every subtask less precisely and more expensively than models tuned to single roles.

A key element of Salesforce’s cost and control strategy is an “agentic harness,” which the company describes as the system-layer that grounds agent behavior in enterprise realities. Salesforce said it is based on its position as a system of record for enterprise work, drawing on telemetry from “hundreds of thousands of use cases across service, sales, and commerce.” The company argues that because each inference produces traces in its telemetry, it can identify common seams across agent workflows and then tune models to those repeatable steps.

In the intent and routing layer, Salesforce says it replaced a slower, general reasoning approach with a purpose-built classifier called the HyperClassifier. The company said it is fine-tuned on GPT-OSS-20B, an open-source Mixture-of-Experts architecture that it says is quantized to run fast, and that the model operates within what Salesforce calls the Einstein Trust Layer. It classifies customer requests across “up to 200 possible labels” and returns an answer in roughly 26 milliseconds, compared with about 1,446 milliseconds for a general frontier model. Salesforce also reported accuracy and stability gains, including an improvement in safety topic accuracy from 95% to 99% and a reduction in drift during multi-turn conversations from about 20% to 10%.

Salesforce said the HyperClassifier has become generally available in Spring 2026 and is now the default for Agentforce Service and Employee Agent templates. It also says the same model powers semantic endpointing in Agentforce Voice, detecting in real time when a speaker has finished an intent rather than simply pausing mid-sentence, a detail Salesforce positions as central to making conversations feel natural.

Beyond routing, Salesforce described multiple separate safety and quality checks layered around generation. For prompt injection and other malicious instructions, it said it built a dedicated injection-detection model trained on common attack surfaces such as CRM field values, inbound email bodies, form inputs, and transcript content. The model classifies six distinct attack vectors, including role-play and persona manipulation, prompt leakage, privilege escalation, encoding attacks like Base64 and hex, privacy attacks, and malicious code generation. Salesforce said every prompt receives an injection score that is logged in an audit trail, and it is developing a blocking mode intended to halt detected attacks before sending them to the reasoner.

For content safety, Salesforce said it launched a specialized model that screens each response for categories like violent, sexually explicit, hateful, or physically harmful content, assigning a toxicity score and logging it to the Einstein Trust Layer Audit Trail. It added that the agent also uses a separate evaluation model to avoid the blind spot it described in which LLMs can judge their own work too generously. Salesforce identified that evaluator as TextEval, which it describes as a 20-billion-parameter model fine-tuned from GPT-OSS-20B. It says TextEval evaluates whether answers are grounded in customer data, whether sources and citations are included, whether the agent followed its instructions, and whether the task was resolved. Salesforce said this check is built into the runtime and cannot be changed by administrators.

For retrieval, Salesforce said it runs its own text ranking model to rerank knowledge results before they enter the reasoning step, warning that the ordering of retrieved documents can steer the agent toward incorrect answers. It said it moved the reranker from third-party hosting to Amazon Bedrock while maintaining “zero customer-facing downtime” and latency parity, and that it now runs across regions including a multilingual reranker. Salesforce said the objective is not to eliminate frontier models but to limit where they are used, keeping the agent’s decision-making modular so customers can swap foundation models while leaving the surrounding harness and guardrails in place.