Two Amazon engineers just published the most compelling case study we've seen for not using a frontier model. Carr Onstott and Mehul Jaiswal swapped Claude Sonnet 4.6 for Strands Decider, a tiny open-weight decision model, on one of Amazon's largest internal ticketing systems — and cut per-ticket AI costs from $0.022 to $0.0005. That's a 98% reduction. The change will save millions of dollars in annual token spend.
This isn't a story about AI getting cheaper. Frontier model prices are already falling. This is a story about a more fundamental question that most businesses deploying AI haven't asked yet: does this task actually need a large language model at all?
What a decision model is — and isn't
A decision model can't write an email, draft a contract, or hold a conversation. It does exactly one thing: given some context and a set of options, it returns a probability for each option plus a confidence score. No text generation. No reasoning chains. Just a fast, calibrated classification.
Strands Decider takes a pre-trained LLM (in this case, Google's Gemma 4) and replaces its text-generation head with a "pointer head" that scores options. The result is a model with roughly 2 to 4 billion effective parameters that answers classification questions in tens of milliseconds on a single GPU. The weights are open-source under Apache 2.0 on Hugging Face.
Amazon's ticketing system routes customer escalations, capacity requests, automated alarms, and security findings into 20 category types. Claude Sonnet 4.6 had been doing this job well — generating JSON reasoning against a fixed set of labels on every ticket. But it was doing it with 6,724 tokens per ticket, including 170 output tokens of structured reasoning that nobody needed. Strands Decider does the same job with 2,010 tokens and a single output token: the choice.
How they actually did it
The engineering deserves attention because it's a template any team could follow.
Onstott and Jaiswal didn't flip a switch. They built a three-way configuration in AWS AppConfig that let them run both models on live production traffic simultaneously. In DUAL mode, both Sonnet and Strands Decider labelled every ticket in parallel, with Sonnet's labels applied to production while Decider's results were logged and compared. Only after validating that Decider matched or beat Sonnet on a separate test set did they switch to DECIDER mode — and even then, rolling back was a 45-second config change.
The architecture itself is standard AWS: ticket events land in an SQS queue, triggering a Lambda function that calls a SageMaker endpoint running Strands Decider on ml.g5.xlarge instances (one A10G GPU, 24 GB). The model uses about 19 GiB and handles roughly 5,600 tickets per hour per instance.
One unexpected finding: JSON key order changed the model's output probabilities by up to 0.19. They now sort all keys before sending requests — a subtle gotcha that any team deploying decision models should note.
The bigger picture: enterprises are overspending on frontier models
Amazon's case study lands in the middle of a growing industry reckoning over AI costs. According to a Vista Equity Partners analysis, the average enterprise spent roughly $7 million on AI model usage in 2025, nearly triple the $2.5 million spent in 2024. Some enterprises now report monthly inference bills in the tens of millions.
The problem isn't that frontier models are expensive — it's that most enterprise tasks don't need them. Only 11% of organisations can forecast their AI costs to within 10% of actuals, and a quarter have delayed or cancelled AI initiatives because of unexpected costs. Agentic workflows compound the issue: according to Gartner, agentic models require between five and thirty times more tokens per task than a standard chatbot interaction.
The model routing industry is responding. TrueFoundry's Auto Routing benchmark showed a 69% cost reduction while retaining 98% of quality compared to an all-Opus setup. LangChain's Switchyard evaluation found that only 7% of agent turns actually needed a frontier model — those calls carried 68% of the total bill, while a 30-billion-parameter model handled the other 93% of turns.
If you've been following our coverage of Uber's tokenmaxxing budget blowout, this is the counter-narrative. The answer isn't necessarily to spend less on AI — it's to spend it where it matters.
What this means for your business
The lesson from Amazon's case study is deceptively simple: audit your AI workloads by task type, not by model preference.
If you're running classification, routing, triage, or labelling tasks through a frontier model, you're almost certainly overpaying. These are closed-choice problems with known option sets — exactly the kind of work that decision models and fine-tuned small models handle at a fraction of the cost.
Vista Equity Partners found that disciplined inference management — combining model routing, context optimisation, and agent design — can deliver cost reductions of 80% or more, with accuracy within one to two percentage points of the most expensive alternative.
The practical steps aren't complicated:
- Identify your closed-choice tasks. Ticket classification, document routing, sentiment labelling, approval workflows — anything with a fixed set of outcomes.
- Run both models in parallel. Amazon's three-way switch pattern (
BEDROCK/DUAL/DECIDER) is the gold standard for safe migration. It told them more than any offline evaluation could. - Measure cost per successful outcome, not cost per token. As Futurum Group noted, only about 36% of AI platform decision-makers track cost per request, and barely 15% track token efficiency.
What to watch
Strands Decider is part of a broader shift. The Strands Agents SDK, the open-source agent framework used in production by Amazon Q Developer, AWS Glue, and VPC Reachability Analyser, is designed around the idea that different parts of an agentic workflow need different models. Decision models handle the routine routing and classification. Frontier models handle the genuinely hard reasoning. The agent orchestrates both.
The arrival of cheap, capable agent models from Meta and others accelerates this trend. We're heading towards a world where the smartest AI systems aren't the ones running the biggest models — they're the ones that know which model to use for which task.
For businesses spending more than a few thousand dollars a month on AI inference, this case study should be a wake-up call. The 98% cost reduction is dramatic, but the methodology — run both, compare on live traffic, switch when confident — is something any team can implement. The only question is whether you'll do it before or after you've overspent.
Sources
- Amazon saves millions by replacing Sonnet 4.6 with Strands Decider — Strands Agents Blog
- Introducing Strands Decider 2B — Strands Agents Blog
- Strands Decider E4B Gemma 4 model card — Hugging Face
- Understanding Inference and the Economics of Enterprise AI — Vista Equity Partners
- The Meter Is Running: Forecasting and Governing AI Spend — Quantum Rise
- From Tokens to Outcomes — Monderman
- Cut LLM Costs with Auto Routing — TrueFoundry
- How many of your agent's calls actually need a frontier model? — LangChain
- The Token Cost Reckoning Arrives in the CFO's Office — Futurum Group
