Project 04
Token Optimization
A small tool for seeing how caching, batching and response length affect model cost without hiding the quality trade-offs.
Baseline
£
Illustrated
£
Rates, assumptions & calculation
Each request starts with 1,000 input tokens and 500 output tokens. Example rates: £3 / million input tokens and £12 / million output tokens.
| Cost | Baseline | Scenario |
|---|---|---|
| Input | £30.00 | £19.50 |
| Output | £60.00 | £44.63 |
| Total | £90.00 | £64.13 |
Base cost = requests × tokens per request × rate ÷ 1,000,000. Cached input is assumed to cost 75% less. Batch requests are assumed to cost 50% less on input and output. Cache discounts apply only outside batches; shorter output applies to both paths.
Illustrative GBP assumptions, not live provider prices or a forecast. Excludes cache-write premiums, retries, tool calls, storage and tax. Savings only help if the answers still meet the quality bar.
What the project explores
A smaller bill starts with understanding the workload.
Models charge for the tokens they read and write. This interactive notebook makes that arithmetic inspectable: choose a workload, change one assumption, then compare the input and output costs. It is a cost modelling experiment; it does not connect to an account or change a production workload.
01 / Reuse stable input
Keep shared instructions and reference material at the start of a prompt. When a provider can reuse that prefix, the cached input can cost less. The slider represents input that actually receives a cache discount.
A repeated prompt is not a guaranteed cache hit. Prefix matching, minimum length and expiry depend on the provider.02 / Move work off the clock
Summaries, evaluations and overnight document processing can often wait. Put eligible requests into a batch instead of paying for an immediate response. This model assumes a 50% batch discount.
Waiting is the trade-off. Keep interactive work on the synchronous path; do not assume cache and batch discounts combine.03 / Spend output deliberately
Ask for the format and length the task needs, then measure the tokens actually returned. In this example output is two thirds of the starting bill, so shorter successful answers can have a meaningful effect.
A smaller token limit can cut an answer short. Compare completeness and accuracy on representative tasks before keeping a change.Further reading: prompt caching and batch processing. Provider rules and prices vary; the figures above are deliberately illustrative.
The project
This is an interactive cost modelling experiment for AI workloads. A model reads input tokens and writes output tokens, often at different prices. The calculator starts from a visible token profile, then shows how cache reuse, delayed batch processing and shorter answers change each part of a monthly bill. It does not connect to a billing account or optimize live requests.
Why I built it
Model cost can be difficult to reason about when several usage choices change at once. This tool makes each assumption adjustable so the effect of a single change is easier to see.
How it works
- Set monthly volume against the stated example profile: 1,000 input tokens and 500 output tokens per request.
- Change actual cached input, the share of requests that can wait for a batch, and the reduction in output tokens.
- Inspect the input/output breakdown and the arithmetic. Evaluate answer quality and latency before applying a change to real work.
What I learned
A lower estimate only matters when the output still does the job. The interface keeps that trade-off explicit instead of turning the result into a promise.
Built with
Selected technologies
- React
- TypeScript
- NumberFlow
- Cost modelling