Cutting LLM inference cost by 70% in a six-step agentic pipeline

September 2026

We ran a supplier-discovery pipeline at Unibloom: a user asks a question, and a chain of Next.js server actions turns it into search queries, fetches and scrapes candidate pages with Firecrawl and Exa, extracts structured facts with an LLM, cross-checks them, and writes an answer. Five to six steps, each with its own system prompt, output schema, tool-call limit and set of callable functions.

It worked. It was also far more expensive than it had any right to be, and the reason turned out to be boring: we were paying to send the same text to the model over and over.

Input tokens are the bill

The intuition that gets people into trouble is thinking of an agentic pipeline as a conversation. It isn't. Each step is a stateless HTTP call, and you are billed for every input token on every call. The model doesn't remember step 1, you re-send it.

So if each step naively carries forward everything that came before, step n pays for the output of steps 1 through n−1, plus every system prompt along the way. The total isn't proportional to the number of steps. It's proportional to the square of it.

Take a pipeline where each step adds roughly 2,000 tokens of context and carries everything forward:

Twelve thousand tokens of actual content, billed as forty-two thousand. The extra thirty thousand buys you nothing, it is the same text, re-sent. Add a step and you don't add 2,000 tokens, you add 14,000.

This is invisible in development. You test with two steps and a short document, the bill is cents, and the shape of the curve never shows itself. It shows itself in production, on real documents, at real volume.

Fix one: stop resending system prompts

Each stage of our pipeline had its own system prompt, the scraper's instructions, the extractor's schema rules, the cross-checker's adjudication policy. Reasonable. The mistake was appending each new one to the accumulated message list rather than replacing it.

By the last step, the model was receiving five sets of instructions, four of which described jobs that were already finished. That is worse than waste. Stale instructions compete with the current ones, and a model told five different things about output format will sometimes pick the wrong one.

The fix is to treat the system prompt as belonging to the step, not to the chain:

// before: instructions accumulate
messages = [...messages, { role: 'system', content: stepPrompt }, userTurn]

// after: each step gets exactly its own instructions
messages = [{ role: 'system', content: stepPrompt }, ...carriedContext, userTurn]

One line of difference. On a six-step chain it removes five redundant prompt copies from the final call, and four from the one before it.

Fix two: pass forward the answer, not the transcript

The larger win. Every step produced structured output, and we were carrying all of it into the next step on the theory that context is good and the model can ignore what it doesn't need.

You are paying for that ignoring. If step 3 scraped eight pages and produced 6,000 tokens of extracted content, and step 4 only needs the three supplier names, then 5,800 of those tokens are being bought purely so the model can skip past them.

So we went through the chain and asked one question at each boundary: what does the next step actually read? Almost everywhere the answer was a small structured subset: a few identifiers, a confidence score, a short list, not the prose the previous step had generated on its way there.

Because every stage already had an output schema, the trimming had a natural place to live. The schema stopped being only a validation boundary and became a budget: whatever isn't in it doesn't travel.

Intermediate output is scaffolding. It exists to get the next step built, and then it should come down.

What the two changes did together

Roughly 70% off inference cost, with no change to output quality. The pipeline still does the same six steps, still reads the same pages, still returns the same answers.

The split between the two fixes is worth naming: prompt hygiene is the tidier problem and the smaller win, because system prompts are short. Trimming intermediate outputs is where the money was, because scraped page content is long and it was being re-sent four or five times over.

A small side effect we didn't plan for: latency improved too. Fewer input tokens is less to transfer and less to prefill, and on the longest chains that was noticeable.

What I'd tell someone starting this

• Log input tokens per step, not per request. A single aggregate number hides the quadratic shape completely. Per-step, the growth is obvious on the first chart you draw.
• Suspect the longest step boundary first. Cost concentrates wherever a step produces bulk text: scraped pages, retrieved documents, tool output.
• Treat the output schema as the contract for what travels. If it isn't in the schema, the next step shouldn't see it. This is the discipline that keeps the fix from rotting as the pipeline grows.
• Re-check after every new step. The whole problem is that adding a step costs more than the step itself. That stays true after you fix it, the constant is just smaller.

None of this is clever. It's the LLM equivalent of noticing you're doing a database query inside a loop. But the pricing model makes it expensive in a way that ordinary redundant work isn't, and nothing in the developer experience points at it. The code that carries everything forward is the code that reads most naturally.

← All articles