← All lab notesWORKSHOP · 2026-05-11

A small experiment in prompt-level cost control

Short write-up from an internal session — read in under five minutes.

↳ 12 min read

This is an internal session write-up, not a formal benchmark — the kind of thing we publish as we go, before it’s polished into a full report.

What we tried

A few of us spent an afternoon looking at where token spend actually goes in a long-running agent session inside Lizardata — the kind that plans a task, calls a few tools, and iterates. We tried three changes, one at a time, against the same fixed task set: trimming the system prompt down to only the instructions that specific workflow needed, caching the parts of the conversation that don’t change between steps, and routing the planning step to a smaller model while keeping the final answer on the larger one.

What we saw

Trimming the system prompt helped the most, and cost nothing to do — most of what we’d written in there was never relevant to the specific task at hand. Caching the static parts of the conversation helped on longer sessions but made almost no difference on short ones. Routing the planning step to a smaller model was the riskiest change: it saved tokens, but on a couple of tasks the smaller model planned a worse first step and the agent needed an extra turn to correct it, which ate back some of the savings.

What we’d do differently

We’d measure the planning-model swap over more tasks before trusting it — one afternoon of testing isn’t enough to call it a net win. The prompt trimming is the one change we’re confident about, and it’s already gone into how we scope new Lizardata pipelines.

More from the lab