Check it yourself #
Maple Harness is built to make a task cost fewer tokens without making the work worse. This page says what that means, how to check it on your own tasks, and what we hold ourselves to. It makes no promise about your bill: that depends on your tasks and your models.
In plain words #
- What it is. How to check, on your own tasks, whether a setting saves tokens without making the work worse.
- What to do. Watch
/costand the setup page's Cache tab, then compare runs withmaple-harness bench. Nothing here is a promise about your bill.
What we aim for #
Efficient. Maple Harness is built to spend fewer tokens for the same work: it keeps requests in a form your provider can reuse, keeps the conversation lean, and stops waste before it compounds.
Efficacious. It does not trade quality for tokens: it does not ask the model to talk in shorthand, what the model sees is cut only where it would be enormous (very large outputs are cut at a limit, with a note saying so), techniques that could change results are opt-in, and the compiler and your tests check the work.
Effective. It works on whole tasks, not single commands: it keeps the model you chose when it fits, notices when an agent is going in circles, and shows you what a task actually cost.
Don't take our word for it #
Please be diligent. A saving that only shows up in one request, or one run, is not a saving. Here is how to check.
- Compare whole runs, three against three. Pick a task you run often. Run it three times with a setting on and
three times with it off, for example trimming of old tool output, or letting the agent choose the model per message.
After each run compare
/costin the agent and the Cache card on the proxy's setup page. - Test your provider's cache on your own account.
maple-proxy cache probe --model <name> --yessends the same small request twice and shows what each call cost. See Check the cache yourself. - Check where the window and the spend are going with
/context, which includes helper agents. - Measure with a spending cap.
maple-harness benchreplays your own tasks under different settings, several times each, and compares them task by task with the uncertainty shown. It plans first, spends nothing until you confirm, and stops at the budget you set. See Measure with bench. - Look at net figures, not single requests. The cost of a whole run, including retries and rewrites of the cache, divided by the tasks that passed, is the number that matters.
- Repeat. Results vary from run to run and task to task, even on the same model. One run proves nothing.
- If a technique does not help on your work, turn it off. Every one of them has a setting.
More habits and what the numbers mean are in See and reduce what a task costs.
What we hold ourselves to #
- We publish results only from repeated, paired runs on real tasks, including the runs where a setting did not help, with the uncertainty shown.
- A technique is on by default only if quality is not worse.
- Nothing here is a promise about your bill.