llmtrim

llmtrim is a local proxy that compresses your LLM API requests so you pay less, with no change to the answers.
It sits between your AI tools and the provider, strips the wasted tokens out of every request, and forwards it on. You get the same answers for a smaller bill.

_{−31% input and −74% output tokens, measured live across 112 A/B cases, with no change in answer quality.}

_{Use it as a proxy, a CLI, an MCP server, or a library (Python · Ruby · Swift · Kotlin · JS · TS · WASM).}

What it does • See it in action • Get started • CLI, MCP & library • Works with • Configuration • Numbers

You run Claude Code, Codex, Cursor, or your own app. Every time it talks to an LLM, it sends a big blob of text: your system prompt, the tool definitions, the whole conversation history, and the raw output of every command it ran. You pay for every one of those tokens, on every single turn.

A lot of that text is waste. A 200-line build log where only 2 lines are errors. A tool schema resent identically 50 times. A JSON array with 500 near-identical rows. The model doesn't need the bulk of it to answer well, but you're billed for all of it.

llmtrim removes the waste before it's sent. It installs as a local proxy that sits between your tool and the LLM provider. Requests pass through it, get compressed, and continue to the provider. The reply comes back unchanged. Your tool doesn't know it's there; you just get a smaller bill.

  before:  your tool ───── full request ─────▶  OpenAI / Anthropic / …
                    ◀──────── reply ──────────

  after:   your tool ──▶ llmtrim ──smaller──▶  OpenAI / Anthropic / …
                            (on your machine)
                    ◀──────── reply ──────────  (same answer)

Everything runs locally. Nothing is ever sent to us.

Here's one real thing llmtrim does, end to end. An AI agent ran a build, and the bash tool returned a 58-line log. Only two lines matter (the errors), but all 58 get sent to the model and billed.

Before, what the model would receive (58 lines, 4,662 chars):

[2026-06-13T10:02:00Z] INFO  compiling module core::worker::task_0 (incremental)
[2026-06-13T10:02:01Z] INFO  compiling module core::worker::task_1 (incremental)
[2026-06-13T10:02:02Z] INFO  compiling module core::worker::task_2 (incremental)
... 27 more near-identical INFO lines ...
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
... 25 more INFO lines ...
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

After, what llmtrim sends instead (5 lines, 978 chars, −79%):

[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)]
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24]
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

Both errors and the summary survive verbatim. The repetitive INFO lines fold into a template plus their values, losslessly, because the range is regular (task_0..task_29). The model still sees exactly what happened; it just costs a fifth as much.

If that's useful to you, a ⭐ helps other people find it.

Try it yourself on any request body:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai

Log-folding is just one of ten compressors. A different one re-encodes bulky JSON arrays into a compact table, with the same data in a third of the tokens:

before:  [{"id":1,"city":"Paris","ok":true},{"id":2,"city":"Lyon","ok":false}, … 200 rows]
after:   [200]{id,city,ok}: 1,Paris,true; 2,Lyon,false; …          (TOON encoding, lossless)

Each compressor fires only where it pays:

Stages run in savings order. Nothing under a cache_control marker is ever rewritten.

Default auto switches each stage on only where it pays. safe runs the lossless stages only. Full config →

# 1. Install (any OS, prebuilt binary, no Rust needed)
npm install -g @llmtrim/cli@latest && llmtrim setup

# 2. Open a new shell. Your AI tools now route through llmtrim automatically.

# 3. Watch the savings add up as you work
llmtrim status --watch

No Node? Use an installer instead:

# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex

Or your own package manager, same binary everywhere: brew install fkiene/tap/llmtrim · cargo binstall llmtrim · scoop install llmtrim · docker run ghcr.io/fkiene/llmtrim. Full options in INSTALL.md.

setup is a local HTTPS proxy, the same technique as mitmproxy, scoped to LLM APIs. It changes exactly three things (a CA certificate in ~/.llmtrim/, a proxy setting in your shell profile, a login service), and llmtrim uninstall reverses all three. No API keys are stored (it forwards your tool's own auth), and your prompts never touch disk; only an anonymous count of tokens saved is kept. Full threat model: SECURITY.md.

1. A private certificate in ~/.llmtrim/, cryptographically constrained to LLM API domains. It cannot read your bank, email, or any other traffic, even if the key were stolen. Check that yourself:

   llmtrim ca   # prints the cert path
   openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"
   # those domains are the only ones it can ever touch

2. A proxy setting in your shell profile (HTTPS_PROXY + NODE_EXTRA_CA_CERTS).
3. A background service that starts at login.

If the service stops, your tools fail fast with a connection error rather than silently bypassing it.

llmtrim status      # health + savings dashboard (aliases: monitor, gain)
llmtrim doctor      # something off? end-to-end diagnosis; each check names its fix
llmtrim start       # start the background proxy
llmtrim stop        # stop it
llmtrim serve       # run in the foreground instead (Ctrl-C to quit)
llmtrim wrap claude # run an agent, guaranteeing this session routes through llmtrim (fails fast if it can't)
llmtrim update      # update to the latest release + restart
llmtrim uninstall   # exact inverse of setup: removes all three changes

llmtrim status --daily (or --weekly / --monthly) gives a time-series report; add --json or --csv to export.

The same compression runs with no proxy and no setup, as a one-shot CLI, an embeddable Rust crate, native bindings for Python, Ruby, Swift and Kotlin, or a WebAssembly module for JavaScript (browser, Node, Cloudflare Worker). No extra model calls, no network: the deterministic engine runs in your process.

CLI. Pipe a request in, get a compressed one out:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai > out.json
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim send     --provider openai   # compress, call, print

Rust. The engine is the llmtrim-core crate (no tokio, no network in its dependency tree):

use llmtrim_core::{compress, compress_with_config, config::DenseConfig, ir::ProviderKind};

// None auto-detects the provider from the request shape.
let out = compress(request_json, Some(ProviderKind::OpenAi))?;
println!("{} -> {} input tokens", out.input_tokens_before, out.input_tokens_after);

// …or pass an explicit preset/config:
let out = compress_with_config(request_json, Some(ProviderKind::OpenAi), &DenseConfig::preset("agent").unwrap())?;

Python / Ruby / Swift / Kotlin. One flat compress(input, provider, preset) call, generated from the same Rust engine via UniFFI. The compiled engine is bundled in each package, so there's no Rust toolchain to install:

import llmtrim

out = llmtrim.compress(request_json, llmtrim.Provider.OPEN_AI, "aggressive")
print(out.input_tokens_before, "->", out.input_tokens_after)

JavaScript / TypeScript (WebAssembly). The same compress(input, provider, preset) call, compiled to WebAssembly, runs in the browser, Node, Bun, Deno, or a Cloudflare Worker with no network or filesystem access. The output type is fully typed in TypeScript:

import { compress } from "@llmtrim/js"; // @llmtrim/wasm is an alias for the same package

const out = compress(requestJson, "openai", "aggressive");
console.log(out.input_tokens_before, "->", out.input_tokens_after);

MCP server. llmtrim mcp speaks the Model Context Protocol over stdin/stdout, so any MCP client can compress payloads and read your savings without the proxy. It exposes three tools: llmtrim_compress (compress a full request body, honoring your ~/.llmtrim config like the proxy), llmtrim_compress_text (shrink one text blob, lossless), and llmtrim_stats (your savings ledger). Every call records to the same ledger, so MCP traffic shows up in llmtrim status.

llmtrim mcp install          # register with Claude Code (one command)
llmtrim mcp install --print  # or print the config to paste into any other client

The printed block is the standard MCP config; for a client you edit by hand it looks like:

{
  "mcpServers": {
    "llmtrim": { "command": "llmtrim", "args": ["mcp"] }
  }
}

Any tool that honors HTTPS_PROXY and an env-provided CA, which is essentially every CLI agent and most Node apps:

Prefer no proxy? Any MCP client (Claude Code, Cursor, custom agents) can call llmtrim directly as tools instead: run llmtrim mcp install, or see CLI, MCP, or library.

Providers come from the llm_providers registry (OpenAI, Anthropic, Google, DeepSeek, Mistral, xAI, Moonshot, Zhipu, Qwen, OpenRouter, …) and update with it. Every non-LLM connection passes through untouched.

Zero config needed. The default (auto) inspects each request and picks the right compressors for its shape: tool-heavy → agent, code → code, long context with a question → rag, otherwise → aggressive.

If you want to force a mode, set one line: LLMTRIM_PRESET=<name> or preset = "<name>" in $XDG_CONFIG_HOME/llmtrim/config.toml:

Every stage is individually tunable via config flags; preset wins over individual flags. The full table is long; see the field list in src/config.rs or run llmtrim compress --help. The most useful knobs:

Env: LLMTRIM_PRESET (preset), LLMTRIM_CONFIG (config-file path), LLMTRIM_DB_PATH (ledger location), LLMTRIM_UPSTREAM_PROXY (route egress through another proxy, see below).

Set LLMTRIM_UPSTREAM_PROXY=http://host:port to make llmtrim send its outbound calls through another proxy instead of dialing the API directly. This covers two cases: a corporate or auth proxy that all egress must pass through, and running llmtrim alongside a second local tool such as headroom on a different port.

The connection to the API is tunneled through the upstream with CONNECT and stays under verifying TLS, so the upstream sees only the encrypted stream. An upstream that points at llmtrim's own listen address (any spelling of localhost on the same port) is rejected, because it would loop traffic back into the daemon; a companion proxy on a different port is allowed. http://user:pass@host:port works for proxy auth, and those credentials are redacted from logs.

Put the variable in the daemon's own launch environment (the launchd plist or systemd unit), not only your shell profile. Credentials written into a shell profile are stored there in plaintext.

Every case is sent twice, once original and once compressed, then both answers are scored and billed at real rates. Cost and quality are measured together, not estimated, across 112 cases:

The token cuts are model-independent (−31% input, −74% output). The dollar saving tracks the model's output-to-input price ratio: −66% here, projecting to −57% at GPT-4o rates and −59% at Claude Sonnet rates. The proxy compresses only the new-content surface and never rewrites the cache-controlled prefix, so your prompt-cache discount survives.

The same A/B on the standard academic suites, at a conservative shape-matched preset (qwen3-next-80b, paired 95% CI). Quality is the score on the original request vs the compressed one. GSM8K comes from the frontier above (n=12); the other three are the named benchmarks readers compare against (n=20 each):

Three rows compress with no quality loss; GSM8K is the one dip:

• BFCL drops the tool schemas the query doesn't need (a menu of 2 to 37 candidates per call).
• SQuAD v2 still answers its unanswerable questions correctly.
• TruthfulQA holds factual accuracy exactly: its ~75-token prompts are almost all answer text, so the safe preset finds nothing to cut.
• GSM8K trades −8pp of accuracy for −71% cost, so measure per workload before enabling its reasoning preset. ¹Its input goes negative because that preset injects a Chain-of-Draft instruction whose payoff is output-side (see the frontier table).

Evidence and a one-line reproduce (named-benchmark snapshot):

python3 crates/llmtrim-cli/bench/scripts/download.py 40 truthfulqa,squad2,bfcl
(cd crates/llmtrim-cli && cargo run -q --features live -- bench quality \
   --corpus bench/data/squad2.jsonl --preset rag \
   --model qwen/qwen3-next-80b-a3b-instruct --route "" --n 20)

Methodology, per-corpus frontier, and confidence intervals: crates/llmtrim-cli/bench/README.md. Reproduce it:

python3 crates/llmtrim-cli/bench/scripts/download.py 40   # pull real corpora (gsm8k, humaneval, dolly, hotpotqa, …)
(cd crates/llmtrim-cli && cargo run -q --features live -- bench suite)  # live A/B across all corpora (needs OPENROUTER_API_KEY)
python3 crates/llmtrim-cli/bench/scripts/chart.py         # regenerate the chart + table

Three neighbors each compress one layer of the problem; llmtrim does the whole round-trip and is quality-gated so it can't increase your bill.

Headroom is the closest comparison, so we measured it head to head at matched aggressiveness on the same o200k_base tokenizer. Headroom compresses harder. It removed more tokens than llmtrim at every matched setting we tried, for example 74% vs 58% on aggressive tool output.

The two tools optimize for different things. Headroom maximizes raw input compression. llmtrim is quality-gated: it reverts any cut that drops answer-relevant content, and it also compresses output and leaves the cached prefix alone, which Headroom doesn't do. In our small answer-survival runs llmtrim held the answer better, but those cases lean on llmtrim's own tool-output corpus, so we don't lead with a quality number. The per-group tables, the caveats, and the repro are in the artifact; reproduce with bench/scripts/vs_headroom.py.

They also stack: RTK shrinks the CLI output, then llmtrim compresses the tool schemas Claude Code resends on top of it. Detailed head-to-heads with RTK, Headroom, and caveman are in crates/llmtrim-cli/bench/README.md.

These are surfaced by the same A/B that proves the savings:

• Anthropic / Gemini token counts are approximate. There's no public exact tokenizer, so a BPE proxy is used and flagged in status. OpenAI is exact.
• Output savings aren't measured live. The proxy compresses input; an output saving needs the A/B counterfactual, which only the offline benchmark runs. status "saved" is input-side.
• The default is quality-gated, not lossless. Lossy stages run only where the eval shows quality holds. Want a byte-faithful round-trip? Use the safe preset.
• "Lossless" is input-side, not response restoration. A lossless stage preserves the information the model reads (a folded log run, a TOON-encoded array, an abbreviation legend the model decodes in-prompt), and the token gate reverts any input cut that doesn't pay off. The engine does not transform the model's response back to an original form.

Every compressor is a deterministic implementation of published research: the ideas are theirs, the engineering and the token gate are ours.

Retrieval & context: BM25 (Robertson & Zaragoza 2009, bm25); BM25+ (Lv & Zhai, CIKM 2011); RM3 (Lavrenko & Croft, SIGIR 2001); TextTiling (Hearst, CL 1997); TextRank (Mihalcea & Tarau, EMNLP 2004); MMR (Carbonell & Goldstein, SIGIR 1998); Submodular objective (Lin & Bilmes, ACL 2011); modified-greedy knapsack maximizer (Tang et al., SIGMETRICS 2021, arXiv:2008.05391); DPP diverse sampling (Chen et al., NeurIPS 2018); Lost in the Middle (arXiv:2307.03172); DSLR (arXiv:2407.03627).

Code: RepoCoder (arXiv:2303.12570); Hierarchical Context Pruning (arXiv:2406.18294); The Hidden Cost of Readability (arXiv:2508.13666); Minification token accounting (arXiv:2606.01326).

Tool output: Drain (He et al., ICWS 2017); Brain (Yu et al., IEEE TSC 2023); LogLSHD (arXiv:2504.02172).

Dedup & abbreviation: SimHash (Charikar, STOC 2002, gaoya); CompactPrompt (arXiv:2510.18043); Maximal repeats (arXiv:1304.0528) + Re-Pair (Larsson & Moffat, DCC 1999).

Output control: Chain-of-Draft (arXiv:2502.18600); TALE (arXiv:2412.18547).

Serialization: TOON (Token-Oriented Object Notation), Johann Schopplich.

Built on tiktoken-rs, tree-sitter, image, whatlang, hudsucker, rusqlite, and more.

Run llmtrim doctor for an end-to-end diagnosis; each failing check names its fix. Found a request it mangled? Set LLMTRIM_CAPTURE_DIR and open an issue with the before/after capture, since a repro is a fix. And if it saved you money, a ⭐ helps others find it.

_{Licensed under MPL-2.0. Use llmtrim freely in your stack, including commercially, with no source-disclosure obligation for your own code; the file-level copyleft applies only to modifications you make to llmtrim's own source files. Contributions via DCO sign-off.}