RustySqueezer
A fake LLM for context compaction. It exposes OpenAI, Anthropic and Ollama compatible endpoints that answer a compaction request with a deterministic, extractive summary of the thread - without calling any language model.
Quick start Integrations Why it exists
Quick start
$ docker run -d -p 8000:8000 ghcr.io/handtrixx/rustysqueezer:latest
$ curl -s localhost:8000/health
{"status":"ok","model":"rustysqueezer"}
$ curl -s localhost:8000/v1/chat/completions \
-H 'content-type: application/json' \
-d @thread.json
# 189k tokens in -> 1.3k tokens out, deterministic, ~4 ms
No model calls
Pure extraction. Deterministic, byte-identical output for the same input, which keeps prefix caches warm.
Drop-in surfaces
OpenAI /v1/chat/completions, Anthropic /v1/messages (including native compaction blocks), Ollama /api/chat and /api/generate.
Tiny
~4 MB RSS, 64 MiB request bodies, a million-token thread compacts in about a tenth of a second.
Private by default
No auth, no outbound network, runs on a private Docker network behind your gateway.
Why this exists
Agents compact a long thread by sending the whole thread to a summarising model. That request has to fit inside the summariser's own context window, so a local model with a modest window (e.g. 131k tokens) cannot compact its own threads - you end up paying a 1M-context cloud model for every compaction event.
The insight: compaction does not need a model. It needs a good extractive summary of what was asked, what was run, what broke, and what was just said.
rustysqueezer does exactly that, in a few milliseconds, for free, and always
produces byte-identical output for the same input.
How it works
flowchart LR
A[Agent thread] -->|POST /v1/chat/completions| B[RustySqueezer]
B --> C[Structured summary, 5-10% of the thread]
C --> D[Compaction model slot in the agent]
- Detect the compaction request. The last user message is matched against
instruction_patterns(Zed, Claude Code-style/compact, generic "summarize the conversation so far") and treated as the instruction, not thread content. - Split the thread into the original task, follow-up user messages and assistant turns.
- Extract facts from tool calls (
tool_callsand tool results): commands that ran, files and paths touched, tool names. - Pick key assistant sentences scored by task-term overlap plus recency - a TF-IDF-ish heuristic, no model.
- Collect error lines using a configurable regex.
- Append the verbatim tail - the last
TAIL_TURNSturns, clipped toTAIL_CHARS.
This is a proof of concept, not a product
There is no authentication, no rate limiting and no abstractive clustering. The compaction itself is solid and evaluated, but treat it as infrastructure you run on a private network.
Deploy
CI publishes :latest, :<sha>, :<version> and :v<version> tags on every
push to main, and the image is public - no login needed.