ranxianglei /
billion-context
A proxy that compresses long coding-agent chats so one session can run for weeks
- 2.4k stars
- TypeScript
- Updated today
- Some experience
- Token & cost savings
Long coding sessions fill the model's context window, and providers charge per token. When the window is full, the session gets worse or stops. billion-context tries to keep one session going for days by compressing the parts of the conversation already used.
It works as a proxy. You start your agent with a bili <client> launcher, or point its base URL at the proxy. The proxy reads each request (Anthropic or OpenAI format), runs compression from the acp-kernel library, adds four context tools, forwards the request to the real model API and rewrites the streamed reply.
The README says it differs from a host's built-in summarizer in three ways. Compression is incremental, so summaries cover small ranges. It is reversible, so the model can restore the original text. And it keeps the prompt prefix stable, so the provider's prefix cache still works. The model decides when and what to compress. The README links a paper with a 4.5-month field study; those figures are the authors' own.
What it does
Model-driven compression
The proxy adds a
compresstool. The model calls it to fold a range of messages into a detailed summary, and the proxy runs it server-side.Reversible summaries
decompressrestores a compressed range when exact details are needed again.search_contextsearches both summaries and visible messages.Context usage overview
acp_statusshows how much context is used and which ranges can still be compressed.Many supported clients
A table covers Claude Code, Codex, OpenCode, pi, Gemini CLI, Aider, Goose, Zed and others. Each has a launcher, a native plugin, or a
/bili/URL prefix, depending on what the client allows.Cache diagnostics
The
/acpand/acp-cachecommands help find out why the prefix-cache hit rate is low.
What you could build
- Run a single Claude Code or Codex session for days without it hitting the context limit.
- Cut token spend on long agent sessions by compressing history already used.
- Work with a model that has a small context window, around 100K tokens.
- Search and restore older parts of a long session when exact details are needed.
- Read the linked paper to study how multi-generation context compression works.
Get started
Install the package with npm, then find your client in the README's table and start it through the matching launcher, for example bili claude or bili codex. Some clients use bili plugin install <client> instead.
npm install -g billion-context --prefix=~/.localCommands are copied from the project's README. Read it on GitHub for the full setup.
More in Token & cost savings
caveman
Makes coding agents answer tersely and shrinks the logs and output they read
rtk
A command-line proxy that shrinks shell output before your AI coding agent reads it
headroom
Shrinks tool outputs, logs, and files before your AI agent sends them to the model
codegraph
A local code knowledge graph that lets AI coding agents look up code in one call
freellmapi
One OpenAI-style API that routes across dozens of free LLM provider tiers
context-mode
Keeps bulky tool output out of your coding agent's context window
9router
Local proxy that routes AI coding tools to subscription, cheap, or free models
pxpipe
Local proxy that renders Claude Code's bulky context as images to cut input tokens
Switchyard
Routing library that picks which LLM handles each agent request
token-optimizer-mcp
MCP server and Claude Code plugin that cuts agent token use and keeps a project graph