A proxy that compresses long coding-agent chats so one session can run for weeks

Long coding sessions fill the model's context window, and providers charge per token. When the window is full, the session gets worse or stops. billion-context tries to keep one session going for days by compressing the parts of the conversation already used.

It works as a proxy. You start your agent with a `bili <client>` launcher, or point its base URL at the proxy. The proxy reads each request (Anthropic or OpenAI format), runs compression from the acp-kernel library, adds four context tools, forwards the request to the real model API and rewrites the streamed reply.

The README says it differs from a host's built-in summarizer in three ways. Compression is incremental, so summaries cover small ranges. It is reversible, so the model can restore the original text. And it keeps the prompt prefix stable, so the provider's prefix cache still works. The model decides when and what to compress. The README links a paper with a 4.5-month field study; those figures are the authors' own.

Read more on interesting repos · GitHub

If you use many AI APIs and cloud services, your credits and bills are scattered across a dozen consoles. Reckoner puts them on a single page so you can see what is left and what you have spent.

It has a FastAPI backend in Python and a React frontend. The backend stores your credentials in a local SQLite database and has one provider file per service. It polls each service in the background every 5 to 30 minutes, and the browser refreshes every 60 seconds.

Some providers work with a normal API key. Others, such as Anthropic, Mistral, Groq, CivitAI and Perplexity, need a session cookie or token copied from your browser's developer tools, so those setups may be less stable. The README marks some balances as best-effort. The author also ships a separate, closed-source iOS app; this repo is only the web version.

Read more on interesting repos · GitHub

AI coding agents waste context by re-reading unchanged files, dumping whole files to see a few lines, and running unbounded searches. They also forget what they worked out in earlier sessions. Token Optimizer targets all of these.

In Claude Code it installs as a plugin that denies large Read, Grep, Glob and similar calls and points the model to cached replacements. Re-reading a file returns only a diff. A local graph of files, symbols, tasks and findings fills in from normal tool traffic, and relevant notes are fed back when the agent touches a file.

The README says it optimizes the provider bill, which depends on prompt caching, rather than bytes removed. It ships benchmarks and admits one loss: on its codebase-exploration workload a competing design reduces slightly more (47.4% against 46.0%). A local dashboard attributes savings per client across 16 CLI clients, and telemetry is opt-in and off by default.

Read more on interesting repos · GitHub

Long prompts cost money. RAG apps, chat agents and coding agents keep sending large documents, logs and old turns to the model. leanctx sits between your code and the provider and trims that input. You change one import line and the rest of your code stays the same.

It classifies each segment of a request. Code, stack traces, JSON and tool-call fields pass through untouched. Prose such as documentation, retrieved passages, logs and earlier turns goes through LLMLingua-2, a small model that runs on your machine and drops less useful words. An optional mode asks a cheap LLM to summarize instead. After rebuilding the request it checks invariants, and if anything fails or times out it sends the original.

The README compares it with three alternatives. Provider prompt caching helps with stable prefixes but not changing content, so it suggests using both. Naive truncation cuts the middle of documents. Hosted compression APIs need your data sent to them, while leanctx runs locally. Savings depend on how much of your traffic is prose. On LongBench v2, layered on top of another compressor, it reports 18.7% fewer tokens for a 1.8 point accuracy drop. It reports about 37% savings on prose-heavy traffic and about 10% on code-heavy traffic.

Read more on interesting repos · GitHub

Coding agents get slow and expensive as a session grows. The built-in compaction in Claude Code and OpenCode replaces the whole conversation with one summary in a fixed template. Your messages, the agent's reasoning and its design decisions get flattened, and the agent has to rebuild its working state from that.

Magic Compact keeps the shape of the conversation instead. Your messages stay word for word, and tool calls stay in place. Each old assistant turn is replaced by its own short summary. Long tool inputs and outputs are swapped for an omission notice with an ID, and the agent can call a `read_omitted_content` tool to fetch the original when it needs it.

Compaction runs once, when you ask for it, so there is no background process and no prompt injection during the agent loop. The README contrasts this with the OpenCode plugins DCP and Magic Context, which manage context continuously and cost more tokens and cache invalidations. On Claude Code the plugin can't edit the current session, so it creates a new compacted session that you resume. The README says development is paused and points to the successor, Operator Memory, but the plugin still works.

Read more on interesting repos · GitHub

If you run AI coding agents a lot, you may have a work Claude plan, a personal one and a Codex subscription. Each has its own 5-hour window, weekly limit and per-model caps, all resetting at different times. Checking by hand usually happens after a run has already died.

ModelDeck has two parts. A SwiftUI menu bar app shows the meters. A small Node.js daemon, listening only on 127.0.0.1, does the reading. Codex usage comes through the official codex app-server protocol. Claude usage comes from Anthropic's usage endpoint, using the credential already stored in that account's profile.

Each account gets its own isolated profile home, so identities don't mix. The README says there is no cloud backend and no telemetry, and that ModelDeck never copies or stores your provider credentials. Update checks are off unless you enable them. The license is PolyForm Noncommercial, so commercial use needs the author's permission.

Read more on interesting repos · GitHub

AI agents often send every request to one expensive model. Switchyard lets you set up a pool of models, such as a cheaper one and a stronger one, and picks which should answer each request. Your gateway or application still makes the actual model call and owns credentials and retries.

You choose a routing algorithm. Some look at the request or the agent's recent tool activity. Others ask a model to judge whether the efficient model can handle the task. Routing can change as the agent keeps working. The default, Auto, is a fixed preset that tries the efficient model first with a confidence threshold of 0.5.

It keeps OpenAI and Anthropic API compatibility, so it can sit behind existing clients. The README says a cheaper call does not guarantee a cheaper finished task, so test the whole setup against a single-model baseline. It is pre-1.0 software and APIs may change.

Read more on interesting repos · GitHub

AI coding agents run shell commands constantly, and the raw output is often long: full git diffs, progress bars, hundreds of passing tests. All of that lands in the model's context and costs tokens. RTK sits between the agent and the command and passes along a shorter version.

It works through a hook. When the agent runs a Bash command like `git status`, the hook rewrites it to `rtk git status`. RTK runs the real command, then trims the result by filtering noise, grouping similar items, truncating, and collapsing repeated lines into counts. The agent never has to call `rtk` itself.

The README is careful about what the numbers mean. Up to 90% is the cut in bash output only, not in your bill, because bash output is just one part of the input and input is only part of the cost. Token counts are estimated as bytes divided by four, so the percentages are more reliable than the absolute numbers. It also says the filtering does not break Claude's prompt cache.

Read more on interesting repos · GitHub

Coding agents like Claude Code re-send a lot of text on every request: the system prompt, tool documentation and old history. An image costs tokens by its pixel size, not by how much text it holds, so dense text packed into a picture can cost far less.

pxpipe runs on your machine between the agent and the API. It renders the bulky, older parts of each request into PNG pages, adds a short factsheet of precision-critical tokens, and keeps recent turns as text. Only the request changes; the model's streamed output is untouched. A dashboard shows tokens saved and each conversion side by side.

The README is open about the trade-offs. Reading images is lossy: exact hex strings are sometimes silently misread, depending on the model. It pays off on token-dense content like code and JSON and loses on sparse prose, so a profitability gate decides what to image. It reports roughly 60 to 70 percent lower bills on Claude Code traffic, checked per request against a count_tokens baseline.

Read more on interesting repos · GitHub

Coding agents and LLM apps send a lot of bulky text to the model: search results, log dumps, JSON from tools. You pay for every token, and long context crowds out useful information. Headroom sits between your agent and the model provider and shrinks that text first.

It runs locally. A router detects the content type and picks a compressor: one for JSON, one for source code (using the syntax tree), and a small Hugging Face model for plain text. Originals are cached locally, so the model can ask for the full text through a retrieve tool when it needs it.

You can call compress() in Python or TypeScript, run a proxy that any OpenAI-compatible client can point at, or use 'headroom wrap' to launch a supported agent already routed through it. The README's benchmarks show 21% to 57% savings on four agent scenarios. It also says savings depend on how repetitive the data is, and that prose compresses very little.

Read more on interesting repos · GitHub

AI coding agents usually learn a codebase the slow way. They grep, glob and read files one at a time, then piece together who calls what. That takes many tool calls and uses a lot of tokens before the real work starts.

CodeGraph indexes your project ahead of time into a knowledge graph of symbols, call edges and dependencies. The graph lives in a local .codegraph/ folder and is served to the agent as an MCP server. The agent asks one question and gets back the relevant source, the call paths between symbols, and what a change could affect. A file watcher keeps the graph current as files change.

The README reports its own benchmarks on seven repos: 44% lower cost and 62% fewer tokens on average, with the biggest gains on questions that needed many tool calls. These are the author's numbers, so check them on your own code. Everything runs locally.

Read more on interesting repos · GitHub

Most AI labs offer a small free tier, a few million tokens a month or a few thousand requests a day. Each one alone is small, and juggling many SDKs and rate limits by hand is a chore. FreeLLMAPI puts them behind one OpenAI-compatible endpoint, so any client that speaks that API can point at your server.

You add your own keys for the providers you want, and they are stored encrypted. For each request, a router ranks the available models by speed, capability and reliability. If a provider returns a rate-limit or server error, it tries the next one. It also counts usage per key so it stays under each free-tier cap.

The README compares it with OpenRouter, LiteLLM and Portkey. Its focus is stacking free tiers, and it also speaks Anthropic, Gemini and Ollama formats so coding agents like Claude Code, Codex CLI and Aider can use it. The README says it is for personal experimentation only. A paid plan ($19/yr) gets new models in the catalog the same day, while free installs get them 30 days later.

Read more on interesting repos · GitHub

AI coding tools like Claude Code, Codex, Cursor and Cline hit rate limits, burn through paid quota, and tie you to one provider. 9Router is a local server you point those tools at instead. It exposes an OpenAI-compatible endpoint at localhost:20128/v1 and forwards each request to a provider you have connected.

Behind that endpoint it translates between API formats (OpenAI and Claude), tracks quota, and refreshes tokens. You use tiers: subscription providers first, then cheap ones such as GLM or MiniMax, then free ones such as Kiro or OpenCode Free. When one tier runs dry, requests move to the next.

It also has a feature called RTK that compresses tool_result content, such as git diff, grep or ls output, before sending it on. The README claims 20-40% token savings. That is the author's figure, so test it on your own work. Free tiers have credit caps, so expect limits, not truly unlimited use.

Read more on interesting repos · GitHub

AI coding agents fill their context windows fast. One browser snapshot, a list of GitHub issues or a log file can add tens of kilobytes of raw text. When the agent later compacts the conversation, it can also lose track of which files it was editing and what you asked for.

Context Mode adds an MCP server whose tools run code or fetch data in a sandbox and return only the printed result. The idea is to have the model write a short script that does the counting or filtering, instead of reading 50 files into context. Hooks nudge the agent to use these tools instead of raw Bash, Read or WebFetch.

For continuity, it logs file edits, git operations, tasks and errors in SQLite, indexes them with FTS5, and pulls back only the relevant parts with BM25 search after a compaction. It does not dictate how the model writes its answers, only where data goes. The README says it supports 17 platforms. The license is Elastic License 2.0, which is not an OSI open-source license.

Read more on interesting repos · GitHub

Makes coding agents answer tersely and shrinks the logs and output they read

AI coding agents are billed by the token. They write long, polite answers and read huge logs, test output and diffs. Caveman goes after both sides. The skill is a rule file that tells the agent to drop the filler and answer in short, caveman-style prose. Code, commands, file paths and error messages are left alone, and security warnings come back in full sentences.

The proxy runs on your machine between the agent and the AI provider. It compresses logs, JSON, test output and diffs before the model sees them, and keeps a backup of every original so the agent can fetch it back. A middleware package does the same inside your own app, wrapping LangChain, Vercel AI SDK, OpenAI or Anthropic calls.

The README is frank about the numbers. A JetBrains test of the skill alone on 86 coding tasks found about 8.5% fewer output tokens with no detectable quality change. The repo's own eval shows only about 3% on top of a plain 'answer concisely' instruction, and 35% at the ultracave level. The authors say most of an agent's bill is reading, which is why the proxy exists.

Read more on interesting repos · GitHub