# leanctx

> Python wrapper that shrinks LLM prompts locally, leaving code and tool calls exact

- Repository: https://github.com/jia-gao/leanctx
- Category: Token & cost savings
- Stars: 322 · Forks: 7 · Language: Python · License: MIT
- Experience: Some experience
- Page: https://ir.adyog.com/repos/jia-gao/leanctx/

Long prompts cost money. RAG apps, chat agents and coding agents keep sending large documents, logs and old turns to the model. leanctx sits between your code and the provider and trims that input. You change one import line and the rest of your code stays the same.

It classifies each segment of a request. Code, stack traces, JSON and tool-call fields pass through untouched. Prose such as documentation, retrieved passages, logs and earlier turns goes through LLMLingua-2, a small model that runs on your machine and drops less useful words. An optional mode asks a cheap LLM to summarize instead. After rebuilding the request it checks invariants, and if anything fails or times out it sends the original.

The README compares it with three alternatives. Provider prompt caching helps with stable prefixes but not changing content, so it suggests using both. Naive truncation cuts the middle of documents. Hosted compression APIs need your data sent to them, while leanctx runs locally. Savings depend on how much of your traffic is prose. On LongBench v2, layered on top of another compressor, it reports 18.7% fewer tokens for a 1.8 point accuracy drop. It reports about 37% savings on prose-heavy traffic and about 10% on code-heavy traffic.

## What it does

- **Drop-in client wrappers**: Import OpenAI, Anthropic or Gemini from leanctx instead of the original package. Responses include usage fields for tokens saved and compression ratio.
- **Loss-tolerance routing**: Code, errors, JSON and tool-call fields stay verbatim. Documentation, retrieved text, logs and prior turns are compressed with LLMLingua-2.
- **Fail-open behavior**: If compression fails, times out or breaks an invariant check, the original uncompressed request goes to your provider.
- **Local by default**: The compression model runs on your own machine. The first call downloads about 1.2 GB of weights, and the only outbound calls go to your existing provider.
- **Reproducible benchmarks**: A CLI runs bundled scenarios with no API key, including one that checks that code and tool-call fields stay unchanged. The LongBench v2 per-item records are committed to the repo.

## What you could build

- Cut input-token costs in a RAG app that sends large retrieved documents.
- Shrink old chat turns and log dumps in a long-running LangChain or LangGraph agent.
- Compress tool-output history in a coding agent without altering code or tool calls.
- Estimate how much a prose-heavy document pipeline would save before committing to it.
- Add a check to CI that confirms compression never changes code or tool-call fields.

## Who it's for

Developers running LLM apps in production with the OpenAI, Anthropic or Gemini Python SDKs, who can accept a small accuracy trade-off and a local model download.

## Get started

Install with the extra for your provider plus lingua, then change your import to come from leanctx and pass a leanctx_config. The first call downloads the model weights; the bench commands let you check the install without an API key.

```
pip install 'leanctx[openai,lingua]'
```
```
leanctx bench list
```
```
leanctx bench run agent-structural --workload agent
```

Tags: prompt-compression, llmlingua, cost-optimization, rag, python