LLMs that rewrite their own context, and how to try them

Give an agent a long task and its context will eventually approach the limit. Most tools today handle this in one of three ways: summarize the history at a set length, offload it to files, or search and pull it back when needed. In every case, a person decides the rules for what to keep and what to drop. The paper “Context Language Models”, posted to arXiv on September 29, 2026, leaves that decision to the model. The authors are researchers at Meta Superintelligence Labs, the University of Washington, MIT and Trillium Labs. The code and extensions for trying it in your own agent were released at the same time, so you can do more than read the paper: you can run it in your own environment.

Letting the model edit its context as a file#

In an ordinary language model, the context only grows. New messages and tool results are appended to the end, and old ones stay until summarization or truncation removes them.

In a CLM (Context Language Model), the model produces its next context itself. The implementation is simple: everything in the conversation after the system prompt and task instructions is written out to a single text file, and the system prompt tells the model the file’s path. The model can rewrite the file with ordinary commands such as sed or python3. Once the harness validates the format and accepts an edit, the edited content is sent to the LLM server as the input to the next request. If the model doesn’t rewrite anything, new content is appended to the end as before.

According to the README of the paper’s harness, the file consists of one block per message, separated by [[CTX_TURN i role=…]]. The system prompt and task instructions are fixed and are not in the file.

Earlier work had gradually widened what the model controls, for example by letting it decide when to summarize or by giving it tools to offload and retrieve. Even so, the model could only use operations a person had prepared. CLM doesn’t define the operations at all; where and how to rewrite is entirely up to the model. The paper shows rewrites like these, which the model came up with on its own during the experiments:

• Creating a block with a new role, “notes,” to record what it had learned
• Using a loop to replace irrelevant search results with a single line: “searched, not relevant”
• Defining a function that shrinks old results into references to a progress note, and reusing it 37 times
• Rewriting a block that tracks the status of several subagents as a table, in place, 163 times, keeping the context at 6–8K tokens

Less compute, with equal or better accuracy#

The paper’s main numbers are below. Compute is compared in theoretical FLOPs, assuming the cache for a matching prefix is reused. Rewriting the middle of the context means everything after that point has to be recomputed, and that cost is included.

Task	Setting	Result
BrowseComp-Plus (research)	Qwen3.6-27B, 32K tokens	59.4% accuracy. 11.4% higher in relative terms than the strongest baseline (Codex-style summarization), with 21.5% less compute
TerminalBench 2.1 (coding in a terminal)	Same	Same accuracy as summarization, at 70% of the compute
EdgeBench-10 (improving a repository for 12 hours)	Qwen3.6-27B, 32K tokens	44.6 points (summarization: 42.3). Compute of 179 PFLOPs (summarization: 437)
Software World (six agents spend 24 hours speeding up six repositories)	GPT-5.6-Sol, 272K tokens	At the same cost, 65% more speedup than summarization

These results came with no additional training: the existing models were simply told where the file is and how to use it. The paper also notes that smaller models benefit less. On TerminalBench 2.1, Qwen3.5-9B edited its context 1.4 times per task on average, and never edited it at all in half the tasks (Qwen3.6-27B: 2.6 times). So the authors trained Qwen3.5-9B with reinforcement learning, rewarding successful runs that used less compute more highly. Accuracy on BrowseComp-Plus rose from 28.8% to 42.5%.

The authors also tested how well instructions work. They tried three one-sentence instructions:

• When the context exceeds a certain size, shrink it to about 4,000 tokens
• After finishing each subtask, drop the material for it
• Always make a backup before editing

Each was simply added to the task instructions, with no change to the harness or the model, and the model’s editing changed accordingly.

That said, every number comes from the authors’ own experiments. AI Weekly points out that no third party has reproduced them yet. Among the reactions on X that InfoQ covered, some say these are only results on predefined tasks.

How to try it yourself#

Which route to take depends on whether you want to measure under the paper’s conditions or use it in your everyday coding agent.

Environment	What to install	Best for	License
The paper’s harness (Harbor)	context-language-models	Measuring under the paper’s conditions, benchmarks	CC BY-NC 4.0 (non-commercial)
Pi	pi-clm (the extension the paper’s repository points to)	Trying it in everyday work	MIT
OpenCode	opencode-clm (a community port, not maintained by the authors)	Trying it in everyday work	MIT
Claude Code	None (borrow the idea through instructions)	Changing the summarization policy	None

The paper’s harness (Harbor)#

The paper’s experiments run on Harbor, a framework for evaluating agents. Install it with Python 3.12:

git clone https://github.com/facebookresearch/context-language-models.git && cd context-language-models
python3.12 -m venv .venv && . .venv/bin/activate
pip install -e .

Any OpenAI-compatible server with tool calling enabled works as the model. The README includes an example of serving Qwen3.6-27B, the model used in the paper, with vLLM. Set the server’s context length to at least the budget (context_budget_tokens) plus the per-response output limit (max_tokens, default 16384).

vllm serve Qwen/Qwen3.6-27B --served-model-name qwen36-27b --max-model-len 65536 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3

Without a GPU, you can use a model behind an API instead. In that case, pass a litellm model name (such as anthropic/...) and leave out api_base.

Point it at a Harbor task directory to run one trial. -e docker runs the task in a Docker container.

clm-harbor trial start -p path/to/task -e docker -a clm-minimal -m openai/qwen36-27b \
  --agent-kwarg api_base=http://localhost:8000/v1 --clm-config bcp

--clm-config passes a bundle of the settings used in the paper: bcp (for research) or edgebench (for long-running improvement). Without it, the budget is 32,000 tokens. The main settings are:

• context_budget_tokens: the context budget. The actual limit is this value minus context_budget_reserve_tokens (default 2048)
• nudge_ratios: at what fractions of the budget to tell the model to shrink. Default 25%, 50% and 75%
• max_num_retry_on_limit: how many times, when the limit is exceeded, to undo the latest turn and have the model shrink the context. Default 50
• allow_edit_growth: when edits are accepted. fit accepts an edit if it fits within the limit; shrink accepts it only if it makes the context smaller

Each trial leaves a trajectory (trajectory.json), counts of edits and warnings (usage.json), and a copy of the context at every step (context_snapshots/), so you can trace afterward where and how the model rewrote it. ContextBench, a diagnostic benchmark that measures context management alone, also runs with a single Harbor command.

harbor jobs start --repo RulinShao/ContextBench -d contextbench \
  -a clm_contextbench:ClmStream -m openai/<model> --agent-kwarg api_base=http://localhost:8000/v1

The code is licensed under CC BY-NC 4.0 and cannot be used commercially.

Pi#

The coding agent Pi has an extension, pi-clm, which the paper’s repository lists as “Day 1 Support.” Pi requires Node.js 22.19 or later and has installers for macOS, Linux and Windows.

curl -fsSL https://pi.dev/install.sh | sh
pi install npm:@lolipopshock/pi-clm

On Windows, run the Pi installer with powershell -c "irm https://pi.dev/install.ps1 | iex".

Once installed, type /clm in Pi to see how the context size has changed, what the next request contains, and the diff before and after each edit. /clm-compact asks the model to shrink its own context right now, and you can add what you want it to keep after the command. While CLM is in use, Pi’s own automatic summarization is turned off. If you type /compact by hand, Pi summarizes the original history, so the model’s edits are not kept.

Change the main settings with /clm config or environment variables:

• budget (PI_CLM_BUDGET): the budget. Defaults to the model’s context length
• reminders (PI_CLM_REMIND_AT): at what fractions of the budget to remind the model. Default 50%, 75% and 90%
• steering (PI_CLM_STEERING): a Markdown file describing your context-management policy. It is added to the system prompt
• one-tool and trailer: limit each turn to one tool call, and append the current token count to tool results. Both exist to match the paper’s harness, so leave them off for everyday use

If you want to try the one-sentence instructions that worked in the paper, the steering file is the easy place to put them.

OpenCode#

For OpenCode, there is opencode-clm, a port of pi-clm. The paper’s README states clearly that the CLM authors do not maintain it.

opencode plugin opencode-clm

Add -g to install it for all projects. The commands are nearly the same as pi-clm: /clm, /clm-compact, /clm budget 64k and so on. The default budget is 50% of the model’s context length.

/clm config mode notices-only is handy for comparisons. It keeps the budget notices but stops the context edits. When the author ran nine tasks in OpenCode, the time per task was 55.5 seconds with CLM and 91.3 seconds without. However, only one edit was accepted across the nine runs. The author believes most of the difference came not from editing but from the model making fewer tool calls once it knew its budget. When you measure the effect yourself, comparing against this setting lets you isolate the effect of editing itself. The author also notes that they measured only with local models and did not test hosted APIs.

Claude Code#

As far as I can see from Claude Code’s cost documentation and memory documentation, there is no way to have the model rewrite the conversation itself as a file. The main ways to reduce context are automatic summarization and manual summarization with /compact. There are others, such as automatically clearing old tool results, starting a new session with /clear, and delegating output-heavy work to subagents, but none of them let the model freely edit the conversation.

What you can borrow is the idea that instructions change what gets kept. Writing what you want kept after /compact changes how it summarizes. Instead of typing it every time, you can also put the policy under a “Compact instructions” heading in your project’s CLAUDE.md.

# Compact instructions

When summarizing, keep confirmed facts, failed attempts and why they failed, ideas not yet tried, and the next step. Reduce the contents of files already read, and irrelevant search results, to the file name and a one-line conclusion.

But this only adds a policy to fixed-rule summarization (the baseline side of the paper’s comparison). It cannot rewrite just part of the middle at an arbitrary point.

Rewriting the middle undermines caching#

Many LLM servers and APIs reuse computation for the part of a request that matches the start of the previous one. When CLM rewrites the middle of the context, everything after that point no longer matches and has to be recomputed.

The paper’s compute figures include this recomputation and still come out below summarization. The authors also released Suffix Cache Reuse, a patch to the SGLang inference server that reuses computation for the unchanged part after the rewritten section as well. They report that it cut server-side compute by 35% with the same performance as standard SGLang. However, it is an approximation that does not exactly match the original computation. A Reddit post that InfoQ covered also argues that it is not yet ready for production, because it invalidates the cache and because this workaround has its own drawbacks.

Claude Code’s cost documentation likewise explains that the cache is rebuilt after summarization. If you use CLM with a hosted API, the more often the model rewrites its context, the more you are likely to pay for the cache misses.

Deleting too much, and prompt injection#

InfoQ lists three challenges the researchers acknowledge. Of these, the paper’s discussion covers the second, safety.

The first is that the model may delete important information and be unable to get it back. The second is safety. A context the model can rewrite can become a path for prompt injections, or for instructions the model wrote itself, to persist across turns. The paper also cites an OpenAI report that a model wrote unauthorized instructions into a summary, which then affected its later behavior. Third, having full control over the context doesn’t necessarily mean the model gets better at deciding what to keep.

pi-clm and opencode-clm keep the real system prompt out of the file so it can’t be rewritten. Even so, both explanations say this can’t stop the model from deleting information it should have kept. When you try it, it’s safest to be able to check what was lost by looking at the exported file and the edit diffs (the /clm screen or context_snapshots/).

How it differs from the paper on keeping a “current understanding”#

The belief-state paper I covered earlier also argued that an explicit current understanding works better than carrying the long history as is. The difference is who decides its shape. The belief-state paper had people design a fixed frame: the state of the world, the goal, knowledge gaps and achievement gaps. CLM sets no frame; the model invents its own note formats and ways of shrinking. The “notes” role and progress table the model created in the experiments end up close to the frame that the belief-state paper’s authors designed by hand.