I Built Compaction Without a Model: 13 Lost Results Against 514

Coding agents compact their context by throwing away evidence. When the context window, the fixed amount of text the model can see at once, fills up, Claude Code and Codex have the model summarize the session and carry on from the prose, and a popular compaction plugin pays a model to guess, result by result, what the agent will need again. At compaction time, everything the agent wrote and ran after a tool result (a file it read, a command's output) is already in the transcript, the file where the session is saved, so whether anything used that result is a lookup, not a forecast. I built better-compaction on that lookup: a plugin that runs in both of them and decides what to keep with five deterministic rules, no model and no network call. Replayed on 10 real Claude Code sessions, it lost 13 results the agent went on to need, and the popular plugin lost 514.

better-compaction never summarizes tool output. For every tool call and result it decides keep, cut to the first part, or drop, and what it keeps stays byte for byte as it was. A long session compacts five or ten times, and each pass can only tighten what the last one decided, until a further pass leaves the history alone. To get inside the hosts I reverse-engineered the shipped Claude Code binary and found a session.compact hook absent from its hooks reference, and routed Codex through the replacement_history record a resumed session rebuilds from. It runs live in both hosts: Claude Code keeps the plugin's rewrite in place of its own summary, and I made Codex resume directly from the pruned history, with no native compaction on the resumed turn. The Codex route tracks the rollout format through codex-cli 0.156.0, and the 340 tests run in about a second.

I graded it against a frozen yardstick on real sessions from both hosts, on the part of each session the rules never saw, and replayed the paid-model plugin's own compiled code, as shipped, through the same harness. Where the hosts' own summaries dropped 583 and 790 needed results at the compactions they ran, the rules dropped 9 and 12, keeping 98.9% and 98.5% of what each session went on to use.

Whether a result was used is a string search over what came after it

Take a file the agent read two hours into a session. By the time compaction runs, every message the assistant wrote and every command it issued after that read is saved. My lookup pulls the read's distinctive tokens and checks whether any of them shows up in a later assistant message or a later tool input. Here a distinctive token means a searchable string, not a word piece. It can be a path, an identifier (a name in code, such as a function or variable name) of six or more characters with a _, a - or camelCase (wordsJoinedLikeThis) in it, a number of four or more digits, a quoted string of eight or more characters, or a whole line with an error marker.

This is the core of the check, from src/core/rules/refs.ts:

for (const token of candidates) {  if (knownBefore.includes(token)) continue; // known before the result existed  if ((resultCountByToken.get(token) ?? 0) > 10) continue; // too generic  const seenAt = lastSeenAt.get(token);  if (seenAt !== undefined && seenAt > pair.msgIndex) matched.push(token);}

The two continue lines do most of the work. A token that was already in the call's own input proves nothing. If you asked to read src/app.ts, seeing src/app.ts again later doesn't mean the file's contents mattered. And a token that turns up in more than ten different results is too common to count. What's left is matched against the last place each token was seen, and a match after the result means a later step relied on it. A result with fewer than ten tool results after it is kept as too recent to judge.

Five deterministic rules decide every tool result

The reference check is the third of five. I gave re-reads, cheap commands and errors a rule each.

Superseded reads: the agent reads a file and later reads it again. If the later read repeats the earlier one or contains its text verbatim, and nothing wrote to that file in between, the earlier read goes. Latest read: the most recent read of each file is the agent's current picture of it, so it stays. Never referenced again: if no distinctive token from a result shows up later, the result goes.

Cheaply re-runnable: the output of ls, git status, git log or a passing npm test (list a folder, show which files changed, show recent commits, run the tests) is cut to its first 200 characters, since running it again gives a fresher answer anyway. Never drop: error results, results a later assistant message quotes, and results followed by an assistant message with a rule word in it (must, never, always, do not) stay no matter what the other rules say. Those are the things a summary drops first: the exact error string, the exact path, the flag the user said never to use. Anything no rule decides is kept.

A dropped file read keeps its call, and its output becomes a note, [output removed by better-compaction: N chars; re-run the call to see it]. The model sees that the read happened and that the contents are gone, and it re-reads the file if it needs them.

The plugin runs inside both hosts, through an unlisted hook and a rebuild record

To replace what Claude Code's compaction produces, you need a hook its published list leaves out. (A hook is a place where a plugin's code gets called when the host does something.) Taking the shipped program apart turned up a session.compact hook with its own error messages, and the plugin uses it to replace native compaction:

precomputed compact: discarded (hook_rewrite): a session.compact hook changed what is compacted

Hook modules load only with Claude Code's early-access flag, CLAUDE_CODE_ENABLE_FUNCTION_HOOKS=1. Anthropic publishes the source of its built-in modules and the type declarations.

Codex has no such hook: its compaction hooks can continue, stop or add a message, and none accepts a replacement history. But Codex keeps each session in a rollout file, and that file has a compacted record whose replacement_history field is what Codex rebuilds from on resume. So the plugin appends one carrying the pruned verbatim history and resumes from a fork, a copy of the rollout under a new session id. To prove it, I appended a record holding only a canary, a made-up codeword, and resumed. The model reported the codeword and nothing from the original conversation. Release 0.156.0 of codex-cli, Codex's command-line program, changed the rollout format three ways (a stricter filename, a sequence number on every line, a new kind of id in a new folder), and I updated the fork to handle all three.

A live session with three reads of one file and two edits between them went through /compact with five tool pairs examined and three dropped, 524 tokens down to 274, in 6.3 ms (thousandths of a second), and the transcript afterward held the last read verbatim and no summary. Another live session went from 70,219 tokens to 47,141, and the model answered from the copy the plugin kept. The plugin recovered a live session from the backend's prompt-too-long error: Claude Code accepted its 56-message rewrite, and the retried request succeeded. On Codex, a live fork of a pruned session resumed at an estimated 148,766 tokens, down from 162,839. Codex skipped its own compaction on the first turn back, and the model saw the removal notes.

Repeated compaction is safe because decisions only tighten

Long work compacts again and again, each pass running against what the last one left. I made repeated compaction monotonic: decisions only tighten, protected results stay untouched, and truncation reaches a fixed point.

One rule may narrow another's decision and never widen it. The rules run as a ladder (pin, never-drop, superseded, latest read, referenced, re-runnable) and the first rule to decide a result wins, with one exception: a later rule can move a keep to a truncate, and nothing else. This is merge(), from src/core/compact.ts:

const merge = (s: Scored): void => {  if (pinned.has(s.callId) || neverDropIds.has(s.callId)) return;  if (s.decision.action === 'truncate' && supersederIds.has(s.callId)) return;  const prev = decided.get(s.callId);  if (!prev) {    decided.set(s.callId, s);    return;  }  if (prev.decision.action === 'keep' && s.decision.action === 'truncate') {    decided.set(s.callId, s);  }};

Pinned and never-drop results can't be touched, and a read that replaced an earlier one can't be shortened. Nothing in merge() moves a decision back toward keep, so the rules can't undo each other.

That fixed point is a state another run leaves exactly as it is. A truncate that wouldn't shorten the text becomes a keep, and so does one on a result an earlier pass already cut, where cutting again would only rewrite the removal note. Neither puts a character back, and a result an earlier pass cut stays exactly as that pass left it. Untouched messages come back as the same objects in memory, not copies, so the unchanged part of the history stays identical down to the byte from pass to pass.

Later passes find more superseded reads, because long sessions re-read the same files, so pass three has more dead reads to remove than pass one did. And a pass that wouldn't remove enough to matter can be told to do nothing: set minRemovalFraction, and a pass that would remove less than that fraction returns the input unchanged.

Across three live compactions in one Claude Code session, every existing removal note stayed byte-identical, with one new note per pass. When the agent needed a file whose read had been removed, it re-read it and answered correctly.

The rules are graded on the part of the session they never saw

To score the rules I built a replay harness and ran it on 20 real sessions, 10 from Claude Code and 10 long Codex rollouts (the file Codex keeps a session in). Each session was cut at 50%, 70% and 90% of its messages, 60 cuts in all. The compactor ran on everything before the cut, and the scorer looked at what the assistant actually wrote and ran after it, which the rules had no access to. A removed result counts as a miss if one of its distinctive tokens shows up after the cut, and the miss rate is misses as a share of results removed. A miss counts as lost if that token appears nowhere in what the compactor left, so the model can't see it at all. The yardstick is a frozen copy of the token extractor, the code that pulls distinctive tokens out of text, pinned to one version, so changing the rules can't change the ruler.

The rules never touch pinned results, which are everything in the first message and the last six messages, plus every error result. On Claude Code they removed 2,340 results, 56.5% of the unpinned ones, and 22.6% of the tokens before the cut (a token here is the model's unit of text, about four characters of English). The miss rate was 2.1%, and 13 removed results were lost, 0.6% of what was removed. On Codex they removed 562, which is 35.9% of the unpinned results, and 11.3% of the tokens, with a 6.8% miss rate and 16 lost, 2.8%. Of the 38 Codex misses, 8 matched only on number-like tokens such as years, version numbers and decimals. Set those aside and the Codex miss rate is 5.3%, and Claude Code's 1.8%.

Against a paid model's guesses, the lookup lost 13 needed results to 514

The lookup replaces a guess, and the popular plugin from the opening is the guess in its plainest form. In September it drew 6,237 GitHub stars (bookmarks on GitHub, the site where most open-source code lives) in its first five days. For each tool result it buys a score from a model for the chance the result is needed again, and keeps the result if the score is at least 0.5.

I compiled the plugin's own compact() from its source, pointed it at its paid model with its shipped defaults (that 0.5 threshold, and the last six messages pinned), and scored it on the same sessions with the same frozen yardstick. That came to 137 requests, 3.56 million input tokens and 18 redactions. For comparison I added a rule with no judgment in it at all, which drops whatever isn't pinned.

(The removed column's denominator includes pinned results, which no method removes; the 56.5% and 35.9% above are shares of unpinned results only. The Claude Code totals differ because one cut was over the paid model's 25,000-token limit, so it's missing from those two rows.)

At its default threshold it removed 3,619 of the 3,632 Claude Code results it was allowed to touch and 1,611 of 1,613 on Codex, matching a drop-everything rule's count exactly at 51 of 59 cuts. Its 514 lost results on Claude Code are 39.5 times the rules' 13, and its 384 on Codex are 24 times their 16. Measured per result removed, its loss rate ran 26 times the rules' on Claude Code and 8 times on Codex. Appendix A has the arithmetic.

The model was never shown what it was judging. Tool output isn't sent by default, for privacy, so each result reaches it as a placeholder like [48213 chars not sent]. The one thing a placeholder tells you is length, and the scores follow length. Ranking by size alone beat it in every evaluation set. On Claude Code, size scored an AUC of 0.90 and the model 0.56. (AUC is the chance that a randomly picked needed result scores higher than a randomly picked unneeded one, so 0.5 is a coin flip.)

It also read the tool name backward. In the Claude Code sessions, Read results were needed later 65% of the time and Edit and Write results (the tools that change a file) 0 times out of 1,429, and it scored Read lowest, 0.267 against 0.279 for Edit and 0.275 for Write. A one-line rule on the tool name removes all 1,429 Edit and Write results and loses nothing the session needed later.

I also tried it as a rescue layer on 24 Codex sessions it had never seen, where its picks left 20 needed results lost, against 16.8 for random picks of the same size and 13 for keeping the largest 20% with no model.

The rules keep what the built-in summary throws away

Measured at the compactions the hosts really ran, 813 of 822 needed results in Claude Code and 793 of 805 in Codex were still visible after the rules. The built-in summaries lost about 65 times as many. Each summary is prose about evidence, and the yardstick measures what that prose drops. "Later" here means everything up to the host's next compaction. Claude Code ran 29, and its summary freed 95.0% of the context on average while 583 of the 822 results used afterward (71%) vanished from view. The rules alone freed 21.4%. Codex ran 27, and its summary freed 95.3% and lost 790 of 805, 98%, because its replacement history keeps the user's messages and a summary and no tool output at all. The rules freed 13.5% there.

The rules keep the evidence the built-in summaries discard, so the plugin runs them first and escalates only when the remaining history exceeds the context budget. Above 60% of the window it cuts further, oldest and most re-readable first, and in Claude Code it hands the session to the built-in summary when even that falls short. Per compaction point, that puts the rules' 9 and 12 against the summaries' 583 and 790.

The rules leave the agent's next step where it was

Cut by 40%, the history still produced a next-step score of 0.72, against 0.75 untouched and 0.55 for a reproduction of the built-in compaction. To check whether the agent does the same thing next, I built a blind next-step test at 60 points in real sessions, 30 in Claude Code and 30 in Codex, where the assistant's next turn was a tool call. At each one a fresh model got each of four versions of the history, one at a time, and named the tool call it would make next. A separate model graded blind, shown the four guesses unlabelled and in random order, and scored each one against the call the real assistant made: 2 for the same step, 1 for related, 0 for different, and the means below are on that scale.

With nothing removed, the mean was 0.75. After the rules, it was 0.73. With the plugin's text escalation level, which also trims old assistant text and was set to cut the history in half, that 40% reduction scored 0.72. A reproduction of Claude Code's own compaction, built from the compaction prompt in the 2.1.280 binary, scored 0.55 across all 60, and on the 30 Codex points it came in 0.30 below full context.

Whether a tool result is still needed shows in the record of what came after it, and better-compaction reads that record with a text search. It lost 13 needed results to the paid model's 514, kept more than 98% of the needed evidence at the hosts' own compactions, and held the agent's next step at 0.73 with the rules alone and 0.72 with two fifths of the context gone, against 0.75 untouched and 0.55 for the built-in reproduction. Every one of those decisions was deterministic, with zero model requests.

Appendix A: The replay arithmetic

The ratios in the replay section come from the table's counts: results removed, and of those, results whose needed content was lost.

# Replay table, better-compaction docs (repo 642712c). Results removed and lost.rows = {    "Claude Code": {"paid": (3619, 514), "rules": (2340, 13)},    "Codex":       {"paid": (1611, 384), "rules": (562, 16)},}for harness, r in rows.items():    paid_rate = r["paid"][1] / r["paid"][0]    rules_rate = r["rules"][1] / r["rules"][0]    print(f"{harness}: paid model lost {r['paid'][1]}, the rules {r['rules'][1]}, "          f"{r['paid'][1] / r['rules'][1]:.1f}x as many")    print(f"{harness}: paid model loses {paid_rate:.2%} of what it removes, "          f"the rules {rules_rate:.2%}, ratio {paid_rate / rules_rate:.2f}x")

Claude Code: paid model lost 514, the rules 13, 39.5x as manyClaude Code: paid model loses 14.20% of what it removes, the rules 0.56%, ratio 25.57xCodex: paid model lost 384, the rules 16, 24.0x as manyCodex: paid model loses 23.84% of what it removes, the rules 2.85%, ratio 8.37x