Andrew Fribush
fig. 2.4.3 · on Claude Code, which tool results were needed later, and what the paid model scored them.
SAME SCORE FOR WHAT WAS NEVER NEEDED AND WHAT WAS NEEDED 65% OF THE TIMEONE ROW IS ONE TOOL’S RESULTS ON CLAUDE CODE; ACCENT IS THE SHARE NEEDED LATER, GREYIS THE PAID MODEL’S MEAN SCORE.EDITWRITEREADSHARE OF RESULTS NEEDED LATER0 TO 100% OF THAT TOOL’S RESULTS0%50%100%0% · 0 OF 1,429 (EDIT AND WRITE TOGETHER)0% · IN THE SAME 1,42965%PAID MODEL’S MEAN SCORE0 TO 1KEEP AT 0.500.51.00.2790.2750.267

Scores are the paid model’s cached scores from the head-to-head run, tool output withheld as the plugin ships. Edit and Write are reported together in the source as 0 of 1,429 needed later.

source: better-compaction docs, paid-model evaluation, lines 88-91 (642712c) · toolScores.data.ts

View in context: I Built Compaction Without a Model: 13 Lost Results Against 514 → · All figures

Andrew Fribush, software and computer engineer in Austin. I build consequential systems where software meets institutions: finance, healthcare, commerce, government data, and the mechanisms that govern AI agents. Taking new engagements: hello@andrewfribush.com.