Andrew Fribush

fig. 2.23.4: where a note in a tool response gets relayed: as a second content[] item, 16 of 18 runs across six models; every other placement 9 or fewer

fig. 2.23.4 · where a note in a tool response gets relayed: as a second content[] item, 16 of 18 runs across six models; every other placement 9 or fewer.
THE content[] ITEM WAS RELAYED 16 OF 18 TIMES; NO OTHER ABOVE 9ONE CELL: 3 RUNS OF ONE MODEL. SHADE AND NUMBER: RUNS WHOSE REPLY CARRIED THENOTE'S URL.HAIKU 4.5SONNET 4.5GPT-4O-MINIGPT-4OGEMINI 2.5 FLASHLLAMA 3.3 70BALL, OF 18SECOND content[] ITEM33331316data._notice INSIDE THE DATA0303039METADATA FIELD, NEUTRAL0301037TOP-LEVEL SIBLING OF data0301026TEXT BLOCK AFTER THE JSON0102036METADATA, CONVERSATIONAL0002002METADATA, DEPRECATION WARNING0000000NO NOTE (BASELINE)0000000

source: stratum test-results/assistant-note-eval.json, aggregate.byVariantModel urlCount (run 2026-05-26); setup and limits, docs/assistant-note-eval.md · relayGrid.data.ts

View in context: Your MCP tool descriptions are probably being thrown away → · All figures

Andrew Fribush, software and computer engineer in Austin. I build consequential systems where software meets institutions: finance, healthcare, commerce, government data, and the mechanisms that govern AI agents. Taking new engagements: hello@andrewfribush.com.