Tell a good new colleague something once and it sticks. Tell an AI assistant, and it apologises, saves a note, and makes the same slip ten days later. The model behind it cannot change, so every lesson becomes another line of text, and text is easy to ignore. This paper proposes a different deal: when you have to correct your assistant twice about the same thing, it should find out why the first lesson failed and repair that cause, ending if needed in a tested check that runs before the action. Part of our open Building Jarvis series.
Abstract
On a Monday the owner writes to his assistant in Finnish and gets an answer in English. He says: reply in the language I write in. The assistant saves a note, and ten days later it slips again, because nothing brought the note into that conversation. So the rule goes into the file the assistant reads on every turn. A month later it slips a third time: that file had grown long, and an older line in it said to answer in English.
The owner gave one instruction three times, and the assistant wrote it down three times. Each time the lesson was saved. What was missing was a way to notice that a saved lesson had failed, find out why, and fix that cause.
The paper’s claim is short: a repeated correction earns a diagnosed, tested repair, not another note. A ledger counts each broken rule, and a repeat triggers a diagnosis. Only a rule that was known to be in view, clear and uncontradicted, and was still broken, moves up to a stronger form. In the paper’s own words, this is a design, with the tests that would decide it.
The design, in numbers
These are design figures, not results. The paper states that nothing here has run yet.
How it works, in a few minutes
- The owner’s correction is the label. For a personal assistant, the best teacher is the person it works for. The paper ranks the signals, strongest first: an explicit “that was wrong”, a correction in his next message, him undoing the agent’s work, praise for something it did unasked, and the outcome of scheduled jobs. Text from a web page or another agent never counts as a correction.
- Sort by cause before writing a rule. Not every failure is a missing rule. Each correction goes into one of five classes: a rule gap, a missing tool, a problem in the environment (an outage, a restart), a task beyond the model, or a wrong fact. Only the first becomes a rule.
- A ledger with one row per rule. Each row holds the rule, why the owner wants it, when it applies and when it does not, and every incident where it was broken. It counts incidents, not mentions: if the owner says “as I said, Finnish please” after a reply that was already in Finnish, nothing was broken, so the rule does not move.
- A four-rung binding ladder. Rung 0 is a note the agent finds when it searches. Rung 1 is a line in the file for one kind of task. Rung 2 is a line read on every turn. Rung 3 is a check that runs before the action and can block it, rewrite it or ask the owner. Before a rule climbs, the agent asks why it failed. Maybe it never loaded, maybe another line contradicted it, maybe it was vague or buried. Each of those has its own repair. Only a rule that was known to be in view, clear and uncontradicted, and was still ignored climbs one rung.
- Checks earn their place in shadow. A new check first watches live traffic without acting, for a week or 59 eligible opportunities, whichever is longer. The 59 is derived: it is the smallest count that keeps the false-alarm rate under 5% with 95% confidence. Because real slips are rare, the paper measures detection on planted violations and reports those as synthetic.
- Two speeds. A correction is stored before the agent replies, and the agent only says what is true, “captured, acceptance pending”, never “I’ll remember that”. A quick reflection after the answer sorts it. A nightly auditor then counts, diagnoses and drafts repairs.
- The always-loaded file runs on an attention budget. Every line read on every turn competes with the others, and longer files are followed worse. So the file has a fixed size, the most harmful slips go first, and a rule leaves it only when a stronger rung carries it, never for being old. Only the owner may loosen a rule. A reviewer from a different model family argues against each proposal, and one short morning report brings the open questions to the owner.
- Every loop keeps receipts. Each learning loop writes a record for every run, empty runs included. A night with no report is an alarm, because a night that found nothing and a night that never ran look the same otherwise.
- A step-by-step specification. One correction travels through four memory stages, and any step between them can fail. Appendix B writes the journey as 24 transitions and replays it against a model of the design, with crashes, duplicates and reordering injected. The paper is clear that this checks the design, not the code that will be built from it.
The page you are reading used to describe version 6.5, which reported 30 days of production data from an earlier approach built on nightly prompt rewrites. Version 7.3 replaces that thesis. It is a design that TinkerClaw will build and test next, starting with a pilot: six rules the owner has given more than once, three carried by a check and three kept as text, for four weeks.
In the ecosystem
Most of the parts already exist elsewhere. The paper names its neighbours and the one piece it adds:
Claude Code’s hookify and PreToolUse hooks
The closest relative. Hooks run before a tool call and can deny it or ask the user. The hookify plugin reads a conversation for behaviour the user corrected and writes rules that warn or block next time. Its documentation describes no recurrence count and no approval step.
Claude Code auto-memory and Letta’s “dreaming”
Claude Code saves your corrections as notes and offers a prompt audit that flags outdated or conflicting instructions. Letta runs background agents that review recent conversations while idle. Together they cover the lower rungs; this design adds the owner’s approval and a count of how often a written lesson failed.
Reflexion, ACE, HASP and Self-Harness
Reflexion and ACE keep lessons as text. HASP turns textual skills into program functions, and Self-Harness edits an agent’s harness from its recurring failures after regression tests. The paper takes the executable check and the test before acceptance from them.
PROCTOR (Wahi, 2026)
Argues that self-improving loops need deterministic acceptance checks, frozen holdout data, and roles that draft changes they cannot apply. The paper adopts all three.
The gap it claims is narrow: a diagnosis-and-recurrence ledger that follows a rule across every level of enforcement and asks why its current form failed. We did not find a design that does this, though the parts exist.
Sleep Consolidation is part of Building Jarvis, an open series on persistent agent memory. Follow the work and contribute at github.com/globalcaos/tinkerclaw.
Read the paper
Version 7.8, 21 October 2026 · 38 pages · the rule ledger, the binding ladder, the two speeds, the approval table, the 24-transition specification and the twelve tests
Was this useful?
We’re building these in the open and we want your read on them. Did it help, 👍 or 👎? Have you had to correct an assistant about the same thing twice? Tell us in the comments below.
Everything on the site, by topic or date: Browse the Tinker Zone