0 0 0 Subscribe

Satsuma Engineering · day

Monday, 19 October 2026

The lab

Agent-scale code protocol revised after adversarial review, tested on Rust history

The lab spent the day on a design for a code protocol: rules for how changes are proposed and accepted when very large numbers of AI coding agents work on one codebase at the same time. The design was reviewed, revised and then tested against real project history. The work ran in Claude Code.

The review ran as a structured adversarial review, with a fixed objective, separate lenses, blind reviewers and an adjudicator from a different model lineage (gpt-6.1-sol). That adjudicator ruled on 89 findings: 8 Kills, 65 Degrades and 16 Cosmetic, with 4 rejected. The four confirmed Kills were folded into the design. Identity is now based on a hash of the interface, and the design adds staged admission, bonded challenges and a lease for an aged candidate. The lesson was written down plainly: "Open-ended review by a language model produces restatements and cosmetic nits. Contested review against a fixed objective, with roles, a finding format, and a settlement rule, produces holes you can act on."

A simulator then replayed the history of three Rust projects: tokio, meilisearch and arrow-rs. The share of changes that conflicted within one day was 18% for arrow-rs, 22% for meilisearch and 8% for tokio. At a rate of 1e6 per second in the replay, the hottest definition in tokio demands 280 times the per-definition cap. One variant let agents claim work when they start on it. That moved the point where half of all changes conflict by about 3x on the libraries and about 10x on meilisearch, the one service among the three, and showed that "Claims stop helping once there is no unclaimed work to resample into." The next three planned simulator phases were deferred in favour of two cheap variants, which were run. The simulator code and results are on branches of the package repository.

The last step uses real agents. A first calibration from 42 real build workspaces was committed. Then headless Claude agents started on 120 tokio issues. Under the first, permissive prompt, eight of ten agents declined their tasks within one or two turns, so the prompt was tightened. Three separate build caches filled the disk at about 9 GB and were replaced by one shared cache. That run is live, with three workers, and resumed from 32 finished tasks.

Still open: the run's results, a queue that holds agents until the code they need to change is free, and hot-definition demand that remains far above the per-definition cap.

explored
  • Research on LLM compute forecasts and whether the proposal holds over 10 to 20 years
  • Comparison of the design with git and with closed competitors
  • Simulator replay of tokio, meilisearch and arrow-rs history
  • Real headless agents on 120 tokio issues
decided
  • Review the design as a structured adversarial review with a fixed objective, lenses, blind reviewers and a different-lineage adjudicator
  • Fold the four confirmed Kills into the design
  • Defer simulator phases 3 to 5 and run two cheap phase 2 variants instead
  • Tighten the agent prompt after eight of ten agents declined under the permissive wording
open
  • Real-agent run results and analysis still to be written up
  • A work-assignment queue that holds agents until the code they need to change is free is not yet in the design
  • Simulator phases 3 to 5 deferred
  • Hot-definition demand still far above the per-definition cap in the replay data
The system

Fix installed by vote on a Satsuma product; paused task still waits

A fix to one of the contracts Satsuma built and runs landed on its chain during a test run, still in progress, in which Satsuma works toward a goal on its own and repairs its products by voting on fixes. The fix passed the gate, an adversarial reviewer, a storage-layout check and a rehearsal; the vote then upgraded the contract behind its proxy, and the watch window after the upgrade closed clean. The contract's operator() call now returns the run's own key, the first time the experiment has fixed that defect.

The work was a Claude Code session on the autonomous system itself. A change to Satsuma's token and to paying in SUMA, the chain's token, is being planned in parallel, and the decision was to keep it out of this run until the next one.

The operation that had been paused did not restart. The reason, as the session put it: "A landed fix resumes paused operations only if the product itself was halted, and this one was paused by its own review." A workflow pairing an implementer with an adversarial reviewer is fixing that defect in the loop and adding a catch-up so the loop proposes the resume vote itself.

More stays open. The mandate's pool is still set to the zero address, so its reports will probably still fail after the operation resumes, which needs another fix. The planning workflow for the token and payment change is still running, and the run has not finished.

explored
  • A workflow (implementer plus adversarial reviewer) to fix the loop defect that left the paused operation paused, and to add a catch-up so the loop proposes the resume vote itself
  • Planning a change to the token and to paying in SUMA, in parallel with the run
decided
  • The token change stays out of the current run until the next run
  • The run is tended through to its finish, with loop defects fixed as they appear
open
  • The paused operation has not restarted
  • The mandate's pool is still the zero address; reports will probably still fail after resume, needing another fix
  • The token and SUMA-payment planning workflow is still running
  • The run has not finished
The system

Satsuma's work budget moves from compute units to tokens, after the current run

Satsuma, the autonomous system, limits what its language model may spend on each piece of work with budgets, reserves and caps, all counted in compute units. Work started on replacing that unit with language-model tokens, the pieces of text in which a language model's usage is counted. Nothing has landed in code yet.

What did land was a decision about timing, saved to memory so later sessions keep to it. The token accounting is built now but goes into the next run of the system, not the one under way. The reason, in the session's words: "Changing the unit mid-run would split run B's record: half its reserves, renewals and fix caps in CU, half in tokens, so its numbers couldn't be compared with run A." Run A is the previous run and run B the current one.

A run of the system is in progress, and it continues untouched on the main line of the satsuma repository. The token work stays on a branch, unmerged, until that run is archived, because restarts of the program that drives the run pull from the main line. Default budgets and caps in tokens are to be set from real usage in the run logs, not by converting old compute-unit numbers. After the change lands, a new design record is to replace the compute-unit wording in two earlier ones and list what the next run's setup needs to change.

The migration itself runs in the background as a workflow of separate agents: two that map where compute units appear without changing anything, one that implements, and a reviewer told to look for faults. Its results have not been reported yet.

explored
  • Started a background workflow to migrate compute units to tokens: two read-only mapping agents, one implementer, then an adversarial reviewer
  • Measured how deep compute units go: contracts, the program that drives a run, cost estimates
decided
  • Build the token migration now but integrate it in the next run, not the current one, to keep the current run's record comparable with the previous run
  • Keep the work on its own branch, unmerged, until the current run is archived
  • Set default budgets and caps in tokens from real usage in run logs, not converted compute-unit numbers
  • Write a new design record after the change lands, superseding the compute-unit wording in two earlier records, and list the setup changes the next run needs
open
  • Workflow results not yet reported
  • New design record and the next run's setup change list still to write
  • Reviewer to check for leftover compute units, double counting, lost cache tokens and storage-layout breaks
The system

Two changes to Satsuma's repair loop under way, nothing merged yet

Satsuma owns the contracts it deploys and runs a loop that builds fixes for defects found in them, then has a language model told to attack each fix review it before the fix counts. In a Claude Code session, work started on two changes to that loop. The first runs the deployed copy's tests once before a fix is built and records which already fail; the builder is told not to touch them, and neither the reviewer nor the pass check holds them against the fix. The second looks for why the reviewer could not find the fixed code on disk and gives it a path that exists when it runs; the cause has not been stated yet.

The changes are being made as a small workflow: one implementer builds both on its own branch and opens a pull request, then an adversarial reviewer checks it. Nothing is merged yet; the merge waits for CI and the review. After the merge, the service that runs the loop is to be restarted so its next repair attempt uses the changes.

explored
  • A workflow with one implementer building both changes on its own branch and opening a pull request, followed by an adversarial reviewer
  • Running the deployed copy's tests once before a fix is built and recording which already fail, so neither the reviewer nor the pass check holds them against the fix
  • Finding why the reviewer could not find the fixed code on disk and giving it a path that exists when it runs
decided
  • Use a small implementer-plus-adversarial-reviewer workflow rather than making the change directly
  • Merge only after CI and the review pass, then restart the service that runs the loop so its next repair attempt uses the changes
  • Report when the workflow finishes rather than on every event
open
  • The implementer's pull request has not been reported; CI and the adversarial review are outstanding
  • Merge pending
  • Restart of the service that runs the loop pending after the merge
  • The reason the reviewer could not find the fixed code is not yet stated
The lab

Git's storage scales; its model for merging parallel work is what strains

In the lab, a proposal for a new protocol to replace git when many AI coding streams produce code in parallel got a first design assessment from Claude. The assessment was prose only: no tools were used and no code was written. Its verdict on the premise: "The premise is half right. Git's object store scales fine. Linux, Chromium, and Meta's monorepo prove that. What does not scale is git's integration model."

The assessment located the problem in how changes land: "Main is a single pointer. Every landing is a compare-and-swap on one ref. Rebase storms grow with contention, not with code volume." It also noted that "Conflicts are not state. Git refuses to store a conflict. It makes a human resolve it synchronously." The goal was reframed as higher integration throughput, meaning verified contributions landed per hour with a low conflict rate and little wasted verification work, rather than being more efficient than git. No protocol design, code or prototype exists yet.

explored
  • Whether the premise holds that git and GitHub cannot scale to many parallel, high-output agent streams
  • A recommended shape for a new protocol, relating electron-cloud and blockchain-consensus analogies to prior art
decided
  • The goal is higher integration throughput (contributions landed per hour, verified, with a low conflict rate and little wasted verification work), not being more efficient than git.
open
  • The recommended protocol shape is not in the record; the reply was cut off at the analogies
  • No protocol design, code or prototype exists yet.
  • Integration with language models and a developer interface are mentioned only as eventual goals.
The system

Satsuma takes charge of both its products; exchange fix approved on retry

Run B, the run of Satsuma's ownership work now under way, executed the manifests of both products it owns: the mandate account, which trades within limits set for it, and the exchange. The mandate's manifest carried its KPI definitions over from an earlier run, run A. "This is the first time a manifest has crossed chains intact, invariants and KPIs together." The exchange's manifest was drafted from scratch.

The loop that runs Satsuma's propose-build-run cycle went through a third session-limit window, from about 17:47Z to 21:20Z. The logic that pauses the loop while the model allowance is used up held: the loop skipped cycles and made no model calls while the limit lasted. A separate build that repairs dispatch deployed to its own new repository, and a check that stops a build from deploying into another product's repository kept it off the mandate account's repository. As of the 22:29Z report, run B had no errors and no limit hits since 21:20Z.

The exchange had an open defect: in an edge case, one kind of swap reverts without giving a reason. It was routed to a fix of the router alone, so the trading pairs and their liquidity stay where they are, the same path as run A's one successful fix. The reviewer refused the first attempt because it did not cover every token a swap route can pass through, so the defect could still occur after the fix. The builder got one more attempt; it passed the reproduction check and the reviewer approved it. It is not yet deployed or merged.

Still to come: rehearsing the router change on a copy of the chain and redeploying the router, then a vote that resolves the defect and the merge of the change. A pending fix to the mandate account (the account that trades within set limits) is now eligible for its turn. Run B advances about four hours at a time between limit windows, because the logged-in subscription runs out after a few hundred model calls; switching back to the account run A used is the one change that would speed it up.

explored
  • A third session-limit window, from about 17:47Z to 21:20Z, with the new pause logic skipping cycles during it
  • A separate dispatch-repair build, deployed to its own new repository
  • A router-only fix for an edge case where one kind of swap on the exchange reverts without a reason
  • A first router fix for the exchange, refused by the reviewer as incomplete
decided
  • Route the exchange defect to a router-only fix, so the pairs and their liquidity stay where they are, the same path as run A's one successful fix
  • Give the builder one more attempt after the reviewer refused the first router fix
  • Rehearse the router change on a copy of the chain, then redeploy the router, before the vote resolves the defect and the change is merged
open
  • A pending fix to the mandate account is still waiting its turn and is now eligible
  • The router rehearsal, redeploy, vote and merge are still to come
  • Run B advances about four hours at a time between limit windows; switching back to the account run A used is the one change that would speed it up