<?xml version="1.0" encoding="UTF-8"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>Automater Intel</title><link>https://automater.ai/intel/</link><atom:link href="https://automater.ai/intel/feed.xml" rel="self" type="application/rss+xml"/><description>Field guides for the agentic era.</description><language>en-us</language><lastBuildDate>Mon, 21 Sep 2026 00:00:00 GMT</lastBuildDate><ttl>60</ttl><item><title>Reconcile Before You Charge Back: Client Estimate, Gateway Meter, Provider Invoice</title><link>https://automater.ai/intel/gateway-chargeback-invoice-reconciliation/</link><guid>https://automater.ai/intel/gateway-chargeback-invoice-reconciliation/</guid><description>AI chargeback starts with reconciliation: price client estimate, gateway meter and provider invoice on one rate file, and block runs that don&apos;t tie out.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>Finance wants one line per team on the third working day of the month. The provider sends one line per credential, and behind a Claude apps gateway there is one credential for everyone: Anthropic’s own docs say the provider’s bill “attributes everything to that credential, not to individual developers.” Every AI chargeback you send is a claim about how to cut that single line, and three meters in your fleet will each suggest a different cut.</p>
<p>This runbook makes the cut defensible before the first internal bill goes out. By month end each lane has a three-column worksheet (client estimate, gateway meter, provider invoice) priced against one dated, versioned rate file. Totals reconcile at multiplier 1.0. Team charges are cut from the invoice, never from a meter. A variance register explains every gap, and anything unexplained above your tolerance stops the run.</p>
<p>Chatbots suggest; agents act, and acting agents spend with nobody reading along. One developer’s afternoon can hold a dozen subagents, two compactions and MCP calls across three plugins, all on the shared credential. The meters watching that spend were built as brakes. Billing people from a brake is how an internal chargeback program loses the room in its first month.</p>
<h2 id="sep-1015-claude-codes-gateway-grows-chargeback-controls">Sep 10–15: Claude Code’s gateway grows chargeback controls</h2>
<p>Three Claude Code releases in six days turned the Claude apps gateway into something that looks like a billing system. Per the <a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a>, 2.1.268 (Sep 10) pushes the gateway’s rates down to clients: “with <code>pricing:</code> set in <code>gateway.yaml</code>, signed-in Claude Code clients receive the same rates through managed settings, so <code>/cost</code> and telemetry match the spend meter.” <code>/cost</code> is an alias of <code>/usage</code>, which is what the docs call it now. The rates travel as the <code>modelPricing</code> managed setting, and the <a href="https://code.claude.com/docs/en/settings-reference#modelpricing">settings reference</a> says clients apply it from v2.1.242.</p>
<p>2.1.271 (Sep 14) raised the ceiling on markup, allowing a <code>multiplier</code> above 1, “up to 10, in the <code>modelPricing</code> managed setting and the Claude apps gateway <code>pricing</code> block, for marked-up internal chargeback rates.” 2.1.273 (Sep 15) added opt-in gateway hint headers (<code>CLAUDE_CODE_GATEWAY_HINT_HEADERS=1</code> sends request class, agent type, previous tool durations and compaction markers) and changed <code>OTEL_LOG_TOOL_DETAILS=1</code> “to also include real agent, skill, plugin and MCP server names on cost and token metrics.”</p>
<p>The docs are more candid than the release notes. The <a href="https://code.claude.com/docs/en/claude-apps-gateway-spend-limits">spend-limits page</a> calls the meter’s amounts “USD estimates, a circuit breaker rather than an invoice; for billing, reconcile against your provider’s usage reporting.” The <a href="https://code.claude.com/docs/en/claude-apps-gateway-config#pricing">gateway configuration reference</a> adds the sentence every chargeback memo should open with: “The multiplier doesn’t change what the upstream provider charges for the requests.”</p>
<p>
<em>Screenshot: Claude Code Docs, “Claude apps gateway configuration - Claude Code Docs” (undated), captured Sep 21, 2026.</em></p>
<p>Other meters moved the same week. GitLab 19.4 (Sep 17) turned its credit usage export into a ZIP of two CSV files, keeping the daily summary and adding a per-event file with “one row for each billable event” on Premium and Ultimate (<a href="https://docs.gitlab.com/releases/19/gitlab-19-4-released/">release notes</a>). GitHub’s Copilot Auto tiers (Sep 14) restated that “Usage is charged based on the model auto selects, regardless of tier” (<a href="https://github.blog/changelog/2026-09-14-configure-cost-and-quality-in-copilot-auto-model-selection/">changelog</a>), the routed-model billing that the <a href="/intel/hosted-router-vs-own-cascade/">hosted-router buyer’s guide</a> takes apart.</p>
<h2 id="three-readings-of-one-month-three-different-blind-spots">Three readings of one month, three different blind spots</h2>
<p>Each column answers a different question. The <strong>client estimate</strong> is what Claude Code computed from its own token counts and whatever rates reached it; <code>/usage</code>, the status line, the Agent SDK’s <code>total_cost_usd</code>, <code>--max-budget-usd</code> and OTel cost metrics all use <code>modelPricing</code> when it is set. The <strong>gateway meter</strong> is what the gateway added to its counters after each response, already multiplied. The <strong>provider invoice</strong> is what the provider charged the shared credential. None is broken by design. Each is blind to something the others see, and the vendor docs name most of those blind spots, which is where Step 5 starts.</p>
<p>The adjacent work is already live. <a href="/intel/three-meters-credits-tokens-sandbox/">Three meters: credits, tokens, sandbox</a> inventories meters and caps them before the spend, down to a <code>rates_version</code> per job; this piece starts after the spend, when three readings of it have to agree. <a href="/intel/agent-cost-anomaly-alerts/">Cost anomaly alerts for agent fleets</a> gives the invoice-versus-meter gap one paragraph. Here it gets a worksheet, an allocation rule and a block.</p>
<h2 id="step-1-price-every-column-against-one-dated-versioned-rate-file">Step 1: Price every column against one dated, versioned rate file</h2>
<p>Before any column holds a number, write down what it will be priced at. One rate file per lane per billing period, versioned, with a UTC effective window and the name of whoever changed it. The file is yours; the gateway’s <code>pricing</code> block is one input, alongside your provider contract and the list prices you checked. Seat and plan lanes record their plan terms instead of per-token rates; <a href="/intel/token-plans-decoded/">how token plans are priced</a> covers reading them.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># rates/claude-gateway-eng/2026-10.yaml  (illustrative shape; your own format)</span></span>
<span class="line"><span style="color:#85E89D">rate_file</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">claude-gateway-eng@2026-10.v1</span></span>
<span class="line"><span style="color:#85E89D">effective_from</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"2026-10-01T00:00:00Z"</span><span style="color:#6A737D">   # gateway monthly reset, 00:00 UTC on the 1st</span></span>
<span class="line"><span style="color:#85E89D">effective_to</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"2026-11-01T00:00:00Z"</span></span>
<span class="line"><span style="color:#85E89D">invoice_period</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"2026-10 (UTC calendar month; confirm against the contract)"</span></span>
<span class="line"><span style="color:#85E89D">sources</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  provider_contract</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;contract id, amendment date&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  list_prices</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;provider price page, fetched date&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  gateway_pricing_sha</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;hash of the gateway.yaml pricing block&gt;"</span></span>
<span class="line"><span style="color:#85E89D">overrides</span><span style="color:#E1E4E8">:                     </span><span style="color:#6A737D"># mirror of gateway pricing.overrides rows, USD per MTok</span></span>
<span class="line"><span style="color:#E1E4E8">  - {</span><span style="color:#85E89D">upstream</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;name&gt;"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">model</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;id&gt;"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">input</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;usd&gt;"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">output</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;usd&gt;"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">cache_read</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;usd&gt;"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">cache_write</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;usd&gt;"</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#85E89D">not_in_client_overrides</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">fast-mode surcharge</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">US-only-inference rate</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">web_search_usd_per_request</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">0.01</span><span style="color:#6A737D">   # list price; the multiplier still applies</span></span>
<span class="line"><span style="color:#85E89D">unknown_model_tier_usd_per_mtok</span><span style="color:#E1E4E8">: {</span><span style="color:#85E89D">input</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">5</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">output</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">25</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#85E89D">gateway_multiplier</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">1.15</span><span style="color:#6A737D">           # what the counters were multiplied by</span></span>
<span class="line"><span style="color:#85E89D">chargeback_markup</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">1.15</span><span style="color:#6A737D">            # what you publish and apply to allocations</span></span>
<span class="line"><span style="color:#85E89D">changed_by</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;name&gt;"</span></span>
<span class="line"><span style="color:#85E89D">changed_at</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"2026-09-30T18:02:11Z"</span></span></code></pre>
<p>Two rules make the file load-bearing. <strong>Change the multiplier and the rate file only at the monthly UTC reset</strong>, lined up with the invoice period; the gateway’s caps reset on UTC calendar boundaries (daily at 00:00, weekly on Monday, monthly on the first). And log every change with a UTC timestamp, even a change you plan to revert, because Step 3 depends on that log.</p>
<h2 id="step-2-fill-the-three-column-worksheet-for-every-lane">Step 2: Fill the three-column worksheet for every lane</h2>
<p>Each lane gets one sheet per month. Pull each column from its own source and convert to list terms before you compare anything.</p>
<table>
<thead>
<tr>
<th>Column</th>
<th>Where it comes from</th>
<th>Convert before comparing</th>
</tr>
</thead>
<tbody>
<tr>
<td>Client estimate</td>
<td>OTel cost metrics from every client (with <code>OTEL_LOG_TOOL_DETAILS=1</code> for agent, skill, plugin and MCP names); Agent SDK <code>total_cost_usd</code> for SDK lanes</td>
<td>Divide events from v2.1.271+ clients by the multiplier; older clients already show costs without it</td>
</tr>
<tr>
<td>Gateway meter</td>
<td>Per-principal period-to-date counters (the gateway’s <code>spend</code> table, or <code>GET /v1/organizations/spend_limits/effective</code>)</td>
<td>Counters are in cents and already multiplied: divide by 100, then by the multiplier in force</td>
</tr>
<tr>
<td>Provider invoice</td>
<td>The provider’s usage reporting for the shared credential; for cloud-provider lanes, the cloud bill</td>
<td>None; this is the anchor</td>
</tr>
</tbody>
</table>
<p>A filled sheet for one lane looks like this. The numbers are illustrative.</p>
<table>
<thead>
<tr>
<th>claude-gateway-eng, Oct 2026 (illustrative)</th>
<th>Client estimate</th>
<th>Gateway meter (÷ 1.15)</th>
<th>Provider invoice</th>
</tr>
</thead>
<tbody>
<tr>
<td>Total at list terms</td>
<td>$38,900</td>
<td>$41,200</td>
<td>$41,650</td>
</tr>
<tr>
<td>Gap to invoice</td>
<td>−$2,750</td>
<td>−$450</td>
<td>—</td>
</tr>
<tr>
<td>Explained by register lines</td>
<td>−$2,410 (lines 1, 2, 4)</td>
<td>−$390 (lines 8, 9)</td>
<td>—</td>
</tr>
<tr>
<td>Unexplained</td>
<td>−$340 (0.8%)</td>
<td>−$60 (0.1%)</td>
<td>—</td>
</tr>
</tbody>
</table>
<p>Read the sheet top to bottom. The client column is expected to drift most, because it depends on which rates reached which session and when. The gateway column should sit close to the invoice. If the gateway column drifts more than the client column, suspect the meter first: an outage window, an unknown model or a truncated export.</p>
<h2 id="step-3-reconcile-totals-at-multiplier-10-and-never-across-a-rate-change">Step 3: Reconcile totals at multiplier 1.0, and never across a rate change</h2>
<p>Reconcile <strong>totals</strong> in list terms, lane by lane. The invoice has no per-developer rows, so there is nothing below the total to reconcile against; per-principal numbers are for allocation, which is Step 4.</p>
<p>The gateway column needs one piece of arithmetic, and it only works under one condition. The spend-limits docs say the meter multiplies each amount by <code>pricing.multiplier</code> before adding it to the counters, so a counter holds marked-up cents. A period-to-date counter divides back to list terms only if one multiplier held for the whole period. The docs don’t cover splitting a counter across a mid-period change; that limit follows from how the counter is built, and a change log can’t repair it, because the counter kept no record of which cents arrived at which rate.</p>
<p>
<em>Illustrative. A mixed-rate counter has no single divisor that recovers list spend. Only the snapshot split lands on the true figure.</em></p>
<p>If a change must land mid-month, do this:</p>
<ol>
<li>Snapshot every principal’s period-to-date spend immediately before the change takes effect.</li>
<li>Apply the change, then snapshot again as soon as it is live, and record both UTC timestamps in the rate-file log.</li>
<li>At month end, divide each segment (start to first snapshot, second snapshot to close) by its own multiplier and add them. Requests that landed between the two snapshots go in the variance register as their own line.</li>
<li>Without those snapshots, mark the lane <strong>mixed-rate and unreconstructible</strong> for the month. Allocate that month from client telemetry, which is event-level and timestamped and can be divided back per event against the change log, and label every charge an estimate.</li>
</ol>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># illustrative: snapshot per-principal to-date spend around a rate change</span></span>
<span class="line"><span style="color:#6A737D"># authenticate the way your gateway's admin API is configured</span></span>
<span class="line"><span style="color:#E1E4E8">ts</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$(</span><span style="color:#B392F0">date</span><span style="color:#79B8FF"> -u</span><span style="color:#9ECBFF"> +%Y%m%dT%H%M%SZ</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#B392F0">curl</span><span style="color:#79B8FF"> -s</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$GATEWAY_URL</span><span style="color:#9ECBFF">/v1/organizations/spend_limits/effective"</span><span style="color:#E1E4E8"> $ADMIN_AUTH </span><span style="color:#79B8FF">\</span></span>
<span class="line"><span style="color:#F97583">  &gt;</span><span style="color:#9ECBFF"> "snapshots/claude-gateway-eng/effective-</span><span style="color:#E1E4E8">$ts</span><span style="color:#9ECBFF">.json"</span></span></code></pre>
<h2 id="step-4-allocate-the-invoice-not-the-meter">Step 4: Allocate the invoice, not the meter</h2>
<p>The amount you charge back is the invoice. The meter decides only the shares. For each principal, take the gateway counter in list terms, roll principals up to teams with your directory mapping, and give each team its share of the invoice total. Then apply the markup you published.</p>
<p><strong>team charge = invoice total × (team gateway spend at list ÷ lane gateway spend at list) × published markup</strong></p>
<table>
<thead>
<tr>
<th>Team (illustrative)</th>
<th>Gateway spend at list</th>
<th>Share</th>
<th>Allocated invoice</th>
<th>Charge at 1.15 markup</th>
</tr>
</thead>
<tbody>
<tr>
<td>payments</td>
<td>$16,480</td>
<td>40%</td>
<td>$16,660</td>
<td>$19,159</td>
</tr>
<tr>
<td>platform</td>
<td>$12,360</td>
<td>30%</td>
<td>$12,495</td>
<td>$14,369</td>
</tr>
<tr>
<td>search</td>
<td>$8,240</td>
<td>20%</td>
<td>$8,330</td>
<td>$9,580</td>
</tr>
<tr>
<td>data</td>
<td>$4,120</td>
<td>10%</td>
<td>$4,165</td>
<td>$4,790</td>
</tr>
<tr>
<td><strong>Lane total</strong></td>
<td><strong>$41,200</strong></td>
<td><strong>100%</strong></td>
<td><strong>$41,650</strong></td>
<td><strong>$47,898</strong></td>
</tr>
</tbody>
</table>
<p>The run asserts one invariant before anything is sent: the sum of team charges equals the invoice total times the markup, within rounding. A run that fails it has billed from a meter somewhere.</p>
<p>
<em>Screenshot: Claude Code Docs, “Claude apps gateway spend limits - Claude Code Docs” (undated), captured Sep 21, 2026.</em></p>
<p>Finer splits come from the client side. Repo, agent, skill, plugin, MCP server and request class all live in client telemetry and hint headers, not in the invoice, and an invoice line can’t be joined to one session (<a href="/intel/vendor-hosted-agent-evidence/">vendor-hosted agent evidence</a> covers why). Show those splits as a breakdown under each team’s charge, labelled <strong>estimate</strong>, scaled to sum to the team’s allocated amount. Never bill a repo line as if the provider had metered it.</p>
<h2 id="step-5-seed-the-variance-register-from-the-vendors-own-divergence-lines">Step 5: Seed the variance register from the vendor’s own divergence lines</h2>
<p>Every documented reason the three columns disagree becomes a numbered register line, with the column it moves and how you detect it. Start from what the docs already admit, so the first month’s gaps have names before they have owners.</p>
<table>
<thead>
<tr>
<th>#</th>
<th>Documented divergence</th>
<th>Column it moves</th>
<th>Detect with</th>
<th>Source</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>A session reports list price until its settings fetch confirms <code>modelPricing</code></td>
<td>Client</td>
<td>Cost events that predate the fetch, per session</td>
<td>Settings reference</td>
</tr>
<tr>
<td>2</td>
<td>Clients below v2.1.242 ignore <code>modelPricing</code>; below v2.1.271 they ignore a multiplier above 1</td>
<td>Client</td>
<td>Client version on every cost event</td>
<td>Settings reference, config</td>
</tr>
<tr>
<td>3</td>
<td>A developer who matches no policy gets no managed settings and stays at list price</td>
<td>Client</td>
<td>Principals in gateway counters with no policy match</td>
<td>Config</td>
</tr>
<tr>
<td>4</td>
<td>Override rows are applied as written, without the fast-mode surcharge or the US-only-inference rate</td>
<td>Client</td>
<td>Fast-mode or US-only usage on the invoice</td>
<td>Settings reference</td>
</tr>
<tr>
<td>5</td>
<td>A rate that only a failover upstream charges stays on the gateway</td>
<td>Client vs gateway</td>
<td>Requests served by the failover upstream</td>
<td>Config</td>
</tr>
<tr>
<td>6</td>
<td>Proxies that strip or rewrite headers push auto mode onto billed client-side classifier requests</td>
<td>Invoice</td>
<td>Classifier request volume after a proxy change</td>
<td>Classifier billing</td>
</tr>
<tr>
<td>7</td>
<td>Cloud-provider usage is invisible to Anthropic’s analytics dashboards and Analytics API</td>
<td>Invoice lives on the cloud bill</td>
<td>Cloud lanes with no vendor-analytics rows</td>
<td>Costs</td>
</tr>
<tr>
<td>8</td>
<td>Client aborts are metered at a floor of about four characters per output token</td>
<td>Gateway</td>
<td>Abort count per principal</td>
<td>Spend limits</td>
</tr>
<tr>
<td>9</td>
<td>Enforcement fails open by default during a Postgres outage, so requests keep flowing while metering is impaired</td>
<td>Gateway (low)</td>
<td>Outage windows; <code>enforcement.fail_closed_on_error</code> value</td>
<td>Spend limits</td>
</tr>
<tr>
<td>10</td>
<td>Unmatched models fall to a $5/$25 per MTok unknown-model tier</td>
<td>Gateway</td>
<td>Model IDs with no override and no list match</td>
<td>Spend limits</td>
</tr>
<tr>
<td>11</td>
<td>Web search is metered at the $0.01 list price, with the multiplier applied</td>
<td>Gateway</td>
<td>Web-search request count</td>
<td>Config</td>
</tr>
<tr>
<td>12</td>
<td>LiteLLM’s legacy <code>/spend/logs</code> stops at the 10,000 most recent rows</td>
<td>Neutral gateway column (low)</td>
<td><code>x-litellm-spend-logs-truncated: true</code> header</td>
<td>LiteLLM</td>
</tr>
</tbody>
</table>
<p>Line 6 comes from the <a href="https://code.claude.com/docs/en/auto-mode-classifier-billing">auto-mode classifier billing page</a>, line 7 from the <a href="https://code.claude.com/docs/en/costs">costs page</a>, which says Claude Code “does not send metrics from your cloud back to Anthropic.” Lines 8 to 10 come from the spend-limits page linked above; line 12 is LiteLLM’s, and Step 7 covers it.</p>
<p>
<em>Every edge of the triangle is a register line waiting to be quantified. The rate file in the middle prices all three corners.</em></p>
<p>Each month, every register line gets a dollar figure or a zero. Whatever is left is the unexplained variance, and a register line that explains the same gap three months running is a configuration fix you haven’t made yet.</p>
<h2 id="step-6-set-a-tolerance-block-the-run-canary-every-change">Step 6: Set a tolerance, block the run, canary every change</h2>
<p>Pick a tolerance per lane before the first run, not after you see the number. A reasonable starting point is the larger of a small percentage of the invoice and a fixed dollar floor, tightened as the register matures. <strong>Any unexplained variance above tolerance blocks the chargeback run for that lane.</strong> No partial send, and no promise to true it up next month.</p>
<p>The block has to fail closed. If the reconciliation job crashes, a column is missing, the invoice hasn’t arrived or a snapshot file is empty, the lane’s run is blocked with the reason written down, and finance gets a late bill instead of a wrong one. The spend caps are a separate brake and keep running in the meantime. Remember that the caps fail open on a Postgres outage unless you set <code>enforcement.fail_closed_on_error: true</code>, which trades developer availability for a hard stop.</p>
<p>After every rate, contract or CLI change, send one canary request through a test principal. Record the client’s <code>/usage</code> figure for that session, the gateway counter delta divided back to list terms, and the provider’s usage line once it posts. Three numbers for one request tell you which column moved before a month of drift does. CLI bumps get the same treatment as the rest of the fleet’s upgrades; <a href="/intel/cli-upgrade-canary-drift/">the CLI upgrade canary</a> owns that replay.</p>
<h2 id="step-7-two-honest-columns-where-theres-no-gateway">Step 7: Two honest columns where there’s no gateway</h2>
<p>Lanes that bill you directly have no independent meter, so the worksheet has two columns: the vendor’s usage report and the invoice. Say so in the sheet’s header. A two-column check can detect a variance in total but rarely attribute it.</p>
<table>
<thead>
<tr>
<th>Lane</th>
<th>Usage column</th>
<th>What the docs give you</th>
<th>What they don’t</th>
</tr>
</thead>
<tbody>
<tr>
<td>GitLab credits (Premium, Ultimate)</td>
<td>19.4 per-event CSV: product, flow type, session, user, namespace, project, credits used, token counts</td>
<td>One row per billable event; background export, download link by email</td>
<td>No price per credit, so price rows with your contract’s rate</td>
</tr>
<tr>
<td>Cursor (Teams, Enterprise)</td>
<td>Usage pools in editor settings and the usage dashboard</td>
<td>Auto bills at the list price of the routed model; a $0.25 per million Cursor Token Rate on third-party models, BYOK included</td>
<td>No documented per-request export of the routed model</td>
</tr>
<tr>
<td>Copilot</td>
<td>Usage billed through Auto</td>
<td>Charged at the model Auto selects, regardless of tier; paid subscribers keep a 10% discount on Auto usage; the model is visible per response in the UI</td>
<td>No documented export or API log of routed models</td>
</tr>
<tr>
<td>LiteLLM (your gateway)</td>
<td><code>/global/spend/report</code> by team, customer, key or user; <code>/spend/logs/v2</code></td>
<td>Per-key, per-user and per-team spend in <code>LiteLLM_SpendLogs</code>; a debugging workflow for “Cost does not match your provider bill?”</td>
<td>No CSV export on the cost-tracking page</td>
</tr>
</tbody>
</table>
<p><a href="https://cursor.com/docs/models-and-pricing">Cursor’s pricing page</a> is the rate-file source for its lane: the Token Rate belongs in the file as its own line, not folded into model prices. GitLab’s export is in credits, and translating credits into tokens is its own discipline; <a href="/intel/credit-vs-token-meter-dialects/">credit versus token meter dialects</a> covers it.</p>
<p>Where you run a vendor-neutral gateway such as LiteLLM, it becomes the middle column. Pull from <code>/spend/logs/v2</code>; the <a href="https://docs.litellm.ai/docs/proxy/cost_tracking">LiteLLM cost-tracking docs</a> say the legacy <code>/spend/logs</code> truncates to the 10,000 most recent matching rows and flags it with a response header, so a reconciliation built on it under-counts in exactly the months that matter.</p>
<h2 id="step-8-publish-the-markup-before-it-goes-live">Step 8: Publish the markup before it goes live</h2>
<p>A markup is a price change for every team, and on the Claude apps gateway it is also a cap change. With an <code>admin:</code> block, the gateway applies the multiplier to spend limits, so developers reach their caps sooner at 1.15 than at 1.0. Publish the new rate, the effective date and the cap impact at least one billing period ahead.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># gateway.yaml (illustrative; see the gateway configuration reference for exact keys)</span></span>
<span class="line"><span style="color:#85E89D">pricing</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  multiplier</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">1.15</span><span style="color:#6A737D">   # greater than 0, at most 10; change only at the monthly UTC reset</span></span></code></pre>
<p>Then check who will see it. Clients below v2.1.271 show costs without the multiplier, and a developer who matches no policy sees list price, so the same session can show two different figures on two laptops. Put the version floor in the announcement, and expect the first disputes to come from that gap rather than from the math.</p>
<h2 id="ai-chargeback-failure-modes-and-the-signal-each-one-leaves">AI chargeback failure modes and the signal each one leaves</h2>
<p><strong>The mixed-rate month.</strong> The multiplier moves on the 12th to cover a contract change, and nobody takes snapshots. Signal: two rate-file versions inside one invoice period. Response: mark the lane unreconstructible, allocate from timestamped client telemetry as an estimate, and move the next change to the reset.</p>
<p><strong>Billing from the brake.</strong> A script exports the gateway’s marked-up counters and sends them as the bill. Signal: charges don’t sum to invoice × markup. Response: the Step 4 invariant, enforced in code before send.</p>
<p><strong>The fail-open gap.</strong> The gateway’s Postgres is down for four hours; requests keep flowing because enforcement fails open, and the provider keeps billing every one of them. Signal: gateway column low by roughly the outage window’s run rate. Response: register line 9, and a decision on <code>enforcement.fail_closed_on_error</code> made on purpose.</p>
<p><strong>The truncated export.</strong> The neutral gateway column comes up short in the busiest month. Signal: the <code>x-litellm-spend-logs-truncated</code> header on any pull. Response: <code>/spend/logs/v2</code>, and a job that fails when the header appears.</p>
<p><strong>The version floor nobody announced.</strong> Half the fleet’s clients predate v2.1.271, <code>/usage</code> shows unmarked costs, and teams dispute charges 15% above what their developers saw. Signal: a client-version histogram weighted by spend. Response: publish the floor with the markup, and treat line 2 as a rollout task.</p>
<p><strong>A split presented as a meter reading.</strong> A team is charged for a repo line that came from telemetry. Signal: a dispute you can’t answer with the invoice. Response: label splits as estimates and scale them to the allocated total.</p>
<h2 id="ai-chargeback-reconciliation-belongs-in-the-fleets-operating-layer">AI chargeback reconciliation belongs in the fleet’s operating layer</h2>
<p>The worksheet, the register and the block are operating-layer work. No vendor’s usage view does them, because each view sees one vendor’s meter and your fleet runs several: a gateway lane, a seat product that bills by credit, a router that bills by whatever model it picked. The reconciliation is where those meters meet, which makes it part of the same layer as the kill switch and the approval queue, the layer a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> runs. The token bill is also only part of what a lane costs you; <a href="/intel/operating-bill-vs-token-bill/">the operating bill versus the token bill</a> prices the rest.</p>
<p>Charge back what you can explain. Everything else waits a month, with its reason written down.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-the-claude-apps-gateway-multiplier-change-what-the-provider-bills">Does the Claude apps gateway multiplier change what the provider bills?</h3>
<p>No. The gateway configuration docs say the multiplier doesn’t change what the upstream provider charges for the requests. It raises what the gateway meters and, with an admin block, how quickly developers reach their spend caps. Reconcile against the invoice in list terms, then apply your published markup to each team’s allocation.</p>
<h3 id="how-do-you-allocate-a-shared-ai-provider-invoice-to-teams">How do you allocate a shared AI provider invoice to teams?</h3>
<p>Start from the invoice total, not a meter total. Divide each principal’s gateway counter back to list terms, roll principals up to teams, and give each team its share of the invoice. Repo, agent and request-class splits come from client telemetry, so show them as labelled estimates under each team’s charge.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a>: 2.1.268 (Sep 10) rates to clients, 2.1.271 (Sep 14) multiplier up to 10, 2.1.273 (Sep 15) hint headers and tool names on cost metrics</li>
<li><a href="https://code.claude.com/docs/en/claude-apps-gateway-spend-limits">Claude apps gateway spend limits</a>: circuit breaker, shared credential, cents counters, UTC resets, abort floor, fail-open metering</li>
<li><a href="https://code.claude.com/docs/en/claude-apps-gateway-config#pricing">Claude apps gateway configuration: pricing</a>: multiplier range, admin-block caps, failover rates, client version floors</li>
<li><a href="https://code.claude.com/docs/en/settings-reference#modelpricing">Claude Code settings reference: modelPricing</a>: list price until the settings fetch lands, surcharges not added to override rows</li>
<li><a href="https://code.claude.com/docs/en/costs">Claude Code costs</a>: cloud-provider usage outside Anthropic’s analytics</li>
<li><a href="https://code.claude.com/docs/en/auto-mode-classifier-billing">Auto mode classifier billing</a>: header-stripping gateways trigger billed classifier fallbacks</li>
<li><a href="https://docs.gitlab.com/releases/19/gitlab-19-4-released/">GitLab 19.4 release notes</a>: per-event credit usage export (Sep 17, 2026)</li>
<li><a href="https://docs.litellm.ai/docs/proxy/cost_tracking">LiteLLM spend tracking</a>: spend by key, user and team; <code>/spend/logs/v2</code>; legacy truncation at 10,000 rows</li>
<li><a href="https://github.blog/changelog/2026-09-14-configure-cost-and-quality-in-copilot-auto-model-selection/">GitHub changelog: Copilot auto model selection tiers</a>: usage charged at the model Auto selects (Sep 14, 2026)</li>
<li><a href="https://cursor.com/docs/models-and-pricing">Cursor models and pricing</a>: Auto at routed-model list price, Cursor Token Rate</li>
</ul>
]]></content:encoded></item><item><title>One Checkout, One Owner: Native CLI Worktrees vs Your Lane Runner</title><link>https://automater.ai/intel/worktree-ownership-one-creator/</link><guid>https://automater.ai/intel/worktree-ownership-one-creator/</guid><description>Codex, Claude Code and Cursor now create git worktrees for AI agents. Give each checkout one owner, lock what your runner owns and prove it with a decoy.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>A lane runner’s teardown goes to remove the checkout it created and finds a second one nested inside it: a <code>.claude/worktrees/</code> directory on branch <code>worktree-fix-auth</code>, because the lane’s launch line still carried <code>claude --worktree fix-auth</code> from the week someone tried the native flag. Two creators, one lane. Each did what it was built to do, and the teardown now has to guess which checkout is its own.</p>
<p>Git worktree management for AI agents used to have one owner per repo: whoever typed <code>git worktree add</code>. In September 2026 three AI coding agents create checkouts on their own. Claude Code does it with <code>--worktree</code>, Cursor does it from its Agents Window, and since Sep 9 Codex CLI does it with <code>--worktree</code> and <code>/worktree</code>. Each binds sessions to checkouts, resumes into them and sweeps them on its own clock, which is also a fair description of your runner.</p>
<p>This is the runbook for giving every checkout exactly one owner. By Tuesday you have an ownership table per repo, one owner per root directory, one creator per lane, a lock carrying your reason on every checkout your runner made, and a decoy test that proves no native sweep touches a checkout it did not create.</p>
<p>Agents act, and they act inside checkouts. One removed under a running agent takes its uncommitted work with it; a session resumed into the wrong one commits a lane’s change on another lane’s base. Neither arrives as an error.</p>
<h2 id="sep-9-codex-cli-becomes-the-third-native-worktree-creator">Sep 9: Codex CLI becomes the third native worktree creator</h2>
<p>Codex CLI 0.154.0 shipped on Sep 9, 2026, and one line in its <a href="https://github.com/openai/codex/releases/tag/rust-v0.154.0">release notes</a> changes who creates checkouts in your repos: “Experimental worktree support lets you create isolated checkouts for new or forked sessions using <code>--worktree</code> or <code>/worktree</code>, then browse and resume them.”</p>
<p>
<em>Screenshot: GitHub, “Release 0.154.0 · openai/codex · GitHub” (Sep 9, 2026), captured Sep 21, 2026.</em></p>
<p>The notes stop there, and the pull requests say more. PR #42196 creates detached, Desktop-compatible worktrees from <code>HEAD</code> or an explicit base. <a href="https://github.com/openai/codex/pull/42652">PR #42652</a> adds an experimental <code>worktrees</code> feature and the <code>--worktree</code> flag to <code>codex exec</code>, binds each checkout to its new thread, and does one more thing: “Share the configured worktree pool with Desktop while leaving automatic cleanup disabled for CLI allocations.” <a href="https://github.com/openai/codex/pull/43069">PR #43069</a> brings the flag to interactive sessions and <code>codex fork --worktree</code>, binds ownership before the first turn, and keeps unbound checkouts after a failed startup with recovery instructions for a human.</p>
<p><a href="https://github.com/openai/codex/pull/43279">PR #43279</a> is the one to read twice. With <code>worktrees</code> enabled, Codex will “include corresponding directories across linked checkouts in resume and fork pickers and <code>--last</code> lookup”. A resume can land in a checkout other than the one you are standing in. <a href="https://github.com/openai/codex/releases/tag/rust-v0.155.0">Codex 0.155.0</a> followed on Sep 17 with “worktree ownership details and confirmed deletion of clean managed worktrees” in the agents overview.</p>
<p>The pool itself is described in the <a href="https://learn.chatgpt.com/docs/environments/git-worktrees">Codex app’s worktree docs</a>. Checkouts live in <code>$CODEX_HOME/worktrees</code> unless someone changes the “Worktree root” setting, sit in detached HEAD, and follow the conversation: “Each chat keeps the same associated worktree over time.” The app keeps the 15 most recent, snapshots before it deletes, and deletes when you archive the chat. CLI allocations are exempt from that sweep, so nothing removes them on a timer.</p>
<p>Claude Code has been a creator for longer. Boris Cherny’s February post introduced <code>claude --worktree</code>, and the <a href="https://code.claude.com/docs/en/worktrees">worktree docs</a> have grown a rulebook since: a marker in the git metadata of every worktree Claude creates, a lock while an agent runs, and name-reuse rules that changed in July.</p>
<p><a href="https://cursor.com/docs/configuration/worktrees">Cursor’s worktree docs</a>, undated, with a cleanup section for 3.5 and later, sweep its root every 6 hours down to 25 worktrees per machine, with every workspace counted against the same limit. They say plainly that worktrees “created outside the manager (for example, worktrees created by <code>/worktree</code> skills or <code>git worktree add</code>) are eligible for deletion”. They do not name the root’s path, or say whether dirty or locked checkouts are spared.</p>
<aside class="social-embed" data-x-embed data-post-id="2025007394967957720">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">February 21, 2026 — Claude Code's lead introduces claude --worktree for running parallel Claude Code sessions in isolated git worktrees inside one repository.</p>
  <a class="social-embed__source" href="https://x.com/bcherny/status/2025007394967957720" target="_blank" rel="noopener noreferrer">View Boris Cherny's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<p>One thing in these rules is reassuring: each native sweep is scoped by its own root or its own marker, and no doc describes a vendor reaching into a directory it doesn’t manage. The danger is narrower and more ordinary: a checkout placed inside a vendor’s root, or created twice because two creators each believed the lane was theirs.</p>
<p>The live guides assume one manager per repo, and inside that assumption they hold. <a href="/intel/interruptible-agent-coordinators/">Interruptible coordinators</a> gives each worker a worktree under a single coordinator, <a href="/intel/subagent-orchestration/">subagent orchestration</a> uses worktrees for parallelism, <a href="/intel/agentic-cicd/">agentic CI/CD</a> removes them after merge and puts TTLs on branches, and <a href="/intel/claude-code-power-guide/">the Claude Code power guide</a> runs one per task. <a href="/intel/claude-code-vs-cursor/">Claude Code vs Cursor</a> goes further and says the two tools’ worktrees coexist with ordinary branch discipline. That is true of branch names and not of placement, because Cursor counts every worktree under its root as a candidate for deletion.</p>
<h2 id="step-1-build-a-git-worktree-ownership-table-for-each-repo">Step 1: Build a git worktree ownership table for each repo</h2>
<p>One table per repo, one column per native manager you actually run, plus your runner. Fill it from the docs first, then from your machines. Every cell that reads “not documented” or “check” becomes a test in step 5.</p>
<table>
<thead>
<tr>
<th>Question</th>
<th>Claude Code</th>
<th>Codex CLI 0.154+</th>
<th>Cursor 3.5+</th>
<th>Your runner</th>
</tr>
</thead>
<tbody>
<tr>
<td>What creates a checkout</td>
<td><code>--worktree</code> / <code>-w</code>; <code>isolation: "worktree"</code> subagents</td>
<td><code>--worktree</code>, <code>/worktree</code>, <code>fork --worktree</code> (experimental <code>worktrees</code> feature)</td>
<td>Agents Window; <code>/worktree</code> and <code>/best-of-n</code> skills in the IDE</td>
<td><code>runner create</code>, or its hook</td>
</tr>
<tr>
<td>Root path</td>
<td><code>.claude/worktrees/&lt;name&gt;/</code> at the repo root</td>
<td><code>$CODEX_HOME/worktrees</code> (app setting “Worktree root”)</td>
<td>not documented; read it off your machine</td>
<td>yours: outside every root to the left, outside any repo</td>
</tr>
<tr>
<td>Branch scheme</td>
<td><code>worktree-&lt;name&gt;</code></td>
<td>detached HEAD</td>
<td>check on your machine</td>
<td><code>lane/&lt;lane&gt;/&lt;run-id&gt;</code></td>
</tr>
<tr>
<td>Reuse by name</td>
<td>on the default fresh base, resets to the default branch only if clean, still on its branch and without commits of its own (or PR merged and remote branch deleted); otherwise reopens at the old tip</td>
<td>checkout bound to one owner thread</td>
<td>check on your machine</td>
<td>never; a new run gets a new checkout</td>
</tr>
<tr>
<td>Resume discovery</td>
<td>resuming returns the session to its worktree</td>
<td>with <code>worktrees</code> on, linked checkouts join the pickers and <code>--last</code></td>
<td>check on your machine</td>
<td>lane registry</td>
</tr>
<tr>
<td>Lock while running</td>
<td><code>git worktree lock</code> while an agent runs; a <code>-p</code> lock stays until a later stale-lock sweep</td>
<td>not documented</td>
<td>not documented</td>
<td><code>runner:&lt;lane&gt;:&lt;run-id&gt;</code>, released in teardown</td>
</tr>
<tr>
<td>Cleanup trigger</td>
<td>sweep of subagent and background worktrees older than <code>cleanupPeriodDays</code></td>
<td>none for CLI allocations; app sweeps on archive or past 15; clean-only delete in the 0.155 overview</td>
<td>every 6 h, down to 25 per machine</td>
<td>teardown only</td>
</tr>
<tr>
<td>Spared by cleanup</td>
<td>changed or untracked files, unpushed commits, <code>--worktree</code> sessions you haven’t backgrounded</td>
<td>app: pinned or in-progress chats, permanent worktrees; snapshot first</td>
<td>dirty or locked: not documented</td>
<td>anything not stopped, exported and clean</td>
</tr>
<tr>
<td>Touches checkouts it didn’t create</td>
<td>no: keeps any worktree without its marker (since 2.1.246)</td>
<td>not documented</td>
<td>yes, inside its root: “eligible for deletion”</td>
<td>never</td>
</tr>
</tbody>
</table>
<p>Three things fall out of a filled table. Two native sweeps count rather than age, so whether your checkout survives depends on what else was created that day. Claude’s sweep is the only one that documents keeping checkouts it didn’t make, and it learned that the hard way: the <a href="https://code.claude.com/docs/en/changelog">2.1.246 changelog</a> (Aug 25, 2026) fixed a background sweep that removed worktrees users had created themselves under <code>.claude/worktrees/</code> when an old background-session record pointed at them. And Codex CLI checkouts are removed by nobody on a schedule, so they pile up unless your runner or a human in the 0.155 overview removes them.</p>
<p>How long anything should be kept is <a href="/intel/agent-session-cleanup-retention/">the cleanup audit’s</a> job, not this one’s.</p>
<p>
<em>Two sweeps count and two don’t. A counted root evicts by everyone’s activity, including the checkouts you parked there.</em></p>
<h2 id="step-2-give-every-root-directory-exactly-one-owner">Step 2: Give every root directory exactly one owner</h2>
<p>A root is the directory a manager creates checkouts in and sweeps. The rule is short: your runner’s root sits outside <code>.claude/worktrees/</code>, outside <code>$CODEX_HOME/worktrees</code>, outside Cursor’s root, and outside any git repository. The last part is Claude’s requirement as much as yours: its docs refuse a hook-created directory inside a repository and tell you to “have the hook create its directories outside any repository”.</p>
<p>Cursor’s root is the awkward one, because the docs never name it. On each machine, create one worktree from the Agents Window, read its path from <code>git worktree list</code>, and write it into the table. Check Codex’s “Worktree root” setting on the same pass, because anyone can move it. If it ever points at your runner’s directory, your checkouts are sitting in Codex’s pool, and the app’s docs describe its rules for Codex-managed worktrees only.</p>
<p>Put the check in the runner rather than in a wiki page. An unknown root fails closed.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># Illustrative runner preflight: refuse a root that overlaps any native manager's root</span></span>
<span class="line"><span style="color:#E1E4E8">RUNNER_ROOT</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">/srv/lanes</span><span style="color:#6A737D">                   # outside any repository</span></span>
<span class="line"><span style="color:#E1E4E8">FORBIDDEN</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">(</span></span>
<span class="line"><span style="color:#9ECBFF">  "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">/.claude/worktrees"</span></span>
<span class="line"><span style="color:#9ECBFF">  "</span><span style="color:#E1E4E8">$CODEX_HOME</span><span style="color:#9ECBFF">/worktrees"</span><span style="color:#6A737D">                # CODEX_HOME as your machines set it</span></span>
<span class="line"><span style="color:#9ECBFF">  "</span><span style="color:#E1E4E8">$CURSOR_WORKTREE_ROOT</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">                # read off the machine in step 2</span></span>
<span class="line"><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">for</span><span style="color:#E1E4E8"> f </span><span style="color:#F97583">in</span><span style="color:#9ECBFF"> "${</span><span style="color:#E1E4E8">FORBIDDEN</span><span style="color:#9ECBFF">[</span><span style="color:#F97583">@</span><span style="color:#9ECBFF">]}"</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#E1E4E8">  [ </span><span style="color:#F97583">-n</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$f</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8"> ] </span><span style="color:#F97583">||</span><span style="color:#E1E4E8"> { </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "native root unknown; fill the table first"</span><span style="color:#F97583"> &gt;&amp;2</span><span style="color:#E1E4E8">; </span><span style="color:#79B8FF">exit</span><span style="color:#79B8FF"> 1</span><span style="color:#E1E4E8">; }</span></span>
<span class="line"><span style="color:#F97583">  case</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$RUNNER_ROOT</span><span style="color:#9ECBFF">/"</span><span style="color:#F97583"> in</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$f</span><span style="color:#9ECBFF">"</span><span style="color:#DBEDFF">/</span><span style="color:#F97583">*</span><span style="color:#F97583">)</span><span style="color:#79B8FF"> echo</span><span style="color:#9ECBFF"> "runner root is inside </span><span style="color:#E1E4E8">$f</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> &gt;&amp;2</span><span style="color:#E1E4E8">; </span><span style="color:#79B8FF">exit</span><span style="color:#79B8FF"> 1</span><span style="color:#E1E4E8">;; </span><span style="color:#F97583">esac</span></span>
<span class="line"><span style="color:#F97583">  case</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$f</span><span style="color:#9ECBFF">/"</span><span style="color:#F97583"> in</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$RUNNER_ROOT</span><span style="color:#9ECBFF">"</span><span style="color:#DBEDFF">/</span><span style="color:#F97583">*</span><span style="color:#F97583">)</span><span style="color:#79B8FF"> echo</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$f</span><span style="color:#9ECBFF"> is inside the runner root"</span><span style="color:#F97583"> &gt;&amp;2</span><span style="color:#E1E4E8">; </span><span style="color:#79B8FF">exit</span><span style="color:#79B8FF"> 1</span><span style="color:#E1E4E8">;; </span><span style="color:#F97583">esac</span></span>
<span class="line"><span style="color:#F97583">done</span></span>
<span class="line"><span style="color:#F97583">if</span><span style="color:#B392F0"> git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$RUNNER_ROOT</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> rev-parse</span><span style="color:#79B8FF"> --git-dir</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF">/dev/null</span><span style="color:#F97583"> 2&gt;&amp;1</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">then</span></span>
<span class="line"><span style="color:#79B8FF">  echo</span><span style="color:#9ECBFF"> "runner root is inside a git repository"</span><span style="color:#F97583"> &gt;&amp;2</span><span style="color:#E1E4E8">; </span><span style="color:#79B8FF">exit</span><span style="color:#79B8FF"> 1</span></span>
<span class="line"><span style="color:#F97583">fi</span></span></code></pre>
<p>Both directions matter. A runner root inside a vendor root is the obvious mistake, and a vendor root moved inside yours is the one nobody notices until a sweep runs.</p>
<p>
<em>Each sweep stays inside its own box. The runner’s job is to never put a checkout in someone else’s.</em></p>
<h2 id="step-3-allow-one-creator-per-lane-the-runner-or-a-hook-that-calls-it">Step 3: Allow one creator per lane, the runner or a hook that calls it</h2>
<p>Pick one of two patterns per lane, write it into the lane manifest, and never run both.</p>
<p><strong>Pattern A: the runner creates, the CLI runs plainly inside.</strong> The runner makes the checkout, locks it (step 4), changes into it and launches the agent with no worktree flag at all.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># Illustrative lane launch, pattern A</span></span>
<span class="line"><span style="color:#E1E4E8">wt</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">$RUNNER_ROOT</span><span style="color:#9ECBFF">/</span><span style="color:#E1E4E8">$LANE</span><span style="color:#9ECBFF">/</span><span style="color:#E1E4E8">$RUN_ID</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> add</span><span style="color:#79B8FF"> -b</span><span style="color:#9ECBFF"> "lane/</span><span style="color:#E1E4E8">$LANE</span><span style="color:#9ECBFF">/</span><span style="color:#E1E4E8">$RUN_ID</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$BASE</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> lock</span><span style="color:#79B8FF"> --reason</span><span style="color:#9ECBFF"> "runner:</span><span style="color:#E1E4E8">$LANE</span><span style="color:#9ECBFF">:</span><span style="color:#E1E4E8">$RUN_ID</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#79B8FF">cd</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#B392F0">claude</span><span style="color:#79B8FF"> -p</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$TASK</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">            # never -w or --worktree on a runner lane</span></span>
<span class="line"><span style="color:#6A737D"># codex exec "$TASK"         # never --worktree; worktrees feature stays off</span></span></code></pre>
<p>Keep Codex’s experimental <code>worktrees</code> feature disabled in the config your runner lanes load. PR #42652 rejects <code>--worktree</code> when the feature is off, so a stray flag stops the launch instead of creating a second checkout, and PR #43279’s cross-checkout <code>--last</code> lookup only applies with the feature on.</p>
<p>Then lint every lane before launch. Fail on <code>--worktree</code> or <code>-w</code> in a Claude or Codex command line, on <code>/worktree</code> in a prompt file, and on <code>isolation: "worktree"</code> in any subagent definition the lane loads. That last one is easy to miss: Claude’s hooks reference lists a finishing <code>isolation: "worktree"</code> subagent among the worktree-removal triggers, so those subagents get checkouts of their own. Without the hook below, those checkouts are Claude’s, under Claude’s sweep.</p>
<p><strong>Pattern B: the vendor flag delegates to the runner.</strong> When a lane needs Claude’s own <code>--worktree</code> flow, for a person working interactively say, hand creation to the runner through Claude’s hook pair. The worktree docs say a <code>WorktreeCreate</code> hook replaces “the default <code>git worktree</code> logic entirely”. The <a href="https://code.claude.com/docs/en/hooks">hooks reference</a> sets the contract: the hook “must return the path” (a command hook prints it on stdout, an HTTP hook returns <code>hookSpecificOutput.worktreePath</code>), and “any non-zero exit code from <code>WorktreeCreate</code> aborts worktree creation”. <code>WorktreeRemove</code> fires when someone exits a <code>--worktree</code> session and chooses removal, when an <code>isolation: "worktree"</code> subagent finishes, and when a background session whose worktree the hook created is deleted.</p>
<p>Illustrative <code>.claude/settings.json</code> fragment for a pattern-B lane (check the hooks reference for matcher and input fields):</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span></span>
<span class="line"><span style="color:#79B8FF">  "hooks"</span><span style="color:#E1E4E8">: {</span></span>
<span class="line"><span style="color:#79B8FF">    "WorktreeCreate"</span><span style="color:#E1E4E8">: [{ </span><span style="color:#79B8FF">"hooks"</span><span style="color:#E1E4E8">: [{ </span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"command"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"command"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"/srv/lanes/bin/runner-hook create"</span><span style="color:#E1E4E8"> }] }],</span></span>
<span class="line"><span style="color:#79B8FF">    "WorktreeRemove"</span><span style="color:#E1E4E8">: [{ </span><span style="color:#79B8FF">"hooks"</span><span style="color:#E1E4E8">: [{ </span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"command"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"command"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"/srv/lanes/bin/runner-hook remove"</span><span style="color:#E1E4E8"> }] }]</span></span>
<span class="line"><span style="color:#E1E4E8">  }</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span></code></pre>
<p>The runner’s hook does what pattern A does: create outside any repo, lock with the runner’s reason, register the lane, print the path. On any failure it exits non-zero, which aborts creation rather than falling back to Claude’s default root. That is the fail-closed direction you want. The hook-created checkout carries no Claude marker, so Claude’s sweep keeps it, and cleanup stays in the runner’s teardown.</p>
<p>The hook is a guardrail with a known edge. It governs Claude only; Codex’s worktree PRs and app docs describe no creation hook, so Codex lanes run pattern A. And it only works if the lane loads the settings file that carries it; skip that file and Claude’s default logic creates the checkout in its own root. <a href="/intel/headless-lane-reproducibility/">Prove a headless lane loads what you declared</a> before you rely on it, and keep step 2’s placement rule and step 5’s decoys as the wall behind the hook.</p>
<h2 id="step-4-lock-what-your-runner-owns-and-unlock-it-only-in-teardown">Step 4: Lock what your runner owns, and unlock it only in teardown</h2>
<p>Lock right after create, with a reason that names the owner: <code>git worktree lock --reason "runner:&lt;lane&gt;:&lt;run-id&gt;" &lt;path&gt;</code>. Claude’s docs back this directly: “The sweep never releases a lock you set yourself with <code>git worktree lock</code>.” Git itself keeps a locked worktree out of <code>git worktree prune</code> and makes a plain <code>git worktree remove</code> refuse it. Neither Codex nor Cursor documents what its cleanup does with a locked checkout, so for those two the lock is a request, and step 5 finds out whether it is honored.</p>
<p>
<em>Screenshot: Claude Code Docs, “Run parallel sessions with worktrees - Claude Code Docs” (undated), captured Sep 21, 2026.</em></p>
<p>The teardown is the only place a lock comes off, and it has one more job. If a lane ever runs <code>claude -p --worktree</code>, Claude’s docs say “Non-interactive runs with <code>-p</code> have no exit prompt, so Claude doesn’t clean up their worktrees, and Claude Code leaves the lock it took on each one at creation in place until a later session’s stale-lock sweep releases it.” On a headless box that later session may never come. Clear the lock yourself, on your own lane’s checkout, once the lane’s Claude process has exited.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># Illustrative teardown; the order is the point</span></span>
<span class="line"><span style="color:#79B8FF">kill</span><span style="color:#79B8FF"> -0</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$AGENT_PID</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> 2&gt;</span><span style="color:#9ECBFF">/dev/null</span><span style="color:#E1E4E8"> &amp;&amp; { </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "agent still running; not tearing down"</span><span style="color:#F97583"> &gt;&amp;2</span><span style="color:#E1E4E8">; </span><span style="color:#79B8FF">exit</span><span style="color:#79B8FF"> 1</span><span style="color:#E1E4E8">; }</span></span>
<span class="line"><span style="color:#B392F0">export_ignored</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">      # logs and transcripts first; see the cleanup audit</span></span>
<span class="line"><span style="color:#E1E4E8">lock</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$(</span><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> list</span><span style="color:#79B8FF"> --porcelain</span><span style="color:#F97583"> |</span></span>
<span class="line"><span style="color:#B392F0">  awk</span><span style="color:#79B8FF"> -v</span><span style="color:#9ECBFF"> p="</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> '$1=="worktree"{cur=substr($0,10)} cur==p &amp;&amp; $1=="locked"{print "L:" substr($0,8)}'</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">case</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$lock</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> in</span></span>
<span class="line"><span style="color:#9ECBFF">  "L:runner:</span><span style="color:#E1E4E8">$LANE</span><span style="color:#9ECBFF">:</span><span style="color:#E1E4E8">$RUN_ID</span><span style="color:#9ECBFF">"</span><span style="color:#F97583">)</span><span style="color:#B392F0"> git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> unlock</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8"> ;;</span></span>
<span class="line"><span style="color:#DBEDFF">  L:</span><span style="color:#F97583">*</span><span style="color:#F97583">)</span><span style="color:#E1E4E8"> [ </span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">$LANE_RAN_CLAUDE_P_WORKTREE</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> =</span><span style="color:#E1E4E8"> yes ] </span><span style="color:#F97583">||</span><span style="color:#E1E4E8"> { </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "foreign lock on </span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">: </span><span style="color:#E1E4E8">$lock</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> &gt;&amp;2</span><span style="color:#E1E4E8">; </span><span style="color:#79B8FF">exit</span><span style="color:#79B8FF"> 1</span><span style="color:#E1E4E8">; }</span></span>
<span class="line"><span style="color:#B392F0">       git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> unlock</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8"> ;;   </span><span style="color:#6A737D"># Claude's leftover -p lock, process gone</span></span>
<span class="line"><span style="color:#9ECBFF">  ""</span><span style="color:#F97583">)</span><span style="color:#79B8FF"> echo</span><span style="color:#9ECBFF"> "warning: </span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF"> was not locked"</span><span style="color:#F97583"> &gt;&amp;2</span><span style="color:#E1E4E8"> ;;</span></span>
<span class="line"><span style="color:#F97583">esac</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> remove</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">             # never --force</span></span></code></pre>
<p>A lock is a guardrail, not a wall. Anyone who passes <code>--force</code> twice to <code>git worktree remove</code> gets past it, and two of the three vendors don’t say whether their cleanup checks. The walls are placement (step 2) and export before delete, which the cleanup audit owns.</p>
<h2 id="step-5-run-the-collision-checks-and-the-decoy-test">Step 5: Run the collision checks and the decoy test</h2>
<p>Run these on a test machine with the same CLI versions as your lanes.</p>
<p><strong>Check 1: <code>--last</code> resumes the right checkout.</strong> On the test machine only, enable Codex’s <code>worktrees</code> feature, start one Codex session in checkout A and another in checkout B, then run <code>codex resume --last</code> from a runner worktree. Pass: it opens the session tied to the checkout you are in, or none. Fail: it opens a session bound to a different checkout. On runner lanes the fix is the feature staying off, which the step 3 lint enforces.</p>
<p><strong>Check 2: Claude’s name reuse, both ways.</strong> Run <code>claude --worktree probe</code>, exit with the tree clean and no commits, keep the worktree when asked, then reuse the name and confirm it reset to the default branch. Commit once in it without merging, exit, reuse the name again, and confirm it reopened at the old tip. Both behaviors are documented; the reset arrived in 2.1.208 on Jul 14, 2026, and before that every reuse reopened the old tip. A runner that reuses lane names is silently betting on one of them.</p>
<p><strong>Check 3: the decoys.</strong> One question: does any sweep touch a checkout outside its own root, or one without its own marker? Plant clean decoys and leave them through at least one full cycle of every sweep: Cursor’s 6-hour pass, a Codex app archive, and Claude’s retention period.</p>
<table>
<thead>
<tr>
<th>Decoy</th>
<th>Where</th>
<th>What the docs predict</th>
<th>Pass</th>
</tr>
</thead>
<tbody>
<tr>
<td>Two plain decoys</td>
<td>runner root</td>
<td>no native sweep is scoped there</td>
<td>both present</td>
</tr>
<tr>
<td>Foreign decoy (<code>git worktree add</code>)</td>
<td><code>.claude/worktrees/</code></td>
<td>kept: no Claude marker, since 2.1.246</td>
<td>present</td>
</tr>
<tr>
<td>Foreign decoy</td>
<td><code>$CODEX_HOME/worktrees</code></td>
<td>docs cover Codex-managed worktrees only</td>
<td>present, or record what removed it</td>
</tr>
<tr>
<td>Foreign decoy</td>
<td>Cursor’s root</td>
<td>eligible for deletion</td>
<td>record; this is the documented risk</td>
</tr>
<tr>
<td>Locked foreign decoy</td>
<td>Cursor’s root</td>
<td>not documented</td>
<td>record whether the lock spared it</td>
</tr>
</tbody>
</table>
<p>Keep decoys clean. An untracked sentinel file makes a checkout dirty, and Claude keeps dirty checkouts for that reason alone, which would hide the marker result. Record decoys in a register outside the tree instead.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># Illustrative decoy plant and check; the register lives outside every root</span></span>
<span class="line"><span style="color:#B392F0">plant_decoy</span><span style="color:#E1E4E8">() {   </span><span style="color:#6A737D"># plant_decoy &lt;path&gt; &lt;label&gt; [lock]</span></span>
<span class="line"><span style="color:#B392F0">  git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> add</span><span style="color:#79B8FF"> --detach</span><span style="color:#9ECBFF"> "</span><span style="color:#79B8FF">$1</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> HEAD</span><span style="color:#F97583"> ||</span><span style="color:#F97583"> return</span><span style="color:#79B8FF"> 1</span></span>
<span class="line"><span style="color:#E1E4E8">  [ </span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">$3</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> =</span><span style="color:#E1E4E8"> lock ] &amp;&amp; </span><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> lock</span><span style="color:#79B8FF"> --reason</span><span style="color:#9ECBFF"> "decoy:</span><span style="color:#79B8FF">$2</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "</span><span style="color:#79B8FF">$1</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#79B8FF">  printf</span><span style="color:#9ECBFF"> '%s\t%s\t%s\n'</span><span style="color:#9ECBFF"> "</span><span style="color:#79B8FF">$1</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "</span><span style="color:#79B8FF">$2</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "$(</span><span style="color:#B392F0">date</span><span style="color:#79B8FF"> -u</span><span style="color:#9ECBFF"> +%FT%TZ)"</span><span style="color:#F97583"> &gt;&gt;</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$DECOY_REGISTER</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#B392F0">check_decoys</span><span style="color:#E1E4E8">() {</span></span>
<span class="line"><span style="color:#F97583">  while</span><span style="color:#9ECBFF"> IFS</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">$'</span><span style="color:#79B8FF">\t</span><span style="color:#9ECBFF">'</span><span style="color:#9ECBFF"> read</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> path</span><span style="color:#9ECBFF"> label</span><span style="color:#9ECBFF"> planted</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> [ </span><span style="color:#F97583">-d</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$path</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8"> ]; </span><span style="color:#F97583">then</span><span style="color:#79B8FF"> echo</span><span style="color:#9ECBFF"> "present  </span><span style="color:#E1E4E8">$label</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">else</span><span style="color:#79B8FF"> echo</span><span style="color:#9ECBFF"> "GONE     </span><span style="color:#E1E4E8">$label</span><span style="color:#9ECBFF"> (planted </span><span style="color:#E1E4E8">$planted</span><span style="color:#9ECBFF">)"</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">fi</span></span>
<span class="line"><span style="color:#F97583">  done</span><span style="color:#F97583"> &lt;</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$DECOY_REGISTER</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span></code></pre>
<p>The Cursor rows only mean something on a machine that actually passes 25 worktrees, so check the count there before you read the result. A missing decoy anywhere else is the signal this whole piece is about. Wire the check into <a href="/intel/cli-upgrade-canary-drift/">the upgrade canary</a>, because these rules moved four times between Jul 14 and Sep 17, and run it weekly between upgrades.</p>
<h2 id="worktree-collisions-between-native-clis-and-your-runner-and-the-signal-for-each">Worktree collisions between native CLIs and your runner, and the signal for each</h2>
<p><strong>A second checkout nested in a lane.</strong> Signal: <code>git worktree list</code> shows a path under a runner checkout, or a <code>worktree-&lt;name&gt;</code> branch appears on a runner lane. Cause: a pattern-A lane that also passed a native flag. Fix: the step 3 lint, failing the launch.</p>
<p><strong>A resume in the wrong checkout.</strong> Signal: the session’s working directory differs from the checkout the lane registry assigned, or commits land on a detached HEAD that belongs to another lane. Cause: <code>--last</code> looking across linked checkouts with <code>worktrees</code> on. Fix: feature off on runner lanes; resume by explicit session.</p>
<p><strong>An old tip where you expected a fresh base.</strong> Signal: a new run starts with commits it didn’t make. Cause: Claude name reuse on a checkout holding unmerged commits, which reopens at its old tip by design. Fix: never reuse names across runs; put the run ID in the name.</p>
<p><strong>A checkout evicted by someone else’s busy day.</strong> Signal: a decoy or a real checkout inside Cursor’s root disappears after the machine passes 25 worktrees. Cause: placement inside a counted root. Fix: step 2.</p>
<p><strong>A lock nobody will release.</strong> Signal: <code>git worktree list --porcelain</code> shows <code>locked</code> on a checkout whose process is gone, with a reason that isn’t yours. Cause: a <code>-p</code> run’s lock waiting for a stale-lock sweep that no later session will run. Fix: the teardown clears it after checking the process.</p>
<p><strong>Leftovers with no owner.</strong> Signal: a checkout in the Codex pool with no owner in the 0.155 overview, or a detached checkout holding commits that no branch reaches. Cause: a startup failure (PR #43069 keeps unbound checkouts and prints recovery steps), or work finished on a detached HEAD. Fix: recover it by hand and name an owner before the next sweep decides for you.</p>
<h2 id="worktree-ownership-belongs-to-the-layer-that-runs-the-fleet">Worktree ownership belongs to the layer that runs the fleet</h2>
<p>No vendor here is wrong about its own checkouts. Each knows its root, its marker and its sweep, and none of them knows your runner exists. The ownership table, the root preflight, the launch lint, the locks and the decoy register can’t live in any one vendor’s settings. They live one layer up, where a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> keeps every CLI’s sessions and checkouts on one board.</p>
<p>One checkout, one owner is a small rule. It only holds if you can name the owner of every checkout on the machine, and the decoy is how you find out whether you can.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-cursor-delete-git-worktrees-it-didnt-create">Does Cursor delete git worktrees it didn’t create?</h3>
<p>Yes, inside its own root. Cursor’s docs say worktrees created outside the manager, including ones made with <code>git worktree add</code>, are eligible for deletion. The sweep runs every 6 hours down to 25 per machine, and the docs don’t say whether dirty or locked checkouts survive. Keep your runner’s checkouts outside Cursor’s root.</p>
<h3 id="can-i-use-claude-codes---worktree-flag-with-my-own-lane-runner">Can I use Claude Code’s <code>--worktree</code> flag with my own lane runner?</h3>
<p>Only through a hook, because each lane needs one creator. If your runner creates the checkout, launch Claude with no worktree flag, or you get a second checkout nested in the lane. When a lane needs <code>--worktree</code>, a <code>WorktreeCreate</code> hook that calls your runner replaces Claude’s default logic, and any non-zero exit aborts creation.</p>
<h3 id="does-git-worktree-lock-protect-a-worktree-from-ai-agent-cleanup">Does git worktree lock protect a worktree from AI agent cleanup?</h3>
<p>Partly. Git keeps a locked worktree out of <code>git worktree prune</code>, a plain <code>git worktree remove</code> refuses it, and Claude’s sweep never releases a lock you set yourself. Passing <code>--force</code> twice still gets past it, and Codex and Cursor don’t document how their cleanup treats locks, so confirm with a decoy.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://github.com/openai/codex/releases/tag/rust-v0.154.0">Codex 0.154.0 release notes</a> — Sep 9, 2026; experimental worktree support</li>
<li><a href="https://github.com/openai/codex/pull/42652">Codex PR #42652</a> — Sep 4, 2026; <code>worktrees</code> feature, shared pool, CLI allocations exempt from cleanup</li>
<li><a href="https://github.com/openai/codex/pull/43069">Codex PR #43069</a> — Sep 5, 2026; interactive sessions, forks, unbound checkouts kept</li>
<li><a href="https://github.com/openai/codex/pull/43279">Codex PR #43279</a> — Sep 6, 2026; linked checkouts in pickers and <code>--last</code></li>
<li><a href="https://github.com/openai/codex/releases/tag/rust-v0.155.0">Codex 0.155.0 release notes</a> — Sep 17, 2026; ownership details, clean-only delete</li>
<li><a href="https://learn.chatgpt.com/docs/environments/git-worktrees">Worktrees, ChatGPT Learn</a> — Codex app pool, keeps 15, snapshot first</li>
<li><a href="https://code.claude.com/docs/en/worktrees">Claude Code worktrees docs</a> — marker, lock, reuse rules, <code>WorktreeCreate</code></li>
<li><a href="https://code.claude.com/docs/en/hooks">Claude Code hooks reference</a> — <code>WorktreeCreate</code> and <code>WorktreeRemove</code> contract</li>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a> — 2.1.208 (Jul 14, 2026); 2.1.246 (Aug 25, 2026)</li>
<li><a href="https://cursor.com/docs/configuration/worktrees">Cursor worktrees docs</a> — 6-hour cleanup, 25 per machine, foreign worktrees eligible</li>
</ul>
]]></content:encoded></item><item><title>A Renamed Tool Is a Disabled Guard: Contract-Test Hook Coverage Before Antigravity’s Oct 5 Cutover</title><link>https://automater.ai/intel/tool-schema-guard-contract-tests/</link><guid>https://automater.ai/intel/tool-schema-guard-contract-tests/</guid><description>Antigravity agent hooks match tool names and fail open, and 09-2026 renamed the file tools. Test guard coverage and replay wire fixtures before Oct 5.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>The Java version of Google’s own privacy-gate sample for Antigravity agent hooks guards file reads by matching the tool name <code>read_file</code>. The Antigravity agent version Google released on Sep 17 has no tool by that name. It reads files with <code>view_file</code>, which takes an <code>AbsolutePath</code> where the old tool took a <code>path</code>, and the Java sample, still on the hooks page as of Sep 21, matches nothing. Nothing errors. A regex that finds no tool is a valid regex, and a hook that never fires looks exactly like a hook that approved every call.</p>
<p>That is the whole failure, and it isn’t Google’s alone. A guard is a string compared against a tool name, plus a script that reads a field from the call. Rename the tool or the field and the guard stays installed, loads cleanly and stops guarding. The May version of the Antigravity agent shuts down on Oct 5, so every lane pinned to it changes tools within two weeks, whether you plan the move or the API makes it for you.</p>
<p>By Tuesday you’ll have a guard contract suite that runs on every agent-version or harness bump and daily. It asserts coverage against the tools the agent actually has, replays recorded payloads and deliberate failures through every guard, and takes write evidence from the disk. You’ll also pin <code>antigravity-preview-09-2026</code> today, with a drill for both things Oct 5 might do.</p>
<h2 id="sep-17-antigravity-09-2026-renames-the-file-tools-and-googles-docs-disagree-about-oct-5">Sep 17: Antigravity 09-2026 renames the file tools, and Google’s docs disagree about Oct 5</h2>
<p>On Sep 17, 2026, the <a href="https://ai.google.dev/gemini-api/docs/changelog">Gemini API changelog</a> released <code>antigravity-preview-09-2026</code> as the replacement for <code>antigravity-preview-05-2026</code>, which it deprecates. Four built-in tools changed name and shape, and two search tools arrived where 05 had only the shell:</p>
<table>
<thead>
<tr>
<th>Job</th>
<th>05-2026</th>
<th>09-2026</th>
</tr>
</thead>
<tbody>
<tr>
<td>Write a file</td>
<td><code>write_file(path, content)</code></td>
<td><code>write_to_file(TargetFile, CodeContent, Overwrite, Description)</code></td>
</tr>
<tr>
<td>Edit a file</td>
<td>full rewrite</td>
<td><code>replace_file_content(TargetFile, StartLine, EndLine, TargetContent, ReplacementContent)</code></td>
</tr>
<tr>
<td>Read a file</td>
<td><code>read_file(path, offset, limit)</code></td>
<td><code>view_file(AbsolutePath, StartLine, EndLine, ContentOffset)</code></td>
</tr>
<tr>
<td>List a directory</td>
<td><code>list_files(path)</code></td>
<td><code>list_dir(DirectoryPath)</code></td>
</tr>
<tr>
<td>Search</td>
<td>shell only</td>
<td><code>find_by_name(SearchDirectory, Pattern, MaxDepth)</code>, <code>grep_search(SearchPath, Query, IsRegex)</code></td>
</tr>
</tbody>
</table>
<p>Two things there matter to a guard. The argument naming the file moved from <code>path</code> to <code>TargetFile</code> or <code>AbsolutePath</code>, so a guard reading the old field reads nothing. And edits moved to line ranges, so the arguments no longer describe the resulting file.</p>
<p>The <a href="https://ai.google.dev/gemini-api/docs/deprecations">deprecations page</a> lists Oct 5, 2026 as the 05-2026 shutdown and defines shutdown generically: the model is turned off completely and “the endpoint is no longer available.” It also calls its dates the earliest possible ones. Google’s <a href="https://github.com/google-gemini/gemini-skills/blob/main/skills/gemini-api-dev/references/migration.md">gemini-skills migration table</a>, a reference file for coding agents merged in <a href="https://github.com/google-gemini/gemini-skills/pull/83">PR #83</a> the same day, says something else: “Requests redirect to <code>antigravity-preview-09-2026</code> after October 5, 2026.” One Google source predicts a hard error. The other predicts a silent swap to an agent whose tools have different names. A stale pin that errors is loud; a stale pin that redirects keeps running with guards keyed to names that no longer exist.</p>
<p>
<em>Screenshot: GitHub, “gemini-skills/skills/gemini-api-dev/references/migration.md at main · google-gemini/gemini-skills” (PR #83 merged Sep 17, 2026), captured Sep 21, 2026.</em></p>
<p>The changelog’s migration advice is narrower than it reads. A remote-sandbox user who reads only <code>output_text</code> or <code>model_output</code> steps can, it says, “update the agent string and nothing else changes”; the next sentence says anyone running tools locally or parsing <code>function_call</code> steps faces changed built-in tools. Our reading, not Google’s sentence: hooks run inside the remote sandbox and match on the tool’s container name, so a <code>hooks.json</code> written for 05 names sits in that no-change path and has just stopped matching.</p>
<p>The <a href="https://ai.google.dev/gemini-api/docs/agent-hooks">agent hooks page</a>, updated the same day, sets the rules this piece tests. Matchers are RE2 regular expressions evaluated against the tool’s container name. The documented decisions are allow and deny (<code>{"decision": "deny", "reason": "..."}</code>). A command that crashes with a non-zero exit, an HTTP hook that returns non-2xx, a timeout (default 30 seconds) or unrecognised JSON all count as allow. Hooks cover <code>code_execution</code> and the filesystem tools, and they don’t fire for custom <code>function</code> tools or <code>mcp_server</code> tools. The page warns that <code>.*_file</code> misses <code>replace_file_content</code> and <code>list_dir</code>. Its Python and JavaScript samples use the 09 names, but the Java tabs of two samples still key on 05 names: a privacy gate on <code>read_file</code> and an audit logger on <code>read_file|write_file</code>.</p>
<p>
<em>Screenshot: Google AI for Developers, “Hooks / Gemini API / Google AI for Developers” (last updated Sep 17, 2026), captured Sep 21, 2026.</em></p>
<p>One naming trap: “Antigravity” here is the Gemini API’s managed Antigravity Agent. The Antigravity CLI and IDE are a separate product with their own hook model, so keep write-ups about them out of these tables.</p>
<h2 id="why-antigravity-agent-hooks-claude-code-matchers-and-codex-hooks-all-fail-quiet">Why Antigravity agent hooks, Claude Code matchers and Codex hooks all fail quiet</h2>
<p>A tool integration breaks loudly: the call fails and somebody sees an error. A guard sits beside the tool rather than in its path, so it breaks quietly. The harness checks whether any matcher applies, and when none does the call simply runs; nothing logs <em>no guard matched</em>. On Antigravity, a guard that matches and then crashes lets the call through too.</p>
<p>Claude Code has hit the same wall repeatedly this year, per its <a href="https://code.claude.com/docs/en/changelog">dated changelog</a>. Version 2.1.191 (Jun 24) fixed comma-separated matchers such as <code>Bash,PowerShell</code> that had silently never fired. Version 2.1.195 (Jun 26) moved hyphenated matchers from substring to exact match, prescribing <code>mcp__brave-search__.*</code> to cover a hyphenated server’s tools. The <a href="https://code.claude.com/docs/en/hooks">hooks reference</a> adds that a matcher on a bare server key never fires for plugin-bundled MCP tools, which are named <code>mcp__plugin_&lt;plugin-name&gt;_&lt;server-name&gt;__&lt;tool&gt;</code>. And 2.1.275 (Sep 17) fixed <code>/update-config</code> writing <code>Write(path)</code> permission rules that file-permission checks don’t match, two months after 2.1.210 (Jul 14) began warning about them. Each was a rule that parsed, loaded and matched something other than what its author meant.</p>
<p>Codex documents its coverage more candidly. Its <a href="https://learn.chatgpt.com/docs/hooks">hooks docs</a> say shell, unified exec, <code>apply_patch</code>, MCP tools and other local function tools fire <code>PreToolUse</code>; hosted tools such as WebSearch don’t; <code>write_stdin</code> doesn’t re-run the hook. They call hooks “a useful guardrail, not a complete enforcement boundary,” and say decisions apply to tools called from code mode. <a href="https://github.com/openai/codex/issues/23411">Issue #23411</a>, open since May 19 with user reproductions through September and no maintainer reply, reports that outer Code Mode <code>exec</code> doesn’t fire <code>PreToolUse</code>. That’s a user report against the docs, and a test case.</p>
<p>Coverage, in other words, is a measurement you repeat on every version, including the ones you didn’t choose.</p>
<h2 id="the-guard-contract-suite-four-asserts-every-bump-and-every-day">The guard contract suite: four asserts, every bump and every day</h2>
<p>The suite is a repo with one tool list, fixture set and guard inventory per vendor per agent version, plus a runner that fails the bump on any red line. It runs on every bump and once a day, because a hosted agent and its docs can change on a day you changed nothing.</p>
<p>
<em>Two inputs, four asserts, one gate. The red link is Google’s own sample meeting Google’s own rename.</em></p>
<h2 id="step-1-assert-coverage-against-the-tools-the-agent-actually-has">Step 1: Assert coverage against the tools the agent actually has</h2>
<p>Build a tool list per vendor per agent version from the vendor’s changelog and from the tool names in recorded <code>function_call</code> steps and hook payloads; when they disagree, the recording wins. Give every tool a class and a hookability flag. On Antigravity, <code>function</code> and <code>mcp_server</code> tools are unhookable by design, and the <a href="https://ai.google.dev/gemini-api/docs/antigravity-agent">Antigravity agent page</a> lists <code>google_search</code> and <code>url_context</code> as default tools whose hook coverage the hooks page doesn’t document. The hooks page names its targetable tools as <code>code_execution</code> plus <code>view_file</code>, <code>write_to_file</code>, <code>replace_file_content</code>, <code>list_dir</code> and <code>delete_file</code>, which leaves the two new search tools undocumented too. On Codex, hosted tools are unhooked.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># tool-lists/antigravity-agent/09-2026.yaml (illustrative shape; a human reviews it per version)</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">code_execution</span><span style="color:#E1E4E8">,       </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">exec</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">write_to_file</span><span style="color:#E1E4E8">,        </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">write</span><span style="color:#E1E4E8">,       </span><span style="color:#85E89D">key</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">TargetFile</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">replace_file_content</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">write</span><span style="color:#E1E4E8">,       </span><span style="color:#85E89D">key</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">TargetFile</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">delete_file</span><span style="color:#E1E4E8">,          </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">delete</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">view_file</span><span style="color:#E1E4E8">,            </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">secret-read</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">key</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">AbsolutePath</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">grep_search</span><span style="color:#E1E4E8">,          </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">secret-read</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">key</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">SearchPath</span><span style="color:#E1E4E8">,      </span><span style="color:#85E89D">hookable</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">undocumented</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">list_dir</span><span style="color:#E1E4E8">,             </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">read</span><span style="color:#E1E4E8">,        </span><span style="color:#85E89D">key</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">DirectoryPath</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">find_by_name</span><span style="color:#E1E4E8">,         </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">read</span><span style="color:#E1E4E8">,        </span><span style="color:#85E89D">key</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">SearchDirectory</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">hookable</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">undocumented</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">google_search</span><span style="color:#E1E4E8">,        </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">network</span><span style="color:#E1E4E8">,     </span><span style="color:#85E89D">hookable</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">undocumented</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">url_context</span><span style="color:#E1E4E8">,          </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">network</span><span style="color:#E1E4E8">,     </span><span style="color:#85E89D">hookable</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">undocumented</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">- {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">each-function-tool</span><span style="color:#E1E4E8">,   </span><span style="color:#85E89D">class</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">by-tool</span><span style="color:#E1E4E8">,     </span><span style="color:#85E89D">hookable</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">false</span><span style="color:#E1E4E8">}   </span><span style="color:#6A737D"># hooks skip function and mcp_server tools</span></span></code></pre>
<p>Next, the guard inventory: every hook, its event, its matcher, the field it keys on, and the tools its author intends it to cover. Then the assert, with four rules:</p>
<ol>
<li>Every hookable write, exec, delete and secret-read tool hits at least one pre-tool guard.</li>
<li>Every matcher matches at least one current tool name. A dead matcher fails the run even when coverage is otherwise complete, because it is a guard somebody believes in.</li>
<li>Every matcher compiles in the vendor’s dialect. Python’s <code>re</code> rejects a lone <code>*</code>, as RE2 does, yet Google’s hooks page lists <code>"*"</code> and <code>""</code> beside <code>".*"</code> as catch-alls, and its auto-format sample uses <code>"*"</code>. Normalize those two documented spellings to <code>.*</code>, fail anything else that doesn’t compile, and replay one fixture to confirm the runtime agrees.</li>
<li>Every matcher’s real match set equals its declared intent. Before 2.1.195, Claude Code treated a hyphenated name as an unanchored regex, so a matcher for <code>code-reviewer</code> also fired for <code>senior-code-reviewer</code>.</li>
</ol>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#6A737D"># coverage_assert.py (illustrative): run per vendor x version; any printed line fails the bump</span></span>
<span class="line"><span style="color:#F97583">import</span><span style="color:#E1E4E8"> re, sys, yaml</span></span>
<span class="line"><span style="color:#79B8FF">EXACT</span><span style="color:#F97583"> =</span><span style="color:#E1E4E8"> re.compile(</span><span style="color:#F97583">r</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">[A-Za-z0-9_</span><span style="color:#85E89D;font-weight:bold">\-</span><span style="color:#79B8FF"> ,|]</span><span style="color:#F97583">+</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">)     </span><span style="color:#6A737D"># Claude Code: only these characters means an exact-name list</span></span>
<span class="line"><span style="color:#79B8FF">CATCH_ALL</span><span style="color:#F97583"> =</span><span style="color:#E1E4E8"> {</span><span style="color:#9ECBFF">"*"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">""</span><span style="color:#E1E4E8">}                          </span><span style="color:#6A737D"># Antigravity: listed beside ".*" as catch-alls</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">def</span><span style="color:#B392F0"> pat</span><span style="color:#E1E4E8">(g):</span></span>
<span class="line"><span style="color:#F97583">    return</span><span style="color:#9ECBFF"> ".*"</span><span style="color:#F97583"> if</span><span style="color:#E1E4E8"> g[</span><span style="color:#9ECBFF">"vendor"</span><span style="color:#E1E4E8">] </span><span style="color:#F97583">==</span><span style="color:#9ECBFF"> "antigravity"</span><span style="color:#F97583"> and</span><span style="color:#E1E4E8"> g[</span><span style="color:#9ECBFF">"matcher"</span><span style="color:#E1E4E8">] </span><span style="color:#F97583">in</span><span style="color:#79B8FF"> CATCH_ALL</span><span style="color:#F97583"> else</span><span style="color:#E1E4E8"> g[</span><span style="color:#9ECBFF">"matcher"</span><span style="color:#E1E4E8">]</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">def</span><span style="color:#B392F0"> hits</span><span style="color:#E1E4E8">(g, name):</span></span>
<span class="line"><span style="color:#E1E4E8">    m </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> pat(g)</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> g[</span><span style="color:#9ECBFF">"vendor"</span><span style="color:#E1E4E8">] </span><span style="color:#F97583">==</span><span style="color:#9ECBFF"> "claude-code"</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#F97583">        if</span><span style="color:#79B8FF"> EXACT</span><span style="color:#E1E4E8">.fullmatch(m):</span></span>
<span class="line"><span style="color:#F97583">            return</span><span style="color:#E1E4E8"> name </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> [s.strip() </span><span style="color:#F97583">for</span><span style="color:#E1E4E8"> s </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> re.split(</span><span style="color:#F97583">r</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">[|,]</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">, m)]</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#E1E4E8"> re.search(m, name) </span><span style="color:#F97583">is</span><span style="color:#F97583"> not</span><span style="color:#79B8FF"> None</span><span style="color:#6A737D">     # any other character: unanchored regex</span></span>
<span class="line"><span style="color:#F97583">    return</span><span style="color:#E1E4E8"> re.fullmatch(m, name) </span><span style="color:#F97583">is</span><span style="color:#F97583"> not</span><span style="color:#79B8FF"> None</span><span style="color:#6A737D">      # Antigravity: RE2, whole-name match assumed</span></span>
<span class="line"></span>
<span class="line"><span style="color:#E1E4E8">tools, guards </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> yaml.safe_load(</span><span style="color:#79B8FF">open</span><span style="color:#E1E4E8">(sys.argv[</span><span style="color:#79B8FF">1</span><span style="color:#E1E4E8">])), yaml.safe_load(</span><span style="color:#79B8FF">open</span><span style="color:#E1E4E8">(sys.argv[</span><span style="color:#79B8FF">2</span><span style="color:#E1E4E8">]))</span></span>
<span class="line"><span style="color:#F97583">for</span><span style="color:#E1E4E8"> g </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> guards:</span></span>
<span class="line"><span style="color:#F97583">    try</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">        re.compile(pat(g))</span></span>
<span class="line"><span style="color:#F97583">    except</span><span style="color:#E1E4E8"> re.error:</span></span>
<span class="line"><span style="color:#79B8FF">        print</span><span style="color:#E1E4E8">(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"BAD REGEX    </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">g[</span><span style="color:#9ECBFF">'id'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">: </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">g[</span><span style="color:#9ECBFF">'matcher'</span><span style="color:#E1E4E8">]</span><span style="color:#F97583">!r</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">); </span><span style="color:#F97583">continue</span></span>
<span class="line"><span style="color:#E1E4E8">    got </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> {t[</span><span style="color:#9ECBFF">"name"</span><span style="color:#E1E4E8">] </span><span style="color:#F97583">for</span><span style="color:#E1E4E8"> t </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> tools </span><span style="color:#F97583">if</span><span style="color:#E1E4E8"> hits(g, t[</span><span style="color:#9ECBFF">"name"</span><span style="color:#E1E4E8">])}</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#F97583"> not</span><span style="color:#E1E4E8"> got:</span></span>
<span class="line"><span style="color:#79B8FF">        print</span><span style="color:#E1E4E8">(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"DEAD MATCHER </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">g[</span><span style="color:#9ECBFF">'id'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">: </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">g[</span><span style="color:#9ECBFF">'matcher'</span><span style="color:#E1E4E8">]</span><span style="color:#F97583">!r</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">    elif</span><span style="color:#E1E4E8"> got </span><span style="color:#F97583">!=</span><span style="color:#79B8FF"> set</span><span style="color:#E1E4E8">(g[</span><span style="color:#9ECBFF">"intent"</span><span style="color:#E1E4E8">]):</span></span>
<span class="line"><span style="color:#79B8FF">        print</span><span style="color:#E1E4E8">(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"DRIFT        </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">g[</span><span style="color:#9ECBFF">'id'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">: matches </span><span style="color:#79B8FF">{sorted</span><span style="color:#E1E4E8">(got)</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">, intent </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">g[</span><span style="color:#9ECBFF">'intent'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">for</span><span style="color:#E1E4E8"> t </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> tools:</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> t[</span><span style="color:#9ECBFF">"class"</span><span style="color:#E1E4E8">] </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> {</span><span style="color:#9ECBFF">"write"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"exec"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"delete"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"secret-read"</span><span style="color:#E1E4E8">} </span><span style="color:#F97583">and</span><span style="color:#E1E4E8"> t.get(</span><span style="color:#9ECBFF">"hookable"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">True</span><span style="color:#E1E4E8">) </span><span style="color:#F97583">is</span><span style="color:#79B8FF"> True</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#F97583">        if</span><span style="color:#F97583"> not</span><span style="color:#79B8FF"> any</span><span style="color:#E1E4E8">(g[</span><span style="color:#9ECBFF">"event"</span><span style="color:#E1E4E8">] </span><span style="color:#F97583">==</span><span style="color:#9ECBFF"> "pre"</span><span style="color:#F97583"> and</span><span style="color:#E1E4E8"> hits(g, t[</span><span style="color:#9ECBFF">"name"</span><span style="color:#E1E4E8">]) </span><span style="color:#F97583">for</span><span style="color:#E1E4E8"> g </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> guards):</span></span>
<span class="line"><span style="color:#79B8FF">            print</span><span style="color:#E1E4E8">(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"UNGUARDED    </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">t[</span><span style="color:#9ECBFF">'name'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF"> (</span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">t[</span><span style="color:#9ECBFF">'class'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">)"</span><span style="color:#E1E4E8">)</span></span></code></pre>
<p>The Antigravity branch assumes whole-name matching. That is an inference from the page’s <code>.*_file</code> advice, not a documented rule, and its <code>""</code> catch-all points the other way, so the fixture replay in step 2 is where you confirm it. Tools flagged <code>undocumented</code> or <code>false</code> skip the unguarded check on purpose; they go to the wall table in step 6.</p>
<p>Run it with the hooks page’s Java samples plus its auto-format example as the inventory and the 09 list as the tools, and the Sep 17 story prints itself (the security gate on <code>code_execution</code> passes, and the catch-all auto-format hook runs only after the fact):</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="text"><code><span class="line"><span>DEAD MATCHER privacy-gate: 'read_file'</span></span>
<span class="line"><span>DEAD MATCHER audit-log: 'read_file|write_file'</span></span>
<span class="line"><span>UNGUARDED    write_to_file (write)</span></span>
<span class="line"><span>UNGUARDED    replace_file_content (write)</span></span>
<span class="line"><span>UNGUARDED    delete_file (delete)</span></span>
<span class="line"><span>UNGUARDED    view_file (secret-read)</span></span></code></pre>
<p>Against a 05 tool list the privacy gate comes back green, which is why nobody looked. The Python tab of the same gate already says <code>view_file</code>: one guard kept in two places drifted in one. A new UNGUARDED line can also mean a guard was removed. If nobody on your team removed it, that is the persistence problem in <a href="/intel/unauthored-agent-hooks-persistence/">unauthored hooks</a>, and it gets an incident.</p>
<h2 id="step-2-replay-wire-fixtures-not-doc-samples-through-every-guard">Step 2: Replay wire fixtures, not doc samples, through every guard</h2>
<p>Docs lag the wire, and Google’s samples prove it. Record real payloads per agent version, the stdin each hook receives and the <code>function_call</code> steps the agent emits, from a disposable lane working on canary files, never production data. Commit each one beside a sidecar saying what it must prove.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="text"><code><span class="line"><span>fixtures/antigravity-agent/09-2026/view_file.canary-secret.json        # raw payload, as recorded</span></span>
<span class="line"><span>fixtures/antigravity-agent/09-2026/view_file.canary-secret.meta.yaml   # tool: view_file, key: AbsolutePath, expect: deny</span></span>
<span class="line"><span>fixtures/claude-code/2.1.278/PreToolUse.Edit.canary.json</span></span>
<span class="line"><span>fixtures/codex/0.155.0/PreToolUse.apply_patch.canary.json</span></span></code></pre>
<p>Replay every fixture through every guard that claims its tool, and assert three things: some guard matched, the field it keys on is present and non-empty, and the verdict is the one the sidecar expects.</p>
<table>
<thead>
<tr>
<th>Guard keys on</th>
<th>05-2026 field</th>
<th>09-2026 field</th>
<th>Assert</th>
</tr>
</thead>
<tbody>
<tr>
<td>Write target</td>
<td><code>path</code></td>
<td><code>TargetFile</code></td>
<td>present, non-empty, resolves inside the workspace</td>
</tr>
<tr>
<td>Read target</td>
<td><code>path</code></td>
<td><code>AbsolutePath</code></td>
<td>present, non-empty, checked against the secrets list</td>
</tr>
<tr>
<td>List or search root</td>
<td><code>path</code> (<code>list_files</code> only)</td>
<td><code>DirectoryPath</code>, <code>SearchDirectory</code>, <code>SearchPath</code></td>
<td>present, non-empty</td>
</tr>
<tr>
<td>Shell</td>
<td><code>command</code> (Claude Code, Codex)</td>
<td>unchanged</td>
<td>present, non-empty</td>
</tr>
</tbody>
</table>
<p>The non-empty assert is the one people skip, and it catches the nastiest version of this bug. Fix the matcher to <code>view_file</code> but leave the guard reading <code>path</code>, and the guard gets an empty string, finds no secrets prefix in it, and allows. The matcher is green, the guard ran, the log says allow, and the canary secret was read.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#6A737D"># replay.py (illustrative): adapters are written against the recordings, never against doc samples</span></span>
<span class="line"><span style="color:#F97583">for</span><span style="color:#E1E4E8"> fx </span><span style="color:#F97583">in</span><span style="color:#79B8FF"> sorted</span><span style="color:#E1E4E8">(glob.glob(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"fixtures/</span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">vendor</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">/</span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">version</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">/*.json"</span><span style="color:#E1E4E8">)):</span></span>
<span class="line"><span style="color:#E1E4E8">    meta </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> yaml.safe_load(</span><span style="color:#79B8FF">open</span><span style="color:#E1E4E8">(fx.replace(</span><span style="color:#9ECBFF">".json"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">".meta.yaml"</span><span style="color:#E1E4E8">)))</span></span>
<span class="line"><span style="color:#E1E4E8">    raw </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> open</span><span style="color:#E1E4E8">(fx).read()</span></span>
<span class="line"><span style="color:#E1E4E8">    args </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> ADAPTERS</span><span style="color:#E1E4E8">[vendor](json.loads(raw))</span></span>
<span class="line"><span style="color:#F97583">    assert</span><span style="color:#E1E4E8"> args.get(meta[</span><span style="color:#9ECBFF">"key"</span><span style="color:#E1E4E8">]), </span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">fx</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">: </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">meta[</span><span style="color:#9ECBFF">'key'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF"> missing or empty"</span></span>
<span class="line"><span style="color:#E1E4E8">    guards </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> guards_for(vendor, meta[</span><span style="color:#9ECBFF">"tool"</span><span style="color:#E1E4E8">])</span></span>
<span class="line"><span style="color:#F97583">    assert</span><span style="color:#E1E4E8"> guards, </span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">fx</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">: no guard matched </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">meta[</span><span style="color:#9ECBFF">'tool'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#F97583">    for</span><span style="color:#E1E4E8"> g </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> guards:</span></span>
<span class="line"><span style="color:#E1E4E8">        verdict </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> run_guard(g, </span><span style="color:#FFAB70">stdin</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">raw)           </span><span style="color:#6A737D"># the real script, fed the real bytes</span></span>
<span class="line"><span style="color:#F97583">        assert</span><span style="color:#E1E4E8"> verdict </span><span style="color:#F97583">==</span><span style="color:#E1E4E8"> meta[</span><span style="color:#9ECBFF">"expect"</span><span style="color:#E1E4E8">], </span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">fx</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">: </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">g[</span><span style="color:#9ECBFF">'id'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF"> said </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">verdict</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">"</span></span></code></pre>
<h2 id="step-3-replay-the-failures-and-record-the-verdict-each-harness-applies">Step 3: Replay the failures and record the verdict each harness applies</h2>
<p>A guard’s contract includes what happens when the guard breaks, and that part belongs to the harness, not to your script. For each vendor, point a disposable lane at a canary action (a read of a canary secret file works everywhere) and swap in guard variants that deny cleanly, ask, print malformed output, exit non-zero and sleep past the timeout. Record what the harness actually did: did the canary run? Documented behaviour as of Sep 21:</p>
<table>
<thead>
<tr>
<th>Replay</th>
<th>Antigravity Agent (API)</th>
<th>Claude Code</th>
<th>Codex</th>
</tr>
</thead>
<tbody>
<tr>
<td>Clean deny</td>
<td>blocks only on <code>{"decision": "deny", ...}</code></td>
<td>blocks on exit 2 or <code>permissionDecision: "deny"</code></td>
<td>blocks on <code>permissionDecision: "deny"</code>, <code>{"decision": "block"}</code> or exit 2</td>
</tr>
<tr>
<td>Ask</td>
<td>not a documented decision: record it</td>
<td>forces a permission prompt</td>
<td>parsed but not supported: hook marked failed, call continues</td>
</tr>
<tr>
<td>Malformed output</td>
<td>allow</td>
<td>not documented for exit 0: record it</td>
<td>not documented: record it</td>
</tr>
<tr>
<td>Non-zero exit, not 2</td>
<td>allow</td>
<td>doesn’t block on its own</td>
<td>not documented: record it</td>
</tr>
<tr>
<td>Timeout</td>
<td>allow (default 30 s)</td>
<td>output discarded, no decision; call continues through the normal permission flow (default 600 s)</td>
<td>not documented (default 600 s): record it</td>
</tr>
</tbody>
</table>
<p>Read the Antigravity column first: only a clean deny blocks, and every documented failure lets the call run. Then read the non-zero-exit row. The Claude Code contract in <a href="/intel/jev-claude-code-hook-not-mcp/">the Claude Code + Jev hook playbook</a> sends every failure path to exit 2, which is right for Claude Code. Port that script to Antigravity and exit 2 is just a non-zero exit, which means allow; the fail-closed guard turns fail-open without a line changing. The same holds for converting verifier errors to exit 2, as <a href="/intel/mcp-approve-once-is-dead/">the approve-once piece</a> teaches: correct on Claude Code, an approval on Antigravity.</p>
<p>On Antigravity the only blocking output is a deny printed by a process that exits cleanly, so wrap every guard to do exactly that on its own errors:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#6A737D">#!/usr/bin/env python3</span></span>
<span class="line"><span style="color:#6A737D"># ag_guard.py (illustrative): on Antigravity a crash or non-zero exit is an allow, so print the deny yourself</span></span>
<span class="line"><span style="color:#F97583">import</span><span style="color:#E1E4E8"> json, signal, sys</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">def</span><span style="color:#B392F0"> _deadline</span><span style="color:#E1E4E8">(</span><span style="color:#F97583">*</span><span style="color:#E1E4E8">_):</span></span>
<span class="line"><span style="color:#F97583">    raise</span><span style="color:#79B8FF"> TimeoutError</span></span>
<span class="line"><span style="color:#E1E4E8">signal.signal(signal.</span><span style="color:#79B8FF">SIGALRM</span><span style="color:#E1E4E8">, _deadline)</span></span>
<span class="line"><span style="color:#E1E4E8">signal.alarm(</span><span style="color:#79B8FF">10</span><span style="color:#E1E4E8">)                                   </span><span style="color:#6A737D"># your own deadline, well under the 30 s hook default</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">try</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">    reason </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> check(json.load(sys.stdin))           </span><span style="color:#6A737D"># None means allow; a string is the deny reason</span></span>
<span class="line"><span style="color:#E1E4E8">    out </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> {</span><span style="color:#9ECBFF">"decision"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"allow"</span><span style="color:#E1E4E8">} </span><span style="color:#F97583">if</span><span style="color:#E1E4E8"> reason </span><span style="color:#F97583">is</span><span style="color:#79B8FF"> None</span><span style="color:#F97583"> else</span><span style="color:#E1E4E8"> {</span><span style="color:#9ECBFF">"decision"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"deny"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"reason"</span><span style="color:#E1E4E8">: reason}</span></span>
<span class="line"><span style="color:#F97583">except</span><span style="color:#79B8FF"> BaseException</span><span style="color:#F97583"> as</span><span style="color:#E1E4E8"> e:</span></span>
<span class="line"><span style="color:#E1E4E8">    out </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> {</span><span style="color:#9ECBFF">"decision"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"deny"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"reason"</span><span style="color:#E1E4E8">: </span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"guard error: </span><span style="color:#79B8FF">{type</span><span style="color:#E1E4E8">(e).</span><span style="color:#79B8FF">__name__}</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#79B8FF">print</span><span style="color:#E1E4E8">(json.dumps(out))</span></span>
<span class="line"><span style="color:#E1E4E8">sys.exit(</span><span style="color:#79B8FF">0</span><span style="color:#E1E4E8">)</span></span></code></pre>
<p>The wrapper narrows fail-open without closing it. An interpreter that never starts, a process killed from outside, or the runtime’s own 30-second timeout still means allow. That is why the timeout replay stays in the suite after the wrapper ships, and why step 6 exists.</p>
<h2 id="step-4-take-write-evidence-from-the-disk-not-the-arguments">Step 4: Take write evidence from the disk, not the arguments</h2>
<p><code>replace_file_content</code> edits a line range and <code>write_to_file</code> carries an <code>Overwrite</code> flag. The arguments describe an intent; what landed is on the disk. For every write-class call, record the path, a hash before, a hash after and the real diff, from a pre-tool and a post-tool hook (<code>pre_tool_execution</code> / <code>post_tool_execution</code> on Antigravity, <code>PreToolUse</code> / <code>PostToolUse</code> elsewhere).</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span><span style="color:#79B8FF">"ts"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"2026-09-22T14:03:11Z"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"lane"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"ag-docs-03"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"agent"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"antigravity-preview-09-2026"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"tool"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"replace_file_content"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"target"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"/workspace/src/billing.py"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"declared_lines"</span><span style="color:#E1E4E8">:[</span><span style="color:#79B8FF">40</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">58</span><span style="color:#E1E4E8">],</span><span style="color:#79B8FF">"pre_sha256"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"9f2c..."</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"post_sha256"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"41ab..."</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"changed_lines"</span><span style="color:#E1E4E8">:[</span><span style="color:#79B8FF">40</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">61</span><span style="color:#E1E4E8">],</span><span style="color:#79B8FF">"evidence"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"complete"</span><span style="color:#E1E4E8">}</span></span></code></pre>
<p>Three asserts run over that log:</p>
<ul>
<li><strong>Every write-class call has both hashes.</strong> Post hooks fail open too, so a missing pair is a failed check, never a pass.</li>
<li><strong>The changed lines sit inside the declared range.</strong> The illustrative record above changed line 61 on a call that declared 40 to 58. Flag it and read the diff.</li>
<li><strong>No hash changes without a recorded call.</strong> Code run through <code>code_execution</code> can write files without calling a file tool, and so can a custom function tool your hooks never see. A sweep of workspace hashes from the runner, outside the agent, is how you notice.</li>
</ul>
<h2 id="step-5-pin-09-2026-now-shadow-the-suite-then-test-both-oct-5-outcomes">Step 5: Pin 09-2026 now, shadow the suite, then test both Oct 5 outcomes</h2>
<p>Across two vendors, five dated changes in three months touched what a guard or permission rule matches, and a sixth is on the calendar.</p>
<p>
<em>Every marker is a documented release or shutdown date. What a 05-2026 request gets after Oct 5 is not documented consistently, so it gets a drill.</em></p>
<p>Work the calendar backwards from Oct 5:</p>
<table>
<thead>
<tr>
<th>When</th>
<th>Do</th>
<th>Pass when</th>
</tr>
</thead>
<tbody>
<tr>
<td>Today, Sep 21</td>
<td>Search every lane config, runner script and SDK call for <code>antigravity-preview-05-2026</code>; pin <code>antigravity-preview-09-2026</code> explicitly</td>
<td>no 05 string anywhere a lane can read it</td>
</tr>
<tr>
<td>This week</td>
<td>Port matchers and key fields to 09 names; record 09 fixtures from a disposable lane</td>
<td>steps 1 to 4 green on 09</td>
</tr>
<tr>
<td>Until Oct 2 (illustrative cut-over)</td>
<td>Run the suite daily in shadow: log every red line, block nothing, give each one an owner</td>
<td>red list empty or owned</td>
</tr>
<tr>
<td>Oct 2</td>
<td>Make the suite blocking for Antigravity lanes</td>
<td>a red line stops the bump and pages the owner</td>
</tr>
<tr>
<td>Oct 5 onward</td>
<td>From one disposable lane with canary tools only, send one request pinned to 05; record error or redirect, with the date</td>
<td>outcome logged</td>
</tr>
<tr>
<td>Oct 5 onward</td>
<td>Runner refuses to start any lane pinned to 05, whatever the API does</td>
<td>refusal in the start log</td>
</tr>
</tbody>
</table>
<p>Both outcomes need a handler before you know which one you’ll get. An error is easy: the lane fails and the runner alerts. A redirect shows up in what comes back: <code>function_call</code> steps from a 05-pinned request that name <code>view_file</code>, <code>write_to_file</code> or <code>list_dir</code> are 09 tools you didn’t ask for, and that lane runs only while the 09 suite is green. The runner’s refusal is the control; the API’s behaviour on the day is evidence.</p>
<p>Oct 5 is also a dated deprecation, so it belongs on the ledger that <a href="/intel/forced-model-routing-continuity/">forced model routing continuity</a> keeps for model-ID swaps; point that ledger entry at this suite as its proof. After any bump, the served-behaviour replay in <a href="/intel/cli-upgrade-canary-drift/">the CLI upgrade canary</a> checks what the lane was actually served. This suite checks that its guards still see it.</p>
<h2 id="step-6-put-a-wall-behind-every-guard-the-suite-cant-prove">Step 6: Put a wall behind every guard the suite can’t prove</h2>
<p>Hooks stay guardrails. The suite proves coverage and verdicts at test time; it can’t make a fail-open runtime fail closed or hook a tool the vendor never routes through hooks. Each such path gets a named wall and its own test:</p>
<table>
<thead>
<tr>
<th>Path the hook can’t hold</th>
<th>Wall behind it</th>
<th>Test</th>
</tr>
</thead>
<tbody>
<tr>
<td>Antigravity custom <code>function</code> tools</td>
<td>your application executes them, so the check lives in that code</td>
<td>a canary function call your code refuses</td>
</tr>
<tr>
<td>Antigravity <code>mcp_server</code> tools</td>
<td>the MCP server’s or gateway’s own authorization</td>
<td>a canary tool call rejected upstream</td>
</tr>
<tr>
<td>Any Antigravity guard failure</td>
<td>keep secrets out of the sandbox; scope what it can reach</td>
<td>the canary secret isn’t in the sandbox at all</td>
</tr>
<tr>
<td>Claude Code guard failure or dead matcher</td>
<td>permission deny rules and the sandbox under destructive tools</td>
<td>post-bump canary repo; rules written as <code>Edit(path)</code>, not <code>Write(path)</code></td>
</tr>
<tr>
<td>Codex hosted tools such as WebSearch</td>
<td>egress policy outside the agent</td>
<td>a canary domain stays blocked with hooks off</td>
</tr>
<tr>
<td>Codex code-mode <code>exec</code> (#23411)</td>
<td>the same shell deny, enforced by the sandbox rather than the hook</td>
<td>the shell canary sent through code mode is refused</td>
</tr>
</tbody>
</table>
<p>For the Claude Code row, run the post-bump canary repo from <a href="/intel/restricted-mode-fleet-policy/">restricted mode as fleet policy</a> in the same gate as this suite; the cross-harness version is <a href="/intel/non-overridable-agent-permissions/">the managed-permissions override test</a>. And remember what 2.1.275 fixed: the wall itself had been written as rules nothing matched. A permission rule is one more matcher, so it goes in the coverage assert too.</p>
<h2 id="where-antigravity-agent-hooks-pass-the-suite-and-still-guard-nothing">Where Antigravity agent hooks pass the suite and still guard nothing</h2>
<p><strong>The tool list came from the changelog.</strong> A tool in <code>function_call</code> steps but not in your list is invisible to every assert. Signal: a recorded tool name missing from the list.</p>
<p><strong>The matcher moved; the field didn’t.</strong> The usual half-fix after a rename. Signal: the key-field assert fails, or the canary secret read logs allow from a guard that matched.</p>
<p><strong>The guard matches too much.</strong> Claude Code 2.1.275 also fixed <code>SubagentStop</code> hooks with a specific matcher that fired for every stopping subagent whose agent type was empty. Signal: DRIFT lines from step 1. Over-broad guards get disabled by the people they annoy, which ends where a dead one does.</p>
<p><strong>Green on Claude’s contract, open on Google’s runtime.</strong> Signal: the non-zero-exit replay reads allow on Antigravity for a guard someone ported from a Claude Code lane. The step 3 wrapper is the fix.</p>
<p><strong>The suite ran once.</strong> Signal: the last green run is older than the last agent bump, or older than a day. Hosted agents and their docs change on their own calendar.</p>
<h2 id="guard-coverage-is-fleet-inventory-not-a-line-in-one-hooks-file">Guard coverage is fleet inventory, not a line in one hooks file</h2>
<p>A fleet that runs Antigravity Agent, Claude Code and Codex lanes has three matcher dialects, three failure policies and three release trains. Reading one <code>hooks.json</code> tells you whether that file parses, not whether the fleet’s writes are guarded. Coverage is a join between what each lane runs today and what each guard matches, and only the layer that starts lanes, pins versions and holds the kill switch has both halves.</p>
<p>That’s where this suite belongs: in the <a href="/intel/multi-agent-command-center/">multi-agent command center</a> layer, beside the runner that refuses to start a lane whose pinned version has a red line. Chatbots suggest; agents act. A guard that stopped being asked is an agent acting with nobody watching.</p>
<h2 id="faq">FAQ</h2>
<h3 id="do-antigravity-agent-hooks-fail-open">Do Antigravity agent hooks fail open?</h3>
<p>Yes, on the Gemini API’s Antigravity Agent. Google’s hooks page says a crashed script, a non-2xx HTTP response, a timeout or unrecognised JSON is treated as allow. Only a deny decision from a working hook blocks the call. Hooks also skip custom function and mcp_server tools, so those need a gate elsewhere.</p>
<h3 id="will-antigravity-preview-05-2026-error-or-redirect-after-october-5">Will antigravity-preview-05-2026 error or redirect after October 5?</h3>
<p>Google’s sources disagree. The deprecations page says a shut-down model’s endpoint is no longer available, which implies an error. Google’s gemini-skills migration table says 05-2026 requests redirect to 09-2026. Pin 09-2026 now, make your runner refuse 05 pins, and test both outcomes from a disposable lane.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://ai.google.dev/gemini-api/docs/changelog">Gemini API changelog</a>: 09-2026 release and tool table, Sep 17, 2026</li>
<li><a href="https://ai.google.dev/gemini-api/docs/deprecations">Gemini API deprecations</a>: 05-2026 shutdown, Oct 5, 2026</li>
<li><a href="https://ai.google.dev/gemini-api/docs/agent-hooks">Gemini API agent hooks</a>: matchers and failure handling, updated Sep 17, 2026</li>
<li><a href="https://ai.google.dev/gemini-api/docs/antigravity-agent">Gemini API Antigravity agent</a>: default tools</li>
<li><a href="https://github.com/google-gemini/gemini-skills/blob/main/skills/gemini-api-dev/references/migration.md">gemini-skills migration reference</a>: the redirect row</li>
<li><a href="https://github.com/google-gemini/gemini-skills/pull/83">gemini-skills PR #83</a>: merged Sep 17, 2026</li>
<li><a href="https://code.claude.com/docs/en/hooks">Claude Code hooks reference</a>: matcher rules, exit codes, timeouts</li>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a>: 2.1.191, 2.1.195, 2.1.210, 2.1.275</li>
<li><a href="https://learn.chatgpt.com/docs/hooks">Codex hooks docs</a>: coverage table and deny shapes</li>
<li><a href="https://github.com/openai/codex/issues/23411">openai/codex issue #23411</a>: Code Mode <code>exec</code> and <code>PreToolUse</code>, open</li>
</ul>
]]></content:encoded></item><item><title>Write Access Isn’t a Run Button: The GitHub Actions Trigger Allowlist for Agent Identities</title><link>https://automater.ai/intel/agent-actions-trigger-allowlist/</link><guid>https://automater.ai/intel/agent-actions-trigger-allowlist/</guid><description>GitHub Actions workflow execution protections decide which CI runs an agent can start. Strip edit and dispatch rights, map every cell, then drill each one.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>A coding agent’s overnight branch changes one line in <code>package.json</code>, so the <code>test</code> script now runs a helper first. Nobody touches <code>.github/workflows/</code>. The push workflow runs <code>npm test</code> on the branch as written, with the repository’s secrets in its environment. The merge gate reviews the diff in the morning, after the run has happened.</p>
<p>Until Sep 17, GitHub’s rule for who may start that run fit in one line: “every user with write access to a repository can trigger workflows.” GitHub Actions workflow execution protections, generally available since Sep 17, 2026, replace that default with an allowlist of actors and events you can target per workflow file. For a fleet of agents pushing as Apps, bots and tokens, it is the first native place to write down which CI runs each agent identity may start.</p>
<p>The move for Tuesday starts underneath the allowlist: strip workflow-edit and dispatch rights from agent tokens as two drilled steps, then write an agent trigger matrix whose every cell names its enforcing control. Encode its allow and deny cells as policies, drill each identity, time a kill switch, and list agent workflows on <code>pull_request_target</code> in public repositories before Nov 2.</p>
<h2 id="sep-17-github-actions-workflow-execution-protections-reach-ga">Sep 17: GitHub Actions workflow execution protections reach GA</h2>
<p>On Sep 17, 2026, <a href="https://github.blog/changelog/2026-09-17-workflow-execution-protections-in-github-actions-generally-available/">GitHub’s changelog</a> made workflow execution protections generally available for enterprises, organizations and repositories, after a public preview that opened Jun 18. The feature lets you “define an allowlist that controls who can trigger an Actions workflow and what events can start it.” In the post’s own words, “Actor rules cover the who, event rules cover the what, and actions evaluate both before a run.”</p>
<p>Per GitHub’s <a href="https://docs.github.com/en/actions/concepts/about-actions-policies">About Actions policies</a> page, actor rules can name users, repository roles, GitHub Apps, Copilot and Dependabot, and event rules cover <code>push</code>, <code>pull_request</code>, <code>pull_request_target</code>, <code>workflow_dispatch</code> and others. GA added workflow-file targeting, so “a single repository can apply different policies to different workflows”, plus insights with an evaluate mode and a REST API. The <a href="https://docs.github.com/en/actions/how-tos/administer/control-workflow-execution">how-to for controlling workflow execution</a> marks Evaluate “GitHub Enterprise Cloud only”, layers policies from enterprise to organization to repository, and is blunt: “Non-allowed actors will not be able to run the specified workflows at all.”</p>
<p>The post also adds a default aimed at “Pwn Request” attacks, where fork code in a <code>pull_request_target</code> workflow can “poison your pipeline and exfiltrate secrets.” Public repositories with no event policy get a rule that disables <code>pull_request_target</code>, and “This default does not apply to private or internal repositories.” It starts in evaluate mode. Then: “On November 2, 2026, we’ll automatically enforce the default rule for affected repositories that were using the default pull_request_target policy before general availability.”</p>
<p>
<em>Screenshot: GitHub Changelog, “Workflow execution protections in GitHub Actions generally available” (Sep 17, 2026), captured Sep 21, 2026.</em></p>
<p>All of this lands on an approval patchwork keyed to which token opened the pull request. Copilot coding agent workflows “do not run until approved by a human with the Approve and run workflows button,” and <a href="https://github.blog/changelog/2026-03-13-optionally-skip-approval-for-copilot-coding-agent-actions-workflows/">since Mar 13</a> admins can switch that off per repository. PRs that a workflow creates or updates with <code>GITHUB_TOKEN</code> produce runs “in an approval-required state,” per <a href="https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow">GitHub’s trigger docs</a>; since Jun 11 such bot PRs can run at all, once someone with write access approves. A PR created or updated by automation with a GitHub App installation token or a PAT runs its <code>pull_request</code> workflows “without the approval prompt described above.” Same agent, three answers; the token decides.</p>
<h2 id="an-agents-push-is-a-run-request-with-the-secrets-attached">An agent’s push is a run request with the secrets attached</h2>
<p>GitHub’s <a href="https://docs.github.com/en/actions/reference/security/secure-use">secure-use reference</a> states the stakes: anyone with write access can read every secret configured in the repository. For a human collaborator, that is a trust decision someone made. For an agent, it is a side effect of the push access it needed.</p>
<p>Editing a workflow file is the obvious attack, and Step 0 drills it. The common path needs no workflow edit. An agent that changes a test script, a <code>package.json</code> script or a Makefile has changed what an existing, secret-bearing push workflow executes, and that workflow runs on the push.</p>
<p>Merge-time controls such as the <a href="/intel/overnight-agent-merge-gates/">overnight merge gates for agent PRs</a> and the <a href="/intel/agentic-cicd/">CODEOWNERS and secret-scanning gates in agentic CI/CD</a> judge the change after CI has already run it. So does GitHub’s Sep 9 rule, in public preview, that blocks merging pull requests with unresolved secret-scanning alerts.</p>
<p>The <a href="/intel/ci-agent-fix-loop-guards/">fix-loop guards</a> scope the agent’s own token, and the <a href="/intel/headless-agent-trust-tier/">headless trust tier</a> decides who may trigger an agent run. This runbook covers the inverse: which CI runs an agent identity can start, and what they reach. It assumes one GitHub App per lane, as in <a href="/intel/agent-service-principal-sso/">the service-principal playbook for agent identity</a>. With one shared identity, every row of the matrix collapses into one.</p>
<h2 id="step-0-remove-edit-rights-and-dispatch-rights-as-two-separate-drills">Step 0: Remove edit rights and dispatch rights as two separate drills</h2>
<p>Workflow edits and dispatch are separate permissions on Apps and fine-grained tokens, so remove and drill them one at a time.</p>
<p><strong>Edit rights.</strong> Take the <code>workflows</code> permission off every agent GitHub App and fine-grained token, and the <code>workflow</code> scope off every classic PAT an agent holds. GitHub’s <a href="https://docs.github.com/en/rest/authentication/permissions-required-for-github-apps">permission table for Apps</a> files the writes that touch workflow files under “Workflows”: <code>PUT</code> and <code>DELETE</code> on contents, git ref writes and release writes. The drill: from the agent’s runtime, with its production token, push a branch that changes a file under <code>.github/workflows/</code>. The push must be rejected.</p>
<p><strong>Dispatch rights.</strong> Take Actions write off every agent App and fine-grained token. The dispatch endpoint, <code>POST /repos/{owner}/{repo}/actions/workflows/{workflow_id}/dispatches</code>, sits under repository “Actions” write for Apps and <a href="https://docs.github.com/en/rest/authentication/permissions-required-for-fine-grained-personal-access-tokens">fine-grained tokens</a> alike. The drill: <code>gh workflow run</code> with the agent’s token must fail. Classic PATs are the exception. The <a href="https://docs.github.com/en/rest/actions/workflows#create-a-workflow-dispatch-event">dispatch reference</a> says “OAuth tokens and personal access tokens (classic) need the repo scope to use this endpoint,” so any classic PAT holding <code>repo</code> can dispatch any workflow that declares <code>workflow_dispatch</code>. Move PAT agents to an App or a fine-grained token, or deny them <code>workflow_dispatch</code> by policy in Step 3.</p>
<p><strong>Admin rights.</strong> The Actions policies API needs Administration write, even for a <code>GET</code>. An agent token that can read your policies can also rewrite them. No lane holds one.</p>
<table>
<thead>
<tr>
<th>Token type</th>
<th>Edit right to remove</th>
<th>Dispatch right to remove</th>
<th>The drill must show</th>
</tr>
</thead>
<tbody>
<tr>
<td>GitHub App, one per lane</td>
<td>“Workflows” permission</td>
<td>“Actions” write</td>
<td>workflow-file push rejected; <code>gh workflow run</code> refused</td>
</tr>
<tr>
<td>Fine-grained PAT</td>
<td>“Workflows” write</td>
<td>“Actions” write</td>
<td>same as an App</td>
</tr>
<tr>
<td>Classic PAT</td>
<td><code>workflow</code> scope</td>
<td>none separate: <code>repo</code> alone can dispatch</td>
<td>push rejected; a dispatched run fails only once a policy denies the event</td>
</tr>
<tr>
<td>Any agent token</td>
<td>never grant “Administration” write</td>
<td>not applicable</td>
<td><code>GET</code> on the policies API refused</td>
</tr>
</tbody>
</table>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># edit-dispatch-drill.sh: illustrative. Run inside the agent lane's runtime with the</span></span>
<span class="line"><span style="color:#6A737D"># token that lane receives in production, against a scratch repo nothing depends on.</span></span>
<span class="line"><span style="color:#6A737D"># The scratch repo holds one workflow that declares workflow_dispatch; git must</span></span>
<span class="line"><span style="color:#6A737D"># authenticate with the same token (credential helper or remote URL).</span></span>
<span class="line"><span style="color:#79B8FF">set</span><span style="color:#79B8FF"> -u</span></span>
<span class="line"><span style="color:#E1E4E8">REPO</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"your-org/trigger-drill"</span></span>
<span class="line"><span style="color:#E1E4E8">WF</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"drill-dispatch.yml"</span></span>
<span class="line"><span style="color:#F97583">export</span><span style="color:#E1E4E8"> GH_TOKEN</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">$AGENT_TOKEN</span><span style="color:#9ECBFF">"</span></span>
<span class="line"></span>
<span class="line"><span style="color:#6A737D"># 1. Edit: a push that touches .github/workflows/ must be rejected.</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#9ECBFF"> switch</span><span style="color:#79B8FF"> -c</span><span style="color:#9ECBFF"> "drill/wf-edit-$(</span><span style="color:#B392F0">date</span><span style="color:#9ECBFF"> +%s)"</span></span>
<span class="line"><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "# drill $(</span><span style="color:#B392F0">date</span><span style="color:#79B8FF"> -u</span><span style="color:#9ECBFF"> +%FT%TZ)"</span><span style="color:#F97583"> &gt;&gt;</span><span style="color:#9ECBFF"> ".github/workflows/</span><span style="color:#E1E4E8">$WF</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#9ECBFF"> commit</span><span style="color:#79B8FF"> -qam</span><span style="color:#9ECBFF"> "drill: workflow edit from agent identity"</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#9ECBFF"> push</span><span style="color:#79B8FF"> -q</span><span style="color:#9ECBFF"> origin</span><span style="color:#9ECBFF"> HEAD</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "FAIL edit: push accepted"</span><span style="color:#F97583"> ||</span><span style="color:#79B8FF"> echo</span><span style="color:#9ECBFF"> "PASS edit: push rejected"</span></span>
<span class="line"></span>
<span class="line"><span style="color:#6A737D"># 2. Dispatch: creating a workflow_dispatch event must fail.</span></span>
<span class="line"><span style="color:#B392F0">gh</span><span style="color:#9ECBFF"> workflow</span><span style="color:#9ECBFF"> run</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$WF</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> --repo</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "FAIL dispatch: run created"</span><span style="color:#F97583"> ||</span><span style="color:#79B8FF"> echo</span><span style="color:#9ECBFF"> "PASS dispatch refused"</span></span>
<span class="line"></span>
<span class="line"><span style="color:#6A737D"># 3. Admin: the policies API needs Administration write; an agent token must not read it.</span></span>
<span class="line"><span style="color:#B392F0">gh</span><span style="color:#9ECBFF"> api</span><span style="color:#9ECBFF"> "repos/</span><span style="color:#E1E4E8">$REPO</span><span style="color:#9ECBFF">/actions/policies"</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF"> /dev/null</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "FAIL admin: policies readable"</span><span style="color:#F97583"> ||</span><span style="color:#79B8FF"> echo</span><span style="color:#9ECBFF"> "PASS admin refused"</span></span></code></pre>
<p>Run it for every lane now and after every token rotation, and keep the exact error each refused call returns.</p>
<h2 id="step-1-write-the-agent-trigger-matrix-and-name-the-control-in-every-cell">Step 1: Write the agent trigger matrix, and name the control in every cell</h2>
<p>The matrix is a checked-in file the platform team owns, reviewed like any change to <code>.github/workflows/</code>.</p>
<p><strong>Rows are the identities agents push or dispatch as.</strong> Five kinds cover most fleets:</p>
<ul>
<li><strong>Copilot</strong>, which actor rules name directly.</li>
<li><strong>One GitHub App per lane</strong>, so a review agent and a dependency agent get different cells.</li>
<li><strong>PAT agents.</strong> The docs don’t say how a PAT-authenticated push is attributed for actor rules, so test whether a policy sees the agent or the token’s human owner.</li>
<li><strong><code>github-actions[bot]</code></strong>, for pull requests your own workflows open.</li>
<li><strong>Dependabot.</strong> GitHub’s own features are exempt, but bot identities such as <code>dependabot[bot]</code> must be added as allowed actors when a custom workflow needs them.</li>
</ul>
<p><strong>Columns are workflows grouped by reach, crossed with the event:</strong> no secrets, repository secrets, environment secrets, OIDC deploy roles, each split by <code>push</code>, <code>pull_request</code>, <code>pull_request_target</code> and <code>workflow_dispatch</code>.</p>
<p><strong>Every cell gets a verdict (allow, approval-required or deny) and its enforcing control.</strong> Execution protections only allow or block; they have no ask. So an approval-required cell means the run may start and the job holding the secret runs in an environment with required reviewers, because “A workflow job cannot access environment secrets until approval is granted by a reviewer.” An approval cell that names no environment is an allow.</p>
<p>The approval prompt is real but narrow. It holds Copilot coding agent runs until an admin turns it off, and runs from <code>GITHUB_TOKEN</code>-created PRs, but never a PR that automation opened with an App token or PAT. Never make it the only control on a secret-bearing column.</p>
<table>
<thead>
<tr>
<th>Identity, <code>push</code> event (illustrative)</th>
<th>No secrets</th>
<th>Repo secrets</th>
<th>Environment secrets</th>
<th>OIDC deploy role</th>
</tr>
</thead>
<tbody>
<tr>
<td>Copilot coding agent</td>
<td>allow · EP</td>
<td>deny · EP</td>
<td>approval · env reviewers</td>
<td>deny · EP</td>
</tr>
<tr>
<td>Lane App: review agent</td>
<td>allow · EP</td>
<td>deny · EP</td>
<td>approval · env reviewers</td>
<td>deny · EP</td>
</tr>
<tr>
<td>PAT agent, attribution untested</td>
<td>deny · replace with an App</td>
<td>deny · replace with an App</td>
<td>deny · replace with an App</td>
<td>deny · replace with an App</td>
</tr>
<tr>
<td><code>github-actions[bot]</code></td>
<td>allow · EP + approval prompt</td>
<td>deny · EP</td>
<td>approval · env reviewers</td>
<td>deny · EP</td>
</tr>
<tr>
<td><code>dependabot[bot]</code></td>
<td>allow · EP, listed actor</td>
<td>deny · EP</td>
<td>deny · EP</td>
<td>deny · EP</td>
</tr>
</tbody>
</table>
<p>EP is an execution-protection policy, and <code>workflow_dispatch</code> is deny for every agent row. The file looks something like this:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># ci/agent-trigger-matrix.yaml: illustrative shape. Owned by the platform team;</span></span>
<span class="line"><span style="color:#6A737D"># reviewed like .github/workflows/. EP = execution-protection policy.</span></span>
<span class="line"><span style="color:#85E89D">identities</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  copilot</span><span style="color:#E1E4E8">:          { </span><span style="color:#85E89D">kind</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">copilot</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">  review-agent</span><span style="color:#E1E4E8">:     { </span><span style="color:#85E89D">kind</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">app</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">app</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">acme-review-agent</span><span style="color:#E1E4E8"> }   </span><span style="color:#6A737D"># one App per lane</span></span>
<span class="line"><span style="color:#85E89D">  deps-agent</span><span style="color:#E1E4E8">:       { </span><span style="color:#85E89D">kind</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">app</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">app</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">acme-deps-agent</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">  legacy-pat-agent</span><span style="color:#E1E4E8">: { </span><span style="color:#85E89D">kind</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">pat</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">owner</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">a-human-login</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">attribution</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">untested</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">  actions-bot</span><span style="color:#E1E4E8">:      { </span><span style="color:#85E89D">kind</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">bot</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">login</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"github-actions[bot]"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">  dependabot</span><span style="color:#E1E4E8">:       { </span><span style="color:#85E89D">kind</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">bot</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">login</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"dependabot[bot]"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">workflow_groups</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  no-secrets</span><span style="color:#E1E4E8">:   [</span><span style="color:#9ECBFF">.github/workflows/lint.yml</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">.github/workflows/unit.yml</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  repo-secrets</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">.github/workflows/integration.yml</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  env-secrets</span><span style="color:#E1E4E8">:  [</span><span style="color:#9ECBFF">.github/workflows/release.yml</span><span style="color:#E1E4E8">]    </span><span style="color:#6A737D"># release job declares environment: release</span></span>
<span class="line"><span style="color:#85E89D">  oidc-deploy</span><span style="color:#E1E4E8">:  [</span><span style="color:#9ECBFF">.github/workflows/deploy.yml</span><span style="color:#E1E4E8">]     </span><span style="color:#6A737D"># deploy job declares environment: production</span></span>
<span class="line"><span style="color:#85E89D">cells</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">  - { </span><span style="color:#85E89D">who</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">copilot</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">review-agent</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">deps-agent</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">dependabot</span><span style="color:#E1E4E8">], </span><span style="color:#85E89D">group</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">no-secrets</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">event</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">push</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">pull_request</span><span style="color:#E1E4E8">], </span><span style="color:#85E89D">verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">allow</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">control</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">EP</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#E1E4E8">  - { </span><span style="color:#85E89D">who</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">copilot</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">review-agent</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">deps-agent</span><span style="color:#E1E4E8">], </span><span style="color:#85E89D">group</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">env-secrets</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">event</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">push</span><span style="color:#E1E4E8">], </span><span style="color:#85E89D">verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">approval</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">control</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"EP allow + environment release with required reviewers"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#E1E4E8">  - { </span><span style="color:#85E89D">who</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">agents</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">group</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">repo-secrets</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">oidc-deploy</span><span style="color:#E1E4E8">], </span><span style="color:#85E89D">event</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">all</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">deny</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">control</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">EP</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#E1E4E8">  - { </span><span style="color:#85E89D">who</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">agents</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">group</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">all</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">event</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">workflow_dispatch</span><span style="color:#E1E4E8">], </span><span style="color:#85E89D">verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">deny</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">control</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"EP + Step 0 token scopes"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#E1E4E8">  - { </span><span style="color:#85E89D">who</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">legacy-pat-agent</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">group</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">all</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">event</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">all</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">deny</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">control</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"replace with an App before any policy relies on it"</span><span style="color:#E1E4E8"> }</span></span></code></pre>
<h2 id="step-2-move-secrets-out-of-reach-of-runs-an-agent-can-start">Step 2: Move secrets out of reach of runs an agent can start</h2>
<p>
<em>Two gates in series. The allowlist decides whether the run starts; the environment decides whether a job gets its secrets.</em></p>
<p>Repository secrets sit between the two gates, open to any job in a run that started, which makes the repo-secrets column of the matrix the one that matters most.</p>
<p>For each workflow an agent identity can start, list the secrets it reads. Delete what it doesn’t need. Move what it needs only for a release, deploy or publish step into an environment with required reviewers, as a separate job that declares that environment.</p>
<p>Set the default <code>GITHUB_TOKEN</code> to read-only, as the secure-use reference advises, and grant write per job. Treat an OIDC deploy role the same way: assume it only inside a reviewed environment, never in the job that runs the agent’s test script.</p>
<p>Then re-sort the columns. A workflow that no longer reads repository secrets moves to the no-secrets column and can stay open to agent pushes. Most of this runbook’s risk reduction happens here, before a single policy exists.</p>
<h2 id="step-3-encode-the-allow-and-deny-cells-as-github-actions-workflow-execution-protections">Step 3: Encode the allow and deny cells as GitHub Actions workflow execution protections</h2>
<p>Only the allow and deny cells become policies. Approval cells are the environment settings from Step 2.</p>
<p>The <a href="https://docs.github.com/en/rest/actions/policies">REST reference for Actions policies</a> gives the fields: <code>name</code>; <code>enforcement</code>, one of <code>disabled</code>, <code>active</code> or <code>evaluate</code>; <code>rules</code>, holding <code>restrict_actions_actors</code> and <code>restrict_action_events</code>; and <code>conditions</code>, using <code>repository_name</code>, <code>repository_id</code>, <code>repository_property</code> or <code>workflow_path</code>. Actor types are User, Bot, Team and App. Three rules keep the translation honest.</p>
<p><strong>Target every policy by workflow file.</strong> The reference says why: “Omitting workflow_path targets all workflows without storing an explicit condition.”</p>
<p><strong>Split before you deny dispatch.</strong> Per the how-to, “If you also restrict events, these users will only be able to trigger workflows with the allowed events.” A policy’s allowed actors get only its allowed events. If one workflow file takes agent pushes and human dispatches, allowing both events lets agents dispatch, and allowing only <code>push</code> takes dispatch away from the humans too. Split it into a push workflow agents may start and a dispatch workflow only humans may start, then deny <code>workflow_dispatch</code> to agent identities on the second.</p>
<p><strong>Shadow it where you can.</strong> Run a week in evaluate mode if you have GitHub Enterprise Cloud; elsewhere, put one repository in <code>active</code> and run the Step 4 drill the same day. Whether non-enterprise organizations can view policy insights is disputed, so check your own org first.</p>
<p>Apply the policy files from a human-owned pipeline that holds the Administration-write credential, never from a lane.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># policies/agent-trigger.yaml: illustrative intent file. A human-owned pipeline</span></span>
<span class="line"><span style="color:#6A737D"># with Administration write translates each entry into the REST fields</span></span>
<span class="line"><span style="color:#6A737D"># (name, enforcement, rules, conditions) and applies it.</span></span>
<span class="line"><span style="color:#E1E4E8">- </span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">release-deploy-humans-only</span></span>
<span class="line"><span style="color:#85E89D">  scope</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">organization</span></span>
<span class="line"><span style="color:#85E89D">  enforcement</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">evaluate</span><span style="color:#6A737D">          # Enterprise Cloud only; elsewhere one repo in active first</span></span>
<span class="line"><span style="color:#85E89D">  conditions</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    workflow_path</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">.github/workflows/release-dispatch.yml</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">.github/workflows/deploy.yml</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  allowed_actors</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">team:release-managers</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  allowed_events</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">push</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">workflow_dispatch</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#E1E4E8">- </span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">tests-agents-may-start</span></span>
<span class="line"><span style="color:#85E89D">  scope</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">organization</span></span>
<span class="line"><span style="color:#85E89D">  enforcement</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">evaluate</span></span>
<span class="line"><span style="color:#85E89D">  conditions</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    workflow_path</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">.github/workflows/lint.yml</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">.github/workflows/unit.yml</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  allowed_actors</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">team:engineering</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">app:acme-review-agent</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">app:acme-deps-agent</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">copilot</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"bot:dependabot[bot]"</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  allowed_events</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">push</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">pull_request</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#E1E4E8">- </span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">kill-switch-agents-off-sensitive</span></span>
<span class="line"><span style="color:#85E89D">  scope</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">organization</span></span>
<span class="line"><span style="color:#85E89D">  enforcement</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">disabled</span><span style="color:#6A737D">          # a human flips this to active (Step 5)</span></span>
<span class="line"><span style="color:#85E89D">  conditions</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    repository_property</span><span style="color:#E1E4E8">: { </span><span style="color:#85E89D">sensitivity</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">high</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">  allowed_actors</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">team:engineering</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  allowed_events</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">push</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">pull_request</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">workflow_dispatch</span><span style="color:#E1E4E8">]</span></span></code></pre>
<h2 id="step-4-drill-from-every-agent-identity-and-write-down-what-you-see">Step 4: Drill from every agent identity, and write down what you see</h2>
<p>A policy you haven’t drilled is a guess. From each agent identity, try each denied cell: push to a branch that triggers a denied workflow, open a PR, dispatch. Add a positive control, a human dispatch that must still run, so you notice when you have denied too much.</p>
<p>Expect a failed run with an error. GitHub documents the text only for event blocks: “Event ‘workflow_dispatch’ is not allowed to trigger Actions workflows. Workflow file: ‘.github/workflows/0-welcome.yml’.” The docs show no text for an actor-rule block, so record the exact error each run gives you rather than matching on a string.</p>
<p>
<em>Screenshot: GitHub Docs, “About Actions policies” (undated), captured Sep 21, 2026.</em></p>
<p>Two tests cover what the docs don’t say. The first finds which identity counts as the actor on a pull request <code>synchronize</code> push, both when an agent pushes to a human’s PR and the reverse. The second finds whether a PAT agent’s run is attributed to the agent or to the token’s owner. Let both answers rewrite the PAT and bot rows.</p>
<table>
<thead>
<tr>
<th>Identity</th>
<th>Workflow</th>
<th>Event</th>
<th>Expected</th>
<th>Observed</th>
<th>Error text, verbatim</th>
<th>Date</th>
</tr>
</thead>
<tbody>
<tr>
<td>app: acme-review-agent</td>
<td>integration.yml</td>
<td><code>push</code></td>
<td>blocked</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>app: acme-review-agent</td>
<td>release-dispatch.yml</td>
<td><code>workflow_dispatch</code></td>
<td>blocked</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>app: acme-review-agent</td>
<td>unit.yml</td>
<td><code>pull_request</code> synchronize on a human’s PR</td>
<td>record the actor</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>PAT agent</td>
<td>unit.yml</td>
<td><code>push</code></td>
<td>record the actor</td>
<td></td>
<td></td>
<td></td>
</tr>
<tr>
<td>human release manager</td>
<td>release-dispatch.yml</td>
<td><code>workflow_dispatch</code></td>
<td>runs (positive control)</td>
<td></td>
<td></td>
<td></td>
</tr>
</tbody>
</table>
<p>Re-run the log after every policy change, token rotation or new lane. I’d keep it beside the matrix, so a reviewer sees a cell change and its drill result together.</p>
<h2 id="step-5-time-an-org-level-kill-switch-for-agent-actors">Step 5: Time an org-level kill switch for agent actors</h2>
<p>Write one more organization policy before you need it. It targets every sensitive workflow, by <code>workflow_path</code> or by a <code>repository_property</code> such as a sensitivity tag, allows human actors only, and sits in <code>disabled</code> (the last entry in the Step 3 file). When a lane misbehaves, a person flips it to <code>active</code> and every agent identity drops off those workflows.</p>
<p>Don’t assume the switch is instant. Measure it: minutes from the decision to the first blocked run from an agent identity. In the same drill, prove the switch blocks an agent that another policy allows.</p>
<p>Policies are evaluated before a run starts, so the switch stops new runs, and runs already in flight need a separate cancel. Rehearse it each quarter with nothing at stake, and keep the credential that flips it with a human.</p>
<h2 id="step-6-before-nov-2-list-agent-workflows-on-pull_request_target-in-public-repos">Step 6: Before Nov 2, list agent workflows on pull_request_target in public repos</h2>
<p>
<em>Every date is from GitHub’s changelog. Nov 2 is scheduled, not yet in force, and covers affected public repositories only.</em></p>
<p>The default rule reaches only public repositories that were on the default <code>pull_request_target</code> policy, and it blocks only one event, so the inventory is small enough to finish this week. GitHub’s secure-use reference already warns that “The <code>pull_request_target</code> and <code>workflow_run</code> workflow triggers, when used with the checkout of an untrusted pull request, expose the repository to security compromises.” Agent review and triage jobs are prime candidates.</p>
<p>For each public repository:</p>
<ol>
<li>List the workflows triggered by <code>pull_request_target</code> (script below).</li>
<li>Mark the ones that run an agent: review bots, triage lanes, auto-fix jobs.</li>
<li>Check whether the repository already has an event policy. If it does, the default rule doesn’t apply, and your policy decides.</li>
<li>Decide per workflow: let the rule block it and move the agent to <code>pull_request</code> with no secrets, or write an explicit event policy that allows <code>pull_request_target</code> for that one workflow file and review what it checks out.</li>
<li>Where insights are visible to you, compare your list with what evaluate mode says would fail.</li>
</ol>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># prt-inventory.sh: illustrative. Lists public repos whose workflow files mention</span></span>
<span class="line"><span style="color:#6A737D"># pull_request_target, for a human to sort into agent and non-agent workflows.</span></span>
<span class="line"><span style="color:#E1E4E8">ORG</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"your-org"</span></span>
<span class="line"><span style="color:#B392F0">gh</span><span style="color:#9ECBFF"> repo</span><span style="color:#9ECBFF"> list</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$ORG</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> --visibility</span><span style="color:#9ECBFF"> public</span><span style="color:#79B8FF"> --limit</span><span style="color:#79B8FF"> 1000</span><span style="color:#79B8FF"> --json</span><span style="color:#9ECBFF"> name</span><span style="color:#79B8FF"> --jq</span><span style="color:#9ECBFF"> '.[].name'</span><span style="color:#F97583"> |</span></span>
<span class="line"><span style="color:#F97583">while</span><span style="color:#9ECBFF"> read</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> repo</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#B392F0">  gh</span><span style="color:#9ECBFF"> api</span><span style="color:#9ECBFF"> "repos/</span><span style="color:#E1E4E8">$ORG</span><span style="color:#9ECBFF">/</span><span style="color:#E1E4E8">$repo</span><span style="color:#9ECBFF">/contents/.github/workflows"</span><span style="color:#79B8FF"> --jq</span><span style="color:#9ECBFF"> '.[].path'</span><span style="color:#F97583"> 2&gt;</span><span style="color:#9ECBFF"> /dev/null</span><span style="color:#F97583"> |</span></span>
<span class="line"><span style="color:#F97583">  while</span><span style="color:#9ECBFF"> read</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> path</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#B392F0">    gh</span><span style="color:#9ECBFF"> api</span><span style="color:#9ECBFF"> "repos/</span><span style="color:#E1E4E8">$ORG</span><span style="color:#9ECBFF">/</span><span style="color:#E1E4E8">$repo</span><span style="color:#9ECBFF">/contents/</span><span style="color:#E1E4E8">$path</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> --jq</span><span style="color:#9ECBFF"> '.content'</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> base64</span><span style="color:#79B8FF"> -d</span><span style="color:#F97583"> |</span></span>
<span class="line"><span style="color:#B392F0">      grep</span><span style="color:#79B8FF"> -q</span><span style="color:#9ECBFF"> 'pull_request_target'</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$repo</span><span style="color:#E1E4E8">  $path</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#F97583">  done</span></span>
<span class="line"><span style="color:#F97583">done</span></span></code></pre>
<h2 id="where-github-actions-workflow-execution-protections-leak-and-the-signal-for-each">Where GitHub Actions workflow execution protections leak, and the signal for each</h2>
<p><strong>The secret-bearing push workflow nobody moved.</strong> Signal: an agent commit that changes <code>package.json</code> scripts, a Makefile or a test helper on a branch whose push workflow reads repository secrets. Fix: Step 2 for that workflow, today.</p>
<p><strong>A PAT agent wearing its owner’s identity.</strong> Signal: the Step 4 drill from a PAT agent succeeds where the same cell blocks the lane App. Fix: move the agent to an App; if the policy sees the human, no actor rule can tell the two apart.</p>
<p><strong>The untargeted policy.</strong> Signal: human runs blocked on workflows you never meant to touch. Cause: a policy with no <code>workflow_path</code>, which targets everything.</p>
<p><strong>The new workflow file.</strong> Signal: a workflow file added since the last policy review that no targeted policy names. Fix: a daily diff of workflow files against policy targets, the same coverage assert <a href="/intel/tool-schema-guard-contract-tests/">the guard contract suite</a> runs against hook matchers.</p>
<p><strong>A gate that isn’t there.</strong> Execution protections are a guardrail on run starts, not a complete boundary. A policy left in <code>disabled</code> or evaluate, a workflow no targeted policy names, GitHub’s exempt own features and a run that started before the switch flipped all get through. The wall behind the allowlist is Step 0’s tokens that can’t edit or dispatch and Step 2’s secrets that wait for a reviewer. Keep both even when the policies look complete.</p>
<h2 id="trigger-rights-belong-in-the-fleets-identity-inventory">Trigger rights belong in the fleet’s identity inventory</h2>
<p>The matrix is an inventory record first and a GitHub setting second. Every agent identity should appear in it the day it is created, next to its token scopes and publish rights; the <a href="/intel/agent-stage-only-publish-tokens/">stage-only token classes for publish lanes</a> are the same exercise for npm. A lane that reads untrusted text and can start a secret-bearing workflow holds all three legs of <a href="/intel/rule-of-two-lane-split/">the Rule of Two</a>, and the matrix is where you will see it first.</p>
<p>That view belongs in the <a href="/intel/multi-agent-command-center/">command center that runs the fleet</a>, beside the kill switch and the drill log, not on a settings page three clicks into one organization. GitHub now supplies a native allow and block. Deciding which identity gets which run, and proving it still holds after next month’s new lane, stays your job.</p>
<h2 id="faq">FAQ</h2>
<h3 id="can-a-coding-agent-with-write-access-trigger-github-actions-workflows">Can a coding agent with write access trigger GitHub Actions workflows?</h3>
<p>Often, yes. Until Sep 17, GitHub’s rule was that every user with write access could trigger workflows, and only some agent tokens hit an approval prompt. Workflow execution protections now let you allowlist the actors and events that may start each workflow file, so you decide which runs each agent identity can start.</p>
<h3 id="does-githubs-default-pull_request_target-rule-apply-to-private-repositories">Does GitHub’s default pull_request_target rule apply to private repositories?</h3>
<p>No. GitHub says the default rule does not apply to private or internal repositories. Automatic enforcement on Nov 2 reaches only public repositories with no event policy that were on the default <code>pull_request_target</code> policy before general availability. Where a public repository has an event policy, the default rule doesn’t apply and your policy decides.</p>
<h3 id="can-a-classic-personal-access-token-dispatch-github-actions-workflows">Can a classic personal access token dispatch GitHub Actions workflows?</h3>
<p>Yes. GitHub’s dispatch reference says classic personal access tokens need the <code>repo</code> scope, so any classic PAT holding <code>repo</code> can dispatch any workflow that declares <code>workflow_dispatch</code>. There is no separate dispatch right to remove. Move PAT agents to a GitHub App or fine-grained token, or deny them <code>workflow_dispatch</code> with an execution-protection policy.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://github.blog/changelog/2026-09-17-workflow-execution-protections-in-github-actions-generally-available/">GitHub Changelog: Workflow execution protections in GitHub Actions generally available</a> (Sep 17, 2026)</li>
<li><a href="https://github.blog/changelog/2026-03-13-optionally-skip-approval-for-copilot-coding-agent-actions-workflows/">GitHub Changelog: Optionally skip approval for Copilot coding agent Actions workflows</a> (Mar 13, 2026)</li>
<li><a href="https://docs.github.com/en/actions/concepts/about-actions-policies">GitHub Docs: About Actions policies</a> (rule types, event-block error)</li>
<li><a href="https://docs.github.com/en/actions/how-tos/administer/control-workflow-execution">GitHub Docs: Controlling who can execute GitHub Actions workflows</a> (evaluate mode, layering)</li>
<li><a href="https://docs.github.com/en/actions/reference/security/secure-use">GitHub Docs: Secure use reference</a> (secrets, <code>pull_request_target</code>, environments)</li>
<li><a href="https://docs.github.com/en/actions/how-tos/write-workflows/choose-when-workflows-run/trigger-a-workflow">GitHub Docs: Triggering a workflow</a> (approval prompt by token type)</li>
<li><a href="https://docs.github.com/en/rest/actions/policies">GitHub REST API: Actions policies</a> (policy fields)</li>
<li><a href="https://docs.github.com/en/rest/actions/workflows#create-a-workflow-dispatch-event">GitHub REST API: Create a workflow dispatch event</a> (classic PAT <code>repo</code> scope)</li>
<li><a href="https://docs.github.com/en/rest/authentication/permissions-required-for-github-apps">GitHub REST API: Permissions required for GitHub Apps</a> (Actions, Workflows)</li>
<li><a href="https://docs.github.com/en/rest/authentication/permissions-required-for-fine-grained-personal-access-tokens">GitHub REST API: Permissions required for fine-grained personal access tokens</a> (Actions, Administration)</li>
</ul>
]]></content:encoded></item><item><title>Your Vendor Lane May Not Be the Last Hop</title><link>https://automater.ai/intel/vendor-lane-last-hop-register/</link><guid>https://automater.ai/intel/vendor-lane-last-hop-register/</guid><description>AI vendor data routing can carry your prompt past the vendor you signed with. Register each lane&apos;s last hop, then cap its data class at what you can verify.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>Anthropic says that over one ten-day stretch, almost 300,000 requests from Moonshot’s customers were answered by Claude instead of Kimi, and that Moonshot showed those customers Claude’s responses. That is Anthropic’s allegation, published Sep 10, and no response from Moonshot has surfaced. The problem it puts in front of anyone running agents doesn’t wait for one: when a lane sends a prompt to a vendor, that vendor may not be the last hop.</p>
<p>AI vendor data routing is invisible from where your runner sits. Your egress log shows the host the runner called. It can’t show whether that host answered, forwarded the request to another provider, or pushed it through someone else’s account and kept a copy. You can see the first hop. Everything after it is a contract or a claim.</p>
<p>This piece turns that gap into a register. By Tuesday every vendor lane in your fleet has one row with five columns: the contracting entity, its stated sub-processors, its onward-routing and resale terms, whether a response names the model that answered, and what you verified versus what you were told. Each lane then carries a data-class cap set by what its last hop can be verified to be, and your runner refuses to send a restricted repo to a lane whose last hop is only a claim.</p>
<p>Chatbots suggest; agents act. A chat user pastes one paragraph. An agent lane ships whatever it can reach, tool output and environment included, on every turn, with nobody reading along, so where its prompts end up is a property of the lane, not of any one conversation.</p>
<h2 id="sep-10-anthropics-threat-report-alleges-relays-customers-never-saw">Sep 10: Anthropic’s threat report alleges relays customers never saw</h2>
<p>Anthropic’s <a href="https://www.anthropic.com/threat-intelligence-report-september-2026">threat-intelligence report</a>, “Detecting and countering misuse of AI”, covers December 2025 to August 2026. The page says only “September 2026”; TechCrunch and CNBC date its release to Thursday, Sep 10. Where most of the report hedges, the distillation section doesn’t: Anthropic says it attributed those campaigns “with high confidence to specific PRC-based labs”.</p>
<p>The Moonshot case, labelled GTG-16002, is the one a register cares about. Anthropic says Moonshot “silently forwarded customer requests to Claude, instead of processing them using Kimi”, and puts a number on one window: “In one instance, over a ten-day period, Moonshot relayed almost 300,000 customer requests to Anthropic, the vast majority of which were routed to Opus.” By Anthropic’s account the traffic ran through a proxy network of 5,380 fraudulent accounts, most of which appeared to be in Singapore and Japan. Anthropic also says Moonshot “captured and saved at least a portion of these exchanges”, and that the rerouted queries included sensitive information about Moonshot’s customers. And it names what it doesn’t know: “We do not know if Moonshot notified their customers that their requests were being rerouted to Anthropic and exposed to a third party.”</p>
<p>
<em>Screenshot: Anthropic, “Countering misuse of AI: September 2026” (September 2026), captured Sep 21, 2026.</em></p>
<p>Moonshot is not the only case in the report, and the others sit closer to how fleets buy access. Anthropic says DeepSeek silently relayed exchanges to Claude, including requests from users working in coding harnesses such as Claude Code, the Claude Agent SDK and OpenCode. It says Xiaomi replayed user sessions, many of them routed through third-party model routing services; that SenseTime bought transcripts from intermediaries that “logged the transcripts and sold them”; and that MiniMax built a proxy network through a shell company. The general line reads like a note to operators: “Many of these exchanges were relayed from users of third-party model routing services commonly used by users in the United States and Europe.”</p>
<p>The totals need care. <a href="https://techcrunch.com/2026/09/10/anthropic-details-distillation-campaigns-from-alibaba-moonshot-ai-and-deepseek/">TechCrunch</a> counts “nearly 200 million exchanges” across five campaigns; Anthropic’s page gives no grand total, and its per-lab scale lines add up to about 190 million. <a href="https://www.cnbc.com/2026/09/11/chinese-ai-labs-moonshot-deepseek-alibaba-anthropic.html">CNBC</a> reported that Alibaba, Moonshot, DeepSeek, Xiaomi and Anthropic “did not immediately respond” to its requests for comment, and no Moonshot statement had appeared by Sep 21.</p>
<p>Read the gaps as carefully as the claims. For the Moonshot case the report names no product surface (not Kimi Code, not the Kimi CLI), doesn’t say how long anything was kept, and doesn’t say whether API or enterprise customers were affected. Nothing in it says a particular lane leaked, and nothing here claims one did. Every allegation above is Anthropic’s. It is also Anthropic’s second pass: its February disclosure named three of the same labs.</p>
<aside class="social-embed" data-x-embed data-post-id="2025997928242811253">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">February 23, 2026 — Anthropic's February 2026 distillation disclosure naming DeepSeek, Moonshot and MiniMax, the precedent for the September report rather than the report itself.</p>
  <a class="social-embed__source" href="https://x.com/AnthropicAI/status/2025997928242811253" target="_blank" rel="noopener noreferrer">View Anthropic's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="step-1-map-ai-vendor-data-routing-with-one-five-column-row-per-lane">Step 1: Map AI vendor data routing with one five-column row per lane</h2>
<p>A lane, for this register, is one path a prompt can take: harness, endpoint, account and plan. The same vendor on a consumer plan and on a signed API agreement is two rows, because the terms are two different documents. Routers, gateways, resellers and BYOK proxies are rows too. Any hop you configure gets one, including the one a developer added last week to save money.</p>
<table>
<thead>
<tr>
<th>Column</th>
<th>What you record</th>
<th>Where it comes from</th>
</tr>
</thead>
<tbody>
<tr>
<td>1. Contracting entity</td>
<td>Legal name on the invoice and the terms, its parent, the host your runner calls</td>
<td>Invoice, order form, egress log</td>
</tr>
<tr>
<td>2. Stated sub-processors</td>
<td>Everyone the vendor says receives prompt content, with the list’s date or version</td>
<td>Data processing agreement annex, published list</td>
</tr>
<tr>
<td>3. Onward routing and resale</td>
<td>Whether the vendor may send your request to another model provider, log it, or sell or share transcripts, and whose account any upstream call runs on</td>
<td>Signed terms; otherwise the public terms</td>
</tr>
<tr>
<td>4. Model identity</td>
<td>Whether each response names the model that answered: exported per call, displayed on screen, or not exposed</td>
<td>Your probe (step 4)</td>
</tr>
<tr>
<td>5. Evidence grade</td>
<td>For columns 1 to 4: verified, contracted, attested, unknown or contradicted</td>
<td>Step 2</td>
</tr>
</tbody>
</table>
<p>Each column maps to a line in the report. The contracting entity matters because, by Anthropic’s account, MiniMax ran its proxy service through a shell company; the brand on the landing page and the entity on the invoice are separate facts. Sub-processors carry a date because the list is a snapshot. Onward routing is the Moonshot question. Resale and logging is the SenseTime question: Anthropic says intermediaries between users and the model logged the transcripts that were later sold.</p>
<p>Column 3 has a second half people skip: whose account the upstream call runs on. Anthropic’s report notes that “Stolen keys and accounts have resale value in established markets”, and it describes Moonshot’s relay as running on fraudulent accounts. A reseller that can’t say it holds its own agreement with each upstream is a hop whose access can vanish overnight and whose logs belong to a stranger.</p>
<p>
<em>Screenshot: Anthropic, “Countering misuse of AI: September 2026” (September 2026), captured Sep 21, 2026.</em></p>
<p>Two neighbouring controls stay out of this register. The keys a lane holds belong with <a href="/intel/agent-stage-only-publish-tokens/">stage-only tokens for agents</a>, and spend spikes on those keys belong with <a href="/intel/agent-cost-anomaly-alerts/">cost anomaly alerts</a>. If you keep the trust ledger from <a href="/intel/chinese-coding-clis/">the Chinese coding CLIs review</a>, with its jurisdiction, terms and paths, add these columns to it rather than starting a second file. <a href="/intel/chinese-frontier-models/">Trust, by usage path</a> classifies workloads by vendor; this register classifies them by where the vendor sends them. How long any hop keeps your prompt is <a href="/intel/zero-retention-vs-agent-memory/">a retention question with its own piece</a>.</p>
<h2 id="step-2-grade-every-cell-verified-contracted-attested-or-unknown">Step 2: Grade every cell verified, contracted, attested or unknown</h2>
<p>The register is only as honest as its grades, so define them before anyone fills a cell.</p>
<table>
<thead>
<tr>
<th>Grade</th>
<th>Means</th>
<th>The cell must point to</th>
</tr>
</thead>
<tbody>
<tr>
<td>Verified</td>
<td>You observed it in something you control</td>
<td>A log line, an invoice, a config you own</td>
</tr>
<tr>
<td>Contracted</td>
<td>A signed term with a notice duty and a remedy; not observed, but enforceable</td>
<td>The clause, the signature, the date</td>
</tr>
<tr>
<td>Attested</td>
<td>The vendor said so: a trust page, a questionnaire answer, a sales email, a model field in a response</td>
<td>The page or message and its date</td>
</tr>
<tr>
<td>Unknown</td>
<td>Nobody has said</td>
<td>Nothing, which is the point</td>
</tr>
<tr>
<td>Contradicted</td>
<td>A credible third-party report says otherwise</td>
<td>The report</td>
</tr>
</tbody>
</table>
<p>Grade honestly and an uncomfortable pattern appears. From the client side you can verify exactly one hop, the first. The egress log proves the host, and the invoice proves the counterparty. Past the vendor’s front door the best available grade is contracted. The only lanes whose last hop is verified by construction are those where the first hop is the last one: open weights you run on hardware you control, with that host’s egress denied, or a gateway you operate in front of such a model.</p>
<p>That isn’t an argument against hosted vendors. Contracts are how companies extend trust to each other, and a lane that tops out at contracted is fine for most code. What the grades stop is quiet promotion: a trust page read as a promise, a sub-processor list read as complete, a model name read as proof. A questionnaire answer is attested however formal it looks. A contracted cell with no link to a signed document gets downgraded on sight.</p>
<p>
<em>You hold the evidence for the first hop. Past it, you are handed evidence, and the cap follows the weakest grade.</em></p>
<h2 id="step-3-ask-the-onward-routing-question-in-writing">Step 3: Ask the onward-routing question in writing</h2>
<p>Anthropic’s numbers are why column 3 exists. The scale lines count the distillation exchanges Anthropic attributes to each lab. The tiles are the Moonshot relay, the part that carried Moonshot’s own customers’ requests, and by Anthropic’s account it happened silently.</p>
<p>
<em>Anthropic’s allegations, charted from its own page on a log scale. The per-lab lines sum to about 190M; TechCrunch rounds the total to nearly 200M.</em></p>
<p>Send every vendor and intermediary in the register the same six questions, and ask for answers in writing from someone who can sign for the company.</p>
<ol>
<li>Is every request answered by the model named in our agreement and in the response? If not, which other models answer, and when?</li>
<li>Is any request, in whole or in part, sent to another model provider? On whose account?</li>
<li>Do you log prompts or completions? Who can read those logs, and are they ever sold, shared or used for training?</li>
<li>Which sub-processors receive prompt content? Send the list with its date.</li>
<li>Do you resell access to another provider’s models? If so, do you hold your own agreement with that provider?</li>
<li>Will you notify us before any of the above changes, and how far in advance?</li>
</ol>
<p>File each answer with its date and signer. A written answer moves a cell from unknown to attested; only a signed term moves it to contracted. A vendor that declines question 2 has still answered it for the register: the cell stays unknown.</p>
<p>Routers get the same questions, and harder. A hosted router is its own row, and its model pool is column 3 by definition. <a href="/intel/hosted-router-vs-own-cascade/">Routers you can’t see inside</a> scores their visibility; carry that grade into column 4 here instead of redoing the work.</p>
<h2 id="step-4-probe-whether-a-response-names-the-model-then-grade-the-name-as-attested">Step 4: Probe whether a response names the model, then grade the name as attested</h2>
<p>Column 4 is the only one you can test from the client. Run a fixed probe from the lane’s own runtime, with its own endpoint and credential, and keep every result.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># probe-model-field.sh (illustrative shape; adapt the request to the lane's API)</span></span>
<span class="line"><span style="color:#6A737D"># Run inside the lane's runtime on a schedule and after every CLI upgrade.</span></span>
<span class="line"><span style="color:#F97583">for</span><span style="color:#E1E4E8"> i </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> $(</span><span style="color:#B392F0">seq</span><span style="color:#79B8FF"> 1</span><span style="color:#79B8FF"> 20</span><span style="color:#E1E4E8">); </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#B392F0">  curl</span><span style="color:#79B8FF"> -s</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$LANE_BASE_URL</span><span style="color:#9ECBFF">/chat/completions"</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#79B8FF">    -H</span><span style="color:#9ECBFF"> "Authorization: Bearer </span><span style="color:#E1E4E8">$LANE_KEY</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> -H</span><span style="color:#9ECBFF"> "Content-Type: application/json"</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#79B8FF">    -d</span><span style="color:#9ECBFF"> "{</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">model</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">:</span><span style="color:#79B8FF">\"</span><span style="color:#E1E4E8">$LANE_MODEL</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">,</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">messages</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">:[{</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">role</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">:</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">user</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">,</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">content</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">:</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">Reply with ok.</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">}],</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">max_tokens</span><span style="color:#79B8FF">\"</span><span style="color:#9ECBFF">:5}"</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#F97583">  |</span><span style="color:#B392F0"> jq</span><span style="color:#79B8FF"> -c</span><span style="color:#79B8FF"> --arg</span><span style="color:#9ECBFF"> lane</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$LANE_ID</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> --arg</span><span style="color:#9ECBFF"> want</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$LANE_MODEL</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#9ECBFF">      '{lane:$lane, requested:$want, reported:(.model // "absent"), ts:(now|todate)}'</span></span>
<span class="line"><span style="color:#F97583">done</span><span style="color:#F97583"> &gt;&gt;</span><span style="color:#9ECBFF"> "register/probes/</span><span style="color:#E1E4E8">$LANE_ID</span><span style="color:#9ECBFF">.jsonl"</span></span></code></pre>
<p>Grade the result on three levels: exported (a model ID per call that your runner stores), displayed (a person can see it on screen) and not exposed. Vendors differ. <a href="https://docs.github.com/en/copilot/concepts/models/auto-model-selection">GitHub’s docs</a> say “You can see which model was used for each Copilot response.” <a href="https://cursor.com/docs/cursor-router">Cursor’s router</a> shows the routed model in its Balance and Intelligence modes only if an admin turns on the Underlying model setting, which is hidden by default. <a href="https://sakana.ai/fugu-max-release/">Sakana</a> names models that are not in Fugu Ultra v2’s pool, which is not the same as naming the one that answered.</p>
<p>Then grade every name you get as attested. The model field is written by the vendor’s server, and the Moonshot allegation is exactly a case where, by Anthropic’s account, users saw one company’s product and received another company’s model. A name can’t verify the hop that wrote it. The probe still earns its place as a tripwire: a model you didn’t request, or a field that goes from a value to absent, is a step 6 trigger.</p>
<h2 id="step-5-cap-each-lanes-data-class-and-enforce-the-cap-at-dispatch">Step 5: Cap each lane’s data class and enforce the cap at dispatch</h2>
<p>Classify repos into four classes, and write the definitions where the runner can read them.</p>
<ul>
<li><strong>Public:</strong> open-source repos, published docs.</li>
<li><strong>Low:</strong> internal but harmless if published, such as build tooling and fixtures with no customer data.</li>
<li><strong>Confidential:</strong> proprietary product code, internal tickets, designs.</li>
<li><strong>Restricted:</strong> customer or regulated data, security-sensitive code, anything with live credentials in reach. Anthropic’s own examples from the relay include an engineer who, in its framing, revealed internal code and live credentials.</li>
</ul>
<p>Then compute each lane’s last-hop grade as the weakest grade across columns 1 to 3. Column 4 stays out of the arithmetic, because for a hosted lane it can never rise above attested; it works as a trigger instead. The grade sets the cap. This mapping is a starting policy, not a standard, so tighten it to your own risk appetite.</p>
<table>
<thead>
<tr>
<th>Last-hop grade</th>
<th>Data-class cap</th>
<th>Typical lane</th>
</tr>
</thead>
<tbody>
<tr>
<td>Verified</td>
<td>Restricted</td>
<td>Open weights on your own hardware, egress denied</td>
</tr>
<tr>
<td>Contracted</td>
<td>Confidential</td>
<td>Signed agreement that names sub-processors, bars onward routing and resale, and requires change notice</td>
</tr>
<tr>
<td>Attested</td>
<td>Low</td>
<td>Click-through terms plus a public trust page</td>
</tr>
<tr>
<td>Unknown or contradicted</td>
<td>Public</td>
<td>Reseller with an untraceable upstream, router with no onward-routing terms, vendor named in an unanswered allegation</td>
</tr>
</tbody>
</table>
<p>Read the bottom two rows together: a lane whose last hop you can neither observe nor enforce gets public or low-class repos only.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># register/lanes.yaml (illustrative shape; the runner reads it, lanes never do)</span></span>
<span class="line"><span style="color:#E1E4E8">- </span><span style="color:#85E89D">lane</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">review-bot-vendor-a</span></span>
<span class="line"><span style="color:#85E89D">  endpoint</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">api.vendor-a.example</span></span>
<span class="line"><span style="color:#85E89D">  plan</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">signed-api-agreement</span></span>
<span class="line"><span style="color:#85E89D">  contracting_entity</span><span style="color:#E1E4E8">: {</span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Vendor A Ltd"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">parent</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Vendor A Group"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">grade</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">verified</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#85E89D">  sub_processors</span><span style="color:#E1E4E8">:     {</span><span style="color:#85E89D">source</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"DPA annex, 2026-07"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">grade</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">contracted</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#85E89D">  onward_routing</span><span style="color:#E1E4E8">:     {</span><span style="color:#85E89D">source</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"MSA s.9, no onward routing or resale"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">grade</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">contracted</span><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#85E89D">  model_identity</span><span style="color:#E1E4E8">:     {</span><span style="color:#85E89D">level</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">exported</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">grade</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">attested</span><span style="color:#E1E4E8">}   </span><span style="color:#6A737D"># vendor-written; a trigger, not an input</span></span>
<span class="line"><span style="color:#85E89D">  last_hop_grade</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">contracted</span><span style="color:#6A737D">                               # weakest of columns 1-3</span></span>
<span class="line"><span style="color:#85E89D">  data_class_cap</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">confidential</span></span>
<span class="line"><span style="color:#85E89D">  open_triggers</span><span style="color:#E1E4E8">: []</span></span>
<span class="line"><span style="color:#85E89D">  reviewed</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2026-09-21</span></span></code></pre>
<p>The dispatch gate runs in the runner before the harness starts, and every path that isn’t a clean yes is a refusal.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#6A737D"># last_hop_gate.py (illustrative shape): exit 0 lets the lane start; anything else refuses</span></span>
<span class="line"><span style="color:#F97583">import</span><span style="color:#E1E4E8"> sys, yaml</span></span>
<span class="line"></span>
<span class="line"><span style="color:#79B8FF">ORDER</span><span style="color:#F97583"> =</span><span style="color:#E1E4E8"> [</span><span style="color:#9ECBFF">"public"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"low"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"confidential"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"restricted"</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#79B8FF">CAP</span><span style="color:#F97583"> =</span><span style="color:#E1E4E8"> {</span><span style="color:#9ECBFF">"verified"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"restricted"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"contracted"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"confidential"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"attested"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"low"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#9ECBFF">       "unknown"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"public"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"contradicted"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"public"</span><span style="color:#E1E4E8">}</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">try</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">    lane_id, repo </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> sys.argv[</span><span style="color:#79B8FF">1</span><span style="color:#E1E4E8">], sys.argv[</span><span style="color:#79B8FF">2</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#E1E4E8">    lanes </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> {l[</span><span style="color:#9ECBFF">"lane"</span><span style="color:#E1E4E8">]: l </span><span style="color:#F97583">for</span><span style="color:#E1E4E8"> l </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> yaml.safe_load(</span><span style="color:#79B8FF">open</span><span style="color:#E1E4E8">(</span><span style="color:#9ECBFF">"register/lanes.yaml"</span><span style="color:#E1E4E8">))}</span></span>
<span class="line"><span style="color:#E1E4E8">    lane </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> lanes[lane_id]                                   </span><span style="color:#6A737D"># unregistered lane = KeyError = refuse</span></span>
<span class="line"><span style="color:#E1E4E8">    classes </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> yaml.safe_load(</span><span style="color:#79B8FF">open</span><span style="color:#E1E4E8">(</span><span style="color:#9ECBFF">"policy/repo-classes.yaml"</span><span style="color:#E1E4E8">))</span></span>
<span class="line"><span style="color:#E1E4E8">    repo_class </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> classes.get(repo, </span><span style="color:#9ECBFF">"restricted"</span><span style="color:#E1E4E8">)            </span><span style="color:#6A737D"># unclassified repo = restricted</span></span>
<span class="line"><span style="color:#E1E4E8">    grade </span><span style="color:#F97583">=</span><span style="color:#9ECBFF"> "unknown"</span><span style="color:#F97583"> if</span><span style="color:#E1E4E8"> lane.get(</span><span style="color:#9ECBFF">"open_triggers"</span><span style="color:#E1E4E8">) </span><span style="color:#F97583">else</span><span style="color:#E1E4E8"> lane.get(</span><span style="color:#9ECBFF">"last_hop_grade"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"unknown"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#E1E4E8">    cap </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> CAP</span><span style="color:#E1E4E8">.get(grade, </span><span style="color:#9ECBFF">"public"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#79B8FF"> ORDER</span><span style="color:#E1E4E8">.index(repo_class) </span><span style="color:#F97583">&gt;</span><span style="color:#79B8FF"> ORDER</span><span style="color:#E1E4E8">.index(cap):</span></span>
<span class="line"><span style="color:#E1E4E8">        sys.exit(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"refuse </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">lane_id</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF"> -&gt; </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">repo</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">: </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">repo_class</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF"> exceeds cap </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">cap</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF"> (</span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">grade</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">)"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">except</span><span style="color:#79B8FF"> Exception</span><span style="color:#F97583"> as</span><span style="color:#E1E4E8"> e:</span></span>
<span class="line"><span style="color:#E1E4E8">    sys.exit(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"refuse: last-hop gate error </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">e</span><span style="color:#F97583">!r</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">)          </span><span style="color:#6A737D"># fail closed</span></span></code></pre>
<p>Be plain about what the gate can’t do. It sees lanes the runner starts, and a developer who launches the harness by hand with the lane’s endpoint skips it. So the gate isn’t the wall. The wall is the lane’s own access: its checkout credential can read only repos at or below its cap, and its egress allowlist reaches only the endpoint in its row. A hand-launched session with the lane’s credential still can’t clone a restricted repo or reach an unregistered host.</p>
<p>Two siblings plug into the same cap. A fallback route inherits every class the primary might carry, so <a href="/intel/jev-gate-fail-mode-rate-limits/">the fail-mode playbook</a> clears one default route for every data class it may see. And <a href="/intel/rule-of-two-lane-split/">the Rule of Two lane split</a> scores what a lane can reach on its sensitive-data leg; the last-hop cap decides where that data may go.</p>
<h2 id="step-6-re-grade-on-triggers-and-treat-an-allegation-as-a-trigger-not-a-verdict">Step 6: Re-grade on triggers, and treat an allegation as a trigger, not a verdict</h2>
<p>A register filled once is a snapshot with a confident filename. Wire these triggers to a re-grade, and record who re-graded and when.</p>
<table>
<thead>
<tr>
<th>Trigger</th>
<th>Signal</th>
<th>Action</th>
</tr>
</thead>
<tbody>
<tr>
<td>Sub-processor list changes</td>
<td>New date or version on the list</td>
<td>Re-grade column 2</td>
</tr>
<tr>
<td>Terms change</td>
<td>New effective date</td>
<td>Re-read column 3; a lost clause drops the grade</td>
</tr>
<tr>
<td>Probe drift</td>
<td>A model you didn’t request, or <code>absent</code></td>
<td>Open a trigger; the lane runs at public until explained</td>
</tr>
<tr>
<td>Vendor adds routing</td>
<td>An auto or router mode, a new pool</td>
<td>New row for the routed path</td>
</tr>
<tr>
<td>Third-party allegation of onward routing</td>
<td>A report like Anthropic’s</td>
<td>Mark contradicted; send the six questions</td>
</tr>
<tr>
<td>Renewal or quarter end</td>
<td>Calendar</td>
<td>Full review of the row</td>
</tr>
</tbody>
</table>
<p>The allegation row needs the most discipline. A credible report that names a vendor is evidence about that vendor’s last hop, and ignoring it is a choice you would have to defend later. It is not proof that your lane leaked, and the register shouldn’t say so. Mark the relevant cells contradicted, let the lane keep running on public repos, send the six questions, and re-grade when the answer arrives: a written reply returns the cell to attested, a signed amendment to contracted. Date every step. If anyone asks in six months what you did the week the report landed, the register answers.</p>
<h2 id="where-ai-vendor-data-routing-slips-past-the-register-and-the-signal-for-each">Where AI vendor data routing slips past the register, and the signal for each</h2>
<p><strong>The unregistered router.</strong> A lane gets pointed at a cheaper gateway or BYOK proxy. Signal: an egress host with no register row.</p>
<p><strong>The plan swap.</strong> Same vendor, but the lane now runs on a consumer or team plan whose terms differ from the row. Signal: the invoice counterparty or account type no longer matches column 1.</p>
<p><strong>Attested, filed as contracted.</strong> A trust-page promise lands in the contracted column. Signal: a contracted cell with no link to a signed document.</p>
<p><strong>Name drift.</strong> The model field changes, or disappears. Signal: the probe log shows a value you didn’t request, or <code>absent</code>, with no open trigger.</p>
<p><strong>Cap rot.</strong> A repo moves up a class, but lanes keep getting dispatched to it on the old answer. Signal: a repo-class change newer than the lane’s last gate record.</p>
<p><strong>The side door.</strong> A lane’s session is exported or imported into another tool whose provider has no row. Signal: session-import events in the audit log for a lane capped below the source repo’s class.</p>
<h2 id="the-last-hop-register-belongs-to-the-layer-that-starts-lanes">The last-hop register belongs to the layer that starts lanes</h2>
<p>None of this lives in a prompt, and none of it can. A model can’t tell you where its own request went, and a vendor’s model field is the vendor talking. The register, the grades, the repo classes, the gate and the probe log sit in the layer that starts lanes, which is also the only place that sees every vendor at once. That layer is what a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> is once you strip the dashboard off: one inventory of lanes that knows what each may carry, and why.</p>
<p>The register won’t make any vendor honest. It makes your own exposure a number you chose, written down with the evidence beside it, and it lets you tighten that number the morning a report like this one lands, without waiting for anyone to answer.</p>
<h2 id="faq">FAQ</h2>
<h3 id="can-an-ai-vendor-send-my-prompts-to-another-model-provider">Can an AI vendor send my prompts to another model provider?</h3>
<p>It depends on the terms, and from the client side you can’t see it happen. Your egress log shows only the first host. Anthropic alleges that Moonshot and DeepSeek relayed customer requests to Claude. Ask each vendor in writing, prefer a signed clause, and cap what each lane may carry.</p>
<h3 id="what-is-a-last-hop-register-for-ai-agents">What is a last-hop register for AI agents?</h3>
<p>A per-lane record of where a prompt can end up: the contracting entity, stated sub-processors, onward-routing and resale terms, and whether responses name the answering model, each graded verified, contracted, attested or unknown. The weakest grade sets the highest data class the runner will send to that lane.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://www.anthropic.com/threat-intelligence-report-september-2026">Anthropic, Detecting and countering misuse of AI (September 2026)</a> — GTG-16002 and the other relay allegations; per-lab scale lines</li>
<li><a href="https://techcrunch.com/2026/09/10/anthropic-details-distillation-campaigns-from-alibaba-moonshot-ai-and-deepseek/">TechCrunch, Anthropic details distillation campaigns from Alibaba, Moonshot AI, and DeepSeek</a> — Sep 10, 2026; the total</li>
<li><a href="https://www.cnbc.com/2026/09/11/chinese-ai-labs-moonshot-deepseek-alibaba-anthropic.html">CNBC on the Anthropic report</a> — Sep 10, 2026; requests for comment</li>
<li><a href="https://docs.github.com/en/copilot/concepts/models/auto-model-selection">GitHub Docs, Auto model selection</a> — model shown per response</li>
<li><a href="https://cursor.com/docs/cursor-router">Cursor Docs, Cursor Router</a> — Underlying model setting</li>
<li><a href="https://sakana.ai/fugu-max-release/">Sakana AI, Fugu Max release</a> — Sep 11, 2026; Fugu Ultra v2 pool exclusions</li>
</ul>
]]></content:encoded></item><item><title>Prove Headless Runs Ignore State You Didn’t Declare</title><link>https://automater.ai/intel/headless-lane-reproducibility/</link><guid>https://automater.ai/intel/headless-lane-reproducibility/</guid><description>Claude Code bare mode plus a clean-versus-used differential run proves a headless lane loads no synced skills, AGENTS.md or memory its manifest didn&apos;t name.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>Take one nightly job: same commit, same prompt, same Claude Code version, run on two machines. On the long-lived runner it reads an <code>AGENTS.md</code>, loads skills synced from whoever signed in to claude.ai there, and writes a memory note on the way out. On a fresh container it does none of that. Neither lane’s config changed; the state walked in through doors that opened last week.</p>
<p>The move is a differential run, with Claude Code bare mode (<code>claude --bare -p</code>) as the launch line it should end on. Run the same unattended job from a clean config home and a used one, record every skill, plugin, instruction file and memory file each run read or wrote, and fail the lane on anything its manifest didn’t declare. Bare mode skips most discovery; the differential proves it did and catches what still gets through.</p>
<p>A chatbot that picks up a stray instruction gives a stranger answer. A headless lane that picks one up opens a pull request or skips a check, and the first person to notice is whoever reviews the diff. You can’t call a run reproducible until you know what its inputs were.</p>
<h2 id="three-new-state-sources-reached-unattended-lanes-on-sep-1618">Three new state sources reached unattended lanes on Sep 16–18</h2>
<p>Three launches in three days added state a headless run picks up with no config edit.</p>
<p><strong>Sep 16: Grok Build memory.</strong> Grok Build now carries conventions, decisions and project facts between sessions. “Capture runs on every completed turn and does not interrupt the session.” Notes are markdown files in a per-project scope plus a global scope that applies everywhere, and memory applies to new sessions (<a href="https://x.ai/news/grok-build-memory">Grok Build memory</a>). The page documents no off switch.</p>
<p><strong>Sep 17: Claude Code 2.1.275.</strong> The changelog entry reads “Added syncing of the skills and plugins enabled on your claude.ai account to terminal sessions signed in with it”, with opt-outs <code>syncClaudeAiSkills: false</code> and <code>syncClaudeAiPlugins: false</code> (<a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a>). The settings reference adds what matters for lanes: sync runs “interactive or non-interactive”, skills land in <code>~/.claude/skills/synced/</code>, plugins load from <code>~/.claude/plugins/synced/</code> as <code>&lt;name&gt;@synced</code>, and “A repository can’t turn it off for you” (<a href="https://code.claude.com/docs/en/settings-reference">Claude Code settings reference</a>).</p>
<p><strong>Sep 18: Claude Code 2.1.277.</strong> “Added AGENTS.md support: in a project with no CLAUDE.md, Claude Code reads AGENTS.md instead”, changeable under “Project instructions” in <code>/config</code>, with a caveat in parentheses: “(not yet on Bedrock, Vertex or Foundry)”. The same repo now loads different instructions depending on which provider sits behind the lane.</p>
<p>
<em>Screenshot: Claude Code Docs, “Claude Code changelog” (2.1.277 entry, Sep 18, 2026), captured Sep 21, 2026.</em></p>
<p>Sync and the AGENTS.md fallback shipped a day apart, and the AGENTS.md read is off in more places than the three providers the changelog names. Step 2 turns that into test design.</p>
<aside class="social-embed" data-x-embed data-post-id="2101009392611278961">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 18, 2026 — Announcement of AGENTS.md support in Claude Code 2.1.277: in a folder with no CLAUDE.md, Claude uses AGENTS.md, with a toggle in /config.</p>
  <a class="social-embed__source" href="https://x.com/trq212/status/2101009392611278961" target="_blank" rel="noopener noreferrer">View Thariq's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="an-instruction-file-is-context-and-a-lane-acts-on-context">An instruction file is context, and a lane acts on context</h2>
<p>Claude Code’s memory docs say it plainly about CLAUDE.md and AGENTS.md: “Claude treats them as context, not enforced configuration” (<a href="https://code.claude.com/docs/en/memory">Claude Code memory docs</a>). A skill is instructions; a memory note is instructions an earlier session wrote. None of it passes a permission rule on its way into the prompt, and all of it shapes what the lane does next.</p>
<p>The <a href="/intel/headless-agent-trust-tier/">headless agent trust tier</a> piece already pins the unattended flag line and hashes instruction files per tier; keep doing that. It predates account sync and per-turn memory, and a hash only covers files you knew to hash. <a href="/intel/agents-md-rot/">AGENTS.md rot</a> records what a repo’s instruction files say, and <a href="/intel/memory-that-burns-quota/">memory that burns quota</a> prices what memory costs. This test does the part none of the three covers: it excludes state and proves the exclusion held.</p>
<h2 id="step-1-write-down-what-the-lane-may-read-and-write">Step 1: Write down what the lane may read and write</h2>
<p>Start with a state manifest per lane. It names every input the runner passes explicitly and every place the lane may write, and it leaves the rest empty on purpose.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># lane-state.yaml (illustrative shape): read by the runner, never shown to the agent</span></span>
<span class="line"><span style="color:#85E89D">lane</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">nightly-deps</span></span>
<span class="line"><span style="color:#85E89D">harness</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">claude-code</span></span>
<span class="line"><span style="color:#85E89D">cli_version</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2.1.278</span></span>
<span class="line"><span style="color:#85E89D">launch</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">claude --bare -p</span></span>
<span class="line"><span style="color:#85E89D">auth</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">api-key</span><span style="color:#6A737D">                  # ANTHROPIC_API_KEY or an apiKeyHelper; no subscription login</span></span>
<span class="line"><span style="color:#85E89D">provider_path</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">anthropic-api</span><span style="color:#6A737D">   # AGENTS.md reading and claude.ai sync depend on it</span></span>
<span class="line"><span style="color:#85E89D">env_allow</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">ANTHROPIC_API_KEY</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">CI</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">LANE_ID</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">reads</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  instructions</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">lane/nightly-deps.md</span><span style="color:#E1E4E8">]   </span><span style="color:#6A737D"># --append-system-prompt-file</span></span>
<span class="line"><span style="color:#85E89D">  settings</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">lane/settings.json</span><span style="color:#E1E4E8">]         </span><span style="color:#6A737D"># --settings</span></span>
<span class="line"><span style="color:#85E89D">  mcp_servers</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">issues-readonly</span><span style="color:#E1E4E8">]         </span><span style="color:#6A737D"># --mcp-config</span></span>
<span class="line"><span style="color:#85E89D">  skills</span><span style="color:#E1E4E8">: []</span></span>
<span class="line"><span style="color:#85E89D">  plugins</span><span style="color:#E1E4E8">: []</span></span>
<span class="line"><span style="color:#85E89D">  memory</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">none</span></span>
<span class="line"><span style="color:#85E89D">writes</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  paths</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">repo working tree</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">lane/out/</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  memory</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">none</span></span></code></pre>
<p>Three rules keep it honest. Every entry is passed by flag or mounted by the runner; nothing is inherited from a home directory. The empty lists are the point, because the test fails on anything that appears in them. And the environment is an allowlist, because one variable can undo bare mode.</p>
<h2 id="step-2-build-a-clean-home-and-a-used-home-and-hold-everything-else-fixed">Step 2: Build a clean home and a used home, and hold everything else fixed</h2>
<p>The two homes differ only in accumulated state. Build the clean one fresh for every run, in a new container or a throwaway user account. The used one is a long-lived runner’s home, signed in the way production runners are, with months of sessions behind it.</p>
<table>
<thead>
<tr>
<th>State</th>
<th>Clean home</th>
<th>Used home</th>
</tr>
</thead>
<tbody>
<tr>
<td>Config home</td>
<td>Empty: new container or throwaway user</td>
<td>A long-lived runner’s</td>
</tr>
<tr>
<td>claude.ai sign-in and sync</td>
<td>None</td>
<td>Signed in, sync at its default</td>
</tr>
<tr>
<td><code>~/.claude/skills/synced/</code>, <code>~/.claude/plugins/synced/</code></td>
<td>Absent</td>
<td>Populated</td>
</tr>
<tr>
<td>Auto memory in <code>~/.claude/projects/&lt;project&gt;/memory/</code></td>
<td>Absent</td>
<td>Notes from earlier sessions</td>
</tr>
<tr>
<td>Grok Build notes, project and global</td>
<td>None</td>
<td>Present</td>
</tr>
<tr>
<td>Sessions since install or upgrade</td>
<td>Zero, so run it twice</td>
<td>Many</td>
</tr>
<tr>
<td>Commit, prompt, flag line, CLI version, provider path, env</td>
<td>Fixed</td>
<td>Identical</td>
</tr>
</tbody>
</table>
<p>The fixture repo carries an <code>AGENTS.md</code> and no <code>CLAUDE.md</code>, because that is the combination 2.1.277 changed.</p>
<p>Two variables flip Claude Code’s AGENTS.md reading and claude.ai sync together. The first is feature-flag fetching, which Claude Code skips when <code>DISABLE_TELEMETRY</code>, <code>DO_NOT_TRACK</code>, <code>DISABLE_GROWTHBOOK</code> or <code>CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC</code> is set, on third-party providers including Amazon Bedrock and Claude Platform on AWS, and in a Claude apps gateway session. Without the flags it loads <code>CLAUDE.md</code> files only and doesn’t sync (<a href="https://code.claude.com/docs/en/env-vars">Claude Code environment variables</a>). Copy the production environment and provider path into both homes, or you are testing a lane you don’t run.</p>
<p>The second is the first session. After an install, or an upgrade that adds a feature, a flag-gated feature can be missing until the flags arrive; the memory docs put it as “Claude reads <code>AGENTS.md</code> from your next session on”. A clean home is a first session by construction, so run it twice and judge the second run.</p>
<p>One more trap: a fresh worktree is not a fresh home. Claude Code derives the auto-memory directory from the git repository, so “all worktrees and subdirectories within the same repo share one auto memory directory”. A lane handed a new worktree on a used runner inherits every note earlier sessions wrote for that repo.</p>
<h2 id="step-3-record-every-read-and-write-three-ways">Step 3: Record every read and write three ways</h2>
<p>One source of evidence isn’t enough; the harness doesn’t report everything it loads.</p>
<ol>
<li><strong>The harness’s own inventory.</strong> Claude Code’s headless <code>system/init</code> event reports <code>plugins</code>, <code>plugin_errors</code>, <code>mcp_servers</code> and <code>mcp_server_errors</code>, and the docs note that a CI gate can fail on a non-empty array (<a href="https://code.claude.com/docs/en/headless">Claude Code headless docs</a>). Compare those arrays with the manifest rather than with emptiness, so a declared MCP server passes and an undeclared one fails.</li>
<li><strong>A filesystem diff for writes.</strong> Hash every file in the home before and after the run. Synced skill and plugin downloads, auto-memory notes and Grok Build notes all show up as new or changed files.</li>
<li><strong>An OS-level read trace.</strong> This is the only reliable way to see an <code>AGENTS.md</code> read. The memory docs say an <code>AGENTS.md</code> read directly is “Not listed” in <code>/memory</code> or <code>/context</code>, and <code>InstructionsLoaded</code> hooks “Don’t fire” for it. An audit built on the harness’s own context view misses exactly the file 2.1.277 added.</li>
</ol>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># record-run.sh (illustrative): one run in one home; repeat for the clean home (twice) and the used home</span></span>
<span class="line"><span style="color:#B392F0">find</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$HOME</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> -type</span><span style="color:#9ECBFF"> f</span><span style="color:#79B8FF"> -exec</span><span style="color:#9ECBFF"> sha256sum</span><span style="color:#9ECBFF"> {}</span><span style="color:#9ECBFF"> +</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> sort</span><span style="color:#79B8FF"> -k2</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF"> before.txt</span></span>
<span class="line"><span style="color:#B392F0">strace</span><span style="color:#79B8FF"> -f</span><span style="color:#79B8FF"> -e</span><span style="color:#9ECBFF"> trace=open,openat</span><span style="color:#79B8FF"> -o</span><span style="color:#9ECBFF"> reads.log</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#9ECBFF">  claude</span><span style="color:#79B8FF"> --bare</span><span style="color:#79B8FF"> -p</span><span style="color:#9ECBFF"> "$(</span><span style="color:#B392F0">cat</span><span style="color:#9ECBFF"> lane/task.md)"</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#79B8FF">    --append-system-prompt-file</span><span style="color:#9ECBFF"> lane/nightly-deps.md</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#79B8FF">    --settings</span><span style="color:#9ECBFF"> lane/settings.json</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#79B8FF">    --mcp-config</span><span style="color:#9ECBFF"> lane/mcp.json</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#79B8FF">    --permission-prompts</span><span style="color:#9ECBFF"> none</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF"> run.out</span></span>
<span class="line"><span style="color:#B392F0">find</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$HOME</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> -type</span><span style="color:#9ECBFF"> f</span><span style="color:#79B8FF"> -exec</span><span style="color:#9ECBFF"> sha256sum</span><span style="color:#9ECBFF"> {}</span><span style="color:#9ECBFF"> +</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> sort</span><span style="color:#79B8FF"> -k2</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF"> after.txt</span></span>
<span class="line"><span style="color:#B392F0">diff</span><span style="color:#9ECBFF"> before.txt</span><span style="color:#9ECBFF"> after.txt</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF"> home-writes.diff</span><span style="color:#F97583"> ||</span><span style="color:#79B8FF"> true</span></span>
<span class="line"><span style="color:#B392F0">grep</span><span style="color:#79B8FF"> -E</span><span style="color:#9ECBFF"> 'AGENTS\.md|CLAUDE(\.local)?\.md|/memory/|/synced/|/skills/|\.mcp\.json'</span><span style="color:#9ECBFF"> reads.log</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF"> state-reads.txt</span><span style="color:#F97583"> ||</span><span style="color:#79B8FF"> true</span></span></code></pre>
<p><code>--permission-prompts none</code> (2.1.259 or later) denies anything that would prompt and tells the model nobody can approve it, so the run never waits on an approver who isn’t there.</p>
<p>Then plant canary tokens, our technique rather than a vendor feature: a distinct marker line in each state source you control (the <code>AGENTS.md</code>, a memory note, a user-level skill, a global Grok note) asking the model to end its reply with that marker. The trace proves a file was opened; a token in the output proves the model used it.</p>
<p>When the recorder itself fails (no trace file, an empty snapshot, an init event that never arrived), fail the lane. Missing evidence is not a pass.</p>
<h2 id="step-4-diff-the-two-runs-and-fail-on-anything-undeclared">Step 4: Diff the two runs and fail on anything undeclared</h2>
<p>Put both runs side by side against the manifest. Here is a first pass on a lane still launched with plain <code>claude -p</code>; the values are illustrative.</p>
<table>
<thead>
<tr>
<th>Item</th>
<th>Clean home, run 2</th>
<th>Used home</th>
<th>In manifest?</th>
<th>Verdict</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>lane/nightly-deps.md</code></td>
<td>read</td>
<td>read</td>
<td>yes</td>
<td>pass</td>
</tr>
<tr>
<td><code>AGENTS.md</code></td>
<td>read</td>
<td>read</td>
<td>no</td>
<td>fail</td>
</tr>
<tr>
<td><code>~/.claude/skills/synced/</code></td>
<td>absent</td>
<td>3 skills read</td>
<td>no</td>
<td>fail</td>
</tr>
<tr>
<td><code>&lt;name&gt;@synced</code> plugin</td>
<td>absent</td>
<td>loaded</td>
<td>no</td>
<td>fail</td>
</tr>
<tr>
<td>Auto memory</td>
<td>absent</td>
<td>1 note read, 1 written</td>
<td>no</td>
<td>fail</td>
</tr>
<tr>
<td>Hook in <code>.claude/settings.json</code></td>
<td>ran</td>
<td>ran</td>
<td>no</td>
<td>fail</td>
</tr>
<tr>
<td>Server in <code>.mcp.json</code></td>
<td>connected</td>
<td>connected</td>
<td>no</td>
<td>fail</td>
</tr>
</tbody>
</table>
<p>Two rules decide the verdict. Any item in either run that isn’t in the manifest fails the lane, even when both runs agree; two contaminated homes agreeing is not cleanliness. And any difference between the runs fails the lane even for a declared item, because an input that loads on one machine and not the other is a reproducibility bug. Keep the verdict and the evidence files in the run record.</p>
<p>
<em>Illustrative. Switching off sync and memory removes the account-borne state; bare mode removes the rest, AGENTS.md included in a 2.1.278 test.</em></p>
<h2 id="step-5-make-claude-code-bare-mode-the-launch-line-and-pass-everything-by-flag">Step 5: Make Claude Code bare mode the launch line and pass everything by flag</h2>
<p>For Claude Code, the documented fix for most of that table is one flag. The headless docs: “Add <code>--bare</code> to reduce startup time by skipping auto-discovery of hooks, skills, custom commands, subagents, plugins, MCP servers, auto memory, and CLAUDE.md.” And the line that makes it the lane default: “Bare mode is useful for CI and scripts where you need the same result on every machine.”</p>
<p>Compare the plain <code>-p</code> default, which the same page warns about: it “runs the hooks in a project’s <code>.claude/settings.json</code> and connects the servers in its <code>.mcp.json</code>, even in a folder you’ve never trusted”, with no workspace trust dialog and no per-server approval prompt.</p>
<p>
<em>Screenshot: Claude Code Docs, “Run Claude Code programmatically” (undated), captured Sep 21, 2026.</em></p>
<p>Bare mode changes authentication. “In bare mode, Claude Code never reads OAuth credentials or the system keychain”, so a subscription login won’t authenticate the lane. Set <code>ANTHROPIC_API_KEY</code> from your secret store or supply an <code>apiKeyHelper</code> in the <code>--settings</code> JSON; Bedrock, Google Cloud’s Agent Platform and Microsoft Foundry keep reading their own provider credentials.</p>
<p>The run gets Bash, file read and file edit, and everything else arrives by flag: <code>--append-system-prompt-file</code>, <code>--settings</code>, <code>--mcp-config</code>, <code>--agents</code>, and <code>--plugin-dir</code> for a plugin you reviewed. That flag line should match the manifest line for line.</p>
<p>
<em>Bare mode stops discovery at one wall. The amber line is why you still run the differential.</em></p>
<p>Three edges to know before you call a bare lane clean.</p>
<ul>
<li><strong>An <code>--add-dir</code> folder brings its skills.</strong> Bare mode loads skills from that folder’s <code>.claude/skills/</code>, while still skipping its <code>.claude/commands/</code> and <code>.claude/agents/</code>. Declare those skills in the manifest or don’t add the folder.</li>
<li><strong>Bare mode skipped AGENTS.md in a capture test.</strong> The bare-mode docs name CLAUDE.md as skipped and never mention AGENTS.md, which ships as the built-in <code>agents-md</code> plugin. A Sep 21 test on Claude Code 2.1.278 settled it for that version: with an <code>AGENTS.md</code>-only repo holding a canary token and the API calls captured locally, plain <code>claude -p</code> sent the file from the second session on, as step 2 predicts, and <code>claude --bare -p</code> never sent it in two runs. One version on the direct API path is not a guarantee, so rerun the fixture after every upgrade: a clean home run twice, the canary repo, and <code>claude --bare -p "what are your project instructions"</code>.</li>
<li><strong>Bare becomes the default later.</strong> The docs say <code>--bare</code> “will become the default for <code>-p</code> in a future release.” Put it in the flag line now, so lanes that quietly rely on discovery break on your schedule rather than the vendor’s.</li>
</ul>
<h2 id="step-6-when-a-lane-cant-use-claude-code-bare-mode-switch-each-source-off-by-name">Step 6: When a lane can’t use Claude Code bare mode, switch each source off by name</h2>
<p>Some lanes need a subscription login, or a plugin that only discovery finds. Those lanes keep plain <code>-p</code> and switch sources off one at a time, from a scope the repo can’t override.</p>
<ul>
<li><strong>Account sync.</strong> Set <code>syncClaudeAiSkills: false</code> and <code>syncClaudeAiPlugins: false</code> in user, local or managed settings, or a file passed with <code>--settings</code>. Claude Code honours only <code>false</code> and ignores it in the committed <code>.claude/settings.json</code>; one plugin can be switched off with <code>"&lt;name&gt;@synced": false</code> in <code>enabledPlugins</code>. A lane that ranks skills ranks whatever sync delivered; <a href="/intel/jev-skill-ranking-agent-decides/">soft-ranking skills with Jev</a> assumes you chose that roster.</li>
<li><strong>Auto memory.</strong> Set <code>CLAUDE_CODE_DISABLE_AUTO_MEMORY=1</code> or <code>autoMemoryEnabled: false</code>, or point <code>autoMemoryDirectory</code> at a directory the lane owns (step 7).</li>
<li><strong>Instruction layers.</strong> <code>~/.claude/CLAUDE.md</code>, a managed CLAUDE.md and <code>.claude/rules/</code> keep loading alongside <code>AGENTS.md</code>. A <code>CLAUDE.local.md</code> counts as a CLAUDE.md, so adding one stops the <code>AGENTS.md</code> read, and so do <code>disableAllHooks</code> and <code>allowManagedHooksOnly</code>, because the reader ships as a plugin. To pin the choice, set <code>pluginConfigs</code> → <code>agents-md@builtin</code> → <code>instructionFiles</code> to <code>claude-md</code> (or <code>claude-md-and-agents-md</code>) in user or managed settings; “Claude Code ignores it in project and local settings files.”</li>
<li><strong>Plugin timing.</strong> Set <code>CLAUDE_CODE_SYNC_PLUGIN_INSTALL=1</code>. Without it, plugins “install in the background and may not be available on the first turn”, so turn one sees a different plugin set from turn five.</li>
</ul>
<p>An illustrative user-settings shape for such a lane’s runner, alongside <code>CLAUDE_CODE_DISABLE_AUTO_MEMORY=1</code> and <code>CLAUDE_CODE_SYNC_PLUGIN_INSTALL=1</code> in its environment:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span></span>
<span class="line"><span style="color:#79B8FF">  "syncClaudeAiSkills"</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">false</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">  "syncClaudeAiPlugins"</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">false</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">  "autoMemoryEnabled"</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">false</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">  "pluginConfigs"</span><span style="color:#E1E4E8">: {</span></span>
<span class="line"><span style="color:#79B8FF">    "agents-md@builtin"</span><span style="color:#E1E4E8">: { </span><span style="color:#79B8FF">"instructionFiles"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"claude-md"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#E1E4E8">  }</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span></code></pre>
<p>Hooks and <code>.mcp.json</code> servers still run under plain <code>-p</code>. Keep the permission deny rules and sandbox that <a href="/intel/restricted-mode-fleet-policy/">restricted mode as fleet policy</a> canary-tests as the wall behind every lane in this piece. The differential tells you what a run read; it does nothing to stop a run acting on it.</p>
<h2 id="step-7-give-memory-without-an-off-switch-an-empty-or-versioned-directory">Step 7: Give memory without an off switch an empty or versioned directory</h2>
<p>Grok Build documents no memory off switch, and its notes include a global scope that applies to every project. So control what exists when the run starts.</p>
<ol>
<li><strong>Fresh or restored home per run.</strong> Start each Grok lane from an empty home, or restore its notes from a versioned copy the lane’s owner reviewed. Grok’s <code>/dream</code> pass also runs on its own in the background, so snapshot notes at run start rather than trusting the previous run’s end state.</li>
<li><strong>Treat new notes as output.</strong> Step 3 already hashes them; route new or changed notes to review before a later run can read them. Grok says task state and secrets are left out of notes. Confirm that in the diff, not on the vendor’s word.</li>
<li><strong>Know what outranks a note.</strong> “Instructions in the current conversation take precedence over anything in a note”, so a lane prompt that states a rule wins. It doesn’t help where the prompt is silent.</li>
<li><strong>Same pattern on Claude Code.</strong> Where a lane genuinely needs memory, set <code>autoMemoryDirectory</code> to a directory under version control, reset it from a pinned commit before each run and review its diff after. That also fixes the shared-worktree trap.</li>
</ol>
<p>Memory outlives compaction, too. Claude Code’s context-window docs list auto memory as “Re-injected from disk” after a compaction (<a href="https://code.claude.com/docs/en/context-window">Claude Code context window docs</a>), so an undeclared note returns after every <a href="/intel/agent-owned-compaction-policy/">compaction the lane’s contract allows</a>.</p>
<h2 id="step-8-re-run-the-differential-after-every-cli-upgrade">Step 8: Re-run the differential after every CLI upgrade</h2>
<p>Both Claude Code changes above arrived as ordinary version bumps that touched no config file. Add the differential to the upgrade gate that <a href="/intel/cli-upgrade-canary-drift/">the CLI upgrade canary</a> runs: the canary diffs what the lane was served (model, effort, compaction point), and this diffs what it read. After every bump, run the clean home twice, since the new version’s first session is when flag-gated features go missing, and fail the upgrade on any new undeclared read or write.</p>
<h2 id="where-the-clean-versus-used-test-lies-to-you">Where the clean-versus-used test lies to you</h2>
<p><strong>A leaked environment variable.</strong> <code>CLAUDE_CODE_DISABLE_AUTO_MEMORY=0</code> forces auto memory on “even when <code>--bare</code> mode or <code>autoMemoryEnabled: false</code> would otherwise disable it”. Signal: memory reads or writes in a bare run. Fix: the environment allowlist from step 1, recorded with every run.</p>
<p><strong>A clean home that’s really a first session.</strong> Signal: clean run 1 and clean run 2 disagree on <code>AGENTS.md</code> or synced content. Fix: judge run 2 and keep run 1 as evidence.</p>
<p><strong>A shared memory directory.</strong> Signal: memory reads on a brand-new worktree. Fix: a lane-owned <code>autoMemoryDirectory</code>, or memory off.</p>
<p><strong>The <code>CLAUDE.local.md</code> flip.</strong> Signal: the instruction-file set changes with no commit touching <code>AGENTS.md</code>. Fix: the <code>instructionFiles</code> pin from step 6.</p>
<p><strong>An opt-out in the wrong file.</strong> Signal: <code>skills/synced/</code> fills up although the repo’s settings say <code>false</code>. Fix: move the opt-out to user, managed or <code>--settings</code> scope.</p>
<p><strong>The invisible read.</strong> Signal: the model follows an instruction that no file in <code>/context</code> contains. Fix: the OS trace and canary tokens from step 3.</p>
<p><strong>The wrong provider path.</strong> Signal: the test passes on an API key while a production lane on Bedrock or behind a gateway loads different instructions. Fix: test on the path production uses.</p>
<h2 id="reproducible-runs-belong-to-the-fleet-layer">Reproducible runs belong to the fleet layer</h2>
<p>None of this lives in the model or the prompt. The manifest, the two homes, the recorder and the fail rule belong to the layer that launches lanes, holds their credentials and keeps their evidence, the same layer that owns kill switches and approvals. A <a href="/intel/fleet-replay/">fleet replay</a> is only evidence if you know what the original run read; the differential turns that input list from a guess into a record.</p>
<p>Vendors will keep adding state that makes interactive sessions smarter by default, which is good for a person at a keyboard. Unattended lanes need the opposite default: nothing loads unless someone wrote it down.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-claude---bare-read-agentsmd">Does <code>claude --bare</code> read AGENTS.md?</h3>
<p>Not on Claude Code 2.1.278, in a Sep 21 capture test: plain <code>claude -p</code> sent the AGENTS.md to the API from the second session on, and <code>claude --bare -p</code> never did. The docs name only CLAUDE.md as skipped, so rerun the test after each upgrade with a canary token in an AGENTS.md-only repo.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a> — 2.1.275 claude.ai sync (Sep 17, 2026); 2.1.277 AGENTS.md support (Sep 18, 2026)</li>
<li><a href="https://code.claude.com/docs/en/headless">Claude Code headless docs</a> — bare mode, authentication, the <code>--add-dir</code> exception, <code>system/init</code>, the untrusted-folder warning</li>
<li><a href="https://code.claude.com/docs/en/memory">Claude Code memory docs</a> — AGENTS.md loading rules and unavailability, auto memory path and switches</li>
<li><a href="https://code.claude.com/docs/en/settings-reference">Claude Code settings reference</a> — <code>syncClaudeAiSkills</code>, <code>syncClaudeAiPlugins</code>, <code>autoMemoryEnabled</code>, <code>autoMemoryDirectory</code></li>
<li><a href="https://code.claude.com/docs/en/env-vars">Claude Code environment variables</a> — feature-flag fetching, the first-session rule, <code>CLAUDE_CODE_DISABLE_AUTO_MEMORY</code>, <code>CLAUDE_CODE_SYNC_PLUGIN_INSTALL</code></li>
<li><a href="https://code.claude.com/docs/en/context-window">Claude Code context window docs</a> — what compaction re-injects</li>
<li><a href="https://x.ai/news/grok-build-memory">Grok Build memory</a> — per-turn capture, workspace and global scopes, new sessions only (Sep 16, 2026)</li>
</ul>
]]></content:encoded></item><item><title>Vendor Permission Verdicts: Which Ones Still Need Your Approval?</title><link>https://automater.ai/intel/vendor-permission-verdict-triage/</link><guid>https://automater.ai/intel/vendor-permission-verdict-triage/</guid><description>Claude Managed Agents permission policy verdicts, normalized: log allows and denials as evidence, queue only real asks, and mark unseen verdicts unavailable.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>A Claude Managed Agents session on the <code>auto</code> permission policy can stamp <code>evaluated_permission: "allow"</code> on a shell call that pushes a branch and run it in the same moment. No prompt, no queue entry, no person. That is the Claude Managed Agents permission policy working as designed: the verdict you used to give now arrives as a field on an event, after the call has happened.</p>
<p>Two Anthropic products moved permission decisions onto Anthropic’s servers in the nine days to Sep 19. The fleet question is no longer whether to approve a call. It is which of the vendor’s verdicts still need you at all, and where the rest belong.</p>
<p>By Tuesday you can answer that for every lane. Each vendor’s evaluation lands in one schema. Allows and denials go to an evaluation log as evidence. Only calls the vendor actually paused reach your approval queue, each with an expiry you set. Tools a person must see first move to the vendor’s native ask. And every call whose verdict you cannot see is written down as <code>unavailable</code> instead of assumed.</p>
<p>Chatbots suggest; agents act. The approval queue used to double as the record of what was decided. Now most decisions happen on the vendor’s side of the wire, and the queue sees only the leftovers.</p>
<h2 id="sep-10-and-sep-19-the-claude-managed-agents-permission-policy-gets-auto-and-claude-code-goes-server-side">Sep 10 and Sep 19: the Claude Managed Agents permission policy gets <code>auto</code>, and Claude Code goes server-side</h2>
<p>On Sep 10, 2026, <a href="https://platform.claude.com/docs/en/release-notes/overview">Anthropic’s platform release notes</a> added <code>auto</code> to the <a href="https://platform.claude.com/docs/en/managed-agents/permission-policies">Managed Agents permission policies</a>, next to <code>always_allow</code> and <code>always_ask</code>. Under <code>auto</code> the server evaluates each agent or MCP tool call and runs it, denies it, or pauses it for your approval when it reaches no determination. The agent toolset defaults to <code>always_allow</code>, MCP toolsets to <code>always_ask</code>, and nothing uses <code>auto</code> until you set it. The same notes shipped version 1.32.0 of the <code>ant</code> CLI with <code>ant beta:sessions connect</code>, which attaches a terminal to a running session so you can follow it and allow or deny waiting calls.</p>
<p>The verdict rides on the event. Under any policy, every <code>agent.tool_use</code> and <code>agent.mcp_tool_use</code> event carries <code>evaluated_permission</code> with a value of <code>allow</code>, <code>ask</code> or <code>deny</code>, and most events also carry an <code>evaluation</code> object. Under <code>auto</code>, a pause carries the <code>reason_code</code> <code>indeterminate</code> and a denial carries <code>high_risk</code>. A denial is final: “The session keeps running, and your client cannot override the denial.” The docs are just as blunt about the other direction.</p>
<blockquote>
<p>“<code>auto</code> is not a human checkpoint. If the server determines that a call is safe, the call runs before anyone sees it, and its effects might not be reversible. If a person must review a tool’s calls before they run, configure <code>always_ask</code> on that tool.”</p>
</blockquote>
<p>
<em>Screenshot: Claude Platform Docs, “Permission policies - Claude Platform Docs” (undated), captured Sep 21, 2026.</em></p>
<p>Nine days later, <a href="https://code.claude.com/docs/en/changelog">Claude Code 2.1.278</a> (Sep 19) switched auto mode for Claude API and Enterprise users, and on Bedrock, Vertex, Foundry and gateways, to a server-side classifier “which does not charge for classifier overhead”. The opt-out, <code>CLAUDE_CODE_AUTO_MODE_SERVER=0</code>, applies on Bedrock, Vertex, Foundry and gateways, and the <a href="https://code.claude.com/docs/en/permission-modes">permission-modes doc</a> says the variable isn’t read on a direct connection to the Anthropic API. Four days earlier, 2.1.273 had set those three clouds to the local classifier by default. How your own classifier stacks against Claude Code’s is covered in <a href="/intel/jev-confidence-gates-tool-calls/">the Jev confidence-gate runbook</a>. What matters here is what Claude Code tells you afterwards: its denials, and never which allows its classifier made.</p>
<p>A third source, GitHub Copilot’s enterprise-managed <code>ask</code>, comes with its own rules about what can’t satisfy it. <a href="/intel/non-overridable-agent-permissions/">The cross-harness override test</a> owns that one.</p>
<aside class="social-embed" data-x-embed data-post-id="2098120135718433024">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 10, 2026 — Claude's developer account announces auto mode for Claude Managed Agents, where each tool call is reviewed against the intent in user.message events and then run, denied or escalated to you.</p>
  <a class="social-embed__source" href="https://x.com/ClaudeDevs/status/2098120135718433024" target="_blank" rel="noopener noreferrer">View ClaudeDevs's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<p><a href="/intel/hitl-approval-queue-hygiene/">Human-in-the-loop queues that don’t become rubber stamps</a> already folds your CLIs into one tiered queue with expiry and budgets. This runbook sits upstream of it. It decides which vendor verdicts get into that queue in the first place.</p>
<h2 id="step-1-write-one-verdict-schema-every-lane-reports-into">Step 1: Write one verdict schema every lane reports into</h2>
<p>Keep one row per tool call per verdict source, append-only. A call that pauses and is later answered gets two rows: the vendor’s <code>ask</code>, then the resolution. Use the same fields on every lane, whichever vendor made the call.</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>What goes in it</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>lane</code>, <code>session_id</code></td>
<td>Your lane name; the vendor’s session ID</td>
</tr>
<tr>
<td><code>event_id</code>, <code>tool_use_id</code></td>
<td>The vendor’s IDs for the event and the call; these are your join keys</td>
</tr>
<tr>
<td><code>tool</code>, <code>call_ref</code>, <code>call_sha256</code></td>
<td>Tool name; a pointer to the exact input in an access-controlled store; its hash as an integrity check</td>
</tr>
<tr>
<td><code>verdict</code></td>
<td><code>allow</code> · <code>ask</code> · <code>deny</code> · <code>unavailable</code></td>
</tr>
<tr>
<td><code>decided_by</code></td>
<td><code>static_policy</code> · <code>vendor_classifier</code> · <code>human</code> · <code>unavailable</code></td>
</tr>
<tr>
<td><code>vendor_raw</code></td>
<td>The documented fields exactly as received, never renamed</td>
</tr>
<tr>
<td><code>source</code></td>
<td>Which stream produced the row: event stream, hook or telemetry</td>
</tr>
<tr>
<td><code>policy_version</code>, <code>harness_version</code>, <code>classifier_location</code></td>
<td>Your agent or settings version; the CLI or API version; <code>server</code>, <code>local</code> or <code>unknown</code></td>
</tr>
</tbody>
</table>
<p><code>static_policy</code> covers anything you configured and versioned, including hooks and a classifier gate of your own. <code>vendor_classifier</code> is a verdict the vendor’s model made and told you about. <code>human</code> is a person answering a pause.</p>
<p><code>unavailable</code> is a first-class value in both columns, and the reason is the rest of this runbook. A verdict you never received is not an allow. The habit of writing one evidence line per gate decision comes from <a href="/intel/pre-action-gates-not-dashboards/">pre-action gates</a>; this schema extends it to decisions somebody else’s server made.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span><span style="color:#79B8FF">"ts"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"2026-09-21T14:02:07Z"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"lane"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"billing-api"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"session_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"&lt;vendor session id&gt;"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"event_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"&lt;vendor event id&gt;"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"tool_use_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"&lt;vendor call id&gt;"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"tool"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"bash"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"call_ref"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"calls/2026-09-21/billing-api/0412.json"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"call_sha256"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"9f2c…"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"verdict"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"allow"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"decided_by"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"vendor_classifier"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"vendor_raw"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"evaluated_permission"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"allow"</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"source"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"managed-agents:events"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"policy_version"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"agent-def@14"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"harness_version"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"managed-agents-2026-04-01"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"classifier_location"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"server"</span><span style="color:#E1E4E8">}</span></span></code></pre>
<p>The row is illustrative. Store the exact call input where your transcripts already live, under the same access rules: tool inputs carry file contents and sometimes secrets, and a hash alone can’t tell a reviewer what ran.</p>
<h2 id="step-2-map-claude-managed-agents-permission-policy-fields-and-no-others">Step 2: Map Claude Managed Agents permission policy fields, and no others</h2>
<p>Your normalizer reads three event types. On <code>agent.tool_use</code> and <code>agent.mcp_tool_use</code>, <code>evaluated_permission</code> is the verdict. <code>decided_by</code> comes from the policy you configured for that tool, which is why <code>policy_version</code> sits in the row: an <code>allow</code> under <code>always_allow</code> came from your configuration, and an <code>allow</code> under <code>auto</code> came from the server’s judgment.</p>
<table>
<thead>
<tr>
<th>What the event carries</th>
<th><code>verdict</code></th>
<th><code>decided_by</code></th>
<th>Destination</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>evaluated_permission: "allow"</code>, tool on <code>always_allow</code></td>
<td><code>allow</code></td>
<td><code>static_policy</code></td>
<td>Evaluation log</td>
</tr>
<tr>
<td><code>evaluated_permission: "allow"</code>, tool on <code>auto</code></td>
<td><code>allow</code></td>
<td><code>vendor_classifier</code></td>
<td>Evaluation log; it already ran</td>
</tr>
<tr>
<td><code>evaluated_permission: "deny"</code>, <code>reason_code</code> <code>high_risk</code></td>
<td><code>deny</code></td>
<td><code>vendor_classifier</code></td>
<td>Evaluation log; not overridable</td>
</tr>
<tr>
<td><code>evaluated_permission: "ask"</code>, tool on <code>always_ask</code></td>
<td><code>ask</code></td>
<td><code>static_policy</code></td>
<td>Approval queue</td>
</tr>
<tr>
<td><code>evaluated_permission: "ask"</code>, <code>reason_code</code> <code>indeterminate</code></td>
<td><code>ask</code></td>
<td><code>vendor_classifier</code></td>
<td>Approval queue</td>
</tr>
<tr>
<td><code>evaluated_permission</code> present, no <code>evaluation</code> object</td>
<td>as stamped</td>
<td><code>unavailable</code></td>
<td>Evaluation log, flagged</td>
</tr>
<tr>
<td>No <code>evaluated_permission</code> (older events)</td>
<td><code>unavailable</code></td>
<td><code>unavailable</code></td>
<td>Evaluation log, flagged</td>
</tr>
<tr>
<td>Unrecognized <code>evaluation.type</code> or <code>reason_code</code></td>
<td>as stamped</td>
<td><code>unavailable</code></td>
<td>Evaluation log, flagged for a map update</td>
</tr>
<tr>
<td><code>agent.custom_tool_use</code></td>
<td><code>unavailable</code></td>
<td><code>unavailable</code></td>
<td>Your app’s gate writes the real row (Step 8)</td>
</tr>
</tbody>
</table>
<p><strong>Missing never means allow.</strong> The <code>evaluation</code> object is absent when the agent names a tool the session doesn’t enable (a deny with no evaluation) and on events recorded before the field existed. Unknown values stay raw: the docs tell clients to tolerate an <code>evaluation.type</code> or <code>reason_code</code> they don’t recognize, so keep it in <code>vendor_raw</code> and leave <code>decided_by</code> at <code>unavailable</code> until you update the map. Custom tool events carry neither field, because permission policies don’t govern custom tools.</p>
<p>An answered pause gets its second row from the confirmation you send (Step 5): same <code>event_id</code>, <code>allow</code> or <code>deny</code>, <code>decided_by: human</code>.</p>
<h2 id="step-3-map-claude-code-honestly-which-mostly-means-denials">Step 3: Map Claude Code honestly, which mostly means denials</h2>
<p>Claude Code gives you two documented sources, and neither can name a classifier allow.</p>
<p>The <a href="https://code.claude.com/docs/en/hooks"><code>PermissionDenied</code> hook</a> runs when auto mode denies a tool call, including a denial made without a classifier verdict. It receives <code>tool_name</code>, <code>tool_input</code>, <code>tool_use_id</code> and a <code>reason</code> (plus <code>mcp_server</code> for MCP tools). “This hook only fires in auto mode”: a denial you click, a <code>PreToolUse</code> block or a matching <code>deny</code> rule never reaches it. Its only output is a request to retry, and exit codes are ignored, so it is a logging point, never a gate.</p>
<p>The OTel <code>tool_decision</code> event in the <a href="https://code.claude.com/docs/en/monitoring-usage">monitoring doc</a> records a <code>decision</code> of <code>accept</code> or <code>reject</code> and a <code>source</code> of <code>config</code>, <code>hook</code>, <code>user_permanent</code>, <code>user_temporary</code>, <code>user_abort</code> or <code>user_reject</code>. <code>config</code> lumps together settings, managed policy, flags and the active permission mode, and the doc is plain about it: “The event doesn’t indicate which of these sources matched.” No documented field or hook distinguishes a classifier allow from a rule allow, so no row of yours should claim one.</p>
<p>
<em>Screenshot: Claude Code Docs, “Hooks reference - Claude Code Docs” (undated), captured Sep 21, 2026.</em></p>
<table>
<thead>
<tr>
<th>Claude Code signal</th>
<th><code>verdict</code></th>
<th><code>decided_by</code></th>
</tr>
</thead>
<tbody>
<tr>
<td><code>PermissionDenied</code>, <code>reason</code> names a verdict such as <code>[Data Exfiltration]</code></td>
<td><code>deny</code></td>
<td><code>vendor_classifier</code></td>
</tr>
<tr>
<td><code>PermissionDenied</code>, <code>reason</code> starts <code>Auto mode could not evaluate this action and is blocking it for safety</code>, or reads <code>Classifier unavailable</code></td>
<td><code>deny</code></td>
<td><code>unavailable</code> (failed closed)</td>
</tr>
<tr>
<td><code>tool_decision</code>, <code>source: "hook"</code></td>
<td>from <code>decision</code></td>
<td><code>static_policy</code></td>
</tr>
<tr>
<td><code>tool_decision</code>, any <code>user_*</code> source</td>
<td>from <code>decision</code></td>
<td><code>human</code></td>
</tr>
<tr>
<td><code>tool_decision</code>, <code>source: "config"</code>, <code>decision: "accept"</code></td>
<td><code>allow</code></td>
<td><code>unavailable</code></td>
</tr>
<tr>
<td><code>tool_decision</code>, <code>source: "config"</code>, <code>decision: "reject"</code></td>
<td><code>deny</code></td>
<td>from the <code>PermissionDenied</code> row on the same <code>tool_use_id</code>, else <code>unavailable</code></td>
</tr>
<tr>
<td>No telemetry exported and no denial</td>
<td><code>unavailable</code></td>
<td><code>unavailable</code></td>
</tr>
</tbody>
</table>
<p>The logger for the first two rows records and cannot block.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#6A737D"># log_denial.py: illustrative PermissionDenied command hook; a logger, not a gate</span></span>
<span class="line"><span style="color:#6A737D"># LANE and LANE_CLI_VERSION are set by your runner, not by Claude Code</span></span>
<span class="line"><span style="color:#F97583">import</span><span style="color:#E1E4E8"> hashlib, json, os, sys, time</span></span>
<span class="line"><span style="color:#E1E4E8">e </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> json.load(sys.stdin)</span></span>
<span class="line"><span style="color:#E1E4E8">reason </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> e.get(</span><span style="color:#9ECBFF">"reason"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">""</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#E1E4E8">no_verdict </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> reason.startswith(</span><span style="color:#9ECBFF">"Auto mode could not evaluate"</span><span style="color:#E1E4E8">) </span><span style="color:#F97583">or</span><span style="color:#E1E4E8"> reason </span><span style="color:#F97583">==</span><span style="color:#9ECBFF"> "Classifier unavailable"</span></span>
<span class="line"><span style="color:#E1E4E8">row </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> {</span></span>
<span class="line"><span style="color:#9ECBFF">    "ts"</span><span style="color:#E1E4E8">: time.strftime(</span><span style="color:#9ECBFF">"%Y-%m-</span><span style="color:#79B8FF">%d</span><span style="color:#9ECBFF">T%H:%M:%SZ"</span><span style="color:#E1E4E8">, time.gmtime()),</span></span>
<span class="line"><span style="color:#9ECBFF">    "lane"</span><span style="color:#E1E4E8">: os.environ.get(</span><span style="color:#9ECBFF">"LANE"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"unknown"</span><span style="color:#E1E4E8">),</span></span>
<span class="line"><span style="color:#9ECBFF">    "session_id"</span><span style="color:#E1E4E8">: e.get(</span><span style="color:#9ECBFF">"session_id"</span><span style="color:#E1E4E8">),</span></span>
<span class="line"><span style="color:#9ECBFF">    "tool_use_id"</span><span style="color:#E1E4E8">: e[</span><span style="color:#9ECBFF">"tool_use_id"</span><span style="color:#E1E4E8">],</span></span>
<span class="line"><span style="color:#9ECBFF">    "tool"</span><span style="color:#E1E4E8">: e[</span><span style="color:#9ECBFF">"tool_name"</span><span style="color:#E1E4E8">],</span></span>
<span class="line"><span style="color:#9ECBFF">    "call_sha256"</span><span style="color:#E1E4E8">: hashlib.sha256(json.dumps(e[</span><span style="color:#9ECBFF">"tool_input"</span><span style="color:#E1E4E8">], </span><span style="color:#FFAB70">sort_keys</span><span style="color:#F97583">=</span><span style="color:#79B8FF">True</span><span style="color:#E1E4E8">).encode()).hexdigest(),</span></span>
<span class="line"><span style="color:#9ECBFF">    "verdict"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"deny"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#9ECBFF">    "decided_by"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"unavailable"</span><span style="color:#F97583"> if</span><span style="color:#E1E4E8"> no_verdict </span><span style="color:#F97583">else</span><span style="color:#9ECBFF"> "vendor_classifier"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#9ECBFF">    "vendor_raw"</span><span style="color:#E1E4E8">: {</span><span style="color:#9ECBFF">"reason"</span><span style="color:#E1E4E8">: reason, </span><span style="color:#9ECBFF">"mcp_server"</span><span style="color:#E1E4E8">: e.get(</span><span style="color:#9ECBFF">"mcp_server"</span><span style="color:#E1E4E8">)},</span></span>
<span class="line"><span style="color:#9ECBFF">    "source"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"claude-code:PermissionDenied"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#9ECBFF">    "harness_version"</span><span style="color:#E1E4E8">: os.environ.get(</span><span style="color:#9ECBFF">"LANE_CLI_VERSION"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"unknown"</span><span style="color:#E1E4E8">),</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#F97583">with</span><span style="color:#79B8FF"> open</span><span style="color:#E1E4E8">(os.path.expanduser(</span><span style="color:#9ECBFF">"~/.fleet/verdicts.jsonl"</span><span style="color:#E1E4E8">), </span><span style="color:#9ECBFF">"a"</span><span style="color:#E1E4E8">) </span><span style="color:#F97583">as</span><span style="color:#E1E4E8"> f:</span></span>
<span class="line"><span style="color:#E1E4E8">    f.write(json.dumps(row) </span><span style="color:#F97583">+</span><span style="color:#9ECBFF"> "</span><span style="color:#79B8FF">\n</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">)</span></span></code></pre>
<p>Then record where the classifier ran, per session. Per the <a href="https://code.claude.com/docs/en/auto-mode-classifier-billing">classifier billing doc</a>, <code>/status</code> has an “Auto mode server” row that reads <code>Enabled</code> while the server decides and <code>Disabled</code> after Claude Code falls back to its own billed classifier requests, which happens when a gateway or proxy strips or rewrites headers, drops unrecognized request fields or edits responses. Headless lanes get the fallback notice on stderr under <code>-p</code> and as a <code>system</code> warning in <code>stream-json</code>; capture it and set <code>classifier_location</code> to <code>local</code>. Pro, Max and Team plans never see that notice, so its absence proves nothing there: leave those sessions at <code>unknown</code>. Keep <code>harness_version</code> on every row, because Bedrock, Vertex and Foundry sessions on 2.1.273 through 2.1.277 default to the local classifier and 2.1.278 onward to the server.</p>
<p>One trap: <code>/permissions</code> lists auto-mode denials under Recently denied, but per the <a href="https://code.claude.com/docs/en/auto-mode-config">auto mode configuration doc</a> not the no-verdict ones. Build the denial log from the hook.</p>
<p>Any lane whose harness documents no verdict field gets <code>unavailable</code> in both columns until it does. Don’t reconstruct a vendor verdict from timing, from a missing prompt, or from a transcript line that reads like approval.</p>
<h2 id="step-4-split-the-stream-evidence-for-allow-and-deny-the-queue-for-ask">Step 4: Split the stream: evidence for allow and deny, the queue for ask</h2>
<p>This split is the answer to the headline. Of the four verdicts, exactly one still needs a person.</p>
<table>
<thead>
<tr>
<th><code>verdict</code></th>
<th>Where it goes</th>
<th>What a person can still do</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>allow</code></td>
<td>Evaluation log</td>
<td>Nothing before the fact. Under <code>auto</code> the call has run; review a weekly sample afterwards</td>
</tr>
<tr>
<td><code>deny</code></td>
<td>Evaluation log</td>
<td>Nothing through the client. Change the policy or the task, then let the agent retry</td>
</tr>
<tr>
<td><code>ask</code></td>
<td>Approval queue, one item per blocking event</td>
<td>Answer it through the vendor’s confirm path (Step 5) before the expiry (Step 6)</td>
</tr>
<tr>
<td><code>unavailable</code></td>
<td>Evaluation log, flagged</td>
<td>Fix the lane: add a verdict source, or put your own gate in front of it</td>
</tr>
</tbody>
</table>
<p>
<em>Allow and deny are evidence. Ask is the only verdict that still waits for a person. Unavailable is a gap you close with your own gate.</em></p>
<p>Allows and denials are evidence, and evidence has a retention clock you don’t control on a vendor-hosted session. Export the evaluation log on a schedule, the same way you’d <a href="/intel/vendor-hosted-agent-evidence/">export the agent audit trail before access changes</a>. The <code>unavailable</code> share per lane is a coverage gap, and it belongs on the weekly page from <a href="/intel/fleet-oversight-three-numbers/">the three oversight numbers</a>, printed rather than hidden.</p>
<p>
<em>Illustrative: modeled shares, not measured data. On a Claude Code auto-mode lane most allows land in <code>unavailable</code>, because no documented field names the classifier.</em></p>
<p>Don’t try to fix a <code>deny</code> by sending a confirmation. The API refuses it: “If you send a <code>user.tool_confirmation</code> for an event whose <code>evaluated_permission</code> is not <code>ask</code>, the API rejects it with a 400 error.” A spike of 400s in your confirm path means your queue is admitting events it shouldn’t.</p>
<h2 id="step-5-answer-each-pause-through-the-vendors-own-confirm-path">Step 5: Answer each pause through the vendor’s own confirm path</h2>
<p>A Managed Agents pause emits <code>session.status_idle</code> with <code>stop_reason.type: requires_action</code> and the blocking IDs in <code>stop_reason.event_ids</code>. Build one queue item per blocking event and carry the lane, the call pointer, the vendor’s <code>reason_code</code> and the session ID on it. The approver’s answer goes back as a <code>user.tool_confirmation</code> with <code>tool_use_id</code>, a <code>result</code> of <code>allow</code> or <code>deny</code>, and an optional <code>deny_message</code>.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"user.tool_confirmation"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"tool_use_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"&lt;id of the paused call&gt;"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"result"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"deny"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"deny_message"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"Pushes to main need a release ticket. Open one and retry."</span><span style="color:#E1E4E8">}</span></span></code></pre>
<p>Illustrative body; the field names are from the docs, and the transport is whatever your client already uses. For a person at a terminal, <code>ant beta:sessions connect</code> covers the same ground, and its <code>--web</code> flag serves the Console’s session viewer locally.</p>
<p>Answer only through that path; a call your runner executes outside the session has no vendor verdict and no place in the event stream. Answer each blocking event separately, so one approval can’t cover a different call that paused in the same idle. Write the resolution row the moment the confirmation is accepted, with the approver’s identity beside <code>decided_by: human</code>.</p>
<p>On Claude Code lanes the confirm path is the session’s own permission prompt. Your record of it arrives as a <code>tool_decision</code> with a <code>user_*</code> source.</p>
<h2 id="step-6-put-your-own-expiry-on-every-native-pause">Step 6: Put your own expiry on every native pause</h2>
<p>“The session waits indefinitely for a response.” That line describes the vendor’s behaviour, and nothing obliges you to copy it. The docs describe no timeout on a pause, so the clock is yours.</p>
<p>Set expiry per consequence tier, using the tiers your approval queue already runs. When a pause expires, send <code>result: "deny"</code> with a <code>deny_message</code> naming the expiry, then write the row as <code>deny</code>, <code>decided_by: static_policy</code>, with the expiry rule’s version in <code>policy_version</code>. A reviewer who clicks later should hit a closed item. Don’t count on the vendor to reject a second confirmation for the same call; close the item on your side.</p>
<p>Count expiry denials per lane each week. A lane whose pauses keep expiring has the wrong tier, a thin reviewer rota, or a tool on the wrong policy.</p>
<h2 id="step-7-move-anything-a-person-must-see-first-to-always_ask">Step 7: Move anything a person must see first to <code>always_ask</code></h2>
<p>An <code>auto</code> allow runs before anyone sees it, and an <code>auto</code> deny can’t be overridden. That leaves a person exactly one lever under <code>auto</code>: the pauses the server chooses to raise. Anything a person must see before it runs goes on <code>always_ask</code>, which pauses every call to that tool.</p>
<p>Walk each lane’s tool list with this checklist:</p>
<ol>
<li><strong>Irreversible effects.</strong> Pushes to protected branches, deploys, deletes, outbound messages, anything that moves money. These go on <code>always_ask</code>, or off the agent entirely.</li>
<li><strong>Tools reachable by relayed end-user text.</strong> What you post in <code>user.message</code> counts as your intent and can lead the server to allow a call it would otherwise deny. Relay untrusted end-user input there and the server reads it as your intent too; Anthropic’s advice is <code>always_ask</code> on tools that user must not run unreviewed. The server doesn’t take intent from tool results, fetched pages, MCP responses or messages between session threads, so the risk is the relay you built.</li>
<li><strong>MCP toolsets.</strong> They default to <code>always_ask</code>. Moving one to <code>auto</code> is a decision with a name and a date on it.</li>
<li><strong>The agent toolset.</strong> It defaults to <code>always_allow</code>, which stamps <code>allow</code> on every call without judging any of them. Choose <code>auto</code> or <code>always_ask</code> per tool on purpose rather than inheriting the default.</li>
</ol>
<p>The normalizer, the log and the queue from Steps 1 to 6 are records and routing. <strong>They don’t enforce anything.</strong> If your pipeline drops an event, the call it described still ran or still stopped exactly as the vendor decided. The walls are the vendor-side policy (<code>always_ask</code>, disabled tools), your harness deny rules and the sandbox under the agent. Watch the pipeline itself: compare event counts per session in the vendor stream against rows in your log, and alert when they diverge.</p>
<h2 id="step-8-gate-custom-tools-in-the-app-that-runs-them">Step 8: Gate custom tools in the app that runs them</h2>
<p><code>agent.custom_tool_use</code> events carry neither <code>evaluated_permission</code> nor <code>evaluation</code>. Your application executes custom tools, so your application is the gate, and the vendor verdict for those calls is permanently <code>unavailable</code>.</p>
<p>Write your gate’s decision into the same schema with <code>decided_by</code> set to <code>static_policy</code> or <code>human</code>, never <code>vendor_classifier</code>. Fail closed: if the gate errors or times out, the row reads <code>deny</code>, the error goes in <code>vendor_raw</code>, and the tool doesn’t run.</p>
<h2 id="four-ways-a-verdict-pipeline-misleads-you-and-the-signal-for-each">Four ways a verdict pipeline misleads you, and the signal for each</h2>
<p><strong>Absent read as allow.</strong> A mapper defaults a missing <code>evaluation</code> or <code>evaluated_permission</code> to <code>allow</code>. Signal: <code>allow</code> rows with an empty <code>vendor_raw</code>. Fix: map absence to <code>unavailable</code>, and fail the mapper’s tests on any default.</p>
<p><strong>Relayed text talks the server into an allow.</strong> An app forwards customer messages as <code>user.message</code> and the server reads them as your intent. Signal: <code>auto</code> allows on sensitive tools in sessions whose <code>user.message</code> came from a relay. Fix: <code>always_ask</code> on those tools, then re-read Step 7.</p>
<p><strong>The classifier moved and nobody noticed.</strong> A CLI upgrade or a header-rewriting proxy changes who decided. Signal: <code>classifier_location</code> flips between <code>server</code> and <code>local</code> within a lane, or the billed-fallback notice appears in headless stderr. Fix: record it per session and treat an unexpected flip as a change event.</p>
<p><strong>The record goes quiet.</strong> A hook stops firing or telemetry export breaks, and the log looks calm. Signal: rows per session fall against the vendor’s event count, or a lane shows zero denials for a week. Fix: alert on the divergence, and remember the enforcement never lived in the log.</p>
<h2 id="one-verdict-schema-is-how-a-fleet-keeps-its-own-record">One verdict schema is how a fleet keeps its own record</h2>
<p>Every vendor will keep shipping its own permission dialect: a server-side policy here, a hook that fires only on denials there, a managed ask with its own override rules somewhere else. None of them will normalize the others for you. The schema, the split and the expiry belong in the layer that runs the whole fleet, beside the kill switch and the session inventory, which is the job of a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> rather than any one vendor’s console.</p>
<p>Run the split for a week. Most rows will be evidence. A few will be real asks with a clock on them. The <code>unavailable</code> count is the honest number, and it is the one to shrink.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-claude-managed-agents-auto-mode-replace-human-approval">Does Claude Managed Agents auto mode replace human approval?</h3>
<p>No. Anthropic’s docs say <code>auto</code> is not a human checkpoint: a call the server judges safe runs before anyone looks, and your client cannot override a denial. Only calls the server can’t decide pause for you. Put <code>always_ask</code> on any tool a person must review before it runs, and give those pauses your own expiry.</p>
<h3 id="can-i-see-which-tool-calls-claude-codes-auto-mode-classifier-allowed">Can I see which tool calls Claude Code’s auto mode classifier allowed?</h3>
<p>Not from any documented field. The <code>PermissionDenied</code> hook fires on auto-mode denials only, and the OTel <code>tool_decision</code> event reports permission-mode decisions under <code>source: "config"</code> without saying which source matched. Log those allows with <code>decided_by: unavailable</code> rather than claiming a classifier verdict you never received.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://platform.claude.com/docs/en/managed-agents/permission-policies">Claude Platform Docs: Permission policies</a> — policy types, verdict fields, confirmations, custom tools, intent</li>
<li><a href="https://platform.claude.com/docs/en/release-notes/overview">Claude Platform release notes</a> — Sep 10, 2026: <code>auto</code>; Ant CLI 1.32.0</li>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a> — 2.1.278 (Sep 19, 2026) and 2.1.273 (Sep 15, 2026)</li>
<li><a href="https://code.claude.com/docs/en/hooks">Claude Code hooks reference</a> — <code>PermissionDenied</code></li>
<li><a href="https://code.claude.com/docs/en/monitoring-usage">Claude Code monitoring</a> — OTel <code>tool_decision</code></li>
<li><a href="https://code.claude.com/docs/en/permission-modes">Claude Code permission modes</a> — server classifier default and opt-out scope</li>
<li><a href="https://code.claude.com/docs/en/auto-mode-classifier-billing">Claude Code auto mode classifier billing</a> — fallback, notice, <code>/status</code> row</li>
<li><a href="https://code.claude.com/docs/en/auto-mode-config">Claude Code auto mode configuration</a> — Recently denied</li>
</ul>
]]></content:encoded></item><item><title>Unauthored Hooks Are Persistence: Remove Them and Prove They Stay Gone</title><link>https://automater.ai/intel/unauthored-agent-hooks-persistence/</link><guid>https://automater.ai/intel/unauthored-agent-hooks-persistence/</guid><description>Claude Code hooks malware survives a declined consent prompt. Baseline hooks, logon tasks and PATH shims off-box, allowlist helpers, prove removal holds.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>Declining <code>llm-interceptor</code>’s consent prompt removes nothing. By the time the prompt appears, its <code>postinstall.js</code> has already registered MCP servers, written a <code>SessionEnd</code> hook into <code>~/.claude/settings.json</code> and set up three separate ways to restart itself on Windows. OSV’s record says the refusal leaves every one of them in place. That is Claude Code hooks malware at its plainest: a line in a settings file that runs someone else’s code each time a session ends, written by a package nobody approved.</p>
<p>Uninstalling the package doesn’t fix it either. npm removes the files it unpacked. It keeps no record of what a script wrote into your home directory, your Task Scheduler or your PATH.</p>
<p>Treat unauthored hooks like any other persistence mechanism: inventory, alert, remove in order, prove. By Tuesday, every agent host should have an off-box baseline of its hooks, logon tasks, autostart entries and PATH shims, with the resolved path and hash of everything each one launches.</p>
<p>A change nobody authored pages someone. Agent-helper packages install only from an allowlist, in a project, through <code>npx</code> or globally. And every cleanup ends with evidence that survives a reboot.</p>
<p>Chatbots suggest; agents act. A hook acts even when nobody is prompting the agent, because the harness launches it on the event, with the user’s rights, under the name of a tool you trust.</p>
<h2 id="sep-6-18-agent-helper-packages-on-npm-claude-code-hooks-malware-among-them">Sep 6: 18 agent-helper packages on npm, Claude Code hooks malware among them</h2>
<p>On Sep 6, 2026, AgentGate published <a href="https://dev.to/agentgate/18-malicious-npm-packages-are-still-remote-controlling-ai-coding-agents-verified-today-3n59">a list of 18 npm packages</a> it calls malicious and says were still installable that day. AgentGate sells an agent-config scanner, so read it as vendor research. Three entries map onto this runbook’s surfaces. <code>llm-interceptor</code> “registers an MCP server in ~/.cursor/mcp.json, runs claude mcp add, installs a Claude Code SessionEnd hook, and on Windows creates a logon task.” <code>remote-claude-daemon</code> takes tasks over an outbound connection and hands them to local agents “with permission prompts explicitly bypassed.” <code>opencode-optimised-toolings</code> “renames your on-PATH opencode aside and installs its own build in its place”.</p>
<p>OSV’s records, with text from Amazon Inspector, add detail. <a href="https://osv.dev/vulnerability/MAL-2026-13370">MAL-2026-13370</a> says <code>llm-interceptor</code> runs its setup before any consent dialog, sends a heartbeat every 15 minutes regardless of consent, and “installs three redundant Windows persistence mechanisms”: a scheduled task with an ONLOGON trigger, a 5-minute pulse and RestartOnFailure; an HKCU Run key; and a Startup-folder shortcut. <a href="https://osv.dev/vulnerability/MAL-2026-13455">MAL-2026-13455</a> says <code>remote-claude-daemon</code> “spawns the local <code>claude</code> binary with <code>--continue -p --dangerously-skip-permissions</code> and the remote-supplied prompt as input”.</p>
<p><a href="https://osv.dev/vulnerability/MAL-2026-13452">MAL-2026-13452</a> says the OpenCode package also rewrites <code>~/.config/opencode/opencode.jsonc</code>, injects a marked block into <code>AGENTS.md</code> and pins itself in <code>tui.json</code>. <a href="https://osv.dev/vulnerability/MAL-2026-5789">MAL-2026-5789</a> says <code>claude-cup</code> runs on the first <code>npx claude-cup</code> and then on every Claude Code tool call once its hooks are in.</p>
<p>
<em>Screenshot: OSV, “MAL-2026-13370 - OSV” (Aug 5, 2026), captured Sep 21, 2026.</em></p>
<p>On Sep 21 we checked the npm registry: all four still resolve, at the latest versions AgentGate listed (0.4.1, 0.7.5, 6.5.2 and 0.9.12). Still resolving means npm hadn’t removed them; it is not a separate finding of malice.</p>
<p>The same day, an OSV query by exact version returned no advisory for <code>remote-claude-daemon@0.7.5</code> or <code>opencode-optimised-toolings@6.5.2</code>, the current latest of each. A scanner that matches OSV by version would pass both. Step 3 turns on that gap.</p>
<p>The thesis has a primary source too. <a href="https://cloud.google.com/security/resources/ai-risk-and-resilience-2026">Mandiant’s AI Risk and Resilience Report</a> (September 2026) includes Case study 3, “Subverting assistant CLI hooks for native remote code execution”. At an IT and software development organization, Mandiant writes, “a threat actor had successfully poisoned an internal AI repository and tampered with the assistant’s underlying CLI hooks.” Its controls include signing and verifying assistant binaries, CLI helpers, plugins and MCP servers before they run, and “multi-party approvals and strict code review policies for all skill/hook modifications”. That second control is how this runbook defines “authored”.</p>
<p>
<em>Screenshot: Google Cloud, “Mandiant AI Risk and Resilience Report 2026” (September 2026), captured Sep 21, 2026.</em></p>
<p>On Sep 14, <a href="https://code.claude.com/docs/en/changelog">Claude Code 2.1.271</a> added <code>--accept-command &lt;sha256&gt;</code> to <code>claude plugin install</code> and <code>claude plugin update</code>, “to accept exactly the command a previous <code>--json</code> run displayed, instead of <code>-y</code>”. The <a href="https://code.claude.com/docs/en/plugin-marketplaces">plugin marketplace docs</a> set its scope: “Claude Code runs only the command it showed, for the archive URL it showed.” That is command approval. It doesn’t hash what the plugin contains, and it says nothing about what the plugin writes into your settings once it runs.</p>
<h2 id="why-claude-code-hooks-malware-outlives-the-package-that-wrote-it">Why Claude Code hooks malware outlives the package that wrote it</h2>
<p>A hook is an entry in a settings file telling the harness to run something when an event fires. Claude Code’s <a href="https://code.claude.com/docs/en/hooks">hooks reference</a> lists five hook types, <code>command</code>, <code>http</code>, <code>mcp_tool</code>, <code>prompt</code> and <code>agent</code>, and the event decides when each one runs; a <code>SessionEnd</code> hook runs as every session closes. Once a package writes one, removing the package doesn’t remove the entry, and the harness keeps launching whatever it names.</p>
<p>An MCP registration gets its command started by every client that reads the config. A logon task runs at each sign-in. A PATH shim runs whenever anyone, human or agent, types the tool’s name.</p>
<p>That is why removing the package is not eradication, and why a scan of <code>node_modules</code> comes back clean on an infected host. The persistence lives in four places npm never tracked.</p>
<p><a href="/intel/marketplace-hygiene-plugin-mcp/">Marketplace hygiene for plugins and MCP servers</a> re-verifies entries you approved, and plugin pinning has its own runbook (step 2 links it). Neither covers what a package writes after install that nobody approved, or proving it’s gone.</p>
<h2 id="step-1-baseline-every-persistence-surface-off-the-box">Step 1: Baseline every persistence surface, off the box</h2>
<p>Keep one record per host, gathered by a collector the host runs and stored where the host can’t rewrite it. The unit is not the entry. It’s the entry plus the resolved path and SHA-256 of whatever the entry launches, because a shim swap leaves every entry’s text unchanged and alters only a hash.</p>
<table>
<thead>
<tr>
<th>Surface</th>
<th>Where to read it</th>
<th>Record for each entry</th>
</tr>
</thead>
<tbody>
<tr>
<td>Agent hooks</td>
<td><code>~/.claude/settings.json</code>, project <code>.claude/settings.json</code>, managed settings, each other CLI’s hook config</td>
<td>event, matcher, hook type, command or URL, resolved target, SHA-256 of target and script</td>
</tr>
<tr>
<td>MCP registrations</td>
<td><code>claude mcp list</code>, <code>~/.cursor/mcp.json</code>, each client’s MCP config</td>
<td>server name, command, args, resolved target, SHA-256</td>
</tr>
<tr>
<td>Logon and boot tasks (Windows)</td>
<td>Task Scheduler, every task with a logon or boot trigger</td>
<td>task path, trigger, action, resolved target, SHA-256</td>
</tr>
<tr>
<td>Run key and Startup folder (Windows)</td>
<td>HKCU <code>Run</code> values, the user’s Startup folder</td>
<td>value or shortcut, resolved target, SHA-256</td>
</tr>
<tr>
<td>PATH shims</td>
<td>every PATH hit for each agent CLI name, in order</td>
<td>order, path, SHA-256, unexpected neighbours</td>
</tr>
<tr>
<td>CLI config and instruction files</td>
<td><code>~/.config/opencode/opencode.jsonc</code>, <code>tui.json</code>, <code>AGENTS.md</code> in working repos</td>
<td>SHA-256, plus any marker block you didn’t write</td>
</tr>
<tr>
<td>Global npm packages</td>
<td>global list for every Node install on the host</td>
<td>name, version</td>
</tr>
<tr>
<td>Autostart on macOS and Linux</td>
<td>user-level launch agents and service units</td>
<td>path, target, SHA-256</td>
</tr>
</tbody>
</table>
<p>Four rules make the record worth diffing.</p>
<p><strong>Resolve before you hash.</strong> For a command like <code>node C:\Users\dev\.tool\index.js</code>, hash both the interpreter and the script. For a shortcut, hash its target. For a bare name, resolve it through the PATH the harness sees, which may not match your interactive shell’s.</p>
<p><strong>Hash every PATH hit, in order.</strong> OSV says that after the OpenCode swap, later <code>opencode</code> runs “execute the author-built binary under the trusted CLI name”. The first hit is what runs; the rest tell you what moved. A file beside the binary with a name like <code>.alonix-toolings-backup</code> is a finding on its own.</p>
<p><strong>Record strings where there is no file.</strong> An <code>http</code> hook has a URL and an <code>mcp_tool</code> hook has a server name. There’s nothing local to hash, so the string itself is the value that must not change.</p>
<p><strong>Ship it off-box, append-only.</strong> A baseline kept on the host is a notebook kept by the suspect. Push every run to a store the host’s user can’t edit, include the collector’s own hash in the record, and treat a missed check-in as an alert.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="powershell"><code><span class="line"><span style="color:#6A737D"># collect-persistence.ps1 - illustrative shape; one JSON line per entry, pushed off-box</span></span>
<span class="line"><span style="color:#E1E4E8">$rows </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> New-Object</span><span style="color:#E1E4E8"> System.Collections.Generic.List[</span><span style="color:#F97583">object</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#F97583">function</span><span style="color:#B392F0"> Sha</span><span style="color:#E1E4E8">($p) { </span><span style="color:#F97583">if</span><span style="color:#E1E4E8"> ($p </span><span style="color:#F97583">-and</span><span style="color:#E1E4E8"> (</span><span style="color:#79B8FF">Test-Path</span><span style="color:#F97583"> -</span><span style="color:#E1E4E8">LiteralPath $p)) { (</span><span style="color:#79B8FF">Get-FileHash</span><span style="color:#F97583"> -</span><span style="color:#E1E4E8">LiteralPath $p </span><span style="color:#F97583">-</span><span style="color:#E1E4E8">Algorithm SHA256).Hash } }</span></span>
<span class="line"></span>
<span class="line"><span style="color:#6A737D"># Agent hooks: repeat for every settings layer and every CLI's hook config</span></span>
<span class="line"><span style="color:#E1E4E8">$cfg </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> Get-Content</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$</span><span style="color:#79B8FF">env:</span><span style="color:#E1E4E8">USERPROFILE</span><span style="color:#9ECBFF">\.claude\settings.json"</span><span style="color:#F97583"> -</span><span style="color:#E1E4E8">Raw </span><span style="color:#F97583">|</span><span style="color:#79B8FF"> ConvertFrom-Json</span></span>
<span class="line"><span style="color:#F97583">foreach</span><span style="color:#E1E4E8"> ($ev </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> $cfg.hooks.PSObject.Properties) { </span><span style="color:#F97583">foreach</span><span style="color:#E1E4E8"> ($g </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> $ev.Value) { </span><span style="color:#F97583">foreach</span><span style="color:#E1E4E8"> ($hk </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> $g.hooks) {</span></span>
<span class="line"><span style="color:#E1E4E8">  $rows.Add(</span><span style="color:#F97583">@</span><span style="color:#E1E4E8">{ surface</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">'hook'</span><span style="color:#E1E4E8">; event</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$ev.Name; matcher</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$g.matcher; type</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$hk.type; cmd</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$hk.command; url</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$hk.url }) } } }</span></span>
<span class="line"></span>
<span class="line"><span style="color:#6A737D"># Logon and boot tasks</span></span>
<span class="line"><span style="color:#79B8FF">Get-ScheduledTask</span><span style="color:#F97583"> |</span><span style="color:#79B8FF"> Where-Object</span><span style="color:#E1E4E8"> { </span><span style="color:#79B8FF">$_</span><span style="color:#E1E4E8">.Triggers </span><span style="color:#F97583">|</span><span style="color:#79B8FF"> Where-Object</span><span style="color:#E1E4E8"> { </span><span style="color:#79B8FF">$_</span><span style="color:#E1E4E8">.CimClass.CimClassName </span><span style="color:#F97583">-match</span><span style="color:#9ECBFF"> 'Logon|Boot'</span><span style="color:#E1E4E8"> } } </span><span style="color:#F97583">|</span></span>
<span class="line"><span style="color:#79B8FF">  ForEach-Object</span><span style="color:#E1E4E8"> { $t </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> $_</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">foreach</span><span style="color:#E1E4E8"> ($a </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> $t.Actions) {</span></span>
<span class="line"><span style="color:#E1E4E8">    $rows.Add(</span><span style="color:#F97583">@</span><span style="color:#E1E4E8">{ surface</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">'task'</span><span style="color:#E1E4E8">; name</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"</span><span style="color:#F97583">$</span><span style="color:#9ECBFF">(</span><span style="color:#E1E4E8">$t.TaskPath</span><span style="color:#9ECBFF">)</span><span style="color:#F97583">$</span><span style="color:#9ECBFF">(</span><span style="color:#E1E4E8">$t.TaskName</span><span style="color:#9ECBFF">)</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">; exe</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$a.Execute; args</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$a.Arguments; sha</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">(Sha $a.Execute) }) } }</span></span>
<span class="line"></span>
<span class="line"><span style="color:#6A737D"># Run key values and Startup shortcuts</span></span>
<span class="line"><span style="color:#E1E4E8">(</span><span style="color:#79B8FF">Get-ItemProperty</span><span style="color:#9ECBFF"> 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Run'</span><span style="color:#E1E4E8">).PSObject.Properties </span><span style="color:#F97583">|</span></span>
<span class="line"><span style="color:#79B8FF">  Where-Object</span><span style="color:#E1E4E8"> Name </span><span style="color:#F97583">-notlike</span><span style="color:#9ECBFF"> 'PS*'</span><span style="color:#F97583"> |</span><span style="color:#79B8FF"> ForEach-Object</span><span style="color:#E1E4E8"> { $rows.Add(</span><span style="color:#F97583">@</span><span style="color:#E1E4E8">{ surface</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">'run'</span><span style="color:#E1E4E8">; name</span><span style="color:#F97583">=</span><span style="color:#79B8FF">$_</span><span style="color:#E1E4E8">.Name; value</span><span style="color:#F97583">=</span><span style="color:#79B8FF">$_</span><span style="color:#E1E4E8">.Value }) }</span></span>
<span class="line"><span style="color:#E1E4E8">$sh </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> New-Object</span><span style="color:#F97583"> -</span><span style="color:#E1E4E8">ComObject WScript.Shell</span></span>
<span class="line"><span style="color:#79B8FF">Get-ChildItem</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$</span><span style="color:#79B8FF">env:</span><span style="color:#E1E4E8">APPDATA</span><span style="color:#9ECBFF">\Microsoft\Windows\Start Menu\Programs\Startup"</span><span style="color:#F97583"> -</span><span style="color:#E1E4E8">Filter </span><span style="color:#F97583">*</span><span style="color:#E1E4E8">.lnk </span><span style="color:#F97583">|</span><span style="color:#79B8FF"> ForEach-Object</span><span style="color:#E1E4E8"> {</span></span>
<span class="line"><span style="color:#E1E4E8">  $tgt </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> $sh.CreateShortcut(</span><span style="color:#79B8FF">$_</span><span style="color:#E1E4E8">.FullName).TargetPath; $rows.Add(</span><span style="color:#F97583">@</span><span style="color:#E1E4E8">{ surface</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">'startup'</span><span style="color:#E1E4E8">; lnk</span><span style="color:#F97583">=</span><span style="color:#79B8FF">$_</span><span style="color:#E1E4E8">.Name; target</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$tgt; sha</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">(Sha $tgt) }) }</span></span>
<span class="line"></span>
<span class="line"><span style="color:#6A737D"># Every PATH hit for each agent CLI, in resolution order</span></span>
<span class="line"><span style="color:#F97583">foreach</span><span style="color:#E1E4E8"> ($n </span><span style="color:#F97583">in</span><span style="color:#9ECBFF"> 'claude'</span><span style="color:#F97583">,</span><span style="color:#9ECBFF">'opencode'</span><span style="color:#F97583">,</span><span style="color:#9ECBFF">'codex'</span><span style="color:#E1E4E8">) {</span></span>
<span class="line"><span style="color:#79B8FF">  Get-Command</span><span style="color:#E1E4E8"> $n </span><span style="color:#F97583">-</span><span style="color:#E1E4E8">All </span><span style="color:#F97583">-</span><span style="color:#E1E4E8">ErrorAction SilentlyContinue </span><span style="color:#F97583">|</span><span style="color:#79B8FF"> ForEach-Object</span><span style="color:#E1E4E8"> { $rows.Add(</span><span style="color:#F97583">@</span><span style="color:#E1E4E8">{ surface</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">'path'</span><span style="color:#E1E4E8">; name</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$n; path</span><span style="color:#F97583">=</span><span style="color:#79B8FF">$_</span><span style="color:#E1E4E8">.Source; sha</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">(Sha </span><span style="color:#79B8FF">$_</span><span style="color:#E1E4E8">.Source) }) } }</span></span>
<span class="line"></span>
<span class="line"><span style="color:#E1E4E8">$rows </span><span style="color:#F97583">|</span><span style="color:#79B8FF"> ForEach-Object</span><span style="color:#E1E4E8"> { </span><span style="color:#79B8FF">$_</span><span style="color:#F97583"> |</span><span style="color:#79B8FF"> ConvertTo-Json</span><span style="color:#F97583"> -</span><span style="color:#E1E4E8">Compress }   </span><span style="color:#6A737D"># pipe to the append-only store, never only to local disk</span></span></code></pre>
<p>The same shape works in shell on macOS and Linux. OSV’s <code>llm-interceptor</code> record describes Windows persistence only. That’s a fact about one record, not about the next package, so baseline the user-level autostart locations on every OS you run.</p>
<p>
<em>Documented behaviour only, from OSV records (text from Amazon Inspector) and AgentGate. A dash means the sources don’t describe it, not that the package doesn’t do it. Each column is a section of your baseline.</em></p>
<h2 id="step-2-alert-on-any-change-you-did-not-author">Step 2: Alert on any change you did not author</h2>
<p>“Authored” needs a definition a script can check. Here it means a change that reached the host through your change path, carrying a reviewed record that names the host group, the surface, the entry and the expected hash, approved by someone other than the requester. That is Mandiant’s multi-party approval for hook changes, applied to all four surfaces. Everything else is unauthored, including a legitimate plugin adding a hook during an update nobody announced.</p>
<table>
<thead>
<tr>
<th>Change in the diff</th>
<th>Example</th>
<th>Response</th>
</tr>
</thead>
<tbody>
<tr>
<td>New entry with no authored record</td>
<td>a <code>SessionEnd</code> hook appears in <code>~/.claude/settings.json</code></td>
<td>Page</td>
</tr>
<tr>
<td>Same entry, new hash</td>
<td><code>opencode</code> on PATH now hashes differently</td>
<td>Page</td>
</tr>
<tr>
<td>First PATH hit changes</td>
<td>a new directory lands ahead of the real binary</td>
<td>Page</td>
</tr>
<tr>
<td>New MCP registration</td>
<td>a server appears in <code>~/.cursor/mcp.json</code></td>
<td>Page, then inventory</td>
</tr>
<tr>
<td>Entry removed</td>
<td>a hook you deployed disappears</td>
<td>Ticket; page if it was a guard</td>
</tr>
<tr>
<td>Authored change, wrong hash</td>
<td>a plugin update wrote something other than the approved diff</td>
<td>Page; hold the rollout</td>
</tr>
<tr>
<td>Missed check-in</td>
<td>collector killed or host offline</td>
<td>Ticket; page on the second miss</td>
</tr>
</tbody>
</table>
<p>Run the collector at logon, on a timer (every 15 minutes is an illustrative starting point), and right after any install or update of an agent CLI, plugin or helper package. The post-install run earns its keep, because it lands the diff next to the command that caused it.</p>
<p>Plugins add hooks legitimately, so make the update itself the authored event. In automation, run <code>claude plugin install</code> or <code>claude plugin update</code> with <code>--json</code>, have a person approve the command it displays, and pass that run’s <code>sha256</code> to <code>--accept-command</code> instead of <code>-y</code>. Record the diff you expect, run the update, then diff for real.</p>
<p>The flag guarantees the command and archive URL a person saw; the diff tells you what that command wrote. Pinning the plugin itself to a commit is covered in the <a href="/intel/plugin4shell-pin-verify-fleet-patch/">Plugin4Shell pin-and-verify runbook</a>.</p>
<p>A new MCP registration is a persistence event and an inventory event at once. Alert on it here, then reconcile it through your <a href="/intel/mcp-server-inventory-ritual/">MCP server inventory ritual</a>. An entry nobody can explain gets the <a href="/intel/shadow-mcp/">shadow MCP</a> treatment.</p>
<h2 id="step-3-allowlist-agent-helper-packages-by-name-for-every-way-npm-runs-code">Step 3: Allowlist agent-helper packages by name, for every way npm runs code</h2>
<p>AgentGate says of its 18: “Each has an upstream OSV/GHSA record”. Per package, that holds. Per version it doesn’t, because two of the four current latest versions returned nothing from OSV on Sep 21.</p>
<p>A denylist knows only the versions someone has already recorded. Allowlist by name, then decide which versions of each name you accept.</p>
<p>The allowlist also has to cover every door, because npm lifecycle scripts run on local, <code>npx</code> and global installs alike. <code>llm-interceptor</code>’s payload starts in <code>postinstall</code>. <code>claude-cup</code>’s runs the first time someone types <code>npx claude-cup</code>, with no lockfile anywhere near it. <code>llm-interceptor</code> can reinstall itself globally on its collector’s instruction.</p>
<p>A lockfile review covers one of those three doors.</p>
<p>Enforce it where the doors meet: the registry path your hosts resolve through. In its case study 1, Mandiant recommends routing “all dependency traffic through secure, internal repositories” and validating AI-recommended dependencies “against cryptographic checksums and approved allowlists”. An internal registry or proxy that serves only allowlisted names does both, for every install mode.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># agent-helpers.allow.yaml - illustrative shape; served by the internal registry, reviewed like code</span></span>
<span class="line"><span style="color:#E1E4E8">- </span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"@your-org/agent-lint"</span></span>
<span class="line"><span style="color:#85E89D">  versions</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"2.4.x"</span></span>
<span class="line"><span style="color:#85E89D">  modes</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">project</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">npx</span><span style="color:#E1E4E8">]            </span><span style="color:#6A737D"># a global install needs its own approval</span></span>
<span class="line"><span style="color:#85E89D">  expected_writes</span><span style="color:#E1E4E8">: []              </span><span style="color:#6A737D"># surfaces it may touch at install or first run; empty = none</span></span>
<span class="line"><span style="color:#85E89D">  approved_by</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">owner-a</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">owner-b</span><span style="color:#E1E4E8">]  </span><span style="color:#6A737D"># two names: the multi-party rule</span></span>
<span class="line"><span style="color:#85E89D">  reviewed</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2026-09-21</span></span>
<span class="line"><span style="color:#E1E4E8">- </span><span style="color:#85E89D">name</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"@your-org/docs-mcp-server"</span></span>
<span class="line"><span style="color:#85E89D">  versions</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"1.9.x"</span></span>
<span class="line"><span style="color:#85E89D">  modes</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">project</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  expected_writes</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">mcp_registration</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  approved_by</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">owner-a</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">owner-c</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  reviewed</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2026-09-21</span></span></code></pre>
<p>The <code>expected_writes</code> field ties this step to step 1. If installing an allowlisted package produces a diff outside what its entry declares, that pages someone.</p>
<p>Decide what counts as an agent helper by behaviour, not by name: anything that writes agent config, registers MCP servers, wraps or replaces an agent CLI, or offers to meter, proxy or remote-control one. Registry keywords such as <code>mcp</code>, <code>claude-code</code> and <code>remote-desktop</code>, all used by packages on the list, can route a package to review. They don’t detect anything.</p>
<p>Be honest about the edges. A developer who points npm straight at the public registry, or copies a tarball by hand, walks around the allowlist; that’s why step 1 exists. Turning install scripts off is a sensible extra wall, but it does nothing about a payload that runs when invoked. The publish side of the same supply chain, keeping your own agents from pushing a poisoned version, is covered in <a href="/intel/agent-stage-only-publish-tokens/">stage-only npm tokens for agents</a>.</p>
<h2 id="step-4-eradicate-in-order-reboot-and-prove-the-hook-task-and-shim-stay-gone">Step 4: Eradicate in order, reboot, and prove the hook, task and shim stay gone</h2>
<p>Order matters, because <code>llm-interceptor</code> is built to come back: RestartOnFailure, a 5-minute pulse, three autostart paths and a self-update. Remove what respawns before you remove what it respawns.</p>
<ol>
<li><strong>Pull the host from agent work.</strong> Stop its sessions, take its lanes off the scheduler, and block outbound traffic to anything but your management plane. OSV says the heartbeat fires every 15 minutes; make the next one fail.</li>
<li><strong>Disable the respawners before killing processes.</strong> Disable the scheduled task and export its definition as evidence before you delete it. Remove the Run value and the Startup shortcut. Only then stop the processes they launch.</li>
<li><strong>Remove the agent-side entries.</strong> Delete the hook from every settings layer, not just <code>~/.claude/settings.json</code>. Remove the MCP registrations from every client, including <code>~/.cursor/mcp.json</code> and Claude Code’s own list.</li>
<li><strong>Restore the trusted binaries and files.</strong> Delete the impostor on PATH, reinstall the CLI from your approved source, and check its hash against the baseline before removing leftovers like <code>.alonix-toolings-backup</code>. Restore <code>opencode.jsonc</code> and <code>tui.json</code> from the baseline copy. Search every repo on the host for injected <code>AGENTS.md</code> blocks, committed ones included, because an instruction-file injection can leave the host inside a pushed commit.</li>
<li><strong>Remove the package from every scope.</strong> That means project <code>node_modules</code>, global installs under every Node version on the host, and the <code>npx</code> cache.</li>
<li><strong>Assume what it read is gone.</strong> <code>llm-interceptor</code> posts transcript turns to its collector. <code>claude-cup</code> harvests <code>.aws/credentials</code>, <code>.npmrc</code>, <code>.git-credentials</code>, SSH keys and <code>.env</code> files. Rotate whatever those held; credential brokering is its own discipline and out of scope here.</li>
<li><strong>Reboot and fire every trigger.</strong> Log on as the affected user, since an ONLOGON trigger needs a logon. Start and end one agent session so any <code>SessionEnd</code> hook fires, and make one tool call. Run each agent CLI by name once. Then leave the host up for at least two heartbeat intervals, which is 30 minutes against OSV’s 15-minute heartbeat.</li>
<li><strong>Diff, then diff again.</strong> Re-run the collector against the pre-incident baseline. The proof passes with zero unauthored delta and no outbound connection to the collector during the wait. Repeat at 24 hours and after the next agent CLI update.</li>
</ol>
<p>Keep the proof as a record, not a feeling:</p>
<table>
<thead>
<tr>
<th>Surface</th>
<th>Trigger you fired</th>
<th>Pass condition</th>
<th>Evidence to keep</th>
</tr>
</thead>
<tbody>
<tr>
<td>Agent hooks</td>
<td>session started and ended, one tool call</td>
<td>no hook outside the baseline</td>
<td>diff output</td>
</tr>
<tr>
<td>MCP registrations</td>
<td>a fresh session in each client</td>
<td>lists match the baseline</td>
<td><code>claude mcp list</code> output, config hashes</td>
</tr>
<tr>
<td>Logon task, Run key, Startup</td>
<td>reboot plus interactive logon</td>
<td>nothing new, nothing re-created</td>
<td>task export, Run values, folder listing</td>
</tr>
<tr>
<td>PATH shims</td>
<td>each agent CLI run by name</td>
<td>every PATH hit matches its hash</td>
<td>hash list</td>
</tr>
<tr>
<td>Config and <code>AGENTS.md</code></td>
<td>CLI launched in each repo</td>
<td>hashes match; no foreign marker blocks</td>
<td>hash list, repo search results</td>
</tr>
<tr>
<td>Packages</td>
<td>24 hours later</td>
<td>helper absent from every scope</td>
<td>package list per scope</td>
</tr>
<tr>
<td>Egress</td>
<td>30-minute wait</td>
<td>no connection to the collector</td>
<td>proxy or firewall log</td>
</tr>
</tbody>
</table>
<p>
<em>Four surfaces, one diff. The loop only closes when the diff comes back empty after a reboot and every trigger has fired.</em></p>
<h2 id="where-the-baseline-lies-to-you-and-what-gives-it-away">Where the baseline lies to you, and what gives it away</h2>
<p>The baseline is detection, not a boundary. It fails in specific ways, and each failure leaves a signal.</p>
<p><strong>The collector runs on the compromised host.</strong> Code that can write <code>~/.claude/settings.json</code> can edit a script in the same profile. Signal: the collector’s hash changes, a check-in goes missing, or a report comes back suspiciously short. Fix: run the collector from a path the user can’t write, and alert on silence. The wall behind it: agents run as standard users, with no admin rights on agent hosts.</p>
<p><strong>Legitimate updates bury the signal.</strong> Every CLI and plugin update produces diffs. If those aren’t authored events, people start closing alerts in bulk. Signal: unauthored alerts closed as expected with no record attached. Fix: route every update through step 2’s authored path, so the surprises left are real ones.</p>
<p><strong>A removed guard looks like a quiet day.</strong> An attacker who deletes your security hook produces a removal, not an addition. Signal: any guard hook missing from the diff, which should page. A guard can also stop matching without being removed, when a harness update renames the tools it watches; catching that is the contract-test problem in <a href="/intel/tool-schema-guard-contract-tests/">a renamed tool is a disabled guard</a>.</p>
<p><strong>The scanner is green and the host isn’t.</strong> Version-matched OSV scanning passes the current latest of two named packages. Signal: an allowlist violation alongside a clean scan. The allowlist wins.</p>
<p><strong>Removal without a reboot.</strong> Entries come back within the task’s pulse or at the next logon. Signal: the step 4 diff fails the second time. Fix: respawners first, then everything else.</p>
<p><strong>The user declined the prompt.</strong> Per OSV, declining removes nothing. Signal: an install of a non-allowlisted helper with no incident attached. Fix: any such install starts step 4, whatever the user clicked.</p>
<h2 id="persistence-is-a-fleet-inventory-not-a-package-ticket">Persistence is a fleet inventory, not a package ticket</h2>
<p>A package ticket closes once the package is gone. Persistence doesn’t, and on a fleet the same package may have reached several hosts through three install modes before anyone opened one. The baseline, the authored-change path, the allowlist and the proof record belong in the layer that runs the fleet, next to the inventory of which hosts run which agents under which identity. That’s the operating-layer argument behind <a href="/intel/corporate-ai-windows-pc/">corporate AI on the Windows PC</a>: the desk that launches agents has to be able to say what else launches on that desk.</p>
<p>The trigger won’t always be an npm list. If <a href="/intel/litellm-mcp-kev-patch-clock/">the LiteLLM MCP bypass on CISA’s KEV list</a> put tool calls with shell reach on your hosts, this runbook is the host-side follow-up. The move is the same either way: baseline, remove, reboot, diff, and keep the diff.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-claude-codes---accept-command-pin-a-plugins-contents">Does Claude Code’s <code>--accept-command</code> pin a plugin’s contents?</h3>
<p>No. Added in 2.1.271 on Sep 14, 2026, it accepts exactly the command and archive URL that a previous <code>--json</code> run displayed, and refuses if either changed. It does not hash the plugin’s files or check what the plugin writes after it runs. Diff your host baseline after every plugin install or update.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://dev.to/agentgate/18-malicious-npm-packages-are-still-remote-controlling-ai-coding-agents-verified-today-3n59">AgentGate: 18 malicious npm packages are still remote-controlling AI coding agents (verified today)</a> — Sep 6, 2026; vendor research</li>
<li><a href="https://osv.dev/vulnerability/MAL-2026-13370">OSV MAL-2026-13370, llm-interceptor</a> — published Aug 5, 2026; record text from Amazon Inspector</li>
<li><a href="https://osv.dev/vulnerability/MAL-2026-13455">OSV MAL-2026-13455, remote-claude-daemon</a> — published Aug 6, 2026</li>
<li><a href="https://osv.dev/vulnerability/MAL-2026-13452">OSV MAL-2026-13452, opencode-optimised-toolings</a> — published Aug 6, 2026</li>
<li><a href="https://osv.dev/vulnerability/MAL-2026-5789">OSV MAL-2026-5789, claude-cup</a> — published Jun 15, 2026</li>
<li><a href="https://cloud.google.com/security/resources/ai-risk-and-resilience-2026">Mandiant AI Risk and Resilience Report 2026</a> — September 2026; case studies 1 and 3</li>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a> — 2.1.271, Sep 14, 2026</li>
<li><a href="https://code.claude.com/docs/en/plugin-marketplaces">Claude Code docs: Create and distribute a plugin marketplace</a> — <code>--accept-command</code> scope</li>
<li><a href="https://code.claude.com/docs/en/hooks">Claude Code docs: Hooks reference</a> — hook types</li>
</ul>
]]></content:encoded></item><item><title>Automatic Agent Session Cleanup: Audit the Settings Before They Fire</title><link>https://automater.ai/intel/agent-session-cleanup-retention/</link><guid>https://automater.ai/intel/agent-session-cleanup-retention/</guid><description>VS Code agent session cleanup, Devin archive and Codex delete can remove sessions, PRs and worktrees. Audit each setting, dry-run it and test restore first.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category><content:encoded><![CDATA[<p>VS Code 1.138 shipped on Sep 16 with two settings that clean up agent sessions on a timer, and both arrive switched off. Nobody has to hunt for them, though: when a merged pull request makes a session eligible, the Mark as Done suggestion carries a button that opens both. VS Code agent session cleanup sits one number away from the default, and so does the question of what it removes at startup, and every hour after, with nobody watching.</p>
<p>In the same nine days, Devin’s archive learned to close child sessions’ pull requests, and Codex 0.155 put delete in its agents overview. Each harness draws its own line around what counts as finished and what leaves with the session: the record, the PRs its children opened, the worktree, and whatever git ignored inside it.</p>
<p>This runbook is the audit you run before typing a number into any of those settings. By Tuesday you will have an inventory of every cleanup trigger in the fleet, a map of what each one touches, a preflight that blocks on unpushed commits, dirty trees, open child PRs and pending approvals, an export of the files git would delete without asking, a dry run, and a restore test you have actually run.</p>
<h2 id="sep-917-vs-code-agent-session-cleanup-devins-archive-cascade-and-codex-delete">Sep 9–17: VS Code agent session cleanup, Devin’s archive cascade and Codex delete</h2>
<p><a href="https://code.visualstudio.com/updates/v1_138">VS Code 1.138</a>, released Sep 16, 2026, puts session cleanup in preview. Two settings drive it. <code>chat.agentSessions.autoMarkAsDoneMergedSessionsAfterDays</code> marks an inactive session done, and <code>chat.agentSessions.autoDeleteArchivedMergedSessionsAfterDays</code> deletes it after a separate grace period. The release notes are plain about the starting point: “Both automatic-cleanup settings are disabled by default.”</p>
<p>The <a href="https://code.visualstudio.com/docs/agents/run/sessions/manage-sessions">manage-sessions docs</a> hold the rules. Cleanup covers inactive Agent Host sessions only; “External sessions aren’t eligible.” A session qualifies when “it isn’t in progress, its last-modified time exceeds the configured threshold, it has at least one merged pull request, and none of its related pull requests are open.”</p>
<p>VS Code checks at startup and hourly while either setting is on. The recommended value is 15 days, 0 switches a stage off, and only sessions VS Code marked done itself are deleted automatically. Restoring one clears its deletion eligibility.</p>
<p>That rule is looser than the nudge, which fires when all of a session’s PRs have merged. Read literally, a session with one merged PR and one closed without merging qualifies. The docs don’t show that case, so treat it as an inference and settle it in step 5.</p>
<p>The worktree gets more care. When cleanup marks a session done or deletes it, VS Code removes the worktree “only when the branch tracks an upstream and has no outgoing commits or uncommitted changes”, keeps it when git state is unknown, and adds: “Cleanup never force-removes a worktree.” <a href="https://github.com/microsoft/vscode/pull/335851">PR #335851</a>, merged Sep 11 for 1.138, separates the paths: manual archive commits uncommitted changes before removing the worktree, while automatic cleanup removes only clean, synchronized ones.</p>
<p>Manual delete is harsher. For Copilot sessions it removes the associated worktrees, and “Deleting a session is irreversible.”</p>
<p>
<em>Screenshot: Visual Studio Code, “Manage agent sessions in VS Code” (undated), captured Sep 21, 2026.</em></p>
<p><a href="https://docs.devin.ai/release-notes/2026">Devin’s release notes</a> moved the reach sideways on Sep 9: archiving a session now also closes the open PRs of the child sessions archived with it, listed in the archive prompt so you can choose which to close. A human clicks archive; the cascade reaches PRs a reviewer may still be reading.</p>
<p><a href="https://github.com/openai/codex/releases/tag/rust-v0.155.0">Codex 0.155.0</a>, on Sep 17, added task hiding, archiving and deletion to the agents overview, plus “confirmed deletion of clean managed worktrees.” The notes say nothing about recoverability. The Codex app’s <a href="https://learn.chatgpt.com/docs/environments/git-worktrees">worktree docs</a> use a count cap instead of a timer: it keeps your 15 most recent managed worktrees by default, deletes on chat archive or to stay under the cap, and “Before deleting a Codex-managed worktree, Codex saves a snapshot of the work on it.” CLI checkouts share that pool, but <a href="https://github.com/openai/codex/pull/42652">PR #42652</a> leaves automatic cleanup disabled for them.</p>
<p>Part of the pressure is disk, as Niels Rogge of Hugging Face warned a month earlier:</p>
<aside class="social-embed" data-x-embed data-post-id="2089666343612072129">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">August 18, 2026 — A practitioner's warning that every Codex app thread creates or reuses a git worktree of about 200 MB, and that after hundreds of chats they pile up unless you clean them regularly.</p>
  <a class="social-embed__source" href="https://x.com/NielsRogge/status/2089666343612072129" target="_blank" rel="noopener noreferrer">View Niels Rogge's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="a-cleanup-setting-is-an-unattended-agent-with-delete-rights">A cleanup setting is an unattended agent with delete rights</h2>
<p>Chatbots suggest; agents act, and an agent session leaves behind things a chat log never had: a branch whose last commits may not be pushed, a worktree holding a test log and a <code>.env</code> that git ignores, child sessions with PRs open, an approval still waiting. Clearing a chat history deletes words. Clearing an agent session can close a colleague’s review or remove the only copy of a failing run’s log.</p>
<p>A timer that fires hourly is an actor with delete rights and nobody at the keyboard. Review it like one: what it may touch, what stops it, how you’d know it ran, and how you undo it.</p>
<h2 id="step-1-inventory-every-cleanup-trigger-and-mark-what-fires-unattended">Step 1: Inventory every cleanup trigger and mark what fires unattended</h2>
<p>List every setting and action in the fleet that can remove a session, a worktree, a branch or a pull request. Two columns set the priority: what starts it, and whether the docs describe an undo. A timer with no undo goes first.</p>
<table>
<thead>
<tr>
<th>Product</th>
<th>Setting or action</th>
<th>Starts it</th>
<th>Touches</th>
<th>Stated guard</th>
<th>Undo, per the docs</th>
</tr>
</thead>
<tbody>
<tr>
<td>VS Code 1.138 (preview)</td>
<td><code>chat.agentSessions.autoMarkAsDoneMergedSessionsAfterDays</code></td>
<td>Timer: at startup, then hourly</td>
<td>Session state; worktree if clean</td>
<td>Not in progress; past threshold; one or more merged PRs, none open</td>
<td>Restore clears deletion eligibility</td>
</tr>
<tr>
<td>VS Code 1.138 (preview)</td>
<td><code>chat.agentSessions.autoDeleteArchivedMergedSessionsAfterDays</code></td>
<td>Timer, after a separate grace period</td>
<td>Session; worktree if clean</td>
<td>Only sessions VS Code marked done itself</td>
<td>None; deletion is permanent</td>
</tr>
<tr>
<td>VS Code</td>
<td>Manual archive</td>
<td>Click</td>
<td>Worktree</td>
<td>Commits uncommitted changes first (PR #335851)</td>
<td>Not stated</td>
</tr>
<tr>
<td>VS Code</td>
<td>Manual delete</td>
<td>Click</td>
<td>Session and its worktrees (Copilot sessions)</td>
<td>Shared worktree kept until its last linked session goes</td>
<td>None; “irreversible”</td>
</tr>
<tr>
<td>Devin</td>
<td>Archive (Sep 9)</td>
<td>Click</td>
<td>Session; child sessions’ open PRs</td>
<td>PRs listed in the prompt; you choose</td>
<td>Not documented</td>
</tr>
<tr>
<td>Codex 0.155</td>
<td>Delete in agents overview</td>
<td>Click</td>
<td>Task; clean managed worktree</td>
<td>Confirmation; clean worktrees only</td>
<td>Not stated</td>
</tr>
<tr>
<td>Codex app</td>
<td>Worktree retention</td>
<td>Cap of 15, or archiving the chat</td>
<td>Managed worktree</td>
<td>Skips pinned, in-progress and permanent; CLI allocations exempt</td>
<td>Snapshot, restore from the chat</td>
</tr>
<tr>
<td>Claude Code</td>
<td>Background retention sweep</td>
<td>Age past <code>cleanupPeriodDays</code></td>
<td>Worktrees it made for subagents and background sessions</td>
<td>Keeps changed, untracked or unpushed work; marker check since 2.1.246</td>
<td>Not stated</td>
</tr>
<tr>
<td>Cursor 3.5+</td>
<td>Worktree cleanup</td>
<td>Every 6 h; cap of 25 per machine</td>
<td>Worktrees in its root, including ones it didn’t create</td>
<td>Dirty and locked handling not documented</td>
<td>Not stated</td>
</tr>
</tbody>
</table>
<p>Add a row for anything your own runner deletes, then mark each row <strong>timer</strong> or <strong>click</strong>, and <strong>local</strong> or <strong>vendor-hosted</strong>. Timer rows come first, because nothing asks you. Click rows with a cascade come second: the click is real, but it gets less attention than the cascade deserves.</p>
<p>Treat every stated guard as a claim about the vendor’s code, not a fact about your repo. Claude Code’s <a href="https://code.claude.com/docs/en/worktrees">worktree docs</a> record the precedent: before v2.1.246 on Aug 25, its retention sweep could remove a worktree you had created yourself when an old background-session record pointed at it. <a href="https://cursor.com/docs/configuration/worktrees">Cursor’s worktree docs</a> make foreign worktrees inside its root eligible on purpose, and don’t say whether dirty or locked ones are spared.</p>
<h2 id="step-2-map-the-shared-state-each-trigger-touches">Step 2: Map the shared state each trigger touches</h2>
<p>A session is the tip of a small graph: the record, the PRs it and its children opened, the worktree, and the branch behind it. Draw each trigger’s arrow into that graph and write its guard on the arrow.</p>
<p>
<em>Session state in the middle, cleanup actions at the sides. The ignored-files band is the one no vendor guard mentions.</em></p>
<p>Three edges deserve a line in your notes.</p>
<ul>
<li><strong>Child PRs.</strong> Devin’s archive closes whichever child-session PRs stay selected in the prompt. Your runbook’s default is none selected; a PR under someone else’s review closes when that person says so.</li>
<li><strong>Shared worktrees.</strong> VS Code keeps a worktree shared by several sessions until the last linked session is deleted or archived, which hides the dependency: removing that last session takes everyone’s work in the checkout.</li>
<li><strong>Branches.</strong> VS Code’s docs don’t say whether cleanup deletes the session’s branch, and neither do Devin’s or Codex’s notes. Write “not documented” and let the step 6 fixture answer it.</li>
</ul>
<p>
<em>Screenshot: Devin Docs, “2026 - Devin Docs” (Sep 9, 2026 entry), captured Sep 21, 2026.</em></p>
<h2 id="step-3-write-the-preflight-and-make-it-stricter-than-any-vendors-rule">Step 3: Write the preflight, and make it stricter than any vendor’s rule</h2>
<p>The preflight is the list of conditions that block removal, checked by a script you own. It runs before any cleanup you trigger, and on a schedule beside any a vendor triggers, so you learn what the vendor’s rule would have taken. Four blockers:</p>
<ol>
<li><strong>Unpushed commits.</strong> No upstream, or commits ahead of it, blocks.</li>
<li><strong>Dirty tree.</strong> Uncommitted changes or untracked files block.</li>
<li><strong>Open child PRs.</strong> Any open PR on the session’s branch, or opened by one of its child sessions, blocks.</li>
<li><strong>Pending approvals.</strong> A session waiting on a human decision blocks, even if the harness counts it as idle.</li>
</ol>
<p>Add a fifth for your own runner: a process still running inside the worktree blocks. A stopped worker is the precondition for everything else.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D">#!/usr/bin/env bash</span></span>
<span class="line"><span style="color:#6A737D"># preflight.sh &lt;worktree&gt; (illustrative): prints CLEAR or BLOCK with a reason; exits 1 on BLOCK.</span></span>
<span class="line"><span style="color:#E1E4E8">wt</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">$1</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#B392F0">block</span><span style="color:#E1E4E8">() { </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "BLOCK </span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">: </span><span style="color:#79B8FF">$1</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">; </span><span style="color:#79B8FF">exit</span><span style="color:#79B8FF"> 1</span><span style="color:#E1E4E8">; }</span></span>
<span class="line"><span style="color:#B392F0">pgrep</span><span style="color:#79B8FF"> -f</span><span style="color:#79B8FF"> --</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF">/dev/null</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#B392F0">block</span><span style="color:#9ECBFF"> "a process is still running in it"</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> rev-parse</span><span style="color:#79B8FF"> --abbrev-ref</span><span style="color:#9ECBFF"> '@{u}'</span><span style="color:#F97583"> &gt;</span><span style="color:#9ECBFF">/dev/null</span><span style="color:#F97583"> 2&gt;&amp;1</span><span style="color:#F97583"> ||</span><span style="color:#B392F0"> block</span><span style="color:#9ECBFF"> "no upstream, or detached HEAD"</span></span>
<span class="line"><span style="color:#E1E4E8">[ </span><span style="color:#F97583">-z</span><span style="color:#9ECBFF"> "$(</span><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">" status </span><span style="color:#79B8FF">--porcelain</span><span style="color:#9ECBFF">)"</span><span style="color:#E1E4E8"> ] </span><span style="color:#F97583">||</span><span style="color:#B392F0"> block</span><span style="color:#9ECBFF"> "uncommitted changes or untracked files"</span></span>
<span class="line"><span style="color:#E1E4E8">[ </span><span style="color:#9ECBFF">"$(</span><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">" rev-list </span><span style="color:#79B8FF">--count</span><span style="color:#9ECBFF"> '@{u}..HEAD')"</span><span style="color:#F97583"> -eq</span><span style="color:#79B8FF"> 0</span><span style="color:#E1E4E8"> ] </span><span style="color:#F97583">||</span><span style="color:#B392F0"> block</span><span style="color:#9ECBFF"> "unpushed commits"</span></span>
<span class="line"><span style="color:#E1E4E8">branch</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"$(</span><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">" branch </span><span style="color:#79B8FF">--show-current</span><span style="color:#9ECBFF">)"</span></span>
<span class="line"><span style="color:#E1E4E8">open</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"$(</span><span style="color:#B392F0">gh</span><span style="color:#9ECBFF"> pr list </span><span style="color:#79B8FF">--head</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$branch</span><span style="color:#9ECBFF">" </span><span style="color:#79B8FF">--state</span><span style="color:#9ECBFF"> open </span><span style="color:#79B8FF">--json</span><span style="color:#9ECBFF"> number </span><span style="color:#79B8FF">--jq</span><span style="color:#9ECBFF"> 'length')"</span></span>
<span class="line"><span style="color:#E1E4E8">[ </span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">$open</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> -eq</span><span style="color:#79B8FF"> 0</span><span style="color:#E1E4E8"> ] </span><span style="color:#F97583">||</span><span style="color:#B392F0"> block</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$open</span><span style="color:#9ECBFF"> open PR(s) on </span><span style="color:#E1E4E8">$branch</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#B392F0">grep</span><span style="color:#79B8FF"> -qxF</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> pending-approvals.txt</span><span style="color:#F97583"> 2&gt;</span><span style="color:#9ECBFF">/dev/null</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#B392F0">block</span><span style="color:#9ECBFF"> "approval pending"</span></span>
<span class="line"><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "CLEAR </span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span></span></code></pre>
<p>Codex’s managed worktrees are detached-HEAD checkouts, so the upstream check blocks them by design; leave those to Codex’s confirmed delete and the app’s snapshot. Pending approvals live in the harness, not in git, so <code>pending-approvals.txt</code> stands for whatever list you can pull from each session view. Child sessions’ worktrees go through the same script.</p>
<p>The removal rule on top is deliberately stricter than VS Code’s: remove a worktree only after the worker is stopped, the branch is pushed and the tree is clean, with git-ignored logs and transcripts exported first (step 4). Never <code>git worktree remove --force</code>. A checkout that fails the preflight becomes a question for a person, not a flag to add. Who owns each checkout, and how to lock the ones your runner created, belongs to <a href="/intel/worktree-ownership-one-creator/">One Checkout, One Owner</a>.</p>
<p>Be honest about what this gate is. Nothing in VS Code, Devin or Codex calls it; if it crashes or never runs, the vendor’s own guard is all that stands between the timer and the worktree. The wall behind both is the step 4 export, which works whether or not anyone read the preflight’s output.</p>
<h2 id="step-4-export-what-git-would-delete-without-asking">Step 4: Export what git would delete without asking</h2>
<p>Git decides clean by tracked changes and untracked files. Ignored files don’t count, so <code>git worktree remove</code> on a clean worktree deletes them without a prompt. That’s git’s behaviour, not a vendor statement, and VS Code’s docs don’t say what happens to ignored files when it removes a worktree. Ignored is exactly where agent runs put their logs, scratch output and sometimes transcripts.</p>
<p>Before any trigger can fire on a worktree, copy its ignored files out, minus secrets:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D">#!/usr/bin/env bash</span></span>
<span class="line"><span style="color:#6A737D"># export-ignored.sh &lt;worktree&gt; &lt;archive-root&gt; (illustrative): copy ignored files out, minus secrets.</span></span>
<span class="line"><span style="color:#E1E4E8">wt</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">$1</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">; dest</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">$2</span><span style="color:#9ECBFF">/$(</span><span style="color:#B392F0">basename</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">")-$(</span><span style="color:#B392F0">date</span><span style="color:#79B8FF"> -u</span><span style="color:#9ECBFF"> +%Y%m%dT%H%M%SZ)"</span></span>
<span class="line"><span style="color:#B392F0">mkdir</span><span style="color:#79B8FF"> -p</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$dest</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> ls-files</span><span style="color:#79B8FF"> --others</span><span style="color:#79B8FF"> --ignored</span><span style="color:#79B8FF"> --exclude-standard</span><span style="color:#79B8FF"> -z</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#F97583">  |</span><span style="color:#B392F0"> grep</span><span style="color:#79B8FF"> -zvE</span><span style="color:#9ECBFF"> '(^|/)(\.env[^/]*|node_modules/.*|.*\.pem)$'</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#F97583">  |</span><span style="color:#E1E4E8"> (</span><span style="color:#79B8FF">cd</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#B392F0">xargs</span><span style="color:#79B8FF"> -0</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> cp</span><span style="color:#79B8FF"> --parents</span><span style="color:#79B8FF"> -t</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$dest</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#E1E4E8">(</span><span style="color:#79B8FF">cd</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$dest</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#B392F0">find</span><span style="color:#9ECBFF"> .</span><span style="color:#79B8FF"> -type</span><span style="color:#9ECBFF"> f</span><span style="color:#79B8FF"> -print0</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> xargs</span><span style="color:#79B8FF"> -0</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> sha256sum</span><span style="color:#E1E4E8">) </span><span style="color:#F97583">&gt;</span><span style="color:#B392F0"> "</span><span style="color:#E1E4E8">$dest</span><span style="color:#B392F0">.sha256"</span></span>
<span class="line"><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "exported $(</span><span style="color:#B392F0">wc</span><span style="color:#79B8FF"> -l</span><span style="color:#F97583"> &lt;</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$dest</span><span style="color:#9ECBFF">.sha256") files from </span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">"</span></span></code></pre>
<p>The exclusions are deliberate: an archive full of credentials is a second incident, so scan what you keep. And mind VS Code’s first stage. Per its docs, worktree removal applies when cleanup marks a session done, not only when it deletes one, so the export runs before you enable mark-as-done.</p>
<p>Session records need the same treatment. Export timing and retention windows for vendor-hosted sessions are covered in <a href="/intel/vendor-hosted-agent-evidence/">Evidence When the Harness Is Vendor-Hosted</a>, and what those records can establish after an incident in <a href="/intel/fleet-replay/">You Can’t Replay What You Can’t See</a>.</p>
<p>For sessions whose transcripts live on local disk, a local library is one way to keep a searchable copy. Automater Lite is a free desktop tray companion for Windows, macOS and Linux. Its homepage says it reads “each tool’s own transcript files and folds them into one library on your machine”, and lists 45 session formats. It does not preserve vendor-hosted cloud sessions: a Devin session, or any cloud session that never wrote a local transcript, is not in it.</p>
<p>Whatever keeps your record, prove it in step 6 by deleting a fixture session and searching for its marker.</p>
<h2 id="step-5-dry-run-vs-code-agent-session-cleanup-before-you-enable-it">Step 5: Dry-run VS Code agent session cleanup before you enable it</h2>
<p>None of the three products documents a dry run, so build one from the preflight. It changes nothing and writes one dated line per worktree:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D">#!/usr/bin/env bash</span></span>
<span class="line"><span style="color:#6A737D"># dry-run.sh (illustrative): what would a cleanup pass remove today? Changes nothing.</span></span>
<span class="line"><span style="color:#B392F0">git</span><span style="color:#9ECBFF"> worktree</span><span style="color:#9ECBFF"> list</span><span style="color:#79B8FF"> --porcelain</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> sed</span><span style="color:#79B8FF"> -n</span><span style="color:#9ECBFF"> 's/^worktree //p'</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> tail</span><span style="color:#79B8FF"> -n</span><span style="color:#9ECBFF"> +2</span><span style="color:#F97583"> |</span><span style="color:#F97583"> while</span><span style="color:#79B8FF"> read</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> wt</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#79B8FF">  printf</span><span style="color:#9ECBFF"> '%s %s\n'</span><span style="color:#9ECBFF"> "$(</span><span style="color:#B392F0">date</span><span style="color:#79B8FF"> -u</span><span style="color:#9ECBFF"> +%FT%TZ)"</span><span style="color:#9ECBFF"> "$(</span><span style="color:#B392F0">./preflight.sh</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$wt</span><span style="color:#9ECBFF">")"</span></span>
<span class="line"><span style="color:#F97583">done</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> tee</span><span style="color:#79B8FF"> -a</span><span style="color:#9ECBFF"> cleanup-dry-run.log</span></span></code></pre>
<p>Run it daily for a week with nothing enabled. Each BLOCK line names a checkout your rule protects, and why. Read the CLEAR lines harder: each is a checkout a sweep could take today, ignored files included.</p>
<p>For VS Code, add the session side: record each Agent Host session’s PR states and last-modified date, and apply the eligibility rule by hand. Include a fixture with one merged PR and one closed without merging to settle the news-section inference.</p>
<p>When the dry run matches what you expected, stage the real thing on one machine: mark-as-done on, delete at 0. Restoring an automatically completed session clears its deletion eligibility, so stage one is recoverable at the session level. It still removes clean worktrees, which is why step 4 came first.</p>
<h2 id="step-6-run-the-restore-test-on-a-fixture-not-on-the-incident">Step 6: Run the restore test on a fixture, not on the incident</h2>
<p>A restore path you haven’t run is a sentence in someone else’s docs. Build a fixture per product: a disposable repo, one session driven to a merged PR, and three markers.</p>
<p>RS-PUSHED is committed and pushed. RS-IGNORED sits in an ignored <code>logs/</code> directory. RS-TRANSCRIPT is typed into the session itself. Let cleanup take the fixture, then try to get each marker back.</p>
<table>
<thead>
<tr>
<th>Trigger</th>
<th>What the docs promise</th>
<th>Test on the fixture</th>
<th>Pass means</th>
</tr>
</thead>
<tbody>
<tr>
<td>VS Code auto mark done</td>
<td>Restore clears deletion eligibility</td>
<td>Let it mark the fixture, restore it, wait past the delete window</td>
<td>Session back, and still there after the grace period</td>
</tr>
<tr>
<td>VS Code auto delete</td>
<td>Permanent deletion</td>
<td>Let it delete; search for RS-TRANSCRIPT everywhere</td>
<td>Found only in your export or local record</td>
</tr>
<tr>
<td>VS Code manual delete</td>
<td>Irreversible; worktrees go too</td>
<td>Don’t test an undo; test that step 4 ran first</td>
<td>Export manifest lists RS-IGNORED</td>
</tr>
<tr>
<td>Codex app retention</td>
<td>Snapshot before delete; restore offered from the chat</td>
<td>Archive the fixture chat, reopen it, restore</td>
<td>RS-PUSHED and RS-IGNORED back, or noted missing</td>
</tr>
<tr>
<td>Codex 0.155 overview delete</td>
<td>Not stated</td>
<td>Delete the fixture task; look for task and worktree</td>
<td>Exactly what’s gone, written down</td>
</tr>
<tr>
<td>Devin archive</td>
<td>No restore documented</td>
<td>Archive a parent with one child PR left unselected</td>
<td>The unselected child PR is still open</td>
</tr>
<tr>
<td>Claude Code sweep</td>
<td>Keeps changed, untracked or unpushed work</td>
<td>Age a clean background-session fixture past <code>cleanupPeriodDays</code></td>
<td>RS-IGNORED recoverable from your export</td>
</tr>
</tbody>
</table>
<p>Record the result per marker. Expect it mixed: the pushed commit survives on the remote, the session may or may not come back, and the ignored log exists only in your step 4 export. Written down, that’s acceptable. Discovered the week an incident needs the log, it isn’t.</p>
<h2 id="step-7-pick-the-numbers-then-re-audit-on-every-upgrade">Step 7: Pick the numbers, then re-audit on every upgrade</h2>
<p>Now enable, with values you chose and wrote down.</p>
<p>
<em>Illustrative. Setting semantics from the VS Code 1.138 docs; the 15-day values are the docs’ recommendation, and the session is modeled.</em></p>
<p>VS Code recommends 15 days for each setting. With both at 15, a session idle past day 15 with a merged PR and none open is marked done at the next hourly check, its worktree goes if clean, and it’s deleted 15 days later unless someone restores it. That second window is your recovery budget, so size it to how long a missing log takes to be missed. Weekly incident reviews fit inside 15; a lane that runs monthly jobs doesn’t.</p>
<p>Write the values into a retention record beside the lane manifest:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># retention.yaml (illustrative; field names are ours, vendor setting IDs appear under settings)</span></span>
<span class="line"><span style="color:#85E89D">reviewed</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2026-09-22</span></span>
<span class="line"><span style="color:#85E89D">settings</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  vscode_1_138</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    chat.agentSessions.autoMarkAsDoneMergedSessionsAfterDays</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">15</span></span>
<span class="line"><span style="color:#85E89D">    chat.agentSessions.autoDeleteArchivedMergedSessionsAfterDays</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">0</span><span style="color:#6A737D">   # stays off until step 6 passes</span></span>
<span class="line"><span style="color:#85E89D">  cursor</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    cursor.worktreeCleanupIntervalHours</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">6</span><span style="color:#6A737D">    # docs default</span></span>
<span class="line"><span style="color:#85E89D">    cursor.worktreeMaxCount</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">25</span><span style="color:#6A737D">               # per machine, across all workspaces</span></span>
<span class="line"><span style="color:#85E89D">  claude_code</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    cleanupPeriodDays</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;your value&gt;"</span><span style="color:#6A737D">         # age limit for its background worktree sweep</span></span>
<span class="line"><span style="color:#85E89D">policy</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  devin_archive_child_prs</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">none-selected-by-default</span></span>
<span class="line"><span style="color:#85E89D">  codex_overview_delete</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">human-only-after-preflight</span></span>
<span class="line"><span style="color:#85E89D">  runner_roots_outside_vendor_roots</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">true</span></span>
<span class="line"><span style="color:#85E89D">  preflight</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">preflight.sh</span></span>
<span class="line"><span style="color:#85E89D">  export_before_remove</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">export-ignored.sh</span></span>
<span class="line"><span style="color:#85E89D">  restore_test_last_passed</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2026-09-22</span></span></code></pre>
<p>Then re-audit on change. VS Code’s docs say the feature “is in preview and might change before it becomes generally available”, and Codex reshaped deletion in a routine release. Add the step 6 fixture to the canned sessions in <a href="/intel/cli-upgrade-canary-drift/">Canary Every CLI Upgrade</a>, and rerun the restore test whenever a harness updates. Keep any session whose work was continued elsewhere until that work merges; <a href="/intel/session-import-fidelity-test/">Test What Survives a Cross-CLI Session Import</a> relies on the source staying put.</p>
<h2 id="five-ways-cleanup-takes-more-than-you-meant-and-the-signal-for-each">Five ways cleanup takes more than you meant, and the signal for each</h2>
<p><strong>The clean worktree with the only log.</strong> The tree passed every guard because the log was ignored. Signal: an incident review asks for a run log whose worktree no longer exists. Response: the step 4 export runs on a schedule, before any trigger can fire.</p>
<p><strong>The cascade nobody read.</strong> A Devin archive closes a child PR someone was reviewing. Signal: a PR closed within a minute of a parent archive, with no comment from its reviewer. Response: none selected by default, and the person archiving reads the list.</p>
<p><strong>The closed PR that didn’t count.</strong> A session with one merged PR and one abandoned one is marked done while the abandoned work still matters. Signal: the step 5 fixture shows up as eligible. Response: close abandoned PRs deliberately, or keep that lane’s threshold at 0.</p>
<p><strong>The machine that was off.</strong> VS Code checks at startup, so a laptop back from a week away runs one pass over everything that aged past the threshold meanwhile. That’s an inference from the schedule, not a documented case. Signal: a burst of done-marks at one boot timestamp. Response: read the dry-run log after any long gap.</p>
<p><strong>The foreign checkout in a vendor’s root.</strong> Cursor’s sweep treats worktrees in its root as eligible whoever made them. Signal: a runner checkout missing after a sweep. Response: runner roots live outside every vendor root, proved by the ownership piece’s decoy test.</p>
<h2 id="retention-is-a-fleet-policy-not-a-settings-page-per-app">Retention is a fleet policy, not a settings page per app</h2>
<p>Each product’s cleanup is sensible inside its own window, but a fleet has several: VS Code’s Agents window, Devin’s sidebar, the Codex overview, Claude Code’s sweep and your runner’s teardown, each with its own clock and idea of finished. The inventory, preflight, export and retention record live above them, in the layer that runs the fleet, where a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> keeps every CLI’s sessions side by side.</p>
<p>Automater works at that desk level, and it enforces none of this; your preflight does. Lite keeps the local, searchable record of sessions whose transcripts exist on the machine, and Pro adds managed sessions you can start, steer, interrupt and stop. Automater Lite is free on automater.ai; Pro is $50/year.</p>
<p>Turn cleanup on. Just let it take only what the dry run said it would.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-vs-code-delete-agent-sessions-automatically">Does VS Code delete agent sessions automatically?</h3>
<p>Only if you enable it. VS Code 1.138 adds two preview settings: one marks inactive Agent Host sessions done, the other deletes them after a separate grace period. Both ship disabled. Deletion applies only to sessions VS Code marked done itself, so sessions you mark done by hand are never deleted automatically.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://code.visualstudio.com/updates/v1_138">Visual Studio Code 1.138 release notes</a> — Sep 16, 2026; both cleanup settings off by default</li>
<li><a href="https://code.visualstudio.com/docs/agents/run/sessions/manage-sessions">VS Code: Manage agent sessions</a> — eligibility, recommended 15, worktree conditions</li>
<li><a href="https://github.com/microsoft/vscode/pull/335851">microsoft/vscode PR #335851</a> — Sep 11, 2026; manual versus automatic worktree cleanup</li>
<li><a href="https://docs.devin.ai/release-notes/2026">Devin release notes 2026</a> — Sep 9, 2026; archive closes child PRs</li>
<li><a href="https://github.com/openai/codex/releases/tag/rust-v0.155.0">Codex 0.155.0 release notes</a> — Sep 17, 2026; delete in the agents overview</li>
<li><a href="https://learn.chatgpt.com/docs/environments/git-worktrees">ChatGPT Learn: Worktrees</a> — Codex app retention and snapshots</li>
<li><a href="https://github.com/openai/codex/pull/42652">openai/codex PR #42652</a> — CLI allocations exempt from auto-cleanup</li>
<li><a href="https://code.claude.com/docs/en/worktrees">Claude Code: Run parallel sessions with worktrees</a> — retention sweep and the 2.1.246 fix</li>
<li><a href="https://cursor.com/docs/configuration/worktrees">Cursor: Worktrees</a> — 6-hour cleanup, 25 per machine</li>
</ul>
]]></content:encoded></item><item><title>Managed Permissions Users Can’t Override — Test It Across Harnesses</title><link>https://automater.ai/intel/non-overridable-agent-permissions/</link><guid>https://automater.ai/intel/non-overridable-agent-permissions/</guid><description>Copilot enterprise managed permissions claim users can&apos;t override them. Prove it: one deny, ten bypass rows, run in Copilot, Claude Code, Codex and JetBrains.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>On Sep 9, GitHub shipped a sentence you can test. Copilot enterprise managed permissions, its changelog says, can’t be weakened by user or workspace settings, auto-approval, or previously saved approvals. Most vendors describe a precedence order and leave it there. This one named the bypass paths, and a named bypass path is a test case.</p>
<p>Your fleet doesn’t run one harness, though. The same deny list has to hold in Copilot, in Claude Code, in whatever Codex lanes you keep, and on the JetBrains seats where the managed sandbox is still a preview. Each delivers policy its own way, and each ships convenience features built to cut prompts. On a chatbot, a permission prompt guards what it says. On an agent, it guards what runs on the machine, so every feature that removes a prompt is a candidate override.</p>
<p>By Tuesday you’ll have one override test: a managed deny and a managed ask on harmless canary operations, a receipt check that proves each lane received the policy, ten bypass rows run on every harness you operate, and a log that records harness, version, expected and observed for every row. Rows that disagree get fixed or flagged, in writing, with an owner.</p>
<h2 id="sep-9-copilot-enterprise-managed-permissions-go-ga-with-the-bypass-paths-named">Sep 9: Copilot enterprise managed permissions go GA with the bypass paths named</h2>
<p>GitHub made <a href="https://github.blog/changelog/2026-09-09-enterprise-managed-permissions-for-github-copilot-agent-operations/">enterprise managed permissions for Copilot agent operations</a> generally available on Sep 9, 2026, for Copilot Business and Enterprise. Admins mark shell commands, file reads and edits, and network domains as blocked, needing human approval, or allowed without a prompt, and can give different enterprise teams their own policies. The GA surfaces are the Copilot app, Copilot CLI and VS Code sessions that use Agent Host. The line to hold GitHub to: “Managed restrictions can’t be weakened by user or workspace settings, auto-approval, or previously saved approvals.”</p>
<p>The <a href="https://docs.github.com/en/enterprise-cloud@latest/copilot/reference/enterprise-administrators/enterprise-managed-settings">enterprise managed settings reference</a> goes further than the changelog. Rules resolve deny &gt; ask &gt; allow. “A deny rule set by any managed settings source blocks the operation for all users regardless of rules in the other sources.” Allowlists intersect across sources instead of adding up. Once any managed source defines a rule or an allowlist, an unmatched supported operation defaults to requiring approval. And the ask rule carries a longer bypass list than the changelog: “A managed ask rule can’t be satisfied by bypass mode (also known as allow-all or YOLO mode), an auto-approval setting, a hook or other approval shortcut, or a grant persisted from an earlier approval.”</p>
<p>
<em>Screenshot: GitHub Docs, “Enterprise managed settings - GitHub Enterprise Cloud Docs” (undated), captured Sep 21, 2026.</em></p>
<p>The same page carries a client support table, and it decides what your test can even measure. <code>permissions.deny</code>, <code>ask</code> and <code>allow</code> are marked Not supported in Copilot cloud agent and in JetBrains IDEs. In VS Code, “these granular permission rules apply to Copilot sessions that use Agent Host.” Step 3 turns that table into your n/a cells.</p>
<p>Two neighbours landed the same fortnight. On Sep 8, GitHub put an <a href="https://github.blog/changelog/2026-09-08-enterprise-managed-sandbox-in-copilot-for-jetbrains/">enterprise-managed sandbox in Copilot for JetBrains</a> into public preview: managed control over sandbox enablement, filesystem and network access, proxy settings, developer-tool access and macOS Keychain access, locked controls in the IDE, and enterprise policy diagnostics to confirm policies are detected and enforced on the device. On Sep 17, <a href="https://code.claude.com/docs/en/changelog">Claude Code 2.1.274</a> added a <code>claude_code.managed_settings_resolved</code> OpenTelemetry event that reports managed-settings sources and policy helper state, with redacted settings and digests when <code>OTEL_LOG_MANAGED_SETTINGS=1</code> is set.</p>
<h2 id="where-copilot-enterprise-managed-permissions-and-claude-code-part-ways">Where Copilot enterprise managed permissions and Claude Code part ways</h2>
<p>One-harness testing fails at the second policy source, because Copilot and Claude Code disagree about what two managed sources mean.</p>
<p>Copilot merges. Any source’s deny wins, and allowlists intersect. Claude Code’s managed tier is first-wins by default: per its <a href="https://code.claude.com/docs/en/managed-settings">managed-settings docs</a>, it uses the highest-ranked source that delivers at least one policy key and ignores the rest rather than merging them. “Claude Code shows no warning for the sources it skips; /status names the source it used and the ones it skipped.” A merge mode exists, opt-in from v2.1.242.</p>
<p>So on Claude Code, a deny delivered through a lower-ranked managed source, while a higher-ranked source delivers even one policy key, is absent on that machine. Nothing errors. The lane runs as if you never wrote the rule, and the same two-source setup on Copilot would have held.</p>
<p>Precedence inside Claude Code, and one house dialect across Claude Code and Codex, are already covered in <a href="/intel/restricted-mode-fleet-policy/">Claude Code permission modes as fleet policy</a> and <a href="/intel/unified-permission-modes/">one permission dialect for Claude Code and Codex</a>. This piece assumes that policy exists. It tests whether each harness keeps it.</p>
<h2 id="the-cross-harness-override-test-step-by-step">The cross-harness override test, step by step</h2>
<h3 id="step-1-write-one-deny-and-one-ask-you-can-trigger-without-harm">Step 1: Write one deny and one ask you can trigger without harm</h3>
<p>Pick three canary operations, one for each class GitHub’s policy covers: a shell command, a file edit, a network domain. Each must be harmless if the wall fails and unmistakable in a log. A shell read of a fake credentials file in a scratch repo, an edit to a file called <code>ask-me.txt</code>, and a fetch to a domain you own and never use all qualify.</p>
<p>Make the shell and network canaries managed denies and the edit a managed ask. Write each rule in that harness’s own managed-settings syntax, from its own docs. Rule selectors differ between vendors, and a rule copied across can parse cleanly and match nothing; that’s the renamed-tool failure in miniature, and <a href="/intel/tool-schema-guard-contract-tests/">contract-testing guard coverage</a> owns it.</p>
<p>Then write the test down as a manifest your runner reads. The shape below is illustrative, your schema rather than any vendor’s:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># override-test.yaml (illustrative; your runner's schema, not a vendor format)</span></span>
<span class="line"><span style="color:#85E89D">policy_version</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">perm-v3</span></span>
<span class="line"><span style="color:#85E89D">canaries</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  shell_deny</span><span style="color:#E1E4E8">:   </span><span style="color:#9ECBFF">read fixtures/fake-credentials.txt from the shell</span></span>
<span class="line"><span style="color:#85E89D">  network_deny</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">fetch https://canary.example.com/ping</span></span>
<span class="line"><span style="color:#85E89D">  edit_ask</span><span style="color:#E1E4E8">:     </span><span style="color:#9ECBFF">edit scratch/ask-me.txt</span></span>
<span class="line"><span style="color:#85E89D">lanes</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#85E89D">id</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">ci-copilot-cli</span></span>
<span class="line"><span style="color:#85E89D">    harness</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">copilot-cli</span></span>
<span class="line"><span style="color:#85E89D">    receipt</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">unmatched-op-prompts</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#85E89D">id</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">laptop-claude-02</span></span>
<span class="line"><span style="color:#85E89D">    harness</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">claude-code</span></span>
<span class="line"><span style="color:#85E89D">    receipt</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">skipped-sources-empty</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">managed-settings-resolved-event</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#85E89D">id</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">ci-codex-01</span></span>
<span class="line"><span style="color:#85E89D">    harness</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">codex</span></span>
<span class="line"><span style="color:#85E89D">    receipt</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">fallback-notice</span><span style="color:#6A737D">     # requirements.toml conflict notice</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#85E89D">id</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">seat-jetbrains-07</span></span>
<span class="line"><span style="color:#85E89D">    harness</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">jetbrains-copilot</span></span>
<span class="line"><span style="color:#85E89D">    receipt</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">policy-diagnostics</span></span>
<span class="line"><span style="color:#85E89D">    scope</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">sandbox-only</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#85E89D">id</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">control-01</span></span>
<span class="line"><span style="color:#85E89D">    harness</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">claude-code</span></span>
<span class="line"><span style="color:#85E89D">    managed_policy</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">none</span><span style="color:#6A737D">         # every canary must succeed here</span></span>
<span class="line"><span style="color:#85E89D">rows</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">user_settings</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">workspace_settings</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">auto_approval</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">saved_approval</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#9ECBFF">       bypass_mode</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">hook_allow</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">second_managed_source</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#9ECBFF">       resumed_session</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">child_agent</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">headless_run</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">record</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">harness</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">version</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">row</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">canary</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">expected</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">observed</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">receipt</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">action</span><span style="color:#E1E4E8">]</span></span></code></pre>
<p>The control lane matters more than it looks. Run every canary on a machine with no managed policy first. If a canary doesn’t fire there, a blocked result elsewhere proves nothing.</p>
<p>Version the policy and stamp that version on every row. When a row flips next month, the first question is whether the policy changed or the harness did.</p>
<h3 id="step-2-prove-each-lane-received-the-policy-before-you-attack-it">Step 2: Prove each lane received the policy before you attack it</h3>
<p>A bypass that works on a lane that never got the policy is a delivery bug, and it looks exactly like a real override. Get a receipt per lane before you run a single bypass row.</p>
<table>
<thead>
<tr>
<th>Harness</th>
<th>Receipt</th>
<th>What it proves</th>
<th>What it doesn’t</th>
</tr>
</thead>
<tbody>
<tr>
<td>Copilot (CLI, app, VS Code Agent Host)</td>
<td>None named in the GA post; trigger an operation your rules don’t mention and expect an approval prompt</td>
<td>The managed policy is loaded on this surface</td>
<td>Which source each rule came from</td>
</tr>
<tr>
<td>JetBrains (preview)</td>
<td>Enterprise policy diagnostics; controls locked in the IDE</td>
<td>Sandbox policies are detected and enforced on the device</td>
<td>Anything about deny, ask or allow, which the docs table marks Not supported there</td>
</tr>
<tr>
<td>Claude Code</td>
<td><code>/status</code> Setting sources and Skipped sources (v2.1.242+); <code>claude doctor</code>; the <code>managed_settings_resolved</code> event (2.1.274+)</td>
<td>Which managed source won and which were dropped</td>
<td>That a deny fired; the event reports sources, not enforcement</td>
</tr>
<tr>
<td>Codex</td>
<td>Admin-enforced <code>requirements.toml</code> (system file, cloud config bundle or macOS MDM); set a disallowed value locally and expect the fallback notice</td>
<td>The requirements reached this client</td>
<td>Which layer delivered them, or that a command rule fired</td>
</tr>
</tbody>
</table>
<p>Codex’s <a href="https://learn.chatgpt.com/docs/enterprise/managed-configuration">managed configuration docs</a> call requirements “admin-enforced constraints that users can’t override”: a local value that conflicts with one falls back to a compatible value and the client notifies the user. Layers don’t simply merge, though. Higher-precedence layers override scalar and list values, while rules and hooks compose field by field, so run the second-source row on Codex too instead of borrowing either vendor’s answer.</p>
<p>The Copilot receipt is derived from GitHub’s own default. When any managed source defines a permission rule, an unmatched supported operation should require approval. If an edit your rules never mention goes through without a prompt, that lane isn’t running your policy, and every bypass row on it will lie.</p>
<p>On Claude Code, read <code>/status</code> before anything else. If Skipped sources isn’t empty, stop: that’s the first-wins shadowing from the section above, and every row on that machine would measure a policy you didn’t intend. <code>claude doctor</code> lists what it dropped. Then collect the <code>managed_settings_resolved</code> event with <code>OTEL_LOG_MANAGED_SETTINGS=1</code>, and keep the digest it logs beside your policy version. A lane whose digest differs from your reference machine’s didn’t get the policy you think it got.</p>
<p>
<em>Screenshot: Claude Code Docs, “Claude Code changelog - Claude Code Docs” (2.1.274, Sep 17, 2026), captured Sep 21, 2026.</em></p>
<p>The event lives in the changelog and not yet in the monitoring docs, so pin its field names from an event you captured, not from memory.</p>
<h3 id="step-3-mark-the-surfaces-that-cant-enforce-the-rule-before-you-test-them">Step 3: Mark the surfaces that can’t enforce the rule before you test them</h3>
<p>Some cells are n/a before you start, and a test that reports them as passes is worse than no test.</p>
<p>
<em>Which managed keys each Copilot surface honours, per GitHub’s docs table on Sep 21. The amber cell is a docs conflict to verify before you rely on it.</em></p>
<p>Copilot cloud agent takes no managed deny, ask or allow, and no <code>disableBypassPermissionsMode</code> either. JetBrains IDEs take no managed deny, ask or allow; their managed control is the sandbox preview, and even that is unsettled, because GitHub’s table marks the <code>sandbox</code> key Not supported for JetBrains while the Sep 8 changelog describes a managed setting that turns the sandbox on or off. VS Code honours the granular rules only in Agent Host sessions, though <code>disableBypassPermissionsMode</code> has broader VS Code support.</p>
<p>Write those cells as n/a, never pass. A lane on an n/a surface isn’t covered by the managed rule at all. It either moves to a surface that enforces the rule, or it gets a wall that doesn’t depend on the harness (step 6).</p>
<h3 id="step-4-run-the-six-bypass-rows-github-names">Step 4: Run the six bypass rows GitHub names</h3>
<p>Now attack. For each row, set up the bypass, fire all three canaries, and write down what happened. The expected column is what each vendor’s docs predict; where the docs are silent, it is your policy intent, marked Intent.</p>
<table>
<thead>
<tr>
<th>Row</th>
<th>How to attempt it</th>
<th>Copilot (CLI, app, VS Code Agent Host)</th>
<th>Claude Code</th>
<th>Codex</th>
<th>JetBrains</th>
</tr>
</thead>
<tbody>
<tr>
<td>User settings</td>
<td>Allow the canaries in the user’s own settings</td>
<td>Deny holds</td>
<td>Deny holds; lower levels can only tighten</td>
<td>Intent: deny holds</td>
<td>n/a</td>
</tr>
<tr>
<td>Workspace settings</td>
<td>Allow them in a committed project config (<code>.claude/settings.json</code> on Claude Code)</td>
<td>Deny holds</td>
<td>Deny holds</td>
<td>Intent: deny holds</td>
<td>n/a</td>
</tr>
<tr>
<td>Auto-approval</td>
<td>Turn on every auto-approve option the surface offers</td>
<td>Deny holds; ask still prompts</td>
<td>Intent: deny holds in every mode you permit</td>
<td>Intent: deny holds</td>
<td>n/a</td>
</tr>
<tr>
<td>Saved approval</td>
<td>Approve the canary persistently before the policy lands; retry after</td>
<td>Deny holds; ask prompts again</td>
<td>Intent: deny holds; watch for <code>user_permanent</code></td>
<td>Intent: deny holds</td>
<td>n/a</td>
</tr>
<tr>
<td>Bypass (YOLO) mode</td>
<td>Launch in the surface’s allow-all mode</td>
<td>Ask still prompts; intent: deny holds</td>
<td>Intent: deny holds</td>
<td>Intent: deny holds</td>
<td>n/a for deny and ask</td>
</tr>
<tr>
<td>Hook returns allow</td>
<td>A pre-tool hook that approves the canary</td>
<td>Ask still prompts; intent: deny holds</td>
<td>Deny still blocks; ask still prompts</td>
<td>Intent: deny holds</td>
<td>n/a</td>
</tr>
</tbody>
</table>
<p>Two cells need their sources. Claude Code’s <a href="https://code.claude.com/docs/en/settings">settings reference</a> puts managed settings above every other level, and its exceptions table lists only cases where a stricter value from a lower level still counts, so no lower level can weaken a managed rule. And Claude Code’s <a href="https://code.claude.com/docs/en/permissions">permissions docs</a> say hook decisions don’t bypass permission rules: a matching deny still blocks and a matching ask still prompts, whatever the hook returned.</p>
<p>Create the saved approvals before the policy lands. That’s how they usually exist: somebody chose a remember-this option weeks ago, and the approval outlived the reason for it.</p>
<p>On Claude Code, the OpenTelemetry <code>tool_decision</code> event shows the outcome of each attempt. Per the <a href="https://code.claude.com/docs/en/monitoring-usage">monitoring docs</a>, <code>decision</code> is accept or reject, and <code>source</code> is one of <code>config</code>, <code>hook</code>, <code>user_permanent</code>, <code>user_temporary</code>, <code>user_abort</code> or <code>user_reject</code>. A canary logged as accept with <code>user_permanent</code> is a saved approval beating your policy. A reject with <code>config</code> means something held, not what: <code>config</code> covers settings, managed policy, flags and the active permission mode, and “The event doesn’t indicate which of these sources matched.” <a href="/intel/vendor-permission-verdict-triage/">Reading vendor permission verdicts</a> covers that stream in depth; here you only need accept versus reject per canary.</p>
<h3 id="step-5-add-the-four-rows-nobody-named">Step 5: Add the four rows nobody named</h3>
<p>These rows are ours, not GitHub’s. They target the places where policy tends to be read once and then assumed.</p>
<table>
<thead>
<tr>
<th>Row</th>
<th>How to attempt it</th>
<th>Copilot</th>
<th>Claude Code</th>
<th>Codex</th>
<th>JetBrains</th>
</tr>
</thead>
<tbody>
<tr>
<td>Second managed source</td>
<td>Ship a second managed policy that allows the canary</td>
<td>Deny holds: any source’s deny wins, allowlists intersect</td>
<td>First-wins: a lower-ranked source is dropped silently; read Skipped sources</td>
<td>Intent: deny holds</td>
<td>n/a (sandbox: docs don’t say)</td>
</tr>
<tr>
<td>Resumed session</td>
<td>Start before the policy lands, resume after</td>
<td>Intent: deny holds</td>
<td>Intent: deny holds (<code>--resume</code>, <code>--continue</code>)</td>
<td>Intent: deny holds</td>
<td>n/a</td>
</tr>
<tr>
<td>Child agent</td>
<td>The session spawns a subagent that fires the canary</td>
<td>Intent: deny holds</td>
<td>Intent: deny holds</td>
<td>Intent: deny holds</td>
<td>n/a</td>
</tr>
<tr>
<td>Headless run</td>
<td>Non-interactive run, nobody there to answer</td>
<td>Intent: deny holds; ask blocks</td>
<td>Intent: deny holds; ask blocks (<code>claude -p</code>)</td>
<td>Intent: deny holds; ask blocks (<code>codex exec</code>)</td>
<td>n/a</td>
</tr>
</tbody>
</table>
<p>The second-source row is where the harnesses diverge by design. Put the allowing policy in the higher-ranked source, expect Claude Code to fail the row on the default, and log that as a finding. The fix is to keep every managed key in one source, or opt into merge and re-run the row. Project-over-group overrides are the same shape one layer down, at the MCP server, and the <a href="/intel/gitlab-mcp-always-ask-writes/">GitLab always-ask audit</a> covers that one.</p>
<p>The resumed-session row catches state carried from before the change: start the session under the old policy, ship the new one, resume, fire. The child-agent row catches policy that binds the parent process but not what it spawns.</p>
<p>In the headless row, the ask canary is the interesting one. With nobody at the keyboard, a managed ask should stop the run, not resolve itself. If a headless lane edits <code>ask-me.txt</code>, an ask turned into an allow somewhere between the policy and the process. Unattended lanes pick up undeclared state in other ways too; <a href="/intel/headless-lane-reproducibility/">proving headless runs ignore undeclared state</a> is the companion test.</p>
<h3 id="step-6-record-expected-against-observed-per-harness-and-version-then-fix-or-flag">Step 6: Record expected against observed per harness and version, then fix or flag</h3>
<p>
<em>Receipt first, then the rows. A lane with no receipt is a delivery bug, not a passing row.</em></p>
<p>Every attempt becomes one log line. The line below is illustrative, including its result:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span><span style="color:#79B8FF">"run"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"2026-09-22T14:05:00Z"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"policy"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"perm-v3"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"lane"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"laptop-claude-02"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"harness"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"claude-code"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"version"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"2.1.278"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"row"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"second_managed_source"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"canary"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"shell_deny"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"expected"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"dropped_first_wins"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"intent"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"deny_holds"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"observed"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"ran"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"receipt"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"skipped_sources=1"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"action"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"fix: move deny into the winning source; re-run"</span><span style="color:#E1E4E8">}</span></span></code></pre>
<p>Keep expected and intent as separate fields. Expected versus observed tells you whether the vendor behaved as documented; a mismatch there is a bug report and a version pin. Intent versus observed tells you whether you’re protected; a mismatch there is your problem regardless of what the docs say.</p>
<p>Every row where observed misses intent gets exactly one of two outcomes:</p>
<ol>
<li><strong>Fix.</strong> Move the rule into the source that wins, turn on merge, set <code>disableBypassPermissionsMode</code>, or move the lane to a surface that enforces deny and ask. Re-run the row and attach the green result to the change.</li>
<li><strong>Flag.</strong> When the harness can’t enforce the rule on that surface or version, write it down with an owner and a review date, keep the lane off work the rule protects, and put a wall underneath that doesn’t depend on the harness: an OS sandbox, egress filtering at the network, credentials the lane never holds.</li>
</ol>
<p>Then re-run the full matrix on every harness upgrade. Version is a column because a green row on one build says nothing about the next, and the <a href="/intel/cli-upgrade-canary-drift/">CLI upgrade canary</a> is where this test belongs in your change process.</p>
<h2 id="five-ways-the-override-test-hands-you-a-false-green">Five ways the override test hands you a false green</h2>
<p><strong>The receipt that isn’t enforcement.</strong> <code>/status</code> and the OTel event say which source won; neither says a deny fired. Signal: receipt present, canary ran. Score it as a failed row.</p>
<p><strong>Silent shadowing.</strong> Claude Code’s first-wins default drops a lower source without a warning. Signal: Skipped sources is non-empty on any lane. Fix delivery before you test anything else there.</p>
<p><strong>An n/a scored as a pass.</strong> A green deny row on Copilot cloud agent, on a JetBrains seat, or in a VS Code session that isn’t on Agent Host means the canary failed for some other reason, because nothing there reads the rule. Signal: the same canary also fails on the control lane.</p>
<p><strong>A canary that can’t fire.</strong> A missing fixture, a sandboxed network, a typo in the path: the block you logged came from the environment. Signal: the control lane can’t run it either. Fix the canary before trusting a single row.</p>
<p><strong>The gate that isn’t the only door.</strong> Managed permissions are the harness enforcing policy on itself, and when that gate fails it fails quietly: a skipped source or an unsupported surface doesn’t error, it runs the canary. A user who can install an unmanaged binary, add a second agent, or run the command in their own shell is outside the harness entirely. Signal: canary operations in shell history or network logs with no agent session attached. That’s why flagged lanes get a wall, and why the destructive and network classes keep one even when every row is green.</p>
<h2 id="policy-is-a-fleet-property-proof-is-per-lane">Policy is a fleet property; proof is per lane</h2>
<p>A managed deny is written once and enforced several times over, by harnesses that each decide how to read it. The only artifact that tells you whether it held is the table from step 6: per lane, per harness, per version, with a receipt behind every row. No vendor console produces that table, because no vendor console sees the other vendors’ lanes. Keeping it is the operating layer’s job, the same argument a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> makes for inventory and kill switches, applied to policy.</p>
<p>GitHub handed you six bypass rows. Add four, run all ten on everything you operate, and write down what you saw.</p>
<h2 id="faq">FAQ</h2>
<h3 id="can-developers-override-copilot-enterprise-managed-permissions">Can developers override Copilot enterprise managed permissions?</h3>
<p>Not through the paths GitHub names. Its Sep 9 changelog says managed restrictions can’t be weakened by user or workspace settings, auto-approval or saved approvals, and the docs add bypass mode and hooks for a managed ask. The rules apply in Copilot CLI, the Copilot app and VS Code Agent Host sessions.</p>
<h3 id="why-is-my-claude-code-managed-deny-missing-on-one-machine">Why is my Claude Code managed deny missing on one machine?</h3>
<p>Most likely first-wins. Claude Code’s managed tier uses the highest-ranked source that delivers at least one policy key and ignores the rest, without a warning. Run /status and read Skipped sources, or run claude doctor. Keep every managed key in one source, or opt into merge mode from v2.1.242.</p>
<h3 id="do-copilot-managed-permissions-work-in-cloud-agent-and-jetbrains">Do Copilot managed permissions work in cloud agent and JetBrains?</h3>
<p>Not the deny, ask and allow rules. GitHub’s managed-settings support table marks them Not supported in Copilot cloud agent and JetBrains IDEs. JetBrains has a separate managed sandbox in public preview since Sep 8, with policy diagnostics. Mark those test rows n/a and protect the lanes another way.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://github.blog/changelog/2026-09-09-enterprise-managed-permissions-for-github-copilot-agent-operations/">GitHub Changelog: Enterprise managed permissions for GitHub Copilot agent operations</a> — Sep 9, 2026; GA for Business and Enterprise, the named bypass paths</li>
<li><a href="https://docs.github.com/en/enterprise-cloud@latest/copilot/reference/enterprise-administrators/enterprise-managed-settings">GitHub Docs: Enterprise managed settings</a> — deny &gt; ask &gt; allow, the managed-ask bypass list, intersected allowlists, the client support table</li>
<li><a href="https://github.blog/changelog/2026-09-08-enterprise-managed-sandbox-in-copilot-for-jetbrains/">GitHub Changelog: Enterprise-managed sandbox in Copilot for JetBrains</a> — Sep 8, 2026; public preview, enterprise policy diagnostics</li>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a> — 2.1.274 (Sep 17, 2026), the <code>managed_settings_resolved</code> event</li>
<li><a href="https://code.claude.com/docs/en/managed-settings">Claude Code docs: Managed settings</a> — first-wins default, opt-in merge, /status Skipped sources, claude doctor</li>
<li><a href="https://code.claude.com/docs/en/settings">Claude Code docs: Settings</a> — managed tier above every other level; stricter-only exceptions</li>
<li><a href="https://code.claude.com/docs/en/permissions">Claude Code docs: Configure permissions</a> — hook decisions don’t bypass deny or ask rules</li>
<li><a href="https://code.claude.com/docs/en/monitoring-usage">Claude Code docs: Monitoring usage</a> — <code>tool_decision</code> decision and source fields</li>
<li><a href="https://learn.chatgpt.com/docs/enterprise/managed-configuration">Codex docs: Managed configuration</a> — admin-enforced requirements, fallback notice, layer precedence</li>
</ul>
]]></content:encoded></item><item><title>The Rule of Two: Split the Lane That Holds All Three</title><link>https://automater.ai/intel/rule-of-two-lane-split/</link><guid>https://automater.ai/intel/rule-of-two-lane-split/</guid><description>The agents Rule of Two for a real fleet: score each lane on untrusted input, private data and egress, split all-three lanes, and lint configs before they run.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>Count the lanes in your fleet that read text a stranger wrote, hold a credential to something private, and can push, post or fetch in the same session. The Agents Rule of Two says that count should be zero. In most fleets it isn’t, because each of those three grants arrived as one reasonable config line, in a different month, approved by a different person.</p>
<p>This is the audit that gets the count to zero. By Tuesday every lane carries a score on three legs, per session and including the subagents it spawns. Every lane that scores all three has dropped a leg, been split into a reader and an actor, or been put under supervision with a human approving each state change and each outbound call. A pre-run lint refuses to start any lane that grants all three without a supervision flag, and one fixture proves the split holds: a handoff that passes the schema, names an attacker’s destination, and gets refused.</p>
<p>Chatbots suggest; agents act. A chatbot that reads a poisoned page can only say something wrong; a lane that also holds the org token and a push tool can do something wrong, unwatched. The rule decides in config, before the run, which of the three a lane gives up.</p>
<h2 id="sep-14-spains-regulator-logs-a-breach-notification-that-names-an-ai-agent">Sep 14: Spain’s regulator logs a breach notification that names an AI agent</h2>
<p>On Sep 14, 2026, Spain’s data protection authority, the AEPD, published a <a href="https://www.aepd.es/prensa-y-comunicacion/blog/primera-notiviacion-brecha-datos-personales-causada-por-ataque-ejecutado-mediante-agente-ia">blog post by Francisco Pérez Bes</a> saying it had received its first personal-data breach notification in which “el incidente habría sido ejecutado mediante un agente de inteligencia artificial” (the incident would reportedly have been carried out through an AI agent). The conditional <em>habría</em> matters, and the AEPD spells out why: the account comes from the affected organisation’s notification and still has to be analysed, and using a particular model does not mean the model or its provider was compromised. The post gives no receipt date, no sector and no model name.</p>
<p>What it does describe is a sequence. The attacking agent searched generic files for weaknesses and logged in successfully. Then it searched the application for vulnerabilities “de forma autónoma” (autonomously), which let it modify personal data and access invoices. Search, log in, modify, read: each step is a tool call some lane in your fleet can already make.</p>
<p>
<em>Screenshot: AEPD, “Primera notificación de una brecha de datos personales causada por un ataque ejecutado mediante un agente de IA” (Sep 14, 2026), captured Sep 21, 2026.</em></p>
<p>That notification concerns an attacker’s agent, and the post never mentions the rule. The link is the regulator itself: its agentic-AI guidance, <a href="https://www.aepd.es/guias/orientaciones-ia-agentica.pdf">V1.2 dated February 2026</a>, presents a “Regla de 2” on page 41, citing <a href="https://chromium.googlesource.com/chromium/src/+/main/docs/security/rule-of-2.md">Chromium’s browser rule</a> and Meta’s reformulation for agents. Its worked example is an email auto-reply agent that receives mail nobody vetted, can reach sensitive data without restriction and acts automatically; holding all three, the guidance says, is a configuration that should not be allowed.</p>
<p>It calls the rule “una regla general de mínimos enfocada a ciberseguridad” (a general minimum rule focused on cybersecurity), a starting point for analysis, and then adds data-protection checks the rule doesn’t cover, such as minimisation and accuracy. Read it as a regulator’s baseline, not a binding standard.</p>
<p>The rule itself is Meta’s. Its <a href="https://ai.meta.com/blog/practical-ai-agent-security/">Oct 31, 2025 post</a> says an agent should satisfy “no more than two of the following three properties within a session”: [A] processing untrustworthy inputs, [B] access to sensitive systems or private data, and [C] changing state or communicating externally. Meta credits Chromium’s Rule of 2 and Simon Willison’s lethal trifecta, which <a href="/intel/securing-ai-agents/">the AI agent security primer</a> already explains, so this piece won’t.</p>
<p>Two more points matter. If a job needs all three without a fresh session, the agent should not run autonomously and needs supervision at minimum. And the rule supplements least privilege; it doesn’t replace it.</p>
<aside class="social-embed" data-x-embed data-post-id="1984355145917088235">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">October 31, 2025 — Announces the Agents Rule of Two as a framework for reasoning about agent security trade-offs, on the day Meta published it.</p>
  <a class="social-embed__source" href="https://x.com/MickAyzenberg/status/1984355145917088235" target="_blank" rel="noopener noreferrer">View Mick Ayzenberg's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="step-1-score-every-lane-on-the-agents-rule-of-two-per-session">Step 1: Score every lane on the Agents Rule of Two, per session</h2>
<p>Score lanes, not models. A lane is one configured job: its model, tools, credentials, inputs, and whatever it can spawn. Score it for one session, which is Meta’s unit, and use this table to decide what counts.</p>
<table>
<thead>
<tr>
<th>Leg</th>
<th>Counts as the leg</th>
<th>The miss that makes fleets under-count</th>
</tr>
</thead>
<tbody>
<tr>
<td>[A] untrusted input</td>
<td>Any text someone outside the lane’s owners could have written: PR bodies and comments, issue text, email, web pages, package READMEs, third-party API responses, files from a fork</td>
<td>Tool output is input. A search result, a log line with a user-agent string, a scraped changelog</td>
</tr>
<tr>
<td>[B] sensitive data or systems</td>
<td>Private repos, customer records, secrets in the environment, production credentials, internal tickets, the user’s home directory</td>
<td>A token scoped to “read” on a private org is still [B]</td>
</tr>
<tr>
<td>[C] state change or outbound call</td>
<td>File writes, git push, PR and issue comments, email and chat, package install or publish, workflow dispatch, web fetch, any MCP tool that calls a remote server</td>
<td>A GET is outbound: the host and query string carry data out</td>
</tr>
</tbody>
</table>
<p>Read [C] literally. Meta’s wording covers changing state <em>or</em> communicating externally, and the second half is where fleets under-count. A web fetch is outbound communication even when it only reads, because the URL is a message and an injected instruction can pick the host. An MCP search tool ships its query string to somebody else’s server.</p>
<p>A git push is [C] twice: it changes the repo, and it can start a CI workflow that holds secrets, which is why <a href="/intel/agent-actions-trigger-allowlist/">the Actions trigger allowlist</a> gives agent identities their own row. Never score [C] as “acts autonomously”. Autonomy is the supervision question in step 4, not a leg.</p>
<p>Then apply the subagent rule. A child agent that reads a web page [A] and returns a summary to a parent holding the org token [B] and a push tool [C] makes the parent an all-three session, whatever the child scores alone. Score the union of every context that flows into another. The only reset is the one Meta names: a fresh session with a fresh context window, fed by something other than the old transcript.</p>
<table>
<thead>
<tr>
<th>Lane (illustrative)</th>
<th>[A]</th>
<th>[B]</th>
<th>[C]</th>
<th>Verdict</th>
</tr>
</thead>
<tbody>
<tr>
<td>PR review bot</td>
<td>PR body, comments</td>
<td>private-org read token</td>
<td>review comments, web fetch</td>
<td>All three: split</td>
</tr>
<tr>
<td>Incident summarizer</td>
<td>alert payloads with user-supplied strings</td>
<td>production log read</td>
<td>posts to the on-call channel</td>
<td>All three: split</td>
</tr>
<tr>
<td>Inbox assistant</td>
<td>inbound customer mail</td>
<td>CRM record</td>
<td>sends replies</td>
<td>All three: supervise (step 4)</td>
</tr>
<tr>
<td>Issue triage, public repo</td>
<td>issue text</td>
<td>none beyond public read</td>
<td>labels</td>
<td>[A C]: passes</td>
</tr>
<tr>
<td>Release publisher</td>
<td>none (tagged build only)</td>
<td>publish credential</td>
<td>publish</td>
<td>[B C]: passes</td>
</tr>
</tbody>
</table>
<p>Write the result down per lane. The illustrative chart shows the shape of the fix: the failing lanes go to zero and the lane count goes up.</p>
<p>
<em>Illustrative, modeled numbers. The split adds lanes; the number to drive to zero is the red one.</em></p>
<h2 id="step-2-drop-a-leg-first-and-split-only-when-the-job-needs-all-three">Step 2: Drop a leg first, and split only when the job needs all three</h2>
<p>Dropping a leg is cheaper than splitting, so try it first. Drop [A] by feeding the lane structured data you control instead of prose: a dependency-update lane reads version numbers from a lockfile diff, not changelog text. Drop [B] by running on a public mirror or a scratch clone with no org credential. Shrink [C] by taking tools away.</p>
<p>Narrowing publish rights to <a href="/intel/agent-stage-only-publish-tokens/">stage-only tokens</a> helps, but GitHub’s changelog says those tokens keep other write rights, dist-tag moves and deprecations included, so the leg shrinks rather than disappears.</p>
<p>When the job genuinely needs all three, split it into two lanes with a typed handoff between them.</p>
<ul>
<li><strong>The reader holds [A B].</strong> It gets the untrusted text plus whatever private context it needs to interpret it. It has no write tools and no egress: no shell, no web fetch, no MCP server that reaches the network, and a sandbox with the network off, because “no fetch tool” is a promise and a closed socket is a fact.</li>
<li><strong>The runner fetches the reader’s input</strong>, by PR number, with a request that carries no private data. The reader never fetches anything.</li>
<li><strong>The reader emits only a handoff</strong> that must validate against a versioned JSON Schema. A failed validation stops the run; nothing is retried with the raw text attached.</li>
<li><strong>The actor holds [B C].</strong> It gets the handoff and its own trusted instructions, never the raw text: not in the prompt, not through a log, and not through a tool that can re-read the same PR. A <code>gh pr view</code> on the actor is a side door.</li>
</ul>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># lanes/pr-review.yaml (illustrative shape; enforced by the runner, never by the prompt)</span></span>
<span class="line"><span style="color:#85E89D">lane</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">pr-review</span></span>
<span class="line"><span style="color:#85E89D">reader</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  legs</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">A</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">B</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  inputs</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">runner.fetch.pr_body</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">runner.fetch.pr_diff</span><span style="color:#E1E4E8">]   </span><span style="color:#6A737D"># fetched by PR number</span></span>
<span class="line"><span style="color:#85E89D">  tools</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">read_file</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">grep</span><span style="color:#E1E4E8">]                               </span><span style="color:#6A737D"># no write, no shell, no network</span></span>
<span class="line"><span style="color:#85E89D">  network</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">off</span></span>
<span class="line"><span style="color:#85E89D">  output_schema</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">handoff/pr-review.v3.json</span></span>
<span class="line"><span style="color:#85E89D">actor</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  legs</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">B</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">C</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  inputs</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">handoff</span><span style="color:#E1E4E8">]                                      </span><span style="color:#6A737D"># never the raw PR text</span></span>
<span class="line"><span style="color:#85E89D">  tools</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">post_review_comment</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">add_label</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  network_allow</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">api.github.com</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  deny_tools</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">gh_pr_view</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">web_fetch</span><span style="color:#E1E4E8">]                    </span><span style="color:#6A737D"># side doors to the raw text</span></span>
<span class="line"><span style="color:#85E89D">supervision</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  required</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">false</span><span style="color:#6A737D">                                        # the lint checks this line</span></span></code></pre>
<p>
<em>The reader never holds a write or egress tool, and the actor never reads the raw text. The fixture proves the gate between them.</em></p>
<h2 id="step-3-treat-every-reader-derived-field-as-tainted-schema-or-no-schema">Step 3: Treat every reader-derived field as tainted, schema or no schema</h2>
<p>The split feels like a firewall because the handoff is typed. A schema checks shape. An injected reader can emit perfectly valid JSON whose <code>notify</code> field is the attacker’s address, whose <code>docs_url</code> points at the attacker’s host, or whose <code>path</code> climbs out of the repo.</p>
<p>The <a href="https://arxiv.org/html/2503.18813">CaMeL paper</a> names this. In the Dual LLM pattern a privileged model plans and a quarantined model reads untrusted data; its authors show injected content can steer the quarantined model into returning attacker-chosen data while the plan stays intact, so “the data flow can still be manipulated.” Their analogy is SQL injection: the attacker changes the parameters, not the query. A later <a href="https://arxiv.org/html/2506.08837">paper on agent design patterns</a>, with several of the same authors, says the same of plan-then-execute agents: the plan holds, and injected text still shapes the planned calls’ inputs.</p>
<p>
<em>Screenshot: arXiv, “Defeating Prompt Injections by Design” (Jun 24, 2025), captured Sep 21, 2026.</em></p>
<p>So the handoff needs four controls, and the actor enforces all of them in code.</p>
<ol>
<li><strong>Tag every field with its source.</strong> Three values are enough: <code>trusted_config</code> (from the lane config or the runner), <code>actor_lookup</code> (the actor fetched it by key from a system of record) and <code>reader_derived</code> (it came out of the reader).</li>
<li><strong>Narrow free-form fields.</strong> Enums, booleans, bounded integers and IDs that must resolve in a trusted system. <code>severity: low | medium | high</code> beats <code>assessment: string</code> for anything the actor branches on.</li>
<li><strong>Gate consequential fields.</strong> A reader-derived value may land in a recipient, URL, path, branch, package name or tool argument only after a deterministic allowlist in code passes it, or a human approves the exact value. If the allowlist lookup errors, times out or can’t parse the value, the answer is refuse and alert. A classifier asked whether the URL looks safe isn’t a gate, because <a href="/intel/jev-gate-injection-jaggedness/">it reads the attacker’s text too</a>.</li>
<li><strong>Narrow the author where you can’t narrow the value.</strong> Meta’s own high-velocity coder example controls [A] by “Using author-lineage to filter all data sources” that reach the agent’s context. If only your team’s commits and tickets feed the reader, most of [A] never arrives.</li>
</ol>
<p>One field type fools people: free text that leaves. The reader holds [B], so any string it emits could carry private data, and an actor that posts that string somewhere public has just exfiltrated it. Free text is consequential when its destination is outside the trust boundary it came from.</p>
<table>
<thead>
<tr>
<th>Handoff field</th>
<th>Type</th>
<th>Source</th>
<th>Consequential</th>
<th>Gate on the actor</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>pr_number</code></td>
<td>integer</td>
<td>trusted_config</td>
<td>yes</td>
<td>must equal the run’s PR</td>
</tr>
<tr>
<td><code>verdict</code></td>
<td>enum: comment, request_changes, approve</td>
<td>reader_derived</td>
<td>yes</td>
<td><code>approve</code> always goes to a human</td>
</tr>
<tr>
<td><code>labels</code></td>
<td>set of repo labels</td>
<td>reader_derived</td>
<td>low</td>
<td>allowlist = the repo’s label list</td>
</tr>
<tr>
<td><code>docs_url</code></td>
<td>URL</td>
<td>reader_derived</td>
<td>yes</td>
<td>host on a fixed list, else dropped</td>
</tr>
<tr>
<td><code>notify</code></td>
<td>email</td>
<td>reader_derived</td>
<td>yes</td>
<td>must resolve in the team directory, else a human</td>
</tr>
<tr>
<td><code>comment_body</code></td>
<td>string, 2,000 chars max</td>
<td>reader_derived</td>
<td>yes, if it leaves the repo</td>
<td>posts only to this PR; secret scan first</td>
</tr>
</tbody>
</table>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span></span>
<span class="line"><span style="color:#79B8FF">  "schema"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"pr-review.v3"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">  "pr_number"</span><span style="color:#E1E4E8">:    { </span><span style="color:#79B8FF">"value"</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">4127</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"source"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"trusted_config"</span><span style="color:#E1E4E8"> },</span></span>
<span class="line"><span style="color:#79B8FF">  "verdict"</span><span style="color:#E1E4E8">:      { </span><span style="color:#79B8FF">"value"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"request_changes"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"source"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"reader_derived"</span><span style="color:#E1E4E8"> },</span></span>
<span class="line"><span style="color:#79B8FF">  "docs_url"</span><span style="color:#E1E4E8">:     { </span><span style="color:#79B8FF">"value"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"https://docs.example.com/testing"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"source"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"reader_derived"</span><span style="color:#E1E4E8"> },</span></span>
<span class="line"><span style="color:#79B8FF">  "comment_body"</span><span style="color:#E1E4E8">: { </span><span style="color:#79B8FF">"value"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Add a test for the retry path."</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"source"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"reader_derived"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span></code></pre>
<h2 id="step-4-supervise-what-you-cant-split-on-every-write-and-every-call-out">Step 4: Supervise what you can’t split, on every write and every call out</h2>
<p>Some jobs need all three and don’t split cleanly. The inbox assistant has to read the customer’s mail, know the customer’s record and reply, and what it should do depends on what the mail says. The CaMeL authors list this as an inherent limit of their own design, “Data requires action”: when the actions to take depend on untrusted data, the plan can’t be fixed in advance. The AEPD guidance’s case 1-2, uncontrolled input plus access to sensitive information, reaches the same place: “se debe impedir cualquier acción automática sin supervisión humana” (any automatic action without human supervision must be prevented), whether its effect lands inside or outside the organisation.</p>
<p>Supervision under this rule has a specific meaning. A human approves each state change <strong>and</strong> each outbound call, fetches included. They see the exact call with its arguments, not a plan summary.</p>
<p>Approvals expire, and expiry is a deny. The approver is never the person who wrote the [A] text.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#85E89D">supervision</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  required</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">true</span></span>
<span class="line"><span style="color:#85E89D">  approve_each</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">state_change</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">outbound_call</span><span style="color:#E1E4E8">]   </span><span style="color:#6A737D"># not "the plan"</span></span>
<span class="line"><span style="color:#85E89D">  show</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">exact_call_with_arguments</span></span>
<span class="line"><span style="color:#85E89D">  expire_after</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">30m</span><span style="color:#6A737D">                             # expiry = deny</span></span>
<span class="line"><span style="color:#85E89D">  owner</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">platform-oncall</span></span>
<span class="line"><span style="color:#85E89D">  review_by</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2026-12-01</span><span style="color:#6A737D">                         # the exception has an end date</span></span></code></pre>
<p>Supervised lanes spend attention, and <a href="/intel/hitl-approval-queue-hygiene/">approval fatigue turns a queue into a rubber stamp</a>. Keep an owner and a review date on each one, and treat the supervised count as a number to shrink.</p>
<h2 id="step-5-ship-the-pre-run-lint-and-make-it-fail-closed">Step 5: Ship the pre-run lint, and make it fail closed</h2>
<p>The lint runs in the runner before the harness starts. It reads the lane’s declared config, maps every tool, credential and input to legs through one classification file the platform team owns, and exits non-zero on any of these:</p>
<ul>
<li>The union of legs across the lane and every spawn profile it may use includes A, B and C, with no <code>supervision.required: true</code>.</li>
<li>A tool, MCP server or credential is missing from the classification file. Unclassified counts as [A] and [C] until someone classifies it.</li>
<li>A reader holds any [C] tool or has network access; an actor has any input or tool that reaches the raw text.</li>
<li>A <code>reader_derived</code> field feeds a consequential parameter with no declared gate.</li>
<li>The config doesn’t parse, the classification file is missing, or the lint itself crashes. A lint that fails open is a suggestion.</li>
</ul>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#6A737D"># rule_of_two_lint.py (illustrative shape): any exit other than 0 keeps the lane from starting</span></span>
<span class="line"><span style="color:#F97583">import</span><span style="color:#E1E4E8"> sys, yaml</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">def</span><span style="color:#B392F0"> legs_of</span><span style="color:#E1E4E8">(item, table):</span></span>
<span class="line"><span style="color:#F97583">    return</span><span style="color:#79B8FF"> set</span><span style="color:#E1E4E8">(table.get(item, [</span><span style="color:#9ECBFF">"A"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"C"</span><span style="color:#E1E4E8">]))   </span><span style="color:#6A737D"># unclassified = untrusted input + egress</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">def</span><span style="color:#B392F0"> lane_legs</span><span style="color:#E1E4E8">(lane, table):</span></span>
<span class="line"><span style="color:#E1E4E8">    legs </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> set</span><span style="color:#E1E4E8">()</span></span>
<span class="line"><span style="color:#F97583">    for</span><span style="color:#E1E4E8"> part </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> [lane] </span><span style="color:#F97583">+</span><span style="color:#E1E4E8"> lane.get(</span><span style="color:#9ECBFF">"spawns"</span><span style="color:#E1E4E8">, []):</span></span>
<span class="line"><span style="color:#F97583">        for</span><span style="color:#E1E4E8"> item </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> part.get(</span><span style="color:#9ECBFF">"inputs"</span><span style="color:#E1E4E8">, []) </span><span style="color:#F97583">+</span><span style="color:#E1E4E8"> part.get(</span><span style="color:#9ECBFF">"tools"</span><span style="color:#E1E4E8">, []) </span><span style="color:#F97583">+</span><span style="color:#E1E4E8"> part.get(</span><span style="color:#9ECBFF">"credentials"</span><span style="color:#E1E4E8">, []):</span></span>
<span class="line"><span style="color:#E1E4E8">            legs </span><span style="color:#F97583">|=</span><span style="color:#E1E4E8"> legs_of(item, table)</span></span>
<span class="line"><span style="color:#F97583">    return</span><span style="color:#E1E4E8"> legs</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">try</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">    table </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> yaml.safe_load(</span><span style="color:#79B8FF">open</span><span style="color:#E1E4E8">(</span><span style="color:#9ECBFF">"policy/legs.yaml"</span><span style="color:#E1E4E8">))</span></span>
<span class="line"><span style="color:#E1E4E8">    lane </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> yaml.safe_load(</span><span style="color:#79B8FF">open</span><span style="color:#E1E4E8">(sys.argv[</span><span style="color:#79B8FF">1</span><span style="color:#E1E4E8">]))</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> lane_legs(lane, table) </span><span style="color:#F97583">&gt;=</span><span style="color:#E1E4E8"> {</span><span style="color:#9ECBFF">"A"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"B"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"C"</span><span style="color:#E1E4E8">} </span><span style="color:#F97583">and</span><span style="color:#F97583"> not</span><span style="color:#E1E4E8"> lane.get(</span><span style="color:#9ECBFF">"supervision"</span><span style="color:#E1E4E8">, {}).get(</span><span style="color:#9ECBFF">"required"</span><span style="color:#E1E4E8">):</span></span>
<span class="line"><span style="color:#E1E4E8">        sys.exit(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"refuse </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">lane[</span><span style="color:#9ECBFF">'lane'</span><span style="color:#E1E4E8">]</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">: A+B+C in one session with no supervision flag"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">except</span><span style="color:#79B8FF"> Exception</span><span style="color:#F97583"> as</span><span style="color:#E1E4E8"> e:</span></span>
<span class="line"><span style="color:#E1E4E8">    sys.exit(</span><span style="color:#F97583">f</span><span style="color:#9ECBFF">"refuse: lint error </span><span style="color:#79B8FF">{</span><span style="color:#E1E4E8">e</span><span style="color:#F97583">!r</span><span style="color:#79B8FF">}</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">)    </span><span style="color:#6A737D"># fail closed</span></span></code></pre>
<p>Be honest about what the lint can’t see. It reads what a lane declares, and the harness can load more at runtime: a synced plugin, an auto-discovered MCP server, a subagent that inherits the parent’s tools.</p>
<p>So the lint isn’t the boundary. The wall behind it is the reader’s sandbox with the network off, permission deny rules on the actor, and credentials the reader’s process never receives. For unattended lanes, the <a href="/intel/headless-agent-trust-tier/">trust-tier inventory of input authors and credentials</a> is already the lint’s best input; feed it in rather than deriving it twice.</p>
<h2 id="step-6-prove-the-split-with-a-malicious-handoff-fixture">Step 6: Prove the split with a malicious-handoff fixture</h2>
<p>Keep a fixture set next to each split lane and run it in CI on every change to the lane config, the classification file or the handoff schema, and after every harness upgrade. The core fixture is a handoff that validates against the schema and names an attacker’s destination. The actor must refuse it.</p>
<table>
<thead>
<tr>
<th>Fixture</th>
<th>Schema-valid value</th>
<th>Expected result</th>
</tr>
</thead>
<tbody>
<tr>
<td>Attacker URL</td>
<td><code>docs_url</code> on a host outside the list</td>
<td>Gate drops it; no fetch; alert names the field and source</td>
</tr>
<tr>
<td>Attacker recipient</td>
<td><code>notify</code> outside the team directory</td>
<td>Routed to a human; no mail sent</td>
</tr>
<tr>
<td>Path escape</td>
<td><code>path</code> that normalises outside the repo root</td>
<td>Refused before any file call</td>
</tr>
<tr>
<td>Protected branch</td>
<td><code>branch</code> set to the default branch</td>
<td>Refused; the lane pushes only to its own branches</td>
</tr>
<tr>
<td>Smuggled secret</td>
<td><code>comment_body</code> containing a token-shaped string</td>
<td>Secret scan blocks the post</td>
</tr>
<tr>
<td>Valid, wrong enum</td>
<td><code>verdict: approve</code> on a PR with failing checks</td>
<td>Human only; never automatic</td>
</tr>
</tbody>
</table>
<p>Judge the result from the sandbox’s egress log, not from the actor’s transcript. The transcript belongs to the component under test; the egress log shows whether a packet left.</p>
<p>Then run the other half: feed the reader a PR body that tells it to set <code>docs_url</code> to the attacker’s host, and put a unique canary string in that body. Expect the reader to comply sometimes, since the design assumes the reader loses. The pass condition is that the gate catches the value and the canary never shows up in the actor’s context. A reader that happens to resist the injection this week isn’t a pass; it’s luck with a timestamp.</p>
<h2 id="agents-rule-of-two-failures-and-the-signal-for-each">Agents Rule of Two failures, and the signal for each</h2>
<p><strong>The reader grows an egress.</strong> Someone adds an MCP server or a fetch tool to the reader for more context. Signal: any connection in the reader sandbox’s egress log, or a lint diff showing a new [C] item on a reader.</p>
<p><strong>The actor finds a side door.</strong> A tool, a log file or a cached artifact lets the actor read the raw text. Signal: the fixture’s canary string appears in the actor’s context.</p>
<p><strong>Free text creeps back.</strong> A <code>notes: string</code> field lands in the schema because an enum felt restrictive. Signal: a schema diff adding an unbounded string with no gate.</p>
<p><strong>The allowlist sprawls.</strong> Signal: wildcard hosts, or entries growing week over week without a named requester.</p>
<p><strong>Supervision decays.</strong> Signal: approval rate near 100% while median decision time falls. Nobody reads a call they approve in two seconds.</p>
<p><strong>Lanes start around the runner.</strong> Someone launches the harness by hand with the lane’s credentials. Signal: harness sessions with no matching lint record.</p>
<p><strong>Subagents inherit too much.</strong> Signal: spawn records whose tool lists differ from the declared spawn profile.</p>
<h2 id="the-rule-of-two-lives-in-the-runner-not-in-the-prompt">The Rule of Two lives in the runner, not in the prompt</h2>
<p>None of the steps above is a prompt instruction. A reader told to ignore instructions in the PR body will follow them eventually; the split works because the reader has no tool to follow them with, and the actor never reads them. That makes the rule a property of the layer that starts lanes: the lane registry, the leg classification file, the lint, the egress logs and the fixture run. It is what a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> turns into once the dashboard comes off, one place that knows every lane, its legs, and who signed the exception.</p>
<p>[B] also covers where the lane’s prompt goes, not only what the lane can reach. If a vendor lane forwards requests to another provider, the data class it may carry is capped by the <a href="/intel/vendor-lane-last-hop-register/">last hop you can verify</a>. Score that too.</p>
<h2 id="faq">FAQ</h2>
<h3 id="what-is-the-agents-rule-of-two">What is the Agents Rule of Two?</h3>
<p>Meta’s Oct 31, 2025 rule for agent security. Within one session, an agent should hold no more than two of three properties: processing untrustworthy input, access to sensitive systems or private data, and the ability to change state or communicate externally. A job that needs all three gets supervision instead of autonomy.</p>
<h3 id="does-splitting-an-agent-into-a-reader-and-an-actor-stop-prompt-injection">Does splitting an agent into a reader and an actor stop prompt injection?</h3>
<p>Not by itself. It keeps injected text away from the tools, but the reader can still be steered into emitting valid, attacker-chosen values. CaMeL’s authors show the Dual LLM pattern protects control flow, not data flow. Gate every reader-derived value in a consequential field with an allowlist or a human.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://www.aepd.es/prensa-y-comunicacion/blog/primera-notiviacion-brecha-datos-personales-causada-por-ataque-ejecutado-mediante-agente-ia">AEPD blog, Primera notificación de una brecha de datos personales causada por un ataque ejecutado mediante un agente de IA</a> — Sep 14, 2026</li>
<li><a href="https://www.aepd.es/guias/orientaciones-ia-agentica.pdf">AEPD, Inteligencia artificial agéntica desde la perspectiva de protección de datos, V1.2</a> — febrero de 2026; “Regla de 2”, p. 41</li>
<li><a href="https://ai.meta.com/blog/practical-ai-agent-security/">Meta, Agents Rule of Two: A Practical Approach to AI Agent Security</a> — Oct 31, 2025</li>
<li><a href="https://arxiv.org/html/2503.18813">Debenedetti et al., Defeating Prompt Injections by Design (CaMeL), full text</a> — Dual LLM data-flow critique; “Data requires action”</li>
<li><a href="https://arxiv.org/html/2506.08837">Beurer-Kellner et al., arXiv 2506.08837, full text</a> — six agent design patterns</li>
<li><a href="https://chromium.googlesource.com/chromium/src/+/main/docs/security/rule-of-2.md">Chromium security docs, rule-of-2.md</a> — cited by the AEPD (footnote 32)</li>
</ul>
]]></content:encoded></item><item><title>Stage-Only npm Tokens for Agents: What They Close and What They Don’t</title><link>https://automater.ai/intel/agent-stage-only-publish-tokens/</link><guid>https://automater.ai/intel/agent-stage-only-publish-tokens/</guid><description>An npm stage-only token blocks an agent&apos;s direct publish but can still move dist-tags and deprecate versions. Token classes, a drill, alerts, a 2FA review.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>Since Sep 18, npm rejects <code>npm publish</code> from a token that is only allowed to stage, even one configured to bypass 2FA for automation. That is the whole of the new npm stage-only token: the lane holding it can put a version in a waiting room, and only a maintainer with a second factor can let it out. For any agent that touches a release, that is a genuine fix, and it closes exactly one door.</p>
<p>The same short changelog names two doors it leaves open. A stage-only token can still move dist-tags and deprecate versions, and GitHub tells you to guard it like any other write token. Add the credentials it never touched, the bypass-2FA tokens already issued and the GitHub OAuth token in a developer’s CLI login, and you have the real shape of the problem.</p>
<p>The move for Tuesday: every lane that can reach the registry gets a token class that matches its job, and none of them gets publish. You write down what the stage class can still do, prove with the agent’s own token that a direct publish fails, alert on every publish, dist-tag move and deprecation in your namespace, and strip long-lived human credentials off agent hosts. The approval stays a person comparing a staged tarball to a CI build. Where the pipeline supports trusted publishing, the long-lived token goes away entirely.</p>
<h2 id="sep-18-the-npm-stage-only-token-splits-stage-from-publish">Sep 18: the npm stage-only token splits stage from publish</h2>
<p>On Sep 18, 2026, <a href="https://github.blog/changelog/2026-09-18-stage-only-npm-tokens-for-safer-automation/">GitHub’s changelog</a> added a new permission level for npm granular access tokens, “Read and write (stage only)”. The workflow runs <code>npm stage publish</code>, a maintainer approves the release with 2FA, and “npm rejects direct npm publish attempts with that token, even if you’ve configured it to bypass 2FA for automation.” You need publish access to the package, 2FA on the account, npm CLI 11.15.0 or later and Node.js 22.14.0 or later. The release is opt-in and “doesn’t change existing tokens or their direct-publish capabilities.”</p>
<p>The lines that matter most sit in the middle of the post: “Stage-only tokens retain other package write permissions, including moving dist-tags and deprecating versions. Protect them with the same care as any other write token.” Further down comes a date: npm “is targeting January 2027 to remove direct publishing through bypass-2FA tokens”, and it points to npm’s <a href="https://github.com/orgs/community/discussions/208130">roadmap discussion</a> for the wider plan.</p>
<p>
<em>Screenshot: GitHub Changelog, “Stage-only npm tokens for safer automation” (Sep 18, 2026), captured Sep 21, 2026.</em></p>
<p>Staging itself is older. GitHub’s changelog announced <a href="https://github.blog/changelog/2026-05-22-staged-publishing-and-new-install-time-controls-for-npm/">staged publishing and new install-time controls for npm</a> on May 22 and <a href="https://github.blog/changelog/2026-09-03-multiple-trusted-publishing-configurations-for-npm/">multiple trusted-publishing configurations</a> on Sep 3. npm’s <a href="https://docs.npmjs.com/staged-publishing/">staged-publishing docs</a> supply the mechanics: <code>npm stage publish</code> “does not require 2FA”, <code>npm stage list</code>, <code>npm stage view</code> and <code>npm stage download</code> show you what is waiting, and approval asks for a second factor “whether you approve the package in the CLI or on npmjs.com.”</p>
<aside class="social-embed" data-x-embed data-post-id="2058225951255986180">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">May 23, 2026 — The Hacker News on the May launch of npm staged publishing: staged releases wait for a human's 2FA approval before anyone can install them, even when CI did the staging.</p>
  <a class="social-embed__source" href="https://x.com/TheHackersNews/status/2058225951255986180" target="_blank" rel="noopener noreferrer">View The Hacker News's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<p>Two days before the tokens shipped, <a href="https://thehackernews.com/2026/09/attacker-hijacks-ai-coding-assistant.html">The Hacker News</a> carried Mandiant’s account of a hijacked AI coding session at an unnamed software-as-a-service provider. The <a href="https://cloud.google.com/security/resources/ai-risk-and-resilience-2026">Mandiant report</a>, dated September 2026, is blunt about the chain: “Once the recommendation was accepted, the attacker used the developer’s active session to install an infostealer using a poisoned PyPI package, harvest GitHub OAuth tokens, and deploy the self-propagating Shai-Hulud worm across approximately 100 internal code repositories.” The attacker then poisoned a package in the company’s official namespace, and a second employee pulled it.</p>
<p>
<em>Screenshot: Google Cloud, “Mandiant AI Risk and Resilience Report 2026” (September 2026), captured Sep 21, 2026.</em></p>
<p>Be precise about what that case shows. The entry vector was a poisoned PyPI package, so a stage-only npm token is not a counterfactual for it, and Mandiant’s report never mentions npm, publish tokens or staged publishing. It names neither the assistant nor the registry behind the poisoned official-namespace package. What the case does show is the part this runbook covers: a hijacked session inherits whatever credentials its host holds, and the second infection came out of the victim’s own namespace.</p>
<h2 id="an-agent-holding-a-publish-token-is-a-release-nobody-watches">An agent holding a publish token is a release nobody watches</h2>
<p>A publish token in a chatbot’s world is a secret someone might paste into a prompt. In an agent’s world it is a capability the lane exercises on its own schedule: the lane decides the version is ready, runs the command, and every downstream install picks it up. Hijack the session, or poison something the session trusts, and the attacker inherits the capability with the same timing.</p>
<p>Stage-only tokens move the irreversible step out of the lane and onto a person with a second factor. They don’t make the token read-only, and they do nothing about credentials the lane was never supposed to hold. The runbook below covers both halves.</p>
<h2 id="step-1-give-every-lane-a-token-class-and-never-the-publish-class">Step 1: Give every lane a token class, and never the publish class</h2>
<p>Four classes cover most fleets. Agents get three of them.</p>
<table>
<thead>
<tr>
<th>Class</th>
<th>Typical lanes</th>
<th>npm credential</th>
<th>GitHub credential</th>
<th>Can it make a version installable?</th>
</tr>
</thead>
<tbody>
<tr>
<td>Read</td>
<td>dependency audits, test runs, codegen that installs packages</td>
<td>none for public packages; a read-only token for private ones</td>
<td>read-only, the lane’s own identity</td>
<td>No</td>
</tr>
<tr>
<td>PR</td>
<td>coding agents that bump versions, edit changelogs, open release PRs</td>
<td>none</td>
<td>the lane’s own App or fine-grained token, scoped to branches and PRs</td>
<td>No</td>
</tr>
<tr>
<td>Stage</td>
<td>the release job an agent runs or triggers</td>
<td>“Read and write (stage only)”, scoped to the packages that job releases</td>
<td>only what the release job needs</td>
<td>No: a human approves with 2FA</td>
</tr>
<tr>
<td>Publish</td>
<td>nobody in the fleet</td>
<td>a maintainer’s own account with 2FA</td>
<td>not applicable</td>
<td>Yes</td>
</tr>
</tbody>
</table>
<p>Scope every stage token to the packages that lane actually releases, and give it the shortest expiry your rotation can live with. The GitHub side of each class, one App or service principal per lane with short-lived tokens, is covered in <a href="/intel/agent-service-principal-sso/">the service-principal playbook for agent identity</a>; broker patterns are their own discipline and out of scope here.</p>
<p>Before you issue a single stage-only token, check the floors in the lane’s real runtime, not on your laptop: npm CLI 11.15.0 or later, Node.js 22.14.0 or later, publish access for the account that owns the token, and 2FA enabled on that account. A lane that fails the preflight must fail loudly. The dangerous version is a helpful release script that notices staging is unavailable and reaches for the old automation token.</p>
<p>Then write down what the stage class can still do. The changelog says “including”, so its list is a floor, not an inventory. Give the register three states: documented, rejected, and test it.</p>
<table>
<thead>
<tr>
<th>Action with the stage-only token</th>
<th>Status</th>
<th>Source</th>
<th>Your control</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>npm stage publish</code></td>
<td>allowed, the point of the class</td>
<td>changelog, npm docs</td>
<td>CI build record per stage (Step 5)</td>
</tr>
<tr>
<td>direct <code>npm publish</code></td>
<td>rejected, even with bypass-2FA set</td>
<td>changelog</td>
<td>the drill in Step 2</td>
</tr>
<tr>
<td>move a dist-tag</td>
<td>allowed</td>
<td>changelog</td>
<td>namespace alert (Step 3)</td>
</tr>
<tr>
<td>deprecate a version</td>
<td>allowed</td>
<td>changelog</td>
<td>namespace alert (Step 3)</td>
</tr>
<tr>
<td>approve a staged version</td>
<td>approval prompts for 2FA</td>
<td>npm docs</td>
<td>the approver is a person (Step 5)</td>
</tr>
<tr>
<td><code>npm unpublish</code></td>
<td>not documented</td>
<td>test it</td>
<td>drill result, logged here</td>
</tr>
<tr>
<td><code>npm access</code> or <code>npm owner</code> changes</td>
<td>not documented</td>
<td>test it</td>
<td>drill result, logged here</td>
</tr>
</tbody>
</table>
<p>Keep the register beside the token inventory and re-run it whenever npm changes the permission model. It is the first page an incident responder should open.</p>
<h2 id="step-2-prove-direct-publish-fails-with-the-agents-own-token">Step 2: Prove direct publish fails with the agent’s own token</h2>
<p>The rejection is npm’s claim. The token under test is yours. Run the drill from the lane’s runtime, with the exact credential the lane receives in production, against a scratch package in your scope that nothing depends on.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># publish-drill.sh: illustrative shape. Run inside the agent lane's runtime,</span></span>
<span class="line"><span style="color:#6A737D"># with the lane's own token, against a scratch package nobody installs.</span></span>
<span class="line"><span style="color:#6A737D"># Setup, once, by a maintainer: publish $PKG@0.0.1 so the residual checks have a target.</span></span>
<span class="line"><span style="color:#E1E4E8">PKG</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"@yourscope/publish-drill"</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#79B8FF"> --version</span><span style="color:#6A737D">        # expect 11.15.0 or later</span></span>
<span class="line"><span style="color:#B392F0">node</span><span style="color:#79B8FF"> --version</span><span style="color:#6A737D">       # expect v22.14.0 or later</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> whoami</span><span style="color:#6A737D">           # expect the lane's account, never a person's</span></span>
<span class="line"></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> publish</span><span style="color:#E1E4E8">;          </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "direct publish exit: </span><span style="color:#79B8FF">$?</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">   # expect non-zero</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> stage</span><span style="color:#9ECBFF"> publish</span><span style="color:#E1E4E8">;    </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "stage publish exit: </span><span style="color:#79B8FF">$?</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">    # expect 0 and a stage ID</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> stage</span><span style="color:#9ECBFF"> list</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$PKG</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">                                   # expect the staged version</span></span>
<span class="line"></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> dist-tag</span><span style="color:#9ECBFF"> add</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$PKG</span><span style="color:#9ECBFF">@0.0.1"</span><span style="color:#9ECBFF"> drill</span><span style="color:#E1E4E8">;             </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "dist-tag exit: </span><span style="color:#79B8FF">$?</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">   # documented: allowed</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> deprecate</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$PKG</span><span style="color:#9ECBFF">@0.0.1"</span><span style="color:#9ECBFF"> "publish drill"</span><span style="color:#E1E4E8">;      </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "deprecate exit: </span><span style="color:#79B8FF">$?</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">  # documented: allowed</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> unpublish</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$PKG</span><span style="color:#9ECBFF">@0.0.1"</span><span style="color:#E1E4E8">;                      </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "unpublish exit: </span><span style="color:#79B8FF">$?</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">  # undocumented: record it</span></span></code></pre>
<p>Score it against a fixed table, and keep every run’s output with the date and the token’s ID.</p>
<table>
<thead>
<tr>
<th>Check</th>
<th>Pass</th>
<th>A fail means</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>npm whoami</code></td>
<td>the lane’s own account</td>
<td>a human credential is on the lane (Step 4)</td>
</tr>
<tr>
<td>direct publish</td>
<td>non-zero exit, no new version in the registry</td>
<td>the lane can reach a token with direct publish; find it before anything else</td>
</tr>
<tr>
<td>stage publish</td>
<td>a stage ID, version not installable</td>
<td>staging is broken; fix it, never fall back</td>
</tr>
<tr>
<td>dist-tag and deprecate</td>
<td>commands succeed and your alert fires inside one polling interval</td>
<td>the watcher is blind (Step 3)</td>
</tr>
<tr>
<td>unpublish, owner, access</td>
<td>outcome recorded either way</td>
<td>the register is out of date</td>
</tr>
</tbody>
</table>
<p>Run the drill on every token rotation, every npm bump in the lane image, and every edit to the release job. It proves one token and says nothing about the others in reach, so pair it with a sweep of the lane’s environment variables, config files and mounted secrets for any other npm credential. The stage-only release left existing tokens untouched, which means an older bypass-2FA automation token is probably still valid somewhere.</p>
<h2 id="step-3-alert-on-every-publish-dist-tag-move-and-deprecation-in-your-namespace">Step 3: Alert on every publish, dist-tag move and deprecation in your namespace</h2>
<p>The direct-publish rejection comes with the token’s permission level, not from a hook on the agent’s host that a hijacked session could skip. Nothing blocks the residual rights, so the control there is detection. Watch the whole namespace, not only the packages agents release: in the Mandiant case the second infection came from the company’s own namespace.</p>
<table>
<thead>
<tr>
<th>Event</th>
<th>Expected when</th>
<th>Alert when</th>
<th>Severity</th>
</tr>
</thead>
<tbody>
<tr>
<td>new version appears</td>
<td>a human approved a stage ID tied to a CI build record</td>
<td>no matching approval or build record</td>
<td>page</td>
</tr>
<tr>
<td><code>latest</code> dist-tag moves</td>
<td>inside a release window, to the version just approved</td>
<td>outside a window, or pointing at an older version</td>
<td>page</td>
</tr>
<tr>
<td>any other dist-tag moves</td>
<td>the release job, during a release</td>
<td>no release in flight</td>
<td>ticket</td>
</tr>
<tr>
<td>a version is deprecated</td>
<td>a maintainer’s recorded decision</td>
<td>no person attached to it</td>
<td>page</td>
</tr>
<tr>
<td>a staged version appears</td>
<td>the release job ran</td>
<td>no job run matches</td>
<td>ticket</td>
</tr>
<tr>
<td>the watcher misses a poll</td>
<td>never</td>
<td>two polls in a row</td>
<td>page</td>
</tr>
</tbody>
</table>
<p>The dist-tag and deprecation rows page for a reason. The <code>latest</code> tag decides what a plain install resolves, so moving it back to an older release you have since patched is a downgrade nobody approved. A deprecation message is text every installer of that version sees, which makes an unexpected one a phishing channel with your package’s name on it. Both are scenarios rather than events from the case, and both sit inside what the changelog says a stage-only token can do.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># namespace-watch.sh: illustrative shape. Run on a schedule from a host</span></span>
<span class="line"><span style="color:#6A737D"># that holds no npm write credential of any kind. notify and heartbeat</span></span>
<span class="line"><span style="color:#6A737D"># stand for whatever your alerting exposes. Note "$pkg@*" resolves to latest</span></span>
<span class="line"><span style="color:#6A737D"># only; "&gt;=0" prints one line per deprecated release version (not prereleases).</span></span>
<span class="line"><span style="color:#F97583">while</span><span style="color:#79B8FF"> read</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> pkg</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#E1E4E8">  key</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$(</span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$pkg</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> tr</span><span style="color:#9ECBFF"> '/@'</span><span style="color:#9ECBFF"> '__'</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#B392F0">  npm</span><span style="color:#9ECBFF"> view</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$pkg</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> dist-tags</span><span style="color:#79B8FF"> --json</span><span style="color:#F97583">   &gt;</span><span style="color:#9ECBFF"> "state/</span><span style="color:#E1E4E8">$key</span><span style="color:#9ECBFF">.tags.new"</span></span>
<span class="line"><span style="color:#B392F0">  npm</span><span style="color:#9ECBFF"> view</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$pkg</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> time</span><span style="color:#79B8FF"> --json</span><span style="color:#F97583">        &gt;</span><span style="color:#9ECBFF"> "state/</span><span style="color:#E1E4E8">$key</span><span style="color:#9ECBFF">.time.new"</span></span>
<span class="line"><span style="color:#B392F0">  npm</span><span style="color:#9ECBFF"> view</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$pkg</span><span style="color:#9ECBFF">@&gt;=0"</span><span style="color:#9ECBFF"> deprecated</span><span style="color:#F97583">     &gt;</span><span style="color:#9ECBFF"> "state/</span><span style="color:#E1E4E8">$key</span><span style="color:#9ECBFF">.dep.new"</span></span>
<span class="line"><span style="color:#F97583">  for</span><span style="color:#E1E4E8"> kind </span><span style="color:#F97583">in</span><span style="color:#9ECBFF"> tags</span><span style="color:#9ECBFF"> time</span><span style="color:#9ECBFF"> dep</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#B392F0">    cmp</span><span style="color:#79B8FF"> -s</span><span style="color:#9ECBFF"> "state/</span><span style="color:#E1E4E8">$key</span><span style="color:#9ECBFF">.</span><span style="color:#E1E4E8">$kind</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "state/</span><span style="color:#E1E4E8">$key</span><span style="color:#9ECBFF">.</span><span style="color:#E1E4E8">$kind</span><span style="color:#9ECBFF">.new"</span><span style="color:#F97583"> ||</span><span style="color:#B392F0"> notify</span><span style="color:#9ECBFF"> "npm </span><span style="color:#E1E4E8">$kind</span><span style="color:#9ECBFF"> changed: </span><span style="color:#E1E4E8">$pkg</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#B392F0">    mv</span><span style="color:#9ECBFF"> "state/</span><span style="color:#E1E4E8">$key</span><span style="color:#9ECBFF">.</span><span style="color:#E1E4E8">$kind</span><span style="color:#9ECBFF">.new"</span><span style="color:#9ECBFF"> "state/</span><span style="color:#E1E4E8">$key</span><span style="color:#9ECBFF">.</span><span style="color:#E1E4E8">$kind</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#F97583">  done</span></span>
<span class="line"><span style="color:#F97583">done</span><span style="color:#F97583"> &lt;</span><span style="color:#E1E4E8"> namespace-packages.txt</span></span>
<span class="line"><span style="color:#B392F0">heartbeat</span><span style="color:#9ECBFF"> npm-namespace-watch</span></span></code></pre>
<p>The watcher is detection, and its polling interval is your exposure window: on a five-minute poll, a moved tag can serve five minutes of installs before anyone knows. Run it from a host with no npm write credentials, alert on its own missed heartbeats, and keep a slower second check on a different host. If the watcher dies quietly, you are back to trusting the token.</p>
<h2 id="step-4-take-long-lived-human-credentials-off-every-agent-host">Step 4: Take long-lived human credentials off every agent host</h2>
<p>Mandiant’s recommended controls include this one: “Isolate local credentials to prevent extensions from accessing raw API keys, long-lived OAuth token, or secrets”. In the case, the attacker worked through the developer’s active session and harvested GitHub OAuth tokens. The fleet version is a hard rule: agents never hold a person’s long-lived credentials.</p>
<table>
<thead>
<tr>
<th>Credential</th>
<th>Where it hides on an agent host</th>
<th>Replace it with</th>
</tr>
</thead>
<tbody>
<tr>
<td>GitHub OAuth token from a person’s CLI login</td>
<td>the CLI’s credential store under the home directory the agent runs as</td>
<td>the lane’s own App or service-principal token</td>
</tr>
<tr>
<td>a person’s PAT, classic or fine-grained</td>
<td>environment variables, shell profiles, git credential helpers, <code>.env</code> files</td>
<td>a per-lane identity with its own scope</td>
</tr>
<tr>
<td>bypass-2FA npm publish token</td>
<td><code>.npmrc</code> files, CI secrets the lane can read, environment variables</td>
<td>a stage-only token, or trusted publishing (Step 6)</td>
</tr>
<tr>
<td>a person’s npm login session</td>
<td><code>.npmrc</code> in that person’s home directory</td>
<td>nothing; the person approves from their own device</td>
</tr>
</tbody>
</table>
<p>Sweep for them by value and by location on every host an agent runs on, including developer workstations where an assistant shares the developer’s session. That last category is the awkward one, because the Mandiant chain ran in exactly that kind of session. You can’t take the developer’s GitHub login away from the developer. You can keep publish tokens and bypass-2FA tokens off that machine entirely, so a hijacked session there finds stage rights at most, and usually nothing.</p>
<p>Don’t wait for January 2027 to retire the old tokens. Inventory every bypass-2FA token now, replace each with a stage-only token or trusted publishing, and revoke the old one the day its replacement passes Step 2. The target date is npm’s stated plan, and a plan can slip; a revoked token can’t.</p>
<p>
<em>Every date is from GitHub’s changelog or The Hacker News. January 2027 is npm’s stated target, drawn as a month because no day is given.</em></p>
<h2 id="step-5-the-approver-checks-the-staged-tarball-against-the-ci-build-then-approves">Step 5: The approver checks the staged tarball against the CI build, then approves</h2>
<p>The human step is the whole security value of staging, and it is easy to hollow out. An approver who clicks through because a chat message said the release was ready is a bypass-2FA token with extra steps. Give the approval a five-minute checklist and a record that proves each check happened.</p>
<ol>
<li>The CI job that stages the version records the tarball’s SHA-256, the commit SHA and the build ID in the release ticket before it runs <code>npm stage publish</code>.</li>
<li>The approver runs <code>npm stage list</code> and <code>npm stage view</code> on the stage ID and confirms the package name and version match the ticket. npm also keeps a separate <a href="https://docs.npmjs.com/cli/v11/commands/npm-stage/"><code>npm stage</code> command reference</a>.</li>
<li>The approver downloads the staged tarball with <code>npm stage download</code>, hashes it, and compares the hash with the CI record. If the hashes differ, unpack both and diff them: a difference only in the compressed bytes is noise, a changed file is a stop.</li>
<li>The approver diffs the staged package against the last published version, looking for new install scripts, new dependencies, new binaries and changed entry points. Any of those without a matching reviewed PR is a stop.</li>
<li>Only then does the approver run <code>npm stage approve</code> or use the Staged Packages tab on npmjs.com, with 2FA from their own device.</li>
</ol>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># approve-check.sh: illustrative. The human approver runs this; no lane ever does.</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> stage</span><span style="color:#9ECBFF"> view</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$STAGE_ID</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">                # name and version match the ticket?</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> stage</span><span style="color:#9ECBFF"> download</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$STAGE_ID</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">            # fetch the staged tarball</span></span>
<span class="line"><span style="color:#B392F0">sha256sum</span><span style="color:#9ECBFF"> ./</span><span style="color:#F97583">&lt;</span><span style="color:#9ECBFF">downloaded-tarbal</span><span style="color:#E1E4E8">l</span><span style="color:#F97583">&gt;</span><span style="color:#9ECBFF">.tgz</span><span style="color:#6A737D">      # equal to the SHA-256 in the CI build record?</span></span>
<span class="line"><span style="color:#B392F0">mkdir</span><span style="color:#79B8FF"> -p</span><span style="color:#9ECBFF"> staged</span><span style="color:#9ECBFF"> prev</span></span>
<span class="line"><span style="color:#B392F0">tar</span><span style="color:#79B8FF"> -xzf</span><span style="color:#9ECBFF"> ./</span><span style="color:#F97583">&lt;</span><span style="color:#9ECBFF">downloaded-tarbal</span><span style="color:#E1E4E8">l</span><span style="color:#F97583">&gt;</span><span style="color:#9ECBFF">.tgz</span><span style="color:#79B8FF"> -C</span><span style="color:#9ECBFF"> staged</span></span>
<span class="line"><span style="color:#E1E4E8">(</span><span style="color:#79B8FF">cd</span><span style="color:#9ECBFF"> prev</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> pack</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$PKG</span><span style="color:#9ECBFF">@</span><span style="color:#E1E4E8">$PREV_VERSION</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8"> &amp;&amp; </span><span style="color:#B392F0">tar</span><span style="color:#79B8FF"> -xzf</span><span style="color:#9ECBFF"> ./</span><span style="color:#79B8FF">*</span><span style="color:#9ECBFF">.tgz</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#B392F0">diff</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> prev/package</span><span style="color:#9ECBFF"> staged/package</span><span style="color:#6A737D">       # install scripts, dependencies, binaries?</span></span>
<span class="line"><span style="color:#6A737D"># every check passed: approve with your own second factor</span></span>
<span class="line"><span style="color:#B392F0">npm</span><span style="color:#9ECBFF"> stage</span><span style="color:#9ECBFF"> approve</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$STAGE_ID</span><span style="color:#9ECBFF">"</span></span></code></pre>
<p>
<em>The amber path is the one that skips the human. The dashed boxes are the controls you add.</em></p>
<p>What the docs leave out matters here. npm’s staged-publishing docs do not say whether a staged version expires, and they document no reject or discard command. A version that fails your check doesn’t vanish; it sits in the list, where a tired approver could approve it next week. Record the failed stage ID in the ticket, alert on stage IDs older than a day, and treat an approval of a stage ID marked failed as an incident.</p>
<p>Keep the approval human. No lane holds a 2FA device or a TOTP seed, no script wraps <code>npm stage approve</code>, and when you have two maintainers, the approver is not the person whose session started the release.</p>
<h2 id="step-6-prefer-trusted-publishing-where-the-pipeline-supports-it">Step 6: Prefer trusted publishing where the pipeline supports it</h2>
<p>The changelog frames stage-only tokens as a bridge: “If you can’t move to trusted publishing yet, stage-only tokens offer a migration path for token-based automation.” <a href="https://docs.npmjs.com/trusted-publishers/">Trusted publishing</a> replaces the long-lived npm token with an identity the CI provider asserts for each run, so there is no token on disk to steal. npm’s staged-publishing docs say staging also works with trusted publishing (OIDC), which gives you both properties at once: no long-lived secret, and a person still approves every version.</p>
<table>
<thead>
<tr>
<th>Pipeline</th>
<th>Posture</th>
</tr>
</thead>
<tbody>
<tr>
<td>a CI provider npm’s trusted publishing supports</td>
<td>trusted publishing plus staging; no npm token exists anywhere</td>
</tr>
<tr>
<td>a supported provider, but the job also releases for humans</td>
<td>the same; approval is per version either way</td>
</tr>
<tr>
<td>a provider trusted publishing doesn’t support</td>
<td>stage-only token, shortest expiry, Steps 2 to 5 in full</td>
</tr>
<tr>
<td>a developer workstation</td>
<td>no automation token at all; the developer stages or publishes interactively with 2FA</td>
</tr>
</tbody>
</table>
<p>Check whether the Sep 3 multiple-configurations change covers the workflow layout that made you keep a shared token. And if you run trusted publishing without staging, whatever can start the publishing workflow can publish, so the trigger surface becomes the credential. That is where <a href="/intel/agent-actions-trigger-allowlist/">the GitHub Actions trigger allowlist for agent identities</a> earns its keep, because an agent that can’t publish can often still start the job that does.</p>
<h2 id="where-an-npm-stage-only-token-leaks-anyway-and-the-signal-for-each">Where an npm stage-only token leaks anyway, and the signal for each</h2>
<p><strong>The old token nobody revoked.</strong> Signal: a version in your namespace with no stage ID and no approval record behind it. Fix: find the token with the Step 4 sweep, revoke it, and rerun the Step 2 drill on every lane that could reach it.</p>
<p><strong>Residual rights, used.</strong> Signal: a Step 3 page for a <code>latest</code> move or a deprecation outside a release window. Response: restore the tag from a clean host, revoke the stage-only token, and read the lane’s session record before issuing a new one.</p>
<p><strong>The rubber-stamp approval.</strong> Signal: approvals landing seconds after staging, or release tickets with no hash recorded. Fix: an approval without the Step 5 record doesn’t count, and the release job refuses to proceed without it.</p>
<p><strong>Staged versions piling up.</strong> Signal: stage IDs older than a day in <code>npm stage list</code>. Response: record each one against its ticket. The docs do not say whether staged versions expire, so treat an old one as live.</p>
<p><strong>The silent fallback.</strong> Signal: a release that succeeded on a lane whose preflight failed. Fix: remove the old token from the lane image entirely; a lane without a publish token has nothing to fall back to.</p>
<p><strong>The upstream this runbook doesn’t touch.</strong> Signal: none from anything above. The Mandiant chain began with a poisoned dependency the assistant recommended, and stage-only tokens do nothing there. Mandiant’s controls for that link are checksum and allowlist verification of AI-recommended dependencies, plus routing dependency traffic through internal repositories. Hooks a package writes after install are <a href="/intel/unauthored-agent-hooks-persistence/">a persistence problem with their own runbook</a>, and a lane that reads untrusted input while holding stage rights is a candidate for <a href="/intel/rule-of-two-lane-split/">the Rule of Two lane split</a>.</p>
<h2 id="publish-rights-belong-in-the-fleet-inventory-not-a-ci-settings-page">Publish rights belong in the fleet inventory, not a CI settings page</h2>
<p>Every control above lives outside the model and outside the registry’s defaults: the token-class table, the register, the drill on a timer, the watcher on a host with no write rights, the ticket that carries a hash. That is the operating layer, the same place that already holds the kill switches and approval queues for the rest of the fleet; <a href="/intel/agentic-ops/">running a fleet of agents</a> is mostly the work of keeping that layer honest. The <a href="/intel/overnight-agent-merge-gates/">overnight merge gates</a> decide what an agent may land in your repository. The publish path decides what everyone else installs, and it deserves at least the same care.</p>
<p>Stage-only tokens are a real improvement, and I’d switch every token-based release lane to them this week. Just write down the two doors they leave open, and watch those doors from somewhere the token can’t reach.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-an-npm-stage-only-token-stop-a-hijacked-agent-from-publishing">Does an npm stage-only token stop a hijacked agent from publishing?</h3>
<p>It stops a direct <code>npm publish</code> with that token: npm rejects it even when the token is set to bypass 2FA, and the version waits until a maintainer approves with 2FA. The same token can still move dist-tags and deprecate versions, and older publish tokens in reach keep working.</p>
<h3 id="does-npm-staged-publishing-work-with-trusted-publishing">Does npm staged publishing work with trusted publishing?</h3>
<p>npm’s staged-publishing docs say it does, and the pairing is the strongest option for agent-driven releases. The pipeline authenticates per run through OIDC, so no long-lived npm token exists to steal, and every version still waits for a person’s 2FA approval before anyone can install it.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://github.blog/changelog/2026-09-18-stage-only-npm-tokens-for-safer-automation/">GitHub changelog: Stage-only npm tokens for safer automation</a> — Sep 18, 2026; the permission level, bypass-2FA rejection, residual rights, version floors, January 2027 target</li>
<li><a href="https://docs.npmjs.com/staged-publishing/">npm docs: Staged publishing</a> — <code>npm stage publish</code> needs no 2FA; list, view, download, approve; works with trusted publishing</li>
<li><a href="https://docs.npmjs.com/cli/v11/commands/npm-stage/">npm docs: npm stage command reference</a> — CLI reference page for <code>npm stage</code></li>
<li><a href="https://docs.npmjs.com/trusted-publishers/">npm docs: Trusted publishers</a> — the OIDC publishing path the changelog recommends</li>
<li><a href="https://github.blog/changelog/2026-05-22-staged-publishing-and-new-install-time-controls-for-npm/">GitHub changelog: Staged publishing and new install-time controls for npm</a> — May 22, 2026</li>
<li><a href="https://github.blog/changelog/2026-09-03-multiple-trusted-publishing-configurations-for-npm/">GitHub changelog: Multiple trusted publishing configurations for npm</a> — Sep 3, 2026</li>
<li><a href="https://github.com/orgs/community/discussions/208130">GitHub community: npm’s roadmap</a> — safer publishing roadmap linked from the Sep 18 changelog</li>
<li><a href="https://cloud.google.com/security/resources/ai-risk-and-resilience-2026">Mandiant AI Risk and Resilience Report 2026</a> — September 2026; case study 1, the PyPI entry vector and the credential-isolation control</li>
<li><a href="https://thehackernews.com/2026/09/attacker-hijacks-ai-coding-assistant.html">The Hacker News: Attacker Hijacks AI Coding Assistant Session, Spreads Shai-Hulud Across About 100 Repositories</a> — Sep 16, 2026</li>
</ul>
]]></content:encoded></item><item><title>LiteLLM MCP on the KEV List: A Patch Clock for Agent Gateways</title><link>https://automater.ai/intel/litellm-mcp-kev-patch-clock/</link><guid>https://automater.ai/intel/litellm-mcp-kev-patch-clock/</guid><description>The LiteLLM MCP vulnerability hit CISA KEV on Sep 2. Block /mcp/ at the proxy, start the clock at the advisory, and prove a forged Bearer now gets a 401.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category><content:encoded><![CDATA[<p>On Sep 2, CISA added CVE-2026-59822 to its Known Exploited Vulnerabilities catalog and gave federal agencies until Sep 16 to act. By that morning the fix had been in a LiteLLM release for 111 days and the vendor advisory had been public for 64. Wiz, the firm that found this LiteLLM MCP vulnerability, says its honeypots caught exploitation on Jul 7, eight weeks before the listing. A patch ticket opened on Sep 2 opened late.</p>
<p>The bug lets a made-up Bearer token through the MCP endpoint of a gateway that teams put in front of the tools their agents use. This piece is the runbook for it and for the next one. By Tuesday every agent gateway you run gets a clock that starts at the vendor advisory, a containment rule you can apply in minutes, a forged-token test that runs after every upgrade and must return 401, a log hunt across the whole exposure window, and a rotation list for everything the gateway fronted.</p>
<p>The history is why it deserves its own clock. This is LiteLLM’s third KEV entry of 2026, and the second on an MCP code path. A component that keeps landing on that list should not wait for CISA to tell it what day it is.</p>
<h2 id="sep-2-cisa-lists-cve-2026-59822-litellms-third-kev-entry-this-year">Sep 2: CISA lists CVE-2026-59822, LiteLLM’s third KEV entry this year</h2>
<p>The <a href="https://github.com/advisories/GHSA-7488-6r32-c95q">advisory</a> is GHSA-7488-6r32-c95q, an MCP authentication bypass through an OAuth2 passthrough fallback, rated High at 8.8 under CVSS 4.0. <a href="https://nvd.nist.gov/vuln/detail/CVE-2026-59822">NVD’s own CVSS 3.1 score</a> is 8.2, so don’t mix the two in a ticket. Every <code>litellm</code> release before 1.84.0 is affected.</p>
<p>When a request to the MCP Streamable HTTP endpoint carried an Authorization header that failed LiteLLM’s key check, a fallback built for OAuth2 passthrough swapped the failed validation for an empty <code>UserAPIKeyAuth()</code> object and let the request continue. The caller could then list and call whatever MCP tools the gateway had configured and reach the services behind them.</p>
<p>The advisory also carries the containment step this runbook opens with: when the upgrade can’t happen right away, switch off the MCP routes, or block <code>/mcp/</code> and the related MCP endpoints at whatever reverse proxy or API gateway sits in front of LiteLLM.</p>
<p>
<em>Screenshot: GitHub, “LiteLLM: MCP Authentication Bypass via OAuth2 Passthrough Fallback” (Jun 30, 2026), captured Sep 21, 2026.</em></p>
<p><a href="https://www.cisa.gov/known-exploited-vulnerabilities-catalog">CISA’s catalog entry</a> (added Sep 2, due Sep 16) describes the same flaw in one line, lists ransomware use as unknown, and tells agencies to apply the vendor’s mitigations under <a href="https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk">BOD 26-04</a>, Prioritizing Security Updates Based on Risk. The entry itself tells no exploitation story.</p>
<p>LiteLLM’s three KEV entries this year:</p>
<ul>
<li><strong>CVE-2026-42208</strong>, a SQL injection an unauthenticated caller could reach with a crafted Authorization header on any LLM API route. <a href="https://nvd.nist.gov/vuln/detail/CVE-2026-42208">NVD</a> and KEV both list it on May 8; the federal due date was May 11, three days later.</li>
<li><strong>CVE-2026-42271</strong>, command execution through two MCP server-preview endpoints, <code>POST /mcp-rest/test/connection</code> and <code>POST /mcp-rest/test/tools/list</code>. Per NVD and <a href="https://github.com/BerriAI/litellm/security/advisories/GHSA-v4p8-mg3p-g94g">its advisory</a> (Apr 21), they accepted a full stdio server config, command included, and checked only for a valid proxy key with no role check; CISA’s entry adds that even a low-privilege internal-user key could run commands on the host. Fixed in 1.83.7, which requires the <code>PROXY_ADMIN</code> role; added to KEV Jun 8, due Jun 22. On Sep 2 CISA also added a Starlette request-smuggling flaw, CVE-2026-48710, noting it could be chained with this one.</li>
<li><strong>CVE-2026-59822</strong>, above.</li>
</ul>
<p>Two of the three sit on MCP code paths, but not the same path. The June bug lives in the preview endpoints, the September bug in the live tool endpoint. That matters when you write the proxy rule.</p>
<aside class="social-embed" data-x-embed data-post-id="2095427951701692787">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 3, 2026 — FOFA's alert on CVE-2026-59822 the day after the KEV listing, with its own count of internet-facing LiteLLM surfaces.</p>
  <a class="social-embed__source" href="https://x.com/fofabot/status/2095427951701692787" target="_blank" rel="noopener noreferrer">View FOFA's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="an-mcp-gateway-bug-hands-out-tools-not-tokens">An MCP gateway bug hands out tools, not tokens</h2>
<p>A model gateway with an auth hole leaks prompts and burns your token budget. An MCP gateway with an auth hole hands a stranger your tools, and each tool holds a credential for something real: a repository, a ticket queue, a database.</p>
<p>Chatbots suggest; agents act, and the gateway is where you gathered the ability to act so you could govern it in one place. <a href="/intel/agent-gateway-control-plane/">The gateway as control plane</a> is still the right design. It also means one CVE in the control plane is a CVE in every connector at once. Neither that piece nor the <a href="/intel/mcp-security-hardening/">MCP hardening checklist</a> names a KEV due date or a negative-auth test, and those are what the day after a listing needs.</p>
<p>The <a href="https://github.com/BerriAI/litellm/pull/26463">fix PR</a> is blunter than the advisory. It says the fail-open, combined with MCP servers marked <code>allow_all_keys=True</code>, gave any unauthenticated caller full tool access on those servers. If you run LiteLLM, that flag is the first column of your blast-radius list.</p>
<h2 id="step-1-contain-today-if-the-upgrade-cant-land-today">Step 1: Contain today if the upgrade can’t land today</h2>
<p>Containment is two moves, and you want both before lunch on advisory day.</p>
<p><strong>Block the MCP routes at the proxy.</strong> The advisory names <code>/mcp/</code>; the June bug’s endpoints live under <code>/mcp-rest/test/</code>. Match on the path, never on the full URL. The fix PR closed a second bypass the advisory doesn’t mention: LiteLLM’s own public-route check looked for <code>.well-known</code> anywhere in the URL, so appending <code>?.well-known</code> to an MCP route skipped auth. A proxy rule that matches the path ignores query-string tricks.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="nginx"><code><span class="line"><span style="color:#6A737D"># Illustrative containment for LiteLLM below 1.84.0. Prefix match on the path only.</span></span>
<span class="line"><span style="color:#6A737D"># "/mcp" also covers /mcp/ and /mcp-rest/. Adjust if LiteLLM sits under a path prefix.</span></span>
<span class="line"><span style="color:#F97583">location</span><span style="color:#F97583"> ^~</span><span style="color:#DBEDFF"> /mcp </span><span style="color:#E1E4E8">{</span></span>
<span class="line"><span style="color:#F97583">    return</span><span style="color:#79B8FF"> 403</span><span style="color:#E1E4E8">;</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span></code></pre>
<p><strong>Take the endpoint off the internet.</strong> Bind the listener to a private interface, or narrow the security group to the addresses your agent hosts egress from. If remote agents need MCP, put the gateway behind your VPN rather than in front of the world.</p>
<p>Then look for the second door. A proxy rule guards only traffic that passes through it: a published container port, a Kubernetes Service on a node port, a second load balancer or someone’s port-forward goes straight to LiteLLM. List every path to the listener and close or test each one.</p>
<p>Containment has a cost. Every agent that reaches tools through that gateway loses them until the upgrade lands. Decide in advance who may accept that outage, and log the containment time; it is the first entry on your clock.</p>
<h2 id="step-2-start-the-patch-clock-at-the-vendor-advisory-not-the-kev-listing">Step 2: Start the patch clock at the vendor advisory, not the KEV listing</h2>
<p>Here is the 59822 clock with T0 at the advisory:</p>
<table>
<thead>
<tr>
<th>Date (2026)</th>
<th>Event</th>
<th>Source</th>
<th>Clock</th>
</tr>
</thead>
<tbody>
<tr>
<td>Apr 25 / Apr 30</td>
<td>Fix PR #26463 opened / merged</td>
<td>GitHub</td>
<td>T−66 / T−61</td>
</tr>
<tr>
<td>May 14</td>
<td>v1.84.0 released; notes list an MCP OAuth2 fallback fix</td>
<td>GitHub release</td>
<td>T−47</td>
</tr>
<tr>
<td>Jun 30</td>
<td>GHSA published to the LiteLLM repo</td>
<td>GitHub</td>
<td><strong>T0</strong></td>
</tr>
<tr>
<td>Jul 7</td>
<td>Exploitation seen in Wiz honeypots</td>
<td>Wiz (vendor)</td>
<td>T+7</td>
</tr>
<tr>
<td>Jul 8</td>
<td>NVD record published</td>
<td>NVD</td>
<td>T+8</td>
</tr>
<tr>
<td>Jul 22</td>
<td>Reviewed into the GitHub Advisory Database</td>
<td>GitHub</td>
<td>T+22</td>
</tr>
<tr>
<td>Sep 2</td>
<td>Added to KEV</td>
<td>CISA</td>
<td>T+64</td>
</tr>
<tr>
<td>Sep 16</td>
<td>Federal due date</td>
<td>CISA</td>
<td>T+78</td>
</tr>
</tbody>
</table>
<p>The advisory is the right T0 because it is the first moment the vendor says, in words a ticket can cite, that a version you run is exploitable and what to do about it. The <a href="https://github.com/BerriAI/litellm/releases/tag/v1.84.0">v1.84.0 release notes</a> were an earlier signal, a line about tightening public-route detection and OAuth2 fallback gating, and teams that read gateway release notes for auth wording got a 47-day head start.</p>
<p>KEV is the latest signal of all. By the listing, <a href="https://www.wiz.io/blog/off-guard-breaking-litellm-from-authentication-bypass-to-cloud-compromise">Wiz says</a> exploitation had been visible in its honeypots for 57 days. Its <a href="https://www.wiz.io/blog/ai-infrastructure-honeypot">honeypot write-up</a> carries the line to pin above the patch queue: “Patch on the assumption that the exploit is already in the wild.”</p>
<p>
<em>Three LiteLLM KEV entries in 2026. For CVE-2026-59822 the advisory came 64 days before KEV; the fixed release, 111.</em></p>
<p>Two things in that table trip teams up. First, one advisory has three publication dates: Jun 30 in the repo, Jul 8 at NVD, Jul 22 in the GitHub Advisory Database. Your dependency scanner keys on one of them, and it may not be the first. Check which, and never let the scanner’s alert date become your clock start.</p>
<p>Second, the discoverer’s timeline disagrees with GitHub on the release date. Wiz lists the fix as released Apr 25, which is the day the fix PR was opened; the release itself is dated May 14. Use the artifact’s date.</p>
<p>
<em>Screenshot: Wiz Blog, “Breaking LiteLLM: From Auth Bypass to Cloud Compromise” (Sep 9, 2026), captured Sep 21, 2026.</em></p>
<p>Write the clock down as policy before the next advisory. This is a starting table, and the numbers are ours, not CISA’s or the vendor’s:</p>
<table>
<thead>
<tr>
<th>Trigger on a gateway you run</th>
<th>Internet-reachable</th>
<th>Internal only</th>
</tr>
</thead>
<tbody>
<tr>
<td>Fixed release with auth or MCP wording, no advisory yet</td>
<td>Upgrade within 14 days</td>
<td>Next maintenance window</td>
</tr>
<tr>
<td>Vendor advisory: auth bypass or code execution on a route you expose</td>
<td>Contain same day; patch within 7 days</td>
<td>Patch within 14 days</td>
</tr>
<tr>
<td>Credible report of exploitation</td>
<td>Contain within the hour</td>
<td>Contain same day</td>
</tr>
<tr>
<td>KEV listing while still open</td>
<td>Incident: contain now, page the owner</td>
<td>Incident</td>
</tr>
</tbody>
</table>
<p>KEV due dates bind federal agencies (the September entry cites BOD 26-04), and the windows vary: 14, 14 and 3 days for LiteLLM’s three. For everyone else they are a backstop. Pull the feed daily, alert when a vendor you run appears, and compare its <code>dateAdded</code> with your own clock start. If the listing was your first alarm, the clock failed, and that is a finding for the retro.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># Daily backstop: KEV entries for a vendor you run (fields from CISA's KEV JSON feed)</span></span>
<span class="line"><span style="color:#B392F0">curl</span><span style="color:#79B8FF"> -s</span><span style="color:#9ECBFF"> https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#F97583">  |</span><span style="color:#B392F0"> jq</span><span style="color:#79B8FF"> -r</span><span style="color:#9ECBFF"> '.. | objects | select(.vendorProject? == "BerriAI")</span></span>
<span class="line"><span style="color:#9ECBFF">           | [.dateAdded, .dueDate, .vulnerabilityName] | @tsv'</span></span></code></pre>
<p>Keep one record per component so the clock survives a shift change:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># patch-clock.yaml (illustrative shape; owner, exposure and server names are yours)</span></span>
<span class="line"><span style="color:#E1E4E8">- </span><span style="color:#85E89D">component</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">litellm-proxy</span></span>
<span class="line"><span style="color:#85E89D">  owner</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">platform-gateway</span></span>
<span class="line"><span style="color:#85E89D">  exposure</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">internet</span><span style="color:#6A737D">              # internet | vpn | localhost</span></span>
<span class="line"><span style="color:#85E89D">  mcp_servers_allow_all_keys</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">tickets</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">warehouse-readonly</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  advisory</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">GHSA-7488-6r32-c95q</span></span>
<span class="line"><span style="color:#85E89D">  cve</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">CVE-2026-59822</span></span>
<span class="line"><span style="color:#85E89D">  clock_start</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2026-06-30</span><span style="color:#6A737D">         # vendor advisory, never the KEV date</span></span>
<span class="line"><span style="color:#85E89D">  fixed_in</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">1.84.0</span></span>
<span class="line"><span style="color:#85E89D">  contained_at</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">null</span></span>
<span class="line"><span style="color:#85E89D">  patched_at</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">null</span></span>
<span class="line"><span style="color:#85E89D">  verified_by</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">forged-bearer-test</span></span>
<span class="line"><span style="color:#85E89D">  kev_added</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2026-09-02</span><span style="color:#6A737D">           # backstop only</span></span></code></pre>
<h2 id="step-3-prove-the-litellm-mcp-vulnerability-is-closed-with-a-forged-bearer">Step 3: Prove the LiteLLM MCP vulnerability is closed with a forged Bearer</h2>
<p>A version string says what you deployed. It doesn’t say what answers on the port. After every upgrade, restart and config change, and once a day besides, send a well-formed MCP <code>initialize</code> request with a fabricated Bearer to every exposed route and transport. The pass condition is a 401, not just any 4xx.</p>
<p>A 403 may be your step 1 proxy rule, which says nothing about the gateway. A 404 means the route moved. A 500 means the auth path errored; the fix PR specifically stopped auth errors being masked as 500s, and a gateway that 500s on a forged token is one refactor away from a 200.</p>
<p>The vendor doesn’t say 401 is the correct answer. This is operator practice, and it’s cheap.</p>
<p>Run it in this order: straight at the gateway from inside, with the proxy block still up; then lift the block; then again through the proxy from every vantage point your agents use.</p>
<p>
<em>Two cuts: the proxy closes the route, the 1.84.0 fix closes the fallback. OAuth2 pass-through targets still need the upstream to refuse.</em></p>
<p>Use four token shapes per route: <code>Bearer x</code> (the single-character shape Wiz saw probing its honeypots, there on a models endpoint rather than <code>/mcp/</code>), a random 40-character string, an empty Bearer and no Authorization header at all. Repeat each with <code>?.well-known</code> appended.</p>
<p>Then one control: the same request with a scoped test key must succeed. Without the control, a gateway that is simply down passes every negative test. The v1.84.0 release also flags breaking changes, so the control catches an upgrade that broke your agents as well.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D">#!/usr/bin/env bash</span></span>
<span class="line"><span style="color:#6A737D"># forged-bearer.sh (illustrative). Non-zero exit on any failure; wire it into the deploy.</span></span>
<span class="line"><span style="color:#79B8FF">set</span><span style="color:#79B8FF"> -u</span></span>
<span class="line"><span style="color:#E1E4E8">GW</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"${</span><span style="color:#E1E4E8">GW</span><span style="color:#F97583">:?</span><span style="color:#E1E4E8">gateway</span><span style="color:#E1E4E8"> base</span><span style="color:#E1E4E8"> URL</span><span style="color:#9ECBFF">; </span><span style="color:#E1E4E8">one</span><span style="color:#E1E4E8"> run</span><span style="color:#E1E4E8"> per</span><span style="color:#E1E4E8"> vantage</span><span style="color:#E1E4E8"> point</span><span style="color:#9ECBFF">}"</span></span>
<span class="line"><span style="color:#E1E4E8">ROUTES</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">(</span><span style="color:#9ECBFF">"/mcp/"</span><span style="color:#9ECBFF"> "/mcp/?.well-known"</span><span style="color:#E1E4E8">)    </span><span style="color:#6A737D"># add every MCP route and transport your logs show</span></span>
<span class="line"><span style="color:#E1E4E8">BODY</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">'{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"&lt;your-spec-version&gt;","capabilities":{},"clientInfo":{"name":"forged-bearer-test","version":"1"}}}'</span></span>
<span class="line"><span style="color:#E1E4E8">fail</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">0</span></span>
<span class="line"><span style="color:#B392F0">probe</span><span style="color:#E1E4E8">() {  </span><span style="color:#6A737D"># $1 route, $2 Authorization header line ("" sends none)</span></span>
<span class="line"><span style="color:#B392F0">  curl</span><span style="color:#79B8FF"> -s</span><span style="color:#79B8FF"> -o</span><span style="color:#9ECBFF"> /dev/null</span><span style="color:#79B8FF"> -w</span><span style="color:#9ECBFF"> '%{http_code}'</span><span style="color:#79B8FF"> --max-time</span><span style="color:#79B8FF"> 10</span><span style="color:#79B8FF"> -X</span><span style="color:#9ECBFF"> POST</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$GW</span><span style="color:#79B8FF">$1</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#FFAB70">    ${2</span><span style="color:#F97583">:</span><span style="color:#E1E4E8">+-H </span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">$2</span><span style="color:#9ECBFF">"</span><span style="color:#FFAB70">}</span><span style="color:#79B8FF"> -H</span><span style="color:#9ECBFF"> 'Content-Type: application/json'</span><span style="color:#79B8FF"> \</span></span>
<span class="line"><span style="color:#79B8FF">    -H</span><span style="color:#9ECBFF"> 'Accept: application/json, text/event-stream'</span><span style="color:#79B8FF"> --data</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$BODY</span><span style="color:#9ECBFF">"</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span>
<span class="line"><span style="color:#E1E4E8">RAND</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$(</span><span style="color:#B392F0">head</span><span style="color:#79B8FF"> -c</span><span style="color:#79B8FF"> 64</span><span style="color:#9ECBFF"> /dev/urandom</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> base64</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> tr</span><span style="color:#79B8FF"> -dc</span><span style="color:#9ECBFF"> 'A-Za-z0-9'</span><span style="color:#F97583"> |</span><span style="color:#B392F0"> head</span><span style="color:#79B8FF"> -c</span><span style="color:#79B8FF"> 40</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">for</span><span style="color:#E1E4E8"> r </span><span style="color:#F97583">in</span><span style="color:#9ECBFF"> "${</span><span style="color:#E1E4E8">ROUTES</span><span style="color:#9ECBFF">[</span><span style="color:#F97583">@</span><span style="color:#9ECBFF">]}"</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#F97583">  for</span><span style="color:#E1E4E8"> auth </span><span style="color:#F97583">in</span><span style="color:#9ECBFF"> "Authorization: Bearer x"</span><span style="color:#9ECBFF"> "Authorization: Bearer </span><span style="color:#E1E4E8">$RAND</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "Authorization: Bearer"</span><span style="color:#9ECBFF"> ""</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">do</span></span>
<span class="line"><span style="color:#E1E4E8">    code</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$(</span><span style="color:#B392F0">probe</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$r</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$auth</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#E1E4E8">    [ </span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">$code</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> =</span><span style="color:#9ECBFF"> "401"</span><span style="color:#E1E4E8"> ] </span><span style="color:#F97583">||</span><span style="color:#E1E4E8"> { </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "FAIL </span><span style="color:#E1E4E8">$r</span><span style="color:#9ECBFF"> [</span><span style="color:#E1E4E8">$auth</span><span style="color:#9ECBFF">] -&gt; </span><span style="color:#E1E4E8">$code</span><span style="color:#9ECBFF">, want 401"</span><span style="color:#E1E4E8">; fail</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">1</span><span style="color:#E1E4E8">; }</span></span>
<span class="line"><span style="color:#F97583">  done</span></span>
<span class="line"><span style="color:#F97583">done</span></span>
<span class="line"><span style="color:#E1E4E8">code</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">$(</span><span style="color:#B392F0">probe</span><span style="color:#9ECBFF"> "/mcp/"</span><span style="color:#9ECBFF"> "Authorization: Bearer ${</span><span style="color:#E1E4E8">CONTROL_KEY</span><span style="color:#F97583">:?</span><span style="color:#E1E4E8">scoped</span><span style="color:#E1E4E8"> test</span><span style="color:#E1E4E8"> key</span><span style="color:#9ECBFF">}"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#E1E4E8">[ </span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">$code</span><span style="color:#9ECBFF">"</span><span style="color:#F97583"> =</span><span style="color:#9ECBFF"> "200"</span><span style="color:#E1E4E8"> ] </span><span style="color:#F97583">||</span><span style="color:#E1E4E8"> { </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "FAIL control -&gt; </span><span style="color:#E1E4E8">$code</span><span style="color:#9ECBFF">, want 200"</span><span style="color:#E1E4E8">; fail</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">1</span><span style="color:#E1E4E8">; }</span></span>
<span class="line"><span style="color:#79B8FF">exit</span><span style="color:#E1E4E8"> $fail</span></span></code></pre>
<p>Two limits on scope. The advisory names the Streamable HTTP endpoint and doesn’t say whether any other transport was affected, so test every transport and route prefix your deployment exposes.</p>
<p>Carve out OAuth2 pass-through: 1.84.0 still allows the empty-auth fallback when every target MCP server is configured <code>auth_type: oauth2</code>, by design, because the token is meant to reach the upstream. For those servers the gateway may accept the forged token; the test is whether the upstream refuses it. Make one read-only tool call with the forged token and assert that you get the upstream’s authorization error and no tool result. Mixed targets, unresolvable server names and empty target lists fail closed after the fix, so they belong in the 401 set.</p>
<p>Wire the script into the deploy, not into a wiki page. A non-zero exit blocks promotion and reopens the clock, and a timeout counts as a failure: a test that can’t reach the gateway has proved nothing.</p>
<p>The test is a check, not a wall. The wall is the patched version plus a listener that isn’t on the internet. The suite that <a href="/intel/tool-schema-guard-contract-tests/">contract-tests guard coverage on every harness bump</a> is the natural home for it.</p>
<h2 id="step-4-hunt-the-mcp-logs-across-the-whole-exposure-window">Step 4: Hunt the /mcp/ logs across the whole exposure window</h2>
<p>The window is not advisory-to-patch. It runs from the first day a pre-1.84.0 build served MCP on a reachable route to the moment step 3 first passed. The advisory gives no lower bound on affected versions, so for most teams the window starts the day MCP went live on that gateway.</p>
<table>
<thead>
<tr>
<th>Signal</th>
<th>Where to look</th>
<th>Why it matters</th>
</tr>
</thead>
<tbody>
<tr>
<td>2xx on an MCP route for a Bearer that matches no issued key</td>
<td>Proxy access log joined to your key inventory by hash</td>
<td>The bypass itself</td>
</tr>
<tr>
<td>One- or two-character Bearer values</td>
<td>Proxy or gateway log, if it records token length or a hash</td>
<td>The probe shape Wiz saw</td>
</tr>
<tr>
<td><code>.well-known</code> in the query string of an MCP route</td>
<td>Proxy access log</td>
<td>The second bypass closed by the same PR</td>
</tr>
<tr>
<td><code>tools/list</code> followed by <code>tools/call</code> from a source no agent host uses</td>
<td>Gateway request log</td>
<td>Enumeration, then use</td>
</tr>
<tr>
<td>500s on MCP auth paths</td>
<td>Gateway error log</td>
<td>Auth failures masked before the fix</td>
</tr>
<tr>
<td>Calls to <code>/mcp-rest/test/</code> from a non-admin key</td>
<td>Gateway request log</td>
<td>The June bug, if you ran 1.74.2 through 1.83.6</td>
</tr>
<tr>
<td>Actions by the gateway’s upstream credential with no matching agent session</td>
<td>Each upstream service’s own audit log</td>
<td>What the tools were used for</td>
</tr>
</tbody>
</table>
<p>The last row matters most and gets skipped most. Join every upstream audit event in the window to your agent session records by time and credential; an action no agent of yours requested is the finding.</p>
<p>Most proxies don’t log Authorization values at all, which is correct and also means the first two rows may come back empty. Write that down. If your logs don’t reach back to the start of the window, treat the window as compromised for step 5 rather than as clean.</p>
<p>If the hunt turns up tool calls with shell or file-write reach on agent hosts, stop treating this as a gateway incident. Assume something was left behind and run the <a href="/intel/unauthored-agent-hooks-persistence/">eradication proof for unauthored hooks</a> on every host those tools touched.</p>
<h2 id="step-5-rotate-everything-the-gateway-fronted-after-the-fix-is-proven">Step 5: Rotate everything the gateway fronted, after the fix is proven</h2>
<p>Order matters. Prove the fix (step 3), rotate, revoke the old credential, then prove the old one fails. Rotating before the fix hands the new credential to the same hole.</p>
<table>
<thead>
<tr>
<th>Credential</th>
<th>Rotate when</th>
<th>Proof the old one is dead</th>
</tr>
</thead>
<tbody>
<tr>
<td>Upstream tokens and keys the gateway holds for MCP servers (repo, tickets, chat, database, cloud)</td>
<td>Always, for anything reachable in the window</td>
<td>A call with the old value gets the upstream’s auth error</td>
</tr>
<tr>
<td>Credentials behind MCP servers marked <code>allow_all_keys=True</code></td>
<td>Always, first</td>
<td>Same as above</td>
</tr>
<tr>
<td>LiteLLM keys issued to agents and teams</td>
<td>When the hunt shows use you can’t attribute, or logs can’t reach back</td>
<td>A request with the old key gets a 401</td>
</tr>
<tr>
<td>The LiteLLM master key</td>
<td>Now, if it is still the default</td>
<td>An admin call with the old key fails</td>
</tr>
</tbody>
</table>
<p>Wiz reports that 9.6% of the 3,074 public LiteLLM instances it examined accepted the default master key or no auth at all, so the last row is not hypothetical. How a gateway should hold upstream secrets in the first place is credential brokering, its own discipline and a different article.</p>
<h2 id="step-6-hand-census-floors-and-asserts-to-the-fleet-patch-drill">Step 6: Hand census, floors and asserts to the fleet patch drill</h2>
<p>Everything else is a drill you may already run. Finding every LiteLLM instance, including the one a team runs in a container on a laptop; setting version floors (1.84.0 for 59822, 1.83.7 for 42271); asserting after deploy that the running version is the one you meant: that is the <a href="/intel/plugin4shell-pin-verify-fleet-patch/">Plugin4Shell pin-and-verify drill</a>, run with LiteLLM rows. For gateways nobody registered, start with the <a href="/intel/shadow-mcp/">shadow MCP sweep</a>. The only columns this piece adds are the clock start and the date step 3 last passed.</p>
<h2 id="where-a-litellm-mcp-vulnerability-patch-clock-goes-wrong">Where a LiteLLM MCP vulnerability patch clock goes wrong</h2>
<p><strong>The second door.</strong> The proxy blocks <code>/mcp/</code>, and a node port serves it anyway. Signal: the forged-Bearer test from inside the agent subnet reaches LiteLLM directly and gets a 200.</p>
<p><strong>The half-rolled deploy.</strong> New image, old replica. Signal: run the test 20 times through the load balancer and get a mix of 401s and 200s. Any mix is a failure.</p>
<p><strong>The 404 that passed.</strong> Someone moved the route and a loosely written negative test went green. Signal: the control request fails too, which is why the control exists. Only a 401 beside a successful control is a pass.</p>
<p><strong>The OAuth2 blind spot.</strong> Every target server is <code>auth_type: oauth2</code>, the gateway lets the forged token through by design, and nobody checks the upstream. Signal: the upstream’s own log shows the forged token and something other than a refusal.</p>
<p><strong>The late clock.</strong> The ticket’s creation date is the KEV date. Signal: for 59822, a ticket opened after Jul 7 was opened after Wiz saw exploitation, and one opened on Sep 2 was opened by CISA, not by you.</p>
<p><strong>Rotation in the wrong order.</strong> New keys went out while the build was still vulnerable. Signal: the rotation timestamp precedes the first passing step 3 run. Rotate again.</p>
<h2 id="agent-gateways-are-fleet-infrastructure-so-the-patch-clock-belongs-to-the-fleet">Agent gateways are fleet infrastructure, so the patch clock belongs to the fleet</h2>
<p>An internet-reachable gateway that fronts private data and write-capable tools holds all three legs of the <a href="/intel/rule-of-two-lane-split/">Rule of Two</a> by construction: untrusted input, sensitive systems, and the ability to change state. You can’t split that lane without giving up the gateway, so you run it with a clock, a test and a hunt instead.</p>
<p>None of those live in the model or the harness. They live in the layer that runs the fleet: the inventory that knows which gateways exist, the policy that says who may accept a containment outage, the deploy gate that runs the forged-Bearer test, and the record that says when the clock started. That layer is what a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> is once you strip the dashboard off it.</p>
<p>LiteLLM will not be the last gateway on KEV. Start the next clock at the advisory.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-litellm-1840-fully-fix-cve-2026-59822">Does LiteLLM 1.84.0 fully fix CVE-2026-59822?</h3>
<p>The advisory and NVD name 1.84.0 as the patched release, and its fix PR also closed a <code>?.well-known</code> query-string bypass. By design it still allows the empty-auth fallback when every target MCP server is configured for OAuth2 passthrough, so for those servers, test that the upstream rejects a forged token.</p>
<h3 id="is-blocking-mcp-at-the-reverse-proxy-enough-to-mitigate-the-litellm-mcp-vulnerability">Is blocking /mcp/ at the reverse proxy enough to mitigate the LiteLLM MCP vulnerability?</h3>
<p>It is the advisory’s own workaround, and it covers only traffic that passes through that proxy. A published container port, a node port, a second load balancer or a port-forward goes around it. Treat the block as containment until the upgrade lands, then prove the fix with a forged-Bearer test that returns 401.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://github.com/advisories/GHSA-7488-6r32-c95q">GitHub advisory GHSA-7488-6r32-c95q</a> — repo advisory Jun 30, 2026; workaround</li>
<li><a href="https://nvd.nist.gov/vuln/detail/CVE-2026-59822">NVD: CVE-2026-59822</a> — Jul 8, 2026; CVSS 3.1 8.2</li>
<li><a href="https://www.cisa.gov/known-exploited-vulnerabilities-catalog">CISA Known Exploited Vulnerabilities Catalog</a> — LiteLLM entries May 8, Jun 8, Sep 2</li>
<li><a href="https://www.cisa.gov/news-events/directives/bod-26-04-prioritizing-security-updates-based-risk">CISA: BOD 26-04</a> — cited by the KEV entry</li>
<li><a href="https://github.com/BerriAI/litellm/releases/tag/v1.84.0">BerriAI/litellm release v1.84.0</a> — May 14, 2026</li>
<li><a href="https://github.com/BerriAI/litellm/pull/26463">BerriAI/litellm PR #26463</a> — the fix, incl. the <code>?.well-known</code> bypass</li>
<li><a href="https://github.com/BerriAI/litellm/security/advisories/GHSA-v4p8-mg3p-g94g">GHSA-v4p8-mg3p-g94g</a> — CVE-2026-42271; Apr 21, 2026</li>
<li><a href="https://nvd.nist.gov/vuln/detail/CVE-2026-42208">NVD: CVE-2026-42208</a> — SQL injection; May 8, 2026</li>
<li><a href="https://www.wiz.io/blog/off-guard-breaking-litellm-from-authentication-bypass-to-cloud-compromise">Wiz: Breaking LiteLLM</a> — discoverer’s timeline; Sep 9, 2026 (vendor)</li>
<li><a href="https://www.wiz.io/blog/ai-infrastructure-honeypot">Wiz: AI infrastructure honeypot</a> — Aug 27, 2026 (vendor)</li>
</ul>
]]></content:encoded></item><item><title>Test What Survives a Cross-CLI Session Import</title><link>https://automater.ai/intel/session-import-fidelity-test/</link><guid>https://automater.ai/intel/session-import-fidelity-test/</guid><description>Import Claude Code session history into another tool only after a fidelity test: mark turns, tool calls, approvals, branch and model carried or dropped.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>The Claude Code lane stops halfway through a bug with its weekly limit spent, and since Sep 14 there is a button for that moment. Cline Desktop will import a Claude Code session, or a Codex one, and carry it on with whatever provider and model you have configured in Cline. Its pages don’t say which parts of the session make the trip.</p>
<p>This runbook turns that silence into a test you run before import becomes your quota failover. By Tuesday you will have two fixture sessions on a throwaway repo, a scorecard that marks every item carried, summarized or dropped, a secret scan that runs before any transcript moves, one probe question aimed at the biggest gap, and a merge rule that names the model that finished the work.</p>
<h2 id="sep-14-cline-desktop-imports-claude-code-and-codex-sessions">Sep 14: Cline Desktop imports Claude Code and Codex sessions</h2>
<p>Cline launched <a href="https://cline.bot/blog/cline-desktop-an-open-source-app-for-open-weight-models">Cline Desktop</a> on Sep 14, 2026, in a post by Etisha Garg that pitches it as an open-source app for open-weight models. The feature that matters to anyone running a fleet is import: bring in a conversation from Claude Code, Codex or another supported agent, then “review what’s already been discussed, and continue with your next instruction”. The post is candid about the use case: “if you’ve been working through a bug in Claude Code where your subscription quota’s up, you can import that conversation into Cline” and finish the job with a cheaper open-weight model.</p>
<p>
<em>Screenshot: Cline, “Cline Desktop: An open-source app for open-weight models” (Sep 14, 2026), captured Sep 21, 2026.</em></p>
<p>The <a href="https://docs.cline.bot/usage/cline-desktop">Cline Desktop docs</a> give the path: open the Import section in the app’s settings, click Import Sessions, and choose the conversation. One sentence describes the result: “The imported conversation gives Cline the earlier context, while the continued task uses the provider and model you have configured in Cline.” The <a href="https://cline.bot/desktop">download page</a> labels the app beta on macOS and Windows.</p>
<p>
<em>Screenshot: Cline Docs, “Cline Desktop - Cline” (undated), captured Sep 21, 2026.</em></p>
<p>That sentence is the whole specification. Neither page lists what the import carries or drops: tool calls and their results, images and attachments, compaction summaries, pending or denied approvals, branch and worktree state, the original model’s identity, timestamps, or what happens to a secret sitting in the transcript. None of that is evidence of loss, only a reason to test. The launch also landed in the week Claude Code’s temporary limit boost ended, which <a href="/intel/claude-code-after-september-14/">Claude Code After September 14</a> prices as a 17% cut, so the quota-failover pitch has an audience ready.</p>
<aside class="social-embed" data-x-embed data-post-id="2099536235350086029">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 14, 2026 — Cline's launch post for Cline Desktop, a native app for open-weight models that works with ClinePass, free models or your own API keys.</p>
  <a class="social-embed__source" href="https://x.com/cline/status/2099536235350086029" target="_blank" rel="noopener noreferrer">View Cline's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="a-coding-agents-session-is-state-not-a-conversation">A coding agent’s session is state, not a conversation</h2>
<p>Chatbots suggest; agents act. In a chat tool the conversation is the product, and an import that keeps the words keeps nearly everything. A coding agent’s session is closer to a working directory with a diary attached.</p>
<p>It holds what the agent read and what those files said at the time, the commands it ran and which of them failed, the approvals you granted and the ones you refused, the branch or worktree the edits live on, and which model made which change. Some of that sits in the transcript as structure rather than prose. Some of it, like the checkout, isn’t in the transcript at all.</p>
<p>An importer can render each of those, flatten it into text, summarize it or skip it, and from inside the continued session all four look alike: a conversation that seems complete. The model on the other side acts on what it was handed.</p>
<p>If the refusal didn’t make the trip, it may propose the command you refused. If the branch didn’t, it edits whatever is checked out. Finding out on a fixture costs an hour. Finding out on the real lane costs a revert, usually at the hour the quota ran out.</p>
<h2 id="step-1-build-two-fixture-sessions-that-produce-every-item-class">Step 1: Build two fixture sessions that produce every item class</h2>
<p>Never run the first test on a real session. Make a disposable repo with a few files and one failing test, and drive two short sessions against it to completion, one in Claude Code and one in Codex. Each session exists to produce every item class the scorecard asks about, each tagged with a marker string you can search for later. The markers make the comparison mechanical: a search either finds FX-TOOL-3 in the imported view or it doesn’t.</p>
<table>
<thead>
<tr>
<th>Item class</th>
<th>How the fixture produces it</th>
<th>Marker</th>
</tr>
</thead>
<tbody>
<tr>
<td>User and assistant turns</td>
<td>Put FX-TURN-1 in a prompt; ask the agent to answer with FX-TURN-2</td>
<td>FX-TURN-1, FX-TURN-2</td>
</tr>
<tr>
<td>Tool calls and results</td>
<td>The agent reads <code>fixture/notes.txt</code> (its only line is FX-TOOL-3) and runs the failing test</td>
<td>FX-TOOL-3</td>
</tr>
<tr>
<td>Images and attachments</td>
<td>Attach a screenshot whose visible text is FX-IMG-4</td>
<td>FX-IMG-4</td>
</tr>
<tr>
<td>Compaction summary</td>
<td>Claude Code: <code>/compact</code> with an instruction to mention FX-SUM-5. Codex: set <code>model_auto_compact_token_limit</code> low so compaction fires early (the key can only lower the default trigger of 90% of the window)</td>
<td>FX-SUM-5</td>
</tr>
<tr>
<td>Denied approval</td>
<td>Ask for a command that prompts, such as deleting <code>fixture/</code>; deny it, then give your reason in the next message with FX-DENY-6</td>
<td>FX-DENY-6</td>
</tr>
<tr>
<td>Pending approval (optional third run)</td>
<td>Quit while a permission prompt is still open, with FX-PEND-7 in the request that triggered it</td>
<td>FX-PEND-7</td>
</tr>
<tr>
<td>Branch and working tree</td>
<td>Claude Code: <code>claude --worktree fx-a</code>. Codex: <code>--worktree</code> with the experimental <code>worktrees</code> feature on</td>
<td>Branch name, HEAD SHA</td>
</tr>
<tr>
<td>Model identity</td>
<td>Note the model and CLI version each session ran</td>
<td>Fixture log</td>
</tr>
<tr>
<td>Secret</td>
<td>A fake credential in a format your scanner flags, in <code>fixture/.env</code>, read by the agent</td>
<td>FX-CANARY-8</td>
</tr>
</tbody>
</table>
<p>The worktree runs are deliberate. Claude Code’s <a href="https://code.claude.com/docs/en/worktrees"><code>--worktree</code> flag</a> puts the session in <code>.claude/worktrees/fx-a/</code> on a branch named <code>worktree-fx-a</code>, and Codex’s experimental worktrees, new in <a href="https://github.com/openai/codex/releases/tag/rust-v0.154.0">0.154.0</a> on Sep 9, check out a detached HEAD bound to the session’s thread. On Claude Code a native resume returns a session to its worktree; whether an import does is exactly what the branch row tests.</p>
<p>Compaction needs one more line. If you run the summary lint from <a href="/intel/compaction-summary-lint/">Lint the Compaction Summary Before the Lane Acts on It</a>, Claude Code’s <code>PostCompact</code> hook already saves the source summary, which it receives as <code>compact_summary</code> (<a href="https://code.claude.com/docs/en/hooks">hooks reference</a>), so you have its exact text to compare. On Codex with the OpenAI provider the compaction result is an encrypted item, so there is no readable source summary. That row reads against the turns Codex kept.</p>
<p>Keep a fixture log as you go: each marker and the turn it appeared in, what you denied and why, the branch and HEAD at the end, the model and CLI version each session ran. The log is the answer key. Without it, step 4 turns into two people squinting at two screens and agreeing they look similar.</p>
<p>End both sessions finished: last turn answered, no tool call in flight, work committed on the fixture branch. The optional third run, killed at an open permission prompt, tests the case quota failover actually produces, because limits don’t wait for a tidy stopping point.</p>
<h2 id="step-2-scan-the-source-transcript-for-secrets-before-import">Step 2: Scan the source transcript for secrets before import</h2>
<p>An import is a data transfer. Per Cline’s docs, the continued task runs on the provider and model configured in Cline, so anything the source session read, tool output included, can reach that provider as context. That is a new last hop for data you had scoped to a Claude or Codex lane, the question <a href="/intel/vendor-lane-last-hop-register/">Your Vendor Lane May Not Be the Last Hop</a> asks of every lane. Scan before the transcript moves.</p>
<p>Find the session’s files on disk. The importer’s docs don’t say which files it reads, so scan everything the source CLI keeps for that session. Use the scanner you already trust on commits, and point it at two places commit scanners rarely look: tool results that echo an environment file or a config dump, and text you pasted into a prompt.</p>
<ol>
<li><strong>Prove the scanner on the fixture.</strong> It must flag FX-CANARY-8 in the source files. If it misses the canary, it will miss a real key the same way, so fix the scanner before anything else.</li>
<li><strong>Search the imported view for the canary</strong> once step 3 is done. If it made the trip, assume real secrets do too.</li>
<li><strong>On a real session with a real hit, don’t import.</strong> Rotate the credential, since it already sits in a transcript on disk, and hand the continuing tool a written handoff instead.</li>
<li><strong>Log the result</strong> in the fidelity record with the session ID.</li>
</ol>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># pre-import.sh (illustrative): run before anyone opens the importer.</span></span>
<span class="line"><span style="color:#6A737D"># secret-scan stands for the scanner you already use on commits.</span></span>
<span class="line"><span style="color:#E1E4E8">session_files</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF">$1</span><span style="color:#9ECBFF">"</span><span style="color:#6A737D">      # everything the source CLI keeps for this session</span></span>
<span class="line"><span style="color:#E1E4E8">record</span><span style="color:#F97583">=</span><span style="color:#9ECBFF">".fidelity/$(</span><span style="color:#B392F0">date</span><span style="color:#79B8FF"> -u</span><span style="color:#9ECBFF"> +%Y%m%dT%H%M%SZ)-scan.json"</span></span>
<span class="line"><span style="color:#F97583">if</span><span style="color:#F97583"> !</span><span style="color:#B392F0"> secret-scan</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$session_files</span><span style="color:#9ECBFF">"</span><span style="color:#79B8FF"> --report</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$record</span><span style="color:#9ECBFF">"</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">then</span></span>
<span class="line"><span style="color:#79B8FF">  echo</span><span style="color:#9ECBFF"> "secret found: rotate it and hand off with a written note; do not import"</span><span style="color:#F97583"> &gt;&amp;2</span></span>
<span class="line"><span style="color:#79B8FF">  exit</span><span style="color:#79B8FF"> 1</span></span>
<span class="line"><span style="color:#F97583">fi</span></span>
<span class="line"><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "scan clean; record at </span><span style="color:#E1E4E8">$record</span><span style="color:#9ECBFF">"</span></span></code></pre>
<p>On the fixture this script must fail, because the canary is doing its job. On a real session it must pass before anyone clicks Import Sessions.</p>
<p>Be honest about what that gate is. It’s a script you run before you click, and nothing in the importer calls it: a scanner that crashes exits non-zero and blocks, but a script nobody runs blocks nothing. The wall behind it is procedural: the runner, not whoever hit the quota, owns the decision to import, and lanes that read secrets are marked import-never in the lane manifest.</p>
<h2 id="step-3-import-the-claude-code-session-and-the-codex-session-then-freeze-both-views">Step 3: Import the Claude Code session and the Codex session, then freeze both views</h2>
<p>Import each fixture through Settings, Import, Import Sessions. Before you type a word into either imported session, capture the view exactly as it arrived: screenshots of the full scroll, and an export if the app offers one (the docs don’t mention one). Record the Cline Desktop version and platform, the provider and model configured for the continued task, and the import time in UTC.</p>
<p>Freeze first: your first instruction changes the view, and the probe in step 5 has to reach the session as imported.</p>
<p>Point the continuing tool at a clean clone of the fixture repo on its default branch, not at the source worktree. That’s the realistic failover: the quota ran out, you opened another app, and you’re wherever it put you. Whether the import finds its way back to the fixture branch is one of the rows.</p>
<p>
<em>The scan gates what leaves the source. The diff, the probe and the merge gate decide what the imported session may do.</em></p>
<h2 id="step-4-compare-source-and-imported-view-item-by-item">Step 4: Compare source and imported view, item by item</h2>
<p>Walk the fixture log row by row and search the imported view for each marker. Define the three marks before you look, so the scoring doesn’t drift to fit what you find:</p>
<ul>
<li><strong>Carried:</strong> the item is there with its structure. The marker appears verbatim, a tool call shows as a call with its result attached, a denial shows as a denial with its reason.</li>
<li><strong>Summarized:</strong> the gist is there but not the item. The marker is missing or paraphrased, a tool result is described instead of shown, several turns are collapsed into prose.</li>
<li><strong>Dropped:</strong> no trace.</li>
</ul>
<p>
<em>Left column: what Cline’s blog and docs state, read Sep 21, 2026. Right: the illustrative scorecard template, empty until you run your own fixtures.</em></p>
<table>
<thead>
<tr>
<th>Item</th>
<th>Look for in the imported view</th>
<th>If it isn’t carried</th>
</tr>
</thead>
<tbody>
<tr>
<td>Turns</td>
<td>FX-TURN-1 and FX-TURN-2, in order, with the right speaker</td>
<td>Restate the task and decisions in the first instruction</td>
</tr>
<tr>
<td>Tool calls and results</td>
<td>The read of <code>notes.txt</code> as a call, with FX-TOOL-3 as its result; the failing test’s output</td>
<td>The new model re-reads files and re-runs tests before any edit</td>
</tr>
<tr>
<td>Images and attachments</td>
<td>The screenshot, and whether the new model can read FX-IMG-4 from it</td>
<td>Re-attach it, or describe it in the first instruction</td>
</tr>
<tr>
<td>Compaction summary</td>
<td>FX-SUM-5, labelled as a summary rather than merged into turns</td>
<td>Lint it as a claim, or rebuild context from the source transcript</td>
</tr>
<tr>
<td>Denied approval</td>
<td>The refusal and FX-DENY-6</td>
<td>Re-create the denial as a deny rule in the continuing tool</td>
</tr>
<tr>
<td>Pending approval</td>
<td>The open call shown as unresolved, not as done</td>
<td>Treat it as denied until a human re-approves</td>
</tr>
<tr>
<td>Branch and working tree</td>
<td>Same branch or SHA at the first shell command</td>
<td>The runner checks out the source branch first</td>
</tr>
<tr>
<td>Source model identity</td>
<td>The earlier turns attributed to the model in the log</td>
<td>Record it in the fidelity record and the merge trailer</td>
</tr>
<tr>
<td>Canary</td>
<td>FX-CANARY-8 present or absent</td>
<td>Present means real secrets travel, so step 2 is mandatory</td>
</tr>
</tbody>
</table>
<p>Two rows need more than a mark. For compaction, note whether the imported view presents the summary as a summary or blends it into the conversation as though those turns happened. A blended summary is a claim the new model treats as memory, which is the problem the compaction lint exists for.</p>
<p>For branch state, check at the continued session’s first shell command, never in the interface: a label naming the right branch proves nothing about the checkout the agent is standing in. <code>git branch --show-current</code> and <code>git rev-parse HEAD</code> are the whole check.</p>
<p>Write the result into a record you keep beside the lane manifest, one file per run:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># .fidelity/2026-09-22-claude-code.yaml (illustrative shape; field names are ours)</span></span>
<span class="line"><span style="color:#85E89D">source</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  cli</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">claude-code</span></span>
<span class="line"><span style="color:#85E89D">  cli_version</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;from the fixture log&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  model</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;from the fixture log&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  session</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;session id&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  checkout</span><span style="color:#E1E4E8">: { </span><span style="color:#85E89D">branch</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">worktree-fx-a</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">head</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;sha&gt;"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">importer</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  app</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">cline-desktop</span></span>
<span class="line"><span style="color:#85E89D">  version</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;as shown in the app&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  platform</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">windows</span></span>
<span class="line"><span style="color:#85E89D">continued_with</span><span style="color:#E1E4E8">: { </span><span style="color:#85E89D">provider</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;configured in Cline&gt;"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">model</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;configured in Cline&gt;"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">secret_scan</span><span style="color:#E1E4E8">: { </span><span style="color:#85E89D">canary_flagged</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">true</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">real_hits</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">0</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">items</span><span style="color:#E1E4E8">:                          </span><span style="color:#6A737D"># carried | summarized | dropped | n/a</span></span>
<span class="line"><span style="color:#85E89D">  turns</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  tool_calls_and_results</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  images</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  compaction_summary</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  denied_approval</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  pending_approval</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  branch_state</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  source_model_identity</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">  canary</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;mark&gt;"</span></span>
<span class="line"><span style="color:#85E89D">probe</span><span style="color:#E1E4E8">: { </span><span style="color:#85E89D">row</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;row&gt;"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">answer</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;verbatim&gt;"</span><span style="color:#E1E4E8">, </span><span style="color:#85E89D">result</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;correct | honest_gap | confabulated&gt;"</span><span style="color:#E1E4E8"> }</span></span>
<span class="line"><span style="color:#85E89D">allowed_use</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"&lt;quota_failover | context_only | start_fresh&gt;"</span></span></code></pre>
<h2 id="step-5-ask-one-question-only-a-dropped-item-could-answer">Step 5: Ask one question only a dropped item could answer</h2>
<p>The scorecard says what the view shows. The probe says what the model believes. Take the highest-risk row you marked summarized or dropped and, before any instruction that can write, ask the continued session one question whose answer lives only in that item.</p>
<table>
<thead>
<tr>
<th>Row you marked dropped or summarized</th>
<th>Probe (ask exactly one)</th>
<th>Answer key</th>
</tr>
</thead>
<tbody>
<tr>
<td>Denied approval</td>
<td><em>Earlier I refused one command. Which one, and why?</em></td>
<td>The denied command and FX-DENY-6</td>
</tr>
<tr>
<td>Tool call result</td>
<td><em>What exactly did fixture/notes.txt contain?</em></td>
<td>FX-TOOL-3</td>
</tr>
<tr>
<td>Branch state</td>
<td><em>Which branch are you on, and is it the one the earlier work used?</em></td>
<td><code>worktree-fx-a</code>, or the Codex SHA</td>
</tr>
<tr>
<td>Compaction summary</td>
<td><em>List what happened before the summary, in order.</em></td>
<td>The pre-compaction turns in the log</td>
</tr>
<tr>
<td>Source model</td>
<td><em>Which model wrote the earlier changes?</em></td>
<td>The model in the log</td>
</tr>
</tbody>
</table>
<p>Score the answer three ways:</p>
<ul>
<li><strong>Correct.</strong> The item reached the model somewhere, perhaps inside a summary. Move the row up one mark and note where the answer came from.</li>
<li><strong>Honest gap.</strong> The model says it doesn’t have that information. The row stays dropped, and at least the session isn’t guessing.</li>
<li><strong>Confabulated.</strong> A confident, specific, wrong answer. This result matters most, because it’s how the continued session will behave on real work: fill the hole and act.</li>
</ul>
<p>Ask one question, not a quiz. Every extra question leaks part of the answer key into the session and teaches it what you’re looking for. To probe a second row, import again and ask a fresh copy.</p>
<h2 id="step-6-decide-what-import-may-be-used-for-lane-by-lane">Step 6: Decide what import may be used for, lane by lane</h2>
<p>Now turn the scorecard into policy. A lane’s allowed use depends on the rows that lane needs, not on an overall grade.</p>
<table>
<thead>
<tr>
<th>Allowed use</th>
<th>When</th>
<th>On failover</th>
</tr>
</thead>
<tbody>
<tr>
<td>Quota failover: continue the task</td>
<td>Turns, both approval rows, branch state and source model carried; tool results carried or covered by a re-read step; probe correct or an honest gap</td>
<td>Scan, import, re-create denials as rules, confirm the checkout, continue; step 7 before merge</td>
</tr>
<tr>
<td>Context only: review, explain, plan</td>
<td>Turns carried; other rows missing; probe not confabulated</td>
<td>Import for reading; no writes until the new model has re-read the files and re-run the tests</td>
</tr>
<tr>
<td>Start fresh</td>
<td>Probe confabulated, or the lane touches secrets and the canary travelled</td>
<td>Written handoff from the source transcript into a new session</td>
</tr>
</tbody>
</table>
<p>Two rules sit on top of the table. First, a refusal carried in the import is still only text the new model has read. Re-create every denied command as a deny rule or a sandbox limit in the continuing tool, because a sentence in the context is advice and a rule is a wall, and even the rule only covers the commands you thought to list.</p>
<p>Second, wherever the answer is start fresh, the failover you want is the one <a href="/intel/openai-agents-api-continuity/">the OpenAI Agents API continuity drill</a> builds: resume from state you keep yourself. Import sits on top of that plan. It doesn’t replace it.</p>
<p>Write the allowed use into each lane’s manifest beside the importer version it was tested on. No record means start fresh.</p>
<h2 id="step-7-re-verify-before-merge-and-mark-which-model-finished-the-work">Step 7: Re-verify before merge and mark which model finished the work</h2>
<p>Work that crossed an import has two authors and a seam, and review should treat it that way.</p>
<ol>
<li><strong>Diff from the import point.</strong> The fixture log has the source’s last commit. Everything after it is the continuing model’s work, reviewed as a whole rather than as a continuation of something already looked at.</li>
<li><strong>Re-run the tests from a clean checkout of the branch.</strong> The continuing model saying the tests pass is a claim.</li>
<li><strong>Check its tool calls against the source’s refusals.</strong> A match with any denied command fails the review, whatever the outcome.</li>
<li><strong>Confirm the branch.</strong> The PR’s head is the branch the source session worked on, or a deliberate new one with a note saying why.</li>
<li><strong>Name the finishing model</strong> in commit trailers and the PR description.</li>
</ol>
<p>Illustrative trailer names; use whatever your repo already parses:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="text"><code><span class="line"><span>Session-Source: claude-code &lt;cli version&gt;, &lt;model&gt;, session &lt;id&gt;</span></span>
<span class="line"><span>Session-Import: cline-desktop &lt;version&gt;, &lt;UTC timestamp&gt;</span></span>
<span class="line"><span>Finished-By: &lt;provider&gt;/&lt;model configured in Cline&gt;</span></span></code></pre>
<p>The trailers answer the question a diff can’t: which model shipped this change. <a href="/intel/fleet-continuity-provider-cutoff/">Rehearse the Provider Cutoff</a> covers the handoff state and provenance any failover should carry. The trailer is the import-specific line of it.</p>
<p>Re-run the whole test when either end changes: a Cline Desktop update, a Claude Code or Codex upgrade, a new provider on the continuing side. Codex alone changed worktrees in 0.154.0 and compaction in <a href="https://github.com/openai/codex/releases/tag/rust-v0.155.0">0.155.0</a>, which switched to always-streamed remote compaction on Sep 17, and a change to the session files a CLI writes can change what an importer sees. <a href="/intel/cli-upgrade-canary-drift/">Canary Every CLI Upgrade</a> is the place to hang it: add the import fixtures to the canned sessions it replays.</p>
<p>Keep the source session until the continued work merges, too. If the import proves lossy after the fact, the source is the only complete record, and automatic cleanup can remove it first. <a href="/intel/agent-session-cleanup-retention/">Audit those cleanup settings before they fire</a>.</p>
<h2 id="six-ways-an-imported-session-goes-wrong-and-the-signal-for-each">Six ways an imported session goes wrong, and the signal for each</h2>
<p><strong>The replayed refusal.</strong> The continued session proposes or runs a command you denied in the source. Signal: a tool call that matches a denied call in the fixture log. Response: denials become deny rules in the continuing tool on every failover.</p>
<p><strong>The wrong checkout.</strong> The source worked in <code>.claude/worktrees/fx-a/</code> or a detached Codex checkout, and the continued session edits whatever the new app opened. Signal: HEAD at its first command differs from the log. Response: the runner checks out the source branch before the first instruction. Codex has its own version of this on native resume: with the <code>worktrees</code> feature on, <a href="https://github.com/openai/codex/pull/43279">linked worktrees join the resume and fork pickers and <code>--last</code> lookup</a>, so even a resume inside Codex can land in a checkout you didn’t expect.</p>
<p><strong>The summary that became memory.</strong> The imported view opens with a compaction summary presented as conversation. Signal: FX-SUM-5 present while the pre-compaction markers are missing. Response: treat the summary as a claim and lint it before the lane acts.</p>
<p><strong>The confident filler.</strong> The probe returns a specific, wrong answer. Signal: <code>confabulated</code> in the record. Response: that lane starts fresh, as the step 6 table says, whatever the other rows say.</p>
<p><strong>The travelling canary.</strong> FX-CANARY-8 shows up in the imported view. Signal: the canary search from step 2. Response: the pre-import scan becomes mandatory for every lane, and lanes that touch secrets stay import-never.</p>
<p><strong>The anonymous merge.</strong> A PR lands with no record that a second model finished it. Signal: commits after an import timestamp with no trailers. Response: the merge check from step 7 blocks until someone adds them.</p>
<h2 id="import-is-a-failover-path-so-the-runner-owns-the-test">Import is a failover path, so the runner owns the test</h2>
<p>None of this assumes Cline gets anything wrong. A beta that documents its import in one sentence is ordinary. What isn’t ordinary is a fleet that turns it into the quota plan without a scorecard.</p>
<p>The fixtures, the record, the deny rules and the trailers live in the layer that runs the fleet, the same place a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> keeps every CLI’s sessions side by side. <a href="/intel/cross-cli-session-memory/">Cross-CLI Memory That Survives the Session</a> covers moving what should outlive a session on purpose, with review. This test is for the day a vendor offers to move all of it at once.</p>
<p>Import the fixture first. Let the real session cross only once you know what it leaves behind.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-cline-desktop-keep-tool-calls-and-approvals-when-you-import-a-claude-code-session">Does Cline Desktop keep tool calls and approvals when you import a Claude Code session?</h3>
<p>Cline’s blog and docs don’t say. They describe the Import Sessions path and one sentence about earlier context, but list nothing about tool calls, approvals, compaction summaries or branch state. Build a fixture session with a marker string for each item, import it, and mark every item carried, summarized or dropped.</p>
<h3 id="can-importing-a-codex-session-leak-secrets-to-another-provider">Can importing a Codex session leak secrets to another provider?</h3>
<p>It can. The continued task runs on the provider and model configured in Cline, so anything the source session read, tool output included, can reach that provider as context. Scan the source transcript before import, and if a real credential turns up, rotate it and hand off with a written note instead of importing.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://cline.bot/blog/cline-desktop-an-open-source-app-for-open-weight-models">Cline: Cline Desktop: An open-source app for open-weight models</a> — Etisha Garg, Sep 14, 2026; the quota use case</li>
<li><a href="https://docs.cline.bot/usage/cline-desktop">Cline Docs: Cline Desktop</a> — Import Sessions path; the “earlier context” sentence</li>
<li><a href="https://cline.bot/desktop">Cline Desktop download page</a> — beta on macOS and Windows</li>
<li><a href="https://code.claude.com/docs/en/worktrees">Claude Code: Run parallel sessions with worktrees</a> — <code>--worktree</code> paths, branches and resume</li>
<li><a href="https://code.claude.com/docs/en/hooks">Claude Code: Hooks reference</a> — <code>PostCompact</code> and <code>compact_summary</code></li>
<li><a href="https://github.com/openai/codex/releases/tag/rust-v0.154.0">Codex 0.154.0 release notes</a> — Sep 9, 2026; experimental worktree support</li>
<li><a href="https://github.com/openai/codex/pull/43279">Codex PR #43279</a> — linked worktrees in resume and fork pickers and <code>--last</code> lookup</li>
<li><a href="https://github.com/openai/codex/releases/tag/rust-v0.155.0">Codex 0.155.0 release notes</a> — Sep 17, 2026; always-streamed remote compaction</li>
</ul>
]]></content:encoded></item><item><title>Canary Every CLI Upgrade: Catch Default and Compaction Drift on Unpinned Lanes</title><link>https://automater.ai/intel/cli-upgrade-canary-drift/</link><guid>https://automater.ai/intel/cli-upgrade-canary-drift/</guid><description>AI CLI upgrade testing for unattended lanes: record the model, effort, caps and compaction point a lane was served, replay every bump, fail undeclared drift.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>On Sep 11 the model behind <code>kimi-for-coding</code> changed and the ID did not. Kimi’s changelog was cheerful about it: “The Model ID is unchanged”, so clients and third-party tools needed no configuration changes, and K2.8 Preview’s default thinking effort is <code>max</code>. Every lane that inherited those defaults was running a different model that morning, at <code>max</code> unless it said otherwise, with a config file that diffed clean against the night before.</p>
<p>That is the hole in most AI CLI upgrade testing. We check that the new version installs, that the flags parse, maybe that the deny rules hold. We rarely check what the lane was served afterwards: which model, at what effort, under what turn cap and wall-clock limit, at what token count it compacted, and what it forgot when it did. The move here is an upgrade canary per lane: record those served values from logs, bump one lane, replay the same long session, diff, and fail the upgrade on any change the lane’s manifest did not declare.</p>
<p>A chatbot whose defaults drift gives you a different answer, and you notice. An unattended lane whose defaults drift runs longer, spends more, compacts at a new point and carries a different summary into its next hour of tool calls. The first person to notice is usually whoever reads the invoice.</p>
<h2 id="twelve-harness-changes-in-eighteen-days-sep-421">Twelve harness changes in eighteen days, Sep 4–21</h2>
<p>September moved lane behaviour three ways, and only one of them arrived with a date on anyone’s calendar.</p>
<p><strong>Defaults.</strong> Codex CLI 0.153.4 (Sep 4) made GPT-6 Astra “the bundled default when no model is explicitly configured” (<a href="https://github.com/openai/codex/releases/tag/rust-v0.153.4">Codex 0.153.4 release notes</a>). Kimi’s K2.8 Preview rolled out Sep 11 behind the unchanged <code>kimi-for-coding</code> ID, and Kimi CLI 0.43.0 (Sep 14) removed goal mode’s 24-hour cap (<a href="https://www.kimi.com/code/docs/en/kimi-code/whats-new.html">Kimi Code what’s new</a>). Claude Code 2.1.277 (Sep 18) started reading <code>AGENTS.md</code> in projects with no <code>CLAUDE.md</code>. GitHub’s Copilot Auto tiers (Sep 14) say “Auto individually evaluates each prompt to select a best-suited model”, and usage is “charged based on the model auto selects, regardless of tier” (<a href="https://github.blog/changelog/2026-09-14-configure-cost-and-quality-in-copilot-auto-model-selection/">GitHub changelog</a>).</p>
<p>
<em>Screenshot: Kimi Code Docs, “What’s New” (Sep 11, 2026 entry), captured Sep 21, 2026.</em></p>
<p><strong>Compaction.</strong> Claude Code 2.1.273 (Sep 15) fixed auto-compact counting advisor-tool turns at roughly twice their real size, which had made it fire at about half the real window (<a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a>). Codex 0.155.0 (Sep 17) moved to always-streamed remote compaction for supported providers and now preserves reasoning effort through compaction (<a href="https://github.com/openai/codex/releases/tag/rust-v0.155.0">Codex 0.155.0 release notes</a>). Kimi CLI 0.43.0 added <code>loop_control.compaction_max_attempts</code> with a default of 5, and 2.0.2 (Sep 19) fixed compaction failing after a switch to a smaller-window model (<a href="https://moonshotai.github.io/kimi-code/en/release-notes/changelog.html">Kimi Code CLI changelog</a>); Kimi Code 2.0.0 (Sep 17) omits the oldest images and videos once a session’s media passes 20 MB. Pi 0.86.0 (Sep 19) added per-model <code>compaction.modelOverrides</code>, and Pi 0.87.0 shipped today with breaking changes for extension authors (<a href="https://github.com/earendil-works/pi/blob/main/packages/coding-agent/CHANGELOG.md">Pi changelog</a>).</p>
<p>
<em>Screenshot: GitHub, “Release 0.155.0 · openai/codex” (Sep 17, 2026), captured Sep 21, 2026.</em></p>
<p>The line that changes how a Codex lane compacts sits between a reviewer-extension refactor and a removed Windows slash command. Nobody reads every row, which is the argument for testing behaviour instead of reading prose.</p>
<p><strong>Calendar.</strong> Antigravity’s May agent version shuts down Oct 5, GPT-5.5 leaves ChatGPT, ChatGPT Work and Codex (not the OpenAI API) on Oct 14, Copilot deprecates a batch of models on Oct 19, and Gemini 3.8 Flash’s price doubles on Jan 1, 2027. Dated swaps belong in the deprecation ledger that <a href="/intel/forced-model-routing-continuity/">forced model routing continuity</a> already runs. This piece covers the undated kinds, which land the moment a lane picks up a new binary.</p>
<aside class="social-embed" data-x-embed data-post-id="2099954190600876533">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 15, 2026 — ChatGPT's notice that GPT-5.5 leaves ChatGPT, ChatGPT Work and Codex on Oct 14, with Codex users told to move to GPT-5.6 Sol or GPT-6 Astra.</p>
  <a class="social-embed__source" href="https://x.com/ChatGPT/status/2099954190600876533" target="_blank" rel="noopener noreferrer">View ChatGPT's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="ai-cli-upgrade-testing-has-to-start-from-what-was-served">AI CLI upgrade testing has to start from what was served</h2>
<p>A Codex lane that never set a model got Astra. A lane pointed at <code>kimi-for-coding</code> got K2.8 at <code>max</code>. A Claude Code lane that used the advisor tool had been compacting at half its window and now runs to the full one: longer context, later summary, bigger bill per turn. In every case the config file is byte-identical before and after, so the config diff most change reviews look at reports nothing.</p>
<p>The config is the declaration; the wire and the session logs are the evidence. The canary compares the two on one session you replay every time and treats any gap the manifest did not predict as a failed upgrade. The milestone compaction habits in the <a href="/intel/context-engineering-playbook/">context engineering playbook</a> assume a human who would notice a summary arriving early. An unattended lane has nobody watching.</p>
<h2 id="step-1-write-a-lane-manifest-that-says-what-the-lane-may-be-served">Step 1: Write a lane manifest that says what the lane may be served</h2>
<p>One file per lane names every served value you care about and the band it may sit in. Pinned lanes fill it with what they pin; unpinned lanes fill it with what they inherited at the last baseline, which is exactly what an upgrade can change.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># lane-manifest.yaml (illustrative shape): one per lane, read by your runner, never by the agent</span></span>
<span class="line"><span style="color:#85E89D">lane</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">nightly-deps</span></span>
<span class="line"><span style="color:#85E89D">harness</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">claude-code</span></span>
<span class="line"><span style="color:#85E89D">cli_version</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">2.1.272</span><span style="color:#6A737D">            # the version the current baseline was recorded on</span></span>
<span class="line"><span style="color:#85E89D">provider_path</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">anthropic-api</span><span style="color:#6A737D">    # auth and billing path; defaults differ by path</span></span>
<span class="line"><span style="color:#85E89D">served</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  model</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">opus-5</span><span style="color:#E1E4E8">]               </span><span style="color:#6A737D"># allowed set; exactly one entry when you pin</span></span>
<span class="line"><span style="color:#85E89D">  effort</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">high</span></span>
<span class="line"><span style="color:#85E89D">  max_turns</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">60</span></span>
<span class="line"><span style="color:#85E89D">  wall_clock_min</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">45</span></span>
<span class="line"><span style="color:#85E89D">instructions</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  agents_md</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">not-loaded</span><span style="color:#6A737D">         # repo carries AGENTS.md and no CLAUDE.md</span></span>
<span class="line"><span style="color:#85E89D">compaction</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  first_fire_turn</span><span style="color:#E1E4E8">: [</span><span style="color:#79B8FF">16</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">20</span><span style="color:#E1E4E8">]     </span><span style="color:#6A737D"># band from two baseline replays</span></span>
<span class="line"><span style="color:#85E89D">  first_fire_tokens</span><span style="color:#E1E4E8">: [</span><span style="color:#79B8FF">460000</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">505000</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  summary</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">readable</span><span style="color:#6A737D">             # readable | opaque</span></span>
<span class="line"><span style="color:#85E89D">survivors</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  pinned_facts</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">freeze-friday</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">no-vendor-edits</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">release-branch</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  note_to_self</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">n/a</span><span style="color:#6A737D">             # "verbatim" on lanes that write one</span></span>
<span class="line"><span style="color:#85E89D">cost_band_usd</span><span style="color:#E1E4E8">: [</span><span style="color:#79B8FF">3.10</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">4.40</span><span style="color:#E1E4E8">]     </span><span style="color:#6A737D"># client-side estimate per replay</span></span>
<span class="line"><span style="color:#85E89D">declared_changes</span><span style="color:#E1E4E8">: []            </span><span style="color:#6A737D"># filled in before a bump, from the changelog range</span></span></code></pre>
<p>Two rules keep it honest. Every field is pinned by config or observed at baseline, never copied from a docs page. And <code>provider_path</code> is a field because one CLI version can serve different features on different paths: Claude Code skips feature-flag fetching on third-party providers and with telemetry disabled, and without the flags it loads <code>CLAUDE.md</code> files only and can’t read <code>AGENTS.md</code> as project instructions (<a href="https://code.claude.com/docs/en/env-vars">Claude Code environment variables</a>). A canary on a different path tests a different product.</p>
<p>
<em>Twelve changes between Sep 4 and Sep 21, 2026, by what they moved. Each maps to a manifest field, and most change served behaviour on upgrade with no config edit.</em></p>
<h2 id="step-2-record-the-baseline-from-wire-and-session-logs-not-from-config">Step 2: Record the baseline from wire and session logs, not from config</h2>
<p>Record before you bump, on today’s version, from sources downstream of the harness where the served value is visible.</p>
<table>
<thead>
<tr>
<th>Manifest field</th>
<th>Where to read it</th>
<th>Why the config can’t tell you</th>
</tr>
</thead>
<tbody>
<tr>
<td>Model served</td>
<td>A logging proxy or gateway between lane and provider; Claude Code’s <code>-p --output-format json</code> per-model breakdown; the model Copilot’s CLI prints per response</td>
<td>An unset key inherits the bundled default; an unchanged ID can front a new model</td>
</tr>
<tr>
<td>Effort</td>
<td>Request parameters at the proxy</td>
<td>Defaults belong to the model and move with it</td>
</tr>
<tr>
<td>Turn cap, wall clock</td>
<td>The run’s end reason, plus start and end timestamps</td>
<td>A renamed key can stop working with only a startup warning</td>
</tr>
<tr>
<td>Compaction fired at</td>
<td>Input tokens on the last request before compaction, plus the turn index; Claude Code’s opt-in <code>x-claude-code-compaction</code> and <code>x-claude-code-context-compacted</code> headers; Pi’s <code>tokensBefore</code></td>
<td>Trigger points come from windows, reserves and counters the harness owns</td>
</tr>
<tr>
<td>Summary</td>
<td>Claude Code <code>PostCompact</code> payload; Pi’s compaction entry <code>summary</code>; Codex marked opaque</td>
<td>Readability depends on the provider path</td>
</tr>
<tr>
<td>Cost</td>
<td><code>total_cost_usd</code> from the JSON output, or proxy token counts times your rate file</td>
<td>Estimates move whenever the model does</td>
</tr>
</tbody>
</table>
<p>The Claude Code headers arrived in 2.1.273 behind <code>CLAUDE_CODE_GATEWAY_HINT_HEADERS=1</code> and let a gateway log tell a compaction request from a work request, the cleanest wire-level record of when a lane compacted. The headless JSON’s cost figures carry the docs’ own warning: “Both figures are client-side estimates and can differ from your actual bill” (<a href="https://code.claude.com/docs/en/headless">Claude Code headless docs</a>). Use them for the diff, never for chargeback. Pi compacts when context tokens exceed the window minus <code>reserveTokens</code> (16,384 by default) and stores <code>tokensBefore</code> on each compaction entry (<a href="https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/compaction.md">Pi compaction docs</a>).</p>
<p>Record the turn index beside the token count. When an upgrade swaps the default model, the same text can tokenize differently; a compaction that moved in tokens but not in turns is a tokenizer change, not a moved threshold.</p>
<h2 id="step-3-build-one-canned-long-session-that-forces-a-compaction">Step 3: Build one canned long session that forces a compaction</h2>
<p>The canary is only as good as the session you replay. Build one per harness family and treat edits to it like edits to a test suite.</p>
<ol>
<li><strong>Fixture repo at a pinned commit.</strong> Same SHA every replay, from a clean checkout, with an <code>AGENTS.md</code> and no <code>CLAUDE.md</code> so instruction-file drift shows up.</li>
<li><strong>Scripted prompts in a fixed order.</strong> Twenty to forty turns shaped like the lane’s real job: read, edit, test, fix.</li>
<li><strong>Pinned facts early.</strong> At turn two, state three facts the lane must honour after compaction: a freeze date, a directory it must never edit, the release branch. Give each an ID.</li>
<li><strong>A context push.</strong> Tool calls that read large files, so the session crosses the compaction point at least once, ideally twice.</li>
<li><strong>Recall probes at the end.</strong> One question per pinned fact, plus one task that breaks a fact if the lane forgot it, such as an edit inside the forbidden directory.</li>
<li><strong>The production flag line.</strong> Same unattended flags, permission mode and environment as the lane. The <a href="/intel/headless-agent-trust-tier/">headless agent trust tier</a> piece already pins that flag line per tier; reuse it.</li>
</ol>
<p>Replay the baseline twice before you bump anything. If the two runs disagree on the compaction turn by more than a few turns, widen the band now, or every upgrade will fail on noise and people will learn to wave failures through.</p>
<h2 id="step-4-bump-one-lane-replay-twice-and-diff-everything-served">Step 4: Bump one lane, replay twice, and diff everything served</h2>
<p>Upgrade the CLI on the canary lane only; every other lane keeps its pinned version until this one passes. Then replay the canned session twice.</p>
<p>The second run exists because of documented Claude Code behaviour: in the first session after an install, or after an upgrade that adds a feature, “a flag-gated feature can be missing”, and it appears in the next session once the flags are fetched (<a href="https://code.claude.com/docs/en/env-vars">Claude Code environment variables</a>). Reading <code>AGENTS.md</code> and claude.ai skill and plugin sync are both flag-gated, so run 1 and run 2 can load different instructions from the same repo. Diff each run against the baseline and against each other. The full clean-home versus used-home differential for instruction files, synced skills and memory is <a href="/intel/headless-lane-reproducibility/">a separate test</a>; the canary only asks whether served behaviour moved.</p>
<p>Here is a weekly Claude Code bump from 2.1.272 to 2.1.278 through the canary. Values are illustrative; the mechanisms are the two changelog entries above.</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Baseline (2.1.272)</th>
<th>Run 1 (2.1.278)</th>
<th>Run 2 (2.1.278)</th>
<th>Declared?</th>
<th>Verdict</th>
</tr>
</thead>
<tbody>
<tr>
<td>Model served</td>
<td>opus-5</td>
<td>opus-5</td>
<td>opus-5</td>
<td>n/a</td>
<td>pass</td>
</tr>
<tr>
<td>Effort</td>
<td>high</td>
<td>high</td>
<td>high</td>
<td>n/a</td>
<td>pass</td>
</tr>
<tr>
<td>First compaction (turn)</td>
<td>18</td>
<td>31</td>
<td>31</td>
<td>yes, 2.1.273</td>
<td>pass</td>
</tr>
<tr>
<td>First compaction (tokens)</td>
<td>485K</td>
<td>950K</td>
<td>955K</td>
<td>yes, 2.1.273</td>
<td>pass</td>
</tr>
<tr>
<td><code>AGENTS.md</code> loaded</td>
<td>no</td>
<td>no</td>
<td>yes</td>
<td>no</td>
<td><strong>fail</strong></td>
</tr>
<tr>
<td>Pinned facts recalled</td>
<td>3/3</td>
<td>3/3</td>
<td>3/3</td>
<td>n/a</td>
<td>pass</td>
</tr>
<tr>
<td>Summary readable at <code>PostCompact</code></td>
<td>yes</td>
<td>yes</td>
<td>yes</td>
<td>n/a</td>
<td>pass</td>
</tr>
<tr>
<td>Cost estimate per replay</td>
<td>$3.60</td>
<td>$5.10</td>
<td>$5.20</td>
<td>band raised with 2.1.273</td>
<td>pass</td>
</tr>
</tbody>
</table>
<p>The compaction move was declared because someone read 2.1.273 before the bump. The <code>AGENTS.md</code> load was not, and it appears only in run 2, so a single replay would have passed this upgrade.</p>
<p>
<em>Two replays before the bump, two after. The second replay after the bump catches features that switch on only once the new version has run once.</em></p>
<h2 id="step-5-check-what-survived-the-compaction-and-whether-you-can-still-read-it">Step 5: Check what survived the compaction, and whether you can still read it</h2>
<p>Served model and effort are the easy half. The expensive drift happens inside the summary.</p>
<ul>
<li><strong>Pinned facts.</strong> Score each recall probe and the forbidden-directory task. A lane that forgets a pinned fact fails the upgrade even if every other field passed; on a real run the next compaction will forget something you did not pin.</li>
<li><strong>The note-to-self.</strong> On lanes that write a handoff note before compacting, check it came back byte for byte. The <a href="/intel/agent-owned-compaction-policy/">compaction contract piece</a> covers how the note crosses the cut; the canary checks it still does. Pi 0.87.0 changed the extension API today, so an extension that intercepts compaction can load cleanly and still stop doing its job.</li>
<li><strong>Readability.</strong> On Claude Code, confirm <code>PostCompact</code> still receives <code>compact_summary</code>; the <a href="/intel/compaction-summary-lint/">summary lint</a> depends on that payload. On Codex lanes whose provider path returns an encrypted compaction item, confirm the manifest says <code>opaque</code> and your outside evidence still captures the transcript. On Pi, confirm the compaction entry still carries readable <code>summary</code> text.</li>
<li><strong>Compaction failures.</strong> Count compaction attempts per run. Kimi’s <code>compaction_max_attempts</code> allows five total attempts for a failing compaction by default, and the 2.0.2 fix shows the failure was real after a model switch. A run that compacted once at baseline and three times after the bump has drifted, even if it finished.</li>
<li><strong>Per-model overrides.</strong> Pi’s <code>compaction.modelOverrides</code> falls back to the ordinary settings when no override matches the served model. When an upgrade changes the default model, a tuned lane quietly loses its override and the compaction point moves with no setting touched.</li>
</ul>
<h2 id="step-6-fail-the-upgrade-on-anything-the-manifest-did-not-declare">Step 6: Fail the upgrade on anything the manifest did not declare</h2>
<p>Every difference between the baseline and either post-bump run must match an entry in <code>declared_changes</code>, or the upgrade fails and the fleet stays pinned.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># declared_changes for the 2.1.272 -&gt; 2.1.278 bump (illustrative)</span></span>
<span class="line"><span style="color:#85E89D">declared_changes</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#85E89D">field</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">compaction.first_fire_turn</span></span>
<span class="line"><span style="color:#85E89D">    expect</span><span style="color:#E1E4E8">: [</span><span style="color:#79B8FF">28</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">34</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">    source</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">Claude Code 2.1.273 changelog entry on advisor-tool turn counting</span></span>
<span class="line"><span style="color:#85E89D">    owner</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">platform-oncall</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#85E89D">field</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">compaction.first_fire_tokens</span></span>
<span class="line"><span style="color:#85E89D">    expect</span><span style="color:#E1E4E8">: [</span><span style="color:#79B8FF">900000</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">967000</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">    source</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">same entry; the lane now compacts near the model default</span></span>
<span class="line"><span style="color:#85E89D">    owner</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">platform-oncall</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#85E89D">field</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">cost_band_usd</span></span>
<span class="line"><span style="color:#85E89D">    expect</span><span style="color:#E1E4E8">: [</span><span style="color:#79B8FF">4.60</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">5.60</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">    source</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">follows from the later compaction point</span></span>
<span class="line"><span style="color:#85E89D">    owner</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">platform-oncall</span></span></code></pre>
<ol>
<li><strong>Declare before you bump.</strong> Someone reads the changelog range, writes the expected value or band for each field it touches, cites the line, and signs it.</li>
<li><strong>Fail on the undeclared.</strong> Any diff with no matching declaration fails the canary, however harmless it looks.</li>
<li><strong>Flag the declared-but-absent.</strong> A declared change that did not show up means the lane is not on the path you think. Check <code>provider_path</code> first.</li>
<li><strong>Pin or declare, then replay.</strong> Either set explicitly the value the lane used to inherit (model, effort, compaction window, cap) and replay until the diff is clean, or declare the new behaviour with a reason.</li>
<li><strong>Promote, then rebaseline.</strong> A passing canary becomes the fleet’s pinned version, and its run 2 becomes the next baseline.</li>
</ol>
<p>Silent config death is what this catches best. Kimi CLI 0.32.0 renamed two <code>[loop_control]</code> keys, including <code>max_steps_per_run</code>, and the changelog says the old keys “stop working with a rename warning at startup”. A lane whose step cap lived under the old key lost it on upgrade. The canary sees a run far longer than the manifest allows; nobody had to read stderr.</p>
<p>Run two sibling tests in the same gate: the deny-rule canary repo from <a href="/intel/restricted-mode-fleet-policy/">restricted mode as fleet policy</a>, which proves permissions still hold, and the <a href="/intel/tool-schema-guard-contract-tests/">guard contract suite</a>, which proves a renamed tool did not slip past a hook matcher.</p>
<h2 id="step-7-for-auto-routers-you-cannot-pin-log-the-model-per-prompt-and-cap-the-minutes">Step 7: For auto-routers you cannot pin, log the model per prompt and cap the minutes</h2>
<p>Copilot Auto picks a model per prompt and bills at that model’s rate, so there is nothing to pin. The manifest declares an allowed set instead.</p>
<ul>
<li><strong>Log the model served on every prompt.</strong> GitHub shows the routed model per response (printed in Copilot CLI, on hover in Chat) but documents no export, so capture it from the CLI output into the run log.</li>
<li><strong>Fail on any model outside the allowed set,</strong> including a newly added one. A router gaining a model is an upgrade you did not install.</li>
<li><strong>Replay three times and keep a distribution.</strong> Per-prompt routing turns the compaction point and cost into ranges; set the bands from the spread.</li>
<li><strong>Cap wall-clock minutes per run.</strong> You cannot cap the rate of a model you did not choose, but you can stop the run, with the same minutes cap the ledger piece applies to forced swaps.</li>
</ul>
<h2 id="where-ai-cli-upgrade-testing-misleads-you-and-the-signal-for-each">Where AI CLI upgrade testing misleads you, and the signal for each</h2>
<p><strong>Noise read as drift.</strong> Nondeterministic tool output or routing moves the compaction turn. Signal: two baseline replays on the old version disagree. Fix: widen the band; never skip the baseline pair.</p>
<p><strong>A tokenizer change read as a threshold change.</strong> Signal: compaction tokens moved while the compaction turn held. Fix: declare it as a model change.</p>
<p><strong>First-session artifacts.</strong> Signal: run 1 matches the baseline and run 2 does not. Fix: judge the upgrade on run 2; keep run 1 as evidence.</p>
<p><strong>The wrong provider path.</strong> The canary runs on an API key while production runs through a gateway or a subscription login. Signal: the canary passes and a production lane still diffs. Fix: a <code>provider_path</code> mismatch fails the canary before it runs.</p>
<p><strong>A stale fixture.</strong> Signal: a baseline replay changes with no upgrade in between. Fix: hash the fixture and prompt script into the manifest and rebaseline when either hash changes.</p>
<p><strong>Estimates read as bills.</strong> Signal: the canary’s cost diff and next month’s invoice disagree. Fix: diff tokens by model and price them from your own rate file.</p>
<h2 id="upgrades-are-change-events-and-the-fleet-layer-owns-them">Upgrades are change events, and the fleet layer owns them</h2>
<p>Nothing in this runbook lives inside the harness. The manifest, the canned session, the two-plus-two replays and the fail rule belong to the layer that decides which lanes run which binary, the same layer that holds kill switches, approvals and evidence. In a <a href="/intel/multi-agent-command-center/">multi-agent command center</a>, that layer is where an overnight CLI update becomes a change record with a baseline, a diff and a name on the approval.</p>
<p>Vendors will keep shipping defaults that are better on average. The canary is for the lane that is not average, and for the night the release notes skipped the line that mattered.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-pinning-the-cli-version-stop-default-drift">Does pinning the CLI version stop default drift?</h3>
<p>Pinning stops the harness from changing under a lane, but not the provider. An unchanged model ID can front a new model, as <code>kimi-for-coding</code> did on Sep 11, and hosted routers such as Copilot Auto pick per prompt. Pin the version, set model and effort explicitly, and replay the canary on every bump anyway.</p>
<h3 id="why-does-claude-code-behave-differently-in-the-first-session-after-an-upgrade">Why does Claude Code behave differently in the first session after an upgrade?</h3>
<p>Some Claude Code features are flag-gated. Its docs say that in the first session after an install, or after an upgrade that adds a feature, the feature can be missing until the flags are fetched. Reading AGENTS.md and claude.ai sync are both gated, so replay twice after every bump and judge run 2.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://www.kimi.com/code/docs/en/kimi-code/whats-new.html">Kimi Code: What’s New</a> — K2.8 Preview, unchanged ID, default <code>max</code> (Sep 11); 24-hour cap removed (Sep 14); 20 MB media rule (Sep 17)</li>
<li><a href="https://moonshotai.github.io/kimi-code/en/release-notes/changelog.html">Kimi Code CLI changelog</a> — 0.43.0 compaction attempts; 2.0.2 fix; 0.32.0 key renames</li>
<li><a href="https://github.com/openai/codex/releases/tag/rust-v0.153.4">Codex 0.153.4 release notes</a> — Astra as bundled default (Sep 4)</li>
<li><a href="https://github.com/openai/codex/releases/tag/rust-v0.155.0">Codex 0.155.0 release notes</a> — streamed remote compaction (Sep 17)</li>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a> — 2.1.273 (Sep 15), 2.1.277 (Sep 18)</li>
<li><a href="https://code.claude.com/docs/en/env-vars">Claude Code environment variables</a> — feature-flag fetching and the first-session rule</li>
<li><a href="https://code.claude.com/docs/en/headless">Claude Code headless docs</a> — JSON cost figures are client-side estimates</li>
<li><a href="https://github.blog/changelog/2026-09-14-configure-cost-and-quality-in-copilot-auto-model-selection/">GitHub changelog: Copilot auto model selection</a> — per-prompt selection and billing (Sep 14)</li>
<li><a href="https://github.com/earendil-works/pi/blob/main/packages/coding-agent/CHANGELOG.md">Pi changelog</a> — 0.86.0 overrides (Sep 19); 0.87.0 breaking changes (Sep 21)</li>
<li><a href="https://github.com/earendil-works/pi/blob/main/packages/coding-agent/docs/compaction.md">Pi compaction docs</a> — trigger, <code>reserveTokens</code>, <code>tokensBefore</code></li>
</ul>
]]></content:encoded></item><item><title>Routers You Can’t See Inside: Fugu Max, Copilot Auto, Cursor Auto vs Your Own Cascade</title><link>https://automater.ai/intel/hosted-router-vs-own-cascade/</link><guid>https://automater.ai/intel/hosted-router-vs-own-cascade/</guid><description>An LLM router that can&apos;t name the model behind each change fails change control. Five acceptance tests for Fugu Max, Copilot Auto, Cursor Auto and your cascade.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category><content:encoded><![CDATA[<p>Three vendors now sell the same thing under different names: a model picker that bills like a model. Sakana prices Fugu Max, an orchestrator, at $2 per million input tokens and $6 per million output. GitHub’s Copilot Auto now takes a cost-or-quality tier and picks a model per prompt. Cursor’s Auto bills each request at the list price of whichever model it chose.</p>
<p>Each is an LLM router you rent instead of build, and each gives a different answer to the question your change control will ask sooner or later: which model wrote this? A chatbot router only has to be right on average. A router in front of agents that edit code, open pull requests and run shell commands has to leave a record a reviewer can read, rerun and roll back against. “Auto” is not a model anyone can roll back to.</p>
<p>This is the buy-side test. You leave with five acceptance criteria to run against any hosted router, a scorecard for the three on offer now, a way to run your own cascade as the control arm, and a rule for the routers that fail: treat each one as a single unpinned vendor model and fence its lane.</p>
<h2 id="what-shipped-fugu-max-copilot-auto-tiers-and-cursors-routed-billing">What shipped: Fugu Max, Copilot Auto tiers and Cursor’s routed billing</h2>
<p><strong>Sakana, Sep 11.</strong> Fugu Max and Fugu Ultra v2 are orchestrators sold as models, both reachable through Sakana’s OpenAI-compatible API. The <a href="https://sakana.ai/fugu-max-release/">release page</a> says Fugu Max routes each task to the leanest model capable of solving it and prices it at $2/$6 per million tokens. That page prices only Max.</p>
<p>Ultra v2’s $5 input and $30 output come from the <a href="https://vercel.com/ai-gateway/models/fugu-ultra-v2">Vercel AI Gateway listing</a>, which also shows a $0.50/M cache-read price, a 1M-token context and a second tier from 272,001 tokens at $10 input and $45 output (cached input $1). Sakana does say which models are not in Ultra v2’s pool: Fable 5, Fable 5.1 and GPT-6-Astra. Nothing on the page says which model answered a given call.</p>
<p>
<em>Screenshot: Sakana AI, “Introducing Fugu Max and Fugu Ultra v2: Orchestrating the Pareto Frontier” (Sep 11, 2026), captured Sep 21, 2026.</em></p>
<aside class="social-embed" data-x-embed data-post-id="2098240407947673868">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 11, 2026 — Sakana AI's co-founder announces Fugu Max and Fugu Ultra v2 as orchestration across the cost and capability frontier.</p>
  <a class="social-embed__source" href="https://x.com/hardmaru/status/2098240407947673868" target="_blank" rel="noopener noreferrer">View David Ha's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<p><strong>GitHub, Sep 14.</strong> Copilot’s auto model selection gained three tiers, Efficiency, Balance and Intelligence, rolling out in VS Code, Copilot CLI and the GitHub Copilot app. Per the <a href="https://github.blog/changelog/2026-09-14-configure-cost-and-quality-in-copilot-auto-model-selection/">changelog</a>, usage is charged at the rate of the model auto selects whatever the tier, and paid subscribers keep a 10% discount on auto usage. The tier steers the choice; GitHub’s own example is a docstring task that may land on a small model even under Intelligence.</p>
<p>Unlike Fugu, Copilot shows its work per response: the <a href="https://docs.github.com/en/copilot/concepts/models/auto-model-selection">auto model selection docs</a> say you can see which model answered each response (hover in Chat, printed in the CLI, at the end of a cloud agent response, beside Auto in the app). Neither page describes an export or API log of routed models.</p>
<p><strong>Cursor, undated docs.</strong> Cursor’s <a href="https://cursor.com/docs/models-and-pricing">models and pricing page</a> bills every Auto mode at the list price of the model each request routes to. On Teams and Enterprise, third-party model requests add a Cursor Token Rate of $0.25 per million tokens, whether you picked the model or Auto did, and the rate covers BYOK usage too; first-party Cursor models (Grok, Composer) are exempt. Cursor’s changelog carries no pricing notice, so treat any reported cutover date for older flat-rate Auto plans as unconfirmed.</p>
<p>The <a href="https://cursor.com/docs/cursor-router">Cursor Router</a> behind Auto launched Jul 22 for Teams and Enterprise. It runs a classifier on each agent request in Balance and Intelligence, while Cost mode keeps the older routing logic. You can’t hand-pick the model, and the pool changes as new models ship.</p>
<p><strong>The reference orchestrator, Sep 4.</strong> GitHub’s <a href="https://github.blog/ai-and-ml/github-copilot/project-hydrafusion-frontier-quality-via-multi-model-orchestration/">Project HydraFusion</a>, experimental in Copilot CLI, picks Single, Cascade or Critique per task. Its best-tuned configuration beat Opus 5 on estimated cost in GitHub’s controlled offline runs, every model at medium reasoning (the build-side playbook in step 4 charts the numbers), and GitHub says the approach works best on first-turn, single-prompt coding tasks. The line that matters to a buyer: the runtime records each leg’s role, cost and outcome internally, and externally the developer gets “one coherent response and one permission-aware change set.”</p>
<h2 id="why-an-agent-lane-needs-the-model-name-a-chatbot-never-did">Why an agent lane needs the model name a chatbot never did</h2>
<p>The served model is a change-control input. You need it to reproduce a regression, to find every change a model made after you stop trusting it, to apply per-model data terms, and to answer an auditor who asks what wrote the payment code. When a provider swaps the model under a fixed API ID, that is <a href="/intel/forced-model-routing-continuity/">forced model routing</a>, and you drill for it. A hosted router makes the swap the product, and it can happen on every prompt.</p>
<p>This is not the MCP gateway question either. <a href="/intel/commercial-mcp-gateway-buy-or-build/">Buying or building an MCP gateway</a> is about proxies in front of tools; a model router decides what does the thinking. Chatbots suggest; agents act, and the router decides which model does the acting.</p>
<h2 id="step-1-turn-llm-router-buying-questions-into-five-passfail-tests">Step 1: Turn LLM router buying questions into five pass/fail tests</h2>
<p>Write the criteria down before any vendor demo, and phrase each as a test with a number in it. A question like “does it support visibility?” gets a yes from every sales deck.</p>
<table>
<thead>
<tr>
<th>#</th>
<th>Criterion</th>
<th>Passes when</th>
<th>Test you run</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>Per-call routed-model visibility</td>
<td>Every call or response yields a model ID your runner stores without a human reading a screen</td>
<td>20 fixed tasks; 20 of 20 records carry a served model, or fail</td>
</tr>
<tr>
<td>2</td>
<td>Spend by routed model</td>
<td>Usage can be split by the model that served it</td>
<td>Split one week of spend by served model; totals match the invoice</td>
</tr>
<tr>
<td>3</td>
<td>Pin and override</td>
<td>A lane can force a named model and exclude others, and the served model matches</td>
<td>Pin, run five tasks, compare served model to pin</td>
</tr>
<tr>
<td>4</td>
<td>Eval on a fixed task set</td>
<td>The same tasks run through the router and your alternatives, scored the same way</td>
<td>Cost per merged PR on a frozen set, every attempt counted</td>
</tr>
<tr>
<td>5</td>
<td>Evidence export</td>
<td>A per-call record (request reference, served model, cost, outcome) leaves the vendor before its retention does</td>
<td>Export last week; join every row to a PR</td>
</tr>
</tbody>
</table>
<p>Grade criterion 1 on three levels, because vendors blur them. <strong>Exported</strong> means a machine-readable model ID per call, from the vendor or from a log you control. <strong>Displayed</strong> means a person can see it on a screen. <strong>Not exposed</strong> means nobody can. A displayed model counts as exported only when your runner captures it reliably, and that capture is a test you rerun after every CLI upgrade.</p>
<p>Criterion 5 is where hosted routers quietly fail even when criterion 1 passes. A model name on a hover card is evidence for the person hovering and nobody else. <a href="/intel/vendor-hosted-agent-evidence/">Vendor-hosted agent evidence</a> covers export timing and retention windows; the router adds one field that has to survive the export.</p>
<h2 id="step-2-score-fugu-copilot-auto-and-cursor-auto-against-the-five">Step 2: Score Fugu, Copilot Auto and Cursor Auto against the five</h2>
<p>Fill the scorecard from the vendor’s own pages, and write “the docs do not say” where they don’t. Anything a salesperson tells you goes in a separate column until it shows up in documentation.</p>
<table>
<thead>
<tr>
<th></th>
<th>Fugu Max / Ultra v2</th>
<th>Copilot Auto</th>
<th>Cursor Auto (Router)</th>
<th>Your own cascade</th>
</tr>
</thead>
<tbody>
<tr>
<td>Which model answered</td>
<td>Not exposed; Sakana lists three models absent from Ultra v2’s pool</td>
<td>Displayed per response in Chat, CLI, cloud agent and app</td>
<td>Admin “Underlying model” toggle, hidden by default; Balance and Intelligence only</td>
<td>Exported: whatever you log</td>
</tr>
<tr>
<td>How it bills</td>
<td>As one model: Max $2/$6; Ultra v2 $5/$30 per Vercel’s listing, $10/$45 from 272,001 tokens</td>
<td>Routed model’s rate, 10% off auto usage for paid subscribers</td>
<td>Routed model’s list price, plus $0.25/M on third-party models (Teams, Enterprise, incl. BYOK)</td>
<td>Each tier at its own rate</td>
</tr>
<tr>
<td>Spend by routed model</td>
<td>Not possible: one price line</td>
<td>Docs do not describe a by-model export</td>
<td>Docs do not describe a per-request export</td>
<td>Yes, from your log</td>
</tr>
<tr>
<td>Pin and override</td>
<td>Choose Max or Ultra v2; Sakana does not describe pinning a sub-model</td>
<td>Admin model policies exclude models from Auto; the only pin is leaving Auto</td>
<td>No hand-pick inside Auto; admins can disable up to two modes; Impose Auto (Hard) locks the picker</td>
<td>Yes: the route table</td>
</tr>
<tr>
<td>Pool changes</td>
<td>Ultra v2 training cutoff 2026-08-28; no change policy stated</td>
<td>May serve evaluation models to individual plans (opt-out exists)</td>
<td>Pool changes as new models ship; router needs Grok 4.6 enabled</td>
<td>Only when you change it</td>
</tr>
<tr>
<td>Fixed-task eval</td>
<td>Whole system only</td>
<td>Whole system, model visible per response</td>
<td>Whole system, model visible only with the toggle</td>
<td>Per arm and per tier</td>
</tr>
</tbody>
</table>
<p>
<em>Four routers, one question. Only the last column is evidence you control.</em></p>
<p>What the scorecard says, router by router:</p>
<ol>
<li><strong>Fugu fails criteria 1 to 3 by design.</strong> It is the purest case of an orchestrator sold as a model: one price line, one answer, no sub-model name. That’s a coherent product, and it means you evaluate Fugu as a model on your task set and fence it like one (step 5).</li>
<li><strong>Copilot Auto passes criterion 1 at the displayed grade.</strong> In Copilot CLI the model prints with each response, so a runner that keeps the terminal transcript can build its own per-response record. The docs do not document an export, so the capture is yours to build and test. Note that GitHub says auto evaluates each prompt individually: one pull request built over a dozen prompts can carry several served models.</li>
<li><strong>Cursor Auto passes criterion 1 only if an admin flips the setting.</strong> Cursor’s docs call hidden “the default and recommended” so answers are judged on merit, a fine rule for chat and the wrong one for a lane under change control. The toggle covers Balance and Intelligence; for Cost mode the docs do not say, so score it not exposed. On Enterprise the router itself is off until someone enables it.</li>
<li><strong>HydraFusion is the yardstick, not a product.</strong> It is experimental, and GitHub’s post doesn’t say users can see the per-leg records its runtime keeps.</li>
</ol>
<p>
<em>Screenshot: Cursor Docs, “Cursor Router” (undated), captured Sep 21, 2026.</em></p>
<h2 id="step-3-price-the-router-before-you-price-the-tokens">Step 3: Price the router before you price the tokens</h2>
<p>Hosted routers split into two pricing shapes, and each hides something different. Fugu gives you a known rate for an unknown model. Copilot and Cursor give you a known model per request, when it is visible, and an unknown rate until the month closes, because the router’s choice sets the price.</p>
<p>
<em>Fugu prices the orchestrator. Copilot and Cursor price whatever it picked. Sources: Sakana release page, Vercel AI Gateway listing, GitHub changelog, Cursor docs.</em></p>
<p>Put real rates against one illustrative month so the shapes are concrete. The volumes are illustrative: 20M input and 4M output tokens for one lane.</p>
<table>
<thead>
<tr>
<th>Router</th>
<th>Rate basis</th>
<th>Illustrative month, 20M in / 4M out</th>
</tr>
</thead>
<tbody>
<tr>
<td>Fugu Max</td>
<td>$2 in / $6 out</td>
<td>$40 + $24 = $64</td>
</tr>
<tr>
<td>Fugu Ultra v2, requests below 272,001 tokens</td>
<td>$5 / $30 (Vercel listing)</td>
<td>$100 + $120 = $220</td>
</tr>
<tr>
<td>Fugu Ultra v2, from 272,001 tokens</td>
<td>$10 / $45 (Vercel listing)</td>
<td>$200 + $180 = $380</td>
</tr>
<tr>
<td>Copilot Auto</td>
<td>Routed model’s rate, less 10% for paid subscribers</td>
<td>Unknown until you see the model mix</td>
</tr>
<tr>
<td>Cursor Auto, third-party routes</td>
<td>Routed list price + $0.25/M</td>
<td>Model mix + $6.00 Token Rate on 24M tokens</td>
</tr>
</tbody>
</table>
<p>Three pricing checks follow from that table:</p>
<ol>
<li><strong>Log input size per request on any Ultra v2 lane.</strong> Long agent contexts drift past 272,000 tokens without anyone deciding to, and Vercel’s listing charges more from there.</li>
<li><strong>Don’t read a Copilot tier as a budget.</strong> The tier steers selection; the bill follows the model. Spend caps belong in your budget layer, not in a tier dropdown.</li>
<li><strong>Track first-party versus third-party routes on Cursor.</strong> The Token Rate applies when Auto routes to a third-party model, BYOK included, so the router’s choice moves the bill even when list prices don’t.</li>
</ol>
<p>Spend by routed model then goes into the monthly close. <a href="/intel/gateway-chargeback-invoice-reconciliation/">The chargeback reconciliation piece</a> builds the three-column worksheet (client estimate, gateway meter, provider invoice); a router lane adds a served-model column to it, and a router that can’t fill that column reconciles only as a single line item.</p>
<h2 id="step-4-run-your-own-cascade-as-the-control-arm">Step 4: Run your own cascade as the control arm</h2>
<p>Your build side already exists as a design. <a href="/intel/jev-cheap-model-router-escalation/">The Jev routing playbook</a> covers the route table, the confidence floor, the never-route list and cost per merged PR on a fixed task set, and <a href="/intel/deepseek-agent-economics/">the DeepSeek agent-economics piece</a> works the cascade-with-escalation arithmetic. Don’t rebuild either here. Wire them into one comparison with every hosted router you are considering.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># router-eval.yaml (illustrative shape; your eval runner reads it, no vendor does)</span></span>
<span class="line"><span style="color:#85E89D">task_set</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">frozen-40-v3</span><span style="color:#6A737D">            # repo snapshots + acceptance tests, never edited mid-run</span></span>
<span class="line"><span style="color:#85E89D">arms</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#9ECBFF">own-cascade</span><span style="color:#6A737D">                   # route table from the build-side playbook</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#9ECBFF">fugu-max</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#9ECBFF">copilot-auto:balance</span><span style="color:#6A737D">          # served model captured from CLI output</span></span>
<span class="line"><span style="color:#E1E4E8">  - </span><span style="color:#9ECBFF">cursor-auto:balance</span><span style="color:#6A737D">           # Underlying model setting turned on</span></span>
<span class="line"><span style="color:#85E89D">record_per_attempt</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">arm</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">task_id</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">attempt</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">served_model</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">tokens_in</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">tokens_out</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#9ECBFF">                     cost_usd</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">escalated</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">tests_passed</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">merged</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">served_model_if_hidden</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">not_exposed</span><span style="color:#6A737D">   # never blank, never guessed</span></span>
<span class="line"><span style="color:#85E89D">score</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">cost_per_merged_pr</span><span style="color:#6A737D">             # every attempt, retry and escalation counted</span></span>
<span class="line"><span style="color:#85E89D">rerun_on</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">new_served_model_seen</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">cli_upgrade</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">monthly</span><span style="color:#E1E4E8">]</span></span></code></pre>
<p>Run rules that keep the comparison honest:</p>
<ul>
<li><strong>Same snapshot, same rubric, same reviewer.</strong> A router judged on a newer repo state than your cascade is judged on a different test.</li>
<li><strong>Count everything.</strong> Retries, escalations, abandoned attempts and reverted merges go in the spend. Only merges go in the denominator.</li>
<li><strong>A served model you haven’t seen before is a pool change.</strong> It triggers a rerun. For a router that exposes nothing, the monthly rerun is the only trigger you have.</li>
<li><strong>Upgrades are pool changes too.</strong> A CLI bump can change which models Auto picks; <a href="/intel/cli-upgrade-canary-drift/">the CLI upgrade canary</a> logs the model served per prompt before and after.</li>
<li><strong>Read HydraFusion as a ceiling on expectations.</strong> GitHub says the approach works best on first-turn, single-prompt tasks. If a vendor quotes those gains against your multi-turn agent work, ask for your task set’s number.</li>
</ul>
<p>The decision rule is short. A hosted router that beats your cascade on cost per merged PR and passes criteria 1, 3 and 5 is worth buying, and you can retire the build. One that beats it and fails any of those three goes to step 5, where it can still earn a lane.</p>
<h2 id="step-5-fence-the-router-that-cant-name-its-model">Step 5: Fence the router that can’t name its model</h2>
<p>If a hosted router can’t tell you which model shipped the change, treat it as one unpinned vendor model. Two consequences follow, and both belong in the lane manifest your runner enforces, not in a wiki.</p>
<p><strong>Log its output apart from other lanes.</strong> Separate lane ID, separate log stream, and the served model recorded as <code>not_exposed</code> rather than left blank. A blank looks like a logging bug; <code>not_exposed</code> is a fact you can count.</p>
<p><strong>Keep it out of any lane whose audit or change-control policy requires naming the model behind each change.</strong> Payment code, infrastructure, security fixes, anything a regulator or customer contract asks you to attribute. The fence is enforced at lane launch and again at merge.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># lanes.yaml (illustrative shape; the runner refuses a launch that violates it)</span></span>
<span class="line"><span style="color:#85E89D">lanes</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  docs-drafts</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    router</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">fugu-max</span></span>
<span class="line"><span style="color:#85E89D">    model_identity</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">not_exposed</span><span style="color:#6A737D">        # exported | displayed | not_exposed</span></span>
<span class="line"><span style="color:#85E89D">    treated_as</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">unpinned_vendor_model</span></span>
<span class="line"><span style="color:#85E89D">    log_stream</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">routers/fugu-max</span><span style="color:#6A737D">       # never merged into other lanes' logs</span></span>
<span class="line"><span style="color:#85E89D">    change_control</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">human-review</span></span>
<span class="line"><span style="color:#85E89D">  payments-service</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    change_control</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">named-model-required</span></span>
<span class="line"><span style="color:#85E89D">    routers_allowed</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">own-cascade</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">    model_identity_required</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">exported</span><span style="color:#6A737D">  # displayed passes only with a tested capture</span></span></code></pre>
<p>Then check at merge time, so a lane that drifts out of policy can’t ship quietly:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># Illustrative pre-merge check: every routed response on a named-model lane has a model</span></span>
<span class="line"><span style="color:#B392F0">jq</span><span style="color:#79B8FF"> -e</span><span style="color:#79B8FF"> --arg</span><span style="color:#9ECBFF"> pr</span><span style="color:#9ECBFF"> "</span><span style="color:#E1E4E8">$PR</span><span style="color:#9ECBFF">"</span><span style="color:#9ECBFF"> '</span></span>
<span class="line"><span style="color:#9ECBFF">  [ .[] | select(.pr == $pr and .lane_policy == "named-model-required") ]</span></span>
<span class="line"><span style="color:#9ECBFF">  | length &gt; 0 and all(.served_model != null and .served_model != "not_exposed")</span></span>
<span class="line"><span style="color:#9ECBFF">'</span><span style="color:#9ECBFF"> routed-calls.json</span><span style="color:#F97583"> ||</span><span style="color:#E1E4E8"> { </span><span style="color:#79B8FF">echo</span><span style="color:#9ECBFF"> "PR </span><span style="color:#E1E4E8">$PR</span><span style="color:#9ECBFF">: routed response without a named model"</span><span style="color:#E1E4E8">; </span><span style="color:#79B8FF">exit</span><span style="color:#79B8FF"> 1</span><span style="color:#E1E4E8">; }</span></span></code></pre>
<p>The check fails closed: a missing or unreadable log exits non-zero and blocks the merge, which is what you want. It is still only a guardrail. Make it a required status on the protected branch, so a skipped run blocks instead of passing, and keep human review on named-model lanes as the wall behind it.</p>
<p>Record the router in the change itself, too. A commit trailer such as <code>Routed-By: copilot-auto/balance</code> with a <code>Served-Models:</code> list, or <code>Served-Models: not-exposed</code>, costs nothing and answers the auditor’s first question without a log search. The list form matters for per-prompt routers, since one change can have several authors.</p>
<p>A router whose pool you can’t see is also a data question: your prompt goes somewhere you can’t name. Put it through <a href="/intel/vendor-lane-last-hop-register/">the last-hop register</a> and cap the lane’s data class at what you can verify about where the request ends up.</p>
<h2 id="where-a-hosted-llm-router-leaks-out-of-change-control-and-the-signal-for-each">Where a hosted LLM router leaks out of change control, and the signal for each</h2>
<p><strong>One PR, several models.</strong> Copilot evaluates each prompt and Cursor’s router classifies each agent request, so a change built over many turns has more than one author. Signal: the per-response log shows two or more served models for one PR while the PR record lists one. Fix: attribution is a list, in the record and the trailer.</p>
<p><strong>Displayed, never captured.</strong> The model name scrolled past in the CLI and nothing stored it. Signal: rows with a null served model on a Copilot lane. Fix: a capture step with a fixture test, rerun after every CLI upgrade; a failed capture downgrades that response to <code>not_exposed</code> and the fence applies.</p>
<p><strong>The toggle nobody turned on.</strong> Cursor ships with the routed model hidden. Signal: Balance or Intelligence responses without the model line at the top. Fix: an admin turns on Underlying model for change-controlled teams, and Cost-mode lanes are scored not exposed until the docs say otherwise.</p>
<p><strong>The tier treated as a budget.</strong> Signal: spend per task rises on a lane whose tier never changed. Fix: caps in your budget layer and a by-model split in the monthly close.</p>
<p><strong>Pool drift under a stable name.</strong> Signal: cost per merged PR or pass rate on the frozen set moves with no config change, or a new model name appears. Fix: rerun the eval and re-approve the lane before it takes new work.</p>
<p><strong>A benchmark budgeted as savings.</strong> Signal: your task set shows a fraction of the published gain. Fix: budget from your own control arm; published results are a ceiling, not a forecast.</p>
<h2 id="the-model-name-is-fleet-evidence-not-a-vendor-setting">The model name is fleet evidence, not a vendor setting</h2>
<p>Routers will keep getting better at picking models, and some of them will keep hiding the pick because hiding it makes the product simpler to sell. That is the vendor’s call. Whether a hidden pick may touch your payment service is yours, and it gets made in the layer that launches lanes, records what ran and refuses what policy forbids.</p>
<p>That layer is what a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> is once you strip the dashboard off it: one place where every lane, whichever CLI or router it runs through, writes the same record, including the field most vendors leave empty. Buy the router that fills it. Fence the one that doesn’t.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-copilot-auto-show-which-model-answered">Does Copilot Auto show which model answered?</h3>
<p>Yes, per response. GitHub’s docs say the model appears on hover in Copilot Chat, prints in the terminal in Copilot CLI, shows at the end of a cloud agent response and sits beside Auto in the Copilot app. The docs do not describe an export or API log of routed models, so capture it yourself.</p>
<h3 id="can-you-pin-a-model-inside-cursor-auto">Can you pin a model inside Cursor Auto?</h3>
<p>No. Cursor’s router docs say you can’t hand-pick the model that handles a request, and the pool changes as new models ship. Admins can show the routed model through the Underlying model setting. To control the model, select one directly instead of Auto, unless an admin has imposed Auto in Hard mode.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://sakana.ai/fugu-max-release/">Sakana AI: Introducing Fugu Max and Fugu Ultra v2</a> — Sep 11, 2026; Fugu Max at $2/$6 per million tokens; models absent from Ultra v2’s pool</li>
<li><a href="https://vercel.com/ai-gateway/models/fugu-ultra-v2">Vercel AI Gateway: Fugu Ultra v2 listing</a> — $5/$30, cache read $0.50/M, second tier from 272,001 tokens</li>
<li><a href="https://github.blog/changelog/2026-09-14-configure-cost-and-quality-in-copilot-auto-model-selection/">GitHub Changelog: Configure cost and quality in Copilot auto model selection</a> — Sep 14, 2026; tiers, billing at the selected model, 10% discount</li>
<li><a href="https://docs.github.com/en/copilot/concepts/models/auto-model-selection">GitHub Docs: Auto model selection</a> — per-response model visibility by surface; evaluation models; admin exclusions</li>
<li><a href="https://cursor.com/docs/models-and-pricing">Cursor Docs: Models and pricing</a> — Auto bills the routed model’s list price; $0.25/M Cursor Token Rate incl. BYOK</li>
<li><a href="https://cursor.com/docs/cursor-router">Cursor Docs: Cursor Router</a> — classifier per request, no hand-pick, Underlying model setting hidden by default</li>
<li><a href="https://github.blog/ai-and-ml/github-copilot/project-hydrafusion-frontier-quality-via-multi-model-orchestration/">GitHub Blog: Project HydraFusion</a> — Sep 4, 2026; Single, Cascade, Critique; offline results; internal per-leg records</li>
</ul>
]]></content:encoded></item><item><title>Jev Soft-Ranks Skills — Agents Still Decide</title><link>https://automater.ai/intel/jev-skill-ranking-agent-decides/</link><guid>https://automater.ai/intel/jev-skill-ranking-agent-decides/</guid><description>Jev skill selection without hiding skills: gate on need, shortlist three, inject one suggestion line, let the agent decide, and log every override it makes.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>Sixty characters. That is how much of each skill’s description an agent on the Hermes harness sees by default when it picks one of 182 skills for a turn. Pick the wrong one and the turn follows the wrong procedure with real tools; load one when nothing applies and the session carries instructions nobody asked for. TypeSafe’s cookbook for Jev skill selection puts a classifier in front of that pick and cuts both errors by more than half, while leaving the pick itself with the agent.</p>
<p>That last clause is the part worth copying. By Tuesday you can run the same shape on your own roster: every skill stays loaded, a gate asks whether the turn needs a skill at all, a ranker shortlists three, one suggestion line goes into the prompt, the agent makes the final call, and every override gets logged. You judge it on wrong and needless loads measured on your roster, and you keep safety where a ranking cannot reach it.</p>
<p>Chatbots suggest; agents act. A skill is a procedure the agent carries out with its tools, so a wrong load is a wrong runbook executed.</p>
<h2 id="what-typesafes-jev-skill-selection-cookbook-measured-and-the-sep-17-sync-that-grows-rosters">What TypeSafe’s Jev skill selection cookbook measured, and the Sep 17 sync that grows rosters</h2>
<p>TypeSafe’s <a href="https://docs.typesafe.ai/cookbooks/skill_suggestion">Skill suggestion cookbook</a> starts from a real catalog: 182 skills in 33 categories from NousResearch’s MIT-licensed Hermes agent, whose index cuts each description to 60 characters by default. The cookbook leaves that index alone and puts two Jev requests in front of each turn.</p>
<p>The first carries one Choice over all 182 skills plus three Nouls asking, from different angles, whether the turn needs a skill at all: action on the user’s own systems, a documented procedure, or an answer prose alone could give. The prose question points the other way, so the code flips it before averaging, and a mean under 0.30 means no suggestion. The second request re-reads the top three with full descriptions and the first 700 characters of each SKILL.md, asks a Choice among them plus a fit Noul per candidate, and drops the shortlist when the best fit is under 0.30.</p>
<p>The winner’s name goes into one line in its own block after the roster. The agent keeps its full index and its own judgment, and because the roster never changes, prefix caching over it still holds.</p>
<p>The published run: 488 single-turn requests, 315 covered by exactly one skill and 173 covered by none, answered by <code>claude-haiku-4-5-20251001</code> with suggestions from jev-1.12, rendered Jul 31, 2026, weeks before Jev’s <a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">public launch on Sep 15</a>. The agent alone loaded the wrong skill on 16.8% of covered requests and a needless one on 9.8% of uncovered ones. With the suggestion, 7.3% and 4.0%. Handed the right answer outright, it still erred on 2.5% and 1.2%: the floor no selection method gets under.</p>
<p>
<em>Screenshot: TypeSafe docs, “Skill suggestion - TypeSafe AI” (cookbook run rendered Jul 31, 2026), captured Sep 21, 2026.</em></p>
<p>Read those numbers as a reference implementation, not a forecast: one vendor-run pass, one agent model, an older Jev version, no independent replication. Claude Sonnet 5 wrote the covered requests from each skill’s own SKILL.md, which TypeSafe itself says makes them easier than real traffic. No token-savings measurement is published. The cookbook runs on Hermes; wiring the same flow into your own harness is custom work.</p>
<p>The roster side moved the same week. <a href="https://github.com/anthropics/claude-code/releases/tag/v2.1.275">Claude Code 2.1.275</a>, released Sep 17, 2026, syncs the skills and plugins enabled on a claude.ai account into terminal sessions signed in with it; <code>syncClaudeAiSkills: false</code> and <code>syncClaudeAiPlugins: false</code> opt out. Per the <a href="https://code.claude.com/docs/en/settings-reference">settings reference</a>, that covers interactive and non-interactive sessions, and a <code>false</code> in a committed <code>.claude/settings.json</code> is ignored, so a repository cannot opt out on the lane’s behalf. Somebody enabled each synced skill, just not necessarily the person who owns the lane.</p>
<aside class="social-embed" data-x-embed data-post-id="2101297116953891201">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 19, 2026 — A Spanish-language Jev explainer whose first example is an agent carrying dozens of skills, with Jev picking the one that fits the request.</p>
  <a class="social-embed__source" href="https://x.com/santtiagom_/status/2101297116953891201" target="_blank" rel="noopener noreferrer">View santtiagom_'s post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="a-skill-is-a-procedure-the-agent-runs-so-selection-errors-are-action-errors">A skill is a procedure the agent runs, so selection errors are action errors</h2>
<p>Roster overhead in context is covered in the <a href="/intel/context-engineering-playbook/">context engineering playbook</a>, and the bill for anything replayed every turn in <a href="/intel/memory-that-burns-quota/">memory that burns quota</a>. This piece is about the pick. A wrong load changes what the agent does next: the wrong release procedure, a checklist for another repo. A needless load turns a question that wanted an answer into a procedure that wants to run.</p>
<p>The tempting fix is to hide the roster and hand the agent the top few. That swaps an overridable mistake for an invisible one: when the ranker is wrong, the right skill is gone. Picking a skill from a request and a list is a call a person makes in seconds, which is why it passes <a href="/intel/jev-ten-second-rule-decision-seat/">the decision-seat tests</a>; writing the skill’s output would not.</p>
<h2 id="step-1-inventory-the-roster-each-lane-actually-loads">Step 1: Inventory the roster each lane actually loads</h2>
<p>Per lane, list every skill a session can load, where it came from, and how many characters of each description the harness index shows; that width, not the full SKILL.md, is what the agent picks from. Hash the list per session so you can tell when it changes.</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>What goes in it</th>
<th>Why it is there</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>skill</code></td>
<td>Name as the harness lists it</td>
<td>The Choice option and the log key</td>
</tr>
<tr>
<td><code>source</code></td>
<td>repo / user / plugin / account-synced</td>
<td>Synced entries arrived without a lane owner</td>
</tr>
<tr>
<td><code>index_chars</code></td>
<td>Description width the agent sees</td>
<td>The ranker should see what the agent sees</td>
</tr>
<tr>
<td><code>owner</code></td>
<td>A person, not a team alias</td>
<td>Who fixes a description that keeps losing</td>
</tr>
<tr>
<td><code>last_loaded</code></td>
<td>From session logs</td>
<td>Candidates for pruning</td>
</tr>
<tr>
<td><code>roster_hash</code></td>
<td>Per session</td>
<td>Every eval and log line ties to one roster</td>
</tr>
</tbody>
</table>
<p>Give account-synced skills their own rows and a decision per lane: keep sync on, or set the opt-out somewhere other than the committed project settings, which Claude Code ignores for these keys. For unattended lanes, prove which skills a headless run actually read with the <a href="/intel/headless-lane-reproducibility/">clean-versus-used differential run</a> rather than trusting the manifest.</p>
<p>Prune before you rank. <a href="https://code.claude.com/docs/en/changelog">Claude Code 2.1.261</a> (Sep 4) added <code>/skill-doctor</code>, which shows which loaded skills go unused and what they cost in context. A skill no lane has loaded in a month is a removal candidate, not a ranking problem.</p>
<h2 id="step-2-build-a-labeled-selection-set-from-your-own-turns">Step 2: Build a labeled selection set from your own turns</h2>
<p>The cookbook’s metrics are the right ones; its requests are not yours. Pull real first turns from session logs and label each one.</p>
<ul>
<li><strong>Covered:</strong> exactly one skill is the right load. Record which.</li>
<li><strong>Uncovered:</strong> no skill applies. Copy the cookbook’s mix of three kinds: everyday requests, technical requests with no matching skill, and specific requests your roster happens not to serve (85, 42 and 46 of its 173).</li>
<li><strong>Ambiguous:</strong> two skills could serve equally. Keep these out of the score and in a review pile; they are description problems, not ranking problems.</li>
</ul>
<p>Score two rates on the first response only, as the cookbook defines them. <strong>Wrong load</strong> is the share of covered turns where the first skill loaded was not the labeled one; loading nothing counts as a miss. <strong>Needless load</strong> is the share of uncovered turns where the agent loaded any skill. The first load is the first skill-read tool call (<code>skill_view</code> on Hermes; your harness has its own).</p>
<p>Run the agent alone for a baseline, then an oracle arm whose prompt names the correct skill. The gap between them is the most any ranker can win; the oracle’s rate is your floor. As an illustrative sizing, a few hundred covered turns and 150 or so uncovered ones keep a swing of a few turns from looking like a trend.</p>
<h2 id="step-3-gate-on-does-this-turn-need-any-skill-before-ranking">Step 3: Gate on “does this turn need any skill?” before ranking</h2>
<p>The gate is what fixes needless loads, and a ranker alone makes them worse, because a Choice always names something. Ask the need question as several Nouls and combine them in code.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># skill-gate.yaml: illustrative shape; question texts paraphrase the cookbook's gate</span></span>
<span class="line"><span style="color:#85E89D">model</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">jev-1.13.0</span><span style="color:#6A737D">              # pinned; the cookbook's published run used jev-1.12</span></span>
<span class="line"><span style="color:#85E89D">gate</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  questions</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    acts_on_user_system</span><span style="color:#E1E4E8">:               </span><span style="color:#9ECBFF">"The request asks for action on the user's own files, accounts, devices or services."</span></span>
<span class="line"><span style="color:#85E89D">    would_follow_documented_procedure</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"An expert would work from a specific written procedure or command set here."</span></span>
<span class="line"><span style="color:#85E89D">    prose_suffices</span><span style="color:#E1E4E8">:                    </span><span style="color:#9ECBFF">"A well-informed generalist could answer completely in writing, with no tools or docs."</span></span>
<span class="line"><span style="color:#85E89D">  inverted</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">prose_suffices</span><span style="color:#E1E4E8">]   </span><span style="color:#6A737D"># scored as 1 - p before averaging</span></span>
<span class="line"><span style="color:#85E89D">  combine</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">mean</span></span>
<span class="line"><span style="color:#85E89D">  suggest_nothing_below</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">0.30</span><span style="color:#6A737D">  # the cookbook's value; tune yours on the labeled set</span></span></code></pre>
<p>Three rules keep the gate honest.</p>
<ol>
<li><strong>Threshold on the Noul probability.</strong> Nouls carry no confidence value, so the probability is the only thing to threshold.</li>
<li><strong>Flip deliberately; never derive.</strong> The cookbook inverts one question on purpose. Do not compute “needs a skill” as one minus some other answer you happen to have: TypeSafe’s <a href="https://docs.typesafe.ai/model-jaggedness/jev-1.13">jev-1.13 jaggedness page</a> shows a question and its negation, asked as two Nouls, summing to 1.19.</li>
<li><strong>Tune the floor on both rates at once.</strong> Raising it cuts needless loads but strips suggestions from covered turns. Set it from your labeled set and version it in the policy file.</li>
</ol>
<h2 id="step-4-rank-the-whole-roster-to-three-in-the-same-call">Step 4: Rank the whole roster to three in the same call</h2>
<p>Put one Choice over the roster in the same request as the gate Nouls, with each skill’s index description as its criterion: the same text the agent sees. Take the three highest probabilities as the shortlist. One call carrying a Choice and several Nouls against one state is the pattern from <a href="/intel/jev-multi-question-policy-vector/">the decision-vector piece</a>: keyed answers, policy in code.</p>
<p>Two documented limits shape this call. Per the <a href="https://docs.typesafe.ai/api">API reference</a>, a Choice takes at most 255 options, so a larger roster needs a first pass by category or source, then a second Choice inside the winner. TypeSafe’s <a href="https://docs.typesafe.ai/models">models page</a> also sets two budgets that apply at once: 64K tokens for state plus every question, 32K for state plus the longest question. The roster Choice is usually the longest, so check state plus roster against 32K in code before sending; the cookbook’s 182-skill roster prompt ran 16,089 characters.</p>
<p>The state is the user’s latest request plus the short context the need questions require, filtered in code; the jaggedness page advises sending only the fields a question needs. A transcript is not a state.</p>
<h2 id="step-5-re-read-the-shortlist-before-suggesting-anything">Step 5: Re-read the shortlist before suggesting anything</h2>
<p>The first call ranks names and short descriptions. The second reads more: for each of the three, the full description and the opening of its SKILL.md (700 characters in the cookbook). Ask a Choice among the three and a per-candidate fit Noul, and drop the shortlist when the best fit falls under your floor.</p>
<p>The pairing is deliberate. A Choice is relative and names the best of three even when all three are poor; a Noul is absolute and can come back low for every candidate. The jaggedness page shows one question returning 0.22 as a Noul and 0.01 on “yes” as a yes/no Choice, so keep separate, versioned floors for the gate, the shortlist Choice and the fit Nouls.</p>
<p>
<em>Two calls, one line, one decision that stays with the agent. Every path that fails ends in no suggestion, and the permission wall sits under all of it.</em></p>
<h2 id="step-6-inject-one-suggestion-line-and-leave-the-roster-alone">Step 6: Inject one suggestion line and leave the roster alone</h2>
<p>All of that produces one line, in its own block, placed after the roster. An illustrative version:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="text"><code><span class="line"><span>&lt;skill_suggestion&gt;</span></span>
<span class="line"><span>Look at release-notes first for this request. Skip it if it does not match what the user asked.</span></span>
<span class="line"><span>&lt;/skill_suggestion&gt;</span></span></code></pre>
<p>The rules around that line are what make it a suggestion rather than a filter.</p>
<ul>
<li><strong>Never remove, reorder or rewrite roster entries.</strong> The roster is a stable prefix. The cookbook places the line after it so caching holds; a reordered roster breaks the cache on every turn.</li>
<li><strong>One name at most.</strong> A list of three is a second roster with worse descriptions.</li>
<li><strong>Nothing is a valid output.</strong> When the gate or the fit floor says no skill applies, inject nothing.</li>
<li><strong>The line grants nothing.</strong> Loading the suggested skill changes no permission, and declining it changes none either.</li>
<li><strong>The ranker has a deadline.</strong> The suggestion must exist before the turn starts, or the turn runs without one.</li>
</ul>
<p>That last rule is the whole failure path, and it is deliberately benign. A 429, a 529, a 422, a timeout or an answer that fails validation all end the same way: no suggestion, one log line with the reason, and an agent choosing unaided as it did last week.</p>
<p>Set the deadline from your own measured p95. TypeSafe’s three demo turns came back sub-second per call, but the cookbook client sets a 120-second timeout, twelve times the <a href="https://docs.typesafe.ai/sdk/python/api/constants">Python SDK’s 10-second default</a>: fine for a batch eval, wrong for a live turn. Retry budgets that outlast a deadline are covered in <a href="/intel/jev-gate-fail-mode-rate-limits/">the Jev fail-mode piece</a>.</p>
<h2 id="step-7-log-every-override-because-suggestions-also-break-turns">Step 7: Log every override, because suggestions also break turns</h2>
<p>The headline numbers hide one line of the cookbook’s own output. Of its 315 covered requests, the suggestion fixed 37 that the agent got wrong on its own and broke 7 that it had right. TypeSafe’s explanation is blunt: “A confident wrong suggestion is more persuasive than no suggestion at all”.</p>
<p>
<em>Screenshot: TypeSafe docs, “Skill suggestion - TypeSafe AI” (cookbook run rendered Jul 31, 2026), captured Sep 21, 2026.</em></p>
<p>That is the reason the agent keeps the final call, and the reason every disagreement is data. Write one record per turn; the JSON line after the table is illustrative.</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>What it records</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>lane</code>, <code>session</code>, <code>turn</code></td>
<td>Ties the record to the transcript</td>
</tr>
<tr>
<td><code>roster_hash</code>, <code>qset</code>, <code>policy</code></td>
<td>Which roster, question set and thresholds were live</td>
</tr>
<tr>
<td><code>model</code></td>
<td>The versioned ID the response reports</td>
</tr>
<tr>
<td><code>gate</code></td>
<td>Each need Noul and the mean</td>
</tr>
<tr>
<td><code>shortlist</code>, <code>fit</code></td>
<td>Top three with probabilities; fit Noul per candidate</td>
</tr>
<tr>
<td><code>suggested</code></td>
<td>A skill name, or <code>none</code> plus the reason (gate, fit, error, deadline)</td>
</tr>
<tr>
<td><code>first_load</code></td>
<td>What the agent actually loaded first</td>
</tr>
<tr>
<td><code>outcome</code></td>
<td>followed / overrode / declined / unprompted</td>
</tr>
<tr>
<td><code>label</code></td>
<td>Filled at review: which load was right</td>
</tr>
</tbody>
</table>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span><span style="color:#79B8FF">"ts"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"2026-09-22T10:14:03Z"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"lane"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"docs-bot"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"session"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"s-81c2"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"turn"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">1</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"roster_hash"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"sha256:4be1"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"qset"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"skill-suggest@2"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"policy"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"skill-policy@3"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"model"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"jev-1.13.0"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"gate"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"acts_on_user_system"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.81</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"would_follow_documented_procedure"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.74</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"prose_suffices"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.22</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"mean"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.78</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"shortlist"</span><span style="color:#E1E4E8">:[[</span><span style="color:#9ECBFF">"release-notes"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">0.46</span><span style="color:#E1E4E8">],[</span><span style="color:#9ECBFF">"changelog-audit"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">0.31</span><span style="color:#E1E4E8">],[</span><span style="color:#9ECBFF">"pr-description"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">0.12</span><span style="color:#E1E4E8">]],</span><span style="color:#79B8FF">"fit"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"release-notes"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.83</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"changelog-audit"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.41</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"pr-description"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.09</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"suggested"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"release-notes"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"first_load"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"changelog-audit"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"outcome"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"overrode"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"label"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">null</span><span style="color:#E1E4E8">}</span></span></code></pre>
<p>The four outcomes are the review queue: <strong>followed</strong> (loaded the suggestion), <strong>overrode</strong> (loaded another skill), <strong>declined</strong> (loaded nothing despite a suggestion) and <strong>unprompted</strong> (loaded a skill when none was suggested). Review overrides and unprompted loads weekly against labels. A right override is a ranker miss worth a description or criteria fix; a wrong one belongs to the agent. Pinning the version the record carries, and why replaying policy differs from asking Jev again, are in <a href="/intel/jev-decision-log-version-pin/">the Jev decision-log piece</a>.</p>
<h2 id="step-8-score-jev-skill-selection-on-your-roster-not-the-vendors">Step 8: Score Jev skill selection on your roster, not the vendor’s</h2>
<p>Re-run the labeled set with suggestions on: same agent model, same roster hash, pinned Jev version. Report both error rates for all three arms, plus the fixed and broken counts from the log.</p>
<table>
<thead>
<tr>
<th>Arm</th>
<th>Wrong load</th>
<th>Needless load</th>
<th>Fixed</th>
<th>Broke</th>
</tr>
</thead>
<tbody>
<tr>
<td>TypeSafe cookbook, agent alone</td>
<td>16.8%</td>
<td>9.8%</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>TypeSafe cookbook, with suggestion</td>
<td>7.3%</td>
<td>4.0%</td>
<td>37 of 315</td>
<td>7 of 315</td>
</tr>
<tr>
<td>TypeSafe cookbook, oracle</td>
<td>2.5%</td>
<td>1.2%</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>Your lane, agent alone</td>
<td>measure</td>
<td>measure</td>
<td>—</td>
<td>—</td>
</tr>
<tr>
<td>Your lane, with suggestion</td>
<td>measure</td>
<td>measure</td>
<td>count</td>
<td>count</td>
</tr>
<tr>
<td>Your lane, oracle</td>
<td>measure</td>
<td>measure</td>
<td>—</td>
<td>—</td>
</tr>
</tbody>
</table>
<p>
<em>Vendor-run on jev-1.12 and Haiku 4.5 over 488 requests. The suggestion cuts both errors by more than half; the oracle row shows the floor is not zero.</em></p>
<p>Ship suggestions on a lane when both rates fall, the broken count is small beside the fixed count, and every broken turn has a cause written next to it in the override log. Re-run the set on every Jev version bump and after any roster change the manifest shows.</p>
<p>Claim context savings only if you measured them. The cookbook measured selection accuracy, not tokens, and never tested loading fewer skills. For a savings figure, compare against your harness’s native lazy loading over the same set, with <code>/skill-doctor</code> or equivalent for per-skill context cost. Loading only the top-k skills is an unmeasured design that removes the agent’s ability to override; treat it as its own experiment.</p>
<h2 id="step-9-keep-safety-policy-outside-the-ranking">Step 9: Keep safety policy outside the ranking</h2>
<p>Skills are instructions, not permissions. A skill ranked first gets no more tool rights than one ranked last, and a skill the ranker never suggests can still be loaded, so nothing about what a skill may do belongs in ranking thresholds.</p>
<p>Tool permissions, deny rules and the sandbox decide what any loaded skill can execute; intake decides which skills reach the roster at all, the discipline in <a href="/intel/marketplace-hygiene-plugin-mcp/">marketplace hygiene for plugins and MCP</a>. A skill you would not run unsupervised comes off the roster or sits behind a permission wall. It is never merely down-ranked.</p>
<p>The same holds for adversarial input. The jaggedness page says content that argues for its own classification can move an answer, so a request written to favour a particular skill can move the suggestion. Because the suggestion grants nothing and the agent still chooses, that attack buys a nudge, not a capability. Keep such turns in your labeled set.</p>
<h2 id="where-jev-skill-selection-goes-wrong-and-the-signal-for-each">Where Jev skill selection goes wrong, and the signal for each</h2>
<p><strong>The persuasive wrong suggestion.</strong> The agent follows a confident miss it would have avoided alone. Signal: broken count rising while fixed holds, and right overrides trending toward zero, which means the agent stopped disagreeing, not that the ranker stopped erring.</p>
<p><strong>Roster drift under a stale eval.</strong> A sync or plugin update adds skills, and your scores describe a roster that no longer exists. Signal: <code>roster_hash</code> changes with no manifest change, or a shortlist names a skill the manifest lacks.</p>
<p><strong>Alias drift.</strong> An unpinned client rides <code>jev-latest</code>, which maps to jev-1.13.0 today and moves on the next release; TypeSafe advises pinning the versioned ID once thresholds are tuned. Signal: the gate-mean distribution shifts with no roster change.</p>
<p><strong>The roster outgrows the call.</strong> Past 255 options, or past 32K for state plus roster, the request breaks a documented limit. Signal: the suggestion rate drops to zero overnight and every <code>none</code> reads <code>error</code>.</p>
<p><strong>Explain-only turns still load skills.</strong> The gate floor is too low for your traffic. Signal: needless loads flat after the gate ships.</p>
<p><strong>The ranker becomes a gate.</strong> Someone adds a branch that hides a low-scoring skill. Signal: any code path where a ranking removes an option instead of suggesting one.</p>
<h2 id="skill-ranking-belongs-to-the-fleets-policy-layer">Skill ranking belongs to the fleet’s policy layer</h2>
<p>Nothing in this runbook asks an agent to choose more carefully. The roster manifest, pinned model, question set, thresholds, one-line template, deadline, override log and weekly review live outside every model and apply the same way on every lane, whichever CLI runs it. That is what makes the result auditable: one roster hash, one policy version, one record per turn saying what was suggested and what the agent did.</p>
<p>It is also why ranking sits beside kill switches, permissions and evidence in the layer that runs the fleet, the place a <a href="/intel/multi-agent-command-center/">multi-agent command center</a> describes. The ranker narrows attention; the session that can abort the work still makes the choice.</p>
<h2 id="faq">FAQ</h2>
<h3 id="does-jev-choose-which-skill-my-agent-loads">Does Jev choose which skill my agent loads?</h3>
<p>No. Jev ranks the roster and, when the gate and fit checks pass, one suggestion line goes into the prompt after the roster. The agent keeps its full skill index and makes the final call, and every override gets logged. When the ranker fails or misses its deadline, the agent chooses unaided.</p>
<h3 id="how-much-does-jev-reduce-wrong-skill-loads">How much does Jev reduce wrong skill loads?</h3>
<p>In TypeSafe’s own cookbook run over 488 requests, wrong loads fell from 16.8% to 7.3% with a suggestion, and needless loads from 9.8% to 4.0%. That was one vendor-run pass on jev-1.12 with Haiku 4.5 and no independent replication, so measure both rates on your own roster before trusting it.</p>
<h3 id="does-a-top-ranked-skill-get-more-tool-permissions">Does a top-ranked skill get more tool permissions?</h3>
<p>No. Skills are instructions, not permissions. A skill ranked first gets no more tool rights than one ranked last, and following or declining the suggestion changes no permission. Tool permissions, deny rules and the sandbox decide what any loaded skill can execute. A skill you would not run unsupervised is never merely down-ranked.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://docs.typesafe.ai/cookbooks/skill_suggestion">TypeSafe: Skill suggestion cookbook</a> — method, thresholds, results (jev-1.12, Jul 31, 2026)</li>
<li><a href="https://docs.typesafe.ai/model-jaggedness/jev-1.13">TypeSafe: Jev 1.13 jaggedness</a></li>
<li><a href="https://docs.typesafe.ai/api">TypeSafe: API reference</a> — question types, 255-option cap, error codes</li>
<li><a href="https://docs.typesafe.ai/models">TypeSafe: Models</a> — budgets, aliases, pinning</li>
<li><a href="https://docs.typesafe.ai/sdk/python/api/constants">TypeSafe Python SDK constants</a></li>
<li><a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">TypeSafe: Introducing System One models and Jev</a> — Sep 15, 2026</li>
<li><a href="https://github.com/anthropics/claude-code/releases/tag/v2.1.275">Claude Code v2.1.275 release</a> — Sep 17, 2026</li>
<li><a href="https://code.claude.com/docs/en/settings-reference">Claude Code settings reference</a></li>
<li><a href="https://code.claude.com/docs/en/changelog">Claude Code changelog</a> — 2.1.261</li>
</ul>
]]></content:encoded></item><item><title>When Jev Returns 429: Decide Fail-Open or Fail-Closed First</title><link>https://automater.ai/intel/jev-gate-fail-mode-rate-limits/</link><guid>https://automater.ai/intel/jev-gate-fail-mode-rate-limits/</guid><description>A Jev rate limit will stall your gates. Write each gate&apos;s fail mode first, budget both TypeSafe limits fleet-wide, and cap SDK retries inside the deadline.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>At 02:14 an unattended lane proposes <code>git push --force</code> to a release branch, and the Jev gate in front of it gets back <code>429 Too Many Requests</code>. What happens next was decided weeks ago, either by whoever wrote the gate or by nobody. If nobody, the SDK decided: it waits, retries, waits again, and the lane sits there looking busy.</p>
<p>A Jev rate limit is not an edge case this month. TypeSafe publishes two limits, says both are moving, and ships SDKs that retry quietly by default. So every Jev gate has three answers (yes, no and nothing yet), and this runbook is about the third. By Tuesday each gate carries a written fail mode, both limits have a fleet-wide budget, timeouts come from latency you measured, SDK retries fit inside them, and four drills prove the late, expired, queued and rules-fallback paths work.</p>
<p>Chatbots suggest; agents act. An agent waiting on a verdict is still holding the tool call it was about to make.</p>
<h2 id="sep-1521-jevs-rate-limits-are-two-numbers-that-can-move-without-notice">Sep 15–21: Jev’s rate limits are two numbers that can move without notice</h2>
<p>TypeSafe launched Jev on Sep 15, 2026, “available today in early access,” per its <a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">launch post</a>. On Sep 20 it opened access to everyone, no waitlist. More users on the same capacity is when limits move, and the <a href="https://docs.typesafe.ai/models">models page</a> says so.</p>
<p>Two limits apply: 250,000 tokens per second and 1,200 requests per minute, and a request over either returns <code>429 Too Many Requests</code>. A warning box adds that the limits are “adjusting dynamically” and “can change without notice” while TypeSafe serves heavy demand, lands new GPU capacity and lets in more users. The page does not say whether the limits apply per key, per account or across the service.</p>
<p>
<em>Screenshot: TypeSafe AI docs, “Models - TypeSafe AI” (undated), captured Sep 21, 2026.</em></p>
<p>The <a href="https://docs.typesafe.ai/api">API reference</a> lists four errors: 401, 422, 429 and 529. A 429 is about your traffic: “You have exceeded your rate limit. Back off and retry after a short delay.” A 529 is about theirs: “TypeSafe is temporarily overloaded. Retry after a short delay.”</p>
<p>For both, the reference says the client SDKs back off automatically, so a client on its default retry policy needs no extra handling. Sound advice for a batch job; for a gate it is the problem, because the defaults are generous (step 4).</p>
<p>On latency, the launch post says “End-to-end response time is 70ms-500ms for TypeSafe,” and notes that its published evals are generally run from laptops on the West Coast, where the service is based. A vendor-reported range from one coast is not an SLA, and it is not your p99.</p>
<aside class="social-embed" data-x-embed data-post-id="2101786156572823624">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 20, 2026 — TypeSafe announces that Jev is open to everyone with no waitlist, five days after launching it in early access.</p>
  <a class="social-embed__source" href="https://x.com/typesafeai/status/2101786156572823624" target="_blank" rel="noopener noreferrer">View TypeSafe AI's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="why-a-jev-gate-needs-its-fail-mode-before-its-threshold">Why a Jev gate needs its fail mode before its threshold</h2>
<p>A classifier gate sits between an actor and an action. When the verdict doesn’t arrive, something still happens to the action: it waits, it proceeds, or it stops. If you haven’t chosen, the harness or the SDK chooses: on Claude Code, a command hook that times out doesn’t block, and the call continues through the normal permission flow. A TypeSafe JavaScript client on default settings can hold one call for minutes.</p>
<p><a href="/intel/fleet-continuity-provider-cutoff/">Rehearsing a provider cutoff</a> already separates timeouts from quota and drills the move to another model. This is the narrower case: the model is up and the verdict still doesn’t come in time. <a href="/intel/pre-action-gates-not-dashboards/">Where gates belong</a> is settled too. What follows is what a gate does when it can’t decide.</p>
<h2 id="step-1-write-each-jev-gates-fail-mode-before-go-live">Step 1: Write each Jev gate’s fail mode before go-live</h2>
<p>One row per gate, in the same file as its thresholds, reviewed before the gate enforces anything. “Fail open” and “fail closed” are too coarse for the four seats Jev usually takes, so name the action that runs when no verdict arrives.</p>
<table>
<thead>
<tr>
<th>Gate</th>
<th>Fail mode when no verdict arrives</th>
<th>What actually happens</th>
<th>Why</th>
</tr>
</thead>
<tbody>
<tr>
<td>Write and destructive tool calls</td>
<td><strong>Closed</strong></td>
<td>Deny with a reason; unattended lanes pause the task for a person</td>
<td>A wrong deny costs a retry; a wrong pass costs the branch</td>
</tr>
<tr>
<td>Read-only tool calls (Jev as an extra veto)</td>
<td>Open to static rules</td>
<td>No Jev opinion; deny and ask rules, the prompt and the sandbox still decide</td>
<td>The veto was an extra; the wall behind it is unchanged</td>
</tr>
<tr>
<td>Model-tier routing</td>
<td>Fixed default route</td>
<td>One named route, the same every time, logged as a fallback</td>
<td>A known route beats a guess</td>
</tr>
<tr>
<td>Alert and ticket triage</td>
<td>Deterministic rules</td>
<td>Owner map and keyword rules; output tagged for re-triage</td>
<td>Triage can be coarse for an hour; it can’t stop</td>
</tr>
</tbody>
</table>
<p>OpenRouter’s <a href="https://openrouter.ai/docs/cookbook/building-agents/gate-tool-calls-with-jev">gate cookbook</a> makes the same call for the closed row: its client throws on a non-2xx response, a missing answer or a probability outside 0 to 1, “so a broken check never turns into an approval or a review.”</p>
<p>Three rules keep the table honest.</p>
<p><strong>Open never means allow.</strong> A gate that fails open returns no opinion, which leaves the call to whatever sat behind the gate before Jev arrived. If nothing sat behind it, you don’t have a fail-open gate. You have an unguarded tool with a latency budget.</p>
<p><strong>The default route is fixed, logged and cleared for every data class.</strong> The fallback doesn’t know which repo it carries, so its route must be allowed for the most sensitive one. Check it against the <a href="/intel/vendor-lane-last-hop-register/">last-hop register</a>: the default is a lane whose last hop you have verified for every data class you route. It also never moves work down a tier: destructive and long-horizon tasks stay where <a href="/intel/jev-cheap-model-router-escalation/">the router’s never-route list</a> pins them, so a 429 can send work up or sideways, never down. If no single lane is cleared for everything, a class without a cleared route fails closed: the task waits.</p>
<p><strong>Rules exist before they are needed.</strong> The triage fallback is a file of deterministic rules (paths to owners, severity keywords, a catch-all queue), tested on go-live day, not drafted during the outage.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># gates.yaml (illustrative shape); enforced by the gate wrapper, never by a prompt</span></span>
<span class="line"><span style="color:#85E89D">gates</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  tool_destructive</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    classes</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">write</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">destructive</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">    on_no_verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">deny</span></span>
<span class="line"><span style="color:#85E89D">    unattended</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">pause_task</span><span style="color:#6A737D">        # a person picks it up; see step 5</span></span>
<span class="line"><span style="color:#85E89D">  tool_read</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    classes</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">read</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">    on_no_verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">no_opinion</span><span style="color:#6A737D">     # static rules and the sandbox still apply</span></span>
<span class="line"><span style="color:#85E89D">  router</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    on_no_verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">route</span></span>
<span class="line"><span style="color:#85E89D">    default_route</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">cleared-all-classes</span></span>
<span class="line"><span style="color:#85E89D">    never_down_tier</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">destructive</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">long_horizon</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#85E89D">  triage</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    on_no_verdict</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">rules</span></span>
<span class="line"><span style="color:#85E89D">    rules_file</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">triage-rules.yaml</span></span>
<span class="line"><span style="color:#85E89D">    tag</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">rules-fallback</span><span style="color:#6A737D">           # re-triage when Jev is back</span></span></code></pre>
<p>Thresholds, the part that turns a verdict into a decision, live in <a href="/intel/jev-confidence-gates-tool-calls/">the Jev confidence-gate table</a>. This file covers only the case where there is no verdict to threshold.</p>
<h2 id="step-2-budget-both-jev-rate-limits-fleet-wide">Step 2: Budget both Jev rate limits fleet-wide</h2>
<p>1,200 requests per minute is 20 a second. 250,000 tokens per second divided by 20 requests is 12,500 tokens each. That is the crossover: below about 12.5K tokens per request the request limit binds first, and above it the token limit does. A gate that sends 30K-token requests, inside the 32K budget for state plus the longest question, runs out at 500 requests a minute, not 1,200.</p>
<p>
<em>Derived from TypeSafe’s two published limits as of Sep 21, 2026; both can change without notice. Your share of the pool is smaller than any bar.</em></p>
<p>Plan as if every lane shares both limits. TypeSafe doesn’t say whether they apply per key or per account, so a key per lane isolates nothing you can confirm. Allocate one pool.</p>
<ol>
<li>List every Jev caller: tool gates, the router, triage, and any backfill or eval job on the same account.</li>
<li>For each, estimate peak requests per minute and tokens per request (state plus all questions).</li>
<li>Compute each caller’s binding share: requests divided by 1,200, or tokens per minute divided by 15,000,000 (250,000 × 60), whichever is larger.</li>
<li>Sum the shares and plan to at most half of the published numbers, an illustrative margin for numbers that move.</li>
<li>Give batch work the leftovers, never the gates’ share.</li>
</ol>
<table>
<thead>
<tr>
<th>Caller (illustrative)</th>
<th>Peak req/min</th>
<th>Tokens per request</th>
<th>Request share</th>
<th>Token share</th>
<th>Binding share</th>
</tr>
</thead>
<tbody>
<tr>
<td>Destructive tool gate</td>
<td>120</td>
<td>3,000</td>
<td>10%</td>
<td>2.4%</td>
<td>10%</td>
</tr>
<tr>
<td>Model router</td>
<td>30</td>
<td>8,000</td>
<td>2.5%</td>
<td>1.6%</td>
<td>2.5%</td>
</tr>
<tr>
<td>Alert triage</td>
<td>60</td>
<td>20,000</td>
<td>5%</td>
<td>8%</td>
<td>8%</td>
</tr>
<tr>
<td>Ticket backfill (batch)</td>
<td>240</td>
<td>30,000</td>
<td>20%</td>
<td>48%</td>
<td>48%</td>
</tr>
<tr>
<td><strong>Total</strong></td>
<td></td>
<td></td>
<td></td>
<td></td>
<td><strong>68.5%, over a 50% plan</strong></td>
</tr>
</tbody>
</table>
<p>The batch job breaks the budget, which is normal: it gets throttled or moved to a quiet hour, and the gates keep their share. Enforce the split in your client with a token bucket per caller, because the server’s 429 doesn’t know which caller mattered.</p>
<p>Gateways add their own ways to run dry. Vercel’s <a href="https://vercel.com/changelog/typesafe-ai-jev-now-available-on-ai-gateway">launch changelog</a> says evaluation calls count toward gateway budgets, and OpenRouter’s native <a href="https://openrouter.ai/docs/api/api-reference/alphadecisions/submit-a-decisions-questions-and-answers-request">Decisions API</a> documents a 402 when credits run out, plus 413 and 524. Map each to the same fail mode as a 429.</p>
<h2 id="step-3-derive-each-gates-timeout-from-your-measured-p99-and-the-actions-deadline">Step 3: Derive each gate’s timeout from your measured p99 and the action’s deadline</h2>
<p>TypeSafe publishes no p99. Measure your own: a week of real gate calls through the route you actually use (direct, Vercel, OpenRouter or Cloudflare, from your lanes’ region), latency recorded for every call, 429s included. Take p99 per route; re-measure when you change door or region.</p>
<p>Then work backward from the action.</p>
<ul>
<li><strong>Action deadline.</strong> How long the gated action can wait before the wait is itself a failure. An interactive shell call can stall a second or two before a person notices; a router picks a tier before a task that will run for an hour; a triage gate feeds a paging SLA.</li>
<li><strong>Gate timeout.</strong> At most the action deadline minus the time needed to act on the verdict, and well inside the harness’s own hook timeout, so the gate always delivers its fail mode itself. Claude Code’s default hook timeout is 600 seconds, and a hook that runs past it doesn’t block; <a href="/intel/jev-claude-code-hook-not-mcp/">the hook contract</a> covers how the script exits 2 long before that.</li>
<li><strong>Retries that fit.</strong> A retry fits only if the timeout covers (retries + 1) × p99 plus backoff. If it doesn’t, the answer is zero retries, not a longer timeout.</li>
</ul>
<table>
<thead>
<tr>
<th>Gate (illustrative)</th>
<th>Action deadline</th>
<th>Your p99 (example)</th>
<th>Gate timeout</th>
<th>Retries that fit</th>
</tr>
</thead>
<tbody>
<tr>
<td>Destructive tool, interactive</td>
<td>3 s</td>
<td>450 ms</td>
<td>1.5 s</td>
<td>1</td>
</tr>
<tr>
<td>Destructive tool, unattended</td>
<td>10 s</td>
<td>450 ms</td>
<td>2 s</td>
<td>2</td>
</tr>
<tr>
<td>Router, before task start</td>
<td>30 s</td>
<td>600 ms</td>
<td>3 s</td>
<td>2</td>
</tr>
<tr>
<td>Alert triage</td>
<td>60 s</td>
<td>600 ms</td>
<td>5 s</td>
<td>2, then rules</td>
</tr>
</tbody>
</table>
<p>If your measured p99 doesn’t fit inside the action deadline at all, the gate can’t be synchronous on that route. Move it closer, shrink the state, or accept that the fail mode is the normal path and redesign the seat.</p>
<p>
<em>One clock per gate. Retries live inside the timeout, the timeout lives inside the deadline, and anything that misses the timeout runs the fail mode.</em></p>
<h2 id="step-4-cap-sdk-retry-time-inside-the-gate-timeout">Step 4: Cap SDK retry time inside the gate timeout</h2>
<p>The SDK defaults are documented, and none of them was written for a one-second gate.</p>
<table>
<thead>
<tr>
<th>Client</th>
<th>Retries</th>
<th>Backoff</th>
<th>Retried on</th>
<th>Per-attempt timeout</th>
<th>Total budget</th>
</tr>
</thead>
<tbody>
<tr>
<td>TypeSafe Python SDK</td>
<td>2</td>
<td>0.5 s doubling to 5 s, jitter 0.25; honours <code>Retry-After</code></td>
<td>408, 429, 500–599, connection and timeout errors</td>
<td>10 s per HTTP operation</td>
<td>30 s per SDK call</td>
</tr>
<tr>
<td>TypeSafe JS SDK</td>
<td>2</td>
<td>500 ms doubling to 5,000 ms, jitter 0.25; server delays honoured up to 60,000 ms</td>
<td>408, 429, 500–599, connection and timeout errors</td>
<td>10,000 ms</td>
<td>None</td>
</tr>
<tr>
<td>AI SDK <code>experimental_evaluate</code></td>
<td>2 (<code>maxRetries</code>)</td>
<td>Set by AI SDK core; no provider-side retry loop</td>
<td>Transient failures such as 429 and 529</td>
<td>Not stated in the reference</td>
<td>Your <code>abortSignal</code></td>
</tr>
</tbody>
</table>
<p>The <a href="https://docs.typesafe.ai/sdk/python/api/retries">Python retry policy</a> at least has a ceiling: “Total retry budget in seconds per SDK call, including the initial attempt and delays.” The JavaScript client’s <a href="https://docs.typesafe.ai/sdk/javascript/api/interfaces/TypeSafeClientConfig">config reference</a> is blunter about its timeout: “Timeout per attempt in milliseconds, without a total retry budget.” Its <a href="https://docs.typesafe.ai/sdk/javascript/api/interfaces/RetryPolicy">retry policy</a> accepts a server-requested delay of up to 60,000 ms. Our arithmetic on those defaults: three 10-second attempts plus up to 60 seconds of server-requested wait before each of the two retries is about 150 seconds for one gate decision. Python stops at 30 seconds. Both are far past a 1.5-second gate, and under a 600-second hook timeout the wait shows up as an agent that quietly stopped working.</p>
<p>
<em>Screenshot: TypeSafe AI docs, “Interface: TypeSafeClientConfig - TypeSafe AI” (undated), captured Sep 21, 2026.</em></p>
<p>Set the budget wherever each client lets you, then enforce one deadline around the whole call anyway. On Python, a <code>RetryPolicy</code> can be passed as <code>retry</code> on the client or per call:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#F97583">from</span><span style="color:#E1E4E8"> typesafe_sdk </span><span style="color:#F97583">import</span><span style="color:#E1E4E8"> RetryPolicy, TypeSafeClient</span></span>
<span class="line"></span>
<span class="line"><span style="color:#79B8FF">GATE_TIMEOUT_S</span><span style="color:#F97583"> =</span><span style="color:#79B8FF"> 1.5</span><span style="color:#6A737D">                  # from step 3 (illustrative)</span></span>
<span class="line"></span>
<span class="line"><span style="color:#E1E4E8">gate_client </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> TypeSafeClient(</span></span>
<span class="line"><span style="color:#FFAB70">    timeout</span><span style="color:#F97583">=</span><span style="color:#79B8FF">0.6</span><span style="color:#E1E4E8">,                      </span><span style="color:#6A737D"># per HTTP operation; default 10.0</span></span>
<span class="line"><span style="color:#FFAB70">    retry</span><span style="color:#F97583">=</span><span style="color:#E1E4E8">RetryPolicy(</span></span>
<span class="line"><span style="color:#FFAB70">        max_retries</span><span style="color:#F97583">=</span><span style="color:#79B8FF">1</span><span style="color:#E1E4E8">,                </span><span style="color:#6A737D"># default 2</span></span>
<span class="line"><span style="color:#FFAB70">        backoff_max</span><span style="color:#F97583">=</span><span style="color:#79B8FF">0.2</span><span style="color:#E1E4E8">,              </span><span style="color:#6A737D"># default 5.0</span></span>
<span class="line"><span style="color:#FFAB70">        timeout</span><span style="color:#F97583">=</span><span style="color:#79B8FF">GATE_TIMEOUT_S</span><span style="color:#E1E4E8">,       </span><span style="color:#6A737D"># whole call, delays included; default 30.0</span></span>
<span class="line"><span style="color:#E1E4E8">    ),</span></span>
<span class="line"><span style="color:#E1E4E8">)</span></span></code></pre>
<p>On the AI SDK path, which is how most Vercel users reach Jev, the <a href="https://ai-sdk.dev/docs/reference/ai-sdk-core/evaluate"><code>experimental_evaluate</code> reference</a> documents both knobs you need:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="ts"><code><span class="line"><span style="color:#F97583">import</span><span style="color:#E1E4E8"> { experimental_evaluate </span><span style="color:#F97583">as</span><span style="color:#E1E4E8"> evaluate } </span><span style="color:#F97583">from</span><span style="color:#9ECBFF"> 'ai'</span><span style="color:#E1E4E8">;</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">const</span><span style="color:#79B8FF"> GATE_TIMEOUT_MS</span><span style="color:#F97583"> =</span><span style="color:#79B8FF"> 1500</span><span style="color:#E1E4E8">;          </span><span style="color:#6A737D">// from step 3 (illustrative)</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">const</span><span style="color:#79B8FF"> result</span><span style="color:#F97583"> =</span><span style="color:#F97583"> await</span><span style="color:#B392F0"> evaluate</span><span style="color:#E1E4E8">({</span></span>
<span class="line"><span style="color:#E1E4E8">  model: </span><span style="color:#9ECBFF">'typesafe-ai/jev'</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#E1E4E8">  state,</span></span>
<span class="line"><span style="color:#E1E4E8">  questions,</span></span>
<span class="line"><span style="color:#E1E4E8">  maxRetries: </span><span style="color:#79B8FF">1</span><span style="color:#E1E4E8">,                       </span><span style="color:#6A737D">// default 2</span></span>
<span class="line"><span style="color:#E1E4E8">  abortSignal: AbortSignal.</span><span style="color:#B392F0">timeout</span><span style="color:#E1E4E8">(</span><span style="color:#79B8FF">GATE_TIMEOUT_MS</span><span style="color:#E1E4E8">),</span></span>
<span class="line"><span style="color:#E1E4E8">});</span></span>
<span class="line"><span style="color:#6A737D">// an abort throws; the gate wrapper maps it to the fail mode</span></span></code></pre>
<p>On the TypeSafe JS client, set <code>timeout</code> per attempt, <code>retry.maxRetries</code>, <code>retry.backoffMaxMs</code> and a low <code>retry.maxRetryAfterMs</code>, and still wrap the call in your own deadline, because the client has no total budget.</p>
<p>A <code>Retry-After</code> longer than the time left is itself an answer: run the fail mode now rather than sleep. The Python client already stops before a retry whose delay would reach the budget and re-raises the last error. And the docs don’t say whether a retried request is billed twice (no idempotency key exists), so count attempts, not decisions, when you cost a gate.</p>
<h2 id="step-5-treat-a-late-verdict-as-evidence-never-as-a-decision">Step 5: Treat a late verdict as evidence, never as a decision</h2>
<p>Put the order of checks in one wrapper so no two gates implement them differently.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#6A737D"># gate wrapper (illustrative); every path writes exactly one record</span></span>
<span class="line"><span style="color:#F97583">def</span><span style="color:#B392F0"> gate</span><span style="color:#E1E4E8">(action, state, questions, policy, bucket, breaker, log):</span></span>
<span class="line"><span style="color:#E1E4E8">    rec </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> log.open(action.id, policy.version)</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> breaker.is_open() </span><span style="color:#F97583">or</span><span style="color:#F97583"> not</span><span style="color:#E1E4E8"> bucket.take():          </span><span style="color:#6A737D"># shed load before calling</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#E1E4E8"> rec.fail_mode(</span><span style="color:#9ECBFF">"shed"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#E1E4E8">    deadline </span><span style="color:#F97583">=</span><span style="color:#79B8FF"> min</span><span style="color:#E1E4E8">(action.deadline, now() </span><span style="color:#F97583">+</span><span style="color:#E1E4E8"> policy.gate_timeout)</span></span>
<span class="line"><span style="color:#F97583">    try</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#E1E4E8">        answer </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> call_jev(state, questions, deadline)   </span><span style="color:#6A737D"># retries capped inside (step 4)</span></span>
<span class="line"><span style="color:#F97583">    except</span><span style="color:#E1E4E8"> (RateLimited, Overloaded, Timeout, OutOfBudget, BadAnswer) </span><span style="color:#F97583">as</span><span style="color:#E1E4E8"> err:</span></span>
<span class="line"><span style="color:#E1E4E8">        breaker.record(err)</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#E1E4E8"> rec.fail_mode(err.kind)</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> now() </span><span style="color:#F97583">&gt;</span><span style="color:#E1E4E8"> deadline:</span></span>
<span class="line"><span style="color:#E1E4E8">        rec.late(answer)                                </span><span style="color:#6A737D"># kept as evidence, never applied</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#E1E4E8"> rec.fail_mode(</span><span style="color:#9ECBFF">"late"</span><span style="color:#E1E4E8">)</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#F97583"> not</span><span style="color:#E1E4E8"> action.still_pending() </span><span style="color:#F97583">or</span><span style="color:#E1E4E8"> action.state_hash() </span><span style="color:#F97583">!=</span><span style="color:#E1E4E8"> hash_of(state):</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#E1E4E8"> rec.expired(answer)                      </span><span style="color:#6A737D"># it judged a call that is gone</span></span>
<span class="line"><span style="color:#F97583">    return</span><span style="color:#E1E4E8"> rec.decide(policy.apply(answer))</span></span></code></pre>
<p><strong>Late.</strong> The verdict arrived after the gate’s timeout. The fail mode runs; the verdict is logged with its latency and never applied, in either direction. A late “safe” applied after a deny would reopen a call the agent has moved past. A late “unsafe” on a read call that went ahead under a fail-open gate is a signal for review, not a rollback trigger.</p>
<p><strong>Expired.</strong> The verdict is about an action that no longer exists in that form: the tool call was cancelled, or the file changed after the state was sent. Bind every request to the action ID, its deadline and a hash of the state you sent, and drop any verdict whose binding doesn’t match the pending call.</p>
<p><strong>Queued.</strong> Under sustained 429s, don’t let gate calls pile up. A queue in front of a synchronous gate turns every verdict into a late one. Use admission control (the per-caller bucket from step 2) and a circuit breaker: after a burst of 429 or 529 responses, open the breaker for a fixed period, let every gate run its fail mode without calling Jev, then send one probe call to close it.</p>
<p>Every path writes one record. <a href="/intel/jev-decision-log-version-pin/">The Jev decision log</a> already carries which path fired (ok, retried, 429, 529, timeout or fallback) with retry count and latency; add <code>late</code>, <code>expired</code> and <code>shed</code> if yours lacks them. An unattended destructive call that failed closed pauses its task and waits for a person, and that wait needs an owner and an expiry. <a href="/intel/hitl-approval-queue-hygiene/">Approval-queue hygiene</a> covers both.</p>
<p>The wrapper is a guardrail, not the wall. If the gate process crashes or hangs, it can’t run its own fail mode, and on Claude Code a hook that exits 1 without valid JSON or times out lets the call continue through the normal permission flow. Deny rules and the sandbox behind the gate must hold on the day the wrapper doesn’t.</p>
<h2 id="step-6-drill-the-four-failure-paths-before-the-gate-enforces">Step 6: Drill the four failure paths before the gate enforces</h2>
<p>Point the gate wrapper at a local stub that returns whatever you tell it. Run the drills on a disposable lane, on every wrapper change and after every harness or SDK upgrade.</p>
<table>
<thead>
<tr>
<th>Drill</th>
<th>How to cause it</th>
<th>Expected</th>
<th>Signal in the record</th>
</tr>
</thead>
<tbody>
<tr>
<td>Late verdict</td>
<td>Stub answers 300 ms after the gate timeout</td>
<td>Fail mode fires at the timeout; the verdict is logged, never applied</td>
<td><code>late</code>, latency above timeout, outcome is the fail mode</td>
</tr>
<tr>
<td>Expired action</td>
<td>Stub delays; cancel the tool call or edit the file before it answers</td>
<td>Verdict dropped; nothing applied</td>
<td><code>expired</code>, state hash mismatch</td>
</tr>
<tr>
<td>Queue under 429</td>
<td>Stub returns 429 with a 30 s <code>Retry-After</code> to every call; replay five times peak traffic</td>
<td>No gate waits past its timeout; breaker opens; batch callers shed first</td>
<td><code>shed</code> and <code>429</code>, breaker-open events, gate latency flat</td>
</tr>
<tr>
<td>Switch to rules</td>
<td>Hold the breaker open for 15 minutes</td>
<td>Triage runs the rules file and tags output; router takes the default route; destructive gate denies</td>
<td><code>fallback</code>, <code>rules-fallback</code> tags, zero Jev calls</td>
</tr>
<tr>
<td>Recovery</td>
<td>Stub returns 200 again</td>
<td>One probe closes the breaker; tagged items re-triaged</td>
<td>breaker closed, re-triage count</td>
</tr>
</tbody>
</table>
<p>Add one-line drills for the door-specific failures: a 529, a revoked key (a 401 should page someone as well as deny), and an exhausted gateway budget or OpenRouter 402. Then run the drill no stub can fake: kill the gate process mid-call and confirm the deny rules and sandbox stop the destructive call on their own.</p>
<h2 id="how-a-jev-rate-limit-turns-into-a-stall-and-the-signal-for-each">How a Jev rate limit turns into a stall, and the signal for each</h2>
<p><strong>The stall that looks like work.</strong> A default JavaScript client honours a long <code>Retry-After</code> and the lane sits inside the gate. Signal: gate latency clustering at multiples of ten seconds, and sessions with a pending tool call and no progress. Fix: step 4’s budget plus your own deadline.</p>
<p><strong>The silent open.</strong> The gate’s deadline creeps past the harness hook timeout, the hook dies first, and the call proceeds. Signal: tool calls in transcripts with no gate record in front of them. Fix: one deadline constant, checked against the hook timeout when settings load.</p>
<p><strong>The noisy neighbour.</strong> A backfill takes the shared pool and the destructive gate starts failing closed. Signal: 429s on the gate while its own request rate is flat. Fix: per-caller buckets; batch work sheds first.</p>
<p><strong>The limit moved.</strong> TypeSafe lowers the numbers without notice. Signal: a rising 429 rate at a call rate that was clean last week. Fix: the 50% plan and a weekly 429-rate check per caller.</p>
<p><strong>The fallback route that wasn’t cleared.</strong> The router’s default sends a restricted repo to a lane whose last hop covers only public code. Signal: fallback records whose data class exceeds the route’s cap. Fix: one default route cleared for every class, or a deny for any class without one.</p>
<p><strong>Rules forever.</strong> Triage stays on the rules file after Jev recovers because nothing closes the breaker. Signal: the <code>rules-fallback</code> share doesn’t fall after the 429s stop. Fix: the half-open probe, drilled.</p>
<h2 id="a-fail-mode-is-fleet-policy-the-classifier-cant-write">A fail mode is fleet policy the classifier can’t write</h2>
<p>Jev gives you a fast verdict when it gives you one. What the fleet does in the other case is a decision about your tools, data classes and deadlines, and it belongs with the rest of fleet policy: a versioned file, one wrapper every lane uses, a breaker with an owner, drills on a calendar. That is the unglamorous core of <a href="/intel/agentic-ops/">running agents as a fleet</a>: the boring parts written down before the incident.</p>
<p>A classifier that answers in half a second is a good neighbour for that layer, not a substitute for it. Decide the fail mode first, then tune the threshold. The 429 arrives either way.</p>
<h2 id="faq">FAQ</h2>
<h3 id="what-are-jevs-rate-limits">What are Jev’s rate limits?</h3>
<p>TypeSafe’s models page lists 250,000 tokens per second and 1,200 requests per minute, and a request over either returns 429. The page says both are adjusting dynamically and can change without notice. It doesn’t say whether they apply per key or per account, so budget them as one shared pool.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://docs.typesafe.ai/models">TypeSafe: Models</a> — 250,000 tokens/s and 1,200 requests/min; either returns 429; limits “adjusting dynamically”</li>
<li><a href="https://docs.typesafe.ai/api">TypeSafe: API reference</a> — 401, 422, 429 and 529; exponential backoff guidance</li>
<li><a href="https://docs.typesafe.ai/sdk/python/api/retries">TypeSafe Python SDK: Retries</a> — <code>RetryPolicy</code> defaults, 30 s total budget per SDK call</li>
<li><a href="https://docs.typesafe.ai/sdk/javascript/api/interfaces/RetryPolicy">TypeSafe JS SDK: RetryPolicy</a> — two retries, 500 to 5,000 ms backoff, server delays honoured up to 60,000 ms</li>
<li><a href="https://docs.typesafe.ai/sdk/javascript/api/interfaces/TypeSafeClientConfig">TypeSafe JS SDK: TypeSafeClientConfig</a> — 10,000 ms per attempt, no total retry budget</li>
<li><a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">TypeSafe: Introducing System One Models &amp; Jev</a> — Sep 15, 2026; early access at launch; 70–500 ms measured from the West Coast</li>
<li><a href="https://ai-sdk.dev/docs/reference/ai-sdk-core/evaluate">AI SDK: <code>experimental_evaluate</code></a> — <code>maxRetries</code> (default 2) and <code>abortSignal</code></li>
<li><a href="https://vercel.com/changelog/typesafe-ai-jev-now-available-on-ai-gateway">Vercel changelog: Jev on AI Gateway</a> — Sep 16, 2026; evaluation calls count toward budgets</li>
<li><a href="https://openrouter.ai/docs/api/api-reference/alphadecisions/submit-a-decisions-questions-and-answers-request">OpenRouter: Decisions API reference</a> — 402 on exhausted credits, 413, 524 and 529</li>
<li><a href="https://openrouter.ai/docs/cookbook/building-agents/gate-tool-calls-with-jev">OpenRouter cookbook: Gate agent tool calls with Jev</a> — a broken check “never turns into an approval or a review”</li>
</ul>
]]></content:encoded></item><item><title>Log Every Jev Decision — and Pin the Version the Record Carries</title><link>https://automater.ai/intel/jev-decision-log-version-pin/</link><guid>https://automater.ai/intel/jev-decision-log-version-pin/</guid><description>A Jev decision log needs one record per call: state reference, versioned questions, answers, reported model and policy. Pin jev-1.13.0, never jev-latest.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p><em>swap if/else prompt soup for typed decisions the session bus can log.</em></p>
<p>Sooner or later someone asks why a gate let a write through, and they arrive with a transcript, a diff and a deadline. What they need is a Jev decision log: one record per call saying what state the classifier saw, which exact questions it answered, what it returned, which model version answered and which policy turned that into a verdict. Without it, the only move left is to ask the model again, and the model you ask may not be the one that answered.</p>
<p>This is the runbook for making that record boring. By Tuesday every Jev call writes one record with the gated tool call and session attached.</p>
<p>The state sits behind an access-controlled reference, hash beside it. The questions carry a version, not just a key. The model is pinned to <code>jev-1.13.0</code> rather than an alias, with a plan for the day it goes away. And your team knows that replaying policy over a stored answer is deterministic and asking Jev again is not.</p>
<p>Agents made this a records problem. A gate’s score is a number code acts on within a second, between an agent and a shell or a task and a model tier. Typed answers are the good news: a probability under a key you chose can be logged, diffed and replayed in a way model prose never could. The record is how you collect on that.</p>
<h2 id="sep-1121-aliases-dated-ids-and-a-model-swap-under-an-unchanged-name">Sep 11–21: aliases, dated IDs and a model swap under an unchanged name</h2>
<p>TypeSafe’s <a href="https://docs.typesafe.ai/models">models page</a> lists one Jev version, <code>jev-1.13.0</code>, behind two aliases. <code>jev-latest</code> points at it and is the SDK default; <code>jev-preview</code> points at the same build, and no preview build exists right now. The page is candid about the cost: an alias moves when a new release ships, so answers can change with no change on your side, while the response’s <code>model</code> field reports the versioned ID that answered. Once you have tuned thresholds against a version, it says to “pin that version’s ID instead of the alias”.</p>
<p>
<em>Screenshot: TypeSafe AI, “Models - TypeSafe AI” (undated), captured Sep 21, 2026.</em></p>
<p><code>GET /v1/models</code> lists only the aliases; versioned IDs are accepted anyway. What the page doesn’t publish is a retirement policy: no support window for a pinned version, no notice period, no changelog from 1.12 to 1.13. TypeSafe promises nothing either way, which makes the plan for losing <code>jev-1.13.0</code> yours to write.</p>
<p>The doors report versions differently: TypeSafe direct and Cloudflare report <code>jev-1.13.0</code>. OpenRouter’s <a href="https://openrouter.ai/docs/guides/community/typesafe-sdk">TypeSafe SDK guide</a> routes <code>jev-1.13</code> as <code>typesafe/jev-1.13</code> and <code>jev-latest</code> as <code>~typesafe/jev-latest</code>, the tilde marking the alias, and its sample responses report a dated <code>typesafe/jev-1.13-20260917</code>. It doesn’t say whether that dated string, or TypeSafe’s three-part <code>jev-1.13.0</code>, is accepted in a request. Vercel AI Gateway’s <a href="https://vercel.com/docs/ai-gateway/sdks-and-apis/typesafe">TypeSafe-compatible docs</a> use <code>typesafe-ai/jev</code> in every request and sample response, and the gateway lists no versioned Jev.</p>
<aside class="social-embed" data-x-embed data-post-id="2100744709589316009">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 18, 2026 — OpenRouter's launch post for Jev, which it calls a beta, presenting it as a System One model that answers with a typed decision and a probability rather than text.</p>
  <a class="social-embed__source" href="https://x.com/OpenRouter/status/2100744709589316009" target="_blank" rel="noopener noreferrer">View OpenRouter's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<p>The precedent belongs to another vendor. On Sep 11, <a href="https://www.kimi.com/code/docs/en/kimi-code/whats-new.html">Kimi’s changelog</a> announced K2.8 Preview fully rolled out in Kimi Code with the model ID unchanged, still <code>kimi-for-coding</code>, so clients needed no configuration changes. Nothing went wrong, and the note was explicit about the swap. But a lane that logged only <code>kimi-for-coding</code> can’t tell from its records which model answered on Sep 10 and which on Sep 12.</p>
<p>The clients moved too. TypeSafe’s <a href="https://docs.typesafe.ai/sdk/javascript/api/interfaces/TypeSafeClientConfig">JavaScript client config</a> falls back to <code>TYPESAFE_DEFAULT_MODEL</code>, then <code>jev-latest</code>, when a call names no model; the Python default is also <code>jev-latest</code>. The <a href="https://docs.typesafe.ai/sdk/python/changelog">Python SDK changelog</a> lists breaking releases on Sep 15 (0.6.0) and Sep 18 (0.7.0), and 0.7.1 landed today.</p>
<h2 id="a-gate-log-line-isnt-a-classifier-record-yet">A gate log line isn’t a classifier record yet</h2>
<p><a href="/intel/pre-action-gates-not-dashboards/">Pre-action gates, not dashboards</a> already argues for one evidence line per gate decision, stamped with the policy version, and <a href="/intel/fleet-continuity-provider-cutoff/">the provider-cutoff rehearsal</a> already says to pin versions and keep the effective model in run records. Take both as settled.</p>
<p>A Jev gate adds three things those lines weren’t built for. Its verdict is a probability from a model that can change behind a name. Its question is prose someone will reword. And the obvious way to check an old decision, calling Jev again, returns a new sample, not the old answer.</p>
<h2 id="build-the-jev-decision-log-in-eight-steps">Build the Jev decision log in eight steps</h2>
<h3 id="step-1-write-one-record-per-call-on-every-path">Step 1: Write one record per call, on every path</h3>
<p>Every Jev call writes exactly one record, whether the gate allowed, denied, routed, timed out, hit a 429 or fell back. A log that exists only on the happy path is a sample of your successes.</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>What it holds</th>
<th>Why it’s there</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>decision_id</code>, <code>ts</code></td>
<td>Your ID and a UTC timestamp</td>
<td>The key everything else joins on</td>
</tr>
<tr>
<td><code>session_id</code>, <code>tool_call_id</code>, <code>lane</code></td>
<td>The agent session and the exact call the verdict gated</td>
<td>Joins the record to the transcript and the tool log</td>
</tr>
<tr>
<td><code>state_ref</code>, <code>state_sha256</code></td>
<td>A retrievable, access-controlled pointer to the exact state sent, plus its hash</td>
<td>The reference is the evidence; the hash only proves nobody changed it</td>
</tr>
<tr>
<td><code>question_set</code>, <code>question_sha256</code></td>
<td>Question set ID and version, and a hash of the full text, criteria and options sent</td>
<td>A key alone hides edits (step 3)</td>
</tr>
<tr>
<td><code>answers</code></td>
<td>Raw answers keyed by question ID: Noul <code>noul</code>; Choice <code>choice</code>, <code>probabilities</code>, <code>confidence</code>; Score <code>score</code>, <code>legend</code>, <code>probabilities</code>, <code>confidence</code></td>
<td>Noul carries a probability only</td>
</tr>
<tr>
<td><code>model_requested</code>, <code>model_reported</code>, <code>version_known</code></td>
<td>What you sent, what the response says answered, and whether that string names a version</td>
<td>The pin, and proof it held (step 4)</td>
</tr>
<tr>
<td><code>door</code>, <code>sdk</code></td>
<td>Endpoint used, and client library with exact version</td>
<td>Two breaking SDK releases in one week</td>
</tr>
<tr>
<td><code>request_ids</code></td>
<td><code>x-typesafe-request-id</code>; OpenRouter <code>id</code>; Vercel <code>generationId</code></td>
<td>The vendor’s handle for a support ticket</td>
</tr>
<tr>
<td><code>policy_version</code>, <code>verdict</code>, <code>outcome</code></td>
<td>Threshold table version, what the gate decided, what actually happened</td>
<td>Replay needs the policy; audits need the outcome</td>
</tr>
<tr>
<td><code>path</code></td>
<td><code>ok</code>, <code>retried</code>, <code>429</code>, <code>529</code>, <code>timeout</code> or <code>fallback</code>, with retry count and latency</td>
<td>Audits start on the failure paths</td>
</tr>
</tbody>
</table>
<p>TypeSafe’s <a href="https://docs.typesafe.ai/confidence">confidence docs</a> say Noul answers don’t carry a confidence, so a Noul row stores <code>noul</code> alone; a confidence on a Noul row means someone’s code invented it. <code>policy_version</code> names <a href="/intel/jev-confidence-gates-tool-calls/">the per-tool-class threshold table</a>, versioned like code. Field names differ by door, so normalize before you write: <a href="/intel/jev-endpoint-dialects-vercel-cloudflare/">the endpoint-dialect runbook</a> maps every door into one shape, and its <code>model_reported</code> and <code>version_known</code> names keep the records joinable.</p>
<p>An illustrative record, one JSON line per call (values made up):</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span><span style="color:#79B8FF">"decision_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"jd-000481"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"ts"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"2026-09-22T14:03:11Z"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"lane"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"billing-migration"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"session_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"s-7f3c"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"tool_call_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"tc-0192"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"state_ref"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"decisions-store://2026/09/22/jd-000481.json"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"state_sha256"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"9b1e..."</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"question_set"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"write-gate@7"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"question_sha256"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"41c0..."</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"answers"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"destructive"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"noul"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"noul"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.12</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"scope"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"choice"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"choice"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"repo_local"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"probabilities"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"repo_local"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.93</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"shared_infra"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.05</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"other"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.02</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"confidence"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.9</span><span style="color:#E1E4E8">}},</span><span style="color:#79B8FF">"model_requested"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"jev-1.13.0"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"model_reported"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"jev-1.13.0"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"version_known"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">true</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"door"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"typesafe"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"sdk"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"typesafe-sdk==0.7.1"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"request_ids"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"typesafe"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"req_..."</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"policy_version"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"gate-table@12"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"verdict"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"no_decision"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"outcome"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"ran"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"path"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"status"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"ok"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"retries"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"latency_ms"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">184</span><span style="color:#E1E4E8">}}</span></span></code></pre>
<p>Decide now what the gate does when the record can’t be written. For write and destructive classes, no record means no action: deny and alarm; for read-only calls, proceed and alarm. The log is evidence, not the wall; permission rules and the sandbox behind the gate keep working when the logger, the classifier or the hook fails, which is why they stay. What each gate does on a 429, a 529 or a timeout belongs in <a href="/intel/jev-gate-fail-mode-rate-limits/">the fail-mode table</a>, and <code>path</code> is where those choices show up afterward.</p>
<h3 id="step-2-store-the-state-as-a-reference-and-scrub-it-once">Step 2: Store the state as a reference, and scrub it once</h3>
<p>The state is the evidence; its hash isn’t. A SHA-256 proves a stored blob is the one you sent, but it can’t show an auditor what the gate saw. So the record points into a store with its own access control and retention at least as long as your audit window, with the hash beside it as an integrity check.</p>
<ol>
<li><strong>Scrub before you send, then store what you sent.</strong> Credentials and customer secrets come out once, before the call, so logged and sent state are the same bytes. Scrub only the logged copy and the hash stops matching.</li>
<li><strong>Keep it out of application logs.</strong> TypeSafe’s JavaScript client logs bodies at <code>debug</code> and redacts only known credential headers. A debug log in a shared sink is a leak with timestamps.</li>
<li><strong>Size it on purpose.</strong> Jev takes at most 64K tokens per request, 32K for state plus the longest question, so one blob is bounded; a year of them isn’t. Retain long for destructive gates, short for triage.</li>
</ol>
<p>If a lane switches on zero data retention, assume the vendor keeps nothing for you. Your store is the only way back.</p>
<h3 id="step-3-version-the-questions-not-just-their-keys">Step 3: Version the questions, not just their keys</h3>
<p>TypeSafe’s <a href="https://docs.typesafe.ai/api">API reference</a> is precise about keys: you choose each one, answers come back under it, and the key “is not sent to the underlying model”. Good for your code, bad for your records. The key <code>destructive</code> can front three different instructions in a month, and the model only ever saw the instructions.</p>
<p>So question text lives in a versioned file, the record stores the version plus a hash of the text sent, and CI refuses any change to instructions, criteria or options that doesn’t bump the version.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="yaml"><code><span class="line"><span style="color:#6A737D"># questions/write-gate.yaml (illustrative)</span></span>
<span class="line"><span style="color:#85E89D">id</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">write-gate</span></span>
<span class="line"><span style="color:#85E89D">version</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">7</span></span>
<span class="line"><span style="color:#85E89D">questions</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">  destructive</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    type</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">noul</span></span>
<span class="line"><span style="color:#85E89D">    instructions</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Would this command delete or overwrite data outside the working tree?"</span></span>
<span class="line"><span style="color:#85E89D">  scope</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">    type</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">choice</span></span>
<span class="line"><span style="color:#85E89D">    instructions</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Where does this command take effect?"</span></span>
<span class="line"><span style="color:#85E89D">    criteria</span><span style="color:#E1E4E8">:</span></span>
<span class="line"><span style="color:#85E89D">      repo_local</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Only files inside this repository"</span></span>
<span class="line"><span style="color:#85E89D">      shared_infra</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Shared infrastructure, cloud resources or other repositories"</span></span>
<span class="line"><span style="color:#85E89D">      other</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Anything else, or it cannot be told from the command"</span></span></code></pre>
<p>Hash the rendered request, not the file, so a template variable that changes the text changes the hash. Old answers stay valid records of the old question; they just aren’t evidence about the new one.</p>
<h3 id="step-4-record-the-model-the-response-reports-and-mark-the-doors-that-dont-say">Step 4: Record the model the response reports, and mark the doors that don’t say</h3>
<p><code>model_requested</code> is what you asked for. <code>model_reported</code> is the response’s <code>model</code> field: <code>jev-1.13.0</code> on TypeSafe direct and Cloudflare, <code>typesafe/jev-1.13-20260917</code> on OpenRouter. Store both verbatim, then set <code>version_known</code> to true when the reported string names a version and false when it’s a slug.</p>
<p>On Vercel AI Gateway the documented answer is false: sample responses report <code>typesafe-ai/jev</code>, and no page shows a way to pin a version. The docs don’t settle whether one exists, so write “not documented” in your door sheet and test it. Until a Vercel lane’s records can name a version, don’t give it a gate whose thresholds were tuned against one.</p>
<p>Alarm on novelty. A <code>model_reported</code> value your fleet has never seen is the cheapest drift detector you’ll build. On a pinned lane it should never fire; when it does, hold that lane’s gated actions and read the records.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="sql"><code><span class="line"><span style="color:#6A737D">-- illustrative: model strings not seen before, last 24 hours</span></span>
<span class="line"><span style="color:#F97583">SELECT</span><span style="color:#E1E4E8"> lane, model_reported, </span><span style="color:#79B8FF">count</span><span style="color:#E1E4E8">(</span><span style="color:#F97583">*</span><span style="color:#E1E4E8">) </span><span style="color:#F97583">AS</span><span style="color:#E1E4E8"> calls</span></span>
<span class="line"><span style="color:#F97583">FROM</span><span style="color:#E1E4E8"> jev_decisions</span></span>
<span class="line"><span style="color:#F97583">WHERE</span><span style="color:#E1E4E8"> ts </span><span style="color:#F97583">&gt;</span><span style="color:#F97583"> now</span><span style="color:#E1E4E8">() </span><span style="color:#F97583">-</span><span style="color:#E1E4E8"> interval </span><span style="color:#9ECBFF">'24 hours'</span></span>
<span class="line"><span style="color:#F97583">  AND</span><span style="color:#E1E4E8"> model_reported </span><span style="color:#F97583">NOT</span><span style="color:#F97583"> IN</span><span style="color:#E1E4E8"> (</span><span style="color:#F97583">SELECT</span><span style="color:#E1E4E8"> model_reported </span><span style="color:#F97583">FROM</span><span style="color:#E1E4E8"> jev_models_seen)</span></span>
<span class="line"><span style="color:#F97583">GROUP BY</span><span style="color:#E1E4E8"> lane, model_reported;</span></span></code></pre>
<h3 id="step-5-pin-the-model-the-default-and-the-sdk">Step 5: Pin the model, the default and the SDK</h3>
<p>Pin three things, not one:</p>
<ul>
<li><strong>The model on every call.</strong> Send <code>jev-1.13.0</code> to TypeSafe and <code>typesafe/jev-1.13</code> to OpenRouter, which uses author-prefixed IDs as-is. Never <code>jev-latest</code>, <code>jev-preview</code> or <code>~typesafe/jev-latest</code> on a lane that gates actions.</li>
<li><strong>The fallback.</strong> Set <code>TYPESAFE_DEFAULT_MODEL=jev-1.13.0</code> in every lane’s environment, so a call that forgets <code>model</code> lands on the pin. Both SDKs read it.</li>
<li><strong>The client.</strong> Lock <code>typesafe-sdk</code> and <code>@typesafe-ai/sdk</code> to exact versions and record them in <code>sdk</code>. A breaking client release changes the wire without touching the model: 0.6.0 made <code>Score.criteria</code> an ordered sequence.</li>
</ul>
<p>Then fail the build on a moving name:</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="bash"><code><span class="line"><span style="color:#6A737D"># illustrative CI check: no moving Jev names in lane config</span></span>
<span class="line"><span style="color:#F97583">if</span><span style="color:#B392F0"> rg</span><span style="color:#79B8FF"> -n</span><span style="color:#9ECBFF"> '(jev-latest|jev-preview|~typesafe/jev-latest)'</span><span style="color:#9ECBFF"> lanes/</span><span style="color:#9ECBFF"> config/</span><span style="color:#E1E4E8">; </span><span style="color:#F97583">then</span></span>
<span class="line"><span style="color:#79B8FF">  echo</span><span style="color:#9ECBFF"> "moving Jev alias in lane config; pin jev-1.13.0"</span><span style="color:#F97583"> &gt;&amp;2</span></span>
<span class="line"><span style="color:#79B8FF">  exit</span><span style="color:#79B8FF"> 1</span></span>
<span class="line"><span style="color:#F97583">fi</span></span></code></pre>
<p>
<em>TypeSafe has published no model change since launch; the code that calls it has changed four times. Dates from TypeSafe’s docs and SDK changelog, PyPI, Vercel, OpenRouter and Kimi, read Sep 21, 2026.</em></p>
<p>From Sep 11 to today TypeSafe published no model change, while the Python client shipped four releases, two of them breaking. The next change might be the model, and an unchanged name won’t warn you.</p>
<h3 id="step-6-replay-policy-over-stored-answers-re-evaluate-only-on-purpose">Step 6: Replay policy over stored answers; re-evaluate only on purpose</h3>
<p>Audits go wrong when these two operations are treated as one.</p>
<p><strong>Policy replay</strong> reruns your code over the stored answers. Tighten the destructive threshold, replay last month, and you get the exact calls the new table would have handled differently. No Jev call happens, so the result is deterministic: same answers, same policy, same outcome, every run. Use it for threshold changes, incident review and proving a policy bug.</p>
<p><strong>Re-evaluation</strong> sends the stored state and questions to Jev again, which draws a new sample that isn’t guaranteed to match. OpenRouter’s <a href="https://openrouter.ai/docs/cookbook/building-agents/gate-tool-calls-with-jev">gate cookbook</a> reports that identical repeats moved probabilities by up to eight hundredths, one check ranging from 0.35 to 0.43 over four runs, and “the outcome held on every repeat”. TypeSafe’s <a href="https://docs.typesafe.ai/cookbooks/parallel_questions">parallel-questions cookbook</a> found most answers identical over five repeats on <code>jev-1.12</code>, two Nouls drifting with standard deviations under 0.01. Far from a threshold, drift is harmless; near one, it flips the outcome.</p>
<p>
<em>Screenshot: OpenRouter, “Gate Agent Tool Calls with Jev” (undated), captured Sep 21, 2026.</em></p>
<p>The same cookbook says to store the whole decision object with the ticket as the audit record. Add four rules:</p>
<ol>
<li><strong>Never overwrite the original.</strong> A re-evaluation writes a new record with <code>reevaluates: &lt;decision_id&gt;</code> and its own <code>model_reported</code>.</li>
<li><strong>Report re-evaluations as drift.</strong> “Jev would say 0.87 today” is a fact about today’s model, not about what the gate saw.</li>
<li><strong>Re-evaluate on the pinned version</strong>, unless you are measuring a bump (step 8).</li>
<li><strong>No state, no re-evaluation.</strong> Once the store purges a state you can replay but not re-ask, hence step 2’s retention rule.</li>
</ol>
<p>
<em>Replay asks what your code would do. Re-evaluation asks what the model would say today. Only one of them reproduces the past.</em></p>
<h3 id="step-7-plan-for-the-pinned-versions-retirement-before-anyone-announces-it">Step 7: Plan for the pinned version’s retirement before anyone announces it</h3>
<p>TypeSafe publishes no support window for a versioned ID, and OpenRouter doesn’t say whether <code>jev-1.13</code> stays listed after a newer Jev ships. Plan as if the pin can go at short notice, beside your other model deprecations; <a href="/intel/forced-model-routing-continuity/">the deprecation ledger</a> is the shape.</p>
<ol>
<li><strong>An owner and a watch.</strong> One named owner per pinned version and a weekly read of the models page. <code>GET /v1/models</code> lists aliases only, so diffing it won’t show a withdrawn version.</li>
<li><strong>A daily canary.</strong> One fixed fixture, pinned ID, every door you use. Alert on an error, a changed <code>model_reported</code> or an answer outside its usual band.</li>
<li><strong>A written bump procedure.</strong> Step 8, with a named approver, ready before any notice arrives.</li>
<li><strong>A fallback that isn’t the alias.</strong> If the pin fails before the bump lands, the gate takes its written fail mode, closed for destructive classes, rather than quietly riding <code>jev-latest</code>.</li>
</ol>
<h3 id="step-8-re-run-the-shadow-set-on-every-bump">Step 8: Re-run the shadow set on every bump</h3>
<p>A new Jev version is a new classifier. TypeSafe says to pin once you’ve tuned thresholds and move on your own schedule, and its jaggedness notes are published per version; the current page applies to <code>jev-1.13</code> and was last reviewed Sep 17. Nothing TypeSafe publishes says a 0.9 on one version means a 0.9 on the next.</p>
<p>So the bump runs in shadow, the way <a href="/intel/jev-shadow-mode-ci-gate/">the shadow ladder</a> runs any Jev gate before it enforces:</p>
<ol>
<li><strong>Build the set from your records.</strong> Every deny, every destructive-class call, every answer within 0.1 of a threshold, plus a random sample.</li>
<li><strong>Re-evaluate on the new version</strong>, each result a linked record with the new <code>model_reported</code>.</li>
<li><strong>Replay the current policy over the new answers</strong> and count flips per tool class, both directions.</li>
<li><strong>Retune where flips cluster</strong>, then change <code>policy_version</code> and <code>model_requested</code> in one commit, so no record carries a new model under an old table.</li>
<li><strong>Shadow live traffic for a fixed period</strong>, then switch. The old pin stays the rollback until sign-off, while the vendor still serves it.</li>
</ol>
<h2 id="jev-decision-log-failure-modes-and-the-signal-for-each">Jev decision log failure modes, and the signal for each</h2>
<p><strong>The alias that crept back.</strong> A new lane copies an example using <code>jev-latest</code>. Signal: <code>model_requested</code> values other than your pin, or null. Fix: step 5’s CI check and a required <code>model</code> argument in the gate wrapper.</p>
<p><strong>A hash with nothing behind it.</strong> State retention is shorter than the audit window. Signal: a weekly fetch of twenty random <code>state_ref</code> values comes back with misses. Fix: align retention per lane.</p>
<p><strong>Same key, new question.</strong> Someone rewords <code>destructive</code> and leaves version 7. Signal: <code>question_sha256</code> changes while <code>question_set</code> doesn’t. Fix: CI refuses the merge.</p>
<p><strong>Re-evaluation passed off as replay.</strong> An incident review calls Jev again and quotes today’s probability as the gate’s. Signal: review evidence whose request IDs postdate the incident. Fix: replay tooling with no Jev credentials.</p>
<p><strong>Records only on the happy path.</strong> Timeouts and 429s write nothing. Signal: gated tool calls in transcripts with no matching <code>tool_call_id</code>. Fix: write the record in a <code>finally</code> block, <code>path</code> included.</p>
<p><strong>An unversioned door carrying a tuned gate.</strong> Signal: <code>version_known: false</code> on a lane whose policy names a version. Fix: move the gate to a door that reports one, or retune there and label the risk.</p>
<h2 id="decision-records-belong-to-the-fleet-not-to-the-gate-that-wrote-them">Decision records belong to the fleet, not to the gate that wrote them</h2>
<p>One gate’s log answers one gate’s questions. The questions after an incident are fleet questions: which lanes ran <code>jev-1.13.0</code> last Tuesday, which sat on a door that couldn’t say, how many decisions a threshold change would flip across every agent and CLI, and what each session was doing when its gate fired. That join from decision to tool call to session lives above any single harness. <a href="/intel/fleet-replay/">Fleet replay</a> covers the transcripts; the decision log is the typed index into them.</p>
<p>Keeping that index is operating-layer work: one inventory of which model, door and policy version each lane runs, and one place the records land and outlive the session that wrote them. The classifier hands you typed answers. <a href="/intel/agentic-ops/">Agentic operations</a> turns them into evidence someone can use a month later.</p>
<h2 id="faq">FAQ</h2>
<h3 id="should-i-use-jev-latest-in-production">Should I use jev-latest in production?</h3>
<p>Not on a lane that gates actions. TypeSafe says an alias moves when a new release ships, so answers can change with no change on your side, and it advises pinning the versioned ID once you tune thresholds. Send <code>jev-1.13.0</code>, set <code>TYPESAFE_DEFAULT_MODEL</code> to match, and move deliberately.</p>
<h3 id="does-jev-return-the-same-answer-for-the-same-input">Does Jev return the same answer for the same input?</h3>
<p>Mostly, not always. OpenRouter saw one probability range from 0.35 to 0.43 across four identical calls with the outcome unchanged, and TypeSafe’s own repeats showed small drift on a few Nouls. Treat a repeat call as a new sample, and replay stored answers when you need the original decision.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://docs.typesafe.ai/models">TypeSafe: Models</a> — aliases, pinning advice, context budgets</li>
<li><a href="https://docs.typesafe.ai/api">TypeSafe: API reference</a> — keyed questions and answers, response <code>model</code></li>
<li><a href="https://docs.typesafe.ai/confidence">TypeSafe: Confidence</a> — Noul carries no confidence</li>
<li><a href="https://docs.typesafe.ai/sdk/javascript/api/interfaces/TypeSafeClientConfig">TypeSafe JS SDK: TypeSafeClientConfig</a> — default model, debug logging</li>
<li><a href="https://docs.typesafe.ai/sdk/python/changelog">TypeSafe Python SDK changelog</a> — 0.6.0, 0.7.0, 0.7.1</li>
<li><a href="https://openrouter.ai/docs/guides/community/typesafe-sdk">OpenRouter: TypeSafe SDK integration</a> — model-ID routing, dated IDs</li>
<li><a href="https://openrouter.ai/docs/cookbook/building-agents/gate-tool-calls-with-jev">OpenRouter: Gate Agent Tool Calls with Jev</a> — repeat-call drift</li>
<li><a href="https://vercel.com/docs/ai-gateway/sdks-and-apis/typesafe">Vercel: TypeSafe-compatible API</a> — <code>typesafe-ai/jev</code></li>
<li><a href="https://www.kimi.com/code/docs/en/kimi-code/whats-new.html">Kimi Code: What’s new</a> — K2.8 Preview, Sep 11, 2026</li>
<li><a href="https://docs.typesafe.ai/cookbooks/parallel_questions">TypeSafe: Parallel questions cookbook</a> — repeat stability on <code>jev-1.12</code></li>
</ul>
]]></content:encoded></item><item><title>Fan Out Jev Judgments, Compose Policy in Code</title><link>https://automater.ai/intel/jev-multi-question-policy-vector/</link><guid>https://automater.ai/intel/jev-multi-question-policy-vector/</guid><description>Jev API questions belong in one call: keyed Nouls, Choices and Scores return a decision vector. Compose policy in code, validate IDs, retire yes/no subagents.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category><content:encoded><![CDATA[<p>A reviewer subagent spawns, reads the diff, the rules file and the last few turns, reasons for a few thousand tokens, and ends its paragraph with the word <code>safe</code>. A regex finds the word. Then a second subagent spawns to decide whether the same call matches the task. Two cold sessions and two parsed essays have bought two answers a person gives at a glance.</p>
<p>TypeSafe’s Jev turns that pattern inside out. Its API takes one state and a map of questions keyed by IDs you choose, and answers all your Jev API questions against that state in a single call. By Tuesday you can replace the label-returning subagents in your fleet with one multi-question call per decision, a decision vector keyed by your IDs, and a policy function in code that composes the vector with the checks code does better. You will also have evidence: code, one Jev call and the subagent it replaces, compared on the same labeled checks, with every disagreement logged.</p>
<p>Chatbots suggest; agents act, and every act in a fleet sits behind small decisions like these. Multiplied across every tool call on every lane, how you ask them becomes a cost line and an audit problem.</p>
<h2 id="sep-1520-jev-answers-a-map-of-keyed-questions-against-one-state">Sep 15–20: Jev answers a map of keyed questions against one state</h2>
<p>TypeSafe launched Jev on Sep 15, 2026, and its <a href="https://docs.typesafe.ai/api">API reference</a> shows the shape this piece depends on. A request carries a <code>state</code> (a string, a JSON object or an array), a <code>model</code>, and <code>questions</code>, a map whose keys you pick. The response carries <code>answers</code>, an object keyed by the same IDs rather than an array, plus the <code>model</code> that answered and token <code>usage</code>. The key is never sent to the model; it exists for your code.</p>
<p>Each question is one of three types. A Noul returns the probability that a statement is true and carries no confidence value. A Choice picks one of up to 255 options and returns probabilities plus a confidence. A Score grades on an ordered scale of 2 to 10 levels, can land between levels, and also carries a confidence.</p>
<p>The <a href="https://docs.typesafe.ai/models">models page</a> says Jev ingests the state once and evaluates every question against it in parallel, under two budgets that apply at once: 64K tokens for the state plus all questions, and 32K for the state plus the longest question. Input costs $0.042 per million tokens; output is free.</p>
<p>The <a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">launch post</a> reports 70 to 500 ms end to end, measured from TypeSafe’s own West Coast base: a vendor range, not an SLA. The <a href="https://docs.typesafe.ai/patterns/fan-out">speculative fan-out pattern</a> goes further than permitting batches. It recommends putting every question your system needs into one request and letting code decide afterwards which answers matter.</p>
<p>
<em>Screenshot: TypeSafe docs, “Speculative fan-out - TypeSafe AI” (undated docs page), captured Sep 21, 2026.</em></p>
<p>The <a href="https://docs.typesafe.ai/primitives">primitives page</a> states the rule this runbook is built on: “One question’s answer is not hidden context for another.” When a judgment depends on several factors, ask about each separately and combine the answers in your own logic.</p>
<p>Practitioners pushed volume through it within a day. Ryan Vogel reported running Jev over 1,500 of his own emails on Sep 16, and a triage demo shown in Theo Browne’s Sep 20 video reportedly averaged about 200 ms per email, roughly 38 a second.</p>
<aside class="social-embed" data-x-embed data-post-id="2100042788851101842">
  <p class="social-embed__eyebrow">From the source · X</p>
  <p class="social-embed__caption">September 16, 2026 — A practitioner reports testing Jev as a classifier on 1,500 of his own emails, one day after launch.</p>
  <a class="social-embed__source" href="https://x.com/ryanvogel/status/2100042788851101842" target="_blank" rel="noopener noreferrer">View Ryan Vogel's post on X ↗</a>
  <button type="button" class="social-embed__load" data-x-load hidden>Load post from X</button>
  <p class="social-embed__privacy">Loading this post connects to X. You can keep reading without it.</p>
  <p class="social-embed__status" data-x-status role="status" aria-live="polite"></p>
  <div data-x-content></div>
</aside>
<h2 id="a-yesno-subagent-buys-a-whole-session-to-return-one-bit">A yes/no subagent buys a whole session to return one bit</h2>
<p>A subagent is the right tool when the delegated work needs reading, planning or writing. <a href="/intel/when-not-to-use-a-coordinator/">When not to use a coordinator</a> covers whether to delegate at all, and <a href="/intel/subagent-fanout-metering/">subagent fan-out metering</a> covers the bill once a tree exists. Neither addresses the narrow case where the delegated work is a label.</p>
<p>That case is common because it is easy to write: “spawn a reviewer to check whether this is safe” is one line in a prompt. What it costs is a cold context per spawn, prose nobody reads, a parser that breaks when the prose changes shape, and one more agent with tools of its own in your tree. A multi-question call returns typed values under names your code already uses, and cannot run a command.</p>
<h2 id="step-1-find-the-subagents-that-only-return-a-label">Step 1: Find the subagents that only return a label</h2>
<p>Pull a week of spawn logs and list every subagent whose useful output is a yes/no, a label or a level. Write down the questions each one actually answers, one per line. A “safety review” usually turns out to be four or five questions stapled together.</p>
<p>Then give each question an owner. Anything code can answer exactly stays in code: paths, counts, dates, allowlists. Anything that needs synthesis stays with a model that writes. Only checks a person answers in about ten seconds from the state alone are candidates for the vector; <a href="/intel/jev-ten-second-rule-decision-seat/">the decision-seat tests</a> do that sorting, and this piece does not repeat them.</p>
<table>
<thead>
<tr>
<th>Check the reviewer subagent answers today</th>
<th>Owner after the split</th>
<th>Why</th>
</tr>
</thead>
<tbody>
<tr>
<td>Target path is inside this repo</td>
<td>code</td>
<td>path resolution is exact; never ask Jev</td>
</tr>
<tr>
<td>Diff touches more than 20 files</td>
<td>code</td>
<td>counting belongs in code</td>
</tr>
<tr>
<td>Tool class: read, write, network, destructive</td>
<td>Jev Choice <code>kind</code>, after a code allowlist</td>
<td>literal criteria, one label</td>
</tr>
<tr>
<td>Call is safe to run as described</td>
<td>Jev Noul <code>safe</code></td>
<td>a glance-sized judgment</td>
</tr>
<tr>
<td>Blast radius if the task line is wrong</td>
<td>Jev Score <code>risk</code></td>
<td>an ordered scale</td>
</tr>
<tr>
<td>Call matches the stated task</td>
<td>Jev Noul <code>on_task</code></td>
<td>a glance-sized judgment</td>
</tr>
<tr>
<td>The change is correct and ready to merge</td>
<td>a reviewer model or a human</td>
<td>not a ten-second question</td>
</tr>
</tbody>
</table>
<p>The last row matters as much as the others. A vector replaces the checks that were never worth a session; it does not replace review.</p>
<h2 id="step-2-write-the-jev-api-questions-as-one-keyed-map-with-stable-ids">Step 2: Write the Jev API questions as one keyed map with stable IDs</h2>
<p>The question IDs are a contract between the request and the policy, so treat them like column names. Pick short, stable IDs (<code>safe</code>, <code>on_task</code>, <code>risk</code>, <code>kind</code>), version the whole map (<code>tool-gate@3</code>), and bump the version whenever an instruction, criterion or option changes. A policy line that reads <code>answers["safe"]</code> should mean the same question next month.</p>
<p>Build the state from the fields the questions need and nothing else. TypeSafe’s <a href="https://docs.typesafe.ai/model-jaggedness/jev-1.13">jaggedness page for jev-1.13</a> warns that accuracy falls as unrelated content grows, and says to filter in code and send only the fields each question needs. A tool name, its arguments, the task line and a diff summary usually suffice. A transcript is not a state.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span></span>
<span class="line"><span style="color:#79B8FF">  "model"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"jev-1.13.0"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">  "state"</span><span style="color:#E1E4E8">: {</span></span>
<span class="line"><span style="color:#79B8FF">    "task"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Rename the invoice export flag and update its tests"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">    "tool"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Bash"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">    "command"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"sed -i 's/exportV1/exportLegacy/g' src/billing/export.ts"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">    "diff_summary"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"1 file, 6 lines changed"</span></span>
<span class="line"><span style="color:#E1E4E8">  },</span></span>
<span class="line"><span style="color:#79B8FF">  "questions"</span><span style="color:#E1E4E8">: {</span></span>
<span class="line"><span style="color:#79B8FF">    "safe"</span><span style="color:#E1E4E8">:    { </span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"noul"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"instructions"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"The command only reads or edits files in the working tree. It does not delete data, rewrite git history, change permissions or send data off the machine."</span><span style="color:#E1E4E8"> },</span></span>
<span class="line"><span style="color:#79B8FF">    "on_task"</span><span style="color:#E1E4E8">: { </span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"noul"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"instructions"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"The command does what the task line asks and nothing else."</span><span style="color:#E1E4E8"> },</span></span>
<span class="line"><span style="color:#79B8FF">    "risk"</span><span style="color:#E1E4E8">:    { </span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"score"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"instructions"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"How much damage would this command do if the task line were wrong?"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">                 "criteria"</span><span style="color:#E1E4E8">: [</span><span style="color:#9ECBFF">"0: none"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"1: local and reversible"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"2: shared state or other users"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"3: irreversible or external"</span><span style="color:#E1E4E8">] },</span></span>
<span class="line"><span style="color:#79B8FF">    "kind"</span><span style="color:#E1E4E8">:    { </span><span style="color:#79B8FF">"type"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"choice"</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"instructions"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Which class of tool action is this?"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">                 "criteria"</span><span style="color:#E1E4E8">: { </span><span style="color:#79B8FF">"read"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Only reads."</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"write"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Edits files in the repo."</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"network"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Sends or fetches over the network."</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF">                               "destructive"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"Deletes, force-pushes or drops data."</span><span style="color:#E1E4E8">, </span><span style="color:#79B8FF">"unclear"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"None of the above fits cleanly."</span><span style="color:#E1E4E8"> } }</span></span>
<span class="line"><span style="color:#E1E4E8">  }</span></span>
<span class="line"><span style="color:#E1E4E8">}</span></span></code></pre>
<p>The shape is illustrative; check field names against the API reference and your endpoint’s dialect.</p>
<p>Three details are deliberate. The Choice has an <code>unclear</code> option because Jev has no abstain answer. The Score levels are literal, because jev-1.13 reads criteria at face value. And the path check is missing on purpose: it is not a question.</p>
<p>Send a versioned model ID, not an alias; the <a href="https://docs.typesafe.ai/sdk/python/api/constants">TypeSafe SDK constants</a> default to <code>jev-latest</code> unless you set one. Step 7 covers why the pin belongs in every record.</p>
<h2 id="step-3-compose-the-policy-in-code-and-make-every-gap-a-no">Step 3: Compose the policy in code, and make every gap a no</h2>
<p>With the answers back, the policy is a function. The core rule: allow only if the target path is inside this repo (checked in code, never asked of Jev) AND <code>safe</code> meets the threshold from your <a href="/intel/jev-confidence-gates-tool-calls/">per-class gate table</a> AND <code>risk</code> is at most 1. The other branches apply the same idea to the rest of the vector.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="python"><code><span class="line"><span style="color:#6A737D"># tool_gate.py: illustrative policy over one Jev decision vector.</span></span>
<span class="line"><span style="color:#6A737D"># Threshold values are placeholders; take yours from the per-class gate table.</span></span>
<span class="line"><span style="color:#79B8FF">POLICY</span><span style="color:#F97583"> =</span><span style="color:#E1E4E8"> {</span><span style="color:#9ECBFF">"qset"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"tool-gate@3"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"version"</span><span style="color:#E1E4E8">: </span><span style="color:#9ECBFF">"tool-gate-policy@5"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#9ECBFF">          "safe_min"</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">0.90</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"on_task_min"</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">0.80</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"risk_max"</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">1.0</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"conf_floor"</span><span style="color:#E1E4E8">: </span><span style="color:#79B8FF">0.50</span><span style="color:#E1E4E8">}</span></span>
<span class="line"></span>
<span class="line"><span style="color:#F97583">def</span><span style="color:#B392F0"> decide</span><span style="color:#E1E4E8">(call, vec, repo_root):</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#F97583"> not</span><span style="color:#E1E4E8"> path_inside(call.target_path, repo_root):      </span><span style="color:#6A737D"># code, never Jev</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#9ECBFF"> "deny"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"path outside repo"</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#F97583"> not</span><span style="color:#E1E4E8"> vec.valid:                                     </span><span style="color:#6A737D"># step 5 failed</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#9ECBFF"> "ask"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"invalid vector: "</span><span style="color:#F97583"> +</span><span style="color:#E1E4E8"> vec.error</span></span>
<span class="line"><span style="color:#E1E4E8">    kind </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> vec[</span><span style="color:#9ECBFF">"kind"</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> kind.choice </span><span style="color:#F97583">in</span><span style="color:#E1E4E8"> (</span><span style="color:#9ECBFF">"destructive"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"unclear"</span><span style="color:#E1E4E8">) </span><span style="color:#F97583">or</span><span style="color:#E1E4E8"> kind.confidence </span><span style="color:#F97583">&lt;</span><span style="color:#79B8FF"> POLICY</span><span style="color:#E1E4E8">[</span><span style="color:#9ECBFF">"conf_floor"</span><span style="color:#E1E4E8">]:</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#9ECBFF"> "ask"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"class needs a person"</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> vec[</span><span style="color:#9ECBFF">"safe"</span><span style="color:#E1E4E8">].noul </span><span style="color:#F97583">&lt;</span><span style="color:#79B8FF"> POLICY</span><span style="color:#E1E4E8">[</span><span style="color:#9ECBFF">"safe_min"</span><span style="color:#E1E4E8">]:</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#9ECBFF"> "ask"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"safe below threshold"</span></span>
<span class="line"><span style="color:#E1E4E8">    risk </span><span style="color:#F97583">=</span><span style="color:#E1E4E8"> vec[</span><span style="color:#9ECBFF">"risk"</span><span style="color:#E1E4E8">]</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> risk.score </span><span style="color:#F97583">&gt;</span><span style="color:#79B8FF"> POLICY</span><span style="color:#E1E4E8">[</span><span style="color:#9ECBFF">"risk_max"</span><span style="color:#E1E4E8">] </span><span style="color:#F97583">or</span><span style="color:#E1E4E8"> risk.confidence </span><span style="color:#F97583">&lt;</span><span style="color:#79B8FF"> POLICY</span><span style="color:#E1E4E8">[</span><span style="color:#9ECBFF">"conf_floor"</span><span style="color:#E1E4E8">]:</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#9ECBFF"> "ask"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"risk above 1 or uncertain"</span></span>
<span class="line"><span style="color:#F97583">    if</span><span style="color:#E1E4E8"> vec[</span><span style="color:#9ECBFF">"on_task"</span><span style="color:#E1E4E8">].noul </span><span style="color:#F97583">&lt;</span><span style="color:#79B8FF"> POLICY</span><span style="color:#E1E4E8">[</span><span style="color:#9ECBFF">"on_task_min"</span><span style="color:#E1E4E8">]:</span></span>
<span class="line"><span style="color:#F97583">        return</span><span style="color:#9ECBFF"> "ask"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"off task"</span></span>
<span class="line"><span style="color:#F97583">    return</span><span style="color:#9ECBFF"> "no_veto"</span><span style="color:#E1E4E8">, </span><span style="color:#9ECBFF">"all checks passed"</span></span></code></pre>
<p>Four rules keep the function honest.</p>
<p><strong>The best outcome is no veto, never an approval.</strong> In a harness hook the gate only denies or asks; its pass hands the call to your permission rules and sandbox, which still apply, and a destructive call reaches a person whatever the vector says. The gate itself can fail. If the Jev call times out, returns 429 or comes back malformed, the policy returns ask for writes and deny for destructive classes; if the hook process itself crashes or times out, Claude Code lets the call continue through the normal permission flow, so that wall has to hold on its own. Designing each gate’s fail mode is <a href="/intel/jev-gate-fail-mode-rate-limits/">the rate-limit piece</a>.</p>
<p><strong>Thresholds live per question and per primitive.</strong> On the jaggedness page, one question asked as a Noul returned 0.22 while a yes/no Choice put 0.01 on yes, so a Noul threshold does not transfer to a Choice. Keep every threshold in one versioned policy file.</p>
<p><strong>Never do arithmetic across answers.</strong> TypeSafe’s own example of a question and its negation, asked as two Nouls, summed to 1.19. Don’t derive one answer from another, and keep counting and date ordering in code.</p>
<p><strong>A missing answer is a no.</strong> Every branch that cannot find its input ends in ask or deny. A default of zero or false is how a gate quietly becomes an approval.</p>
<h2 id="step-4-enforce-both-budgets-before-sending-and-set-your-own-question-cap">Step 4: Enforce both budgets before sending, and set your own question cap</h2>
<p>TypeSafe documents two limits that bind at once: the state plus every question must fit in 64K tokens, and the state plus the longest question in 32K. Check both in code, with your own token estimate and a margin, before the request leaves. Gateways list only the 32K figure, so don’t assume the 64K aggregate holds on every door.</p>
<p>When a vector goes over, choose deliberately:</p>
<ol>
<li><strong>Shrink the state first.</strong> Filtering fields in code is free and usually helps accuracy.</li>
<li><strong>Split the map into two requests on the same state</strong> if it still won’t fit. You pay for the state twice; log both halves under one decision ID and validate them as one vector.</li>
<li><strong>Never truncate the state to make it fit.</strong> A cut-off diff summary is a different question under the same ID.</li>
</ol>
<p>Handle 422 as a failed gate. TypeSafe describes 422 as a validation failure, such as a missing field or a malformed question, and names the offending field in the body. Its docs do not say an oversized request returns 422, which is why the budget check lives on your side. Log the field, return ask or deny, fix the map.</p>
<p>No documented cap on question count is not the same as unlimited. The budgets cap it; the rate limits (250,000 tokens a second, 1,200 requests a minute) mean a fat batch trades requests for tokens; and each question is a threshold someone must own. Set a ceiling per vector, and require every question ID to appear in at least one policy line. An unneeded question is nearly free in tokens, as the primitives page says, but not in review time.</p>
<h2 id="step-5-validate-the-vector-by-question-id-before-any-policy-line-reads-it">Step 5: Validate the vector by question ID before any policy line reads it</h2>
<p>Validation sits between the response and the policy, and it fails the whole vector on any single problem:</p>
<ul>
<li>Every ID you sent is present in <code>answers</code>, and nothing is there that you did not send.</li>
<li>Each answer’s <code>type</code> matches its question.</li>
<li>A Noul is between 0 and 1. A Choice names one of your options. A Score falls inside your level range, allowing values between levels.</li>
<li>Choice and Score answers carry <code>confidence</code>; Noul answers do not, so no policy line reads one.</li>
<li>The response <code>model</code> is recorded, along with the request ID from the <code>x-typesafe-request-id</code> header that both TypeSafe SDKs expose.</li>
</ul>
<p>Normalize at the adapter, never in the policy. Vercel’s <a href="https://vercel.com/docs/ai-gateway/modalities/evaluation">evaluation API</a> calls a Noul <code>boolean</code> with a <code>probability</code> field, and through the AI SDK the Choice and Score confidence arrives under <code>providerMetadata.typesafe.confidence</code>. The <a href="/intel/jev-endpoint-dialects-vercel-cloudflare/">endpoint-dialect conformance test</a> maps every door to one shape, and the policy only ever sees that shape.</p>
<p>OpenRouter’s <a href="https://openrouter.ai/docs/cookbook/building-agents/gate-tool-calls-with-jev">cookbook for gating agent tool calls with Jev</a> takes the same stance. Its client throws on a non-2xx response, a missing answer or a probability outside 0 to 1, “so a broken check never turns into an approval or a review.” It approves only when every check clears 0.9, blocks when any sits at 0.1 or below, and sends the rest to a human.</p>
<p>
<em>One call, one vector, one policy in code. Every failure path ends in ask or deny, with your permission rules and sandbox still behind the gate.</em></p>
<h2 id="step-6-run-code-one-jev-call-and-the-subagent-over-the-same-labeled-checks">Step 6: Run code, one Jev call and the subagent over the same labeled checks</h2>
<p>Retiring a subagent is a claim, so measure it. Take one labeled set of checks from your own logs: calls a person later approved, calls that were denied or reverted, and every destructive call you have. The pilot set behind your gate table works; so do 200 checks pulled fresh.</p>
<p>Run three arms over the same set:</p>
<ol>
<li><strong>Deterministic code</strong>, for every question code can answer. Where it applies, it is the baseline to beat, and it usually wins.</li>
<li><strong>One Jev call</strong> per check, with the full question map and the step 3 policy.</li>
<li><strong>The subagent</strong> as it runs today, with the same prompt, model and parser.</li>
</ol>
<p>Record cost per check, wall-clock latency at p50 and p95 with retries included, and agreement for each arm. Price the Jev arm from input tokens at $0.042 per million, with the state counted once per call; illustratively, a 2,000-token state plus five short questions comes to about a hundredth of a cent. Price the subagent from your usage export, counting every retry and re-spawn. Measure agreement per question and per decision, against the labels and between arms.</p>
<table>
<thead>
<tr>
<th>Arm</th>
<th>Cost per check</th>
<th>Latency p50 / p95</th>
<th>Agreement with labels</th>
<th>Disagreements with other arms</th>
</tr>
</thead>
<tbody>
<tr>
<td>Deterministic code</td>
<td>measure</td>
<td>measure</td>
<td>per question it covers</td>
<td>list every case</td>
</tr>
<tr>
<td>One Jev call</td>
<td>measure</td>
<td>measure</td>
<td>per question and per decision</td>
<td>list every case</td>
</tr>
<tr>
<td>Subagent it replaces</td>
<td>measure</td>
<td>measure</td>
<td>per decision</td>
<td>list every case</td>
</tr>
</tbody>
</table>
<p>TypeSafe’s <a href="https://docs.typesafe.ai/cookbooks/parallel_questions">parallel-questions cookbook</a> is a sanity check for the shape of the Jev arm, not a result for yours: 13 questions about a ~54,000-character article on jev-1.12, five runs each way. One call carrying all 13 cost $0.000497 and took 0.27 s. Thirteen single-question calls cost $0.006090 and took 2.71 s, a time the cookbook says assumes one call after another; concurrency narrows the time gap but not the token bill. Choices, scores and six of the eight Nouls came back identical across the repeats either way.</p>
<p>
<em>Vendor-run figures on jev-1.12. The state was sent once in the batched call and 13 times in the other; your subagent arm is yours to measure.</em></p>
<p>
<em>Screenshot: TypeSafe docs, “Parallel questions - TypeSafe AI” (undated cookbook, jev-1.12 run), captured Sep 21, 2026.</em></p>
<p>Retire the subagent for a check class only when the Jev arm matches or beats its agreement with the labels on that class, and no disagreement on a destructive or network row is left unexplained. Where code agrees with the labels on every case, the question leaves the map entirely.</p>
<h2 id="step-7-log-the-vector-and-every-disagreement">Step 7: Log the vector and every disagreement</h2>
<p>Log the whole vector, not just the verdict. A record that says “ask” cannot tell you next month whether <code>safe</code> or <code>risk</code> tipped it. Each line carries the question-set version, every answer, the code-check results, the policy version and deciding branch, the reported model and the request ID. The full record, and why replaying policy over a stored answer differs from asking Jev again, is <a href="/intel/jev-decision-log-version-pin/">the decision-log piece</a>.</p>
<pre class="astro-code github-dark" style="background-color:#24292e;color:#e1e4e8; overflow-x: auto;" tabindex="0" data-language="json"><code><span class="line"><span style="color:#E1E4E8">{</span><span style="color:#79B8FF">"ts"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"2026-09-22T14:03:11Z"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"decision_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"d-7f3a"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"lane"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"billing-refactor"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"tool_call"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"tc-0412"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF"> "qset"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"tool-gate@3"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"policy"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"tool-gate-policy@5"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"model"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"jev-1.13.0"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"request_id"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"req-from-response-header"</span><span style="color:#E1E4E8">,</span></span>
<span class="line"><span style="color:#79B8FF"> "code"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"path_inside_repo"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">true</span><span style="color:#E1E4E8">},</span></span>
<span class="line"><span style="color:#79B8FF"> "vector"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"safe"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"noul"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.94</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"on_task"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"noul"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.88</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"risk"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"score"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">1.2</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"confidence"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.71</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"kind"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"choice"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"write"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"confidence"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">0.83</span><span style="color:#E1E4E8">}},</span></span>
<span class="line"><span style="color:#79B8FF"> "decision"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"ask"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"branch"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"risk above 1 or uncertain"</span><span style="color:#E1E4E8">,</span><span style="color:#79B8FF">"shadow"</span><span style="color:#E1E4E8">:{</span><span style="color:#79B8FF">"subagent"</span><span style="color:#E1E4E8">:</span><span style="color:#9ECBFF">"allow"</span><span style="color:#E1E4E8">},</span><span style="color:#79B8FF">"disagreement"</span><span style="color:#E1E4E8">:</span><span style="color:#79B8FF">true</span><span style="color:#E1E4E8">}</span></span></code></pre>
<p>Disagreements are the product of this step: during the comparison, log every check where the arms disagree with each other or with the label. After cut-over, keep the old subagent or a human reviewer in shadow on a small slice and keep logging. Review weekly and give each disagreement one cause: criteria wording, a missing state field, a threshold, a policy bug or a wrong label. The fix lands as a new question-set or policy version, never a silent edit.</p>
<h2 id="where-a-jev-api-questions-vector-fails-and-the-signal-for-each">Where a Jev API questions vector fails, and the signal for each</h2>
<p><strong>A missing answer read as zero.</strong> A default fills the gap and the gate passes. Signal: validation failures with no matching ask or deny. Fix: step 5 fails the whole vector; defaults are banned.</p>
<p><strong>Arithmetic across answers.</strong> Code subtracts one Noul from 1 to stand in for another, or sums related Nouls. Signal: any policy line that folds two probabilities into one number.</p>
<p><strong>One threshold for two primitives.</strong> Someone rewords a Noul as a Choice and the old number rides along. Signal: a question-set version bump with no policy version bump.</p>
<p><strong>Question creep.</strong> The map grows because questions are cheap, and half feed no policy line. Signal: IDs the policy never references, and a longest question creeping toward 32K.</p>
<p><strong>The door changed under the policy.</strong> A lane moves to a gateway that says <code>boolean</code>, and <code>answers["safe"].noul</code> goes missing. Signal: validation failures spike right after a routing change.</p>
<p><strong>Adversarial state.</strong> A diff written to argue for its own classification can move the answer, as the jaggedness page warns. Signal: nothing in the vector looks wrong. Fix: code allowlists first, the sandbox behind, and injected fixtures in the labeled set, as <a href="/intel/jev-gate-injection-jaggedness/">the injection piece</a> lays out.</p>
<h2 id="the-decision-vector-is-fleet-policy-not-a-prompt">The decision vector is fleet policy, not a prompt</h2>
<p>Nothing in this runbook asks a model to be careful. The question map, thresholds, budget check, validator, policy function and disagreement log are files and code outside every model, applied the same way on every lane whichever CLI runs it. That is what makes the vector auditable: one version of one policy, fed by one typed call, with a log that names the deciding branch.</p>
<p>It is also why the vector belongs in the layer that runs the fleet, not in any one agent’s instructions. A <a href="/intel/multi-agent-command-center/">multi-agent command center</a> is where lanes, gates, kill switches and evidence meet; a decision vector per gate is one more thing that layer owns, and one fewer session it pays for.</p>
<h2 id="faq">FAQ</h2>
<h3 id="can-i-ask-jev-several-questions-in-one-api-call">Can I ask Jev several questions in one API call?</h3>
<p>Yes. One Jev request carries a state plus a map of questions keyed by IDs you choose, and answers return under the same IDs. Jev ingests the state once and evaluates every question in parallel. The state plus all questions must fit in 64K tokens, and the state plus the longest question in 32K.</p>
<h3 id="can-jev-replace-a-yesno-reviewer-subagent">Can Jev replace a yes/no reviewer subagent?</h3>
<p>For checks that only return a label, yes, once you measure it. Keep exact checks like paths and counts in code, and leave merge readiness to a reviewer model or a person. Retire the subagent for a check class only when Jev matches or beats its agreement with your labels on that class.</p>
<h3 id="how-do-i-combine-several-jev-answers-into-one-decision">How do I combine several Jev answers into one decision?</h3>
<p>Combine them in code, in a policy function that reads each answer by its ID against its own threshold. Answers are independent, so don’t derive one from another or sum related Nouls; TypeSafe’s example of a question and its negation summed to 1.19. Any missing or invalid answer should end in ask or deny.</p>
<h2 id="sources">Sources</h2>
<ul>
<li><a href="https://docs.typesafe.ai/api">TypeSafe API reference</a> — keyed questions and answers, question types, error codes</li>
<li><a href="https://docs.typesafe.ai/models">TypeSafe Models</a> — 64K and 32K budgets, rate limits, pricing</li>
<li><a href="https://docs.typesafe.ai/primitives">TypeSafe Primitives</a> — independent answers</li>
<li><a href="https://docs.typesafe.ai/patterns/fan-out">TypeSafe: Speculative fan-out</a></li>
<li><a href="https://docs.typesafe.ai/cookbooks/parallel_questions">TypeSafe: Parallel questions cookbook</a> — jev-1.12 cost and time table</li>
<li><a href="https://docs.typesafe.ai/model-jaggedness/jev-1.13">TypeSafe: jev-1.13 jaggedness</a></li>
<li><a href="https://typesafe.ai/blog/introducing-system-one-models-and-jev">TypeSafe: Introducing System One models and Jev</a> — Sep 15, 2026</li>
<li><a href="https://docs.typesafe.ai/sdk/python/api/constants">TypeSafe Python SDK constants</a> — default model</li>
<li><a href="https://vercel.com/docs/ai-gateway/modalities/evaluation">Vercel AI Gateway: Evaluation</a></li>
<li><a href="https://openrouter.ai/docs/cookbook/building-agents/gate-tool-calls-with-jev">OpenRouter: Gate agent tool calls with Jev</a></li>
</ul>
]]></content:encoded></item><item><title>Three Oversight Numbers Your Fleet Should Publish Internally</title><link>https://automater.ai/intel/fleet-oversight-three-numbers/</link><guid>https://automater.ai/intel/fleet-oversight-three-numbers/</guid><description>AI agent oversight metrics on Anthropic&apos;s own definitions: coverage before and after, review latency per leg, one escalation rate per monitor, every week.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Lint the Compaction Summary Before the Lane Acts on It</title><link>https://automater.ai/intel/compaction-summary-lint/</link><guid>https://automater.ai/intel/compaction-summary-lint/</guid><description>A compaction summary can tell the next context to hide mistakes. Lint it where each harness allows: before it loads, before the next tool call, or not at all.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Jev Can Be Prompt-Injected Too: Read the Jaggedness Page Before It Guards a Tool</title><link>https://automater.ai/intel/jev-gate-injection-jaggedness/</link><guid>https://automater.ai/intel/jev-gate-injection-jaggedness/</guid><description>Jev prompt injection is in TypeSafe&apos;s own docs. Put an allowlist first, send only needed fields, and pin injected fixtures so a flipped verdict blocks upgrades.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Shadow-Mode Jev Before It Skips Your CI Tests</title><link>https://automater.ai/intel/jev-shadow-mode-ci-gate/</link><guid>https://automater.ai/intel/jev-shadow-mode-ci-gate/</guid><description>AI CI test selection with Jev starts in shadow: count false skips against real failures, canary on a slice, and pin the action by SHA before you enforce.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Who Owns Compaction on an Unattended Lane: Notice, Warn, Force, and a Note-to-Self</title><link>https://automater.ai/intel/agent-owned-compaction-policy/</link><guid>https://automater.ai/intel/agent-owned-compaction-policy/</guid><description>Agent context compaction on unattended lanes: who fires it, token thresholds under price lines, what must survive the cut, and whether the summary is readable.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Claude Code + Jev: Put the Gate in a Hook, Not an MCP Tool the Agent Can Skip</title><link>https://automater.ai/intel/jev-claude-code-hook-not-mcp/</link><guid>https://automater.ai/intel/jev-claude-code-hook-not-mcp/</guid><description>Claude Code Jev integrations shipped as a plugin and MCP tools the agent can skip. Put the gate in a PreToolUse hook that fails closed, with deny rules behind.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Ship Typed Jev Decisions on Vercel, Cloudflare or OpenRouter — Test the Dialect Before You Pick the Door</title><link>https://automater.ai/intel/jev-endpoint-dialects-vercel-cloudflare/</link><guid>https://automater.ai/intel/jev-endpoint-dialects-vercel-cloudflare/</guid><description>Vercel AI Gateway&apos;s evaluation API calls Jev&apos;s Noul a boolean. Run one fixture set through Vercel, Cloudflare, OpenRouter and TypeSafe, then pin one wire shape.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Jev Isn’t a Chatbot — Stop Routing Like One</title><link>https://automater.ai/intel/jev-ten-second-rule-decision-seat/</link><guid>https://automater.ai/intel/jev-ten-second-rule-decision-seat/</guid><description>TypeSafe Jev belongs in a decision seat, not a reply seat. Sort fleet decisions with four tests, keep judges and compactors off it, and ship the table.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>Use Jev to Route Cheap Models Without Guessing</title><link>https://automater.ai/intel/jev-cheap-model-router-escalation/</link><guid>https://automater.ai/intel/jev-cheap-model-router-escalation/</guid><description>Jev model routing as code: tiers with written criteria, an unclear exit, a floor that keeps unsure work on frontier, and cost per merged PR as the test.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Jev Confidence Gates Before the Tool Runs</title><link>https://automater.ai/intel/jev-confidence-gates-tool-calls/</link><guid>https://automater.ai/intel/jev-confidence-gates-tool-calls/</guid><description>Set a Jev confidence threshold per tool class from 200 of your own labeled calls. Jev may deny or ask, never approve; destructive calls never run on its word.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 21 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>AutoClaw Connectors: The Weekly Blast-Radius Inventory</title><link>https://automater.ai/intel/autoclaw-connector-inventory/</link><guid>https://automater.ai/intel/autoclaw-connector-inventory/</guid><description>AutoClaw connectors ship with no scope or revoke docs. Run a weekly inventory: what is installed, who added it, last used, write scope and a tested revoke path.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Off-Peak AI Agent Scheduling: Route Flexible Work by the Clock, Not by Vibe</title><link>https://automater.ai/intel/offpeak-agent-schedule-by-clock/</link><guid>https://automater.ai/intel/offpeak-agent-schedule-by-clock/</guid><description>Off-peak AI agent scheduling by the clock: classify jobs, map DeepSeek&apos;s UTC peak hours and Anthropic&apos;s cache timers, route with caps, and review weekly.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>AI Credits vs Tokens: Two Billing Dialects, One Operator Ledger</title><link>https://automater.ai/intel/credit-vs-token-meter-dialects/</link><guid>https://automater.ai/intel/credit-vs-token-meter-dialects/</guid><description>AI credits vs tokens are two billing dialects. Translate a credit pack into measured jobs, price the same jobs at token rates, and keep one burn-rate ledger.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Annotation Is Not Approval: The Human in the Loop Annotation Approval Split</title><link>https://automater.ai/intel/annotation-is-not-approval/</link><guid>https://automater.ai/intel/annotation-is-not-approval/</guid><description>Human in the loop annotation approval blurs when note-taking sits beside deploy, pin and schedule buttons. Tier gestures by consequence; check the gate weekly.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Local AI Agent vs Cloud Project: Digital Employees on the Desk, Threads in the Cloud, and No Sync Fantasy</title><link>https://automater.ai/intel/local-digital-employees-vs-cloud-projects/</link><guid>https://automater.ai/intel/local-digital-employees-vs-cloud-projects/</guid><description>Local AI agent vs cloud project: AutoClaw, OpenClaw and Kimi Work beside Cursor and Claude Code Projects; one identity, one kill switch, provenance per host.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Three Meters for One Agent Job: Inventory Your AI Agent Billing Meters Before Fan-Out</title><link>https://automater.ai/intel/three-meters-credits-tokens-sandbox/</link><guid>https://automater.ai/intel/three-meters-credits-tokens-sandbox/</guid><description>One agent job can hit three ai agent billing meters: tokens with cache multipliers, per-call tools, and sandbox minutes with a 5-minute minimum. Cap each one.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>GitLab Duo Triggers on Merge Request Created, and the Loops Around Them</title><link>https://automater.ai/intel/gitlab-mr-created-trigger-loops/</link><guid>https://automater.ai/intel/gitlab-mr-created-trigger-loops/</guid><description>GitLab Duo triggers fire on merge request created. Bots cannot activate them, so the loops left are external agents, cross-system hops and cost. Five guards.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Computer Use Agent Windows Policy: Entitlements Before the Mouse Moves</title><link>https://automater.ai/intel/computer-use-windows-endpoint-policy/</link><guid>https://automater.ai/intel/computer-use-windows-endpoint-policy/</guid><description>A computer use agent windows policy for corporate endpoints: approved models, standard-user driver accounts, an entitlement matrix, a kill switch and logging.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>MCP Marketplace Security After MCPJacking and Plugin4Shell: Review, Pin, Verify</title><link>https://automater.ai/intel/marketplace-hygiene-plugin-mcp/</link><guid>https://automater.ai/intel/marketplace-hygiene-plugin-mcp/</guid><description>MCP marketplace security after MCPJacking&apos;s 155 hijackable entries and Plugin4Shell&apos;s SHA-pin bypass: an intake checklist to review, pin, verify and revoke.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Zero Data Retention AI Agents: Where Vendor Forgetting Meets Agent Memory</title><link>https://automater.ai/intel/zero-retention-vs-agent-memory/</link><guid>https://automater.ai/intel/zero-retention-vs-agent-memory/</guid><description>Zero data retention AI agents need memory to finish overnight jobs. What Anthropic&apos;s Covered Models and OpenAI&apos;s endpoint table retain, plus a boundary table.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Model Deprecation Routing Is a Continuity Drill, Not a Changelog Line</title><link>https://automater.ai/intel/forced-model-routing-continuity/</link><guid>https://automater.ai/intel/forced-model-routing-continuity/</guid><description>Model deprecation routing runbook: when a provider swaps the model under your ID, run ordered fallbacks, smoke tests, spend caps per lane and human notify.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>IM Channel Allowlists for Digital Employees: Gate the AutoClaw Telegram WhatsApp Bot Before It Joins #ops</title><link>https://automater.ai/intel/im-channel-allowlists-digital-employees/</link><guid>https://automater.ai/intel/im-channel-allowlists-digital-employees/</guid><description>An AutoClaw Telegram WhatsApp bot can drive a local agent from any group it joins. Write the channel allowlist, rate cap and revoke path before it hits #ops.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Multi-Agent Abort With Three Coordinators, One Human, and No Shared Bus</title><link>https://automater.ai/intel/vendor-coordinator-mashup-abort-bus/</link><guid>https://automater.ai/intel/vendor-coordinator-mashup-abort-bus/</guid><description>Multi-agent abort runbook for Claude Code Projects, Cursor Projects and the OpenAI Agents API: one human interrupt path, evidence export per vendor, pause all.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Kimi Work Scheduled Tasks Are Unattended Desktop Cron: Tier Them Like a Headless Run</title><link>https://automater.ai/intel/widget-scheduled-tasks-trust-tier/</link><guid>https://automater.ai/intel/widget-scheduled-tasks-trust-tier/</guid><description>Kimi Work scheduled tasks turn a live widget into unattended desktop cron. Tier it like a headless run: allowlists, no prod creds, run records, a human gate.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Kimi Pin to Desktop Allowlist: Always-Visible Agents Need a Gate</title><link>https://automater.ai/intel/pin-to-desktop-widget-allowlist/</link><guid>https://automater.ai/intel/pin-to-desktop-widget-allowlist/</guid><description>Kimi Pin to Desktop makes a widget always-visible. Write a pin allowlist like an MCP allowlist: who may pin, what data it may show, always-on-top and expiry.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>AutoClaw Cluster Mode Progress Panels vs Tray Stall Flags: Two Ways to See an Agent Is Still Working</title><link>https://automater.ai/intel/cluster-mode-vs-stall-flags/</link><guid>https://automater.ai/intel/cluster-mode-vs-stall-flags/</guid><description>AutoClaw Cluster Mode shows step status in chat; a tray shows stall flags. Two ways to see still working, neither a kill switch. Who owns abort on one laptop.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Cyber Capability Gates Are Becoming Product SKUs: Allowlist AI Cyber Model Access by Agent Role</title><link>https://automater.ai/intel/cyber-sku-agent-allowlist/</link><guid>https://automater.ai/intel/cyber-sku-agent-allowlist/</guid><description>Gemini 3.8 Flash Cyber, Mythos 5.1 and Astra Daybreak turned cyber capability into gated SKUs. Map AI cyber model access by agent role, log it, rehearse revoke.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Misalignment Disclosures Are a Fleet Evidence Problem: AI Agent Incident Evidence Before the Session Rotates</title><link>https://automater.ai/intel/misalignment-incident-evidence-export/</link><guid>https://automater.ai/intel/misalignment-incident-evidence-export/</guid><description>OpenAI disclosed six misalignment classes from its own runs. Build AI agent incident evidence for your fleet: tool trail, approvals, cost, diff and environment.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>AWS MCP Server Lambda Triage: Split Diagnostic Reads From Deploy Writes</title><link>https://automater.ai/intel/aws-fat-lambda-mcp-read-write-split/</link><guid>https://automater.ai/intel/aws-fat-lambda-mcp-read-write-split/</guid><description>The AWS MCP Server Lambda capability triages in one read-only call. Keep deploy writes on a separate IAM role, MCP config entry and human gate, logged apart.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>A Keyless MCP Server Is Its Own Trust Tier</title><link>https://automater.ai/intel/keyless-mcp-trust-tier/</link><guid>https://automater.ai/intel/keyless-mcp-trust-tier/</guid><description>A keyless MCP server has no grant to revoke, no published rate limit and feeds open-world text into context. The allowlist tier, five controls, a probe script.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Agent Dashboard Cards: One Operating Metaphor per Surface, or Your Team Drowns in Toast</title><link>https://automater.ai/intel/card-operating-metaphor-fleet-ux/</link><guid>https://automater.ai/intel/card-operating-metaphor-fleet-ux/</guid><description>Agent dashboard cards now mean four things: widgets, sub-agents, deliverables, threads. A dialect map of what each card is, what it can do, who owns the kill.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>OpenClaw vs AutoClaw: Buy the Metered Car or Build on the Engine</title><link>https://automater.ai/intel/openclaw-vs-autoclaw-buy-vs-build/</link><guid>https://automater.ai/intel/openclaw-vs-autoclaw-buy-vs-build/</guid><description>OpenClaw vs AutoClaw is an ownership question, not a price one. Decide who owns the gateway, skills, model and revoke path before a chat bot holds prod tokens.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Claude Fable 5.1 Cache Reads: Meter Dollars per Overnight Job</title><link>https://automater.ai/intel/fable-51-cache-read-overnight-meter/</link><guid>https://automater.ai/intel/fable-51-cache-read-overnight-meter/</guid><description>Claude Fable 5.1 cache reads cost 0.025x base input. Meter dollars per overnight agent job and cache-hit rate, learn what breaks the cache, set effort per job.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>DeepSeek V4.1-Flash: Rewrite the Off-Peak and Cache-Hit Runbook</title><link>https://automater.ai/intel/deepseek-v41-flash-offpeak-cache/</link><guid>https://automater.ai/intel/deepseek-v41-flash-offpeak-cache/</guid><description>DeepSeek V4.1-Flash halves prices off-peak and bills cache hits at $0.006/MTok. The UTC windows, the prompt shape that hits cache, and the re-based budgets.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>GitLab MCP 19.4: Always-Ask Writes as the Default Dialect</title><link>https://automater.ai/intel/gitlab-mcp-always-ask-writes/</link><guid>https://automater.ai/intel/gitlab-mcp-always-ask-writes/</guid><description>GitLab MCP 19.4 turns fixed tool rules into settings: reads allow, writes ask, deletes deniable. Copy that dialect into every agent on your repos, then audit.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Claude Code Projects: Anthropic&apos;s Cloud Coordinator, and Who Owns Abort When Threads Spawn</title><link>https://automater.ai/intel/claude-code-projects-cloud-coordinator/</link><guid>https://automater.ai/intel/claude-code-projects-cloud-coordinator/</guid><description>Claude Code Projects runs cloud threads on their own branches under one coordinator. Stop, Pause, Archive, Delete semantics, 200 threads a day, who owns abort.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Plugin4Shell: SHA Pinning Without HEAD Verify Is Theater</title><link>https://automater.ai/intel/plugin4shell-pin-verify-fleet-patch/</link><guid>https://automater.ai/intel/plugin4shell-pin-verify-fleet-patch/</guid><description>Plugin4Shell beat SHA pinning in four coding agents. A Tuesday drill: version floors by harness, post-checkout HEAD assert, auto-update posture, plugin census.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>GPT-6 Astra Computer Use: Inventory Every Host That Can Drive the Mouse</title><link>https://automater.ai/intel/gpt-6-astra-computer-use-host-inventory/</link><guid>https://automater.ai/intel/gpt-6-astra-computer-use-host-inventory/</guid><description>GPT-6 Astra computer use makes every screen a model can reach a trust tier. Build the host ledger first: who can drive it, which account, kill switch, evidence.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Kimi Work Dashboard Cards vs Automater Lite Home Cards: The Widget and the Status Wall</title><link>https://automater.ai/intel/kimi-dashboard-vs-lite-session-cards/</link><guid>https://automater.ai/intel/kimi-dashboard-vs-lite-session-cards/</guid><description>Kimi Work Dashboard cards are widgets the agent builds; Automater Lite Home cards are a status wall for running CLIs. The decision table, limits and kill rules.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>AutoClaw: Inventory the Digital Employee Before Cluster Mode Fans Out</title><link>https://automater.ai/intel/autoclaw-digital-employee-inventory/</link><guid>https://automater.ai/intel/autoclaw-digital-employee-inventory/</guid><description>AutoClaw is a digital employee on the machine. Inventory credit plan, bot tokens, connectors, Cluster Mode, wake-lock and Hermes gate, then test the kill path.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 19 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Human in the Loop Approval Fatigue: Queues That Don&apos;t Become Rubber Stamps</title><link>https://automater.ai/intel/hitl-approval-queue-hygiene/</link><guid>https://automater.ai/intel/hitl-approval-queue-hygiene/</guid><description>Human in the loop approval fatigue turns agent approvals into rubber stamps. Tier by consequence, batch the low-risk, expire stale prompts, measure the reflex.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>AI Agent Environment Setup Is Two-Thirds of the Failure: Fleet Readiness Before the Prompt</title><link>https://automater.ai/intel/agent-env-setup-readiness/</link><guid>https://automater.ai/intel/agent-env-setup-readiness/</guid><description>AI agent environment setup caused 65% of GitTaskBench failures. Run this six-gate preflight (image, lockfile, toolchain, smoke test, budget, score) first.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Agent Evaluation CI Gates: Fail the PR When the Agent Regresses</title><link>https://automater.ai/intel/agent-eval-ci-regression-gates/</link><guid>https://automater.ai/intel/agent-eval-ci-regression-gates/</guid><description>Build an agent evaluation CI gate: deploy the agent, run a fixed prompt set, score its tool choices, and block the PR on regression. Thresholds and YAML inside.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>The Agent Memory Benchmark You Can Run on a Tuesday</title><link>https://automater.ai/intel/memory-layer-tuesday-bakeoff/</link><guid>https://automater.ai/intel/memory-layer-tuesday-bakeoff/</guid><description>Run an agent memory benchmark on your repos in one afternoon: a frozen suite, three arms, P99 latency, cost per 1k lookups, harm cases, and a decision rule.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>From Dashboard Theater to Pre-Action Gates: Where AI Agent Security Gates Belong</title><link>https://automater.ai/intel/pre-action-gates-not-dashboards/</link><guid>https://automater.ai/intel/pre-action-gates-not-dashboards/</guid><description>AI agent security gates should act before each tool call: approve, prune, broker, revoke, and record. Use the matrix, vendor questions, and Tuesday drill.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>Windows IT After GitSpawn: An AI Coding Agent Endpoint Policy</title><link>https://automater.ai/intel/windows-it-agent-endpoint-policy/</link><guid>https://automater.ai/intel/windows-it-agent-endpoint-policy/</guid><description>After GitSpawn, an AI coding agent endpoint policy for Windows IT: standard-user accounts, AppLocker version floors, git overrides by policy, intake quarantine.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>AI PR Review Agent Policy: Agents Propose, Humans Merge</title><link>https://automater.ai/intel/pr-review-agent-policy/</link><guid>https://automater.ai/intel/pr-review-agent-policy/</guid><description>An AI PR review agent should propose, never merge. The policy: always-human paths, a CODEOWNERS shape, no bot-approves-bot, and reviewer quality you can track.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>When Not to Use an AI Agent Coordinator</title><link>https://automater.ai/intel/when-not-to-use-a-coordinator/</link><guid>https://automater.ai/intel/when-not-to-use-a-coordinator/</guid><description>An AI agent coordinator copies its first mistake N times. The decision rule, a sixty-second self-test, and four brakes for sensitive or air-gapped work.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>AI Agent Cost Alerts: Anomaly Thresholds, Paging, and Auto-Pause for Agent Fleets</title><link>https://automater.ai/intel/agent-cost-anomaly-alerts/</link><guid>https://automater.ai/intel/agent-cost-anomaly-alerts/</guid><description>AI agent cost alerts for fleets: baseline two weeks, define a 3x-day anomaly plus velocity and worker triggers, page with the right facts, pause spawns first.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Claude Code Permission Modes, Codex Sandboxes, One House Dialect</title><link>https://automater.ai/intel/unified-permission-modes/</link><guid>https://automater.ai/intel/unified-permission-modes/</guid><description>Map Claude Code permission modes and Codex sandbox flags onto three house tiers, encode them once, audit which host drifted, and learn what subagents inherit.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Managed Agents Comparison: Bedrock, Claude, OpenAI. Pick Failover First.</title><link>https://automater.ai/intel/managed-agents-failover-matrix/</link><guid>https://automater.ai/intel/managed-agents-failover-matrix/</guid><description>A managed agents comparison of Bedrock AgentCore, Claude Managed Agents, and the OpenAI Agents API on state, tool schemas, evidence, kill switch, and residency.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>codex exec Headless Modes: Unattended Is a Different Trust Tier</title><link>https://automater.ai/intel/headless-agent-trust-tier/</link><guid>https://automater.ai/intel/headless-agent-trust-tier/</guid><description>codex exec headless runs, Claude Code print mode, and Actions wrappers need their own trust tier: flag shapes per tier, no prod creds, an evidence pack per run.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Interrupt AI Agent Coordinators Safely: What Pause, Redirect, and Abort Must Mean</title><link>https://automater.ai/intel/interruptible-agent-coordinators/</link><guid>https://automater.ai/intel/interruptible-agent-coordinators/</guid><description>How to interrupt AI agent coordinators without orphaning work: a signal ladder, tool-boundary stops, branch-per-worker git rules, a redirect protocol, a drill.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>CI Failure to Fix: The AI Agent CI Loop That Doesn&apos;t Thrash</title><link>https://automater.ai/intel/ci-agent-fix-loop-guards/</link><guid>https://automater.ai/intel/ci-agent-fix-loop-guards/</guid><description>An AI agent CI loop that retries every red check will thrash. Cap attempts at three, gate on new failure signatures, set a cost ceiling per PR, then hand off.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>MCP Server Inventory Before Allowlist: The Weekly Ritual</title><link>https://automater.ai/intel/mcp-server-inventory-ritual/</link><guid>https://automater.ai/intel/mcp-server-inventory-ritual/</guid><description>Run a 45-minute weekly MCP server inventory: find every config, attribute each server, score blast radius, diff versions, then keep, prune, or pin each row.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Evidence When the Harness Is Vendor-Hosted: Export the AI Agent Audit Trail Before Access Changes</title><link>https://automater.ai/intel/vendor-hosted-agent-evidence/</link><guid>https://automater.ai/intel/vendor-hosted-agent-evidence/</guid><description>Vendors keep the agent session record. Export the AI agent audit trail: tool calls, approvals, costs, final diff, and environment before access changes.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>AI Agent Identity as a Service Principal: Stop Sharing Human SSO</title><link>https://automater.ai/intel/agent-service-principal-sso/</link><guid>https://automater.ai/intel/agent-service-principal-sso/</guid><description>Give every AI agent identity of its own: a service principal, GitHub App, or IAM role, scoped grants, short tokens, a broker, and revocation that spares users.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Overnight Agents: The AI Agent Merge Gates That Define Done</title><link>https://automater.ai/intel/overnight-agent-merge-gates/</link><guid>https://automater.ai/intel/overnight-agent-merge-gates/</guid><description>Overnight agents open PRs while you sleep. AI agent merge gates define done: tests, a diff ceiling, secret scan, path rules, an eval threshold, human approval.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Hybrid Agent Fleets: Laptop + Cloud VM Without the Sync-Everything Fantasy</title><link>https://automater.ai/intel/hybrid-laptop-cloud-fleet/</link><guid>https://automater.ai/intel/hybrid-laptop-cloud-fleet/</guid><description>Run a hybrid agent fleet across laptop, cloud VM, and a vendor&apos;s computer: per-host identity, transcript provenance, a kill switch per host, git-only hand-offs.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>When the Coordinator Spawns a Thousand Subagents: Metering Subagent Token Cost at Fan-Out</title><link>https://automater.ai/intel/subagent-fanout-metering/</link><guid>https://automater.ai/intel/subagent-fanout-metering/</guid><description>Subagent token cost climbs one worker at a time. Cap concurrent workers, budget per task, log every spawn, and kill orphans before a coordinator fans out.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Slack AI Agent Subscriptions Without a Control Plane</title><link>https://automater.ai/intel/slack-agent-subscriptions-policy/</link><guid>https://automater.ai/intel/slack-agent-subscriptions-policy/</guid><description>Slack AI agent subscriptions turn every channel message into a worker. The runbook: channel allowlist, spawn cap, event dedupe, human gate on merge, revoke path</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>The Sandbox Is a Suggestion: What Black Hat&apos;s AI Agent Sandbox Escapes Broke for Operators</title><link>https://automater.ai/intel/sandbox-is-a-suggestion/</link><guid>https://automater.ai/intel/sandbox-is-a-suggestion/</guid><description>An AI agent sandbox escape hit Claude Code Action, Gemini CLI, and Codex at Black Hat 2026. The compensating controls operators can install this week.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>GitSpawn Week: Untrusted Repos Are an Intake Problem</title><link>https://automater.ai/intel/gitspawn-repo-intake-checklist/</link><guid>https://automater.ai/intel/gitspawn-repo-intake-checklist/</guid><description>GitSpawn turns opening a folder into code execution. A Tuesday intake checklist: clone-only policy, read .git/config first, version floors, a quarantine user.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Nightfall&apos;s MCP Gateway: Buy the Proxy or Build the Six Checks</title><link>https://automater.ai/intel/commercial-mcp-gateway-buy-or-build/</link><guid>https://automater.ai/intel/commercial-mcp-gateway-buy-or-build/</guid><description>A buying guide for the MCP gateway decision: score Nightfall&apos;s proxy against six checks, run a two-week acceptance test, and see what no SaaS gateway covers.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>OpenAI Agents API Is a Managed Harness. Run the Continuity Drill Anyway.</title><link>https://automater.ai/intel/openai-agents-api-continuity/</link><guid>https://automater.ai/intel/openai-agents-api-continuity/</guid><description>The OpenAI Agents API rents you the Codex loop. Inventory model, harness, and sandbox dependencies, write failover routes, and keep the record on your disk.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Cursor Projects vs. the Tray: Who Owns Your Fleet Coordinator</title><link>https://automater.ai/intel/cursor-projects-vs-tray-fleet/</link><guid>https://automater.ai/intel/cursor-projects-vs-tray-fleet/</guid><description>Cursor Projects puts a fleet coordinator in the IDE. The ownership table: what it runs, what a local tray owns (stall flags, kill switch), and what breaks.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 13 Sep 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>Open-Source Agent Gateway vs Vendor Security Suite: Pick the Layer You Refuse to Lock In</title><link>https://automater.ai/intel/open-gateway-vs-vendor-suite/</link><guid>https://automater.ai/intel/open-gateway-vs-vendor-suite/</guid><description>Agent gateway open source vs vendor suite: a six-layer scoring runbook for which control-plane layers stay portable, plus an exit test and vendor questions.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 06 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Rehearse the Provider Cutoff: Model Failover for Agent Fleets</title><link>https://automater.ai/intel/fleet-continuity-provider-cutoff/</link><guid>https://automater.ai/intel/fleet-continuity-provider-cutoff/</guid><description>Use the proposed Cursor cutoff to rehearse model provider failover: inventory dependencies, validate supported routes, test quality, and price capacity.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 06 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>AI Agent Identity: Treat Every Agent as a Privileged User</title><link>https://automater.ai/intel/agents-as-privileged-users/</link><guid>https://automater.ai/intel/agents-as-privileged-users/</guid><description>AI agent identity runbook: workload identities, scoped grants, credential brokers, provider expiry limits, and revocation tests across each trust boundary.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 05 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>GPT-6 Astra at the Critical Cyber Threshold: An Operator’s Runbook</title><link>https://automater.ai/intel/astra-cyber-threshold/</link><guid>https://automater.ai/intel/astra-cyber-threshold/</guid><description>Assess GPT-6 Astra enterprise access and prepare production controls: contain active agents, reconstruct their actions, and gate consequential writes.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 05 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Shadow MCP Is the New Shadow IT</title><link>https://automater.ai/intel/shadow-mcp/</link><guid>https://automater.ai/intel/shadow-mcp/</guid><description>Shadow MCP is the new shadow IT. One-week runbook: sweep harness configs, build an approved MCP inventory, quarantine unregistered servers, catch drift nightly.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 04 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Stateless MCP Is a Gift to Gateways</title><link>https://automater.ai/intel/stateless-mcp-gateways/</link><guid>https://automater.ai/intel/stateless-mcp-gateways/</guid><description>Route stateless MCP by validated headers, separate transport logs from tool outcomes, and migrate legacy clients with explicit policies, cache keys, and checks.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 04 Sep 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Deadbugz Killed Approve-Once: The Runtime Controls MCP Needs Now</title><link>https://automater.ai/intel/mcp-approve-once-is-dead/</link><guid>https://automater.ai/intel/mcp-approve-once-is-dead/</guid><description>Deadbugz hid malicious MCP metadata behind ordinary calls. Build a runtime loop with pinned manifests, re-approval, bounded egress, and call-time evidence.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 04 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>The Agent Gateway Is the Control Plane for Enterprise Agents</title><link>https://automater.ai/intel/agent-gateway-control-plane/</link><guid>https://automater.ai/intel/agent-gateway-control-plane/</guid><description>An agent gateway is the control plane between enterprise agents and their tools. Run six checks this week: access, approvals, secrets, audit, revoke, tenancy.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 04 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Stop Trusting the Mermaid</title><link>https://automater.ai/intel/stop-trusting-the-mermaid/</link><guid>https://automater.ai/intel/stop-trusting-the-mermaid/</guid><description>Review AI-generated diagrams as claims: inspect the transcript, diff, command outputs and current tests, then reproduce suspicious behavior before merge.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 01 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Cold Restarts on Windows: Sessions That Don&apos;t Come Back</title><link>https://automater.ai/intel/cold-restarts-on-windows/</link><guid>https://automater.ai/intel/cold-restarts-on-windows/</guid><description>Recover AI sessions after a Windows reboot: verify transcripts, restore WSL and Docker bottom-up, inspect the working tree, then resume or re-brief each agent.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 01 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Companion vs Harness vs Computer</title><link>https://automater.ai/intel/companion-harness-computer/</link><guid>https://automater.ai/intel/companion-harness-computer/</guid><description>AI companion vs harness vs computer-use agent vs ADE: define each layer, map who commands whom, and identify the capability a product actually sells.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 01 Sep 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>A Day on the Windows Tray</title><link>https://automater.ai/intel/a-day-on-the-windows-tray/</link><guid>https://automater.ai/intel/a-day-on-the-windows-tray/</guid><description>A composite day of Windows AI fleet management: recover after a reboot, inspect a stalled session, review a diff, meter parallel work, and redact a secret.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 01 Sep 2026 00:00:00 GMT</pubDate><category>Product</category></item><item><title>Memory That Burns Quota</title><link>https://automater.ai/intel/memory-that-burns-quota/</link><guid>https://automater.ai/intel/memory-that-burns-quota/</guid><description>Measure Claude memory cost across auto memory, CLAUDE.md, and plugins, then replace indiscriminate replay with a bounded, archive-first retrieval policy.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 01 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>You Can&apos;t Replay What You Can&apos;t See</title><link>https://automater.ai/intel/fleet-replay/</link><guid>https://automater.ai/intel/fleet-replay/</guid><description>An incident playbook for AI session replay: search supported session records together, inspect tool calls, match them to the diff, and resume where supported.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 01 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Claude Code Permission Modes Are Fleet Policy</title><link>https://automater.ai/intel/restricted-mode-fleet-policy/</link><guid>https://automater.ai/intel/restricted-mode-fleet-policy/</guid><description>Choose Claude Code permission modes by repository trust, define who can escalate them, and audit the same policy across an AI-agent fleet.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 01 Sep 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Perplexity Personal Computer vs a Windows Tray Companion</title><link>https://automater.ai/intel/perplexity-computer-vs-tray/</link><guid>https://automater.ai/intel/perplexity-computer-vs-tray/</guid><description>Compare Perplexity Personal Computer for Windows with a local tray companion across role, price, data boundary, platform, and vendor risk.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 31 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>&apos;AI With No Restrictions&apos; Usually Means Local</title><link>https://automater.ai/intel/ai-with-no-restrictions/</link><guid>https://automater.ai/intel/ai-with-no-restrictions/</guid><description>Most people searching for AI with no restrictions don&apos;t want jailbreaks. They want local: models, transcripts, and installs no vendor can cap or cut off.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 31 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>What Is an AI Computer? A Daily-Driver Definition</title><link>https://automater.ai/intel/what-is-an-ai-computer/</link><guid>https://automater.ai/intel/what-is-an-ai-computer/</guid><description>What is an AI computer? One name covers three layers — the machine, the computer-use agent, and the operating layer. A definition with 2026&apos;s products mapped.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 31 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>When the Harness Is a Plugin Bag, You Still Need a Boss</title><link>https://automater.ai/intel/plugin-bag-needs-a-boss/</link><guid>https://automater.ai/intel/plugin-bag-needs-a-boss/</guid><description>DeepSeek Harness makes the loop, sandbox, and model swappable plugins. That eases agent harness lock-in — and still leaves your fleet without a boss layer.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 31 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Plugins, Gists, and Rotting AGENTS.md</title><link>https://automater.ai/intel/agents-md-rot/</link><guid>https://automater.ai/intel/agents-md-rot/</guid><description>AGENTS.md files drift, conflict, and multiply until no two agents run the same job. The playbook: what stays, what becomes a skill, and the quarterly rot audit.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 31 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Muse Code&apos;s Session Bus: Inter-Session Messaging as the Operator Primitive</title><link>https://automater.ai/intel/muse-code-session-bus/</link><guid>https://automater.ai/intel/muse-code-session-bus/</guid><description>The Muse Code session bus is live: inter-session messaging over a local socket, plans from $5. What the primitive does to visibility, token burn, and replay.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 31 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>The Desktop ADE: Session Explorer, WSL, Docker, and the Topology Chat Never Shows</title><link>https://automater.ai/intel/desktop-ade-session-explorer/</link><guid>https://automater.ai/intel/desktop-ade-session-explorer/</guid><description>Tour the Automater Desktop beta: searchable cross-provider Session Explorer plus a separate live topology for WSL, Docker stacks, containers, and hosts.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 30 Aug 2026 00:00:00 GMT</pubDate><category>Product</category></item><item><title>Cursor After SpaceX: The Proposed Cutoff Is a Command-Center Problem</title><link>https://automater.ai/intel/cursor-after-spacex/</link><guid>https://automater.ai/intel/cursor-after-spacex/</guid><description>SpaceX closed its Cursor acquisition August 14; OpenAI proposed ending model access November 12. Here is the record and a practical continuity checklist.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 30 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Claude Code After September 14: The 17% Cut You Will Feel</title><link>https://automater.ai/intel/claude-code-after-september-14/</link><guid>https://automater.ai/intel/claude-code-after-september-14/</guid><description>Claude Code limits change September 14: the +50% boost ends and settles at +25%, a 17% reduction from the temporary allowance. See the math and checklist.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 30 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Many Assistants, One Boss — What Grok Bot Changes</title><link>https://automater.ai/intel/one-boss-grokbot/</link><guid>https://automater.ai/intel/one-boss-grokbot/</guid><description>What Grok Bot is, where its data lives, and how to manage it beside local AI CLIs without blurring cloud storage, permissions, or transcript boundaries.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 30 Aug 2026 00:00:00 GMT</pubDate><category>Product</category></item><item><title>Stall Flags and Keepalive: AgentOps From the System Tray</title><link>https://automater.ai/intel/stall-flags-and-keepalive/</link><guid>https://automater.ai/intel/stall-flags-and-keepalive/</guid><description>AI agent monitoring from the Windows tray: what a stall flag means, amber vs. green, what keepalive prevents, and what a tray honestly can&apos;t fix.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 30 Aug 2026 00:00:00 GMT</pubDate><category>Product</category></item><item><title>Cross-CLI Memory That Survives the Session</title><link>https://automater.ai/intel/cross-cli-session-memory/</link><guid>https://automater.ai/intel/cross-cli-session-memory/</guid><description>AI session memory that outlives one tool: keep supported histories searchable and local, keep preferences small, and keep secrets out of both.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 30 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Home AI Without a Dedicated AI Workstation</title><link>https://automater.ai/intel/home-ai-without-a-workstation/</link><guid>https://automater.ai/intel/home-ai-without-a-workstation/</guid><description>Build a useful home AI setup on the PC you own. Learn when local inference hardware earns its cost, and when archive, monitoring, and metering matter more.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 29 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Local-First AI: The Vault and Session Archives That Stay Local</title><link>https://automater.ai/intel/local-first-ai-vault/</link><guid>https://automater.ai/intel/local-first-ai-vault/</guid><description>Local-first AI as operating practice: keep the session archive on your disk, scrub secrets before indexing, and map every optional connected data path.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 29 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>$0 Tray vs $29/Year vs $20/Month: The Operating Bill, Not the Token Bill</title><link>https://automater.ai/intel/operating-bill-vs-token-bill/</link><guid>https://automater.ai/intel/operating-bill-vs-token-bill/</guid><description>Your AI subscription cost is only half the bill. Price the other half — the operating bill: $0 tray vs $29/year vs $20/month — with two fleet scenarios.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 29 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Corporate AI on a Windows PC: What IT Is Actually Buying</title><link>https://automater.ai/intel/corporate-ai-windows-pc/</link><guid>https://automater.ai/intel/corporate-ai-windows-pc/</guid><description>IT is being asked to put AI agents on company computers. A corporate AI checklist that works: where sessions live, what leaves disk, who sees the fleet.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 29 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>What People Mean by &apos;AI Computer&apos; on Windows in 2026</title><link>https://automater.ai/intel/ai-computers-on-windows/</link><guid>https://automater.ai/intel/ai-computers-on-windows/</guid><description>Searching for AI computers? On Windows in 2026 the phrase means agents on the PC you own — computer-use workers, the tray boss that runs them, no new hardware.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 29 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>Automater Lite: The Windows Tray That Watches Your AI Fleet</title><link>https://automater.ai/intel/automater-lite-windows-tray/</link><guid>https://automater.ai/intel/automater-lite-windows-tray/</guid><description>Automater Lite is a free Windows tray companion with a local AI-session Library, fleet status, search, usage meters, and signed updates. Pro is $29/year.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 29 Aug 2026 00:00:00 GMT</pubDate><category>Product</category></item><item><title>EU AI Act Article 50 Is Live: What Agent Builders Must Disclose Now</title><link>https://automater.ai/intel/eu-ai-act-agents/</link><guid>https://automater.ai/intel/eu-ai-act-agents/</guid><description>EU AI Act Article 50 took effect August 2, 2026. What agent builders must disclose, how to mark AI content, who is in scope, and a practical checklist.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 28 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Evals for AI Agents: How to Measure Software That Acts</title><link>https://automater.ai/intel/evals-for-ai-agents/</link><guid>https://automater.ai/intel/evals-for-ai-agents/</guid><description>What are evals in AI? A plain definition, four grader types, pass@k worked examples, and a five-step plan for your first agent eval suite in one week.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 28 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>Types of AI Agents, Explained With Real 2026 Examples</title><link>https://automater.ai/intel/types-of-ai-agents/</link><guid>https://automater.ai/intel/types-of-ai-agents/</guid><description>The five textbook types of agents in AI, then the 2026 taxonomy that matters: four axes, eight real tools mapped, and a straight answer about Copilot.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 28 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>OpenAI Codex: Harness, Models, and Plans — the Daily-Driver Review</title><link>https://automater.ai/intel/openai-codex-deep-dive/</link><guid>https://automater.ai/intel/openai-codex-deep-dive/</guid><description>OpenAI Codex reviewed as a daily driver: the Rust CLI, cloud fan-out, IDE extension, GPT-5.6-era models, and what each ChatGPT plan actually sustains.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 27 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>LangGraph in 2026: Deep Review, Real Use Cases, and When to Use Something Else</title><link>https://automater.ai/intel/langgraph-guide-alternatives/</link><guid>https://automater.ai/intel/langgraph-guide-alternatives/</guid><description>An honest LangGraph review for 2026: the graph model, checkpointing and interrupts, three real builds, platform pricing scrutiny, and when to skip it.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 27 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>Agentic AI vs Generative AI: What Actually Changes When Software Gets Goals</title><link>https://automater.ai/intel/agentic-ai-vs-generative-ai/</link><guid>https://automater.ai/intel/agentic-ai-vs-generative-ai/</guid><description>Agentic AI vs generative AI, minus the vendor gloss: the architecture gap, a real comparison table, cost and risk asymmetries, and when a plain prompt wins.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 27 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>Amp, Crush, and OpenClaw: The New-Wave Harnesses Worth Watching</title><link>https://automater.ai/intel/new-wave-harnesses/</link><guid>https://automater.ai/intel/new-wave-harnesses/</guid><description>Eight new AI coding tools mapped: Amp, Crush, OpenClaw, Ecodex, and more — with evidence tiers, survivor criteria, and a safe two-week trial protocol for 2026.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Wed, 26 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>MCP Goes Stateless: What the 2026-07-28 Spec Changes for Builders</title><link>https://automater.ai/intel/mcp-2026-spec/</link><guid>https://automater.ai/intel/mcp-2026-spec/</guid><description>The 2026-07-28 MCP spec retires sessions, replaces elicitation with MRTR, and hardens OAuth. What changed, why, and how to migrate servers and clients.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Wed, 26 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Beyond the Terminal: Cowork Tools and Autonomous AI Agent Workspaces</title><link>https://automater.ai/intel/cowork-tools-autonomous-workspaces/</link><guid>https://automater.ai/intel/cowork-tools-autonomous-workspaces/</guid><description>What an AI agent workspace really delivers in 2026 — Manus, Genspark, Devin, ChatGPT agent, and Claude Cowork compared on task fit, pricing, and trust.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Wed, 26 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>DeepSeek Harness: The &quot;Everything Is a Plugin&quot; Agent Runtime, Reviewed</title><link>https://automater.ai/intel/deepseek-harness-review/</link><guid>https://automater.ai/intel/deepseek-harness-review/</guid><description>DeepSeek Harness reviewed: the MIT agent runtime where everything is a plugin. Four modes, a session-log core, real sandboxing — and who should switch now.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 25 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>GitHub Copilot CLI, Grok, and Amazon Q: The Other American Harnesses</title><link>https://automater.ai/intel/other-us-harnesses/</link><guid>https://automater.ai/intel/other-us-harnesses/</guid><description>GitHub Copilot CLI after the June 2026 AI-credit switch, Grok&apos;s missing CLI, Amazon Q&apos;s blocked signups: one honest review of the second-tier US harnesses.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 25 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Securing AI Agents: Prompt Injection, Least Privilege, and the New Attack Surface</title><link>https://automater.ai/intel/securing-ai-agents/</link><guid>https://automater.ai/intel/securing-ai-agents/</guid><description>AI agent security in practice: the lethal trifecta, prompt injection, MCP hardening per the June 2026 government guidance, and controls that bound blast radius.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 25 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>AgentOps: Running a Fleet of AI Agents Without Losing Your Mind</title><link>https://automater.ai/intel/agentic-ops/</link><guid>https://automater.ai/intel/agentic-ops/</guid><description>Agentic ops, defined by people who run agent fleets daily: the five-layer AgentOps stack, the four metrics that matter, and a starter incident runbook.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 25 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>Goodbye, Llama? Muse Spark and Meta&apos;s Proprietary Pivot</title><link>https://automater.ai/intel/meta-muse-spark-llama-pivot/</link><guid>https://automater.ai/intel/meta-muse-spark-llama-pivot/</guid><description>Muse Spark is Meta&apos;s first proprietary model since Llama. What it is, what happens to Llama open source, and how teams standardized on it should hedge.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 24 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>The Local AI Workstation, 2026 Edition: DGX Spark, Ryzen AI Max, or a Mac?</title><link>https://automater.ai/intel/local-ai-workstation/</link><guid>https://automater.ai/intel/local-ai-workstation/</guid><description>DGX Spark, Ryzen AI Max 395, Mac Studio, or used 3090s? The 2026 local LLM hardware guide: bandwidth vs capacity, three priced builds, and when local wins.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 24 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>The Great Harness Die-Off: Gemini CLI, iFlow, and How to Pick Survivors</title><link>https://automater.ai/intel/harness-die-off/</link><guid>https://automater.ai/intel/harness-die-off/</guid><description>The Gemini CLI shutdown was no one-off: iFlow, Roo Code, Cascade, and Phind died in 2026 too. Get the dated casualty list, survivor traits, and exit checklist.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 24 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Databricks and the Enterprise Agent Platform Race</title><link>https://automater.ai/intel/databricks-enterprise-agents/</link><guid>https://automater.ai/intel/databricks-enterprise-agents/</guid><description>What Databricks is, who owns it, and whether the lakehouse can own enterprise agents: the data-gravity thesis, the MCP counter-case, and a verdict by workload.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 24 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>OpenAI&apos;s Agentic Stack: From Research Papers to Dev Day to What You Can Build</title><link>https://automater.ai/intel/openai-agentic-stack/</link><guid>https://automater.ai/intel/openai-agentic-stack/</guid><description>Every OpenAI Dev Day decoded, 2023–2026: the agent primitives that survived, what the Assistants API sunset teaches, and what to build on without whiplash.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 24 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>Fleet and Swarm Agentic Workflow Architectures in 2026</title><link>https://automater.ai/intel/fleet-swarm-agentic-workflow-architectures/</link><guid>https://automater.ai/intel/fleet-swarm-agentic-workflow-architectures/</guid><description>Compare fleet and swarm agentic workflow architectures in 2026: durable state, bounded delegation, isolated worktrees, evaluations, permissions, and costs.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 23 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>The Mid-2026 Frontier Scorecard: Fable 5 vs GPT-5.6 vs Gemini 3.1</title><link>https://automater.ai/intel/frontier-model-scorecard-2026/</link><guid>https://automater.ai/intel/frontier-model-scorecard-2026/</guid><description>Which is the best AI model in 2026? Claude Fable 5, GPT-5.6 (Sol), and Gemini 3.1 scored for real agent work — plus the open models closing the gap fast.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 23 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>The Test Harness, Reinvented: QA for Agentic Software</title><link>https://automater.ai/intel/testing-agentic-software/</link><guid>https://automater.ai/intel/testing-agentic-software/</guid><description>The test harness in software testing, defined in 49 words — then rebuilt for AI agents: sandboxes, replayed tools, trajectory checks, and budget caps.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 23 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>The Best Agentic AI Tools in 2026, Tested by Daily-Driver Standards</title><link>https://automater.ai/intel/best-agentic-ai-tools/</link><guid>https://automater.ai/intel/best-agentic-ai-tools/</guid><description>Agentic AI tools ranked by daily-driver testing: coding CLIs, IDEs, workflow platforms, frameworks, and the operating layer — with a rubric you can rerun.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 22 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>The Open-Source AI Agent Stack Serious Builders Run in 2026</title><link>https://automater.ai/intel/open-source-ai-agent-stack/</link><guid>https://automater.ai/intel/open-source-ai-agent-stack/</guid><description>What open source artificial intelligence really means, and the agent stack that runs on it in 2026: models, runtimes, orchestration, MCP, and three recipes.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 22 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>How AI Agents Actually Work: The Loop Behind the Magic</title><link>https://automater.ai/intel/how-ai-agents-work/</link><guid>https://automater.ai/intel/how-ai-agents-work/</guid><description>Artificial intelligence agents are a loop: a model deciding, tools acting, results feeding back. See a real annotated trace, then build one in 50 lines.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 22 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>The Subscription Squeeze: Copilot Goes Metered and the Flat-Rate Era Wobbles</title><link>https://automater.ai/intel/subscription-squeeze/</link><guid>https://automater.ai/intel/subscription-squeeze/</guid><description>The GitHub Copilot pricing change swapped premium requests for metered AI credits on June 1, 2026. Why flat-rate AI plans are wobbling — and how to defend.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 21 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>What Is an Agent Harness? The 2026 Field Map of AI Coding Harnesses</title><link>https://automater.ai/intel/agent-harness-landscape/</link><guid>https://automater.ai/intel/agent-harness-landscape/</guid><description>An AI agent harness turns a model into a working agent. Get the practitioner&apos;s definition plus the 2026 field map: survivors, casualties, and the new wave.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 21 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>The DeepSeek Effect: Open Models and the Collapsing Cost of Running Agents</title><link>https://automater.ai/intel/deepseek-agent-economics/</link><guid>https://automater.ai/intel/deepseek-agent-economics/</guid><description>DeepSeek collapsed the cost of running AI agents. We price one real workflow across three tiers — down to $0.14/M — and show you which steps to re-route.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 21 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>What Is Agentic Coding? From Autocomplete to Autonomous Engineering</title><link>https://automater.ai/intel/what-is-agentic-coding/</link><guid>https://automater.ai/intel/what-is-agentic-coding/</guid><description>What is agentic coding? Get the practitioner&apos;s definition, the August 2026 tool map, core team practices, and the honest anti-patterns that burn teams.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 21 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>Open-Source Coding Agents: Aider, Cline, OpenCode, Goose, and Friends</title><link>https://automater.ai/intel/open-source-harnesses/</link><guid>https://automater.ai/intel/open-source-harnesses/</guid><description>Field review of six open source coding agents — Aider, Cline, OpenCode, Goose, OpenHands, Crush — with maintenance health, BYOK math, and honest picks.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 20 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Agentic Browsers in 2026: Atlas, Comet, and the Injection Problem That Won&apos;t Die</title><link>https://automater.ai/intel/agentic-browsers-2026/</link><guid>https://automater.ai/intel/agentic-browsers-2026/</guid><description>Atlas, Comet, and Dia can browse and act for you. What agentic browsers do well in 2026, why prompt injection may never be solved, and a safe-use playbook.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Wed, 19 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Context Engineering: The 2026 Playbook for Agents That Don&apos;t Forget</title><link>https://automater.ai/intel/context-engineering-playbook/</link><guid>https://automater.ai/intel/context-engineering-playbook/</guid><description>Context engineering keeps agents sharp past turn 30. See what actually fills the window, six techniques with real configs, and a one-week adoption plan.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Wed, 19 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Ox Alpha: The Free Stealth Model Nobody Will Claim</title><link>https://automater.ai/intel/ox-alpha-mystery-model/</link><guid>https://automater.ai/intel/ox-alpha-mystery-model/</guid><description>Ox Alpha appeared free and anonymous on August 20, 2026 — 1M context, no maker named. The benchmark that collapsed, the GLM fingerprints, what&apos;s safe to send.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 18 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Harness Engineering: The Discipline Your Agents Were Missing</title><link>https://automater.ai/intel/harness-engineering/</link><guid>https://automater.ai/intel/harness-engineering/</guid><description>Harness engineering is why one team ships clean agent PRs while another babysits loops. Learn the six subsystems, day-one practices, and a maturity ladder.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 18 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Muse Code: Meta&apos;s Terminal Coding Agent, Reviewed</title><link>https://automater.ai/intel/meta-muse-code/</link><guid>https://automater.ai/intel/meta-muse-code/</guid><description>Muse Code reviewed: Meta&apos;s beta terminal coding agent, the Muse Spark 1.2 engine, its 59.3% DeepSWE standing, open questions, and how to trial it safely.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 17 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>AI Agent Frameworks in 2026: How to Actually Choose</title><link>https://automater.ai/intel/ai-agent-frameworks/</link><guid>https://automater.ai/intel/ai-agent-frameworks/</guid><description>Skip the listicles. A working decision guide to AI agent frameworks in 2026: a taxonomy, an eight-check rubric, a decision tree, and when to use none at all.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 17 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>Kimi K3, GLM-5.2, DeepSeek V4: The Open Models Crowding the Frontier</title><link>https://automater.ai/intel/open-weight-scorecard-2026/</link><guid>https://automater.ai/intel/open-weight-scorecard-2026/</guid><description>Kimi K3, GLM-5.2, DeepSeek V4, Qwen3-Coder-Next: verified figures, prices, deployment lanes, and how to pick the best open source model 2026 for agent work.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 16 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Hardening MCP in Production: The Government-Guidance Era</title><link>https://automater.ai/intel/mcp-security-hardening/</link><guid>https://automater.ai/intel/mcp-security-hardening/</guid><description>The June 2026 government CSI made MCP security official. Get the threat classes, a hardening checklist mapped to the guidance, and the new auth upgrades.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 16 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Open-Weight Models That Can Actually Drive a Harness</title><link>https://automater.ai/intel/open-weight-models-for-agents/</link><guid>https://automater.ai/intel/open-weight-models-for-agents/</guid><description>Which open-weight models can actually drive a coding agent in 2026? We define agent-fitness, profile DeepSeek V4 to Kimi K3, and map serving and hardware.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 16 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>Gemini CLI Is Gone: The Antigravity Era of Google&apos;s Agentic Coding</title><link>https://automater.ai/intel/gemini-cli-antigravity/</link><guid>https://automater.ai/intel/gemini-cli-antigravity/</guid><description>Google shut Gemini CLI down on June 18, 2026, and CI pipelines broke overnight. What happened, how Antigravity CLI replaces it, and the 15-minute migration.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 16 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Claude Code vs Cursor in 2026: The Agentic Coding Shootout</title><link>https://automater.ai/intel/claude-code-vs-cursor/</link><guid>https://automater.ai/intel/claude-code-vs-cursor/</guid><description>Cursor AI code editor or Claude Code? We compare autonomy, review ergonomics, and real heavy-user pricing math, then give verdicts by persona. Updated for 2026.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 16 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Talk to Your Agents: Voice-Driven Development Grows Up</title><link>https://automater.ai/intel/voice-driven-development/</link><guid>https://automater.ai/intel/voice-driven-development/</guid><description>Voice-driven development grew up: push-to-talk hotkeys, local GPU speech-to-text, and voice-to-spec pipelines. Where dictation beats typing, plus a setup guide.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 15 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>UiPath and the Agentic Pivot: What RPA&apos;s Reinvention Means for Automation</title><link>https://automater.ai/intel/uipath-agentic-automation/</link><guid>https://automater.ai/intel/uipath-agentic-automation/</guid><description>What is UiPath in 2026? The RPA leader&apos;s agentic pivot explained: Agent Builder, Maestro, an honest RPA-vs-agents comparison, and who should buy — or skip.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sat, 15 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Ten Assistants, One Boss: Running Multiple AI Coding Agents Without the Chaos</title><link>https://automater.ai/intel/multi-agent-command-center/</link><guid>https://automater.ai/intel/multi-agent-command-center/</guid><description>Running Claude Code, Codex, and Kimi side by side? Manage multiple AI agents with one searchable archive, fleet health alerts, and local token metering.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 14 Aug 2026 00:00:00 GMT</pubDate><category>Product</category></item><item><title>What Is an Agentic Workflow? Anatomy, Patterns, and Real Examples</title><link>https://automater.ai/intel/what-is-an-agentic-workflow/</link><guid>https://automater.ai/intel/what-is-an-agentic-workflow/</guid><description>Learn what an agentic workflow is: the seven-stage anatomy, six core patterns, 11 real examples, and when to skip agents — from a team that runs them daily.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Fri, 14 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>Subagent Orchestration: Fleet Patterns for Daily Drivers</title><link>https://automater.ai/intel/subagent-orchestration/</link><guid>https://automater.ai/intel/subagent-orchestration/</guid><description>Subagent orchestration without framework theory: five fleet patterns — worktrees, planner/worker, skeptic pairs, swarms, background agents — with real setups.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 13 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>The RL Environment Gold Rush: Why Agent Gyms Are the New Training Data</title><link>https://automater.ai/intel/rl-environments-gold-rush/</link><guid>https://automater.ai/intel/rl-environments-gold-rush/</guid><description>RL environments are the new training data. Why labs pay for agent gyms, who sells them, and how reward hacking and benchmark contamination could sour the rush.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 13 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>Agentic CI/CD: Self-Healing Pipelines and Git When Machines Commit</title><link>https://automater.ai/intel/agentic-cicd/</link><guid>https://automater.ai/intel/agentic-cicd/</guid><description>Agentic CI/CD runs both ways: agents heal failing pipelines, and pipeline gates govern machine commits. Get the playbook, git rules, and adoption plan.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 13 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>MCP Explained: The Model Context Protocol for Power Users</title><link>https://automater.ai/intel/mcp-model-context-protocol/</link><guid>https://automater.ai/intel/mcp-model-context-protocol/</guid><description>Anthropic MCP explained for power users: how the Model Context Protocol works after the 2026 stateless spec, real client configs, security, and server builds.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 13 Aug 2026 00:00:00 GMT</pubDate><category>The Agent Stack</category></item><item><title>Agentic Software: How AI Agents Are Turning Code Into Colleagues</title><link>https://automater.ai/intel/agentic-software/</link><guid>https://automater.ai/intel/agentic-software/</guid><description>What a software agent is, how agentic software actually works, and how to adopt it without chaos — architecture, lifecycle, SDLC patterns, and governance.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Thu, 13 Aug 2026 00:00:00 GMT</pubDate><category>Agentic Foundations</category></item><item><title>Spec-Driven Development: From Vibe Coding to Contracts Your Agents Can Ship</title><link>https://automater.ai/intel/spec-driven-development/</link><guid>https://automater.ai/intel/spec-driven-development/</guid><description>Vibe coding broke at review time. Spec-driven development fixes it: a four-artifact stack, one full worked example, real tooling, and metrics that prove it.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Wed, 12 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>When SWE-bench Saturates: Reading Agent Benchmarks in 2026</title><link>https://automater.ai/intel/agent-benchmarks-2026/</link><guid>https://automater.ai/intel/agent-benchmarks-2026/</guid><description>SWE-bench Verified is saturating — open models post 78–93%. What scores still predict, how vendors dress them up, and a checklist for reading agent benchmarks.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 11 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>DeepSeek V4 Migration Guide: Life After V3 and R1</title><link>https://automater.ai/intel/deepseek-v4-migration/</link><guid>https://automater.ai/intel/deepseek-v4-migration/</guid><description>DeepSeek deprecated V3 and R1 on July 24, 2026. Migrate to DeepSeek V4 Pro or Flash with real config swaps, an eval-first sequence, and a rollback plan.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 11 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Token Plans Decoded: What Heavy AI Users Actually Pay in 2026</title><link>https://automater.ai/intel/token-plans-decoded/</link><guid>https://automater.ai/intel/token-plans-decoded/</guid><description>AI subscription plans decoded for heavy users: Claude Max, ChatGPT Pro, Copilot&apos;s new AI credits, Chinese flat plans, and the math that picks your stack.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 11 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>Understanding Anthropic: The Company Betting Everything on Agents</title><link>https://automater.ai/intel/anthropic-agentic-bet/</link><guid>https://automater.ai/intel/anthropic-agentic-bet/</guid><description>Anthropic explained for builders: the founders, the safety strategy, Claude Fable 5 and Mythos 5, MCP, real critiques, and how to bet on the agent-first lab.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Tue, 11 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>GLM-5.3: Zhipu&apos;s Post-Training Leap and the Two-Week Weights Wait</title><link>https://automater.ai/intel/glm-5-3-release/</link><guid>https://automater.ai/intel/glm-5-3-release/</guid><description>Zhipu shipped GLM-5.3 on August 14 through the GLM Coding Plan, with weights two weeks out. What the vendor claims, what&apos;s verified, and how to try it today.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 10 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>Chinese AI Models for Agent Work: DeepSeek, Qwen, Kimi, GLM</title><link>https://automater.ai/intel/chinese-frontier-models/</link><guid>https://automater.ai/intel/chinese-frontier-models/</guid><description>Kimi K3, GLM-5.2, DeepSeek V4, and Qwen3-Coder-Next: a lab-by-lab guide to Chinese AI models for agent work, covering capability, licenses, access, and trust.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 10 Aug 2026 00:00:00 GMT</pubDate><category>Industry Analysis</category></item><item><title>The Anthropic API and Console: A Builder&apos;s Guide</title><link>https://automater.ai/intel/anthropic-api-console-guide/</link><guid>https://automater.ai/intel/anthropic-api-console-guide/</guid><description>Master the Anthropic Console: mint an API key, make streaming Claude calls in Python and TypeScript, cut costs with caching, and ship a small agent service.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Mon, 10 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Train Your Own Mini-Model: From nanochat to a Fine-Tuned Daily Driver</title><link>https://automater.ai/intel/train-your-own-mini-model/</link><guid>https://automater.ai/intel/train-your-own-mini-model/</guid><description>Train your own LLM in 2026: QLoRA fine-tunes with Unsloth, distillation from open teachers, or a $100 nanochat run. Real configs, costs, and honest limits.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 09 Aug 2026 00:00:00 GMT</pubDate><category>Playbooks</category></item><item><title>Claude Fable 5 and Mythos 5: Inside Anthropic&apos;s Mythos-Class Era</title><link>https://automater.ai/intel/claude-fable-5-mythos/</link><guid>https://automater.ai/intel/claude-fable-5-mythos/</guid><description>Anthropic split the frontier in two on June 9, 2026: public Claude Fable 5, gated Mythos 5. What Mythos-class means for agent builders, minus the hype.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 09 Aug 2026 00:00:00 GMT</pubDate><category>News &amp; Trends</category></item><item><title>The Chinese CLI Wave: Qwen Code, Kimi Code CLI, and the $3 Coding Plan</title><link>https://automater.ai/intel/chinese-coding-clis/</link><guid>https://automater.ai/intel/chinese-coding-clis/</guid><description>Qwen Code, Kimi Code CLI, and the Z.ai GLM Coding Plan reviewed for August 2026: real prices, quotas, Claude Code wiring recipes, and a calm trust checklist.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 09 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item><item><title>Claude Code: The Power User&apos;s Field Guide</title><link>https://automater.ai/intel/claude-code-power-guide/</link><guid>https://automater.ai/intel/claude-code-power-guide/</guid><description>Master Claude Code beyond the basics: CLAUDE.md discipline, hooks, subagents, headless CI runs, and cost control — the field guide daily drivers bookmark.</description><dc:creator>Russell Winslow</dc:creator><pubDate>Sun, 09 Aug 2026 00:00:00 GMT</pubDate><category>Tools &amp; Reviews</category></item></channel></rss>