Rollouts

A rollout ships one version of a configuration or bundle to a channel in stages. Each stage reaches more of the fleet, and the next stage starts only when the agents already on the new version report it applied and healthy. If an agent rejects it, Ampbase rolls back, or pauses for you to decide. When the last stage is green, the rollout promotes: every rule it moved serves the new version. When a version needs build components, stages are judged only on agents whose supervisor reports theirs — see What automatic judgement needs from your fleet below.

A rollout is built from pieces you already know: it moves the routing rules that serve the version it replaces, the channel default among them, it judges each stage from what your agents report on every check-in, and promoting it sets each of those rules to the new version, the channel default as Everyone now would. Its advantage over editing a rule's weights yourself is that a rollout records its progress, holds each stage until the fleet has shown it works, and can be stopped or rolled back from its own page.

A channel runs one rollout at a time.

What a rollout reaches

A rollout of a configuration version reaches every rule that serves that configuration: a rule whose target is the configuration, at any version, and a rule whose target is a bundle carrying it. The channel default is one of them when it serves the configuration, or a bundle carrying it. A rollout of a bundle version reaches every rule whose target is that bundle. Each rule a rollout reaches is one of its slots.

Take a channel whose default serves a base configuration and whose signal and prod rules route their environments to bundles built on base. Rolling base from v13 to v14 reaches all three:

slot                    serves before        serves once promoted
signal → bundle signal  bundle signal s7     bundle signal s8
prod → bundle prod      bundle prod p4       bundle prod p5
channel default         config base v13      config base v14

• At Start the rollout records what each slot serves. A rule that tracks its target's latest version is held on the version it serves until the rollout ends, so a save made meanwhile can't reach its agents unjudged. For each bundle slot the rollout makes a new bundle version, as Update to latest does, but moving only the rolled-out configuration's entry to the new version; the bundle's history shows it minted by rollout with a link to the rollout.
• Each stage routes the stage's agents on every slot to that slot's new version; the rest of each rule's agents keep what they run. A stage is judged on its agents across every slot together.
• Promotion sets every slot to its new version. A rule that tracked its latest version before the rollout tracks it again, unless its configuration or bundle moved on, or was deleted, during the rollout; then it stays on the slot's version, and the rollout's page says why after kept pinned, because …: the version the slot stays pinned to, and the newer version its configuration or bundle moved to, or that it was deleted.
• Rollback returns every slot to what it served before. A rule that tracked its latest version tracks it again only if its latest is still what it served before; once a bundle version has been made for it, or the configuration has moved on, it stays pinned, because tracking would serve a version the rollback removed. The Routing page shows the pin on the rule. The bundle versions the rollout made stay in each bundle's history, and the bundle page says the latest was minted by a rollout that rolled back.

A rule that is disabled, or a variant weighted 0, is a slot too: it is held and promoted or returned with the rest, and reaches no agents while it routes none. Start refuses a rollout it can't carry through, naming the rule and what to do. Among the refusals:

• a rule that splits its agents between two targets, which a rollout can't divide — fold the split first;
• a bundle that already carries the configuration at a newer version than the one being rolled out, which promotion would move backwards;
• a rule pinned to a bundle version older than that bundle's latest — set it to the latest, or tracking latest, first;
• a version no enabled rule routes any agent to, which leaves the rollout nothing to move.

Starting a rollout: the deploy sheet

Open a configuration (or bundle) and click Deploy this version, either at the top of the page for the current version or on any row of the versions table. The deploy sheet opens with two choices:

• Staged rollout — Stage by stage, each judged before the next. The rest of this page is about this one.
• Everyone now — Set as the channel default at once. Every agent receives the version on its next check-in. See Configurations, under Deploy is the verb.

The plan

A staged rollout needs a plan: how each stage widens.

• By percentage — Stages, in percent of eligible agents, for example 10, 50, 100. Each stage routes that share of the channel's eligible agents to the new version. The numbers must increase and end at 100. An agent's share is decided by a hash of its ID, so an agent routed in the 10% stage stays routed in the 50% stage; nobody moves back and forth between versions as the rollout widens. Because the hash decides membership only when an agent is evaluated, a percentage stage's count is an estimate, shown as ~50 expected.
• By cohort — a Cohort label (an attribute your agents report, such as env) and its Values, in ramp order, for example integration, staging, prod. Each stage adds the next value: the first stage routes agents with env: integration, the second routes integration and staging, and so on. A cohort is a fact about your fleet rather than a claim about a hash, so its counts are exact. Where your agents already carry a label that says where they sit in your promotion order, prefer a cohort plan.

Attributes are what you set under agent.attributes in supervisor.yaml; see Agents.

As you edit the plan, the sheet previews it: a row per stage with what it Reaches and how many of those agents Can run it — whose builds contain every component the version needs (see Build eligibility in Routing). The preview is computed by the same check that starting the rollout runs, so a plan the preview accepts is one Start rollout accepts, and a plan it refuses shows the reason in place of the table.

The two automation toggles

• Advance to the next stage when this one is green (on by default). Turn it off and every green stage waits for someone to click Advance now.
• Roll back when a tripwire fires (otherwise pause) (on by default). Turn it off and a fired tripwire pauses the rollout instead, leaving the decision to you.

The two are independent: you can let Ampbase advance while you keep the decision to roll back, or advance by hand while Ampbase protects you.

Every stage is held to the same gates:

• Soak — 30 minutes since the stage started. A stage can't go green before then, but the tripwires are read on every tick from the start: one can fire, and roll the rollout back or pause it, a minute into the soak.
• Applied — at least 90% of the stage's agents report the new version applied and healthy.
• Tripwires — one each for Apply rejections, Degraded agents and Configuration drift. Each fires on the first agent it counts.

Confirms the sheet may ask for

• A version that arms enforcement (an eBPF security policy that blocks or kills, for example) asks you to type the same confirm phrase deploying it directly would. Its first stage is a canary and must stay within the canary ceiling the sheet states; a plan whose first stage reaches more is refused until you narrow it. A cohort stage routes every agent it matches, so for a small first stage use a percentage or a cohort of one host.
• A version no reporting agent can run asks you to tick a coverage confirm before starting. See the next section.

What automatic judgement needs from your fleet

A stage is judged from what its agents report, and an agent is only routed to a stage if its build can run the version. Ampbase knows what a build can run from the component inventory the supervisor reports (see Supervisor reference). An agent whose supervisor has never reported an inventory is unchecked, not fine: if the version needs any components, that agent is withheld — it stays on the channel default and is not part of any stage.

So what a rollout can judge depends on what your fleet reports:

• Where no agent in the channel reports an inventory and the version needs any components, every agent is withheld from every stage. Each stage reads No agent has been routed to this stage yet, nothing advances on its own, and nothing about the version has been checked. Upgrade the supervisors first; until then a staged rollout of such a version is not a canary.
• Where some agents report and some don't, the stages are made of the reporting agents alone. Each rule on the Routing page counts the rest as withheld.
• Agent types with no build-time components — Refinery, Tetragon, Falco — are never withheld, so every agent is judged whether or not it reports an inventory.

The coverage confirm is the other side of the same rule. The sheet asks for it when agents have reported their builds and none of them can run the version:

No agent that reported its build can run this version, and the last stage's promotion will refuse it without this confirm. Start anyway; agents that cannot load it keep their last-applied config.

Promotion is the reason. When the rollout finishes, every slot serves the version. A rule other than the default still withholds it from a build that can't run it, but the channel default is never gated: it reaches every agent no rule above it serves, whatever its build — including the agents the stages withheld and never judged. That is the same friction Everyone now attaches to setting a default the fleet cannot run, collected once at the start so the rollout does not stall at the end. A fleet where nothing has reported never asks for it.

How a stage is judged

   Start rollout
        │
        ▼
   widen every slot to the next stage ◀────────────┐
        │                                          │
        ▼                                          │
   judge the stage (every minute)                  │
        │                                          │
        ├─ Red: a tripwire fired                   │
        │     ├─ roll back ─▶ what it replaced     │
        │     └─ pause ─────▶ you decide           │
        │                                          │
        ├─ Not yet judged ─▶ hold, judge again     │
        │                                          │
        └─ Green ─▶ advance ───────────────────────┘
                    (or wait for Advance now)

   after the last stage is green: promote — every
   slot serves the new version

Once a minute the rollout sorts every agent in the stage into one of five buckets:

From those counts it records a verdict — green, not yet judged, or red — as a sentence saying why, coloured to match, for example 3 of 110 agents in this stage rejected this version; the Apply rejections tripwire allows at most 0. Rolling the rollout back. The Rollouts list, the rollout's page and the Serving panel on the configuration page all show that one recorded verdict; no page works out its own.

Not yet judged is never red. A stage that is still soaking, or whose agents have not all applied yet, holds; it is not failing. Unknown counts against the stage: an agent that has gone quiet is in the stage's total but never counts as applied, so a stage cannot pass because its agents stopped reporting. The sentence says so: No news is not evidence.

When there is nothing to judge

A rollout's pages keep three different kinds of "nothing" apart, because they mean different things:

• No agents in the stage. No agent has been routed to this stage yet, so there is nothing to judge. The stage holds. On a percentage stage, agents join as they check in; if it lasts, check each rule's withheld count on the Routing page (above).
• Evidence not in yet, or stale. Before the first stage is judged, the page says No evidence yet: the first stage has not been judged. While a stage is rolling out or paused, the rollout rewrites its verdict every minute; if it has not recorded a newer one for more than two minutes, the verdict is shown with Stale: last judged and its time: the numbers are the last ones recorded, not current ones. The page does not mark a verdict stale while the rollout is waiting for evidence or for an operator.
• A read failed. Could not read this rollout; nothing is known about its evidence. Nothing is claimed either way; reload to try again. A failed read inside the rollout itself changes nothing either: it leaves the last verdict in place, which then shows as stale.

Commands

The rollout's page carries four buttons. Each asks you to confirm, and Roll back and an override ask for a reason, which is recorded. A button is disabled when the rollout is in a state where it does nothing; if the rollout moves on between the page loading and your click, the dialog shows the refusal instead.

While the rollout is Promoting, Roll back is the only command it accepts.

When a tripwire fires

With Roll back when a tripwire fires on, the rollout stops, and every rule it moved is handed back serving what it did before, so every agent the rollout reached returns to what it ran. The page reads Rolled back with the tripwire that failed, and the Rollouts list shows the failed gate on the row.

With it off, the rollout pauses. The banner says The policy paused it, names the tripwire, and the choice is yours: Roll back, or Override gates and advance with a reason. Resume is offered too, but it only restarts the soak: if nothing has changed, the same tripwire fires again when the stage is next judged, so it is not a way out.

After a rollback, the rollout's page keeps what the rollout saw: the Agents table and its counts are the rollout's last reading, so an agent that rejected the version still shows Rejected with its error. Below it, Reverted to what each slot served before lists the same agents as they are now: Back on what it ran before, Not back yet, or Unknown for one that has gone quiet or left the channel. The rollout reads Rolled back once none is left Not back yet.

The verdict line on a finished rollout is the gate's last verdict, quoted with its time. A verdict that says Rolling the rollout back is what the gate said when it fired; the state and the banner say how it ended.

The rollout stays in the Rollouts list with the gate that failed. Fix the configuration, save a new version, and deploy that one from Deploy this version.

Webhooks

A webhook can subscribe to five rollout events: Rollout Started, Rollout Advanced, Rollout Gate Failed, Rollout Rolled Back and Rollout Promoted. Each delivers the rollout record with its current verdict. Rollout Gate Failed fires once, never per tick, and only when a tripwire rolls the rollout back or a stage cannot be reached, and Rollout Promoted once every rule it moved is handed back and the rollout reads complete. A tripwire set to pause sends nothing: the rollout waits, paused with a red verdict, on its own page, so don't wait for a Rollout Gate Failed delivery that will not come.

Reading a rollout

Slots. A rollout's pages list its slots: each slot is one rule the rollout moves, named after the rule and the target that rule serves, such as signal → bundle signal. The channel default is one of them, named channel default, when it serves the version being replaced; when it does not, the page says so and the rollout leaves it alone. Each slot reads as what it became: read-only while the rollout runs, promoted, serving this version, or back on what it served before.

The Rollouts list (channel sidebar → Rollouts) shows live rollouts first, then finished ones, newest first: the target and version with its slots, the stage, the verdict, when the soak ends, how many of the agents routed to the stage have applied, and who started it. It updates as rollouts change.

A rollout's page has the state and commands at the top, a banner for paused, waiting and finished states, the stages, the Evidence panel with the five counts and the verdict sentence, and the Agents table, filterable by bucket, with each agent's detail — the error an agent rejected the version with, for example.

A new stage starts unjudged. From the moment a stage starts until its first reading, the list, the banner and the Evidence panel say Stage 2 starting; not judged yet rather than showing the previous stage's verdict and counts. The same holds after Resume, which restarts the stage's soak.

The Activity column lists everything that happened to the rollout, oldest first: when it started, each stage advance, every pause, resume, advance request and override, a failed gate, and how it ended. Each row names who did it — a person, or the rollout itself for what it decided — and the reason they gave. The same events are in the channel's audit log.

Times are absolute. Every time on these pages is a UTC time, such as 2026-09-28 15:49 UTC, with how long ago (or how soon) beside it. Nothing on the page counts down; it updates when the rollout changes, and the evidence refreshes once a minute while a stage is live.

On the configuration page, the Serving panel shows the channel default and the rollout in progress (with View rollout), with its slots and how many agents each holds. Its Withheld card points to the Routing page, which counts the agents each rule withheld. The versions table's Serving column reads Default, Rolling out · stage 2 of 3, or Not serving, and the version being rolled out cannot be deployed a second time.

The rules a rollout moves

A rollout creates no rule of its own. It moves the channel's rules that serve the version it replaces, the channel default among them: each stage gives each of those rules a temporary variant for the agents the stage reaches, and the rest of the rule's agents keep what they run. On the Routing page each rule a rollout moves carries a Managed by rollout chip linking to the rollout. Opening it explains: This rule is read-only and keeps its place: its rollout sets the weights as each stage advances. While the rollout is live, the channel's rules can't be saved, and the API refuses changes the same way.

While the rollout runs, the Routing page is read-only, with one banner naming the rollout: The rules are read-only while rollout … runs. Roll it back or let it finish, then edit. If a rollout's run stopped without handing its rules back, the page is editable again and its banner says the rollout no longer runs but still holds rules here. The next save releases them.

When the rollout ends, every rule it moved is handed back and can be edited again. Promoted, each serves the new version; rolled back, each serves what it did before the rollout.

On a bundle's page, while a rollout that will make a new version of the bundle is running, the Serving panel's banner for an entry behind its configuration's latest links that rollout (Rolling out: view rollout) in place of Update to latest, which would race it.

One rollout per channel

While a rollout is live or rolling back, the channel can't start another, and Everyone now is refused:

rollout … is live or rolling back on this channel; roll it back or let it finish before setting a version as the channel default

The refusal links to the rollout (View rollout) and to its commands (Roll it back there). This is what keeps the rollout honest: a default set underneath it would change what the channel default served before the rollout, which is what a rollback returns it to.

Rollouts are available on channels that offer Bundles; AI coding agent channels don't have a rollout target yet.