Maintenance
Use maintenance mode to pause new work while updating or diagnosing DevKits.
Kerrigan blocks new work and SDK pairing, waits for active work to finish, and
stops and disables affected GitHub runner services. Ending maintenance rediscovers
all actions.runner.*.service units and enables and starts them, including runners
that were disabled before entry or installed during the window. It does not
restore their prior enabled/disabled state.
A maintenance window is explicitly opened and closed. It has no automatic expiry, and its intent survives controller restarts. Finishing a firmware update does not automatically end maintenance.
What maintenance blocks
maintenance.blocked is an admission and scheduling gate, not a prohibition on
every device operation. It becomes true as soon as any applicable maintenance
scope is enabled and remains true through entering, active, exiting, and
error. It becomes false only after the device returns to normal.
The following work is blocked for the entire window:
| Operation | Behavior while maintenance.blocked is true |
|---|---|
| New execution tasks and workload leases | No new task is assigned. Work that was already running is not cancelled; maintenance entry waits for it to finish. |
| SDK connect or pairing | New connections are rejected. Disconnect and cleanup remain available. |
| P2P allocation, preparation, and connection | New P2P work is rejected. Stop, disconnect, and release operations remain available. |
| Managed GitHub runner configuration | Runner installation, removal, group changes, and reconciliation wait until maintenance ends. |
| GitHub Actions runner work | Entry waits for an active Runner.Worker process. At active, discovered actions.runner.*.service units are stopped and disabled. Pause standalone workflows at their source while entry is pending so they do not claim more work. |
The following maintenance work is admitted only after the device reaches
active. It is blocked during entering, exiting, and error:
| Operation | Additional admission rules still enforced |
|---|---|
| DevKit TFTP firmware recovery | The device must be idle and reachable, with a downloaded target image and no execution lease, P2P lease, bundle rollout, or conflicting firmware operation. Device and pool recovery preserve the maintenance window. |
| DevKit OTA update | The device must be idle and reachable, with no execution lease, P2P lease, bundle rollout, or other firmware operation. |
| Device bundle rollout | The device must be healthy, online, idle, unleased, and free of another device reservation. Automatic rollouts also verify that GitHub Actions is idle. |
| Developer Portal credential projection | The device must be healthy, online, idle, unleased, configured for managed authorization, and free of another device reservation. |
These controls remain available throughout the window, subject to their normal authorization and safety checks:
| Operation | Notes |
|---|---|
| Inventory, registration, heartbeat, status, and reporting | Agents continue reporting, but task polling returns no assignment. |
| Maintenance scope changes | Operators can add or clear intent. Clearing intent does not reopen scheduling until exit completes. |
| Power status, power on/off/cycle, and reboot | Maintenance does not replace the operation's normal availability and validation checks. |
| Authorized browser terminal | A new terminal can reserve an online, idle device with no active execution lease. Closing it preserves the maintenance scheduling block. |
| Completion, disconnect, release, recovery, and cleanup | Existing work is not automatically cancelled. Cleanup paths remain available so reservations and transient device state can be released safely. |
Maintenance never overrides another safety gate. An operation listed as allowed can still wait or fail because the device is offline, unhealthy, busy, leased, already reserved, or otherwise ineligible.
Choose a local scope
| Scope | Devices affected | Local dashboard control |
|---|---|---|
Appliance (global/all) | All devices on this Kerrigan instance | Appliance administration Maintenance tab |
Pool (pool/{pool_id}) | Devices belonging to that local pool | Edit the pool and expand Maintenance mode |
Device (device/{target_id}) | One DevKit | Device Maintenance tab |
Pools are local to a Kerrigan instance. Appliance and pool maintenance are inherited by their devices. A device remains blocked while any applicable scope is enabled. Turning off a narrower scope does not override an enabled parent; the dashboard identifies the controlling scope and disables the inherited toggle. End maintenance there first, then clear any remaining narrower-scope intent. Pool maintenance also applies to devices subsequently added to that pool.
Open and close a window
- Sign in to the local dashboard as an administrator and open the appropriate control from the table above.
- Optionally enter a reason, then turn maintenance on.
- Wait until every affected device reports active before starting work. Saving the setting records intent; active jobs and offline devices can delay entry.
- Perform the update or diagnostic work according to the admission rules above. If maintenance is turned off during a bundle rollout or credential projection, exit waits for that reserved operation to release the device. On 3.0 platforms, OTA recovery reinstalls saved GitHub runner units after reboot but keeps them stopped and disabled until maintenance ends.
- Turn maintenance off at the scope where it was enabled. Wait for normal and
blocked: falsebefore returning devices to service.
| Phase | Meaning and operator action |
|---|---|
entering | Wait for active work to finish and the device to be reachable. |
active | Entry is complete; the device is ready for maintenance. |
exiting | Runner restoration is pending; new work remains blocked. |
normal | Reconciliation is complete; verify blocked: false. |
error | Inspect and correct the reported error. Kerrigan keeps the device blocked and retries. |
Maintenance state is separate from firmware-update state. Cancelling a stale OTA job does not clear maintenance, and clearing maintenance does not cancel an OTA job. Check both before returning a manually updated device to service.
Local REST controls
Read a scope with GET /v1/maintenance/{scope}/{scope_id}. Change it with
PUT to the same path using an authenticated administrator session:
PUT /v1/maintenance/pool/sdk-3-0-0
Content-Type: application/json
Cookie: kerrigan_session=<your-session>
{"enabled": true, "reason": "Manual firmware update"}
Use global/all for appliance scope or device/{target_id} for one device.
The reason is optional and limited to 500 characters. To close the window, send
{"enabled": false} to the same scope.
A successful write returns 202 Accepted, acknowledging saved intent rather
than completed runner changes. The response and subsequent GETs include the
scope's own enabled and reason, inherited_scopes, and affected devices
with their effective maintenance state. Inspect each device's phase, blocked,
and error; the scope's own enabled: false does not rule out inherited blocking.
Cloud controls and reporting
The MQTT maintenance command accepts explicit device IDs and changes device scopes only. Expanding a pool in a cloud UI to a device list does not create persistent local pool policy.
The controller advertises maintenance capability separately from pool membership. Track per-device maintenance state to determine when entry or exit is complete.