Skip to main content

Pools and Devices

Pools group DevKits by ownership, purpose, firmware policy, or hardware class. Devices can inherit pool metadata and add their own labels.

For local maintenance controls, see Maintenance.

Default Pool​

Newly discovered bootstrapping devices are placed into the default pool. Lab operators can move them into a more specific pool after they are provisioned.

Registration​

The device agent registers with Fleet Manager using:

  • target ID
  • board
  • target type
  • MAC addresses
  • current firmware version
  • OS type
  • labels and agent metadata

Registration is closed by default when settings come from environment. Only allowed MAC prefixes can register or heartbeat unless KERRIGAN_OPEN_REGISTRATION=1.

Labels​

Labels are key/value strings used by the scheduler. Pool labels describe a group of devices. Device labels refine a specific device. Kerrigan also generates read-only implicit labels such as:

kerrigan.pool_id=default
kerrigan.target_id=modalix-01
kerrigan.board=modalix
kerrigan.target_type=modalix_devkit

For matching rules and workflow syntax, see resource selection.

Device management address​

The device agent opens a TCP connection to the host and port in its configured Fleet Manager URL and reports that connection's local source IP on registration and every heartbeat. An overlay route to the controller therefore selects the overlay address even when Internet traffic uses a different interface. Discovery or command-line changes to the controller URL are reflected in the next report.

For compatibility, the API field latest_ip remains the management address used for SSH, SDK pairing, and other controller-to-device operations. The separate reported_ip field records the board's general/display address; it never overrides the management address. DHCP notifications and lease scans preserve these agent addresses for recently seen, registered devices; the lease address remains in the bootstrap job record. Newly discovered devices still use DHCP for bootstrap access. Device identity continues to use MAC/serial identity, not an IP address. Existing agents must be upgraded to use controller-route selection.

Address precedence is:

  1. KERRIGAN_MANAGEMENT_IP or management_ip in /etc/kerrigan/agent.toml.
  2. Legacy KERRIGAN_LATEST_IP or latest_ip in the same file.
  3. The source IP of a connection to the configured controller.

Environment variables override file values for each setting. Automatic selection is refreshed rather than cached at startup. If connecting to the controller fails, the agent reports a null management address rather than an unrelated default-route address. Registration and both heartbeat endpoints accept the same explicit address contract. A heartbeat that omits latest_ip preserves the stored address; an explicit null clears it.

Registration and heartbeat handlers trust the address declared by agents on the device network. They do not infer a management address from the HTTP peer or forwarding headers: that peer may be a reverse proxy or NAT gateway. With a proxy, NAT, asymmetric routes, or outbound-only access, configure an explicit address that the controller can reach and allow inbound SSH to it. An outbound controller connection and a recent heartbeat do not prove inbound SSH reachability. SDK operations test the actual SSH connection and identify the attempted management endpoint on failure; a WireGuard handshake failure is reported separately as an SDK tunnel failure.

USB relay power control​

System administrators can open a device's Power tab to assign a detected USB relay channel, select COM–NC (default) or COM–NO wiring, and save it. Channels already assigned to another device are excluded; the API also enforces exclusive ownership atomically. The durable identity is the normalized DevKit MAC address, not its editable Kerrigan target ID. Assignments therefore persist across target renames, deletion, and rediscovery of the same MAC. Unplugging a relay retains its assignments but disables controls.

For manufactured appliances, replace the complete MAC-to-channel table in one atomic request rather than configuring devices one at a time. Kerrigan validates that every MAC is registered, every MAC and channel is unique, and every channel fits the detected relay before replacing the old table. Configuration changes do not initialize or switch relay outputs, and every replacement is recorded in the persistent power-channel audit table.

Upgrading a database that used target-ID assignments preserves each assignment when its registered DevKit has exactly one valid MAC and the MAC ownership index agrees. Bootstrap IDs must also match that MAC. Existing MAC mappings take precedence over conflicting legacy assignments. The migration does not access relay hardware or change outputs.

All legacy rows are archived for recovery, and the migration is audited. Missing, ambiguous, or conflicting identities require an administrator to validate and configure the mapping explicitly. Previously archived rows are not automatically reactivated on restart, so intentionally cleared mappings stay cleared. The empty legacy table is retained for binary rollback compatibility: an older Fleet Manager also sees unassigned controls instead of interpreting the new mapping. Re-upgrading restores use of the separately retained MAC mapping.

The power switch represents DevKit power, not coil activation. With NC wiring, Power On releases the relay and Power Off energizes it. Power cycle interrupts power for five seconds, then restores power. Commands verify FTDI pin readback; this does not measure contact position, load voltage, or successful boot. The Power tab reads FT245R pin levels even after a service restart, without initializing the controller or first issuing a power operation. A failed read reports unknown state with an error. Status queries, discovery, and assignment never initialize or switch the outputs. USB disconnects, server failure or loss of coil power can change contact state; software cannot guarantee restoration after those failures.

Connect the relay USB cable to the Kerrigan server and provide the relay board's specified coil supply (12 V DC for the SRD-12VDC board). Supported hardware is currently the eight-channel FT245R USB FIFO relay. The driver matches both USB IDs and product name to avoid treating ordinary FT232 serial adapters as relays, and uses the serial number as persistent identity. Duplicate relay identities fail closed instead of selecting one device by USB enumeration order. The server image includes PyUSB/libusb and its fleet-manager service already has access to /dev. Run one fleet-manager process per physical controller; do not operate relayctl or another USB controller concurrently.

Channel capacity comes from the detected driver's hardware profile, defaulting to eight. Future larger boards require a matching driver/profile; increasing the FT245R channel count alone cannot provide more than its eight output bits.

API (system administrator session required):

  • GET /v1/power/relays: detected boards, capacities and channel assignments.
  • PUT /v1/power/mappings: atomically replace all assignments with { "mappings": [{ "mac": "aa:bb:cc:dd:ee:ff", "channel": 1 }] }. When exactly one relay is connected, its ID is inferred; otherwise include relay_id explicitly. Send { "mappings": [] } to clear retained mappings, including mappings whose DevKit has already been removed.
  • GET /v1/power/targets/{target_id}: assignment, availability and power state.
  • PUT /v1/power/targets/{target_id}: { "relay_id": "...", "channel": 8, "wiring": "nc" }. Send { "relay_id": null } to unassign without changing outputs.
  • POST /v1/power/targets/{target_id}/actions: { "action": "on" }, off, or cycle.

Unavailable hardware returns 503; duplicate channel assignment returns 409. Hardware discovery refreshes while the Power tab is open. Configuration changes must be saved before operating the channel.

Maintenance mode​

Administrators can pause a DevKit in its Maintenance tab, all devices in a pool in the pool editor, or the entire appliance under System → Maintenance. A reason can be recorded with the change. The panel reports inherited scopes, transition progress, runner services, and errors.

Maintenance is persistent and separate from device health. Any enabled global, pool, or device scope blocks new work and SDK pairing immediately. Clearing one scope does not override another. The displayed health is in-maintenance; the underlying health and connectivity remain available in reported metadata.

Kerrigan discovers systemd units named actions.runner.*.service on each DevKit. It waits for active Kerrigan leases and GitHub Runner.Worker processes, then stops and disables those services. On final exit, it enables and starts them. No runner is a successful no-op. Offline devices and failures remain blocked and are retried; check that the panel says active before starting maintenance. Entry/exit can take longer while existing work drains. New standalone GitHub work should also be paused at its workflow source while waiting for runners to stop.

The maintenance gate does not treat every operation alike. New execution work, SDK pairing, P2P connections, and managed-runner reconciliation are blocked for the whole window. OTA updates, device bundle rollouts, and Developer Portal credential projection are admitted only after the device reaches active. Status/reporting, power and reboot controls, authorized cloud terminals, and cleanup or release operations remain available subject to their normal safety checks. Existing sessions are not automatically cancelled. See What maintenance blocks for the complete operator matrix. Exiting maintenance restores the actual health state, not an assumed healthy state.

Maintenance API​

  • GET /v1/maintenance/{scope}/{scope_id}: scope intent and per-device results.
  • PUT /v1/maintenance/{scope}/{scope_id}: administrator-only asynchronous change; body {"enabled": true, "reason": "Software update"}, HTTP 202.
  • Scopes: global/all, pool/{pool_id}, device/{target_id}.

Device responses include maintenance.enabled (effective intent), blocked (admission gate), phase (normal, entering, active, exiting, error), scopes, services, error, and updated_at. Clearing intent does not reopen admission until runner restoration succeeds. Requests are idempotent in intent; repeated reconciliation verifies service state without creating new devices.

The maintenance toggle shows On · inherited and is disabled when System maintenance applies to a pool or DevKit, or pool maintenance applies to a DevKit. End maintenance at the controlling scope to unlock the local toggle. A separately enabled local scope remains enabled after the parent scope is cleared.

Maintenance runner reconciliation requires a fresh device heartbeat before SSH. An offline device stays pending and blocked until it reports again; Kerrigan does not use a stale cached IP address to change runner services.

Agents may continue registration and heartbeat reporting during maintenance. Task polling returns an empty assignment so older agents stay in their normal heartbeat loop; maintenance still prevents execution work from being assigned. Re-registration preserves the last real heartbeat timestamp rather than clearing or refreshing it.