ICE Peer Pairing

ICE Peer Pairing is a cloud-infrastructure-agnostic Kerrigan capability for connecting an authorized developer's machine to one allocated DevKit when either peer may be behind NAT. It makes a best-effort direct UDP connection and uses a TURN relay candidate when direct connectivity cannot be established. Kerrigan is the DevKit-side safety authority; an authorized cloud control plane provides allocation, signaling, and session lifecycle control.

This mechanism addresses remote development and validation workflows where a user needs a temporary peer connection to a specific DevKit but neither side has a stable public UDP endpoint. It is not a generic VPN, a cloud traffic relay, or a claim that TURN allocation alone proves end-to-end connectivity.

For a managed SDK endpoint with a known address—such as a controlled EC2 instance—use SDK Connection instead. It configures the two WireGuard peers directly and mounts an NFS workspace without ICE signaling or TURN. ICE Peer Pairing is for the complementary case: a developer machine whose network path is not known in advance.

What ICE is

Interactive Connectivity Establishment (ICE) is the IETF method for finding a usable network path between two peers when one or both may be behind NAT or a firewall. Each peer gathers several possible addresses, called candidates: a local host address, a public server-reflexive address learned through Session Traversal Utilities for NAT (STUN), and, when necessary, a relay address from Traversal Using Relays around NAT (TURN). The peers exchange those candidates over their authenticated signaling channel, run connectivity checks, and select a working candidate pair. The signaling channel coordinates the setup only; it does not carry the peer traffic.

ICE Peer Pairing implements the UDP ICE procedures in RFC 8445 and is intended to interoperate with RFC 8445-compatible ICE peers. The STUN service follows RFC 8489, and the TURN relay service follows RFC 8656. ICE discovers and verifies a path; it does not itself create a general-purpose encrypted network interface. Kerrigan uses the selected UDP path for the authenticated WireGuard data plane that carries workload traffic such as SSH, NFS, and SDK access.

The device-side native forwarder currently uses Pion ICE to gather candidates. Complete interoperability depends on the entire session path—including candidate exchange, connectivity checks, selected-path forwarding, WireGuard handshake, and cleanup—being enabled and validated. Candidate gathering or a successful TURN allocation alone is not a conformance or end-to-end connection claim.

Integration flow

Open the sequence diagram in a full page.

General workflow

1. Configure and advertise capability. An operator installs a current Kerrigan device agent and native forwarder on approved DevKits, then pairs the appliance to its cloud profile. The appliance reports whether ICE is available for each permitted DevKit.
2. Allocate a DevKit. The control plane reserves one eligible Development DevKit. Kerrigan verifies the allocation, lease, health, maintenance state, and scheduler ownership before accepting a peer session.
3. Create an ICE session. The control plane creates an allocation-bound session and issues different, short-lived signaling credentials to the user and the device. The user receives only its own credential.
4. Start the device role. The control plane publishes devices.p2p.ice.start to the selected appliance. Kerrigan validates that the command exactly matches its configured capability and active allocation, then starts the privileged device forwarder with a mode-0600 runtime configuration.
5. Signal and connect. The two forwarders use the authenticated signaling service to exchange ICE information, try candidates, and select a direct or relayed path. ICE candidates, passwords, and TURN credentials must remain in the forwarders and signaling transport; they are not durable Kerrigan state.
6. Confirm actual traffic. Treat a selected candidate or TURN allocation as preliminary. Verify application or WireGuard traffic in both directions, then record the device-side connection result through the normal command event path.
7. Stop and clean up. On user disconnect, allocation release, expiry, or a failed connection, publish devices.p2p.ice.stop. Kerrigan stops the forwarder and removes its runtime files before it clears the pairing state. If cleanup fails, the lease stays held and cleanup is retried; do not reuse the DevKit until the completed stop result is observed.

Required infrastructure

An integrator needs all of the following:

Component	Responsibility
Kerrigan appliance and approved DevKit	Runs Fleet Manager, the device agent, and the device-role native forwarder. The forwarder must be installed before start commands are enabled.
Cloud profile	Limits the feature to authorized target_ids and supplies the exact STUN, TURN, and signaling endpoints that Kerrigan will accept. Updating the cloud does not update a connected appliance; roll out the required device bundle and Fleet Manager first.
STUN service	Lets each forwarder discover server-reflexive candidates. It does not relay application traffic.
TURN service	Provides authenticated, expiry-bounded relay candidates when direct UDP cannot work. A successful allocation is not an end-to-end data-plane test.
Authenticated signaling service	Relays only ICE negotiation messages between the two role-scoped connections. It is not the data path and must use TLS (wss).
User-side forwarder	Uses its own user-role signaling credential and performs the actual peer/data-plane setup. Keep this credential, TURN credential, and any generated private keys out of logs and durable configuration.

Open the required outbound UDP and TLS paths for the appliance and DevKit network. Exact firewall policy depends on the chosen STUN/TURN deployment. Do not document or hard-code staging endpoints, keys, or credentials in an integration.

MQTT control APIs

These commands use the standard MQTT Control API envelope and command-specific result events. They are control-plane operations for the allocator/signaling service, not end-user commands. Use a stable command_id when redelivering the same logical command.

Operation	Purpose	Terminal success	Important behavior
devices.p2p.reserve	Reserves an eligible DevKit for an allocation.	status: reserved	Creates the durable appliance-side allocation and lease.
devices.p2p.ice.start	Starts the device-role ICE forwarder for an existing allocation.	status: started	Requires an exact match to the configured signaling, STUN, and TURN capability. Credentials are passed through to the device agent but are not stored in Fleet Manager history or results.
devices.p2p.ice.stop	Stops the forwarder and removes its local session material.	status: closed	A completed stop is the acknowledgement that permits ICE lock cleanup. It is safe to retry.
devices.p2p.release	Releases the allocation after peer cleanup.	status: closed	Use after the ICE stop acknowledgement. A failed cleanup keeps the reservation.

The detailed argument and result contracts are in the MQTT Control API.

The developer-facing service or UI owns developer authentication and never exposes the MQTT command channel. Its allocator publishes the commands above; Kerrigan makes the final appliance-local admission and cleanup decision.

Integration rules and recovery

• Callers must not create arbitrary ICE starts. Kerrigan rejects a command if it is outside the configured profile, names an unauthorized DevKit, does not match the active allocation, is expired, or conflicts with maintenance or another peer session.
• Do not persist signaling_token, TURN password, ICE credentials, or candidates in the caller's normal request log, command journal, or ticket.
• A command.completed result for start means Kerrigan launched the device role; it does not prove the peer/data plane connected. Require forwarder traffic verification and the device's connection report.
• Stop is idempotent. Retry the same logical stop command after a delivery or event-consumption failure. Do not clear an allocation from an external system solely because its cloud record expired.
• Validate direct traffic, relayed traffic, loss/expiry behavior, and cleanup on real DevKits before relying on the integration for developer workflows.

Direct-path preference and diagnostics

In automatic P2P sessions, the native forwarder allows ten seconds from the start of ICE checks before nominating a relay pair. It gathers and checks TURN candidates concurrently, so a working relay can be selected after this window. Successful direct pairs can be selected earlier. Direct checks have a larger retry budget to remain eligible throughout this window. The server's setup expiry remains authoritative; a shorter deadline can end setup sooner. Explicit relay-only (routed) sessions retain fast relay nomination.

Both peers emit bounded ICE diagnostics and ICE pair records during setup and when a path is selected or setup fails. These include elapsed time, candidate types, checklist state, nomination, check/response counts, inbound checks, and round-trip time. no_response_yet means checks have been sent without a response; checks_exhausted means the pair failed its checks. Neither identifies a specific firewall or NAT as the cause. connectivity_confirmed distinguishes a working pair from an untested or failing pair, even when another pair is selected. ICE socket records classify local send failures, such as unreachable networks, permission errors, or unavailable source addresses, once per socket. These are local operating-system errors, not evidence of a remote firewall rule. At most five periodic snapshots and one final snapshot are emitted, each with up to 64 pairs. Pair indexes are snapshot-local. Candidate addresses, ports, raw candidate IDs, and credentials are excluded. Device summaries retain these safe diagnostic records alongside the forwarder log.

CloudEx does not require an existing VPN. On macOS, STUN discovery now runs across eligible interfaces, including physical and VPN interfaces, instead of only the default route. This broadens candidate discovery; it does not bypass macOS routing or a managed VPN policy. Signaling, TURN, and the final WireGuard path remain subject to system routing. The forwarder does not disconnect VPNs, change routes, or promise a physical-interface path while a VPN is active. Both host and DevKit forwarders need the updated binary for the full behavior.