New-line runtime support
New line, in development. This page describes SHC as it is on the
devbranch, not the released product. See The new line.
The new line has four runtimes. compose (docker) and podman are driven through the compose CLI: executor.New answers them in core/executor/executor.go, and they are not registered packages. incus (incus-compose) and apple (Apple’s container CLI, on macOS) register from their packages and are blank-imported in daemon/imports.go, so shcd links them. An empty --runtime is detected (modules/cluster/local_init.go detectRuntimeFrom): a live docker socket selects compose, a live podman socket selects podman, and otherwise compose. A runtime that cannot run a key of a deployment keeps its replicas off that runtime’s nodes (What SHC does to your compose file).
--runtime swarm and a join runtime of swarm are refused (X400422CLSRUN). A deployment runtime of swarm is refused (X400140RTMSWM). --fabric swarm is refused (X400371FABSWM).
shc cluster converge-swarm and shc node swarm-tokens no longer exist.
Enrollment carries the cluster identity and mTLS credentials; rqlite is the
only quorum. SHC does not call docker stack, docker service, docker node
or docker swarm to run its workloads.
Backups, restore, repositories and integrations remain product features. Imports from other platforms may still read those platforms’ Swarm state. Docker and Podman use their existing install, exec and log paths.
On the shc fabric a container asks the node’s DNS from its own fabric
address on compose and on podman. Docker’s embedded resolver forwards from the
container’s namespace. Podman’s aardvark-dns forwards from the host, so a
Podman node of the fabric has no aardvark-dns at all (X400398FABAVK
otherwise), and SHC keeps the engine’s resolver off every network a Podman
container joins (podman-compose 1.4.0 or newer) so its resolv.conf is the
node’s DNS. The names only the engine knows differ as a result: a compose
container name or an authored network alias resolves on a Docker node and
not on a Podman node. Service names resolve on both, except on a single
Podman node, where the node’s DNS has no mesh address to listen on and
nothing answers them.
Events each runtime tells
Section titled “Events each runtime tells”The node agent restarts, keeps down or completes a service from the events
its runtime tells. The same start, death, kill, stop and removal mean the
same thing on every runtime: core/executor/hosttest drives one service up
healthy, kills it, starts it, stops it and takes it down on each runtime,
and fails the runtime whose events mean anything else (SHC-2588).
| Event | docker | podman | incus | apple |
|---|---|---|---|---|
| start | told | told | told | told, from the polls |
| death, with its exit code | told | told | told, the exit the exec helper recorded | told, the exit the exec helper recorded |
| a stop asked through the engine | told after the death | not told: podman records the stop as asked, and the node agent reads that record | told after the death: one asked through Incus’s API, and SHC’s own stop | told after the death: SHC’s own stop |
| a kill through the engine, a crash | docker kill | podman kill | Incus has no kill; a signal that ends the entry is a crash | the CLI has no kill; the same |
| health | told | told | told | told, SHC probes it |
| removal | told | told | told | told, from the polls |
| network and volume events, in the event feed | every one | every one | created, deleted and updated, networks and custom storage volumes | none |
Apple’s container CLI has no event stream, so the apple executor polls
the container list and tells what the polls see (SHC-2646): a run first
seen up is a start, a container no longer listed is a removal, and the
death of a run that began before a stop SHC asked is followed by a stop,
which the next run ends. A run that starts and ends between two polls is
told as its death alone.
Container isolation from the node plane
Section titled “Container isolation from the node plane”Linux’s node-plane rules use HostInput to compile the nft table
inet shc_node_plane. A pinned container enters on its own veth, the
host end of a Docker or Podman pin or an Incus routed NIC; a container on a
deployment’s own network enters on that network’s bridge.
An apple node holds its containers off the node plane
with the pf child anchor com.apple/shc/node-plane, compiled from the same
HostInput as the Linux nft table inet shc_node_plane, and loaded by shcd
while the Mac is a member. The design is in
Security, mesh networking.
Single-writer watchdog
Section titled “Single-writer watchdog”A container SHC labels shc.single_writer must stop once its node loses its
lease, or a replacement started elsewhere writes beside it. shcd stops it
itself while it runs. A stopped, hung or suspended shcd renews nothing and
stops nothing, so a second process does it: shcd watchdog, outside shcd,
on every node (ce modules/watchdog). It reads the lease expiry shcd writes
after each committed renew (node-lease-expiry.json in the state home) and,
once that has passed, stops each labelled container, escalating to a kill,
and appends what it did to watchdog.jsonl there. It needs no network and no
control plane.
It watches every engine it finds on the node and looks again every 30
seconds: docker and podman through their CLIs, incus through the local
Incus socket (/var/lib/incus/unix.socket, label
user.label.shc.single_writer), and on macOS Apple’s container CLI (ce
modules/watchdog/runtime.go, incus.go, apple.go). A node with no engine
it knows records one unenforced line and keeps looking.
The docker and podman CLIs dial the endpoint shcd’s own children of that
engine dial, which shcd writes into each lease expiry (engines): what
DOCKER_HOST, CONTAINER_HOST, a docker context or socket discovery points
shcd at. Only an absolute unix socket on the node is followed. Any other
endpoint (tcp://, ssh://) is refused and journaled, so fencing never
reaches another machine’s engine. With none recorded, the watchdog asks the
root engine (/var/run/docker.sock, or podman without a socket).
On Linux the package installs and starts the shc-watchdog systemd unit,
deliberately not tied to shc.service. On macOS shcd’s activation installs a
second LaunchDaemon, org.selfhostedcloud.shc-watchdog, beside its own (ce
core/hostactivate/platform_launchd.go); a failure there is a warning and
does not fail shcd’s start.
The deleted scheduler’s capabilities came back through these implementation steps, planned on 2026-09-30. On next:
| Capability | Step | On next |
|---|---|---|
| Assignments and cross-node operations | A3, SHC-1111 | Built. shc forward through the public route still resolves only on the daemon it reaches (SHC-1692) |
| Restart policy | A4, SHC-1112 | Built: the node agent restarts a replica |
| Replicas across nodes | B4, SHC-1145 | Built, each on the node it is bound to (SHC-1426) |
| Global deployments | B5, SHC-1146 | Built |
| Rescheduling and failover | C1-C3, SHC-1149 through SHC-1151 | Built: stateless replicas move after their grace; a replica with a volume moves through the dead-source move, and a single writer on a STONITH confirm or the lease proof from a node that runs the watchdog (SHC-1150); a pinned replica waits for its node |
| Rolling updates | D2, SHC-1155 | Built |
| Drain on the scheduler | D3, SHC-1156 | Built, through the placement engine |
| Configs and secrets staging | E2, SHC-1159 | Built: configs staged per node, secrets from the vault in RAM |
| Cross-node fabric | 7.1, SHC-1055 | Built |
| Ingress on every node | 8.1, SHC-1137 | Built |
| Service resolver | 8.4, SHC-1140 | Built: shcd answers DNS |
A forward whose engine can’t list the deployment’s containers is refused
X503929FWDENG, and a container gone between list and inspect is
X404921FWDCNT (SHC-1688). The SHC-1166 placeholder skips in the bats suites
still name these steps although the steps landed. A
successful single-node deployment is not proof of multi-node parity.