The control plane and its voters
The control-plane database is replicated by raft. A write commits when a majority of the voters agree. The voters are chosen from the control plane: the largest odd number of its members, up to five. Three members give three voters; far01 is outside the control plane, so it replicates everything and votes on nothing.
$ shc cluster nodes -vNAME ID HOSTNAME INTERNAL ADDRESS UNDERLAY ADDRESS PUBLIC ADDRESS CLOUD LOCATION MESH ADDRESS RUNTIME CPU CORES MEMORY BYTES CLUSTER STATE AVAILABILITY IS LEADER IS VOTER BUILD STORE CONDITION FREE CPU MILLICORES FREE MEMORY BYTES COMMITTEDfar01 node-194c5250de86 far01 10.48.0.4:4003 192.168.234.14 far01 learn learn/far 10.48.0.4 docker 2 8130596864 healthy active false false 0.1.51 replica ready 2000 8130596864 master/traefik/defaulthome01 node-1312bc7a966c home01 10.48.0.1:4003 192.168.234.11 203.0.113.11 learn learn/home 10.48.0.1 docker 2 8130596864 healthy active true true 0.1.51 voter ready 2000 8130596864 master/keycloak/default, master/postgres/default, master/traefik/defaulthome02 node-460e3fb4e787 home02 10.48.0.2:4003 192.168.234.12 203.0.113.11 learn learn/home 10.48.0.2 docker 2 8130592768 healthy active false true 0.1.51 voter ready 2000 8130592768 master/traefik/defaulthome03 node-5668c56ca532 home03 10.48.0.3:4003 192.168.234.13 203.0.113.11 learn learn/home 10.48.0.3 docker 2 8130596864 healthy active false true 0.1.51 voter ready 2000 8130596864 master/traefik/default$ shc label show control-planeLABEL control-planesource systemdescription -reserved nogranted to -backup repo -created 2026-11-07 02:42:52
NODESNODE CONDITION AVAILABILITY SOURCEhome01 ready active derivedhome02 ready active joinhome03 ready active join
CONSTRAINTS none
DEPLOYMENTS nonewarning: control-plane has no spare, fewer than four nodes$ shc node ls -v...One voter leads. Every node knows who, and only the leader has watched the others.
$ shc leader statusROLE LEADER OBSERVED ATleader node-1312bc7a966c 2026-11-07T02:49:27Z$ shc leader healthNODE ID STATE OBSERVED AT ISSUEnode-1312bc7a966c healthy 2026-11-07T02:49:27Z -node-194c5250de86 healthy 2026-11-07T02:49:27Z -node-460e3fb4e787 healthy 2026-11-07T02:49:27Z -node-5668c56ca532 healthy 2026-11-07T02:49:27Z -$ shc leader statusROLE LEADER OBSERVED ATfollower node-1312bc7a966c 2026-11-07T02:49:28Z$ shc leader healthNODE ID STATE OBSERVED AT ISSUEnode-1312bc7a966c healthy 2026-11-07T02:49:27Z -node-194c5250de86 healthy 2026-11-07T02:49:27Z -node-460e3fb4e787 healthy 2026-11-07T02:49:27Z -node-5668c56ca532 healthy 2026-11-07T02:49:27Z -A node’s row holds everything the cluster knows about it.
$ shc node info far01ID HOSTNAME STORE RUNTIME AVAILABILITY INTERNAL ADDRESS UNDERLAY ADDRESS PUBLIC ADDRESS CLOUD LOCATION IS VOTER FABRIC CHUNKS FABRIC ADDRESSESnode-194c5250de86 far01 replica docker active 10.48.0.4:4003 192.168.234.14 far01 learn learn/far false 10.128.1.0/26 3 held
┌─ committed ──────────────────────────────────────────────────────────────────────────────┐│ DEPLOYMENT RESERVED CPU RESERVED MEMORY LEDGER CPU LEDGER MEMORY ││ master/traefik/default 0.0 0B 1.1 640.0M │└──────────────────────────────────────────────────────────────────────────────────────────┘$ shc node health far01NAME STATUS HEALTH STATUS CHECKED ATfar01 active healthy 2026-11-07T02:49:29Z
┌─ checks ─────────────┐│ connection ok ││ runtime docker │└──────────────────────┘
┌─ resources ──────────────────────┐│ disk_available_bytes 37.13 GB ││ disk_bytes 1.94 GB ││ disk_path / ││ disk_total_bytes 39.08 GB │└──────────────────────────────────┘Membership in the control plane is a node property. Adding far01 promotes nothing: four members still give three voters, and the three are taken.
$ shc node update far01 --control-plane=true✔ set control-plane of node "far01" to true$ shc cluster statuscluster name: shc-clusterleader: home01nodes: 4 healthy of 4voters: 3quorum: healthy, 2 voters requiredleader-only work, recorded by home01 55s ago:...Taking home03 out of the control plane leaves a voter outside it. The leader
runs one suffrage change a minute, a promotion before a demotion: far01 is
promoted first, then home03 is demoted. Run shc node ls until far01 votes
and home03 does not.
$ shc node update home03 --control-plane=false✔ set control-plane of node "home03" to false$ shc node lsNAME STATE CONDITION AVAIL CLOUD LOCATION ROOM BUILD STORE LEADER VOTERfar01 healthy ready active learn learn/far ok 0.1.47 voter no yeshome01 healthy ready active learn learn/home ok 0.1.47 voter yes yeshome02 healthy ready active learn learn/home ok 0.1.47 voter no yeshome03 healthy ready active learn learn/home ok 0.1.47 replica no no$ shc cluster statuscluster name: shc-clusterleader: home01nodes: 4 healthy of 4voters: 3quorum: healthy, 2 voters requiredleader-only work, recorded by home01 56s ago:...A vote in far means a write may wait on the far link. Put the control
plane back the way it was: home03 in, far01 out, and wait for the two
passes again.
$ shc node update home03 --control-plane=true✔ set control-plane of node "home03" to true$ shc node update far01 --control-plane=false✔ set control-plane of node "far01" to false$ shc node lsNAME STATE CONDITION AVAIL CLOUD LOCATION ROOM BUILD STORE LEADER VOTERfar01 healthy ready active learn learn/far ok 0.1.47 replica no nohome01 healthy ready active learn learn/home ok 0.1.47 voter yes yeshome02 healthy ready active learn learn/home ok 0.1.47 voter no yeshome03 healthy ready active learn learn/home ok 0.1.47 voter no yes$ shc cluster statuscluster name: shc-clusterleader: home01nodes: 4 healthy of 4voters: 3quorum: healthy, 2 voters requiredleader-only work, recorded by home01 36s ago:...Voters are one of the store’s three tiers. One node in every
store.replica_every, which is 50 unless set, runs a replica of the store,
with at least one in every location, and serves the nodes near it. Every
other node is a client: it runs no store and reaches one through a replica.
A cluster of no more nodes than store.replica_every has no clients, so
every node here runs the store. shc node update --store holds a node to a
tier: voter, which only a control-plane node can be, replica or
client. An empty value lets the leader choose again. The leader moves one
node at a time, and a move that a pin set or freed halfway through no longer
fits is dropped before the node is touched. A move open longer than five
minutes is a warning in shc doctor and amber on cluster.store_tier_moves
in shc status.
$ shc config set store.replica_every 20 --scope global$ shc node update far01 --store client$ shc node update far01 --store ""The daemon logs on every node, and its raft engine runs inside it, so the engine’s lines are in the daemon’s log. The last lines of it on home02:
$ shc cluster logs home02 -S shc -n 3...shc node update also rewrites, in place, the metadata a join recorded:
the cloud, the location and the public address.
$ shc node update far01 --cloud learn --location far --public-address far01...$ shc node info far01...Two group verbs answer on their own. shc cluster alone prints what
shc cluster status prints, and shc node alone lists the nodes,
--verbose with the detail shc node ls -v shows.
$ shc cluster...$ shc node...$ shc node --verbose...Next: Locations and labels.