Skip to content
SHC Docs

The control plane and its voters

The control-plane database is replicated by raft. A write commits when a majority of the voters agree. The voters are chosen from the control plane: the largest odd number of its members, up to five. Three members give three voters; far01 is outside the control plane, so it replicates everything and votes on nothing.

home01
$ shc cluster nodes -v
NAME ID HOSTNAME INTERNAL ADDRESS UNDERLAY ADDRESS PUBLIC ADDRESS CLOUD LOCATION MESH ADDRESS RUNTIME CPU CORES MEMORY BYTES CLUSTER STATE AVAILABILITY IS LEADER IS VOTER BUILD STORE CONDITION FREE CPU MILLICORES FREE MEMORY BYTES COMMITTED
far01 node-194c5250de86 far01 10.48.0.4:4003 192.168.234.14 far01 learn learn/far 10.48.0.4 docker 2 8130596864 healthy active false false 0.1.51 replica ready 2000 8130596864 master/traefik/default
home01 node-1312bc7a966c home01 10.48.0.1:4003 192.168.234.11 203.0.113.11 learn learn/home 10.48.0.1 docker 2 8130596864 healthy active true true 0.1.51 voter ready 2000 8130596864 master/keycloak/default, master/postgres/default, master/traefik/default
home02 node-460e3fb4e787 home02 10.48.0.2:4003 192.168.234.12 203.0.113.11 learn learn/home 10.48.0.2 docker 2 8130592768 healthy active false true 0.1.51 voter ready 2000 8130592768 master/traefik/default
home03 node-5668c56ca532 home03 10.48.0.3:4003 192.168.234.13 203.0.113.11 learn learn/home 10.48.0.3 docker 2 8130596864 healthy active false true 0.1.51 voter ready 2000 8130596864 master/traefik/default
$ shc label show control-plane
LABEL control-plane
source system
description -
reserved no
granted to -
backup repo -
created 2026-11-07 02:42:52
NODES
NODE CONDITION AVAILABILITY SOURCE
home01 ready active derived
home02 ready active join
home03 ready active join
CONSTRAINTS
none
DEPLOYMENTS
none
warning: control-plane has no spare, fewer than four nodes
$ shc node ls -v
...

One voter leads. Every node knows who, and only the leader has watched the others.

home01
$ shc leader status
ROLE LEADER OBSERVED AT
leader node-1312bc7a966c 2026-11-07T02:49:27Z
$ shc leader health
NODE ID STATE OBSERVED AT ISSUE
node-1312bc7a966c healthy 2026-11-07T02:49:27Z -
node-194c5250de86 healthy 2026-11-07T02:49:27Z -
node-460e3fb4e787 healthy 2026-11-07T02:49:27Z -
node-5668c56ca532 healthy 2026-11-07T02:49:27Z -
home02
$ shc leader status
ROLE LEADER OBSERVED AT
follower node-1312bc7a966c 2026-11-07T02:49:28Z
$ shc leader health
NODE ID STATE OBSERVED AT ISSUE
node-1312bc7a966c healthy 2026-11-07T02:49:27Z -
node-194c5250de86 healthy 2026-11-07T02:49:27Z -
node-460e3fb4e787 healthy 2026-11-07T02:49:27Z -
node-5668c56ca532 healthy 2026-11-07T02:49:27Z -

A node’s row holds everything the cluster knows about it.

home01
$ shc node info far01
ID HOSTNAME STORE RUNTIME AVAILABILITY INTERNAL ADDRESS UNDERLAY ADDRESS PUBLIC ADDRESS CLOUD LOCATION IS VOTER FABRIC CHUNKS FABRIC ADDRESSES
node-194c5250de86 far01 replica docker active 10.48.0.4:4003 192.168.234.14 far01 learn learn/far false 10.128.1.0/26 3 held
┌─ committed ──────────────────────────────────────────────────────────────────────────────┐
│ DEPLOYMENT RESERVED CPU RESERVED MEMORY LEDGER CPU LEDGER MEMORY │
│ master/traefik/default 0.0 0B 1.1 640.0M │
└──────────────────────────────────────────────────────────────────────────────────────────┘
$ shc node health far01
NAME STATUS HEALTH STATUS CHECKED AT
far01 active healthy 2026-11-07T02:49:29Z
┌─ checks ─────────────┐
│ connection ok │
│ runtime docker │
└──────────────────────┘
┌─ resources ──────────────────────┐
│ disk_available_bytes 37.13 GB │
│ disk_bytes 1.94 GB │
│ disk_path / │
│ disk_total_bytes 39.08 GB │
└──────────────────────────────────┘

Membership in the control plane is a node property. Adding far01 promotes nothing: four members still give three voters, and the three are taken.

home01
$ shc node update far01 --control-plane=true
✔ set control-plane of node "far01" to true
$ shc cluster status
cluster name: shc-cluster
leader: home01
nodes: 4 healthy of 4
voters: 3
quorum: healthy, 2 voters required
leader-only work, recorded by home01 55s ago:
...

Taking home03 out of the control plane leaves a voter outside it. The leader runs one suffrage change a minute, a promotion before a demotion: far01 is promoted first, then home03 is demoted. Run shc node ls until far01 votes and home03 does not.

home01
$ shc node update home03 --control-plane=false
✔ set control-plane of node "home03" to false
home01
$ shc node ls
NAME STATE CONDITION AVAIL CLOUD LOCATION ROOM BUILD STORE LEADER VOTER
far01 healthy ready active learn learn/far ok 0.1.47 voter no yes
home01 healthy ready active learn learn/home ok 0.1.47 voter yes yes
home02 healthy ready active learn learn/home ok 0.1.47 voter no yes
home03 healthy ready active learn learn/home ok 0.1.47 replica no no
$ shc cluster status
cluster name: shc-cluster
leader: home01
nodes: 4 healthy of 4
voters: 3
quorum: healthy, 2 voters required
leader-only work, recorded by home01 56s ago:
...

A vote in far means a write may wait on the far link. Put the control plane back the way it was: home03 in, far01 out, and wait for the two passes again.

home01
$ shc node update home03 --control-plane=true
✔ set control-plane of node "home03" to true
$ shc node update far01 --control-plane=false
✔ set control-plane of node "far01" to false
home01
$ shc node ls
NAME STATE CONDITION AVAIL CLOUD LOCATION ROOM BUILD STORE LEADER VOTER
far01 healthy ready active learn learn/far ok 0.1.47 replica no no
home01 healthy ready active learn learn/home ok 0.1.47 voter yes yes
home02 healthy ready active learn learn/home ok 0.1.47 voter no yes
home03 healthy ready active learn learn/home ok 0.1.47 voter no yes
$ shc cluster status
cluster name: shc-cluster
leader: home01
nodes: 4 healthy of 4
voters: 3
quorum: healthy, 2 voters required
leader-only work, recorded by home01 36s ago:
...

Voters are one of the store’s three tiers. One node in every store.replica_every, which is 50 unless set, runs a replica of the store, with at least one in every location, and serves the nodes near it. Every other node is a client: it runs no store and reaches one through a replica. A cluster of no more nodes than store.replica_every has no clients, so every node here runs the store. shc node update --store holds a node to a tier: voter, which only a control-plane node can be, replica or client. An empty value lets the leader choose again. The leader moves one node at a time, and a move that a pin set or freed halfway through no longer fits is dropped before the node is touched. A move open longer than five minutes is a warning in shc doctor and amber on cluster.store_tier_moves in shc status.

explained, not walked: a cluster of four has no store clients, and this ladder keeps every node on the store
$ shc config set store.replica_every 20 --scope global
$ shc node update far01 --store client
$ shc node update far01 --store ""

The daemon logs on every node, and its raft engine runs inside it, so the engine’s lines are in the daemon’s log. The last lines of it on home02:

home01
$ shc cluster logs home02 -S shc -n 3
...

shc node update also rewrites, in place, the metadata a join recorded: the cloud, the location and the public address.

home01
$ shc node update far01 --cloud learn --location far --public-address far01
...
$ shc node info far01
...

Two group verbs answer on their own. shc cluster alone prints what shc cluster status prints, and shc node alone lists the nodes, --verbose with the detail shc node ls -v shows.

home01
$ shc cluster
...
$ shc node
...
$ shc node --verbose
...

Next: Locations and labels.