Upgrading the daemon
SHC is upgraded by installing the newer package on each node and
restarting the daemon there. There is no shc upgrade: the package
manager installs, the daemon runs its schema migrations when it starts,
and the cluster carries on. Every node runs the same version, so a cluster
upgrade is a rolling one: one node at a time, never more than one voter
away, the leader last.
This page installs the package you kept at ~/shc.deb on every machine.
It is the version already running, so the version does not change; a newer
package goes through exactly these steps.
$ shc versionshc version 0.1.2build: embedded-tuiUNLICENSED/trial watermarked, see `shc license status`The leader writes a snapshot of the control-plane database every night
into /var/lib/shc/snapshots/control-plane; it is what a failed upgrade
is rolled back from. Read when the next one runs. A cluster younger than a
day has none yet.
$ shc cron ls...Take one now rather than wait for the night. shc db snapshot create
writes a snapshot through the nightly’s own dump into the same directory.
It runs on the leader only, home02 here; on any other node it refuses and
names the leader. shc db snapshot ls lists the snapshots the node it runs
on holds, newest first, the on-demand one beside any nightly, with its time
and size.
$ shc db snapshot create...$ shc db snapshot ls...Start with far01, the worker. A maintenance window tells the scheduler the node is away on purpose, so nothing fails over while the daemon restarts.
$ shc node maintenance far01 --window 10mwarning: node far01 is cordoned; this binds SHC's own placement only: it is excluded from install-time scoring and from drain relocation targets, but a compose deploy typed against it directly with `shc install` on that node or `--node far01` still lands there✔ node "far01" is in maintenance until 2026-10-11T05:51:29Z, availability cordoned: nothing is rescheduled from it before then$ dpkg -i ~/shc.deb...$ systemctl restart shc$ shc versionshc version 0.1.2build: embedded-tuiUNLICENSED/trial watermarked, see `shc license status`Wait for far01 to read healthy, then end the window.
$ shc node ls --where node=far01NAME STATE CONDITION AVAIL CLOUD LOCATION ROOM BUILD LEADER VOTERfar01 healthy cordoned cordoned learn learn/far ok 0.1.0 no no$ shc node maintenance far01 --end✔ maintenance on node "far01" lifted, availability activeThe voters next, one at a time. With three voters, two must answer at every moment, so wait for each to be healthy before the next.
$ shc node maintenance home03 --window 10mwarning: node home03 is cordoned; this binds SHC's own placement only: it is excluded from install-time scoring and from drain relocation targets, but a compose deploy typed against it directly with `shc install` on that node or `--node home03` still lands there✔ node "home03" is in maintenance until 2026-10-11T05:51:53Z, availability cordoned: nothing is rescheduled from it before then$ dpkg -i ~/shc.deb...$ systemctl restart shc$ shc cluster statuscluster name: shc-clusterleader: home02nodes: 4 healthy of 4voters: 3quorum: healthy, 2 voters requiredcordoned: 1 of 4 nodesleader-only work, recorded by home02 1m4s ago:...$ shc node maintenance home03 --end✔ maintenance on node "home03" lifted, availability active$ shc node maintenance home01 --window 10mwarning: node home01 is cordoned; this binds SHC's own placement only: it is excluded from install-time scoring and from drain relocation targets, but a compose deploy typed against it directly with `shc install` on that node or `--node home01` still lands there✔ node "home01" is in maintenance until 2026-10-11T05:57:11Z, availability cordoned: nothing is rescheduled from it before then$ dpkg -i ~/shc.deb...$ systemctl restart shc$ shc cluster statuscluster name: shc-clusterleader: home02nodes: 4 healthy of 4voters: 3quorum: healthy, 2 voters requiredcordoned: 1 of 4 nodesleader-only work, recorded by home02 16s ago:...$ shc node maintenance home01 --end✔ maintenance on node "home01" lifted, availability activeThe leader last. While its daemon restarts the other two voters elect a leader between them; the restarted node comes back as a follower.
$ shc node maintenance home02 --window 10mwarning: node home02 is cordoned; this binds SHC's own placement only: it is excluded from install-time scoring and from drain relocation targets, but a compose deploy typed against it directly with `shc install` on that node or `--node home02` still lands there✔ node "home02" is in maintenance until 2026-10-11T06:00:33Z, availability cordoned: nothing is rescheduled from it before then$ dpkg -i ~/shc.deb...$ systemctl restart shc$ shc cluster statuscluster name: shc-clusterleader: home01nodes: 4 healthy of 4voters: 3quorum: healthy, 2 voters requiredcordoned: 1 of 4 nodesleader-only work, recorded by home01 5s ago:...$ shc node maintenance home02 --end✔ maintenance on node "home02" lifted, availability activeEvery node reports the version it runs. Check them all.
$ shc versionshc version 0.1.2build: embedded-tuiUNLICENSED/trial watermarked, see `shc license status`$ shc versionshc version 0.1.2build: embedded-tuiUNLICENSED/trial watermarked, see `shc license status`$ shc versionshc version 0.1.2build: embedded-tuiUNLICENSED/trial watermarked, see `shc license status`$ shc versionshc version 0.1.2build: embedded-tuiUNLICENSED/trial watermarked, see `shc license status`An upgrade is also when the platform’s modules are re-ensured. shc cluster reconcile installs any module that init skipped and now fits;
--modules names the set, here the one the daemon already has.
openobserve still needs a node with three CPUs, and this cluster has none,
so reconcile skips it again and says so.
$ shc cluster reconcile --modules postgres,keycloak,openobserve,traefik...$ shc reconcile --modules postgres,keycloak,openobserve,traefik...Every deployment ran through the whole roll.
$ shc ls --where owner=learnSTACK TENANT ENV APP STATUS NODEa1 prod prod /root/learn/cluster/hello running home01a2 prod prod /root/learn/cluster/hello running far01a3 prod prod /root/learn/cluster/hello running home02a6 prod prod /root/learn/cluster/hello running home02A package older than the one running goes back the same way, node by node, as long as the newer one added no schema migration: a daemon whose database holds a migration it does not ship refuses to start. Keep the package you are replacing; the package manager does not.
The control-plane database keeps every event. shc db purge reports what
its rules would take and takes nothing without --yes; --before bounds
it. A cluster a day old has little to take.
$ shc db purge...$ shc db purge --before 30d...--yes performs the purge, --batch-size bounds each delete, and
--audit takes the compliance records with it, which no second copy
holds.
$ shc db purge --before 30d --batch-size 500 --yes$ shc db purge --audit --before 30d --yesAn image stays on a node after nothing there runs it. shc node prune
removes those on the nodes it names, or on every node with --all. It
keeps an image a container on the node runs, one a deployment there names,
one a rollback would need and one the node could not pull back, at any
age. It prints per node what it removed, what it kept and why.
--dry-run prints the same and removes nothing.
$ shc node prune --all --dry-run...The platform’s own Keycloak has an admin password the genesis minted. An
operator who lost it resets it on the node that holds the keycloak stack;
the recorded seed follows and the stack redeploys. Without --password a
new one is generated and printed once. The access ladder’s
Users and SSO page types it and logs in with the
new password, so it is not typed here.
$ shc system keycloak reset-admin-password --password P@ssw0rdNext: Repositories.