Layer 4. Primitives
Layer 4. Primitives. Layer page.
Primitives are mechanisms. They have no owner and no HUD page. Clusterfs, the node lease, labels and constraints, and pressure sit here. An owner and a HUD page is an object.
On this pin
Section titled “On this pin”The materialiser in shcd is the only writer of a node’s clusterfs.
Each section has its own directory, <clusterfs>/<section>/, with its
own rev-<n> revisions and its own current link. A build renders every
section into a staging directory inside the section’s directory. A
section whose content changed is fsynced, renamed to its next revision,
and its current swapped by rename; one that did not change keeps its
revision. The root REVISION is written last and marks the build
complete (ce modules/clusterfs/materialiser.go). A reader of a section
sees one revision or the next.
A container binds one section, never the whole tree:
/etc/shc/clusterfs/traefik for the system Traefik,
/etc/shc/clusterfs/collector for the collector,
/etc/shc/clusterfs/auth for forward-auth. It reads through the
section’s current/, which resolves inside the bind, so it sees each
swap on its next open and sees no other section. Each section stamps
FRESH with the revision and digest a clean build proved and when.
The auth section holds what forward-auth decides from: realms.json
(each realm’s issuer and client, and each host the cluster routes with
its realm, deployment, cookie domain and gating), platform_admins.json,
revocations.json, scoped_clients.json and ca-bundle.pem (ce
modules/app/auth_section.go).
The declared platform admins come from one row the leader writes as user
ids, so every node renders the same file. Each realm’s forward-auth client
secrets live in a secret section on the RAM root, auth-secrets, one file
per realm, bound at /etc/shc/auth-secrets. A revocation, a change to the
declared admins and a stamp of a new client secret each move the
revision, so every node carries it at its next build.
forward-auth decides from those files and calls shcd for no verdict. It
verifies the token against the realm’s keys, applies the revocations and
the declared admins, and decides with ce core/gatedecide, the package
shcd’s own gate handler uses. A gate handle is bound to its session with
one call to shcd at login and one per renewal, so the launcher on a gated
host still knows who is looking, and a logout proves its revocation with
the session’s own token as the bearer. The Traefik plugin’s realm lookups,
for the gate and the launcher, read realms.json and auth-secrets too.
Until a node has built the section, forward-auth and the plugin ask shcd as
a v0.1.21 node did, so an upgrade in any order serves on. A section that is
there but whose proof is older than five minutes, or names another digest,
is stale: every gated request is the coded 503 stale_replica, and
shc doctor’s system.auth-section names the node.
forward-auth answers its health check 503 until its node has built the
section, so Traefik sends it nothing before it can decide. shcd’s realm
lookup routes, realm-for-host and realm, answer only while a system
Traefik replica still runs a render that binds no auth section: the leader
lists those replicas into one kv row on every pass, and once the list is
empty both routes answer 410 X410059RLMGON. The list never latches, so a
node back from being offline with an old replica opens them again until it
is rolled.
The licence directory /etc/shc/license is the license section,
flat: each file it serves is named at the top of its directory as a
link through current/, so the system Traefik and Keycloak read
<bind>/license.lic as they always did. Every node renders it from the
store: the licence rows, the estate domain, the DNS record the leader
looked up, and the ingress intents the apps menu lists (ce
modules/app/license_section.go, shc modules/license/section.go). A
build starts from what the section served, so a row that cannot be read
keeps the file it fed. Secrets sections (certs) live on the RAM root
beside the run directory, never on this disk.
A spec stored on v0.1.21 still binds the whole tree, or a package file
through the root current. For those containers the root current
names a legacy/ directory of links, one per section to that section’s
current, so they keep reading live sections until they are
re-rendered. A render that binds the bare /etc/shc/clusterfs is
refused, so no new one appears. After each pass over its replicas a
node writes to the store what on it still reads through the view, and
every node’s build decides from all of those reports: it keeps the view
while any node reports one, and makes it again if one comes back. Once
every node reports none, the build reads the reports again from the
leader and removes legacy/, the root current and the whole-tree
revisions v0.1.21 left. A node with no report yet, an older build or
one that has not finished its first pass, leaves each view as it is. A
node the cluster cannot reach counts by the last report it wrote, and a
node that cannot reach the leader keeps its own view. A container that
reads through the view again once it is gone, a rollback to a spec
stored on v0.1.21 say, finds no current until its node’s next pass
reports it and the next build makes the view again.
shc doctor names each container still reading through it
(pods.clusterfs-whole-tree), with the shc update that moves it to
its own section.
A bind of a path below current/ holds the directory it resolved to
when the container started, and a revision is unlinked three builds
after it stops being current. So what a container binds, an app’s
package files, sits in the section’s store beside its revisions, and
the section links to it from its subtree (ce
modules/pods/files_section.go). An entry stays while any revision of
the section on disk links into it (SHC-1648).
While this node leads, the node lease writes its transitions and a
coalesced renewal (ce modules/nodelease/steward.go).
Placement is node labels and the selectors that match them: --where
and --prefer, named node selectors and constraints. shc node label puts a
label on a node; shc label, shc node selector and shc constraint read and
store the rest (ce modules/fence/commands/label.go, selector.go,
constraint.go). shc fence is retired; each of its verbs
refuses with X400251FNCGON and names its replacement (fence.go). The
placement term is location.
Automatic moves freeze while any pressure signal is red, and thaw
only after every signal has stayed calm (ce core/placement/freeze.go,
SHC-1208). The scheduler reports replicas waiting on the engine as
pressure (ce modules/scheduler/pressure.go).
The mechanisms are this page. Where the materialiser sits in a deploy is render once. A relocate that waits on the freeze is relocate.