Storage providers

SHC supports three storage provider types for stack volumes and backups:

An SHC cluster can have multiple providers — e.g. local for per-node fast volumes plus nfs for shared backups. Each stack picks which provider to use based on the app’s requires.storage whitelist and operator-chosen preferences.

Source of truth (ce module): modules/storage/storageapi/types.go (types, ProviderConfig, the Capabilities set), modules/storage/providers/ (the provider implementations), modules/storage/commands/ (CLI).

For the per-stack resolution logic (how SHC picks which provider for a given stack) see architecture/modules/storage.md.

Every SHC installation starts with a local provider pointing at the daemon’s state directory (typically /var/lib/shc/storage). No configuration needed — shc init sets it up.

Capabilities are auto-detected by detection.go based on the filesystem backing the state directory:

• ZFS → supports_quotas=true, supports_snapshots=true, native snapshots, native quotas
• XFS (with project quotas enabled) → supports_quotas=true, snapshot via rsync
• Btrfs → supports_snapshots=true (CoW snapshots), supports_quotas=true (qgroups)
• ext4 → supports_quotas=false, supports_snapshots=false (rsync fallback for snapshots works, but expensive)

To see what was detected:

shc storage ls
shc storage show local

• Production single-node, heavy IO: ZFS with a dedicated dataset. Best snapshot + quota + IO story SHC supports.
• Production multi-node with local-mode stacks: Btrfs subvolumes, or ZFS if you have it. Fall through to NFS / CephFS for shared volumes.
• Development or small deployments: ext4 is fine. You won’t get instant snapshots but rsync-based backup will still work, just slower.

You can add a second local provider pointing at a different mount — e.g. a fast SSD for DB data, keeping the default provider on slower storage:

shc storage add fast-nvme --path /mnt/nvme/shc --node node-a
shc storage link fast-nvme node-a node-b    # attach to specific nodes

Options:

• --path — mount path. Must be writable by the shc user.
• --node — single-node scope. Omit for cluster-wide.

Commands:

• shc storage add <name> --path <path> [--node <id>] — commands/add.go
• shc storage link <name> <node-id> [<node-id>...] — attach a provider to specific nodes (commands/link.go)
• shc storage unlink <name> <node-id> — detach
• shc storage rm <name> — remove a provider (refuses if stacks are using it)

Reserved names that you cannot use for a user-added provider: local, nfs, cephfs — these are the managed singletons created via their dedicated enable commands.

• Multi-node cluster where the same stack can schedule to any node and needs the same data
• Shared backup repository (backup.repo in default.yaml)
• You want the simplest possible shared-storage setup — NFS “just works” over a LAN

SHC ships a managed NFS singleton — one NFS server across the cluster, with SHC handling exports, client mounts, and reconnects.

shc storage nfs enable --export-path /mnt/shc/nfs --network 10.0.0.0/24

What happens:

1. SHC designates one node as the NFS server (the one you ran the command on, unless overridden) and installs nfs-kernel-server.
2. Exports <export-path> to the given network CIDR.
3. Installs nfs-common on every other cluster node.
4. Mounts the export on each client node under /mnt/shc/nfs-mount (or similar — see shc storage show nfs).
5. Registers an nfs provider in the control-plane database pointing at the mount.

Source: modules/storage/commands/managed.go (the shc storage nfs verbs), modules/storage/shared_manager.go + modules/storage/nfs/ (exports, cleanup).

• Shared access: yes — that’s the whole point.
• Snapshots: rsync-based — SHC stages snapshots by rsync’ing to a .snapshots directory on the NFS export. Works on any backing filesystem but isn’t atomic.
• Quotas: no — NFS doesn’t model per-directory quotas in a way SHC can enforce.
• Locking: SHC uses application-level locking (database-backed) for concurrency, so NFS’s own locking quirks don’t bite you.
• Performance: fine for most web-app workloads. Not suitable for databases that fsync heavily — run those on local storage and only back them up to NFS.

shc storage nfs enable           # create the singleton (run once per cluster)
shc storage nfs status           # check server & client mount health
shc storage nfs join             # have *this* node (re)mount the NFS share
shc storage nfs leave            # have this node unmount (pre-decommission)
shc storage nfs disable          # tear down the singleton (CAREFUL — affects all nodes)

If you already have an NFS server you want SHC to consume (instead of letting SHC manage one), add it as an external provider:

# Mount the NFS export yourself (via /etc/fstab or autofs),
# then register it as a local-type external provider:
shc storage add team-nfs --path /mnt/team-nfs

SHC will treat it like a local provider backed by whatever filesystem the mount exposes.

• You already run Ceph for other workloads and want SHC to participate
• You need native atomic snapshots of shared storage (NFS snapshots aren’t atomic)
• Larger clusters where you want to decouple storage from SHC-managed infrastructure

CephFS is configured by connecting SHC to an existing Ceph cluster — SHC does not provision Ceph for you.

shc storage cephfs enable \
  --monitors 10.0.0.10:6789,10.0.0.11:6789,10.0.0.12:6789 \
  --user shc \
  --secret-file /etc/ceph/shc.keyring \
  --fs-name shc_fs

SHC will:

1. Validate the Ceph credentials can reach a monitor.
2. Create the SHC subdirectory on the CephFS filesystem if needed.
3. Mount it on every cluster node via mount -t ceph ....
4. Register a cephfs provider in the control-plane database.

Source: modules/storage/commands/managed.go (the shc storage cephfs verbs), modules/storage/ceph/manager.go.

• Shared access: yes
• Snapshots: native, atomic — faster and safer than NFS+rsync
• Quotas: limited — Ceph supports directory quotas; SHC exposes them where the subdirectory is set up with quota attributes
• Setup complexity: you need a working Ceph cluster first. If you don’t already run Ceph, NFS is the pragmatic choice.

Beyond the filesystem providers above, SHC can back an individual compose mount with a provider-managed block volume — a real cloud disk (Hetzner Cloud volume, OpenStack Cinder, AWS EBS, GCE PD, Azure Disk, DigitalOcean, Linode, Vultr, CloudStack) that is created, attached, and mounted for that one mount, instead of a directory on the node’s filesystem.

An app opts a mount in with the x-shc-volumes compose extension:

x-shc-volumes:
  pgdata:
    size: 10Gi   # Ki/Mi/Gi/Ti (binary), K/M/G/T (decimal), or bare bytes
    access: rwo  # rwo = single-node read-write (default), rox = read-only multi-attach
volumes:
  pgdata:

What you get compared to a plain directory:

• Survives the node. The disk is a first-class cloud resource; a deployment move re-attaches it to the target node (same location) instead of copying files.
• Sized and enforced per mount — size is the disk’s real capacity, not a quota approximation.
• Backup-reconstructable. Backups record the volume identity, and a restore can re-provision + re-attach the disk from the backup when the original is gone.
• Fenced single-writer. rwo volumes carry a monotonic attach fence, so a partitioned node can never keep writing after the cluster rebinds the volume elsewhere.

The cloud credentials come from the connection registry (shc connection), the same place the cluster’s provisioning credentials live — the storage module resolves the node’s provider (for example hcloud on a Hetzner node) and calls the cloud API with the registered connection. Nodes without a block-volume provider fall back to the app’s whitelisted filesystem providers.

SHC’s storage module (modules/storage/, ce module) picks a provider at install time based on:

1. App requirement: requires.storage: [local, nfs] in the app’s app.yaml whitelists which providers are acceptable.
2. Per-stack override: shc install <app> --storage <provider> at install time forces a specific provider (must still be in the whitelist).
3. Node affinity: the stack can only use providers that are linked to its assigned node.
4. Capability requirements: if the app or install flow needs snapshots or quotas, providers without those capabilities are filtered out.
5. Operator default: if multiple providers qualify, SHC picks the cluster default (configurable in default.yaml).

If no provider qualifies, install fails with X400937QNOSUP (quotas not supported) / X400942SNOSUP (snapshots not supported) / X404914STGNOF (storage provider not found).

shc storage ls
shc doctor run --category storage    # structured diagnostics

The storage module registers a storage.local diagnosis: it walks every registered provider and checks that the path exists, is a directory, and is a mount point. A missing or non-directory path is an error; an existing path that is not a mount point is a warning. Source: modules/storage/doctor.go in the ce module.

“Provider is accessible but stacks fail with X400942SNOSUP on backup.” → The backing filesystem doesn’t support snapshots. Install stacks on a provider whose capabilities include snapshots, or accept the rsync-fallback path (slower, non-atomic).

“NFS mount keeps dropping on reboot.” → Check that shc storage nfs join has registered the mount in the SHC-managed systemd unit. Distro-level /etc/fstab entries bypass SHC and should be removed if present.

“CephFS mount requires ceph kernel module.” → Load it (modprobe ceph) and add to /etc/modules-load.d/. SHC’s cephfs-enable checks for this and refuses if missing.

“Adding a provider fails with ‘name is reserved’.” → local, nfs, and cephfs are reserved for managed singletons. Use shc storage nfs enable / cephfs enable instead of storage add nfs.

Deleting a provider refuses if any stack still uses it. First migrate stacks away (via shc move or reinstall), then:

shc storage rm <name>

For the managed singletons, use the disable commands — they tear down the server/mount infrastructure as well:

shc storage nfs disable      # CAREFUL — removes the cluster-wide mount
shc storage cephfs disable   # unmounts Ceph on all nodes

Disable refuses if any stacks are still using the provider.

• Multi-node cluster setup — plan storage before adding nodes
• Upgrading SHC — storage considerations during upgrade
• Architecture: storage module — internals
• App definition — how apps declare storage requirements
• CLI reference: storage — full command-line reference (see also shc storage nfs --help, shc storage cephfs --help)