Security model

How SHC authenticates callers, how it authorizes actions against tenants and resources, how tenants are isolated from each other, and how cluster-internal traffic is secured. This page is the canonical reference; the core/api/auth.go, modules/auth/types.go, and modules/auth/middleware.go modules are the source of truth.

Every request resolves a caller through Service.CurrentUser in modules/auth/service.go, using the auth precedence extracted by ExtractAuth in core/api/auth.go. There are exactly two transports that can authenticate a caller:

Requests arriving over the UDS listener (default /run/shc/shc.sock) are treated as local admin requests and minted by NewLocalUser (modules/auth/types.go) as a synthetic *User with roles ["platform:manage"] and a wildcard permission {Resource: "*", Scopes: ["*"]} (wildcard tenant access comes from IsPlatformAdmin() / CanAccessTenant, not a tenants set). Access to the socket file is gated by filesystem permissions — the socket is owned by the shc user, and its group is shc; members of that group (set with usermod -aG shc <user>) can open it. Everything else cannot.

The detection lives in the udsMarkerMiddleware that tags the request with Transport=uds at ingress; CurrentUser mints the local admin when that marker is present. Authorization bypass (see §2.4) keys off the unexported isLocal bool flag on *User (set only by NewLocalUser), not a claim string or a Go type assertion — so a Keycloak user whose sub happens to equal "local-user" cannot escalate.

Every other transport on the public listener — HTTP(S) and WebSockets — must present a Bearer token that validate_keycloak_token(token) accepts. The token is validated against the Keycloak realm configured in authentication.keycloak.* (see config keys). On success, the returned User includes the sub, roles, tenant claims, and resolved permissions (§2.2).

Failures raise X401900NOAUTH (no credentials) or X401901* (token invalid). Failed authentications emit A401000AUTHFL audit events with a reason tag.

The Keycloak-protected, tenant REST API (what clients and the CLI use over the network) listens plaintext on 0.0.0.0:8000 (overridable with server.tcp.bind, or SHC_PORT for per-daemon test isolation; the public host is SHC_HOST) (the http listener subsystem in subsystems/impl/http.go). TLS is not terminated inside SHC — traefik (installed as a system app) fronts this listener and handles TLS, Let’s Encrypt, and routing by host+path. ACME resolvers are injected by the daemon’s traefik seed (see apps/traefik/compose.yaml), not by an app-level plugin.

Once a caller is authenticated, every endpoint that changes state (and most that read it) is wrapped with RequirePermissions(...) from modules/auth/middleware.go. The middleware runs four checks in order.

RequirePermissions pulls the Ctx object from the request context. If no Ctx is available the request is rejected with X403000AUTHZF — this is a programming error, not an auth failure, and indicates a route that forgot to take a ctx: Ctx = Depends(get_ctx) parameter.

Non-UDS requests must carry a resolved user. Missing user ⇒ X403903NOUSER.

Before any scope check, ctx.user.can_access_tenant(ctx.tenant_name) must return True. The tenant on the Ctx was derived from an untrusted source — the X-SHC-Tenant HTTP header, or the --tenant CLI flag — so this check is mandatory. Without it, a user with apps:write on tenant A could set the header to tenant B and write apps in B.

CanAccessTenant (modules/auth/types.go) considers three cases (the User model carries a single TenantName string field — there is no multi-valued tenants set):

1. Empty tenant (tenantName == "", cluster-wide probe) → allow; the per-scope check is the real gate.
2. Platform admin (IsPlatformAdmin() — holds platform:manage, or is the synthetic LocalUser) → allow.
3. user.TenantName == requested → allow. This single-tenant equality is the primary match for Keycloak users.

Denial raises X403904NOTENT and emits an A403002TENTDNY audit event with the attempted tenant and the user’s sub.

The required scopes are the arguments to @permissions(...) — e.g. @permissions("stack:write"). The current resource is derived from the Ctx by _build_resource:

• If the endpoint declared resource_type="tenant", or the Ctx has no stack and no environment → resource is "tenant".
• If the Ctx has a stack → resource is "stack:<stack_name>".
• If it only has an environment → resource is "environment:<name>".
• Otherwise → "tenant".

HasPermission(userPermissions, resource, requiredScopes) then walks the user’s Permission objects:

// pseudocode
for _, perm := range user.Permissions {
    if matchResource(perm.Resource, currentResource) {
        for _, userScope := range perm.Scopes {
            for _, required := range requiredScopes {
                if matchScope(userScope, required) {
                    return allow
                }
            }
        }
    }
}
return deny

Denial raises X403900NOPERM(scope=…, resource=…) and emits an A403001PERMDNY audit event.

LocalUser instances short-circuit this entire pipeline — that’s the bypass from §1.1.

Scopes are "<noun>:<verb>":

• match_scope(user, required) allows:
  • "*" → matches everything.
  • Exact match (stack:write = stack:write).
  • Noun wildcard: user has *:read, required is stack:read → allow.
  • Verb wildcard: user has stack:*, required is stack:delete → allow.

match_resource(user, required) uses fnmatch globs, so a policy with resource="stack:prod-*" will match stack:prod-web but not stack:dev-web.

Most modules gate access via CanAccessTenant + a request-scoped tenant check rather than a per-noun @permissions vocabulary. The explicit RequirePermissions(...) middleware guards only a handful of routers with these scopes. Source: rg "RequirePermissions\(" over the ce module and this repo’s trees (non-test call sites).

Users don’t carry permissions directly in their token; they carry roles. The server translates roles → permissions by evaluating Policy objects (typically sourced from Keycloak custom policies; see Policy.from_keycloak). Each Policy carries:

• name
• roles — if the user has any of these, the policy matches.
• resources — which resource patterns (stack:*, stack:prod-web, tenant, …) the policy grants access to.
• scopes — which scope strings (stack:read, backup:write, …) are granted on those resources.

get_permissions_for_roles(policies, roles) flattens matching policies into the Permission list that the user object carries into RequirePermissions.

Policy evaluation is expensive (Keycloak round-trip for policy lookup), so results are cached per-client in PolicyCache with a TTL taken from keycloak.policy_cache_ttl (default 60 seconds). The cache is an in-process singleton — if a policy changes in Keycloak, it takes up to the TTL to propagate. Call PolicyCache.invalidate() to force a refresh.

Tenants are the top-level namespace in SHC. Every stack, environment, schedule, backup, audit row, and vault secret is rooted at a tenant. Isolation is enforced at four layers:

1. Authorization (above): the IDOR guard in @permissions blocks cross-tenant requests before the handler runs.
2. Database: every table that carries user data has a tenant_name column; every query includes it in the WHERE clause (see the queries.go files in each module).
3. Filesystem: per-tenant directories under paths.TenantsDir(tenant) (see core/paths/paths.go). Compose project names (via build_docker_project_name(tenant, stack, env)) carry the tenant so container names don’t collide.
4. Docker networks: the network module creates per-tenant overlay networks (ensure_tenant_network) so stacks in the same tenant can reach each other while stacks in other tenants cannot.

What is not isolated: host kernel resources, the control plane process, and the underlying docker daemon. SHC is a multi-tenant-within-one-trust-boundary system, not a hostile- multi-tenant system. Operators should not run untrusted-code tenants on the same node.

Cluster-internal traffic falls into two kinds:

The internal API (/internal/*, mounted by buildRouter in modules/cluster/router.go) is used for leader → follower operations, cross-node shc stack exec, log streaming, etc. It is authenticated with mTLS: each node presents a client certificate signed by the cluster CA (modules/cluster/cert_provider.go). Certificates are minted at cluster-join time using a short-lived join token (§5.3) and rotated on a schedule (subsystems/cluster_cert_rotation.go).

The internal API is not exposed to end users. It runs on a single listener (TCP 4003) and, by default, binds to all interfaces (0.0.0.0) so joining nodes can reach it before the mesh is established. Exactly one port, always TLS.

The listener is configured with ssl_cert_reqs=CERT_OPTIONAL and ssl_ca_certs=<cluster CA>. This is a deliberate two-layer design, not a weakened mTLS mode:

1. TLS layer. OpenSSL chain-verifies any client cert the peer presents against the cluster CA during the handshake. A cert that fails verification aborts the handshake — it never reaches the application. A peer that presents no cert at all is allowed through the handshake (so a joining node with no cert yet can reach /internal/enroll) but that is the only thing CERT_OPTIONAL allows past TLS.
2. Application layer. Every route except the enrollment allowlist depends on VerifyInternalAuth (core/api/auth.go), which reads the validated peer cert off the TLS connection state and rejects TLS connections that don’t carry one. A cryptographic “cluster membership” gate, just done one layer up from the kernel socket so the single listener can also serve /enroll — the exact endpoint a joiner-with-no-cert needs.

Net effect on the wire: every non-enrollment route is equivalent to CERT_REQUIRED, and every byte on the socket is TLS-encrypted, including the enrollment exchange itself. A plaintext HTTP request to port 4003 is physically impossible; the server only speaks TLS.

Still:

• Production deployments MUST firewall TCP 4003 from the public internet. Block it at your host / cloud firewall, leaving only the mesh subnet (defaults to 10.42.0.0/16) and, for new nodes during join, the specific source IP(s) that will run shc node join.
• Operators who want a stricter physical-network boundary bind the internal API to a specific NIC with shc init --nic <iface> / shc node join --nic <iface> (the same pin drives every cluster-internal address), or set the internal_api.bind config key (SHC_INTERNAL_API_BIND) for an explicit address. See configuration files reference.
• The enrollment endpoint (/internal/enroll) is explicitly allowlisted not to require a cluster-CA client cert (the joiner doesn’t have one yet) but it does still run over TLS. The cluster-CA public certificate ships embedded in the single-use, TTL-bound shc node token join token, so the joining node pins the leader’s server cert against the cluster CA during /internal/enroll — closing the MITM window that would otherwise exist when the joiner has no prior trust anchor. Authentication of the call itself is the bootstrap token (§5.3); firewalling 4003 closes even that narrow window.

Durable cluster state (tenants, stacks, schedules, quotas, audit log, …) lives in rqlite — raft-replicated SQLite, run as an external rqlited process the daemon supervises. Raft replication runs on :4002 between members; members join as voters by default, and a non-voter member replicates read-only.

Transport security (#58 raft mTLS). The raft plane is gated by the RaftTLS posture recorded in node_bootstrap.json:

• Genesis default-on — a from-empty shc init always writes RaftTLS: true (modules/cluster/service.go), so every fresh cluster runs raft replication (:4002) and the rqlited HTTP API (:4001) under mutual TLS: rqlited gets -node-cert/-node-key/-node-ca-cert + -node-verify-client and -http-cert/-http-key/-http-ca-cert (core/rqlited/manager.go), using the per-node cluster certs under {state}/certs/ (ca.pem, node.pem, node-key.pem).
• Leader-inherited — a joiner never decides its own posture; it adopts the leader’s RaftTLS from the enroll response (modules/cluster/enroll_handler.go), so a cluster is always uniformly TLS or uniformly plaintext.
• Legacy plaintext — a node_bootstrap.json written before the field existed decodes RaftTLS as false, and that cluster stays plaintext through binary upgrades; flipping it to TLS is a deliberate all-nodes operation, not an upgrade side effect. For those clusters, firewalling :4002 between members remains the only protection.
• File-gated bring-up — if the posture says TLS but the cert material is missing, genesis falls back with a logged warning (X503200RFTTLW) while a joiner refuses to start (X500211RFTTLF) rather than silently joining plaintext.

Short-lived join tokens are minted by the leader (shc node token) and presented by a joining node to bootstrap mTLS. Token generation lives in modules/cluster/token_router.go (the shc node token route) backed by the single-use ledger in modules/cluster/meshtoken_repo.go; the handshake itself is in modules/cluster/enroll_handler.go. Tokens are single-use and expire quickly (default: minutes).

When the optional Nebula mesh is enabled (shc init --nebula or shc cluster mesh enable — it is off by default), cluster traffic that isn’t raft replication flows through a Nebula overlay mesh. Each node holds a Nebula certificate signed by the cluster CA; the mesh provides authenticated, encrypted UDP transport even across hostile networks. Without the mesh, non-raft cluster traffic rides the mTLS/shared-token internal API (:4003) directly on the cluster interface. Ownership: the nebula module.

User-facing secrets (ref+vault://… references in compose templates, app vars marked @secret) are stored via the vault module, which is itself multi-provider: Local (SQLite with a cluster-distributed KEK), AWS Secrets Manager, HashiCorp Vault, Azure Key Vault, GCP Secret Manager, 1Password.

Secrets are resolved lazily at read time by core/config/secrets.go: Config.GetResolved parses a ref+vault://… value via ParseSecretRef and dispatches it through the provider Dispatcher() resolver. They are never committed to the rendered compose file that Docker sees — they are injected directly as environment variables or mounted as tmpfs files at container start.

Every authorization decision that matters emits an audit event:

The full reference lives at errors.md. Audit rows are hash-chained (see audit module) so tampering is detectable.

• Not a Kubernetes / Coolify replacement for hostile multi-tenancy. One trust boundary per node. Tenant isolation is for organizational separation, not adversarial separation.
• Not a full IAM system. Role and policy management happens in Keycloak; SHC only consumes the resulting token. Role design is the operator’s responsibility.
• Not a secret manager. Vault providers are external systems; the vault module is a thin, uniform facade.

• Architecture: vault module
• Architecture: cluster module
• Architecture: node module
• Architecture: audit module
• Error reference — X40* / X50* auth errors
• Config keys — authentication.* and authorization.*