Flow: app install

Step-by-step trace of what happens when an operator runs shc install <app> --stack <name>. Source of truth: modules/app/install_async.go’s RunInstallWithObserver (orchestrating modules/app/install_runner.go); the numbered steps below correspond to the code there.

This page covers the success path and the rollback path. For the lifecycle-hook contract, see docs/app-development/lifecycle-hooks.md. For upgrade / uninstall flows, see upgrade.md and uninstall.md.

If a stack_name is supplied, validate it is lowercase-alphanumeric with hyphens. Invalid names raise X400069INVNME immediately — nothing else happens.

Resolve app_name to a row in AppModel:

1. Split parent/child syntax if present (used by VApps).
2. Scope the search to repos visible to the current tenant + environment:
   • the caller’s own tenant_name or the system tenant
   • the caller’s environment or the system environment
3. Optionally filter by repo_name (--repo flag) and app_version (--version flag).
4. Order by repo priority ascending, version descending.
5. First row wins; no match ⇒ X404002APPNOF.

This is how SHC decides which app named foo to install when multiple repos contain one — tenant-scoped repos override the system repos by repo priority.

If the resolved app has is_vapp=True, delegate to service._install_vapp(...). VApps don’t create their own stack — they extend another running stack (e.g. the acme VApp adds TLS to a running Traefik). See docs/app-development/vapps.md and the app module for the VApp model.

For the rest of this flow, assume a regular app.

service.get_stack(name, environment) — if a stack with this name already exists in this environment, raise X409003STKEXS. SHC does not auto-increment or replace.

Two directories are created via fsops.Mkdir(...):

• paths.deployment_dir(tenant, name, environment) — operational state for this deployment (rendered compose, vars.yaml, logs).
• paths.app_dir(tenant, name, environment) — the app’s source files, copied from the app repo.

App sources are resolved via GetAppSourcePath(db, app) (handles git / path / archive repos — see repo module) and copied in with CopyAppFiles.

LoadAppDefinition(appDir) parses the app’s app.yaml into a typed AppDefinition — the schema, hooks, platform constraints, integration declarations, and vars.

If the caller pinned a node (--node), verify it satisfies the app’s platform constraints (OS, arch, required features). Raises if not.

MergeVariables(appDef, userVars, integrations) combines:

1. Default var values from app.yaml
2. User-supplied --var key=value overrides
3. Values derived from declared integrations (plugs ⇄ sockets)

This is a declarative merge — the result is a flat map[string]string; no rendering yet.

PreRenderCompose(...) renders the compose file with the merged variables just enough to extract resource hints. This is needed so the quota / capacity checks below see the real numbers (CPU, memory) the app will ask Docker for.

ExtractComposeResources reads cpu / memory reservations out of the pre-rendered compose, plus disk from the --disk flag, and EnforceStackCreationQuota either allows the install to proceed or raises a quota error. See the quota module.

If runtime=swarm, do three additional checks before any DB write:

1. Pool capacity: sum of cpu/memory across all replicas fits the total free capacity of the swarm.
2. Largest replica fits some node: not just “fits in total” but “can actually be placed somewhere.”
3. Live bin-pack: CheckSwarmCanPlaceReplicas talks to the swarm and runs a real placement preflight.

Any failure raises X403014SWMFRG with a free-capacity report so the operator sees why.

If the install is pinned to a specific node (--node), run a compose-runtime capacity check (EnforceNodeCapacity). Swarm uses its own scheduler for this; this is only for compose.

service.resolveStorage(...) picks the storage provider (local, NFS, a cluster-aware provider) based on: --storage-provider flag, app.yaml hints, pinning, and runtime. See the storage module.

A DeploymentModel struct is constructed with status=CREATED and every resolved field (app name/version, runtime, storage provider, HA flag, scale overrides, deployment fingerprint). Not yet committed.

Six boolean flags are declared — stackRowCommitted, ingressAllocated, hostModePortsAllocated, composeRendered, composeStarted, finalizeCompleted. Each flag corresponds to a piece of externally-visible state that could leak on a mid-flight failure. The error-handling block in §18 uses them to know what needs undoing.

From here everything runs with deferred rollback that triggers on any error.

db.Create(&stack)

stackRowCommitted = true. Now the cluster can see “an install is in flight” even if the process crashes.

The merged variables are written to deployment_dir/vars.yaml. This is what subsequent upgrades and re-renders read.

If the user passed --host foo.example.com / --port 443/tcp, service.allocateIngress reserves the host/port in the ingress bookkeeping table and configures DNS records. Sets ingressAllocated = true.

For services using network_mode: host, reserve ports against the owning node so no two stacks collide. Sets hostModePortsAllocated = true.

If --integrate <other-stack> was passed, CreateIntegrations wires the plug/socket connections into persistent state so that later renders can resolve integration+…:// references.

ValidateRequiredPlugs(appDef, integrations, mergedVariables) — if app.yaml declares a plug as required: true and no integration or inline var satisfied it, raise. This is the “you can’t install Keycloak without a Postgres somewhere” check.

Two passes with the real variables now:

• renderComposeFile — writes the final compose.yaml (or swarm stack file) that Docker will consume.
• renderYAMLFiles — writes any other templated YAML files the app declares (e.g. *.job.yaml, *.exporter.yaml).

Sets composeRendered = true.

If the stack is owned by another node, syncDeploymentToRemote ships the rendered files over the internal API to the owning node before the next step tries to run docker there.

User-defined pre_install hook (shell job inside a throwaway container, or host command — see lifecycle hooks). A non-zero exit rolls the whole install back.

EnsureNetworksExist creates the per-tenant overlay network and per-stack private network if they don’t already exist (idempotent — see network module).

Three cases:

1. If LoadComposeData returned a parsed struct, use ComposeUp(...) which goes through the structured compose pipeline.
2. If the app dir has a compose file but no parsed data (edge case), fall back to composeCommand(name, env, "up", "-d", …).
3. If the app has no compose file at all (job-only apps), treat it as success with no-op.

On failure ⇒ X500907STKFAL with compose exit code + stderr + stdout embedded.

On success ⇒ composeStarted = true.

If runtime=swarm, poll until replicas converge (or timeout). This catches swarm-mode deploys where ComposeUp returns success but actual containers never reach running.

finalizeSuccessfulInstall:

• Runs the post_install hook (unless skipPostInstall=true).
• Registers the stack’s jobs with the scheduler.
• Installs any cron schedules declared in app.yaml.
• Updates tenant usage counters.
• Publishes the A200100STKINS audit event.

Sets finalizeCompleted = true.

Final db.Save() persists any state changes from finalize, then return the DeploymentModel.

The deferred rollback handler catches any error — including programming errors. The rollback runs in reverse order of forward progress so we don’t, say, delete the stack row before the ingress release has a chance to look up the allocations.

Key property: every cleanup step is individually guarded. A failure in one rollback sub-step is logged (X200003OPFAIL with an operation tag) and does not prevent later rollback steps from running. The original install error is returned at the end.

1. Dump container logs (if composeStarted) — so the operator can see why the container failed, even after the stack is gone.
2. Usage counter decrement (if finalizeCompleted).
3. Unregister jobs (if finalizeCompleted).
4. Delete schedules (if finalizeCompleted).
5. Release ingress (if ingressAllocated or hostModePortsAllocated). Queried from the DB rather than from the flags, because allocateIngress may have persisted rows before returning an error. Must run before the stack row delete — CASCADE FKs would otherwise wipe the allocation rows.
6. rollbackFailedInstall (the shared helper): compose down, network removal, integration cleanup, stack row delete.
7. Deployment / app dir cleanup (if composeRendered or stackRowCommitted): remove any rendered compose / copied sources the shared helper missed.

If noRollback=true was passed (--no-rollback), skip steps 2-7 — the operator has chosen to leave the mess for manual inspection. This is rarely used outside of debugging.

• pre_install hook side-effects: if the hook already ran a migration against an external database, SHC does not know about it.
• External DNS propagation: the ingress release asks the DNS provider to remove the record, but TTLs are what they are.
• Vault secrets: secrets written during pre_install stay. Callers that need rollback-safe secret allocation should write under the stack’s vault namespace (which is deleted by the shared helper).

Full reference: errors.md.

• Architecture: app module — owns this pipeline
• Architecture: quota module — step 10
• Architecture: storage module — step 13
• Architecture: network module — step 16.10
• Lifecycle hooks — steps 16.9 / 16.13
• Plugs & sockets — steps 8, 16.5, 16.6