Skip to content
SHC Docs

Origin ingest gate + materialized transpile (2026-07-17)

Status: DESIGN. Part of the platform-compat program — faithful capture of docs/handoff/2026-07-16-hel1-reprovision-and-platform-compat.md (§C3, §D, §E), no new design. Siblings: 2026-07-17-manifest-adapters.md (the per-origin transpilers this gate feeds), 2026-07-17-app-metadata-schema.md (the canonical fields the transpile targets), 2026-07-17-notes-templating.md (the post-deploy notes the adapt stage synthesizes), 2026-07-17-origin-repos-and-migrations.md (the repo surface that runs this pipeline at sync).

Terminology (origin, adapter, adapted app) is the collision-checked set defined in 2026-07-17-manifest-adapters.md.

SHC wants to run apps packaged for other self-hosting origins (Umbrel, CasaOS, Runtipi, Coolify, Cloudron, …) — gated, opt-in per app; native SHC apps untouched. Two failure modes have to be designed out up front:

  1. Runtime shimming. Adapting a foreign manifest in-memory at deploy time produces packages that are unpinnable, undiffable, and unreviewable — and forces the enrichment core to know about origins forever. The transpile must instead be materialized at ingest: the output is a real, on-disk SHC package.
  2. Silent partial coverage. Not every foreign app can be adapted (host networking, device passthrough, cross-app dependency shell). A catalog where ~40% of apps silently break is worse than one that honestly rejects them. Every reject must be recorded with its reason and stay visible (shc repo apps --rejected).

There is also a safety class the gate exists to catch: origin packages assume a private LAN box. Umbrel’s app_proxy is a transparent session-auth gateway and many Umbrel apps carry NO auth of their own (the ecosystem tracks PROXY_AUTH_ADD). Stripping the proxy and publishing on public DNS + LE would mint unauthenticated admin panels with a green padlock. The adapt stage therefore flags forward-auth default-on for every adapted app (see the adapt stage below).

Per app, at ingest, before any transpile output is produced:

RejectWhy
network_mode: hostbreaks tenancy/network isolation; no overlay placement
device passthrough (/dev)host hardware coupling (Home Assistant/USB class)
privilegedunreviewable third-party compose with host root
torOnlySHC exposes via public DNS + LE, not Tor hidden services
dependencies / exports.sh (v1)arbitrary vendor shell sourced into env resolution + cross-stack volume mounts (${APP_BITCOIN_DATA_DIR}) — the exact backup-boundary violation class the boundary campaign purged
>1 HTTP UIa single host_spec cannot represent it; unique-exposer rule
unparseable manifestnothing trustworthy to transpile

Non-HTTP (L4) ports are either covered by an explicit L4 publish policy or are an ingest REJECT (unique-exposer + tenant port allocation) — the policy choice is open (see below).

Rejects are first-class output: each rejected app stays listed in the origin repo with its reject reason. Honest coverage, never a silent 40%.

Per accepted app, the adapter transpile writes a complete SHC package to disk: pinned, diffable, reviewable. Package identity = origin app version + origin catalog git sha. Never adapt in-memory at deploy — installing an adapted app is a plain shc install; the enrichment core, tenancy, ingress, and backup machinery cannot tell it’s adapted.

What shc repo add --origin umbrel <git-url> + every subsequent sync does (the repo-registration surface itself is 2026-07-17-origin-repos-and-migrations.md):

  1. Pin — clone and record the upstream git sha. Nothing floats; every sync is a snapshot.
  2. Enumerate — discover app packages using the origin’s layout (umbrel: dir-per-app with umbrel-app.yml; casaos: compose with x-casaos; runtipi: config.json + compose; cloudron: manifest + Dockerfile).
  3. Gate — per app, the reject taxonomy above. Every reject recorded WITH ITS REASON (shc repo apps --rejected).
  4. Adapt — per accepted app, the transpile:
    • manifest → canonical app.yaml (merit fields + compat: bag — 2026-07-17-app-metadata-schema.md);
    • strip the origin proxy and synthesize host_spec from the proxy service’s APP_HOST/APP_PORT env, NOT the manifest port (the manifest port is where the proxy listens; using it routes to nothing for much of the catalog);
    • bind-mounts → named volumes + seed-file init job (volume seeding is a missing core primitive: origin packages ship seed files copied into ${APP_DATA_DIR} pre-start, but named volumes populate from the IMAGE, not the package);
    • secret contract (APP_SEED/SERVICE_PASSWORD_*@secret mints; APP_SEED is mint-once, IMMUTABLE, non-rotatable — apps persist seed-derived creds at first-init, rotation = lockout the platform can’t see, restore must replay the SAME seed);
    • auto-backup.yaml from the volume set;
    • notes template from default-creds / postInstallMessage (2026-07-17-notes-templating.md);
    • healthcheck synthesis where the origin declares one;
    • forward-auth default-on flag, per-app opt-out requiring ingest-gate evidence of self-auth (the Huly ?token= mangling class means the opt-out mechanism must exist day one).
  5. Materialize — write the complete SHC package to disk (above).
  6. Index — the accepted set is served as a normal SHC repo.
  7. Re-sync — new upstream sha → re-run 2–6 and DIFF the normalized output; changed packages surface as app updates. This is the rot-prevention CI: without pinned snapshots + re-transpile diffing + conformance fixtures per origin, the adapters rot in a quarter (see the steady-state section of 2026-07-17-manifest-adapters.md).
  • Transpile is materialized at ingest — on-disk, pinned, diffable packages; no runtime adaptation layer (“adapter”, not “shim”).
  • Rejects are recorded with reasons and stay visible (shc repo apps --rejected).
  • Forward-auth gate DEFAULT-ON for every adapted app; opt-out is per-app and requires evidence of self-auth. (Reverses the earlier draft where stripping app_proxy had no replacement gate — that default was dangerous as drafted.)
  • dependencies/exports.sh are cut from v1 and rejected at ingest (not executed, not emulated).
  • APP_SEED is immutable: no standard vault-rotation UX on it.
  • host_spec synthesis reads the proxy’s env, never the manifest port.
  • Compat volumes pin to LOCAL (never network-backed) volume providers.
  • L4 ports: explicit publish policy vs plain ingest reject — unresolved (P2P apps: 8333/9735 are common on Umbrel).
  • exports.sh apps: reject forever, or curated reimplementation of high-value dependency-carrying apps later?
  • Quota defaults for quota-less manifests (noisy-neighbor guard): the requirement is decided, the default values are not.
  • Image pinning / provenance policy for unreviewed third-party compose: required by the steady-state analysis, not yet specified.
  • Overlapping subpath mounts need a stated shared-parent-volume rule; UID/ownership first-boot failures are expected and need a documented remediation shape.