Origin repos (foreign catalogs) + migrations to native apps (2026-07-17)

Status: DESIGN. Part of the platform-compat program — faithful capture of docs/handoff/2026-07-16-hel1-reprovision-and-platform-compat.md (§C3, §C4, §C6, §D, §E), no new design. Siblings: 2026-07-17-origin-ingest-gate.md (the gate + pipeline every sync runs), 2026-07-17-manifest-adapters.md (the per-origin adapters + terminology), 2026-07-17-app-metadata-schema.md (the canonical fields + compat: bag), 2026-07-17-notes-templating.md (post-deploy notes).

Terminology (origin, adapter, adapted app, origin repo, import, migration) is the collision-checked set defined in 2026-07-17-manifest-adapters.md.

Two gaps beyond single-app adaptation:

1. Whole catalogs. Adapting apps one at a time doesn’t scale to ecosystems of hundreds of packages. A whole foreign catalog should register as an shc repo and serve its adaptable subset like any other repo — with honest, visible coverage.
2. Adapted is not the destination. An adapted app runs with a dumb backup — volume snapshots that are crash-consistent but conform to NOTHING (no exporter format, no per-app backup contract), second-class for export/import. Users need a path from an adapted app to the official native SHC app(s), data included — and people running a real Umbrel/CasaOS/Runtipi/Cloudron box need a way onto SHC at all.

Register a whole foreign catalog as an shc repo:

shc repo add --origin umbrel https://github.com/getumbrel/umbrel-apps

(same for casaos / runtipi / coolify / cloudron-packages). Semantics:

• Sync = pin + transpile + gate. Each sync pins the upstream to a git sha, runs the ingest gate + ManifestAdapter transpile per app (the 7-stage pipeline in 2026-07-17-origin-ingest-gate.md), and serves the ACCEPTED subset as a normal SHC repo (index.yaml + packages). Rejected apps stay listed with their reject reason (shc repo apps --rejected) — honest coverage, no silent 40%.
• Re-sync is the rot-prevention CI. New upstream sha → re-pin, re-transpile, DIFF the normalized output; changed packages surface as app updates. Conformance fixtures per origin live with the adapter (steady-state requirements in 2026-07-17-manifest-adapters.md).
• Foreign metadata rides the compat: passthrough bag (2026-07-17-app-metadata-schema.md).
• Licensing diligence applies per catalog before serving it beyond private use — a legal decision, not engineering (the Umbrel platform is PolyForm Noncommercial; catalog + gallery screenshots need diligence, given SHC is commercially deployed).

Installing from an origin repo is a plain shc install — the enrichment core, tenancy, ingress, and backup machinery cannot tell the package is adapted.

THE KEY INSIGHT: an adapted app running on SHC has a dumb backup — volume snapshots that are crash-consistent but conform to NOTHING (no exporter format, no per-app backup contract). Migration’s whole job is to transform that dumb backup into conforming SHC backup(s) / a backup GROUP that fit the native SHC app(s) being migrated to — because then the EXISTING restore machinery (restore = deploy-from-backup + data-replay) does the rest. Migration is a data transpile whose output is exactly what restore already consumes. No new deploy path.

A *.migration.yaml (shipped in the catalog next to apps, versioned, testable) declares:

name: runtipi-erpnext
source:
  platform: runtipi              # umbrel | casaos | runtipi | cloudron | coolify
  app: erpnext
  match:
    version: '^(1[45])\.(\d+)\.(\d+)$'   # REGEX w/ capture groups — one file
                                          # covers every tipi ERPNext version
targets:                          # 1..N native SHC apps (splits are normal:
  - app: frappe                   #  bundled-DB foreign app -> app + db app)
    version: '${1}'               # capture substitution — target version is
                                  # DERIVED from source version…
    transform:
      job: migrate-erpnext.job.yaml   # runs with source volumes mounted RO
      produces: exporter              # emits the frappe exporter-format export
  - app: mariadb
    version:
      map:                        # …or an explicit mapping table for
        '14.*': '10.6'            # irregular version relationships
        '15.*': '10.8'
    transform:
      from_volume: db-data
      produces: dump              # logical dump extracted from the snapshot
group: consistency                # one backup GROUP -> all-or-nothing restore

Load-bearing properties:

• Version mapping is regex/range-based, not per-version files (the ERPNext requirement). Capture substitution for regular mappings, map: tables for irregular ones. And it is NOT cosmetic: the target app version must match the DATA format (the same-DB-major restore constraint; frappe major ↔ site schema) — the mapping selects the restore-compatible target.
• 1→N splits produce one conforming backup per target + a consistency group; restore of the group brings up the native apps together (the group-backup + #91 manifest-inventory machinery already exists and validates the output).
• Transforms run as jobs with the source volumes mounted read-only (never mutate the source), producing whatever the target’s backup contract expects (exporter-format export, logical dump, file tree). Dry-run = produce + validate the group without cutover.
• Cutover: stop foreign stack → final dumb backup → transform → restore targets → move hosts → foreign stack kept (protected) until the operator confirms, then destroyed.
• Layout + matching: migrations are defined by the app they migrate INTO — apps/<targetApp>/migrations/<origin>/<sourceApp>.migration.yaml (e.g. apps/postgres/migrations/runtipi/linkwarden.migration.yaml), with the transform *.job.yaml files beside the recipe. The loader DISCOVERS the owning app by globbing the catalog for the (origin, source-app) key; two apps claiming the same source is an error. A multi-target recipe lives under its PRIMARY target (the app the source natively is); auxiliary datastore targets ride in the same file. New apps add migration paths as a drop-in under their own dir — no shc source edit — and recipes ship automatically because app dirs are packaged recursively. (NOT the repo-root migrations/, which is the Atlas SQL schema catalogue.) Match key = (origin, app, version-matcher). Same target app from different origins = separate files (the data layouts differ). NO match (or unmappable version) → the adapted app simply cannot be migrated yet and keeps running as-is; shc app info on an adapted app shows migratable to: <apps> when a path exists. Adapted apps are first-class migration sources regardless of how they arrived (fresh install from an origin repo, or shc origin import off a real box) — identical dumb-backup shape.

For someone actually RUNNING umbrel/casaos/tipi/cloudron on their own box, the journey is two hops, both terminating in the existing restore machinery:

Hop 1 — shc origin import --origin runtipi ssh://root@box [app]

1. The importer reads the box’s app registry + the INSTALLED manifest version (known layouts: umbrel ~/umbrel/app-data/<id>, casaos /DATA/AppData/<id>, tipi app-data/<id> + user-config; CLOUDRON: consume its native per-app backup tarball directly, no SSH).
2. The adapter transpiles that exact manifest version → the adapted SHC package.
3. The importer quiesces the origin app, pulls its data, packages it as a DUMB BACKUP of the adapted app.
4. shc restore deploys the adapted app from that backup.

Import = adapter (package) + importer (data) + existing restore.

Hop 2 — later, optional: run the migration.yaml above (adapted app → conforming backup group → restore as official native SHC app(s)). The migration cannot tell whether the adapted app arrived by fresh install or by import — same input either way. Users may stop at hop 1 indefinitely (the adapted app already gains SSO-gating, per-app backups, LE ingress).

• Sync pins the upstream to a git sha; nothing floats. Rejects are recorded with reasons and stay visible (shc repo apps --rejected).
• The accepted set is served as a normal SHC repo; install from it is a plain shc install.
• Re-sync = re-pin + re-transpile + diff the normalized output; conformance fixtures per origin live with the adapter.
• Migration output = conforming backup(s) + a consistency group, consumed by the EXISTING restore machinery — no new deploy path.
• Version matching is regex/map-based (never per-version files) and selects the restore-compatible target version.
• Transforms mount source volumes read-only; dry-run produces + validates the group without cutover.
• Cutover keeps the foreign stack (protected) until the operator confirms.
• Migrations are app-defined (apps/<targetApp>/migrations/<origin>/<sourceApp>.migration.yaml, shipped with the target app); same source app from different origins = separate files.
• Unmigratable adapted apps keep running as-is; shc app info surfaces migratable to: when a path exists.
• Import = adapter + importer + existing restore; Cloudron’s importer consumes its native per-app backup tarball (no SSH). Hop 2 is optional and may never happen.

• Flag spelling: the draft uses both shc repo add --type <origin> and --origin <origin>; the collision-checked terminology settles on origin as the noun — the flag name follows it here, to be confirmed at implementation. Likewise the example’s source.platform: key predates the terminology ruling.
• Licensing diligence per catalog (legal, not engineering) before an origin repo is served beyond private use.
• Transform-job contract details (what validates a produces: output against the target’s backup contract) ride the existing backup/restore machinery — not re-specified here.