Migrating from another platform
Apps already running on another self-hosting platform come over two ways. An adapter loads one as it is: it reads the platform’s own files for the app, writes an SHC app from them and pulls its data into a backup. A migration moves the data into a native SHC app instead, through a recipe the native app ships for that platform.
$ shc adapter...$ shc adapter ls...An old box
Section titled “An old box”Any machine with Docker can play the old box. Here home01 does, so the pull
stays on one machine. Dockge keeps each stack as a compose file in its own
directory under /opt/stacks, with the stack’s data beside it. Write one
called jotter, with a line already in its log.
services: writer: image: busybox:1.36 command: ["sh", "-c", "while true; do date -u >> /data/log; sleep 30; done"] volumes: - ./data:/data restart: unless-stoppedwritten on the old box$ docker compose -f /opt/stacks/jotter/compose.yaml up -d...The adapter reaches a box over SSH, the way you would. Give it a key of its own and let that key into the box.
$ ssh-keygen -t ed25519 -N "" -q -f ~/.ssh/learn...$ cat ~/.ssh/learn.pub >> ~/.ssh/authorized_keys...$ ssh-keyscan -t ed25519 localhost > ~/.ssh/learn_hosts...Load the stack as it is
Section titled “Load the stack as it is”--dry-run reads the box and writes the SHC app under --dest, and pulls
no data. --known-hosts checks the box’s host key against the file
ssh-keyscan wrote.
$ shc adapter load dockge ssh://root@localhost jotter --identity ~/.ssh/learn --known-hosts ~/.ssh/learn_hosts --dest ~/learn/adapted --dry-run...$ ls -1 ~/learn/adapted/jotter...$ cat ~/learn/adapted/jotter/compose.yaml..../data became a named volume, data. An importer may also write keys an
app author does not: x-shc-http-ui marks the service that serves the app’s
own web page. Read it on a small import of your own.
name: importedversion: "1.0.0"title: Importeddescription: The keys an importer writes, validated and never installedcategory: demolicense: id: MITservices: web: image: nginx:1.27-alpine expose: - "80" x-shc-http-ui: true x-shc-scale: max: 1$ shc validate ~/learn/imported...Now pull the data too. The adapter
stops the app with Dockge’s own docker compose stop, copies the volume,
starts it again, and assembles a backup the restore reads. --backup-dir
says where. Every load writes <app>.import.sh beside the app: the install
and the restore, in the order they have to run.
$ shc adapter load dockge ssh://root@localhost jotter --identity ~/.ssh/learn --insecure-host-key --dest ~/learn/adapted --backup-dir ~/learn/adapted/backups...$ cat ~/learn/adapted/jotter.import.sh...$ sh ~/learn/adapted/jotter.import.sh...$ shc exec -s jotter -S writer -- head -n 1 /data/logwritten on the old box--insecure-host-key skips the host key check: for a lab, never a box you do
not own. --no-quiesce copies the volume while the stack runs, and
--no-assemble leaves one tar per volume instead of a backup.
$ shc adapter load dockge ssh://root@localhost jotter --identity ~/.ssh/learn --insecure-host-key --dest ~/learn/adapted-live --no-quiesce --no-assemble...$ ls -1 ~/learn/adapted-live/backups...$ shc adapter load dockge ssh://root@oldbox jotter --password P@ssw0rd --root /srv/stacksA platform with a public app store is a repository too: --adapter makes
shc repo sync read that store through the adapter.
$ shc repo add casaos https://github.com/IceWhaleTech/CasaOS-AppStore/archive/refs/heads/main.zip --adapter casaosA migration recipe
Section titled “A migration recipe”The loaded jotter is an adapted app: the stack’s compose file, kept as the
platform wrote it. notebook already does what jotter does, so the data can
move into notebook instead. A recipe says how. It lives in the native app,
migrations/<platform>.migration.yaml, the platform in the file’s name.
name: dockge-jottersource: app: jotter match: version: '^1\.'targets: - app: notebook version: "1.0.0" transform: from_volume: data produces: files artifact: notesgroup: consistencysourceis the app on the old platform and the versions of it the recipe takes, a regular expression on the version you give.targetsare the native apps its data becomes. One stack can become several, an app and its database.from_volumetakes the files of one of the source’s volumes as they are; ajobinstead runs a container over them, for a database dump.artifactis what the files become in the target’s backup. Withproduces: filesit is one of the target’s volumes, notebook’snotes, andshc recoverwrites the files into it before notebook starts. They take the owner notebook declares fornotes, 65534, not root’s from the old box, sonotescan read them, and the volume’s own directory takes its declared mode,0750. Adumpis handed to the target’s restore jobs instead.groupnames the set of backups the run assembles, restored all or none.
--apps-root is the directory that holds the target apps. --dry-run finds
the recipe, plans each target and runs nothing.
$ shc adapter migrate jotter --adapter dockge --version 1.0.0 --from ssh://root@localhost --identity ~/.ssh/learn --known-hosts ~/.ssh/learn_hosts --apps-root ~/learn --dry-run...Without --dry-run the migration copies the source volume into
--work-dir, runs each transform over it and writes the group to --out.
$ shc adapter migrate jotter --adapter dockge --version 1.0.0 --from ssh://root@localhost --identity ~/.ssh/learn --insecure-host-key --apps-root ~/learn --work-dir ~/learn/migrate-jotter --out ~/learn/jotter.bkg...$ shc backup inspect ~/learn/jotter.bkg...A recovered group installs each target by name, so notebook has to be in a catalogue. Publish it in yours, recipe and all.
$ tar -czf ~/learn/shelf/files/notebook-1.0.0.app -C ~/learn/notebook app.yaml notes.conf shc.yaml backup.yaml notes.exporter.yaml migrations...api_version: v2apps: - name: guestbook description: A page, an API under its path and a counter category: demo tags: [demo, redis, nginx] homepage: https://example.com/guestbook versions: - version: "1.3.1" archive: guestbook-1.3.1.app - version: "1.3.0" archive: guestbook-1.3.0.app - version: "1.2.1" archive: guestbook-1.2.1.app - version: "1.2.0" archive: guestbook-1.2.0.app - name: notebook description: Writes a line every 30 seconds and keeps them category: demo tags: [demo] versions: - version: "1.0.0" archive: notebook-1.0.0.app$ shc update shelf...$ shc repo sync mine...notebook already runs here, so recover the group under another name. Its log then starts with jotter’s first line.
$ shc recover -f ~/learn/jotter.bkg --rename notebook=jotted --dry-run...$ shc recover -f ~/learn/jotter.bkg --rename notebook=jotted --yes...$ shc exec -s jotted -S notes -- head -n 1 /srv/notes/logwritten on the old box$ shc exec -s jotted -S notes -- ls -ln /srv/notes...From the adapted app
Section titled “From the adapted app”The data does not have to come from the old box again. --stack takes it
from the adapted app SHC already runs: it pauses jotter, copies its
volumes, and starts it again. It reads the volumes on the node it runs on,
so it runs on the node that holds jotter, with the recipe there too; on
this cluster of one that is home01. --emit-only leaves each target’s backup
loose and prints the shc recover line that would take them.
$ shc adapter migrate jotter --adapter dockge --version 1.0.0 --stack jotter --apps-root ~/learn --work-dir ~/learn/migrate-stack --emit-only...$ shc adapter migrate jotter --adapter dockge --version 1.0.0 --stack jotter --apps-root ~/learn --work-dir ~/learn/migrate-live --no-quiesce --emit-only...Some platforms name an app on the box differently from its catalogue name:
Portainer by container, Coolify by a resource id. --source-app and
--box-app give that name, and --password stands in for a key.
$ shc adapter migrate wordpress --adapter portainer --version 6.5 --from ssh://root@oldbox --source-app wpstack-wordpress-1 --password P@ssw0rd$ shc adapter migrate gitea --adapter coolify --version 1.22 --from ssh://root@oldbox --box-app k4c8s0wClean up
Section titled “Clean up”$ shc uninstall jotted --yes...$ shc uninstall jotter --yes...$ docker compose -f /opt/stacks/jotter/compose.yaml down...Every page so far ran on home01 alone. The next one needs more nodes.
Next: Groups and HA.