Splitting the sso socket into oidc and saml
Status: LANDED. (Corrected 2026-08-13. This line read “PROPOSED, awaiting owner ruling. Design only; no socket, plug, or estate was changed in the lane that produced this document” for eight days after the split shipped, was merged and was deployed. The recommendation below was accepted and executed; see §0.)
Date: 2026-08-05. Measured against shc origin/main @ 07ace7b62 and ce
origin/main @ 1053a03, both in fresh worktrees.
0. Outcome: accepted and shipped
Section titled “0. Outcome: accepted and shipped”Measured on shc and ce main 2026-08-13, not recalled:
| claim | state |
|---|---|
ce/builtin/keycloak/ | serves oidc.socket.yaml, saml.socket.yaml and keycloakAdmin.socket.yaml |
sso.socket.yaml anywhere | 0; none survives in ce/builtin/ or under shc/apps/ |
oidc.plug.yaml | 43: 41 under shc/apps/, 2 in ce/builtin/ (traefik, openobserve) |
sso.plug.yaml | 0 |
saml.plug.yaml | 2: apps/flagship/mautic/, apps/flagship/sonarqube/ |
the declared socket: key | wired, and guarded by tests/unit/catalog/plug_socket_agreement_test.go |
| the state migration of §7 | shipped as ce modules/app/oidc_plug_migration.go |
Landing commits in ce: 1c8aba6 (“feat(app)!: the identity socket is
oidc …”, which deleted sso.socket.yaml), 9ed5af5 (“feat(keycloak)!:
keycloak serves a saml socket…”), 96e7354 (the fail-closed X400023HKNOID
identity guard), 7a96731.
One thing §1 anticipated has now landed too (2026-08-14). Fleet, the
third SAML consumer, and the one with no OIDC path at any 4.x pin, was
packaged on the unmerged branch feat/fleet-saml-sso (be2544a4) against
apps/security/fleet/, a path main had retired. The branch’s content was
re-landed by hand against the bucketed tree: apps/integrated/fleet/
(moved out of standalone/ in the same commit, because an app with a
working SSO plug is not “no SSO”) now ships saml.plug.yaml, and fleet is
recorded as an UNPROVEN consumer in
tests/unit/catalog/plug_socket_agreement_test.go’s
samlConsumersAwaitingLoginProof until a real browser round-trip lands.
docs/evidence/fleet-sso/00-verdict.md remains the record of the
pre-package state.
The paired record in the huly repo is docs/socket-grammar-split.md, whose
own status line carried the same staleness and was corrected on 2026-08-11.
1. Recommendation
Section titled “1. Recommendation”Do the split. It is not a cosmetic honesty fix; it closes a live, silently-failing integration in the catalogue today, and it is measurably CHEAPER and SAFER than the rename-only alternative.
Four findings drive that, in order of weight:
-
One catalogue app already needs SAML and is silently broken.
apps/mauticdeclares ansso.plug.yamlwhosepost_deploymentjob configures Mautic as a SAML SP (saml_idp_*inlocal.php, entity ID fromintegration.plug.data.clientId, IdP descriptor fetched from/realms/<realm>/protocol/saml/descriptor), while the socket it plugs into mints a client with"protocol":"openid-connect"(ce builtin/keycloak/sso.socket.yaml:161,163). Keycloak’sSamlServicerejects an AuthnRequest whose client protocol is notsaml(“Wrong client protocol”). Both jobs print success.apps/mautic/app.yaml:37documents the dishonesty in its own words:clientSecret@secret: SAML client secret (unused by SAML but kept for parity with OIDC plugs). The only test (apps/mautic/tests/tests.bats:547) asserts the job EXISTS and thatsaml_idp_metadataappears in its command, a guard that is green while the thing it guards cannot work. So the answer to “does any catalogue app need SAML today?” is YES, one, and it is a live defect, not a hypothetical. -
The split’s two-socket shape is what makes the migration survivable.
findSocketForPlug(ce modules/app/integration_render.go:1172) falls back to the provider’s sole*.socket.yamlwhen the plug name misses. A rename-only change leaves keycloak with one socket, so a stale persistedplug: "sso"row still resolves a socket and drivescreate-oidc-clientwith an empty realm and empty clientId, the exact/admin/realms//clients?clientId=→ 404 → exit-22 rollback recorded atintegration_render.go:442-447. With TWO sockets the fallback cannot fire, so the stale edge degrades to a no-op instead of a hard failure. The split is the less destructive of the two changes. -
The mechanism that makes a mis-plugged app fail VISIBLY already exists in the grammar, is documented, is authored in four files, and is INERT.
docs/app-development/plugs-and-sockets.md:154documents a plug’ssocket:field;apps/matrix/postgres.plug.yaml,apps/matrix/postgresMas.plug.yaml,apps/postiz/postgres.plug.yamlandapps/postiz/postgresTemporal.plug.yamlall declare it. Nothing on the install path reads it;diskPlugSpecs(ce modules/app/integration_disk_register.go:30) hardcodesSocket: stem, andResolveIntegrationsnever opens the key. Without wiring it, asaml.plug.yamlpointed at a single-socket OIDC provider (e.g.apps/external-oidc) silently binds the OIDC socket via the same sole-socket fallback, reproducing the very false-success the split exists to remove. Wiring the declaredsocket:key is a REQUIRED companion, not an optional extra. Designed in §9, priced in §11. -
The July 2026 precedent is a warning, not a template.
shc cc21767c2(“feat(apps)!: capability-named sockets: keycloak→sso, mail→smtp/imap (clean break)”) did this exact rename with no data migration, and its own commit body says so: “their persisted integration edges store plug=keycloak/mail; every such app must be REDEPLOYED to rewrite plug=sso/smtp edges.” That worked because nothing then read the plug key for teardown andedge_statesdid not exist. Both are now false (ce 988d4c2). The same recipe today leaves credentials un-revoked on uninstall (§5).
Recommended sequencing: land the sibling credential-scope fix FIRST, the split SECOND, in the SAME release window: one estate redeploy wave, two attributable commits. Reasoning in §10. This refines the owner’s prior (“one socket-contract break instead of two”): agree on the window, disagree on one commit.
2. Testing the case as stated
Section titled “2. Testing the case as stated”| Claim | Verdict | Evidence |
|---|---|---|
"protocol":"openid-connect" hardcoded in both client-creation bodies | TRUE | ce builtin/keycloak/sso.socket.yaml:161 (secret branch) and :163 (mint-and-return branch) |
the word saml appears nowhere in the socket | TRUE | grep -i saml over ce matches one unrelated line, builtin/openobserve/app.yaml:7 |
plug schema is pure OIDC (clientId, clientSecret, redirect + post-logout URIs) | TRUE with one correction | schema is realm/clientId/clientSecret/redirectUris (:12-16). Post-logout is not a plug field; it is DERIVED in the job by sed-ing the origin off redirectUris (:158) |
the name over-promises: an sso plug can expect any SSO protocol and silently gets OIDC or nothing | TRUE, and worse than “or nothing” | apps/mautic gets a perfectly-created OIDC client it cannot use, and reports success. See §1, finding 1 |
| a SAML client needs a different shape entirely | TRUE | §6 |
| certificate exchange rather than a shared secret | TRUE, with an important simplification | the IdP half needs no socket-published certificate: Keycloak serves /realms/<realm>/protocol/saml/descriptor unauthenticated, which is what apps/mautic/sso.plug.yaml:46 already fetches. Only the SP→IdP direction needs a real cert push, and only when saml.client.signature is on. See §6 |
| splitting makes a SAML-only app fail visibly | TRUE ONLY IF the declared socket: key is wired | §1 finding 3, §9 |
the two live estates carry sso edges right now | TRUE | §5.1 |
3. Measured reference counts
Section titled “3. Measured reference counts”Counts are of structural references (files to rename, template namespaces
to rewrite, code identifiers to change), not every prose mention of “SSO”.
Raw \bsso\b hit counts for scale: shc 897, ce 452.
shc (origin/main @ 07ace7b62)
Section titled “shc (origin/main @ 07ace7b62)”| Surface | Count | Note |
|---|---|---|
sso.plug.yaml files under apps/ | 32 | the app list is below |
sso.socket.yaml files | 1 | apps/external-oidc/sso.socket.yaml |
apps/*/compose.yaml reading integrations.sso.* | 23 | |
apps/*/sso.plug.yaml reading integrations.sso.* | 18 | subset of the 32 renames |
apps/*/vars.yaml naming the socket | 1 | apps/mailcow/vars.yaml |
apps/*/tests/tests.bats naming the socket/plug | 39 | includes 7 apps with no sso.plug.yaml (prose/negative assertions) |
tests/unit/** files | 30 | |
tests/builtin/** files | 3 | |
tests/proxmox/test_multinode.bats | 1 | ★ deliberately exercises the -i keycloak=… fallback spelling at :3687 |
terraform/provider/** files | 5 | fixtures + comments only; no hardcoded socket name in provider logic |
docs/** + CHANGELOG.md + HANDOFF.md | 20 |
The 32 consumers: affine blockscout defectdojo-classic dependency-track frappe gitea gitlab highlight huly immich ivatar journiv kutt librechat mailcow matomo matrix mautic minio minio-classic monica netbird nextcloud paperless peertube penpot postiz supabase vaultwarden wazuh windmill wordpress.
Plug-file cardinality matters for §5: 30 of 32 ship MULTIPLE *.plug.yaml
(so findPlugForName’s sole-plug fallback cannot rescue a stale row). Only
minio and minio-classic ship exactly one.
Only 8 of 32 carry a consumer-side jobs.unintegration: mailcow (2 jobs),
wordpress, windmill, nextcloud, mautic, immich, gitea,
defectdojo-classic. The other 24 have none, which is why the teardown
failure in §5 is silent rather than reported.
Only 3 of 32 actively consume the master-realm admin credentials into their
own jobs: netbird, mailcow, nextcloud (adminUser/adminPassword). The
other 29 receive them in the render context regardless, the sibling lane’s
subject.
ce (origin/main @ 1053a03)
Section titled “ce (origin/main @ 1053a03)”| Surface | Count | Note |
|---|---|---|
sso.socket.yaml | 1 | builtin/keycloak/sso.socket.yaml, the one to SPLIT |
sso.plug.yaml | 2 | builtin/traefik/, builtin/openobserve/ |
templates reading integrations.sso.* | 3 | builtin/traefik/compose.yaml, builtin/openobserve/compose.yaml, builtin/openobserve/sso.plug.yaml |
| real Go identifier changes | 4 | see below |
SocketSSO reference sites | 8 | |
Go *_test.go files with a structural "sso" | 10 |
The four real Go identifier changes; everything else in ce is a comment:
| Site | Change |
|---|---|
modules/adapter/adapterplug.go:36 SocketSSO = "sso" | → SocketOIDC = "oidc" + new SocketSAML = "saml". The cloudron "oidc" addon (adapterplug.go:213) then maps "oidc" → SocketOIDC, an identity instead of a translation. |
modules/app/integration_render.go:1310 keycloakPlugName = "sso" | → "oidc". Two call sites (:504, unintegration_runner.go:354) gate InjectDerivedKeycloakIssuer, which is OIDC-specific and must NOT fire for saml; a SAML SP needs a descriptor URL, not an OIDC issuer. |
modules/app/openobserve_basepath_reconcile.go:208 in.Plug == "sso" | → "oidc" |
modules/app/integration_service.go:164 + modules/app/sso_edge_gate.go:41 | ★ licensing. Both gate the identity-edge product hook on == adapter.SocketSSO. They must become `SocketOIDC |
★ Two identifiers that look like the socket name and MUST NOT change
Section titled “★ Two identifiers that look like the socket name and MUST NOT change”| Identifier | Why |
|---|---|
ce modules/app/sso_redirect.go:68 defaultSSOBypassQuery = "sso" | the operator-facing ?sso=false URL query parameter on the auto-redirect escape hatch. Renaming it changes a documented public URL and can lock a local admin out of a password form. |
shc modules/license/license.go:90 FeatureSSO = "sso" | a licence entitlement key, baked into every minted licence and verified in-process by the Keycloak SPI’s embedded Ed25519 path. Renaming it invalidates every issued licence. |
A blind sso → oidc sweep breaks both. Any implementation must exclude them
explicitly, with a fitness test that asserts the exclusion.
4. Correction to the brief: the sidecar is manifest.json, not state.json
Section titled “4. Correction to the brief: the sidecar is manifest.json, not state.json”ce modules/app/manifest.go:18-27; ADR-0002 collapsed the two on-disk files
into manifest.json, and the note is emphatic that the pre-T17
state.json is “GONE, not read, not pruned, not named”, verified by
find /var/lib/shc -name state.json returning empty on all six nodes of both
live estates. Path:
<stateHome>/deployments/<tenant>/<stack>/<environment>/manifest.json # 0600, atomic temp+renameSeveral ce comments still say “state.json” (e.g.
daemon/services_recovery.go:395); those are stale prose, not a second file.
The migration in §7 targets manifest.json.
5. ★ What persisted state breaks: the crux
Section titled “5. ★ What persisted state breaks: the crux”5.1 What is live today
Section titled “5.1 What is live today”Established from the estate repos’ authoritative Terraform, not a live read (an
SSH probe was attempted with
bitspur-ops/hetzner/hel1/infra/.ssh/cluster against 37.27.215.210 and was
refused for every candidate user; no further probing was attempted, per the
read-only rule).
| Estate | Tenant-app sso edges | System-app sso edges |
|---|---|---|
| bitspur (hel1) | corp_affine, corp_huly; both integrate.sso = { stack = "KEYCLOAK" } | traefik, openobserve |
| promanager (hel1) | corp_affine, corp_huly; same | traefik, openobserve |
8 live edges. The four system-app edges are not in Terraform: they come from
ce modules/app/system_apps.go:86,88, which hardcodes
integrations: []string{"keycloak"}, the bare provider-ref form. That form
is rename-proof (inferPlugFromProvider re-derives the plug from the
provider’s socket stems against the consumer’s plug stems on every install), so
no Go string there changes. Their persisted rows, however, are not: R1a
resolves the bare token to a plug before it becomes state
(install_async.go:663), so both sidecars record plug: "sso".
5.2 The three keys, and how they diverge
Section titled “5.2 The three keys, and how they diverge”| Record | Keyed by | Today’s value |
|---|---|---|
manifest.json → integrations[].plug | the operator’s raw -i token, after R1a plug resolution | "sso" |
manifest.json → edge_states map key | the RESOLVED plug name (ce modules/app/edge_state.go:98) | "sso" |
manifest.json → edge_states[k].socket_name | the RESOLVED socket name (edge_state.go:66) | "sso" |
integration table → consumer_plug_name, provider_socket_name | resolved names | "sso", but the table is empty in production: RegisterStack has no install-path writer, and the only production Connect caller is the recover path’s WireIntegrations (integration_disk_register.go:13-18) |
backup archive manifest → IntegrationRecord.ConsumerPlugName / .ProviderSocketName | resolved names | "sso", immutable; cannot be migrated (§7.4) |
resolveTeardownPlugName (unintegration_runner.go:257) bridges the first two
by re-resolving through findPlugForName against the deployed app tree.
5.3 Impact table: traced, not estimated
Section titled “5.3 Impact table: traced, not estimated”consumerAppDir on the install path is the catalogue app dir
(install_runner.go:652 passes appDir), while the teardown and socket-job
paths read the deployed trees. That asymmetry is what produces the
different outcomes below.
| Path | Rename only (keycloak keeps ONE socket) | Rename + SPLIT (keycloak ships oidc + saml) |
|---|---|---|
shc update <consumer>, 30 multi-plug apps | ★ HARD FAIL, rollback. findSocketForPlug(kc,"sso") → sole-socket fallback → oidc; findPlugForName(catalogue,"sso") → miss (>1 plug) → PlugData empty; RunSocketIntegrationJobs stats oidc.socket.yaml, hits, runs create-oidc-client with REALM=""/CLIENT_ID="" → the documented /admin/realms//clients?clientId= 404 → exit-22 rollback | Silent no-op for the stale edge. findSocketForPlug returns "","" (2 sockets, no exact match) so ic.SocketName is never set; socketBase falls back to "sso"; stat sso.socket.yaml misses → job skipped. The deploy succeeds; the freshly-resolved oidc edge (from the restated -i or the bare system-app form) does the real work |
shc update, minio / minio-classic (single-plug) | Self-heals. sole-plug fallback resolves oidc, ic.Plug normalises, plug data renders | same; self-heals |
shc uninstall <consumer>, 24 apps with NO plug-side unintegration | Survives: es.SocketName="sso" → sole-socket fallback → oidc.socket.yaml → delete-oidc-client runs against the FROZEN realm/clientId from edge_states["sso"] | ★★ SILENT CREDENTIAL LEAK. findSocketForPlug(kc,"sso") → "","" → socketJobs empty; findPlugForName on the deployed tree hits the old sso.plug.yaml but it declares no jobs → plugJobs empty → unintegration_runner.go:390 if len(plugJobs)==0 && len(socketJobs)==0 { return }, an unconditional early return with no leak report at all. The OIDC client and its secret stay live in Keycloak forever |
shc uninstall, 8 apps WITH plug-side unintegration | Survives, as above | Plug jobs run (consumer stops using the credential); the provider-side delete-oidc-client never runs and nothing is reported; socketJobs is empty, not failed, so no X500927UNILEK and no retained_cleanup frame |
-i sso! disconnect leg | Matches (classification is by syntax, install_async.go:666; the live set still says sso); the row and sidecar entry drop. jobs.unintegration does not run, a pre-existing documented gap (DisconnectRemovedIntegrations, job 3 of 4), so the Keycloak client leaks either way. Unchanged by the split | same |
integration table row | Empty in production; a Connect after the rename fails X404007PLGNOF (the plug lookup derives from the deployed tree) | same. Cheap to migrate anyway (§7.3) |
| Recover from a PRE-rename backup | WireIntegrations per-edge failure is a warning, not fatal (services_recovery.go:391-398), and its own message says “shc update reconverges it”, but that update is the HARD FAIL above. Recover reports success with warnings and leaves an unfixable stack | The stale replayed spec becomes a silent no-op; the bare-<provider> candidate from plan.Requires (services_recovery.go:182) re-derives the correct oidc plug when keycloak is a plan-group member. For a magic-token cross-tenant KEYCLOAK (both estates’ shape) keycloak is NOT in the group, so the operator must supply --skip-integration sso and re-wire. Documented remedy, not a shim |
| Vault-held client secret | Unaffected. The ref is ref+vault://keycloak_client_secret?autogen, keyed off the vars.keycloak.* path, not the plug name | same |
| Running containers | Unaffected. The deployed compose is already rendered to literal values; a plug rename is invisible until the next render | same |
The two headline outcomes. A rename-only change is a fleet-wide update outage (loud). A rename+split is a silent credential leak on uninstall (worse, because nothing tells you). Both are closed by the same one-shot data migration, and with the migration in place neither occurs.
5.4 Two residual holes the split does not create but does expose
Section titled “5.4 Two residual holes the split does not create but does expose”-i keycloak=<stack>dies. That spelling works only throughfindSocketForPlug’s sole-socket fallback on the keycloak app (integration_render.go:1201says so verbatim). Two sockets kills it. It is documented (docs/reference/cli/app.md:264,273), authored in ~10 app header comments, and deliberately exercised bytests/proxmox/test_multinode.bats:3687(“Deliberately the fallback spelling”). It must be removed from docs, comments and that test in the same commit, and the CHANGELOG must name it as a break.inferPlugFromProviderpicks byos.ReadDirorder. With keycloak shipping both sockets, a consumer shipping both anoidcand asamlplug getsoidcby alphabetical accident (integration_render.go:150-181). No app does this today. Worth a comment; not worth a mechanism.
6. Minimum viable SAML field set
Section titled “6. Minimum viable SAML field set”Confirmed against the Keycloak 26.7 admin REST API (builtin/keycloak/vars.yaml:1
pins upstream 26.7.0 + the SHC layers): POST /admin/realms/{realm}/clients
with a ClientRepresentation.
The socket’s create-client body
Section titled “The socket’s create-client body”{ "clientId": "<entityId>", "enabled": true, "protocol": "saml", "frontchannelLogout": true, "redirectUris": ["<acsOrigin>/*"], "attributes": { "saml_assertion_consumer_url_post": "<acsUrl>", "saml_single_logout_service_url_post": "<sloUrl>", "saml_name_id_format": "email", "saml_force_name_id_format": "false", "saml.server.signature": "true", "saml.assertion.signature": "true", "saml.signature.algorithm": "RSA_SHA256", "saml.client.signature": "false", "saml.encrypt": "false", "saml.authnstatement": "true" }}★ saml.client.signature MUST be written explicitly. Keycloak defaults it
to true regardless of what the caller omits (keycloak/keycloak#28390; the
REST convertClientDescription() path ignores AuthnRequestsSigned where the
console honours it). A body that leaves it out produces a client that requires
the SP to sign AuthnRequests and rejects every login from an SP with no
keypair, the same “created successfully, cannot authenticate” shape this whole
exercise exists to eliminate. It is the single most important field in the set.
clientId is the SAML entity ID. redirectUris is the ACS allowlist;
the ACS endpoint itself comes from saml_assertion_consumer_url_post (or,
legacy, adminUrl as the Master SAML Processing URL). There is no secret.
schema.plug for saml.plug.yaml
Section titled “schema.plug for saml.plug.yaml”| Field | Required | Maps to |
|---|---|---|
realm@string! | yes | realm segment, same as OIDC |
entityId@string! | yes | clientId |
acsUrl@url! | yes | attributes.saml_assertion_consumer_url_post |
sloUrl@url | no | attributes.saml_single_logout_service_url_post |
nameIdFormat@string | no, default email | attributes.saml_name_id_format (username|email|persistent|transient) |
signAssertions@bool | no, default true | attributes.saml.assertion.signature |
clientSignatureRequired@bool | no, default false, always emitted | attributes.saml.client.signature |
spCertificate@string | required iff clientSignatureRequired | attributes.saml.signing.certificate (PEM body, no headers) |
encryptAssertions@bool | no, default false | attributes.saml.encrypt |
spEncryptionCertificate@string | required iff encryptAssertions | attributes.saml.encryption.certificate |
Ten fields, two of them conditionally required. The two conditional
requirements are exactly where a required:-style guard earns its keep: an
operator who flips clientSignatureRequired without supplying a cert must be
refused, not handed a client that 400s at login.
schema.socket: what the SAML socket adds over the OIDC one
Section titled “schema.socket: what the SAML socket adds over the OIDC one”Nothing certificate-shaped, and no shared secret. The IdP half of the trust
is Keycloak’s realm SAML descriptor, served unauthenticated at
<publicUrl><basePath>/realms/<realm>/protocol/saml/descriptor, which
apps/mautic/sso.plug.yaml:46 already fetches unaided. Since realm is a PLUG
value the socket cannot render the full URL, so:
- Drop from the SAML socket:
scopes(OIDC-only),kcadmPath(already documented as legacy atsso.socket.yaml:26-29). - Optional, recommended for both sockets: publish
realmsBase=<publicUrl><basePath>/realms. It is protocol-agnostic and would delete the same string concatenation from ~20 consumer plug jobs. Scope creep; flagged, not assumed.
What the SAML socket’s jobs.integration emits
Section titled “What the SAML socket’s jobs.integration emits”Nothing via ::shc-connection-value::. The OIDC socket’s mint-and-return
(sso.socket.yaml:171-184) exists because Keycloak generates the client secret;
SAML has no secret, and the descriptor is derivable. Do not invent a
certificate-return channel.
7. The migration: a migration, not a shim
Section titled “7. The migration: a migration, not a shim”Doctrine: no backcompat code, no aliases; data-migration tolerance is fine.
The baseline is append-only (make migrate, never flatten) and Atlas table
REBUILDS are forbidden.
7.1 What must be rewritten
Section titled “7.1 What must be rewritten”Per deployment sidecar manifest.json, for every edge whose plug is sso:
integrations[i].plug:"sso"→"oidc"edge_statesmap key:"sso"→"oidc"edge_states["oidc"].plug:"sso"→"oidc"edge_states["oidc"].socket_name:"sso"→"oidc"
All existing edges migrate to oidc, including mautic’s. That is factually
correct about what exists: the socket has only ever created
"protocol":"openid-connect" clients, so every live edge IS an OIDC edge.
Mautic’s re-wire to saml is a separate, deliberate operator action,
precisely because it changes what is provisioned in Keycloak.
7.2 Where it runs, and why not on the read path
Section titled “7.2 Where it runs, and why not on the read path”Not in migratePersistedIntegrationPlugs (install_async.go:1515). That
function is the natural-looking home; it is already a lenient read-side
sidecar rewriter, and its existing branches (backfill / drop / decline) are the
right shapes. But a permanent sso → oidc map there is an alias, and it is
the thing doctrine forbids. It also would not help the teardown path, which
never calls it.
Recommended: a one-shot, node-local sidecar migration run at daemon
start, ordered after the DB migration subsystem
(subsystems/impl/migrations.go), ledgered so it runs exactly once per node:
- walks
<stateHome>/deployments/*/*/*/manifest.json(plus*/vapps/*/), - applies the four rewrites above,
- writes through
WriteManifestso the atomic temp+rename and the0600mode are inherited rather than re-implemented, - logs one coded line per rewritten deployment and one summary; a sidecar it cannot parse is skipped and reported, never rewritten blind.
★ It must run on EVERY node, not the leader. The sidecar is node-local
(manifest.go:11-17); a leader-only migration leaves followers stale, the
same class as the follower-mTLS P1 (“leader-only checks are not checks”). The
ledger must therefore be node-scoped, not a replicated row.
7.3 The DB half
Section titled “7.3 The DB half”Two pure UPDATE statements in one new append-only Atlas migration file. No
new table, no column change, no rebuild, so the
PRAGMA foreign_keys = off rebuild hazard that destroys ON DELETE CASCADE
children is not in play at all:
UPDATE `integration` SET `consumer_plug_name` = 'oidc' WHERE `consumer_plug_name` = 'sso';UPDATE `integration` SET `provider_socket_name` = 'oidc' WHERE `provider_socket_name` = 'sso';The unique index integration_tenant_name_consumer_deployment_id_consumer_plug_name
cannot collide: no deployment can already hold an oidc plug, because oidc
does not exist before this change. The table is empty in production anyway
(§5.2), so this is cheap insurance. If the node-scoped ledger for §7.2 is to
live in the DB, add it as a new table in the same file, additive only.
7.4 Backup archives: not migratable, and that is the honest answer
Section titled “7.4 Backup archives: not migratable, and that is the honest answer”Every existing archive’s manifest carries ConsumerPlugName: "sso" and is
immutable. A restore of a pre-rename archive after the split replays a stale
-i sso=<provider> spec that resolves nothing. The remedy is the mechanism
that already exists: RecoverRequest.SkipIntegration
(recover.go:233-235, skip[rec.ConsumerPlugName]):
shc recover <archive> --skip-integration ssoshc update <stack> -i oidc=KEYCLOAKThis must be a release-note line and a docs/guide entry. It is a documented
operator step, not a compatibility layer. Do not add a translation at the
recover boundary; that is the alias doctrine forbids, on the one path where
being wrong is unrecoverable.
7.5 How the migration is PROVEN
Section titled “7.5 How the migration is PROVEN”The five guards that passed while the thing they guarded was broken (v0.0.12) are the standard to beat. Assert the post-migration artifact, and mutation-test in a disguised shape:
- Golden-sidecar test. A fixture
manifest.jsoncarrying anssoedge with a populatededge_states["sso"](frozenrealm/clientId,secret_fields: ["clientSecret"]held as a raw template) → run the migration → assert all four rewrites, byte-compare the rest, and assert mode0600. - Teardown replay across the migration. Build a deployed-tree fixture with
oidc.plug.yaml+ a keycloak tree withoidc.socket.yamlandsaml.socket.yaml, a MIGRATED sidecar, and assertRunUnintegrationJobsdispatchesdelete-oidc-clientwith the frozenrealm/clientId. Then run the same fixture with an UNMIGRATED sidecar and assert it reports a leak rather than returning silently. That second assertion is the mutation test for the §5.3 early return. - Idempotence. Run the migration twice; the second run rewrites nothing and reports zero.
- Multi-node. Assert the ledger is node-scoped: a second node with its own stale sidecars still migrates.
- Exclusion fitness test. Assert
defaultSSOBypassQueryis still"sso"andlicense.FeatureSSOis still"sso"after the sweep (§3). - Live proof, owed. One real consumer on one estate: capture the
pre-migration sidecar, roll, confirm the rewrite,
shc uninstall, and confirm in Keycloak that the client is gone. A green unit suite is not this proof.
8. Shared infrastructure vs per-protocol
Section titled “8. Shared infrastructure vs per-protocol”{% include %} / {% extends %} / {% import %} are deliberately refused
by the render engine (ce core/jinja/gonja.go:72-76, noLoader), YAML anchors
cannot cross files, and socket jobs carry no volumes: support (the
unintegration gate refuses volumes on a provider socket job outright,
unintegration_runner.go:642). There is no template-level sharing mechanism.
Protocol-agnostic: must exist ONCE, in one implementation
Section titled “Protocol-agnostic: must exist ONCE, in one implementation”| Piece | Where today |
|---|---|
check-keycloak-admin-api discovery preflight, bounded at 24 × 5s ≈ 120s | sso.socket.yaml:66-98 |
the admin-token mint (grant_type=password, client_id=admin-cli) | :89-93, :121-125, :235-239 |
★ curl-not-kcadm: every kcadm.sh call boots a cold ~20-30s Quarkus JVM, and chaining five-plus of them blew the daemon’s RPC exec deadline | the comment at :113-119; this lesson must survive verbatim into both sockets |
| realm ensure by HTTP code (200 vs 404), 409 tolerated on create | :129-141 |
client-existence probe + the grep -c '"id":"' fail-safe count (the historical grep -c "id" || echo "0" emitted "0\n0" and made the create branch unreachable) | :143-149 |
reconcile-on-every-run posture (client exists → PUT the declared state) | :185-216 |
| delete-by-CID teardown | :245-256 |
socket data:: hostname, port, publicUrl, basePath, adminUrl, adminUser, adminPassword | :18-25 |
Per-protocol
Section titled “Per-protocol”| OIDC only | SAML only |
|---|---|
"protocol":"openid-connect", publicClient:false, secret | "protocol":"saml", the ten saml* attributes |
post.logout.redirect.uris derived by sed off redirectUris (:150-158) | saml_single_logout_service_url_post |
the ::shc-connection-value::clientSecret= mint-and-return (:171-184) | nothing returned; descriptor is derivable |
scopes | nameIdFormat, signature/encryption posture |
InjectDerivedKeycloakIssuer (gated on the plug name; must NOT fire for saml) | the realm SAML descriptor URL |
How to share it: recommendation
Section titled “How to share it: recommendation”Now: duplicate the ~40-line preamble across the two socket files and guard it
with an architectural fitness function (ADR-0005; scripts/arch-fitness.sh
already runs in make lint). The test extracts the shared block from both
files and asserts byte-identity, so the cold-JVM lesson and the fail-safe
count idiom cannot drift apart.
Later: move it into the SHC keycloak image as /opt/keycloak/bin/shc-kc-admin.sh,
sourced by both jobs; one implementation, no duplication. Correct, but it needs
an image digest bump, and builtin/keycloak/vars.yaml:31-35 is explicit that a
broken image here is “an estate-wide SSO outage” and “mvn test green is not
a boot”. Do not couple the split to an image rebuild. Ride the next one
that happens for another reason.
9. Naming, and the reserved-word check
Section titled “9. Naming, and the reserved-word check”oidc is the right name; it fixes a symmetry break rather than creating a collision
Section titled “oidc is the right name; it fixes a symmetry break rather than creating a collision”Socket names and app names are different namespaces (socket identity is the
<stem>.socket.yaml filename stem). And the external-* family is a perfect
pattern with exactly one exception:
| App | Serves socket |
|---|---|
external-imap | imap |
external-mongodb | mongodb |
external-mysql | mysql |
external-postgres | postgres |
external-redis | redis |
external-s3 | s3 |
external-smtp | smtp |
external-oidc | sso ← the only break |
apps/external-oidc/oidc.socket.yaml restores 8-of-8. It also makes the
cloudron adapter’s "oidc" addon key map to SocketOIDC, an identity where
today it is a translation (tests/unit/modules/adapter/adapterplug_test.go:30).
Useful symmetry, no collision.
Reserved words: clear
Section titled “Reserved words: clear”docs/reviews/2026-07-21-adapter-program/terminology-sweep-plan.md:8 names the
reserved set: promote (rqlite flip), convert/converter (backup
engine + converters: schema + ConverterName), adopt (cluster-CA
genesis). Neither oidc nor saml is on it, and neither appears in
ce core/validation/reserved_names.go:38 (which governs tenant/stack/environment
names, not socket names, and reserves none of these).
SocketLDAP = "ldap" (adapterplug.go:37) is the standing precedent for a
capability socket name declared with no provider app, so saml would be
well-formed even if keycloak were not going to serve it.
The required companion: wire the plug’s declared socket: key
Section titled “The required companion: wire the plug’s declared socket: key”This is what converts the split from a rename into an honest contract, and it must land WITH the split:
- Read the optional top-level
socket:key from<plug>.plug.yaml(default: the file stem). It is already documented (docs/app-development/plugs-and-sockets.md:154) and already authored in 4 files. - In
ResolveIntegrations, after the plug resolves andic.SocketNameis known, refuse with a new coded error whendeclaredSocket != ic.SocketName. - Fix
diskPlugSpecsto read the declared key instead of hardcodingSocket: stem, soConnect’s existingX400016TYPMIScheck (integration_service.go:147) starts telling the truth on the recover path.
Behaviour this preserves and this refuses:
| Case | Declared socket | Resolved socket | Outcome |
|---|---|---|---|
matrix’s postgresMas.plug.yaml → postgres’s sole socket | postgres (declared) | postgres | passes; today’s working pattern is preserved |
-i keycloak=<kc> on traefik (sole-plug fallback → oidc) | oidc (stem) | oidc | passes, but see §5.4: the spelling dies anyway once keycloak ships two sockets |
a saml.plug.yaml against external-oidc (one socket) | saml | oidc | ★ REFUSED, the visible failure the split is for |
a stale -i sso=… after the migration | sso (no file) | oidc | REFUSED with a coded error instead of the current cryptic empty-realm 404 |
Note that no sso plug in the catalogue declares required: today (only
apps/journiv/sso.plug.yaml:43, and it says required: false). So the
X400994PLGREQ “required plug not connected” refusal is available but not
what produces the visible failure; the socket-type check above is.
10. Sequencing against work in flight
Section titled “10. Sequencing against work in flight”The sibling lane is fixing the socket’s credential scope: adminUrl,
adminUser, adminPassword are published in integration.socket.data to
every consumer (3 of 32 actually read them into their own jobs: netbird,
mailcow, nextcloud; the other 29 get them in the render context anyway),
and the fix is per-consumer scoped service accounts.
Ruling requested: SAME RELEASE WINDOW, TWO ORDERED COMMITS, credential fix first. This agrees with the owner’s prior on the window and refines it on the count.
Why the same window: the estate redeploy wave is the expensive part, and it is driven by the credential fix (the split alone does not require a redeploy; §5.3 last row). One wave, one CHANGELOG breaking entry, one migration.
Why credential-fix-first rather than one combined commit:
- It is one file instead of two. The credential fix rewrites the auth
mechanism of
create-oidc-client/delete-oidc-clientin today’s singlesso.socket.yaml. Doing it after the split means writing it twice, and it means authoring the brand-newsaml.socket.yamlagainst a mechanism that is about to be replaced. - The split then inherits the corrected mechanism. The fragile shared preamble of §8 gets authored once, already right, into both sockets, and the byte-identity fitness test locks it there.
- Attributability. Both changes can break client minting. If they land as one commit and a client fails to mint on an estate, you cannot tell which half did it. Two commits in one release keep the bisect one step wide.
- The migration belongs to the split alone. Keeping it in the second commit keeps its blast radius attributable, and keeps commit 1 revertible without touching persisted state.
The other three active lanes are untouched by this: the v0.0.19 estate roll
(the split must land after it, not into it; do not add a socket-contract
break to a release already in flight), the Nextcloud competing-provider fix
(same file family, apps/nextcloud/sso.plug.yaml, so it must land and be
merged before the 32-file rename sweep, or the rename will conflict with it),
and the frappe assets fix (apps/frappe/sso.plug.yaml is in the rename set for
the same reason).
11. Price
Section titled “11. Price”| Work | Size | Risk |
|---|---|---|
Split sso.socket.yaml → oidc.socket.yaml + new saml.socket.yaml | 2 files, ~260 + ~200 lines | medium; the SAML body is new code against a live IdP |
Rename 32 apps/*/sso.plug.yaml + 1 apps/external-oidc/sso.socket.yaml + 2 ce builtin/*/sso.plug.yaml | 35 git mv | low, mechanical |
Rewrite integrations.sso.* → integrations.oidc.* | 23 compose + 18 plug + 3 ce templates + 1 vars | low, mechanical, make lint runs validate-templates over apps/ |
4 ce Go identifier changes + 8 SocketSSO sites, incl. the licensing gate widening to SocketOIDC || SocketSAML | ~6 files | ★ medium; a missed gate site is an unlicensed external-IdP bypass |
Exclusion fitness test for defaultSSOBypassQuery + FeatureSSO | 1 test | low, high value |
Wire the declared plug socket: key + new coded error + fix diskPlugSpecs (§9) | ~40 lines + 1 error code + tests | medium; changes a resolution path; the four existing declarations become the regression fixtures |
| One-shot node-local sidecar migration + node-scoped ledger + 2-statement Atlas migration | ~150 lines + 1 SQL file | ★ high consequence, low complexity; this is the piece that must not be got wrong |
Convert apps/mautic to a real saml.plug.yaml and delete its OIDC-shaped clientSecret var | 3 files | medium; the first genuine SAML consumer; needs a real login proof |
Retire -i keycloak=<stack>: docs, ~10 app header comments, tests/proxmox/test_multinode.bats:3687, CHANGELOG | ~15 files | low |
| Tests: 39 app bats + 30 unit + 3 builtin + 1 proxmox + 10 ce Go + 5 TF fixture files | ~88 files | low, mechanical, but the volume is where a blind sweep hits §3’s two exclusions |
| Docs: 20 shc files | ~20 files | low |
Estate TF: 4 integrate.sso → integrate.oidc (2 per estate) | 2 files | low. ★ Note the TF diff emits Unintegrations: ["sso"] + a new -i oidc=KEYCLOAK (diffRemovedIntegrations, app_grammar.go:805), which is benign: same realm, same clientId, so create-oidc-client reconciles the existing client |
~200 files touched, of which ~180 are mechanical. The real engineering is three things: the SAML socket body, the sidecar migration, and the socket-type check. The value: one live defect closed (mautic), one silent-teardown class closed (the §5.3 early return, which is a latent hazard today for any provider that grows a second socket), and one contract made honest.
If the owner rules against the split, the mautic defect still needs fixing, and
the only alternatives are to delete mautic’s SSO plug (losing the feature) or to
teach the single sso socket a protocol switch, which is the flag-on-the-path
shape the owner already rejected, and which would leave the socket-type check
unwired and the next SAML app in the same trap.
12. What is explicitly NOT recommended
Section titled “12. What is explicitly NOT recommended”- No aliases. No
sso→oidcmap on any read path, no dual-name lookup, no compat stem. The migration rewrites state once; after thatssois not a name the platform knows. - No translation at the recover boundary. Pre-rename archives use
--skip-integration sso+ a re-wire (§7.4). - No
protocol:flag on one socket. Rejected by the owner, and §6 shows the shapes do not overlap enough to justify it. - No image rebuild coupled to this change (§8).
- No renaming of
FeatureSSOordefaultSSOBypassQuery(§3). - No
--redact-*flags on anything here; the topology-ships-plainly ruling stands.
13. Open questions for the owner
Section titled “13. Open questions for the owner”- Window. Confirm: credential-scope fix then split, two commits, one release, landing after v0.0.19 is rolled. (§10)
- Scope of the socket-type check. It is a required companion in this design (§9). Confirm it lands with the split rather than as its own change; it is the only thing that makes the refusal visible, but it does touch a resolution path used by every socket.
realmsBaseon both sockets. Would delete ~20 copies of string concatenation from consumer plug jobs. In or out? (§6)apps/mautic. Convert tosamlin the same window (and own the live login proof), or land the split first and convert mautic separately? The defect is pre-existing either way.- Ledger home. Node-scoped file marker under
<stateHome>, or a new node-keyed table in the Atlas baseline? Both are additive; the table is inspectable, the file cannot drift from the sidecars it guards.