Skip to content
SHC Docs

Integration-edge identity: the model

Status: design document, revision 3. Revision 1 was implemented in modules/app (six of seven rules) and then found wrong in three places by an adversarial review of its own output. Revision 2 fixed those three — and was itself broken by a second adversarial review in two places: the merge could still produce a set in which one removal token addressed two rows whose printed spellings were IDENTICAL (so X400972INTAMB’s advice was unfollowable), and formatIntegrationSpec had two residual totality holes where the one spelling it handed an operator was a token that could not address the row it described. Revision 3 closes both — the identity invariant now holds BY CONSTRUCTION (one identity function, and the merge normalises to one row per identity) — and the code matches it. §14 records every round, so the next reader does not re-derive an earlier revision from first principles.

Every behavioural claim below was executed against the real code in a pinned scratch copy, not inferred from reading.

  • Implementation read: /var/tmp/shc-work/ce-cleanup/modules/app/{install_async.go, integration_render.go,host_spec.go,port_mappings.go,var_overrides.go,plug_required.go, install_runner.go,integration_service.go,types.go,errors.go}, /var/tmp/shc-work/ce-cleanup/modules/app/commands/update.go, /var/tmp/shc-work/ce-cleanup/ent/schema/integration.go, and the Terraform provider’s app_grammar.go / app_ssh.go in both /var/tmp/shc-work/terraform/provider/ (pre-marker) and /var/tmp/shc-work/release-retry/terraform/provider/ (post-marker).
  • Verification harness: /var/tmp/shc-work/ce-cleanup copied to <scratchpad>/ce-pin, probes added as modules/app/zz_probe{,2,3}_test.go, run with GOWORK=off go test ./modules/app/ -run TestProbe -v. The probe files are left in place as evidence. The CE tree did compile and vet clean at the time of the copy (GOWORK=off go vet ./modules/app/ silent), so nothing here is blocked on the concurrency-owned files.
  • Provider line numbers are as of 01:29 on the read day; release-retry/.../app_grammar.go was modified by a concurrent workflow during this read (mtime moved 01:27), so provider anchors are given by function name and the line numbers should be treated as approximate.
  • New error codes are specified below. Per the concurrency rule they must land in a new modules/app/errors_edges.go, never in errors.go.

0. Ground truth — the facts the model is not free to contradict

Section titled “0. Ground truth — the facts the model is not free to contradict”

These are properties of the running system, not opinions. Every rule below is derived from them.

G1. The render namespace is keyed by plug, and only by plug. BuildIntegrationsContext ends with out[ic.Plug] = entryintegration_render.go:791. Two edges that resolve to the same plug collide in a Go map: the last one silently wins and the first vanishes from integrations.<plug> with no diagnostic anywhere.

G2. The database’s natural key for an edge is the consumer plug. ent/schema/integration.go:64: index.Fields("tenant_name", "consumer_deployment_id", "consumer_plug_name").Unique(). IntegrationService.Connect is a guarded upsert on that triple (integration_service.go:163-173); IntegrationService.Disconnect takes (tenant, stack, environment, plugName) and nothing else (integration_service.go:180). The schema comment says it outright: “a consumer plug binds to at most one provider socket.”

G3. The sidecar persists what the operator typed, not what the system resolved. install_runner.go:1197-1200 builds persistedIntegrations by direct conversion from opts.Integrations. PersistedIntegration (install_runner.go:2211-2236) carries Plug verbatim. A bare -i keycloak therefore persists Plug: "". The plug the render inferred is written into a copy (ic.Plug, integration_render.go:310-316) and thrown away.

G4. G1+G2 and G3 disagree, and that disagreement is the entire bug family. The system’s identity for an edge is the plug. The state’s identity is “whatever string was on the command line”. Every critical from R3 through R5 is a different symptom of code trying to match persisted state under an identity the persisted state does not carry.

G5. The three sibling surfaces already have an explicit identity/value split. Integrations is the only one that does not.

surfaceidentityvaluewhere
--varthe keythe valuetrivially, MergeVarOverrides var_overrides.go:7
--host(hostname, path)(service, port)hostSpecIdentity host_spec.go:490
--port(connection, external, protocol)(service, container, nodeport)portKey port_mappings.go:68
-iunstatedunstatedsameIntegrationEdge install_async.go:716 guesses per-call

G6. The Terraform provider is already plug-keyed on every integration surface. diffRemovedIntegrations returns map keys, which validateIntegrationKey forces to be bare plug names. buildIntegrationSpec always emits plug=… from the map form. The provider has never emitted a bare -i provider or a provider! removal. Anything the model does to bare tokens is CLI-surface-only blast radius.

G7. ApplyMagicToken already normalises specs on both the parse path and the sidecar read path (integration_render.go:137, install_async.go:264), rewriting keycloak(stack=keycloak, tenant=master). Spec normalisation on read is an existing, accepted concept — the model leans on it for migration.

G8. A persisted row with no plug is not an edge. It is a consequence of G1 and G2, not a new claim: the render namespace is keyed by plug, so a row with no plug contributes no integrations.<plug> entry; the DB’s natural key is the plug, so it writes no integration row; PersistedConnectedPlugs contributes it to no gate. The row is inert. Revision 1 called these “plugless edges” and tried to abolish them (R1a); revision 2 names them UNRESOLVED INTENTS and gives them an identity, because abolishing them is not something the implementation can honestly deliver — see §3.2 and §14.2.


sameIntegrationEdge (install_async.go:716-724) answers “it depends which side has a plug”:

if a.Plug != "" && b.Plug != "" {
return a.Plug == b.Plug
}
return a.ProviderStack == b.ProviderStack &&
a.ProviderTenant == b.ProviderTenant &&
a.ProviderEnvironment == b.ProviderEnvironment &&
a.ProviderConnection == b.ProviderConnection

That is not an identity. It is two different identities selected by an accident of how the other operand was spelled. It is wrong, and it is the R5 critical.

Verified (TestProbeBareAddDropsPluggedSibling, TestProbe3R5Normalised):

prior = [{cache→postgres}, {db→postgres}] adds = [-i postgres]
merged = [{Plug:"" →postgres}, {db→postgres}]
dropped = []

The cache edge is gone. dropped is empty, so the integration edge removed log line (install_async.go:308-313) never fires and DisconnectRemovedIntegrations (install_async.go:788) never tears down the DB row. Silent, on all three surfaces at once.

The mirror is just as bad (TestProbe3PluggedThenBareNormalised):

prior = [{sso→keycloak}] adds = [-i keycloak]
merged = [{Plug:"" →keycloak}]

The plug is erased from persisted state. The render still works (it re-infers sso), so nothing looks broken — until the next Terraform converge emits -i sso!, which is the only handle the provider has (G6), and it matches nothing.

candidate-i sso=authentik over sso=keycloak{cache=postgres, db=postgres}verdict
plug alonerepoint ✅two edges ✅satisfies both
plug + providerduplicatetwo edges ✅fails the repoint requirement; also unrepresentable — two rows on one plug violate G2’s UNIQUE index and collide in G1’s map
provider + environmentrepoint ✅one edgefails the distinctness requirement
the connectionundefined for stack-backed edgesundefinednot a general identity; exists only for @conn edges

Only “plug alone” survives, and it survives because G1 and G2 already enforce it. An identity that permits two simultaneous edges on one plug is describing a state the render namespace and the database cannot hold.

An edge’s identity is (tenant, stack, environment, plug). Within one deployment, the plug alone. The provider stack, the provider tenant, the provider environment and the provider connection are the edge’s VALUE.

Corollary R1a (the load-bearing half): no INPUT can create a plugless edge. The bare -i <provider> form stays as CLI sugar, but it is resolved to a plug before it becomes state, on every verb. A token that cannot be resolved to exactly one plug is refused rather than persisted.

Corollary R1b: a persisted row that carries no plug is an UNRESOLVED INTENT, not an edge (G8), and its identity is its provider reference. Two intents with the same provider reference are the same intent.

R1a is what makes R1 true of the state rather than only of the render. Revision 1 stated it as a property of the SIDECAR (“there are no plugless edges”) and every rule below leaned on that. It is not deliverable: inference reads the provider’s app directory, that directory is absent when the provider is not deployed on this node, and dropping such a row would silently disconnect an edge that heals the moment the provider comes back. The model itself blesses leaving those rows alone (§1.4 change 1.3) — so it asserted a premise its own migration rule contradicts.

R1b is the honest replacement. It costs nothing (an intent is inert by G8, so nothing downstream needs a special case for it) and it buys the thing the missing premise was load-bearing for: the unconditional addressability proof in §3.2.

#changefile:line
1.1sameIntegrationEdge: a plugged spec is the same edge as another iff the plugs match. The provider-ref comparison survives for the both-plugless case ONLY, where under R1b it is not a fallback but the definition of an intent’s identity — it is what dedupes two intents on one provider and so what makes @conn! / stack@env! unambiguous (§3.2). (Revision 1 said “deleted”; that was a consequence of the R1a it could not deliver.)modules/app/install_async.go
1.2Persist the resolved plug. persistedIntegrations must be built from the resolved contexts (which already carry ic.Plug, ic.ProviderTenant, ic.ProviderEnvironment) rather than from the raw opts.Integrations. OWEDinstall_runner.go / register.go were owned by another workstream.modules/app/install_runner.go:1197-1200, consuming integrationContexts from :487
1.3Read-side migration: a sidecar row with Plug == "" is resolved on read via the same inference and rewritten on the next deploy. Resolvable → backfilled; provider unreadable on this node → left alone (it is an intent, and it heals when the provider returns); provider readable but inference empty → dropped with X200969INTDRP; inference ambiguous → left alone with a warn.modules/app/install_async.go (migratePersistedIntegrationPlugs); precedent X200996HSTDRP
1.4removedEdgePlugNames’ plugless branch (the PlugRow.ConnectedTo prefix scan) is the INTENT path and stays. Revision 1 called it dead-under-R1a; it is not, and it needed a guard rather than a deletion — see R5a / change 5.5.modules/app/install_async.go

Changes behaviour? Yes — 1.1 and 1.2 change what a bare add does to state (see §2) and change what the sidecar contains. The repoint case (sso=authentik over sso=keycloak) already works and is merely written down: verified TestProbeRepointmerged=[{Plug:sso ProviderStack:authentik}].


2. What does a bare add mean when the deployment already has edges on that provider?

Section titled “2. What does a bare add mean when the deployment already has edges on that provider?”

Implicitly, via sameIntegrationEdge’s fallback branch: “replace the first persisted edge whose provider reference matches, whatever plug it was on.” That is R5. It is not one of the three options on the table; it is a fourth one nobody chose.

Verified behaviours:

priorbare addresultverdict
[{cache→postgres},{db→postgres}]-i postgres[{Plug:"" →postgres},{db→postgres}], dropped=[]silent destruction
[{Plug:"" →postgres}]-i postgres[{Plug:"" →postgres}]benign dedupe
[{sso→keycloak}]-i postgresappendbenign
[{sso→keycloak}]-i keycloak[{Plug:"" →keycloak}]plug erased from state

2.2 The options, and why none of the three offered is the answer

Section titled “2.2 The options, and why none of the three offered is the answer”
  • Append a new unplugged edge — creates a second edge on a plug the render will infer, so G1 makes one of them silently disappear at render time and G2 makes the second Connect upsert overwrite the first row. It writes a state the system cannot represent.
  • Replace the single existing one — only defined when there is exactly one; on {cache, db} it has to pick, and picking is R5.
  • Refuse as ambiguous — right instinct, wrong place. The ambiguity is not “which edge does this add collide with”; it is “which plug does this provider mean”. Refusing at the collision site makes -i postgres fail on a two-edge deployment and succeed on a one-edge deployment, which is an identity that depends on how much state you already have.

A bare add is resolved to a plug FIRST; then it is an ordinary add-or-repoint on that plug. It is never matched positionally against a provider reference.

  • Resolution uses the existing inferPlugFromProvider (integration_render.go:145-192): the provider’s *.socket.yaml stems matched against the consumer’s *.plug.yaml stems, with the existing “provider ships exactly one socket” fallback.
  • Exactly one candidate → the add is <plug>=<provider> and repoints that plug.
  • Zero candidates → refuse (X400973INTPLG), naming the provider’s sockets and the consumer’s plugs, and telling the operator to type plug=provider.
  • More than one candidate → refuse, same code. (See open question OQ2 — this is the one clause with catalogue blast radius.)

Under R2, {cache=postgres, db=postgres} + -i postgres resolves the token to whatever single plug the postgres socket maps to, and repoints exactly that one. It never touches the sibling. If the provider’s socket stem is postgres and neither plug file is named postgres, the “sole socket” fallback resolves the plug name to postgres, and the result is a third edge on plug postgres — which is correct and, critically, is now named in state, so it is addressable and removable.

Note what R2 also buys: -i sso=keycloak -i sso=authentik in one invocation currently resolves to authentik with no complaint (verified TestProbe2TwoAddsOnePlug). Under R1 that is two adds on one identity, and the same last-wins rule applies — but now it is a stated rule about a stated identity, and it can be warned about (§4) rather than being an emergent property of slice order.

#changefile:line
2.1inferPlugFromProvider returns (name string, candidates int) instead of a first-match string.modules/app/integration_render.go:145-192
2.2New ResolveSpecPlugs(specs, appDir, stateHome, tenant, env) ([]IntegrationSpec, error) fills Plug on every plugless spec, refusing X400973INTPLG on 0 or >1. Called on addedIntegrations and on the sidecar-read priors before MergeIntegrationSpecs.new fn; call sites modules/app/install_async.go:228 and :262-272, before :303
2.3New code X400973INTPLG{provider, sockets, plugs} — “-i <provider> cannot be resolved to one consumer plug”.new file modules/app/errors_edges.go
2.4MergeIntegrationSpecs may then assume every spec has a plug; its add loop is a plug-keyed map upsert.modules/app/install_async.go:679-700

Changes behaviour? Yes, decisively. This is the R5 fix, and it is a fix by specification rather than by another special case — the plugless spec, which is what every previous round was patching around, stops existing.


3. What does a removal token mean — and what does it still mean on the second run?

Section titled “3. What does a removal token mean — and what does it still mean on the second run?”

resolveIntegrationRemovals (install_async.go:534-566) picks the first handle from {plug, connection, providerStack} that matches anything, then SplitRemovedIntegrations (install_async.go:738-762) removes every edge matching under that handle.

Verified:

priortokendroppedproblem
[{cache→postgres},{db→postgres}]postgres!bothwide, silent, unnameable-in-advance
[{s3→@aws-primary},{backup→@aws-primary}]aws-primary! or @aws-primary!bothsame class on the connection handle
[{postgres→redis},{Plug:"" →postgres}]postgres!{postgres→redis} onlythe bare edge on stack postgres is unaddressable by any token
[{keycloak→authentik},{sso→keycloak}]keycloak!{keycloak→authentik}plug wins — but sso→keycloak is still nameable as sso!

The last two rows were read, in revision 1, as “plug-first precedence is correct and it strands only plugless edges, which R1a abolishes”. Both halves of that reading are wrong. R1a does not abolish them (§1.3), and precedence is not the property that matters — what the token means on the NEXT run is, and precedence gives no answer at all to that (§14.1).

Revision 1 answered this with a total order — try the plug handle, then the connection, then the provider stack, and take the first that matches anything — plus a count-of-one rule. That answer is wrong, and §4 is what makes it wrong; the two are individually right and jointly destructive. The argument is in §14.1. The rule is now:

A removal token’s IDENTITY CLASS is decided by the token’s SYNTAX, and by nothing else. It then addresses exactly one row, or nothing, or the update is refused.

tokenclassaddresses
name!plugthe WIRED EDGE whose consumer plug is name
@name!intentthe UNRESOLVED INTENT whose provider connection is name
name@env!intentthe UNRESOLVED INTENT on provider stack name, in environment env
plug=provider!refused, X400002VALINP
! / empty / @ / name@refused, X400002VALINP
  1. A WIRED edge is addressed by its plug and by nothing else. Its provider stack, provider environment and provider connection are its VALUE (R1), and you never address by value.
  2. An UNRESOLVED INTENT has no plug, so its provider reference IS its identity (R1b), and the two value-shaped tokens address intents only.
  3. The environment in name@env is the one the row resolves in (ProviderEnvironmentFor), not the literal string it persisted. A row written as -i relay in environment prod carries ProviderEnvironment: "" and means prod; comparing literally would leave it unnameable. Resolution is a pure function of the row and the deployment’s own environment, so it does not reintroduce state-dependence.
  4. Count the rows the token addresses. 1 → remove it. 0 → §4. >1X400972INTAMB, remove nothing.

Why syntax and not “what matches”. A token’s meaning must not be a function of how much state you already have. That is this document’s own argument in §2.2, made against refusing a bare add at the collision site — and revision 1 built exactly that property into removals without noticing. Under a total order, once the plug a token named is gone, the SAME token silently re-classifies as a provider-stack lookup and can match a DIFFERENT edge. Verified, and catalogue-real (§14.1).

Every persisted row is addressable, unconditionally. The proof no longer has a false premise:

  • The two ROW classes are disjoint by construction: a row carries a plug or it does not.
  • A wired edge is named by its plug, and plugs are unique within a deployment (G1 + G2). (Revision 3 adds the grammar precondition this silently assumed: a plug name carrying @ or = is a name the token grammar cannot spell, so it is refused at input — errIntegrationPlugName — and never becomes a wired edge’s identity. See §14.6.)
  • An intent is named by its provider reference, and two intents with the same provider reference are the same intent (R1b). (Revision 3: “the same provider reference” is decided by integrationIdentity — resolved environment, tenant excluded — the ONE function every consumer of identity asks, because letting the merge and the removal grammar hold separate opinions is exactly how the unfollowable ambiguity shipped. See §14.5.)
  • The two TOKEN classes are disjoint by syntax.

⇒ every persisted row is named by exactly one token, and every token addresses at most one row. There is no state in which an operator is stuck — including {→postgres, cache→postgres}, the state revision 1 could not express, where the plugless row was nameable by no token at all and the ambiguity error’s own advice (“name one of them by its plug”) was unfollowable.

X400972INTAMB therefore becomes an INVARIANT CHECK, not an operator-facing rule. It fires only for a persisted set holding two rows with one identity — which, as of revision 3, the merge is incapable of producing BY CONSTRUCTION rather than by argument: normalizeIntegrationIdentities collapses the prior set to one row per identity (last wins, mirroring the render’s out[ic.Plug] = entry) BEFORE removals resolve against it, and again after the adds land. The only way to reach the code is to hand a duplicate-identity set directly to SplitRemovedIntegrations (a hand-edited sidecar read outside the deploy path). Its message no longer advises disambiguation — two rows with one identity CANNOT be told apart by any token — it says the state is inconsistent and that a plain shc update normalises it, which is the remedy that actually works. Keeping it is cheap and it refuses rather than guessing.

What is deliberately given up:

  • “disconnect everything on provider X” as an implicit reading of -i X! — the silent over-removal class the whole effort exists to kill.
  • the bare -i keycloak! spelling for “the edge pointing at keycloak”. It read as the mirror of the bare -i keycloak ADD, but R1a already broke that mirror: the bare add is resolved to a plug before it becomes state, so the edge is sso=keycloak and sso! is its name. An operator who types the old spelling gets a reported no-op (§4) listing the live edges, which contains the token they wanted. One extra round trip, loud, and never destructive.
  • -i aws-primary! (sigil-less) for a connection-backed edge. Same reasoning: that edge has a plug.

Blast radius on the terraform provider: none. diffRemovedIntegrations returns map keys and validateIntegrationKey forces them to be bare plug names (G6), and a bare token is a plug. The provider’s removals were already in the only class it can emit.

#changefile:line
3.1New classifyIntegrationRemoval(tok, live, environment) puts a token in one class from its shape. integrationRemoval carries the classified fields (plug | connection | stack+environment) instead of a re-parsed IntegrationSpec. handleProviderStackhandleProviderRef.modules/app/install_async.go
3.2matchesIntegrationRemoval gains the spec.Plug == "" guard on both value handles, and compares ProviderEnvironmentFor(spec, environment) rather than the literal field.same
3.3resolveIntegrationRemovals / countIntegrationMatches / SplitRemovedIntegrations / MergeIntegrationSpecs all thread environment.same
3.4formatIntegrationSpec renders an INTENT as its removal token (@conn, stack@env) rather than as a bare stack — the old rendering handed the operator a token that could not address the row it described, and on a deployment with a matching plug addressed a different one.same
3.5X400972INTAMB’s message ends “name one of them by its own identity” (was “by its plug”, which an intent does not have). Superseded in revision 3: two rows with one identity print identically, so “name one of them” was unfollowable — the message now says the state is inconsistent and that a plain shc update normalises it (§14.5).modules/app/errors_edges.go
3.6(revision 3) One identity function: integrationIdentity computes THE token that names a row; sameIntegrationEdge, matchesIntegrationRemoval and formatIntegrationSpec all ask it, and normalizeIntegrationIdentities makes the merge hold one row per identity by construction. formatIntegrationSpec prints prose, never a token shape, for the two row kinds no token can address (§14.5, §14.6).modules/app/install_async.go

Changes behaviour? Yes, twice over. Revision 1’s multi-match removal became a refusal; revision 2 narrows what each token can reach at all, so several revision-1 spellings become reported no-ops.


4. Is an unmatched removal an error or a no-op?

Section titled “4. Is an unmatched removal an error or a no-op?”

4.1 What today’s code answers — verified, and it is inconsistent four ways

Section titled “4.1 What today’s code answers — verified, and it is inconsistent four ways”
flagunmatched removalevidence
--var key!silent no-opMergeVarOverrides var_overrides.go:18-20 is a bare delete(); probe → map[a:1] unchanged
--host h!silent no-opMergeHostSpecs host_spec.go:693-713 builds a set and filters; probe → route set unchanged
--port p!silent no-opMergePortMappings port_mappings.go:302-316; probe → mapping set unchanged
-i ref!hard error, whole update abortsX400971INTNOF, install_async.go:549/561, errors.go:205

The live hazard is not hypothetical. Verified end-to-end in TestProbe3ProviderConverge, replaying the identical provider argv twice (the “state write lost after a successful apply” case that X400971INTNOF’s own doc comment at errors.go:198-204 predicts):

converge #1: merged=[{sso→keycloak}] dropped=[{s3→@aws-primary}] err=<nil>
converge #2: merged=[] dropped=[]
err=[X400971INTNOF] integration removal s3 matches no integration ...

The second run aborts before ExecuteCompose, so the whole update is lost — including any version bump, var change or route change that rode the same converge. A partially-applied ssh update leaves the estate unable to converge at all until someone hand-edits tfstate.

“Error” and “silent no-op” are not the only options, and picking either loses something real:

  • Error everywhere. Removals are the one operation that is naturally idempotent, and idempotence is what makes converge-shaped callers safe. Erroring makes the second run of a correct command fail. The failure mode is asymmetric: a failing ADD is harmless (nothing changed, retry works); a failing REMOVAL means you cannot converge at all.
  • Silent no-op everywhere. This is the original sin — shc update myblog -i keyclaok! exiting 0 with the edge still wired is exactly what the removal marker was built to eliminate. Retreating to it would undo the point of the effort.

One rule for all four flags: an unmatched removal is a NO-OP, and it is REPORTED. Never silent, never fatal.

  • The update proceeds. Every other part of the invocation applies.
  • One coded warning per unmatched token — X200970RMVNOP{flag, token, live} — carrying the same payload the error carried: the token as typed and the live set it failed to match.
  • The warning goes to corelog.Warn. It should ALSO ride the operation observer, so it appears in shc update’s streamed output rather than only in the daemon journal — this half is OWED, not done, and the reason is structural: operations.DeploymentPayload is a deployment progress snapshot with nowhere to put a sentence, so an advisory frame kind has to be added and the CLI-side decoders that read that payload have to be taught it. Until then -i is QUIETER at the terminal than the hard error it replaced, which is a real regression on that one surface and the honest price of the reversal. See OQ4.
  • --strict-removals (flag, or SHC_STRICT_REMOVALS=1) promotes the warning back to a hard error for operators who want the old -i guarantee in a script. Default off — and it is also the interim answer for a caller that needs the failure visible at the terminal.

R4 is only safe because of R3’s syntax-classing, and that dependency is the whole of §14.1: a reported no-op is harmless only if the token that produced it could not have meant something else. Under revision 1’s total order it could, so the second run of a correct removal destroyed a different edge and --strict-removals could not see it, because the token matched.

Read alongside R3, the rule is: matched-nothing is a loud no-op; matched-too-many is a hard error. That asymmetry is the whole point and it is defensible in one line — a no-op cannot destroy anything, and a guess can.

The operator learns nothing happened because the warning names the token and prints the live set, which is the same message X400971INTNOF prints today. The only thing that changes is the exit code. That is precisely the property a converge needs and the property a typo does not.

#changefile:line
4.1MergeVarOverrides returns (merged map[string]string, unmatched []string).modules/app/var_overrides.go:7-25; caller install_runner.go:217
4.2MergeHostSpecs returns unmatched []string (removal identities that matched no prior route).modules/app/host_spec.go:652-714; caller install_runner.go:377
4.3MergePortMappings returns unmatched []string.modules/app/port_mappings.go:297-332; caller install_runner.go:206
4.4resolveIntegrationRemovals returns unmatched tokens instead of raising X400971INTNOF. X400971INTNOF is retired (its registry row at errors.go:1675 and factory at :205 go, with a memorial comment recording why the hard failure was tried and what it cost).modules/app/install_async.go:534-566, errors.go:184-209
4.5New code X200970RMVNOP{flag, token, live}. One code, one flag param, four surfaces.modules/app/errors_edges.go
4.6One emit point per surface, after each merge, warning + observer frame.modules/app/install_runner.go:206/217/377, modules/app/install_async.go:303-313 (reuse the existing dropped log loop’s position)
4.7Register --strict-removals on shc update; thread as UpdateRequest.StrictRemovals *bool.modules/app/commands/update.go:179-215, modules/app/types.go:~270

Changes behaviour? Yes, on all four, and this is the largest change in the document. -i loses a hard guarantee that a previous review round explicitly asked for. This one needs the owner’s ruling — see OQ1.


5. Precedence when an add and a removal collide in one invocation

Section titled “5. Precedence when an add and a removal collide in one invocation”

5.1 What today’s code answers — verified. It is a 2–2 split, not three behaviours

Section titled “5.1 What today’s code answers — verified. It is a 2–2 split, not three behaviours”
surfaceorderX and X! on one lineevidence
--hostadditions folded in, then removals filterremoval wins (result empty)host_spec.go:676-713; TestProbeHostAddVsRemoveSameLineout=[]
--varprior copied, adds overlaid, then delete()removal winsvar_overrides.go:11-20
--portremovals filter prior, then adds appendadd winsport_mappings.go:302-330; TestProbePortAddVsRemoveSameLine → the add survives
-iremovals resolve against prior only, adds land lastadd winsinstall_async.go:679-699; install_async.go:663-676 states it as a contract

So {hosts, vars} are removal-wins and {ports, integrations} are add-wins. The task’s “ports are a third” is true of a different axis — ports resolve removals against prior only and a protocol-less removal widens to both protocols (verified TestProbe2PortBareRemovalDropsBothProtocols: bare 53! drops tcp and udp; 53/udp! drops only udp). On the add/remove collision itself ports agree with -i.

One rule: merged = (prior − removals) + adds. Removals resolve against PERSISTED STATE ONLY; adds are applied afterwards and always win.

Corollary R5a — a removal an add put back is SHADOWED, and a shadowed removal must be withheld from the TEARDOWN, not merely reported. dropped is not a log line: it is the input to DisconnectRemovedIntegrations, which runs after a successful ExecuteCompose and keys on the plug alone (G2). So -i sso=authentik -i sso! deployed the new edge and then deleted the integration row the deploy had just written. Nothing re-Connects on any deploy path — the only writers are the recover path and AutoConnectPending, which has no callers — so the divergence between the compose, the sidecar and shc integration ls was permanent.

The merge therefore partitions what the removals took into dropped (torn down) and shadowed (reported only). The same guard is repeated one layer down for the case the merge cannot see: an UNRESOLVED INTENT carries no plug, so the plug it tears down is recovered from the DB rows inside DisconnectRemovedIntegrations itself, which is handed the merged set and skips any plug in it.

Three reasons, in order of weight:

  1. It is the only rule under which the two halves of a converge cannot fight. The provider emits adds and removals in one argv by construction (buildDeployUpdateArgv, app_ssh.go:211), and its diffs are prior-vs-plan. A token can therefore appear in both lists only when prior and plan disagree — which is exactly the moment the plan (the add) must win.
  2. It makes the invocation order-independent. Under removal-last, --host a.com! --host a.com and --host a.com --host a.com! are the same command (both delete) but neither reads that way. Under add-wins both mean “a.com is present afterwards”, which is what a declarative caller means and what a human reading the line assumes.
  3. It composes with R4. A removal overridden by a same-line add and a removal that matched nothing become the same reported event class, so an operator who typed both by accident is told, rather than getting a silent delete under one flag and a silent keep under another.

The honest counter-argument: removal-last is the more conservative reading of an ambiguous command line — “if the operator asked for a delete, delete” — and it is what --var does today, which is the flag most likely to be typed by hand. But conservative-for-a-human is destructive-for-a-converge, and the converge is the caller that actually emits both halves. R4’s report closes the human gap.

#changefile:line
5.1MergeHostSpecs: move the removal filter to run over prior before the additions loop. Build merged from (prior − removeSet), then apply resolvedAdditions through the same index.modules/app/host_spec.go:652-714 — move :693-700 above :676
5.2MergeVarOverrides: move the delete() loop between the prior copy and the adds overlay.modules/app/var_overrides.go — move :18-20 between :14 and :15
5.3MergePortMappingsno change, already compliant.modules/app/port_mappings.go:297-332
5.4MergeIntegrationSpecs — order already compliant; it gains the R5a partition, returning (merged, dropped, shadowed, unmatched, err).modules/app/install_async.go
5.5DisconnectRemovedIntegrations takes the merged set as kept and skips any recovered plug that is in it.modules/app/install_async.go

Changes behaviour? --host and --var: yes. --port and -i: no — this writes down what they already do.


6. Does a removal token name the identity, or the full spec?

Section titled “6. Does a removal token name the identity, or the full spec?”

6.1 What today’s code answers — three different answers

Section titled “6.1 What today’s code answers — three different answers”
flagtodayevidence
--portidentity only, extras refused (X400991PRTIDN)ParseCLIPortRemovals + portRemovalNamesATarget, port_mappings.go:124-156; probe → 80:web refused
--hostfull spec parsed, selector silently discardedMergeHostSpecs:693-700 parses with ParseHostSpec and keys on hostSpecIdentity (host_spec.go:490). Verified: api:8080@a.example.com! deleted the web:80@a.example.com route, no error, no warning
-iplug=provider refused (errUnintegrationToken, install_async.go:650), but provider@env accepted as a narrowing qualifier
--vardegenerate — the token is the identity

Yes, it is a general rule: you remove by the identity you would replace by, and a token carrying anything beyond that identity is REFUSED, not stripped.

Per surface:

  • --port — unchanged. It already is the rule, and ParseCLIPortRemovals’ doc comment (port_mappings.go:105-123) already states the argument correctly: there is at most one mapping per identity, so a target segment can never select anything — it can only agree or be wrong.

  • --host — tightened. Identity is hostname[/path]. A removal token carrying a service[:port]@ selector is refused with a new X400974HSTIDN, worded as the mirror of X400991PRTIDN. The check is syntactic, like portRemovalNamesATarget: web:80@a.com! is refused even when web:80 is exactly what is persisted, because “agrees or is wrong” is not a selector.

  • -i — tightened, and R6 finally holds of it without a carve-out. Revision 1 kept the provider / connection / provider@env tokens and classified them as “lookups, not identities” — a non-plug token resolved to whatever it matched, and the removal was then a plug removal. That carve-out is precisely where the replay defect lived (§14.1): a lookup’s result is a function of state, so its meaning was too.

    Revision 2 deletes the concept. Every -i removal token names an identity. A bare token names the identity of a WIRED edge (its plug); @conn and stack@env name the identity of an UNRESOLVED INTENT (its provider reference, R1b). Neither addresses anything by VALUE, so R6 reads the same on all four flags with no special case. What goes away is the reading under which provider! removes an edge at all.

6.3 Provider-visible blast radius of applying R6 to --host: none

Section titled “6.3 Provider-visible blast radius of applying R6 to --host: none”

Verified against the release-retry provider:

  • buildRemoveDeploymentHostSpec returns in.Hostname + hostPathSuffix(in.Path) — no selector, ever. Its own doc says so, and TestBuildRemoveDeploymentHostSpec asserts it deliberately ("the remove identity is hostname[/path] — selectors never ride removals").
  • diffRemovedDeploymentHosts keys on routeKey() = hostname|path, so a selector flip on the same route key is an in-place replace and never emits a removal at all — asserted by TestDiffRemovedDeploymentHostsByRouteKey.
  • buildDeployUpdateArgv (app_ssh.go:211) emits --host removalToken(h) where h came from buildRemoveDeploymentHostSpec.

The provider therefore cannot emit a selector-carrying host removal, and refusing them cannot break a converge. The blast radius is CLI-surface only: an operator who has been typing --host web@a.com! gets an error where they previously got a deletion that was correct only by accident. The same is true of the -i tightening, by G6.

#changefile:line
6.1New ParseCLIHostRemovals(raw []string) ([]HostSpec, error) mirroring ParseCLIPortRemovals: parse, then refuse any token containing @.new fn in modules/app/host_spec.go, pattern at modules/app/port_mappings.go:124-156
6.2MergeHostSpecs consumes the screened removals instead of re-parsing raw.modules/app/host_spec.go:693-700
6.3Thread the raw removal strings to the screen the way RemovePortsRaw is threaded.modules/app/install_async.go:412-413 (RemovePortsRaw precedent), :427
6.4New code X400974HSTIDN{token, identity} — “--host removal {token} names a service or port selector, a route is removed by its identity {identity} only”.modules/app/errors_edges.go
6.5Update the shc update long help, which currently documents the port rule but not a host rule.modules/app/commands/update.go:46-52

Changes behaviour? --host: yes. --port: no (confirms). -i: indirectly, via R3.


ValidateRequiredPlugs (plug_required.go:46) is called at install_runner.go:517 with gateConnected built at :504-516:

gateConnected := integrationsCtx // the RESOLVED (merged) edge set
if persisted := PersistedConnectedPlugs(priorState.Integrations, ...); len(persisted) > 0 {
// union: every plug the PRE-removal sidecar connected
}

priorState is the manifest read at install_runner.go:198-205before any removal. On the update path opts.Integrations is already the merged set (install_async.go:303:416), so a removed edge is correctly absent from integrationsCtx. But its plug is still in priorState.Integrations, so the union puts it back and the gate passes.

Net: shc update myblog -i sso! on an app whose sso.plug.yaml declares required: true succeeds and deploys a stack with no sso edge. The gate never fires. (Established by reading; not executed end-to-end — it needs a real deployment dir with plug files, which the probe harness does not build.)

Yes: the gate must see the merged, about-to-be-deployed edge set. A gate evaluated against state the deploy is about to overwrite is not a gate.

But the union must be replaced by a selection, not deleted — see 7.3.

UNBLOCKED (2026-07-29): the flat-required: true sharp edge is moot — the OQ3 catalogue audit ran and found zero flat declarations (see OQ3, resolved), so R7 needs no catalogue migration, no bypass flag, and no owner ruling; only the implementation in §7.4 is still owed.

7.3 What breaks if it does, stated exactly

Section titled “7.3 What breaks if it does, stated exactly”

The union is not gratuitous. It is the only thing keeping clone / node-move / the recreate seams from tripping the gate: those callers reach ExecuteCompose without restating -i, so opts.Integrations is empty, integrationsCtx is empty, and without the persisted union every required-plug app would fail to clone or move. The comment at install_runner.go:498-503 says this in as many words. Deleting the union trades one silent hole for a loud, estate-wide regression.

The fix is to make the union removal-aware — or, more precisely, to make it a selection on “did the caller state intent?”:

  • Caller stated the edge set (install; update, where install_async.go:303 has already folded prior ⊕ adds ⊖ removals) → the gate sees exactly integrationsCtx. No union.
  • Caller did not state it (clone, move, recreate: no -i, no removals) → the gate falls back to PersistedConnectedPlugs(priorState.Integrations, …).

Mechanically that is one condition: thread the raw removal tokens into ComposeRunOptions alongside Integrations, and gate the union on len(opts.Integrations) == 0 && len(opts.RemoveIntegrations) == 0. A bare shc update myblog 2.4.1 still takes the “stated” branch, because the fold at install_async.go:303 has already populated opts.Integrations from the sidecar — which is correct, and is exactly why the fold belongs where it is.

Three further things break, and they should be accepted rather than worked around:

  1. A required plug becomes genuinely undisconnectable via -i sso! alone. That is the point, but it has a sharp edge. For the catalogue’s canonical conditional form (required: "{{ not vars.postgres.hostname }}") the escape is one invocation, because resolvedVars is the merged var set (install_runner.go:475-486) and the gate evaluates the condition against it: shc update myblog -i sso! --var oidc.issuer=https://… passes. For a flat required: true there would be no escape at all — but that branch is vacuous in the shipped catalogue: the audit this item asked for has run and found zero flat declarations (every required: is conditional or false — see OQ3, resolved). No bypass flag is needed, and the recommendation against ever adding one (it would be a --force on a correctness gate) stands for any future flat declaration.
  2. The gate becomes reachable from --var alone. An update that changes only a var can flip a conditional required: from false to true and now fail, where the pre-removal union would have masked it if the plug had ever been connected. That is correct behaviour and a new failure mode for var-only updates. It is loud and coded (X400994PLGREQ), so it is recoverable, but it will surprise someone.
  3. Cost of ordering. The gate runs after ResolveIntegrations, so an update that will fail the gate still pays for the resolve of the remaining edges first. Already true; not made worse.
#changefile:line
7.1ComposeRunOptions gains RemoveIntegrations []string (raw tokens), mirroring RemoveVars / RemoveHosts / RemovePortsRaw.modules/app/install_runner.go (ComposeRunOptions decl); populated at modules/app/install_async.go:399-447
7.2Gate the union on len(opts.Integrations) == 0 && len(opts.RemoveIntegrations) == 0.modules/app/install_runner.go:504-516
7.3Update PersistedConnectedPlugs’ doc comment: it is the unstated-caller fallback, not an unconditional union. Its bare-edge plug-inference branch (:102-108) becomes dead under R1a and should be deleted with removedEdgePlugNames’s twin.modules/app/plug_required.go:85-124

Changes behaviour? Yes — a removal of a required plug now fails the update.


An integration edge is identified by its consumer plug, within (tenant, stack, environment). Everything else about the edge — provider stack, provider tenant, provider environment, provider connection — is its value.

  1. No input can create a plugless edge. A persisted row that carries no plug is an unresolved intent, not an edge: it is inert, and its identity is its provider reference.
  2. A bare -i <provider> is sugar: it is resolved to a plug before it is anything else, and refused if it cannot be resolved to exactly one.
  3. merged = (prior − removals) + adds, on all four flags. Removals see persisted state only. Adds win — and a removal an add put back is reported and withheld from the teardown.
  4. You remove by the identity you would replace by, and which identity a token names is decided by the token’s syntax, never by what happens to match: a bare token is a plug, @name and name@env are the provider reference of an unresolved intent. A token carrying more than an identity is refused, never stripped.
  5. A removal that matches nothing is a reported no-op. A removal that matches more than one is a hard error. (A no-op cannot destroy anything; a guess can.) Rule 4 is what makes rule 5 safe: a no-op is only harmless if the token could not have meant something else.
  6. The required-plug gate sees the set that is about to be deployed.

An implementer can follow this mechanically. P = persisted edge set, A = this invocation’s adds, R = this invocation’s removal tokens.

#inputconditionactioncode
A1-i plug=providerupsert on plug
A2-i plug=provider@envupsert on plug, value carries env
A3-i plug=@connupsert on plug, value carries connection
A4-i PROVIDER (magic)resolves to 1 plugrewrite to A1 with tenant=master, upsert
A5-i providerinference → exactly 1 plugrewrite to A1, upsert
A6-i providerinference → 0 plugsrefuseX400973INTPLG
A7-i providerinference → >1 plugrefuse (OQ2)X400973INTPLG
A8two adds, same pluglast wins, reportX200970RMVNOP-class advisory
A9-i @conn (bare, no plug)refuse — a connection is not a deployed stack, ships no sockets, and the mint needs the plug to pick a capabilityX400973INTPLG

A9 was missing from revision 1’s table, so the code refusing it was neither required nor forbidden. It is required. OWED: the refusal is implemented on update only — install (register.go, concurrency-owned at the time of writing) hands -i straight to ComposeRunOptions.Integrations, so install still accepts tokens update refuses, and persists them plugless. R1a is a rule about inputs and it is only enforced on one of the two verbs.

The class is read off the token’s SHAPE first; the live set is consulted only to count matches.

#token shapeclassmatchesactioncode
D1plug!plug1remove the wired edge on that plug
D2plug!plug0no-op, report — including when the token is also a live provider stack or connection nameX200970RMVNOP
D3@conn!intent1remove the unresolved intent on that connection
D4@conn!intent0no-op, report — a WIRED connection edge is not in this class; it is plug!X200970RMVNOP
D5provider@env!intent1remove the unresolved intent on that provider, in that resolved environment
D6provider@env!intent0no-op, report — a WIRED edge on that provider is not in this classX200970RMVNOP
D7anyany>1refuse — invariant violation, not an operator errorX400972INTAMB
D8provider! (bare)plugit is D1/D2. There is no bare provider-stack removal.
D9plug=provider!refuse (unchanged)X400002VALINP
D10! / empty / @ / provider@refuse (unchanged)X400002VALINP

The environment in D5 is the one the row RESOLVES in (ProviderEnvironmentFor), so an intent that persisted ProviderEnvironment: "" in a prod deployment is provider@prod!. This also settles the gap revision 1 left open — under it, an edge created as -i mail=relay in prod persisted "" and was NOT removable by relay@prod!, so §6.2’s “you remove by the identity you would replace by” did not hold for the environment qualifier.

9.3 Collisions and precedence (all four flags)

Section titled “9.3 Collisions and precedence (all four flags)”
#situationrule
C1X and X! in one invocationremoval applies to P only; add applies after and wins. Report the shadowed removal.
C1a-i only: the shadowed removalwithheld from the teardown list, not merely reported — dropped feeds DisconnectRemovedIntegrations, which runs after the deploy and keys on the plug (R5a).
C2X! matches nothing in Pno-op; report X200970RMVNOP{flag, token, live}.
C3X! matches >1 in P-i only (the other three are 1:1 by identity): refuse X400972INTAMB. Unreachable from a merge-produced set.
C4--strict-removals setC2 becomes a hard error carrying the same payload.
flagidentityaccepted removal tokenrefusedcode
--varkeykey!comma/semicolon/whitespace in keyX400002VALINP (unchanged)
--hosthostname[/path]hostname[/path]!anything with @X400974HSTIDN (new)
--port[conn@]external[/proto]sameany : after the conn prefixX400991PRTIDN (unchanged)
-iplug (wired edge) / provider ref (unresolved intent)plug!, @conn!, provider@env!anything with =; half-written @ formsX400002VALINP (unchanged)

-i is the one flag whose token shapes name two different identity classes, and that is not an exception to R6 but a consequence of G8: the two classes of persisted row have two different identities, so they have two different spellings. Neither spelling addresses anything by value.

#calleropts.Integrationsopts.RemoveIntegrationsgate reads
G1installstatedemptyintegrationsCtx
G2update (any)folded set (never empty when P non-empty)may be non-emptyintegrationsCtx
G3update, -i x! emptying the last edgeemptynon-emptyintegrationsCtx (empty) → gate fires
G4clone / move / recreateemptyemptyPersistedConnectedPlugs(priorState)

Every row distinguishes a combination the rules distinguish. Suggested homes: integration rows → modules/app/integration_unintegrate_test.go; host rows → modules/app/host_spec_path_test.go; port rows → modules/app/port_spec_marker_test.go; parse rows → modules/app/commands/removal_marker_test.go; gate rows → a new modules/app/plug_required_gate_test.go.

#priorinvocationexpect
T1{sso→keycloak}-i sso=authentik{sso→authentik}, one edge, dropped=[]
T2{cache→postgres, db→postgres}bare updateboth survive, unchanged
T3{mail→relay@prod}-i mail=relay@stagingone edge, env flipped
T4{s3→@aws-primary}-i s3=@aws-secondaryone edge, connection flipped
T5{s3→minio}-i s3=@aws-primaryone edge, kind flipped stack→connection
T6sidecar with Plug:""bare updatesidecar rewritten with the resolved plug (migration)
T7sidecar with Plug:"", plug unresolvablebare updatecoded warn, edge dropped, deploy succeeds

10.2 Bare add (R2) — the R5 regression suite

Section titled “10.2 Bare add (R2) — the R5 regression suite”
#priorinvocationexpect
T8{cache→postgres, db→postgres}-i postgresexactly one edge repointed; the sibling byte-identical. Regression guard for R5.
T9{sso→keycloak}-i keycloakplug sso preserved in the persisted spec
T10{}-i postgres, consumer declares one matching plugedge on that plug
T11{}-i postgres, consumer declares none, provider ships one socketedge on the socket stem, plug recorded
T12{}-i postgres, provider ships 2 sockets both matching consumer plugsX400973INTPLG
T13{}-i postgres, no inference possibleX400973INTPLG
T14{}-i sso=keycloak -i sso=authentikone edge on authentik + advisory report
T15{}-i postgres -i postgresone edge (dedupe)
#priortokenexpect
T16{sso→keycloak}sso!removed
T17{cache→postgres, db→postgres}postgres!no-op + report — a bare token is a plug and neither edge is on plug postgres; the report lists both, which names the tokens that work
T18{s3→@aws-primary, backup→@aws-primary}@aws-primary!no-op + report — both are WIRED edges, named s3! and backup!
T19{s3→@aws-primary, backup→@aws-primary}aws-primary!no-op + report (bare ⇒ plug)
T20{s3→@aws-primary}s3!removed
T21{keycloak→authentik, sso→keycloak}keycloak!the edge on PLUG keycloak removed; and sso! still removes the other (addressability, half 1 and 2)
T21a{sso→keycloak} (i.e. after T21)keycloak!no-op + report. THE REPLAY GUARD. Under revision 1 this removed sso→keycloak — a different edge, for a command that named neither it nor its provider.
T21b{postgres→postgres, postgresMas→postgres}postgres! twicerun 1 removes plug postgres; run 2 is a no-op + report. The catalogue shape (apps/matrix, apps/postiz).
T22{Plug:"" →relay@prod}relay@prod!removed (the intent form)
T23{mail→relay@prod}relay@prod!no-op + report — a WIRED edge is not in the intent class; it is mail!
T24{Plug:"" →relay} in env prodrelay@prod!removed — the qualifier is the RESOLVED environment
T24a{Plug:"" →relay} in env prodrelay@dev!no-op + report
T24b{Plug:"" →postgres, cache→postgres}postgres@default!the INTENT removed, cache untouched. The defect-3 guard: revision 1 had no token for the intent at all.
T24cevery row of a mixed setits own printed spellingeach removes exactly itself, and no row is reachable by another row’s token (the addressability proof, executed)
T25{sso→keycloak}plug=provider!X400002VALINP (unchanged)
T26{sso→keycloak}wire {"unintegrations":["zzz=keycloak"]}X400002VALINP (unchanged; CLI/wire parity)
T26a{→x, →x} (hand-edited sidecar)x@default!(revised in revision 3) through the MERGE the duplicate collapses last-wins and the token addresses the survivor (TestMergeCannotProduceTwoRowsOnOneIdentity); handed directly to SplitRemovedIntegrations it refuses X400972INTAMB (TestAmbiguousRemovalIsAnInvariantGuard)
T26b(rev 3) {→relay, →relay@prod} in env prod; magic-keycloak beside {→keycloak@default}; {sso→a, sso→b}bare update, then the row’s own tokenmerge holds ONE row per identity, last wins, and the token addresses it — TestMergeCannotProduceTwoRowsOnOneIdentity, TestOneIdentityFunctionDecidesForEveryCaller
T26c(rev 3) every row class incl. {}, {@weird→relay}, {mail@relay→relay}its printed spellingprint totality: a token-shaped spelling addresses exactly its row; the unaddressable rows print PROSE that names plug+provider and never reads as a token — TestPrintedSpellingIsFollowable, TestRowNamingNoProviderIsDroppedOnRead, TestUnaddressablePlugNameIsRefused

10.4 Unmatched removal (R4) — one row per flag, both modes

Section titled “10.4 Unmatched removal (R4) — one row per flag, both modes”
#flagpriortokendefault--strict-removals
T27--var{a:1}nope!no-op + X200970RMVNOPerror
T28--hosta.example.comabsent.example.com!no-op + X200970RMVNOPerror
T29--port80809999!no-op + X200970RMVNOPerror
T30-i{sso→keycloak}absent!no-op + X200970RMVNOPerror
T31-i{} (no edges at all)anything!no-op + report, live renders (none)error
T32anythe warning text carries both the token and the live set
T33converge replayafter a successful -i s3! converge, replay the identical argvsucceeds, reports one no-operror

T33 is the live-hazard regression guard. It fails today.

#flagpriorinvocationexpect
T34--host{a.com→web:80}--host api:8080@a.com --host a.com!a.com present, selector api:8080
T35--host{a.com→web:80}--host a.com! --host api:8080@a.comidentical to T34 (order independence)
T36--var{k:old}--var k=new --var k!k=new
T37--var{k:old}--var k! --var k=newk=new
T38--port{8080}--port 8080:web:80 --port 8080!add survives (confirms today)
T39-i{sso→keycloak, auth→authentik}-i auth=keycloak -i auth!auth→keycloak wired, the removal shadowed
T40all fourthe shadowed removal is reported
T40a-i{sso→keycloak}-i sso=authentik -i sso!merged sso→authentik, dropped EMPTY, shadowed = the old edge. The teardown must not delete the row the deploy just wrote (R5a).
T40b-iintent {→kc-main} dropped, merged {sso→kc-main}DisconnectRemovedIntegrations(dropped, kept)the sso row survives — the plug recovered from the DB rows is skipped because the deploy wired it
#flagtokenexpect
T41--hostweb:80@a.example.com!X400974HSTIDNfails today, deletes silently
T42--hostweb@a.example.com!X400974HSTIDN
T43--hosta.example.com/api!removes only /api; root route survives
T44--hosta.example.com!removes only the root route; /api survives
T45--port80:web!X400991PRTIDN (confirms)
T46--port53!removes tcp and udp (confirms)
T47--port53/udp!removes udp only (confirms)
T48providerbuildRemoveDeploymentHostSpec on a selector-carrying hostInputstill bare hostname[/path] — proves zero provider blast radius
T49providerround-trip: every diffRemovedDeploymentHosts output passes ParseCLIHostRemovalsno provider-emitted removal is ever refused
T50providerround-trip: every diffRemovedIntegrations output resolves to exactly one edge under D1provider converges never hit X400972INTAMB

T49 and T50 are the cross-repo contract tests. They are the cheapest possible insurance against another round of “the fix broke the provider”.

#app plug declpriorinvocationexpect
T51required: true{sso→keycloak}-i sso!X400994PLGREQpasses today
T52required: "{{ not vars.oidc.issuer }}"{sso→keycloak}-i sso!X400994PLGREQ
T53required: "{{ not vars.oidc.issuer }}"{sso→keycloak}-i sso! --var oidc.issuer=https://xsucceeds
T54required: true{sso→keycloak}bare updatesucceeds (fold keeps the edge)
T55required: true{sso→keycloak}clone (no -i, no removals)succeeds via the persisted fallback — regression guard for the union
T56required: true{sso→keycloak}node-movesucceeds
T57required: "{{ not vars.pg.hostname }}"{} never connected--var pg.hostname= (blanking)X400994PLGREQ — new failure mode, assert it is coded and names the plug

11. Which answers CHANGE behaviour, and which write down what already happens

Section titled “11. Which answers CHANGE behaviour, and which write down what already happens”
Qverdictdetail
1 — identityCHANGES, partly confirmsRepoint-by-plug already works (verified T1-equivalent). What changes: the plug is backfilled into persisted state, sameIntegrationEdge’s provider-ref fallback is narrowed to the both-plugless case (where it is now load-bearing: it is what makes an intent’s identity unique), and a plugless row is reclassified as an intent rather than an edge.
2 — bare addCHANGESThis is the R5 fix. Today a bare add silently destroys a plug-qualified sibling; it becomes a plug-resolved repoint, or a refusal.
3 — bare removalCHANGES; revision 2 REVERSES revision 1’s answerRevision 1 kept the plug→connection→stack total order and added a count-of-one rule. Revision 2 replaces the order with syntax-classing: a bare token is a plug, always. The handle order was not merely suboptimal, it was unsafe under replay (§14.1).
4 — unmatched removalCHANGES all four-i: hard error → loud no-op (retires X400971INTNOF). --var/--host/--port: silent no-op → loud no-op. Largest change; needs a ruling. The terminal-visibility half is owed (OQ4).
5 — precedenceCHANGES 2, confirms 2, plus R5a--host and --var flip from removal-wins to add-wins. --port and -i already comply. R5a (a shadowed -i removal is withheld from the teardown) is new in revision 2 and is a correctness fix, not a report.
6 — token shapeCHANGES --host and -i, confirms --port--host stops silently discarding selectors and refuses them. -i is tightened: revision 1’s “lookups, not identities” carve-out is deleted, and every token names an identity.
7 — required-plug gateCHANGES — NOT IMPLEMENTEDGate should read the merged set for stating callers, with the persisted union as the unstated-caller fallback. No longer blocked — OQ3 is resolved (zero flat required: true in the catalogue) — but not landed: ComposeRunOptions has no RemoveIntegrations field and the union is still unconditional. shc update <app> -i sso! on a required plug still succeeds.

Net: of the seven, five change behaviour materially, one (§5) changes two of four surfaces, and none of the seven is purely descriptive. That is the honest measure of how under-specified the subsystem was — there was no rule to write down.


12. New error codes (all in a new modules/app/errors_edges.go)

Section titled “12. New error codes (all in a new modules/app/errors_edges.go)”
codeclassparamsmeaning
X400972INTAMB4xxtoken, matchesintegration removal addresses more than one persisted row, which share one identity and so cannot be told apart by any token; nothing removed, a plain shc update normalises the state. Invariant check — unreachable from a merge-produced set BY CONSTRUCTION (normalizeIntegrationIdentities), reachable only by handing a hand-edited set directly to SplitRemovedIntegrations (§14.5)
X400973INTPLG4xxprovider, sockets, plugs-i <provider> cannot be resolved to exactly one consumer plug
X400974HSTIDN4xxtoken, identity--host removal names a service/port selector; remove by identity only
X200970RMVNOP2xx warnflag, token, livea removal matched nothing; nothing was removed
X200969INTDRP2xx warntenant, stack, edge, reasona persisted row’s plug could not be inferred and the row was dropped from this deploy (the read-side migration)

Retired: X400971INTNOF (factory errors.go:205, registry row errors.go:1675) — replaced by X200970RMVNOP. Leave a memorial comment recording that the hard failure was tried, what it bought (a typo can no longer exit 0 silently) and what it cost (T33), so the next round does not re-derive it from first principles.


OQ1 — R4 is a reversal and needs an explicit ruling. Retiring X400971INTNOF’s hard failure undoes something a prior review round asked for on purpose, and errors.go:190-204 argues the case for it well. My recommendation is the loud no-op plus --strict-removals, on the ground that a failing removal makes a correct command fail on its second run and takes an entire converge down with it (T33). But this is a judgement about which failure mode the estate would rather live with, and it is the owner’s to make. If the ruling is “keep the hard error”, then R4 becomes “error on all four” and --var/--host/--port get X4009xx codes instead — which is internally consistent, and which I would still prefer over today’s four-way split.

OQ2 — should ambiguous plug inference refuse, or pick with a warning? — SETTLED, refuse. Rule A7 refuses when a provider ships two sockets that both match consumer plugs. The catalogue sweep the question asked for has since been run: the only multi-socket provider in apps/ is mailcow {imap, smtp}, and no app declares an imap plug, so sockets ∩ plugs is 0 or 1 everywhere. Blast radius nil; refusing stands.

OQ3 — how many plug files declare flat required: true? — RESOLVED, ZERO (moot). Measured against the catalogue at origin/main (2026-07-29): 54 plug files under apps/ declare required: — 48 conditional Jinja, 6 required: false — and the ce built-ins add 2 more conditionals. No flat-truthy spelling exists anywhere (true/yes/on/1, bare or quoted, all swept). The undisconnectable case of 7.3.1 is therefore vacuous, no catalogue migration is needed, and per this question’s own decision rule R7 lands first. (The doc’s “48 files” was plug_required.go’s count when written; the catalogue has since grown to 54 — all still conditional or false.)

OQ4 — do the R4 warnings ride the operation observer, or only the daemon log? — STILL OPEN, and the code answers “only the daemon log”. operations.StepObserver (core/operations/observer.go:107) exists and the update path threads one (RunUpdateWithObserver) — the observer is even in scope at the emit point — but operations.DeploymentPayload is a deployment progress snapshot (services, progress counts, status, phase) with no field an advisory sentence can ride. Emitting one needs a new frame kind and the CLI-side decoders in commands/operation_live.go / commands/daemon_client.go taught to render it, which is a cross-surface change this round did not take.

Consequence, stated plainly because it is a real cost: shc update myblog -i keyclaok! exits 0 having printed nothing at the terminal. corelog.Warn fans out to slog, the event bus and the notifier, none of which the CLI renders. For -i specifically this is a net REGRESSION in operator visibility against the retired X400971INTNOF, which did surface. Making the no-op visible was R4’s whole justification for retiring the hard error, so this is the one place where the implementation does not yet earn the reversal. Interim: --strict-removals.

OQ5 — a repoint orphans a connection-backed edge’s minted credential, and this model does not fix it. Out of scope but adjacent, and newly sharper under R1. Verified (TestProbe2PlugRepointAcrossKinds): repointing s3=@aws-primary to s3=minio returns dropped=[], so DisconnectRemovedIntegrations (install_async.go:788) never sees the old edge and revokeConnectionEdges — wired only into ExecuteComposeDown — never runs. The scoped IAM user survives. DisconnectRemovedIntegrations’ own doc (:764-786) already admits leg 4 is unwired for removals; a repoint does not even reach that code path. If R1 lands, the cheap fix is to make MergeIntegrationSpecs return replaced-value edges in a further list (repointed) so the same teardown hook can see them — the merge now already partitions dropped from shadowed (R5a), so the shape exists. Worth deciding now, because R1 is the change that makes “the old value of this edge” a nameable thing for the first time.


14. Revision history — what each revision got wrong

Section titled “14. Revision history — what each revision got wrong”

Revision 1 was implemented (six of seven rules, mutation-checked) and then reviewed against the running code; three of its claims did not survive (§14.1–14.3). Revision 2 fixed those and was itself broken by a second adversarial review in two places (§14.5–14.6). All are recorded here rather than quietly edited away, because the pattern in each is the reason this subsystem produced one critical per round for five rounds: a fix that is correct in isolation, composed with another fix that is correct in isolation — and, in revision 2’s case, one predicate re-implemented three times.

14.1 The handle order and the reported no-op were jointly destructive

Section titled “14.1 The handle order and the reported no-op were jointly destructive”

R3 (revision 1) fixed the handle order — plug, then connection, then provider stack, first match wins — and R4 made an unmatched removal a loud no-op so a converge could retry safely. Neither is wrong alone. Together:

prior: {postgres→postgres, postgresMas→postgres} # apps/matrix, verbatim
run 1: shc update matrix -i postgres! → removes plug `postgres` ✅
run 2: shc update matrix -i postgres! → plug handle misses,
FALLS THROUGH to the stack handle,
removes plug `postgresMas` ❌

The second run is the exact case R4 exists to make safe — an operator re-running, or a converge retrying after a lost state write — and it destroys an edge the command never named. --strict-removals cannot catch it, because the token matched. Verified end to end through RunUpdateWithObserver, which logs integration edge removed … plug=postgresMas for a command that said postgres.

The model proved that every edge is ADDRESSABLE and never asked the dual question: what does a token MEAN once the plug it named is gone? Under a total order the answer is “something else”, so a removal token’s meaning is a function of how much state you already have — which is the property this document rejects for adds in §2.2, in as many words, and then built into removals without noticing.

Revision 2’s answer is §3.2: the identity class is read off the token’s SYNTAX. postgres! means the plug postgres on an empty deployment, on a one-edge deployment and on a fifty-edge deployment.

The catalogue makes this concrete, not hypothetical: apps/matrix declares postgres.plug.yaml AND postgresMas.plug.yaml, both on socket: postgres, and builtin/postgres ships postgres.socket.yaml, so the provider stack is named postgres too. apps/postiz is the same shape with postgresTemporal.

14.2 The addressability proof had a false premise

Section titled “14.2 The addressability proof had a false premise”

§3.2 (revision 1) proved “every edge remains addressable” in two lines from R1a: every persisted edge carries its plug, plugs are unique, therefore the plug handle names exactly one.

R1a is not deliverable as a property of the sidecar. Inference reads the provider’s app directory; that directory is absent when the provider is not deployed on this node; and the model’s own migration rule (§1.4 change 1.3) blesses leaving those rows alone, precisely because dropping them would silently disconnect an edge that heals when the provider returns. Ambiguous inference is left alone for the same reason. So plugless rows exist, {→postgres, cache→postgres} is reachable, and under revision 1 no token named the plugless one — while the ambiguity error it raised for postgres! advised “name one of them by its plug”, and printed, as that candidate’s spelling, the ambiguous token itself.

Revision 2 replaces the premise instead of asserting it. R1a becomes a rule about INPUTS (nothing can create a plugless edge) and R1b names the residue: a plugless row is an UNRESOLVED INTENT whose identity is its provider reference. The proof is then unconditional, because it rests on two disjoint row classes and two disjoint token shapes rather than on a property the code cannot guarantee (§3.2).

14.3 C1 treated a shadowed removal as a reporting problem

Section titled “14.3 C1 treated a shadowed removal as a reporting problem”

Revision 1’s C1 said: a removal an add put back is a no-op with a matched token, so report it.

dropped is also the input to DisconnectRemovedIntegrations, which runs AFTER ExecuteCompose and keys on the plug alone. So -i sso=authentik -i sso! deployed the new edge and then deleted the integration row the deploy had just written. Nothing re-Connects on any deploy path — the only writers are daemon/services_recovery.go (recover) and AutoConnectPending, which has no callers — so shc integration ls and the recover path stayed wrong permanently.

Revision 2 adds R5a: the merge partitions dropped from shadowed, and the teardown sees only dropped. The same guard is repeated inside DisconnectRemovedIntegrations for the intent case, where the plug is recovered from the DB rows and so cannot be compared before the call.

  • §9.1 had no row for a bare -i @conn ADD, so the code refusing it was neither required nor forbidden. It is now row A9, and it is required. (The install half remains owed — see A9’s note.)
  • §6.2’s “you remove by the identity you would replace by” did not hold for the environment qualifier: an edge created as -i mail=relay in prod persists ProviderEnvironment: "" and was NOT removable by relay@prod!. Revision 2 compares the RESOLVED environment (D5), which is what makes an intent nameable at all.

14.5 Revision 3 — the merge really could produce the unfollowable ambiguity

Section titled “14.5 Revision 3 — the merge really could produce the unfollowable ambiguity”

Revision 2 claimed (§3.2): “a token matching more than one row means the persisted set holds two rows with one identity, which the merge cannot produce.” An adversarial review produced it, from the merge, with no hand-editing:

prior: {→relay}, {→relay@prod} deployment environment: prod

The first intent was written -i relay and persists ProviderEnvironment: ""; the second was written -i relay@prod and persists it literally. sameIntegrationEdge compared the value fields LITERALLY ("" != "prod"), so the merge kept both rows. matchesIntegrationRemoval compared the RESOLVED environment (ProviderEnvironmentFor, per D5), so the token relay@prod! addressed both. And formatIntegrationSpec also printed the resolved environment, so both candidates printed as relay@prodX400972INTAMB listed two identical spellings and advised “name one of them by its own identity”, which was precisely what could not be done. Neither row was ever removable. The same shape reached through the magic table with no environment qualifier at all: ApplyMagicToken({→keycloak}) stamps ProviderTenant: master (which sameIntegrationEdge also compared literally, and which no removal token can spell) beside a plain {→keycloak@default}.

The defect was not a missing case; it was three hand-written comparisons holding two opinions of “one identity” — the merge’s equality, the removal grammar’s match and the print each re-derived it. The fix removes the possibility of drift rather than the instance of it:

  • integrationIdentity IS the model. It returns THE one token that addresses a row (plug / @connection / stack@resolved-env, tenant excluded — the grammar has no tenant slot, so a tenant-bearing identity would be an unnameable one). sameIntegrationEdge is now integrationIdentity(a) == integrationIdentity(b), matchesIntegrationRemoval is integrationIdentity(row) == token, and formatIntegrationSpec prints the identity’s own spelling. One function decides; nothing can drift from it. (TestOneIdentityFunctionDecidesForEveryCaller walks every pair and requires the three consumers to agree.)
  • normalizeIntegrationIdentities makes the invariant constructive. The merge normalises the PRIOR set to one row per identity (last wins, mirroring the render’s out[ic.Plug] = entry) BEFORE removals resolve against it — the ordering is load-bearing: normalise after the split and the operator still gets the unfollowable error — and again after the adds land. Every set MergeIntegrationSpecs returns holds exactly one row per identity. (TestMergeCannotProduceTwoRowsOnOneIdentity.)
  • The production read path reports what normalisation does. migratePersistedIntegrationPlugs collapses duplicate-identity sidecar rows with a warn naming the identity; the in-merge normaliser is the silent structural backstop for direct callers.
  • X400972INTAMB is reworded for the only state that can still reach it (a duplicate-identity set handed directly to SplitRemovedIntegrations): two rows with one identity cannot be told apart by ANY token, so the message stopped advising disambiguation and now says the state is inconsistent and that a plain shc update normalises it — the remedy that works.

The pattern to carry forward is revision 1’s, one level up: revision 1 shipped two individually correct rules that were jointly destructive; revision 2 shipped three individually plausible comparisons that were jointly inconsistent. Where several functions must agree, the model is only implemented when they are one function.

14.6 Two residual totality holes: the print handed out tokens that could not address the row

Section titled “14.6 Two residual totality holes: the print handed out tokens that could not address the row”

The same review found two row shapes for which formatIntegrationSpec’s one spelling was a well-formed token that could not address the row it described — an unfollowable message of the same class as the ambiguity error:

  1. The identity-less row ({} — no plug, no connection, no provider stack; reachable from a hand-edited sidecar or an install-verb write). It printed as @<environment> — a well-formed CONNECTION token that matches zero rows. It renders as nothing, writes no DB row, gates nothing, and NO token addresses it, so nothing could ever remove it either. Disposition: it has no identity, so it is not a row the system can hold — formatIntegrationSpec now prints (an edge naming no provider), migratePersistedIntegrationPlugs DROPS it with X200969INTDRP (quoting that spelling), and normalizeIntegrationIdentities refuses to pass one through the merge even when the migration has not run. (TestRowNamingNoProviderIsDroppedOnRead.)
  2. A wired edge whose plug the grammar cannot spell ({@weird→relay}, {mail@relay→relay} — reachable from a hand-edited sidecar or a catalogue *.plug.yaml stem carrying @). Syntax-classing silently REQUIRED plug names to carry no @ or =, and nothing checked: the row printed as @weird=relay / mail@relay=relay, whose identity half — the removal token, by the print’s own contract — classifies as a CONNECTION and a PROVIDER-REF token respectively, so it addresses nothing forever (and, beside a look-alike intent, addresses a DIFFERENT row). Disposition, in layers:
    • Refused at input (errIntegrationPlugName, X400002VALINP): ResolveIntegrationPlugs screens both the typed plug (-i mail@relay=postgres) and the INFERRED one (a mail@relay.plug.yaml stem), so no new such edge can be created. Quoting/escaping was rejected: plug names are file stems and Jinja context keys, where neither character works either. (TestUnaddressablePlugNameIsRefused.)
    • Declined at backfill: the migration never promotes an intent onto an unaddressable inferred name — the row stays an intent, which IS addressable (stack@env!).
    • Kept but honest for the persisted survivor: a wired row that already carries such a plug is NOT dropped (it is wired — dropping would be wiring loss) — the migration keeps it with a loud warn, and formatIntegrationSpec prints PROSE naming the plug and the provider ((an edge on plug "@weird", provider relay, which no removal token can spell — rename the *.plug.yaml stem …)) instead of the token-shaped lie. The look-alike intent beside it stays reachable by its own token, and the two print differently. (TestPrintedSpellingIsFollowable, which executes the totality claim: every printed spelling either carries a token that addresses exactly its row, or is prose — and prose appears only for rows no token can address.)

The rule both fixes instantiate: a spelling handed to an operator is a claim, and the print must not be able to make a claim the grammar cannot honour. Where the claim can be made true, print the token; where it cannot (no identity; an unspellable name), print prose that describes the row and hand out no token at all.

#owedwhy it is not done here
1R7, the required-plug gate (§7)was blocked on OQ3, which is now resolved-moot (zero flat required: true in the catalogue) — implementation-only. Not landed. shc update <app> -i sso! on a required plug still succeeds.
2R1a on the INSTALL verb (row A9)register.go / install_runner.go own the install input path; ResolveIntegrationPlugs is called only from RunUpdateWithObserver, so install still persists plugless rows and accepts -i @conn.
3change 1.2 — persist the resolved plug at installsame files. Consequence: R1a holds only after the first update.
4change 2.1 — one inference implementationintegrationPlugCandidates was added beside inferPlugFromProvider rather than replacing it (integration_render.go was owned by another workstream). They are pinned together by a test; two implementations remain divergeable by construction.
5OQ4 — the advisory frameDeploymentPayload cannot carry a sentence; see OQ4.
6X400971INTNOF’s factory + registry rowerrors.go is owned elsewhere. No raise sites remain; the memorial is in errors_edges.go.
7errors_edges.go specs in appErrorSpecsregistry.Errors panics on a second registration for one module, so the five codes reach the coded registry but not the module manifest.