Plan: 5-level config scoping + DNS target resolver (LB override + swarm multi-A)
Status: IMPLEMENTED on feat/config-5level-dns-target (phases 0–6, commits
60fcd18e..91e7415a). Remaining follow-ups: real-DNS e2e against a
provider-configured harness; worker-side docker swarm join port (until it
lands only the init node carries a swarm_node_id, so multi-A degrades
conservatively); wiring the core-config loader’s DBProvider impl (interface is
5-level-ready, the DB layer of the 13-layer loader remains an unfinished
pre-existing port phase).
UPDATE —
dns.targetwas later REFACTORED off scoped config (2026-07). The 5-level config ladder shipped and remains. Butdns.targetno longer resolves as a scoped config key: a subsequent phase moved the DNS target to per-deployment state (persisted in the deployment’s state, not the config store). There is nodns.targetconfig tier anymore — seeinternal/modules/app/dns_target.go. The scoped-dns.targetexamples below (the--scope environmentLB-override walkthrough) describe the now-retired config-tier resolution and are historical.
Why these are one change
Section titled “Why these are one change”A user asked for managed DNS to be able to point a --host at a load-balancer
target (A/AAAA/CNAME) instead of node IPs, overridable per tenant / environment /
deployment, with a config fallback, falling back to node IPs (swarm = one record per real
swarm node; compose = single record to the advertise address).
Resolving “what does the record point to” needs a scoped config lookup. But SHC config
is only 3-level (global/tenant/stack) while secrets/vault are 5-level
(base/tenant/stack/environment/deployment). Owner decision: promote config to the full
5-level ladder so config — and therefore every provider selector that reads config
(dns.provider, vault.provider, backup.*, storage.*) — becomes environment- and
deployment-aware. The DNS target resolver is then a thin consumer of that unified scoping.
The one scope ladder (single source of truth)
Section titled “The one scope ladder (single source of truth)”Narrowest wins. Matches the existing vault path cascade (internal/modules/vault/paths.go:114-140):
Deployment = tenant + stack + environment (most specific)Stack = tenant + stackEnvironment = tenant + environmentTenant = tenantGlobal/Base (broadest fallback)Stack outranks Environment (confirmed: vault CascadePaths lists Stack at :121 before
Environment at :126). The config koanf merge must load layers Global → Tenant →
Environment → Stack → Deployment (koanf = last-wins ⇒ that order yields the ladder above).
A new shared helper encodes this ONE ordering; vault, config, and DNS all consume it — no
drift. Proposed: internal/core/scope/ — type Scope enum + Cascade(ctx ScopeCtx) []Scope
(narrowest→broadest) + ScopeCtx{Tenant, Stack, Environment}. Vault paths.go and config
layering refactor onto it.
Decisions locked
Section titled “Decisions locked”| Decision | Choice |
|---|---|
--host target syntax | Auto-detect only: [service:]host[=target]. Type inferred by dns.ResolveTarget (service.go:63): IPv4→A, IPv6→AAAA, hostname→CNAME. No a:/cname: prefix. |
| Scope model | 5-level on both config + vault. Promote config now. |
| Precedence | Deployment > Stack > Environment > Tenant > Global. |
| IPv6 | Supported — AAAA via auto-detect; node-IP fallback emits AAAA for IPv6 advertise addrs; reconcile/delete key on (name, type) so A/AAAA/CNAME sets are independent. |
| Swarm node filter | A “real swarm node” = node row with non-empty swarm_node_id (NOT node.runtime, which is "compose" even for swarm members). Lighthouse/overlay-only nodes aren’t node rows → excluded for free. |
Parity vs net-new
Section titled “Parity vs net-new”- Parity (already ported): provider-selectable DNS create-on-deploy / delete-on-uninstall.
- Net-new: swarm multi-A, LB/CNAME/AAAA target override, scoped
dns.targetconfig, AND the 5-level config promotion (Python config was 3-level; only secrets were 5-level).
Invariants
Section titled “Invariants”dns.targetis config, not a secret — plaintext config store, never the vault. (Standing rule: secrets MUST be encrypted in the vault; a DNS target is not a secret.)- No config→vault→config cycle — provider-selector keys (
*.provider,*.providers.*.*) stay plain strings resolvable before secret-ref expansion. The 5-level merge is applied at the same pre-resolution layer as today. - DNS work is best-effort & additive — a resolver/provider error must not fail the
install (mirror the existing
createDeploymentDNSRecordsrollback semantics). - Coded errors only on prod paths (X-factories / coded.New/Wrap; no plain
errors.New/fmt.Errorf;shclog.Warnnotslog.Warn). POSIX sh for bats.
Phased implementation (TDD — test first each step)
Section titled “Phased implementation (TDD — test first each step)”Phase 0 — shared scope cascade helper
Section titled “Phase 0 — shared scope cascade helper”- New
internal/core/scope/scope.go:Scopeenum,ScopeCtx{Tenant,Stack,Environment},Cascade(ScopeCtx) []Scope(narrowest→broadest),Classify(ScopeCtx) Scope. Unit-test the full matrix. - Refactor
vault/paths.goCascadePaths/BuildPathto derive ordering fromscope.Cascade(behavior-preserving; existing vault tests must stay green).
Phase 1 — config store → 5 levels (highest blast radius)
Section titled “Phase 1 — config store → 5 levels (highest blast radius)”ent/schema/config.go: addenvironment,deployment(String().Optional().Nillable()); update unique index to(path, tenant_name, stack_name, environment, deployment)+ lookup index. Regenerate ent. Migration SQL adds the nullable columns (back-compat: existing rows = global/tenant/stack as today).internal/core/config/config.go: extendDBProviderwithEnvironment(ctx,tenant,env)+Deployment(ctx,tenant,stack,env); insert them into the Load merge in order Global→Tenant→Environment→Stack→Deployment (config.go:150-166). AddenvironmentFromCtx.internal/modules/config/service.go: extendlookupPrecedenceRows(:378) +Get/List/ListMergedprecedence walk to 5 tiers;Set/Unsetaccept env+deployment.internal/modules/config/commands/*:--scopeacceptsglobal|tenant|environment|stack|deployment;defaultScopederives the narrowest scope present in ctx.- Thread
environmentinto the config ctx at install/update Load sites.
- Tests: precedence matrix (all 5 tiers, stack>env), migration round-trip, CLI scope parse,
dns.provider/vault.providernow overridable at env/deployment scope.
Phase 2 — node projection widening (unblocks swarm multi-A)
Section titled “Phase 2 — node projection widening (unblocks swarm multi-A)”noderepo/adapter.go(:144) populateSwarmNodeID/SwarmRolefrom the node model; add them toNodeListRow(cluster/nodes_router.go:32). Test: a node withswarm_node_idsurfaces it throughlistNodes.
Phase 3 — DNS target resolver
Section titled “Phase 3 — DNS target resolver”- New
internal/modules/app/dns_target.go:parseHostTarget(auto-detect;=splits host from target).realSwarmNodeIPs(ctx) []netTargetviacluster.GetNodeRegistry().Listfiltered on non-emptySwarmNodeID,hostOnly(InternalAddress), IPv6→AAAA / IPv4→A classification. Returns nil ⇒ caller falls to compose single-A(/AAAA).resolveDNSTargets(ctx, host, scope) []DNSTargetencoding precedence: flag →config.Get("dns.target", 5-level scope)→ swarm multi → compose single.
- Unit-test the precedence matrix with fake config + fake registry (flag>config>swarm>compose; multi-A; IPv6→AAAA).
Phase 4 — DNS service typed + multi-record create/delete
Section titled “Phase 4 — DNS service typed + multi-record create/delete”dns/interfaces.go+service.go: add typed multi-target reconcile, e.g.SyncRecordsForHost(ctx, host, rt RecordType, contents []string, providerOverride, ttl, proxied)usingProvider.ListRecords(dnsapi/types.go:103) to add-missing / delete-stale (the currentCreateOrUpdatekeys on(zone,name,type)and would overwrite a 2nd A — must reconcile the full set). WidenDeleteRecordForHostto delete ALL records for(host,type). No provider-interface change (providers already takeRecord{Type}).- Tests: 3 swarm IPs → 3 A records; node removed → stale A pruned; uninstall removes all; A + AAAA coexist; CNAME replaces.
Phase 5 — wiring
Section titled “Phase 5 — wiring”createDeploymentDNSRecords(dns_lifecycle.go:34): signature gainstenant, stackName, environment(+ parsed host targets); replace the singleadvertiseAddress()/CreateRecordForHostwith per-hostresolveDNSTargets→SyncRecordsForHost; track(host,type,content)for rollback.- Call sites: install (
install_runner.go:574— tenant/stack/env onopts) andRegisterDeploymentDNS(move/recover,services_clone_remote.go:185). persistedState: addDNSTargets(alongsideHosts) so uninstall/move re-point exactly.
Phase 6 — e2e + docs
Section titled “Phase 6 — e2e + docs”- Bats:
--host h=cname:lb.example.com→ CNAME;--host h=2001:db8::1→ AAAA;config set dns.target lb.example.com --scope environmentthen bare--host→ CNAME from env scope; 2-real-swarm-node fixture → 2 A records; flag beats config. - Docs: config 5-level scoping (update
docs/architecture+ B11 note),dns.targetkey,--hosttarget syntax, swarm multi-A behavior.
- Config migration blast radius — every config consumer reads the widened table. Mitigated by nullable columns (old rows unchanged) + behavior-preserving Phase 0 refactor + the precedence matrix tests before Phase 2+.
StringMapsubtree scoping — provider credential subtrees (dns.providers.<n>.*) merge per-layer in koanf; verify a deployment-scoped partial subtree overlays (not replaces) the tenant subtree. Add an explicit test.- ent regen drift — regenerate, don’t hand-edit
ent/; run the dqlite-backed config tests (note: dqlite go-tests fail in the sandbox — gate behind the real harness).
Non-goals (separate changes)
Section titled “Non-goals (separate changes)”- ttl/proxied per-host CLI tuning. Three-tier SSE deploy events. The install-time
live-service-tracking spinner (separate plan). Per-app (
app_name) config scope (vault has it; config promotion stops at deployment for now).