Skip to content
SHC Docs

Plan: 5-level config scoping + DNS target resolver (LB override + swarm multi-A)

Status: IMPLEMENTED on feat/config-5level-dns-target (phases 0–6, commits 60fcd18e..91e7415a). Remaining follow-ups: real-DNS e2e against a provider-configured harness; worker-side docker swarm join port (until it lands only the init node carries a swarm_node_id, so multi-A degrades conservatively); wiring the core-config loader’s DBProvider impl (interface is 5-level-ready, the DB layer of the 13-layer loader remains an unfinished pre-existing port phase).

UPDATE — dns.target was later REFACTORED off scoped config (2026-07). The 5-level config ladder shipped and remains. But dns.target no longer resolves as a scoped config key: a subsequent phase moved the DNS target to per-deployment state (persisted in the deployment’s state, not the config store). There is no dns.target config tier anymore — see internal/modules/app/dns_target.go. The scoped-dns.target examples below (the --scope environment LB-override walkthrough) describe the now-retired config-tier resolution and are historical.

A user asked for managed DNS to be able to point a --host at a load-balancer target (A/AAAA/CNAME) instead of node IPs, overridable per tenant / environment / deployment, with a config fallback, falling back to node IPs (swarm = one record per real swarm node; compose = single record to the advertise address).

Resolving “what does the record point to” needs a scoped config lookup. But SHC config is only 3-level (global/tenant/stack) while secrets/vault are 5-level (base/tenant/stack/environment/deployment). Owner decision: promote config to the full 5-level ladder so config — and therefore every provider selector that reads config (dns.provider, vault.provider, backup.*, storage.*) — becomes environment- and deployment-aware. The DNS target resolver is then a thin consumer of that unified scoping.

The one scope ladder (single source of truth)

Section titled “The one scope ladder (single source of truth)”

Narrowest wins. Matches the existing vault path cascade (internal/modules/vault/paths.go:114-140):

Deployment = tenant + stack + environment (most specific)
Stack = tenant + stack
Environment = tenant + environment
Tenant = tenant
Global/Base (broadest fallback)

Stack outranks Environment (confirmed: vault CascadePaths lists Stack at :121 before Environment at :126). The config koanf merge must load layers Global → Tenant → Environment → Stack → Deployment (koanf = last-wins ⇒ that order yields the ladder above).

A new shared helper encodes this ONE ordering; vault, config, and DNS all consume it — no drift. Proposed: internal/core/scope/type Scope enum + Cascade(ctx ScopeCtx) []Scope (narrowest→broadest) + ScopeCtx{Tenant, Stack, Environment}. Vault paths.go and config layering refactor onto it.

DecisionChoice
--host target syntaxAuto-detect only: [service:]host[=target]. Type inferred by dns.ResolveTarget (service.go:63): IPv4→A, IPv6→AAAA, hostname→CNAME. No a:/cname: prefix.
Scope model5-level on both config + vault. Promote config now.
PrecedenceDeployment > Stack > Environment > Tenant > Global.
IPv6Supported — AAAA via auto-detect; node-IP fallback emits AAAA for IPv6 advertise addrs; reconcile/delete key on (name, type) so A/AAAA/CNAME sets are independent.
Swarm node filterA “real swarm node” = node row with non-empty swarm_node_id (NOT node.runtime, which is "compose" even for swarm members). Lighthouse/overlay-only nodes aren’t node rows → excluded for free.
  • Parity (already ported): provider-selectable DNS create-on-deploy / delete-on-uninstall.
  • Net-new: swarm multi-A, LB/CNAME/AAAA target override, scoped dns.target config, AND the 5-level config promotion (Python config was 3-level; only secrets were 5-level).
  1. dns.target is config, not a secret — plaintext config store, never the vault. (Standing rule: secrets MUST be encrypted in the vault; a DNS target is not a secret.)
  2. No config→vault→config cycle — provider-selector keys (*.provider, *.providers.*.*) stay plain strings resolvable before secret-ref expansion. The 5-level merge is applied at the same pre-resolution layer as today.
  3. DNS work is best-effort & additive — a resolver/provider error must not fail the install (mirror the existing createDeploymentDNSRecords rollback semantics).
  4. Coded errors only on prod paths (X-factories / coded.New/Wrap; no plain errors.New/fmt.Errorf; shclog.Warn not slog.Warn). POSIX sh for bats.

Phased implementation (TDD — test first each step)

Section titled “Phased implementation (TDD — test first each step)”
  • New internal/core/scope/scope.go: Scope enum, ScopeCtx{Tenant,Stack,Environment}, Cascade(ScopeCtx) []Scope (narrowest→broadest), Classify(ScopeCtx) Scope. Unit-test the full matrix.
  • Refactor vault/paths.go CascadePaths/BuildPath to derive ordering from scope.Cascade (behavior-preserving; existing vault tests must stay green).

Phase 1 — config store → 5 levels (highest blast radius)

Section titled “Phase 1 — config store → 5 levels (highest blast radius)”
  1. ent/schema/config.go: add environment, deployment (String().Optional().Nillable()); update unique index to (path, tenant_name, stack_name, environment, deployment) + lookup index. Regenerate ent. Migration SQL adds the nullable columns (back-compat: existing rows = global/tenant/stack as today).
  2. internal/core/config/config.go: extend DBProvider with Environment(ctx,tenant,env) + Deployment(ctx,tenant,stack,env); insert them into the Load merge in order Global→Tenant→Environment→Stack→Deployment (config.go:150-166). Add environmentFromCtx.
  3. internal/modules/config/service.go: extend lookupPrecedenceRows (:378) + Get/List/ListMerged precedence walk to 5 tiers; Set/Unset accept env+deployment.
  4. internal/modules/config/commands/*: --scope accepts global|tenant|environment|stack|deployment; defaultScope derives the narrowest scope present in ctx.
  5. Thread environment into the config ctx at install/update Load sites.
  • Tests: precedence matrix (all 5 tiers, stack>env), migration round-trip, CLI scope parse, dns.provider/vault.provider now overridable at env/deployment scope.

Phase 2 — node projection widening (unblocks swarm multi-A)

Section titled “Phase 2 — node projection widening (unblocks swarm multi-A)”
  • noderepo/adapter.go (:144) populate SwarmNodeID/SwarmRole from the node model; add them to NodeListRow (cluster/nodes_router.go:32). Test: a node with swarm_node_id surfaces it through listNodes.
  • New internal/modules/app/dns_target.go:
    • parseHostTarget (auto-detect; = splits host from target).
    • realSwarmNodeIPs(ctx) []netTarget via cluster.GetNodeRegistry().List filtered on non-empty SwarmNodeID, hostOnly(InternalAddress), IPv6→AAAA / IPv4→A classification. Returns nil ⇒ caller falls to compose single-A(/AAAA).
    • resolveDNSTargets(ctx, host, scope) []DNSTarget encoding precedence: flag → config.Get("dns.target", 5-level scope) → swarm multi → compose single.
  • Unit-test the precedence matrix with fake config + fake registry (flag>config>swarm>compose; multi-A; IPv6→AAAA).

Phase 4 — DNS service typed + multi-record create/delete

Section titled “Phase 4 — DNS service typed + multi-record create/delete”
  • dns/interfaces.go + service.go: add typed multi-target reconcile, e.g. SyncRecordsForHost(ctx, host, rt RecordType, contents []string, providerOverride, ttl, proxied) using Provider.ListRecords (dnsapi/types.go:103) to add-missing / delete-stale (the current CreateOrUpdate keys on (zone,name,type) and would overwrite a 2nd A — must reconcile the full set). Widen DeleteRecordForHost to delete ALL records for (host,type). No provider-interface change (providers already take Record{Type}).
  • Tests: 3 swarm IPs → 3 A records; node removed → stale A pruned; uninstall removes all; A + AAAA coexist; CNAME replaces.
  • createDeploymentDNSRecords (dns_lifecycle.go:34): signature gains tenant, stackName, environment (+ parsed host targets); replace the single advertiseAddress()/CreateRecordForHost with per-host resolveDNSTargetsSyncRecordsForHost; track (host,type,content) for rollback.
  • Call sites: install (install_runner.go:574 — tenant/stack/env on opts) and RegisterDeploymentDNS (move/recover, services_clone_remote.go:185).
  • persistedState: add DNSTargets (alongside Hosts) so uninstall/move re-point exactly.
  • Bats: --host h=cname:lb.example.com → CNAME; --host h=2001:db8::1 → AAAA; config set dns.target lb.example.com --scope environment then bare --host → CNAME from env scope; 2-real-swarm-node fixture → 2 A records; flag beats config.
  • Docs: config 5-level scoping (update docs/architecture + B11 note), dns.target key, --host target syntax, swarm multi-A behavior.
  • Config migration blast radius — every config consumer reads the widened table. Mitigated by nullable columns (old rows unchanged) + behavior-preserving Phase 0 refactor + the precedence matrix tests before Phase 2+.
  • StringMap subtree scoping — provider credential subtrees (dns.providers.<n>.*) merge per-layer in koanf; verify a deployment-scoped partial subtree overlays (not replaces) the tenant subtree. Add an explicit test.
  • ent regen drift — regenerate, don’t hand-edit ent/; run the dqlite-backed config tests (note: dqlite go-tests fail in the sandbox — gate behind the real harness).
  • ttl/proxied per-host CLI tuning. Three-tier SSE deploy events. The install-time live-service-tracking spinner (separate plan). Per-app (app_name) config scope (vault has it; config promotion stops at deployment for now).