Skip to content
SHC Docs

Health probes

Layer 6. Services. Services

New line, in development. This page describes SHC as it is on the dev branch, not the released product. See The new line.

A service’s health is one word: starting, healthy or unhealthy. SHC gets it two ways, and everything that reads health reads the same word whichever way it came.

TestWho runs itWhere
["CMD", ...] or ["CMD-SHELL", "..."]the container runtimeinside the container
["SHC-HTTP", ":PORT/PATH"] or ["SHC-TCP", ":PORT"]the node agentfrom the node, at the replica’s address

A command needs its binary in the image. A distroless or scratch image has no shell, no curl and no wget, so a check such as wget -qO- http://127.0.0.1:8080/healthz cannot run there and the container never reads healthy. An SHC probe needs nothing in the image.

services:
web:
image: gcr.io/distroless/static-debian12
expose: ["8080"]
healthcheck:
test: ["SHC-HTTP", ":8080/healthz"]
interval: 10s
timeout: 3s
retries: 3
start_period: 30s
db:
image: postgres:17
expose: ["5432"]
healthcheck:
test: ["SHC-TCP", ":5432"]

SHC-HTTP sends a GET for the path to that container port. Any 2xx or 3xx passes, and a redirect is the answer, not followed. A third entry names the one status that passes: ["SHC-HTTP", ":8080/ready", "204"]. The path may carry a query. SHC-TCP passes when a connect to the port succeeds.

The target is :PORT, never a host, because the probe always dials the replica’s own address. Write the list form: a string test is a shell command, so test: "SHC-HTTP :8080/healthz" is refused.

interval, timeout, retries, start_period and start_interval mean what they mean to Docker, with Docker’s defaults: 30s, 30s, 3, 0 and 5s. A field left out takes Docker’s default, never the image’s. A probe replaces an image’s own HEALTHCHECK, as any compose test does.

A probe that cannot run is refused at install with X400713HLTPRB, which names the service and the fault: a misspelt kind such as SHC-HTTPS, no port or one outside 1 to 65535, a host in the target, a path on a TCP probe, a status outside 100 to 599, or a timing that is not a duration.

In shc.yaml the same healthcheck block sits on the service.

No runtime sees the SHC form. The render writes healthcheck: {disable: true} in its place, so docker, podman, incus and apple are never handed a test they cannot run and no image HEALTHCHECK runs beneath the probe. The probe rides on container labels under shc.probe. (ce modules/app/inject_native_probes.go).

Each replica’s project adds where the node probes the entry (ce core/pod/probe.go): shc.probe.address, the entry’s fabric address, which is the address the front door dials. A member that joins another member’s network namespace is probed at that member’s address, and one with network_mode: host at 127.0.0.1. On apple, which assigns addresses itself, the label is shc.probe.network and the node reads the address apple gave the container on that network. An shc.* label an app writes is still refused, X400357LBLSHC.

The node agent of the node the replica runs on, from the host (ce core/executor/healthcheck/native.go Prober, subsystems/impl/health_probes.go). The timing is Docker’s: start_interval while the check is starting inside start_period, a failure inside start_period does not count, retries failures in a row make it unhealthy, and one pass makes it healthy. A container that starts again begins at starting.

The same things that read a Docker healthcheck:

  • shc status and the replica’s readiness. An install or a rollout waits through the probe’s start period as it would for a command’s.
  • The front door and a group’s address, which route only to replicas that read healthy. The probe’s word reaches the container’s row the moment it changes.
  • x-shc-health.restart_after, which restarts a container its probe has found unhealthy that long (x-shc directives). The sustained-unhealth warning X500115CTRUNH quotes what the last failed probe found, such as GET http://10.128.0.7:8080/healthz answered 500.
  • Another pod’s depends_on with condition: service_healthy.

A member that waits for another member of its own pod with condition: service_healthy cannot wait on that member’s probe: compose orders a pod’s members and sees only the runtime’s checks. The replica’s render refuses it, naming both services. Wait with service_started, or give the awaited member a CMD check.

A container the node has no address for, which happens only on a node that runs no replica projects, fails every probe and reads unhealthy.