DoD gate D12-LB-allocation root cause: k3s registers nodes with
providerID=k3s://<hostname> instead of hcloud://<server-id>. hcloud-ccm
rejects every LB allocation:
hcops/LoadBalancerOps.ReconcileHCLBTargets: providerID does not have
one of the expected prefixes (hcloud://, hrobot://, hcloud://bm-):
k3s://catalyst-t115-omani-works-nbg1-1-cp1
This blocked clustermesh-apiserver Service from getting an external
IP on every secondary region → AutoEstablishClusterMesh (PR #1508)
couldn't write peer entries → D10/D11 fail.
Caught on t115.omani.works (577be15281be2587, 2026-05-16) after PR
#1509 flipped clustermesh-apiserver Service to LoadBalancer. The
NodePort default in the old chart masked this k3s-vs-hcloud-ccm
incompatibility until the LoadBalancer flip exposed it.
Fix (k3s server install line in cloudinit-control-plane.tftpl):
+ --disable-cloud-controller
+ --kubelet-arg=cloud-provider=external
Fix (k3s agent install line in cloudinit-worker.tftpl):
+ --kubelet-arg=cloud-provider=external
The k3s server flag tells the embedded cloud controller to stay out.
The kubelet flag tells kubelet to wait for an external CCM to set
providerID. hcloud-ccm (bootstrap-kit slot 36) then matches each
node to its Hetzner server by name and sets providerID=hcloud://<id>,
unblocking LB allocation, Volume CSI, and node-external-ip.
The node is briefly tainted node.cloudprovider.kubernetes.io/
uninitialized=NoSchedule until the CCM removes it — Flux's
bootstrap-kit Kustomization tolerates this taint via SOPs.
Co-authored-by: claude <claude@anthropic.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7f0de7fa82