openova/openova
1
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity
66e7768e8e
openova/platform/k8s-ws-proxy/README.md
e3mrah 639b94fe55
feat(epic-4): K+P+X1+G — k8s-ws-proxy + projector + WebSocket logs + Guacamole chart (#1099) (#1164) 
EPIC-4 Slice K+P+X1+G — bundled backend infrastructure for the
"k9s-on-web" Cloud Resources experience:

K1 — core/cmd/k8s-ws-proxy/ — per-node WebSocket exec proxy.
HMAC-signed (X-Catalyst-HMAC: SHA256({timestamp}:{path})) WebSocket
upgrades on /proxy/exec/{ns}/{pod}/{container} bridged to the local
kube-apiserver via in-cluster ServiceAccount. v4.channel.k8s.io
subprotocol echo. Optional TMUX_CASCADE wraps in a shared
catalyst-ops tmux session. Shipped as a DaemonSet + Service with
internalTrafficPolicy=Local in platform/k8s-ws-proxy/chart/.

P1 — core/cmd/projector/ — NATS catalyst.events JetStream → Valkey
KV projector. Canonical key shape:
  cluster:{cluster-id}:kind:{kind}:{namespace}/{name}
Cold-start does a full LIST across DefaultKinds, then catches up on
the 24h replay window. Multi-replica safe (durable consumer queue
group, last-write-wins on namespacedName). Shipped as a default-OFF
Deployment + RBAC under products/catalyst/chart/templates/services/projector/.

X1 — products/catalyst/bootstrap/api/internal/handler/k8s_logs.go —
WebSocket Pod-log streaming endpoint:
  GET /api/v1/sovereigns/{id}/k8s/logs/{ns}/{pod}/{container}
      ?follow&tailLines&since=<rfc3339>&previous
Reads from kubelet via client-go GetLogs().Stream(); each WS frame =
one log line. Supports `since` resume. Reuses RequireSession middleware
+ chroot cluster-id resolver. New k8scache.Factory.CoreClient(id)
accessor exposes the per-cluster typed client without duplicating
kubeconfig parsing.

G1 — platform/guacamole/chart/ — full Apache Guacamole chart:
guacd Deployment + Service, Tomcat webapp Deployment + Service,
Cilium Gateway HTTPRoute, SeaweedFS-PVC for recordings (RWO,
hcloud-volumes), SealedSecret placeholder for Keycloak OIDC client
secret, NetworkPolicy (default-deny + selective egress to KC +
k8s-ws-proxy + SeaweedFS + NATS), and ConfigMap consumed by
keycloak-config-cli post-deploy Job (mirrors platform/keycloak
realm-config pattern). Default-OFF gate; full-ON renders 9
resources. Empty image.tag / hostname / oidc.issuer fail-fast at
helm template time per INVIOLABLE-PRINCIPLES #4a/#5. ONE Guacamole
per Sovereign per ADR-0001 §11. Blueprint manifest uses
v1alpha1 + version "0.1.0" + upgrades.from ["0.x"].

Tests:
- k8s-ws-proxy: HMAC happy/expired-old/expired-future/malformed/
  bad-signature, path-only signature, WS upgrade + protocol echo,
  bad path, bad HMAC, denied namespace via httptest.
- projector: Apply ADD/MOD/DEL/validation, key shape (ns-scoped +
  cluster-scoped), handleOne ack/nak/term routing with fakeMsg,
  cold-start LIST + project + error continuation via dynamicfake.
- X1: parseLogOptions defaults + edge cases + bad query params,
  503/404/400 paths + full WS happy-path with kfake clientset.
- G1: chart/tests/render.sh — default-OFF=0, empty-tag fail-fast,
  full-ON=9 resources, every required kind present, realm-config
  wires OIDC client.
- bp-k8s-ws-proxy chart: chart/tests/render.sh — default-OFF=0,
  empty-tag fail-fast, full-ON=5 resources.

Pre-existing test status: TestPinIssue and TestBootstrapKit/gitea
remain flaky on main per canon §7 — verified not introduced by
this slice.

Co-authored-by: hatiyildiz <hati.yildiz@openova.io>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 09:27:39 +04:00

1.4 KiB
Raw Blame History

bp-k8s-ws-proxy — k8s-ws-proxy Blueprint

Catalyst-built Go binary + Helm chart wrapping the per-node WebSocket exec proxy (core/cmd/k8s-ws-proxy/).

Why this exists

Browsers can't reach the kube-apiserver directly without exposing kubeconfig tokens (INVIOLABLE-PRINCIPLES #5). Putting a per-node DaemonSet in front lets:

  1. The catalyst-api forward exec requests with HMAC-signed WebSocket upgrades — no kubeconfig in the browser.
  2. Sessions stay node-local (internalTrafficPolicy: Local) — the kube-proxy short-circuits onto the same node's pod, eliminating cross-node hops.
  3. NetworkPolicy gates exec traffic at the per-node DaemonSet's pod IPs (one selector, one policy).

See core/cmd/k8s-ws-proxy/DESIGN.md for the wire contract + failure-mode matrix.

Default-OFF gate

values.yaml ships k8sWsProxy.enabled: false. Per-Sovereign overlay flips on AND populates:

  • k8sWsProxy.image.tag — SHA-pinned (CI populates)
  • k8sWsProxy.hmacSecret.name — name of the SealedSecret holding the shared HMAC key (operator pre-creates with kubeseal)

Empty values for either fail the helm template render.

Render check

# 0 resources when off
helm template bp-k8s-ws-proxy . | grep -c '^kind:'

# Full set when on
helm template bp-k8s-ws-proxy . \
  --set k8sWsProxy.enabled=true \
  --set k8sWsProxy.image.tag=abc1234 \
  | grep -c '^kind:'