Skip to content

fix(eval): chaos capture requires an explicit, non-prod kube context - #213

Merged
leo-aa88 merged 1 commit into
mainfrom
fix/capture-kube-context-guard-209-m3
Sep 26, 2026
Merged

leo-aa88 merged 1 commit into
mainfrom
fix/capture-kube-context-guard-209-m3

Conversation

@leo-aa88

Copy link
Copy Markdown
Member

Step 2 of the M3 sequence frozen on #209 (protocol): make the capture tool safe to run before anyone runs it.

Problem

scripts/eval/otel_corpus.py --chaos ran kubectl apply/delete -f <manifest> with no --context, so it used the kubeconfig's current context. The workstation that will drive M3 has a live production EKS cluster as its current context. A real chaos run from there would have injected pod-kill, network-partition, IO and CPU faults into production.

Fix

The target is now affirmative. There is no implicit fallback.

rule behaviour
--kube-context required A real --chaos run without it exits 2 before any kubectl call.
must exist The context has to be in the kubeconfig, which is read with kubectl config view. That command is read-only and contacts no cluster.
prod/live refusal The run refuses when the context name, its cluster or its API server contains prod, prd or live. Checking the cluster and server catches innocuous aliases, e.g. a context named sandbox → arn:…:cluster/…-live-prod.
--allow-context <exact-name> It must repeat --kube-context exactly. It overrides only the marker check, never the existence check, for non-prod names that trip it, e.g. delivery-dev (it contains "live").
every call names the target kubectl --context <resolved> apply/delete …

The target is resolved before any capture state is built. --dry-run still never touches kubectl, and the flag path (flagd HTTP/file, no kubectl) is unchanged. Matching is substring-based on purpose, so it fails closed. False positives cost one extra flag.

Tests

tests/unit/test_otel_corpus.py::TestKubeContextGuard swaps in a fake subprocess.run that serves a synthetic kubeconfig and records every call. No test can reach a real kubectl. The tests cover:

  • the missing flag;
  • a prod-named context;
  • a prod-cluster alias;
  • an unknown context;
  • a mismatched --allow-context;
  • --allow-context on a missing context;
  • a marker false positive with and without --allow-context;
  • every apply/delete carrying --context;
  • dry-run making no kubectl calls.

make lint ✅ · make test-unit ✅ (1505 passed). The docs (deploy/otel-demo/README.md) now show --kube-context in the real-run command and explain the rules.

Eval delta

None. This touches scripts/ and docs only, not src/core/, so the analysis path is untouched.

Refs #209.

🤖 Generated with Claude Code

A real `otel_corpus.py --chaos` run kubectl-applied Chaos Mesh manifests
against whatever the kubeconfig's current context was. On a workstation
whose current context is a live production cluster, that would inject
faults into production. The M3 capture (#209) runs this driver, so the
target is now affirmative:

- --kube-context is required for a real chaos run; there is no
  fallback to the current context.
- The run refuses a context that is not in the kubeconfig, and one whose
  name, cluster or API server contains prod / prd / live. Checking the
  cluster and server catches innocuous aliases for prod clusters. The
  kubeconfig is read with `kubectl config view`, which contacts no cluster.
- --allow-context <exact same name> overrides the marker check only (not
  the existence check), for non-prod contexts that trip it.
- Every kubectl apply/delete passes --context <resolved>.
- The target is resolved before any capture state is built; dry-run
  still never touches kubectl.

Tests swap in a fake subprocess.run serving a synthetic kubeconfig, so
none can reach a real kubectl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@leo-aa88

Copy link
Copy Markdown
Member Author

/review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Open in Web View Automation 

Sent by Cursor Automation: Code Reviewer

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Open in Web View Automation 

Sent by Cursor Automation: Code Reviewer

@leo-aa88
leo-aa88 merged commit c53e9fc into main Sep 26, 2026
7 checks passed
@leo-aa88
leo-aa88 deleted the fix/capture-kube-context-guard-209-m3 branch September 26, 2026 12:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant