fix(eval): chaos capture requires an explicit, non-prod kube context - #213
Merged
Merged
Conversation
A real `otel_corpus.py --chaos` run kubectl-applied Chaos Mesh manifests against whatever the kubeconfig's current context was. On a workstation whose current context is a live production cluster, that would inject faults into production. The M3 capture (#209) runs this driver, so the target is now affirmative: - --kube-context is required for a real chaos run; there is no fallback to the current context. - The run refuses a context that is not in the kubeconfig, and one whose name, cluster or API server contains prod / prd / live. Checking the cluster and server catches innocuous aliases for prod clusters. The kubeconfig is read with `kubectl config view`, which contacts no cluster. - --allow-context <exact same name> overrides the marker check only (not the existence check), for non-prod contexts that trip it. - Every kubectl apply/delete passes --context <resolved>. - The target is resolved before any capture state is built; dry-run still never touches kubectl. Tests swap in a fake subprocess.run serving a synthetic kubeconfig, so none can reach a real kubectl. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
|
/review |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Step 2 of the M3 sequence frozen on #209 (protocol): make the capture tool safe to run before anyone runs it.
Problem
scripts/eval/otel_corpus.py --chaosrankubectl apply/delete -f <manifest>with no--context, so it used the kubeconfig's current context. The workstation that will drive M3 has a live production EKS cluster as its current context. A real chaos run from there would have injected pod-kill, network-partition, IO and CPU faults into production.Fix
The target is now affirmative. There is no implicit fallback.
--kube-contextrequired--chaosrun without it exits 2 before any kubectl call.kubectl config view. That command is read-only and contacts no cluster.prod,prdorlive. Checking the cluster and server catches innocuous aliases, e.g. a context namedsandbox→arn:…:cluster/…-live-prod.--allow-context <exact-name>--kube-contextexactly. It overrides only the marker check, never the existence check, for non-prod names that trip it, e.g.delivery-dev(it contains "live").kubectl --context <resolved> apply/delete …The target is resolved before any capture state is built.
--dry-runstill never touches kubectl, and the flag path (flagd HTTP/file, no kubectl) is unchanged. Matching is substring-based on purpose, so it fails closed. False positives cost one extra flag.Tests
tests/unit/test_otel_corpus.py::TestKubeContextGuardswaps in a fakesubprocess.runthat serves a synthetic kubeconfig and records every call. No test can reach a real kubectl. The tests cover:--allow-context;--allow-contexton a missing context;--allow-context;--context;make lint✅ ·make test-unit✅ (1505 passed). The docs (deploy/otel-demo/README.md) now show--kube-contextin the real-run command and explain the rules.Eval delta
None. This touches
scripts/and docs only, notsrc/core/, so the analysis path is untouched.Refs #209.
🤖 Generated with Claude Code