fix: reinforce no-script-execution constraint in devops-health-check#845
Open
AbhitejJohn wants to merge 2 commits into
Open
fix: reinforce no-script-execution constraint in devops-health-check#845AbhitejJohn wants to merge 2 commits into
AbhitejJohn wants to merge 2 commits into
Conversation
AbhitejJohn
requested review from
a team,
JanKrivanek,
ViktorHofer and
webreidi
as code owners
Contributor
Skill Coverage Report
Uncovered:
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the DevOps Daily Health Check agent instructions to more explicitly avoid attempts at script execution (e.g., python3 -c ...) that are blocked by the bash allowlist/security policy. It also introduces Claude marketplace/plugin-manifest changes for the dotnet-msbuild plugin.
Changes:
- Strengthen
devops-health-checkguidance to state script interpreters (python/node) are not available and to usejqfor JSON parsing. - Add
plugins/dotnet-msbuild/.claude-plugin/plugin.jsonfor Claude-specific plugin metadata. - Extend
.claude-plugin/marketplace.jsonwith anmcpServersdefinition fordotnet-msbuild.
Show a summary per file
| File | Description |
|---|---|
plugins/dotnet-msbuild/.claude-plugin/plugin.json |
Adds a Claude plugin manifest for dotnet-msbuild (but currently uses likely-incorrect relative paths). |
.github/workflows/devops-health-check.md |
Clarifies that python/node interpreters are not in the bash allowlist and directs JSON processing to jq. |
.claude-plugin/marketplace.json |
Adds an mcpServers entry for dotnet-msbuild in the Claude marketplace manifest. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 3/3 changed files
- Comments generated: 2
The agent was hallucinating Python script creation (run_analysis2.py) and attempting to execute it, which was correctly blocked by the bash allowlist security policy. Strengthen the efficiency guideline to explicitly state that python/python3/node are NOT in the bash allowlist and will be blocked, directing the agent to use jq instead. Fixes #841 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
AbhitejJohn
force-pushed
the
abhitejjohn/fix-devops-health-check-script-hallucina
branch
from
541c543 to
04abecd
Compare
AbhitejJohn
force-pushed
the
abhitejjohn/fix-devops-health-check-script-hallucina
branch
from
04abecd to
45c6faf
Compare
Contributor
|
👋 @AbhitejJohn — this PR has 3 unresolved review thread(s). When you're ready, please address the feedback and push an update; the triage bot will pick up the next state automatically. (Add the |
Addresses review feedback: 'any script interpreter' was too broad since jq/sed are technically interpreters but ARE in the allowlist. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
github-actions
Bot
added
waiting-on-review
Contributor
|
✅ Evaluation passed for |
Contributor
|
✅ Evaluation passed for |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Strengthens the prompt in the DevOps Daily Health Check workflow to explicitly tell the agent that python, python3, and node are NOT in the bash allowlist and will be blocked by the security policy.
Problem
The Copilot agent was hallucinating Python script creation (python3 -c "import json,...") to process JSON data, which was correctly blocked by the bash tool's security policy. Despite existing instructions saying "don't create scripts," the agent interpreted inline python3 -c as a command rather than a script, and retried multiple times - burning token budget.
Failed run: https://github.com/dotnet/skills/actions/runs/28311460592
Fix
Made the efficiency guideline explicitly state that script interpreters are not available, directing the agent to use jq for all JSON processing.
Fixes #841