Skip to content

CLI-547 Experiment with a new agentic workflow#350

Draft
damien-urruty-sonarsource wants to merge 3 commits into
masterfrom
task/dam/night-owl
Draft

CLI-547 Experiment with a new agentic workflow#350
damien-urruty-sonarsource wants to merge 3 commits into
masterfrom
task/dam/night-owl

Conversation

@damien-urruty-sonarsource

@damien-urruty-sonarsource damien-urruty-sonarsource commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary by Gitar

  • New Agentic Workflow:
    • Implemented night-owl agentic workflow for automated Jira ticket processing.
    • Added prepare-jira-context.sh to automate Atlassian CLI authentication and Jira context gathering.
    • Created night-owl.md and night-owl.lock.yml to define the new workflow architecture.
  • Workflow Migration:
    • Refactored ci-failure-triage-agent to align with the new night-owl concurrency patterns and operational standards.
  • Documentation:
    • Updated CLAUDE.md with guidelines and instructions for implementing the new agentic workflow.

This will update automatically on new commits.

@hashicorp-vault-sonar-prod hashicorp-vault-sonar-prod Bot changed the title Experiment with a new agentic workflow CLI-547 Experiment with a new agentic workflow Jun 1, 2026
@hashicorp-vault-sonar-prod

hashicorp-vault-sonar-prod Bot commented Jun 1, 2026

Copy link
Copy Markdown

CLI-547

Comment thread .github/workflows/night-owl.md Outdated

concurrency: ci-triage-${{ github.run_id }}

concurrency: night-owl

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Quality: ci-failure-triage-agent.md fully repurposed but keeps old name

The ci-failure-triage-agent.md workflow has been entirely replaced with the Night Owl logic (identical env vars, prompt, safe-outputs). This means the original CI failure triage functionality is lost. The workflow file name, the GitHub Actions name: field ("CI Failure Triage Agent"), and the run-name in the lock file are now misleading. Consider either:

  • Keeping the CI failure triage agent intact and only adding the new night-owl.md workflow, or
  • Renaming/removing the old file to avoid confusion.

The comment on line 124 says it's "temporarily repurposed", but having two identically-functioning workflows (night-owl.md and ci-failure-triage-agent.md) with the same concurrency group (night-owl) and the same cron schedule could cause unintended mutual cancellation.

Was this helpful? React with 👍 / 👎

@sonarqubecloud

sonarqubecloud Bot commented Jun 3, 2026

Copy link
Copy Markdown

Agentic Analysis: Early Results

Agentic Analysis and Context Augmentation are available on your project. Here are some issues that could have been prevented. Follow the links to learn how to put them into action.

19 issue(s) found across 1 file(s):

Rule File Line Message
shelldre:S7679 .github/scripts/night-owl/prepare-jira-context.sh 14 Assign this positional parameter to a local variable.
shelldre:S7679 .github/scripts/night-owl/prepare-jira-context.sh 14 Assign this positional parameter to a local variable.
shelldre:S7679 .github/scripts/night-owl/prepare-jira-context.sh 73 Assign this positional parameter to a local variable.
shelldre:S7679 .github/scripts/night-owl/prepare-jira-context.sh 80 Assign this positional parameter to a local variable.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 83 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 132 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 169 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 176 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 201 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 204 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 207 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 214 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 293 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 314 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 334 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 335 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 359 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 382 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.
shelldre:S7688 .github/scripts/night-owl/prepare-jira-context.sh 417 Use '[[' instead of '[' for conditional tests. The '[[' construct is safer and more feature-rich.

Analyzed by SonarQube Agentic Analysis in 3.5 s

Comment on lines +124 to +137
issue_description_text() {
local issue_json="$1"
local description_text

description_text="$(
jq '.fields.description // .description // null' <<<"$issue_json" | adf_to_text
)"

if [ -n "$(compact_text "$description_text")" ]; then
printf '%s\n' "$description_text"
else
printf '_None provided._\n'
fi
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Security: Untrusted Jira content injected verbatim into autonomous agent prompt

prepare-jira-context.sh builds context_markdown from Jira ticket summary, description, comments, and linked-issue fields (issue_description_text/format_comments_markdown/format_links_markdown) and emits it to GITHUB_OUTPUT. Both night-owl.md and ci-failure-triage-agent.md then inline ${{ needs.night_owl_prepare.outputs.context_markdown }} directly into the agent prompt and instruct the agent to 'Treat the prepared Jira content above as the source of truth.' The agent has the create_pull_request safe output, so a Jira ticket (any CLI ticket labeled for-agent) whose description/comments contain adversarial instructions becomes an XPIA / prompt-injection vector that can steer autonomous code changes and PRs.

This is partially mitigated by the gh-aw xpia.md guard prompt, and the surface is not strictly new (Jira was previously read via MCP). However, the change moves the content into the static prompt body rather than tool results. Consider: (1) clearly delimiting the Jira content as untrusted data in the prompt rather than 'source of truth', and (2) tightening the agent job's firewall — it still allows api.atlassian.com / sonarsource.atlassian.net even though the agent no longer needs Jira access (the prepare job does), so those domains are unnecessary egress/exfiltration surface for the agent.

Was this helpful? React with 👍 / 👎

Comment on lines +139 to +149
has_open_pr_for_key() {
local key="$1"
local pattern="(^|[^A-Z0-9])${key}([^A-Z0-9]|$)"

jq -e --arg pattern "$pattern" '
any(
.[]?;
([.title // "", .body // "", .headRefName // ""] | join("\n")) | test($pattern; "i")
)
' <<<"$OPEN_PULL_REQUESTS_JSON" >/dev/null
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Edge Case: Open-PR dedup capped at 100 PRs and relies on exact key text match

OPEN_PULL_REQUESTS_JSON is fetched with gh pr list --state open --limit 100, and has_open_pr_for_key only matches the Jira key as a token in title/body/headRefName. If a repository has more than 100 open PRs, an in-flight ticket may not be found and Night Owl could start duplicate work on a ticket already covered by an open PR. The match also fails entirely if a previous night-owl PR did not embed the Jira key in its title/body/branch. Consider raising/removing the limit (or filtering by the night-owl label/author) and ensuring created PRs always embed the Jira key.

Was this helpful? React with 👍 / 👎

--json number,title,body,headRefName,url
)"

SEARCH_RESULTS_JSON="$(search_workitems)"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Quality: Malformed acli search output is silently reported as 'starving'

The candidate loop is fed by a process substitution: done < <(normalize_collection <<<"$SEARCH_RESULTS_JSON" | jq -c '.[]?'). With set -euo pipefail, a failure inside a process substitution does not abort the parent shell. If acli ... search returns output that is not valid JSON (or an unexpected shape normalize_collection cannot match), the pipeline fails/produces nothing, the loop iterates zero times, candidate_count stays 0, and the script emits a 'No Jira ticket matched' starving Slack message and exits 0 — masking an actual infrastructure/query error as routine starvation. Consider validating SEARCH_RESULTS_JSON parses as JSON (e.g. jq empty with explicit error handling) before the loop so genuine failures route to night_owl_prepare_failure_notify instead of the starving path.

Was this helpful? React with 👍 / 👎

@gitar-bot

gitar-bot Bot commented Jun 3, 2026

Copy link
Copy Markdown
Code Review 👍 Approved with suggestions 1 resolved / 5 findings

Introduces the night-owl agentic workflow for Jira ticket processing and refactors CI triage infrastructure. Rename the repurposed ci-failure-triage-agent.md, sanitize Jira content before injection, and improve pull request deduplication logic for robustness.

💡 Quality: ci-failure-triage-agent.md fully repurposed but keeps old name

📄 .github/workflows/ci-failure-triage-agent.md:7 📄 .github/workflows/ci-failure-triage-agent.md:122-124 📄 .github/workflows/ci-failure-triage-agent.lock.yml:73 📄 .github/workflows/ci-failure-triage-agent.md 📄 .github/workflows/ci-failure-triage-agent.lock.yml:59 📄 .github/workflows/ci-failure-triage-agent.lock.yml:61-62 📄 .github/workflows/night-owl.lock.yml:61-62 📄 .github/workflows/night-owl.lock.yml:73 📄 .github/workflows/ci-failure-triage-agent.md:3-4 📄 .github/workflows/night-owl.md:3-4 📄 .github/workflows/night-owl.md:7

The ci-failure-triage-agent.md workflow has been entirely replaced with the Night Owl logic (identical env vars, prompt, safe-outputs). This means the original CI failure triage functionality is lost. The workflow file name, the GitHub Actions name: field ("CI Failure Triage Agent"), and the run-name in the lock file are now misleading. Consider either:

  • Keeping the CI failure triage agent intact and only adding the new night-owl.md workflow, or
  • Renaming/removing the old file to avoid confusion.

The comment on line 124 says it's "temporarily repurposed", but having two identically-functioning workflows (night-owl.md and ci-failure-triage-agent.md) with the same concurrency group (night-owl) and the same cron schedule could cause unintended mutual cancellation.

💡 Security: Untrusted Jira content injected verbatim into autonomous agent prompt

📄 .github/scripts/night-owl/prepare-jira-context.sh:124-137 📄 .github/scripts/night-owl/prepare-jira-context.sh:151-165 📄 .github/scripts/night-owl/prepare-jira-context.sh:183-197 📄 .github/scripts/night-owl/prepare-jira-context.sh:434-446 📄 .github/workflows/night-owl.md:200 📄 .github/workflows/night-owl.md:217-219 📄 .github/workflows/night-owl.lock.yml:805

prepare-jira-context.sh builds context_markdown from Jira ticket summary, description, comments, and linked-issue fields (issue_description_text/format_comments_markdown/format_links_markdown) and emits it to GITHUB_OUTPUT. Both night-owl.md and ci-failure-triage-agent.md then inline ${{ needs.night_owl_prepare.outputs.context_markdown }} directly into the agent prompt and instruct the agent to 'Treat the prepared Jira content above as the source of truth.' The agent has the create_pull_request safe output, so a Jira ticket (any CLI ticket labeled for-agent) whose description/comments contain adversarial instructions becomes an XPIA / prompt-injection vector that can steer autonomous code changes and PRs.

This is partially mitigated by the gh-aw xpia.md guard prompt, and the surface is not strictly new (Jira was previously read via MCP). However, the change moves the content into the static prompt body rather than tool results. Consider: (1) clearly delimiting the Jira content as untrusted data in the prompt rather than 'source of truth', and (2) tightening the agent job's firewall — it still allows api.atlassian.com / sonarsource.atlassian.net even though the agent no longer needs Jira access (the prepare job does), so those domains are unnecessary egress/exfiltration surface for the agent.

💡 Edge Case: Open-PR dedup capped at 100 PRs and relies on exact key text match

📄 .github/scripts/night-owl/prepare-jira-context.sh:139-149 📄 .github/scripts/night-owl/prepare-jira-context.sh:274-280

OPEN_PULL_REQUESTS_JSON is fetched with gh pr list --state open --limit 100, and has_open_pr_for_key only matches the Jira key as a token in title/body/headRefName. If a repository has more than 100 open PRs, an in-flight ticket may not be found and Night Owl could start duplicate work on a ticket already covered by an open PR. The match also fails entirely if a previous night-owl PR did not embed the Jira key in its title/body/branch. Consider raising/removing the limit (or filtering by the night-owl label/author) and ensuring created PRs always embed the Jira key.

💡 Quality: Malformed acli search output is silently reported as 'starving'

📄 .github/scripts/night-owl/prepare-jira-context.sh:282 📄 .github/scripts/night-owl/prepare-jira-context.sh:289 📄 .github/scripts/night-owl/prepare-jira-context.sh:332 📄 .github/scripts/night-owl/prepare-jira-context.sh:334-338

The candidate loop is fed by a process substitution: done < <(normalize_collection <<<"$SEARCH_RESULTS_JSON" | jq -c '.[]?'). With set -euo pipefail, a failure inside a process substitution does not abort the parent shell. If acli ... search returns output that is not valid JSON (or an unexpected shape normalize_collection cannot match), the pipeline fails/produces nothing, the loop iterates zero times, candidate_count stays 0, and the script emits a 'No Jira ticket matched' starving Slack message and exits 0 — masking an actual infrastructure/query error as routine starvation. Consider validating SEARCH_RESULTS_JSON parses as JSON (e.g. jq empty with explicit error handling) before the loop so genuine failures route to night_owl_prepare_failure_notify instead of the starving path.

✅ 1 resolved
Security: Credentials in job output not masked with add-mask

📄 .github/workflows/night-owl.md:64-65 📄 .github/workflows/ci-failure-triage-agent.md:64-65
The atlassian_mcp_auth job writes Base64-encoded Jira credentials directly to $GITHUB_OUTPUT without first calling ::add-mask::. GitHub Actions does not automatically mask job outputs, so the authorization_header value (which is just Basic <base64(user:token)>) could appear in plain text in workflow logs of downstream jobs that reference it. Adding ::add-mask:: before writing the output ensures the value is redacted from all logs.

🤖 Prompt for agents
Code Review: Introduces the night-owl agentic workflow for Jira ticket processing and refactors CI triage infrastructure. Rename the repurposed `ci-failure-triage-agent.md`, sanitize Jira content before injection, and improve pull request deduplication logic for robustness.

1. 💡 Quality: ci-failure-triage-agent.md fully repurposed but keeps old name
   Files: .github/workflows/ci-failure-triage-agent.md:7, .github/workflows/ci-failure-triage-agent.md:122-124, .github/workflows/ci-failure-triage-agent.lock.yml:73, .github/workflows/ci-failure-triage-agent.md, .github/workflows/ci-failure-triage-agent.lock.yml:59, .github/workflows/ci-failure-triage-agent.lock.yml:61-62, .github/workflows/night-owl.lock.yml:61-62, .github/workflows/night-owl.lock.yml:73, .github/workflows/ci-failure-triage-agent.md:3-4, .github/workflows/night-owl.md:3-4, .github/workflows/night-owl.md:7

   The `ci-failure-triage-agent.md` workflow has been entirely replaced with the Night Owl logic (identical env vars, prompt, safe-outputs). This means the original CI failure triage functionality is lost. The workflow file name, the GitHub Actions `name:` field ("CI Failure Triage Agent"), and the `run-name` in the lock file are now misleading. Consider either:
   - Keeping the CI failure triage agent intact and only adding the new `night-owl.md` workflow, or
   - Renaming/removing the old file to avoid confusion.
   
   The comment on line 124 says it's "temporarily repurposed", but having two identically-functioning workflows (`night-owl.md` and `ci-failure-triage-agent.md`) with the same concurrency group (`night-owl`) and the same cron schedule could cause unintended mutual cancellation.

2. 💡 Security: Untrusted Jira content injected verbatim into autonomous agent prompt
   Files: .github/scripts/night-owl/prepare-jira-context.sh:124-137, .github/scripts/night-owl/prepare-jira-context.sh:151-165, .github/scripts/night-owl/prepare-jira-context.sh:183-197, .github/scripts/night-owl/prepare-jira-context.sh:434-446, .github/workflows/night-owl.md:200, .github/workflows/night-owl.md:217-219, .github/workflows/night-owl.lock.yml:805

   `prepare-jira-context.sh` builds `context_markdown` from Jira ticket summary, description, comments, and linked-issue fields (issue_description_text/format_comments_markdown/format_links_markdown) and emits it to GITHUB_OUTPUT. Both `night-owl.md` and `ci-failure-triage-agent.md` then inline `${{ needs.night_owl_prepare.outputs.context_markdown }}` directly into the agent prompt and instruct the agent to 'Treat the prepared Jira content above as the source of truth.' The agent has the `create_pull_request` safe output, so a Jira ticket (any CLI ticket labeled `for-agent`) whose description/comments contain adversarial instructions becomes an XPIA / prompt-injection vector that can steer autonomous code changes and PRs.
   
   This is partially mitigated by the gh-aw `xpia.md` guard prompt, and the surface is not strictly new (Jira was previously read via MCP). However, the change moves the content into the static prompt body rather than tool results. Consider: (1) clearly delimiting the Jira content as untrusted data in the prompt rather than 'source of truth', and (2) tightening the agent job's firewall — it still allows `api.atlassian.com` / `sonarsource.atlassian.net` even though the agent no longer needs Jira access (the prepare job does), so those domains are unnecessary egress/exfiltration surface for the agent.

3. 💡 Edge Case: Open-PR dedup capped at 100 PRs and relies on exact key text match
   Files: .github/scripts/night-owl/prepare-jira-context.sh:139-149, .github/scripts/night-owl/prepare-jira-context.sh:274-280

   `OPEN_PULL_REQUESTS_JSON` is fetched with `gh pr list --state open --limit 100`, and `has_open_pr_for_key` only matches the Jira key as a token in title/body/headRefName. If a repository has more than 100 open PRs, an in-flight ticket may not be found and Night Owl could start duplicate work on a ticket already covered by an open PR. The match also fails entirely if a previous night-owl PR did not embed the Jira key in its title/body/branch. Consider raising/removing the limit (or filtering by the night-owl label/author) and ensuring created PRs always embed the Jira key.

4. 💡 Quality: Malformed acli search output is silently reported as 'starving'
   Files: .github/scripts/night-owl/prepare-jira-context.sh:282, .github/scripts/night-owl/prepare-jira-context.sh:289, .github/scripts/night-owl/prepare-jira-context.sh:332, .github/scripts/night-owl/prepare-jira-context.sh:334-338

   The candidate loop is fed by a process substitution: `done < <(normalize_collection <<<"$SEARCH_RESULTS_JSON" | jq -c '.[]?')`. With `set -euo pipefail`, a failure inside a process substitution does not abort the parent shell. If `acli ... search` returns output that is not valid JSON (or an unexpected shape `normalize_collection` cannot match), the pipeline fails/produces nothing, the loop iterates zero times, `candidate_count` stays 0, and the script emits a 'No Jira ticket matched' starving Slack message and exits 0 — masking an actual infrastructure/query error as routine starvation. Consider validating `SEARCH_RESULTS_JSON` parses as JSON (e.g. `jq empty` with explicit error handling) before the loop so genuine failures route to `night_owl_prepare_failure_notify` instead of the starving path.

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud

sonarqubecloud Bot commented Jun 3, 2026

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant