Skip to content

feat(AF-1026): deployment verification-failed notifications and audit - #1235

Merged
babltiga merged 2 commits into
mainfrom
feature/AF-1026-deploygov-verification-notifications
Nov 11, 2026
Merged

babltiga merged 2 commits into
mainfrom
feature/AF-1026-deploygov-verification-notifications

Conversation

@babltiga

Copy link
Copy Markdown
Contributor

Closes #1026 (part of epic #1022).

What

A breached post-deployment verification (#1025) used to flip the request EXECUTED → FAILED without telling anyone. This change notifies the people who can act on it and records the breach in the audit log.

  • Audit. A new DEPLOYMENT_VERIFICATION_FAILED AuditAction is written by DeploymentRequestStateService.markVerificationFailed. It is a system row: the actor is null and trigger=verification. Metadata holds pipeline_id, version, check and detail; check is left out when the config snapshot could not be read. markVerificationFailed now also takes the check name.
  • Notification. A new DEPLOYMENT_VERIFICATION_FAILED NotificationEventType goes to the reviewers who approved the deployment plus the submitter. Duplicates are removed and inactive users dropped. A request with no approvers (auto-approved or break-glass) notifies the submitter alone.
    • It fans out to every active channel in the org. It never pages and never opens a ticket, because PagerDutyTrigger and TicketingTrigger do not map it; tests pin that.
    • The wording recommends and never claims a rollback happened: "The release is still running — AccessFlow did not roll it back. Decide whether to roll back, then report ROLLED_BACK."
  • Email. A new template, email/deployment-verification-failed.html, is wired into all three switches in the email strategy. Its keys are in all seven messages*.properties files.
  • Chat. Slack, Teams, Telegram and Discord each add a Verification field and a Next-step field, built by the shared DeploymentVerificationText.
  • Webhook. The webhook body shape is unchanged; the detail travels in the existing deployment.decision_reason.
  • In-app. The notification payload gains a verification_detail field, so the bell names the failed check.
    • The bell routes to /deployments/{id}, never to the rollback worklist, because nothing was rolled back.
    • A notification.created for this event refreshes the deployment detail and list queries.
    • The audit-log filter gains the action, and all seven locales have the key.
  • Docs. Updated docs/08-notifications.md, docs/18-deployment-governance.md, docs/03-data-model.md, docs/05-backend.md, docs/04-api-spec.md and docs/06-frontend.md. help-corpus/ was regenerated because en.json changed.

No migration. audit_log.action and user_notifications.event_type are both VARCHAR.

Website: no change. No website page lists deployment notification events, and the epic's website sweep is #1028.

E2E: none added. No existing flow changes behaviour. The verification UI and its spec are #1027.

Screenshots: none. The only UI change is the text of one bell entry.

Verification

  • mvn -o verify on the first commit: 12,169 tests, 0 failures, 6 skipped. It ran before the review fixes in the second commit. The 291 backend tests those fixes touch were re-run afterwards and pass, including ApplicationModulesTest and MessagesParityTest.
  • Frontend:
    • npm run lint passes with 0 errors (warnings were already there).
    • npm run typecheck and npm run build pass.
    • npm run test:coverage: 3,187 tests pass, with 96.7% line and 89.5% branch coverage.
  • node .github/scripts/build-help-corpus.mjs produces no further changes when re-run.

Review notes

Five read-only reviewers (verifier, cross-cutting, Java, frontend, content) ran in parallel. None found a blocker. The fixes for their concerns are in the second commit:

  • af-java-reviewer: the email body claimed a check reached its threshold even on the path where the config could not be read. It is reworded in all six languages so it is true on both paths.
  • af-reviewer: the bell could not name the failed check, so the payload now carries verification_detail. The audit-action reference in docs/03 and the paragraph in docs/05 also needed the new action.
  • af-content-reviewer: docs/08 described the detail format wrongly and didn't mention the unreadable-config message; both are fixed.

Concerns left open for a human reviewer:

  • af-reviewer: the detail reads <check>: <failed>/<failure_threshold> failed samples, and at breach time it is usually "3/3". That can be read as "every sample failed". The format comes from deploygov: DeploymentVerificationJob — window, sampling and breach → FAILED #1025 and is part of the documented API (verification_detail in docs/04/docs/05), so I kept it and spelled out the meaning in docs/08. Rewording it is a possible follow-up.
  • af-java-reviewer: deployments are triggered with an API key, so the "submitter" is often a service account rather than the person named in on_behalf_of_user_id. This matches the design agreed on the issue and the existing APPROVED/REJECTED behaviour. Notifying that person is a possible follow-up.
  • af-reviewer: the unreadable-config detail is the hard-coded English string from deploygov: DeploymentVerificationJob — window, sampling and breach → FAILED #1025. It now reaches recipients who read other languages.

@github-actions

Copy link
Copy Markdown
Contributor

Backend Test Results

12 170 tests  +20   12 170 ✅ +20   9m 50s ⏱️ - 2m 31s
 1 266 suites + 1        0 💤 ± 0 
 1 266 files   + 1        0 ❌ ± 0 

Results for commit 18e927e. ± Comparison against base commit 0dbfbc7.

@github-actions

Copy link
Copy Markdown
Contributor

Backend Code Coverage

Overall Project 95.48% 🍏
Files changed 100% 🍏

File Coverage
DeploymentNotificationListener.java 100% 🍏
NotificationContext.java 100% 🍏
NotificationEventType.java 100% 🍏
DeploymentRequestStateService.java 100% 🍏
AuditAction.java 100% 🍏
DeploymentVerificationText.java 100% 🍏
DefaultDeploymentVerificationService.java 99.39% 🍏
NotificationContextBuilder.java 91.97% 🍏
MsTeamsPayloadFactory.java 89.78% 🍏
SlackBlockKitFactory.java 89.38% 🍏
TelegramMessageFactory.java 88.86% 🍏
DiscordPayloadFactory.java 88.71% 🍏
NotificationDispatcher.java 84.79% 🍏
EmailNotificationStrategy.java 79.4% 🍏

@github-actions

Copy link
Copy Markdown
Contributor

Frontend Test Results

    1 files    360 suites   10m 24s ⏱️
3 185 tests 3 185 ✅ 0 💤 0 ❌
3 187 runs  3 187 ✅ 0 💤 0 ❌

Results for commit 18e927e.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report for Frontend Coverage (frontend)

Status Category Percentage Covered / Total
🟢 Lines 96.69% (🎯 90%) 4500 / 4654
🟢 Statements 95.41% (🎯 90%) 4973 / 5212
🟢 Functions 94.85% (🎯 90%) 1384 / 1459
🟢 Branches 89.51% (🎯 80%) 3254 / 3635
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
frontend/src/realtime/websocketManager.ts 87.97% 73.19% 94.44% 91.41% 35, 64, 87, 92, 99-100, 146, 149, 155, 163, 193-194, 249-262
Generated in workflow #1709 for commit 18e927e by the Vitest Coverage Report Action

@babltiga
babltiga merged commit d3bacb0 into main Nov 11, 2026
35 checks passed
@babltiga
babltiga deleted the feature/AF-1026-deploygov-verification-notifications branch November 11, 2026 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

notifications: DEPLOYMENT_VERIFICATION_FAILED fan-out and audit

1 participant