Skip to content

chore(release): make daily tag flow semi-automatic - #8485

Open
cjagwani wants to merge 2 commits into
mainfrom
codex/semi-automatic-daily-tag
Open

chore(release): make daily tag flow semi-automatic#8485
cjagwani wants to merge 2 commits into
mainfrom
codex/semi-automatic-daily-tag

Conversation

@cjagwani

@cjagwani cjagwani commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Before this change, the release skill required a candidate-bound E2E ledger before a maintainer-confirmed tag. After this change, the 4 PM release remains maintainer-confirmed and locally signed, but E2E runs asynchronously on every push to main and never blocks tagging or merging.

Changes

  • Keep the daily tag semi-automatic: the agent verifies the merged dated changelog and housekeeping, generates the release plan, exercises the maintainer workstation's signer, and requests the exact confirmation phrase.
  • Keep release signing keys out of GitHub Actions and do not introduce a release bot.
  • Remove the obsolete pre-tag E2E evidence command, implementation, and tests.
  • Update maintainer skills and policy for SHA-bound post-merge E2E triage from 4 PM through 8 AM while merges continue normally.
  • Update the maintainer skill and policy contract tests.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This changes maintainer-only release policy and removes an internal E2E evidence helper; no public CLI, configuration, runtime default, or support claim changes.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: The existing local signing boundary remains unchanged; policy and contract tests explicitly reject a GitHub Actions signing secret or release bot.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: No docs/ page owns the maintainer-only daily tag procedure. Carlos's merged ci(e2e): run every workflow E2E on main #8477 owns and documents full E2E on every main push; historical changelog entries remain unchanged.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — initial skill/policy suite: npm exec vitest run -- test/maintainer-skills-policy.test.ts test/maintainer-e2e-skill.test.ts --testTimeout=30000 (33 passed); review fix: npm exec vitest run -- test/maintainer-skills-policy.test.ts --testTimeout=30000 (19 passed), plus direct portable command execution against v0.0.103 (exactly one match)
  • Applicable broad gate passed — Not applicable; focused maintainer skill/policy contracts and normal hooks cover this change.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Charan Jagwani cjagwani@nvidia.com

Summary by CodeRabbit

  • Release Process

    • Release tagging is now scheduled for 4 PM Pacific and requires a merged changelog, housekeeping verification, explicit maintainer confirmation, and local signing.
    • Releases can proceed independently of E2E test status.
    • Overnight E2E results are handled asynchronously for post-release stabilization and triage.
  • Documentation

    • Updated maintainer guidance to reflect the revised release schedule, confirmation process, and E2E workflow.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@cjagwani cjagwani self-assigned this Aug 6, 2026
@coderabbitai

coderabbitai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The release process now uses a 4 PM locally signed tag based on the planned origin/main commit. E2E runs are SHA-bound diagnostic evidence and no longer block tagging or merging. Maintainer guidance, policy references, and tests reflect asynchronous overnight stabilization.

Changes

Release flow

Layer / File(s) Summary
Release-tag preparation and verification
.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md, .agents/skills/nemoclaw-skills-guide/SKILL.md, test/maintainer-skills-policy.test.ts
The release skill verifies the dated changelog, generates a current-main plan, confirms a local signer, creates a signed tag after exact maintainer confirmation, and validates release state. The tests validate semver tags and matching changelog headings.
Daily cadence and overnight stabilization
.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md, .agents/skills/nemoclaw-maintainer-evening/SKILL.md, .agents/skills/nemoclaw-maintainer-policies/references/*
The 4 PM release boundary permits continued merging. E2E failures move to overnight SHA-based triage instead of blocking tags or merges.
SHA-bound diagnostic evidence
.agents/skills/nemoclaw-maintainer-e2e/SKILL.md, test/maintainer-e2e-skill.test.ts
Full E2E runs against origin/main are classified as diagnostic evidence tied to the validated SHA. Release E2E evidence generation and its dedicated tests were removed.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ReleaseAgent
  participant Maintainer
  participant originMain
  participant E2EWorkflow

  ReleaseAgent->>originMain: Generate 4 PM release plan
  ReleaseAgent->>Maintainer: Request exact tag confirmation
  Maintainer->>ReleaseAgent: Confirm local signing
  ReleaseAgent->>originMain: Create signed tag with release:cut
  originMain->>E2EWorkflow: Trigger SHA-bound E2E run
  E2EWorkflow-->>ReleaseAgent: Return diagnostic result for overnight triage
Loading

Possibly related PRs

Suggested labels: area: ci, area: e2e, release-target

Suggested reviewers: apurvvkumaria, cv

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the main change: making the daily release tag flow semi-automatic.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/semi-automatic-daily-tag

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md:
- Around line 20-21: Update the changelog validation instructions around the
line-40 search to use the selected release version rather than the literal
vX.Y.Z placeholder. Require exactly one dated docs/changelog/YYYY-MM-DD.mdx file
containing the exact planned ## vX.Y.Z heading, and fail validation when zero or
multiple entries match.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 026e4a89-ea4b-4838-9ba9-f79f2f697508

📥 Commits

Reviewing files that changed from the base of the PR and between affe641 and 5257437.

📒 Files selected for processing (12)
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts
  • .agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md
  • .agents/skills/nemoclaw-maintainer-e2e/SKILL.md
  • .agents/skills/nemoclaw-maintainer-evening/SKILL.md
  • .agents/skills/nemoclaw-maintainer-policies/references/daily-flow.md
  • .agents/skills/nemoclaw-maintainer-policies/references/release-train.md
  • .agents/skills/nemoclaw-skills-guide/SKILL.md
  • package.json
  • test/maintainer-e2e-skill.test.ts
  • test/maintainer-skills-policy.test.ts
  • test/release-e2e-evidence.test.ts
💤 Files with no reviewable changes (3)
  • package.json
  • test/release-e2e-evidence.test.ts
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts

Comment thread .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/maintainer-skills-policy.test.ts (1)

246-254: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Test changelog validation through observable behavior.

These assertions only check that implementation fragments occur in the Markdown. They do not prove that one matching changelog succeeds or that zero and multiple matches fail. Execute the validation command against temporary Git fixtures, or expose it through a testable script and assert its exit status and output.

As per path instructions, tests must provide behavioral confidence rather than implementation lock-in.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/maintainer-skills-policy.test.ts` around lines 246 - 254, Replace the
fragment-based assertions in the changelog validation section of
maintainer-skills-policy tests with behavioral tests that run the release
validation command against temporary Git fixtures. Cover exactly one matching
dated changelog heading as success, and zero or multiple matches as failure,
asserting exit status and relevant output rather than script text.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/maintainer-skills-policy.test.ts`:
- Around line 246-254: Replace the fragment-based assertions in the changelog
validation section of maintainer-skills-policy tests with behavioral tests that
run the release validation command against temporary Git fixtures. Cover exactly
one matching dated changelog heading as success, and zero or multiple matches as
failure, asserting exit status and relevant output rather than script text.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c7fde54b-1ef5-48bc-a935-bfc551fc7b64

📥 Commits

Reviewing files that changed from the base of the PR and between 5257437 and 26ef0cf.

📒 Files selected for processing (2)
  • .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md
  • test/maintainer-skills-policy.test.ts

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Failed after a partial review · low confidence · 2 blockers · 1 warning · 0 suggestions

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

3 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • justified — semi-automatic at .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md:12: Keep the term with the stated maintainer-confirmation and local-signing contrast.
  • define — asynchronous E2E stabilization at .agents/skills/nemoclaw-maintainer-policies/references/release-train.md:48: Retain the section definition that specifies the time window, inputs, and non-gating result.
  • justified — SHA-bound diagnostic evidence at .agents/skills/nemoclaw-maintainer-e2e/SKILL.md:340: Keep the modifier and the adjacent statement that limits this evidence to diagnostic use.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: cloud-inference, cloud-onboard, security-posture

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

@github-code-quality

github-code-quality Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 26ef0cf in the codex/semi-automatic... branch remains at 96%, unchanged from commit 0d33997 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 26ef0cf in the codex/semi-automatic... branch remains at 81%, unchanged from commit 0d33997 in the main branch.

Show a code coverage summary of the most impacted files.
File main 0d33997 codex/semi-automatic... 26ef0cf +/-
src/lib/inferen...a/model-size.ts 96% 83% -13%
src/lib/shields/index.ts 83% 82% -1%
src/lib/shields...nsition-lock.ts 84% 83% -1%
src/lib/adapter...teway-server.ts 0% 79% +79%
src/lib/voice-g...nclaw-client.ts 0% 83% +83%
src/lib/voice-g...dential-file.ts 0% 88% +88%
src/lib/actions...ateway/serve.ts 0% 91% +91%
src/lib/voice-g...sion-service.ts 0% 93% +93%
src/lib/hermes-...anaged-route.ts 0% 94% +94%
src/lib/voice-g...ay/contracts.ts 0% 100% +100%

Updated August 06, 2026 23:56 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant