Skip to content

fix: improve commit message reliability#36

Merged
SnowCheetos merged 4 commits into
mainfrom
fix/commit-message-reliability
Jun 29, 2026
Merged

fix: improve commit message reliability#36
SnowCheetos merged 4 commits into
mainfrom
fix/commit-message-reliability

Conversation

@SnowCheetos

Copy link
Copy Markdown
Contributor

Summary

Two root-cause fixes addressing issues found in real-world usage of autocommit on PR rust-osdev/acpi#306, where the maintainer noted that commit messages were confusing, hallucinated, and individual commits were not useful for review.

1. Literal model-free path for small diffs

Bypasses the LLM entirely for diffs under 20 lines across at most 2 files. The new literal_report() function in reduce.rs generates commit messages directly from diff structure:

  • Counts additions and deletions
  • Extracts symbol declarations from both added and removed lines
  • Detects renames (same short prefix on added and removed symbol)
  • Classifies as feat (additions only), refactor (removals/renames), fix (mixed), docs, or test
  • Zero inference cost, zero hallucination risk

2. Version bumps only when manifest is explicitly changed

Changed should_evaluate in version_bump.rs to require the manifest file itself (Cargo.toml, package.json, etc.) to be in the diff. Previously, touching any source file in a crate would trigger a version bump suggestion, causing noisy suggestions on every intermediate commit within a PR.

Additional improvements

  • Prompt rules now explicitly ban style: for any change touching executable code
  • Analyze prompt warns when a chunk is under 15 lines: describe what was literally changed, do not infer intent
  • Boilerplate risk notes (no security risks, no data loss) are filtered from commit output; risk section omitted when level is low
  • --no-bump flag added as explicit opt-out for version suggestions

Files changed

  • crates/core/src/pipeline/reduce.rs - literal_report() and helpers (add/delete counting, declaration extraction, classification)
  • crates/core/src/pipeline/analyze.rs - route small diffs to literal path
  • crates/core/src/llm/prompts.rs - better type classification rules, small-diff warnings
  • crates/cli/src/cmd/version_bump.rs - only evaluate bumps when manifest changed
  • crates/cli/src/cmd/commit.rs - --no-bump flag, boilerplate risk note filtering

Copilot AI review requested due to automatic review settings June 29, 2026 04:43
…on and noisy version bumps

Two root-cause fixes addressing issues found in real-world usage
(see rust-osdev/acpi#306 for concrete examples of failing output):

1. Literal model-free path for small diffs (<=20 lines, <=2 files):
   Bypass the LLM entirely for tiny changes. The new literal_report()
   function generates commit messages directly from diff structure:
   counts additions/deletions, extracts symbol declarations from +/- lines,
   detects renames, and classifies as feat/refactor/fix/docs/test.
   Zero inference cost, zero hallucination risk.

2. Version bumps only when manifest is explicitly changed:
   Previously, touching any source file in a crate would trigger a version
   bump suggestion. Changed should_evaluate to require the manifest file
   itself (Cargo.toml, package.json, etc.) to be in the diff. This stops
   noisy version bump suggestions on every intermediate commit within PRs.

Additional improvements:
- Prompt rules explicitly ban style: for any change touching executable code
- Analyze prompt warns when a chunk is <15 lines to discourage intent inference
- Boilerplate risk notes ("no security risks", "no data loss") are filtered
  from commit output; risk section omitted entirely when level is low
- --no-bump flag added as an explicit opt-out for version suggestions
@SnowCheetos
SnowCheetos force-pushed the fix/commit-message-reliability branch from 81b5cd4 to 869033d Compare June 29, 2026 04:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves commit message reliability by introducing a model-free “literal” reporting path for very small diffs and by reducing noisy version bump recommendations unless a manifest file actually changes.

Changes:

  • Add literal_report() (and helpers) to synthesize commit metadata directly from diff structure for small diffs, bypassing the LLM.
  • Route DraftOnly + very small diffs through the literal path in analyze.rs.
  • Tighten prompt guidance around type classification, small-diff behavior, and risk-note boilerplate; add --no-bump and filter boilerplate risk notes in commit output.
  • Only evaluate version bumps when the manifest file itself is present in the diff.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
crates/core/src/pipeline/reduce.rs Adds literal diff-to-report synthesis and tests; includes change classification and title/intent generation helpers.
crates/core/src/pipeline/analyze.rs Routes very small DraftOnly diffs to the new literal report path; avoids moving key_symbols.
crates/core/src/llm/prompts.rs Tightens prompt rules and injects a small-diff warning into analyze prompts.
crates/cli/src/cmd/version_bump.rs Limits version bump evaluation to cases where the manifest file changed.
crates/cli/src/cmd/commit.rs Adds --no-bump to skip bump logic and filters boilerplate risk notes / omits low-risk sections when empty.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/core/src/pipeline/reduce.rs
Comment thread crates/core/src/pipeline/reduce.rs Outdated
Comment thread crates/core/src/pipeline/reduce.rs Outdated
Comment on lines +237 to +242
if path_lower.ends_with(".md")
|| path_lower.ends_with("readme")
|| path_lower.contains("doc")
{
return TypeTag::Docs;
}
Comment thread crates/core/src/llm/prompts.rs Outdated
SnowCheetos and others added 3 commits June 28, 2026 23:51
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Cass Sheng <86272122+SnowCheetos@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Cass Sheng <86272122+SnowCheetos@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Signed-off-by: Cass Sheng <86272122+SnowCheetos@users.noreply.github.com>
@SnowCheetos
SnowCheetos merged commit afb7322 into main Jun 29, 2026
7 of 8 checks passed
@SnowCheetos
SnowCheetos deleted the fix/commit-message-reliability branch July 6, 2026 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants