Skip to content

feat(ci): enrich architecture drift report with actionable details#30

Open
tuannx wants to merge 2 commits into
mainfrom
feat/enrich-drift-report
Open

feat(ci): enrich architecture drift report with actionable details#30
tuannx wants to merge 2 commits into
mainfrom
feat/enrich-drift-report

Conversation

@tuannx

@tuannx tuannx commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Problem

The Architecture Drift Report posted on PRs (e.g. #29 comment) was too sparse to be actionable:

  • All baseline metric columns showed — metrics were never persisted in baseline.json, so no delta could ever be computed
  • Entity movements were a bare count — "52 entity movement(s)" tells you nothing about WHAT moved WHERE
  • No component breakdown — reviewers can't see the current state at a glance
  • No visual — no diagram to orient reviewers
  • No smell guidance — "Concern Overload: Tests" with no recommendation
  • Detailed analysis only on main-targeted PRs — PRs to feature branches (e.g. codex/rust-parser) never got the rich "🤖 Architecture Analysis Summary" because ci.yml had pull_request: branches: [main]

Solution

1. Baseline persists metrics (backwards compatible)

save_architecture() now accepts an optional metrics dict. load_architecture() returns (Architecture, dict[str, float]). Old baselines without metrics still load fine (empty dict).

Once --update-baseline runs with this code, all future PR reports will show real deltas with direction indicators:

  • 🟢 metric improved
  • 🔴 metric degraded
  • ⚪ unchanged (< 0.005 threshold)

2. Entity movement details

Top-15 individual entity movements listed as entity → Component, with "… and N more" overflow.

3. Component breakdown table

Component Entities Responsibility

4. Mermaid component dependency diagram

5. Actionable smell recommendations

Each smell now includes a 💡 recommendation (split component, break cycle, reduce fan-in, stabilize interface).

6. Self-dogfooding runs on ALL PRs

Removed branches: [main] filter from ci.yml's pull_request trigger. Now every PR (including those targeting feature branches) receives the full "🤖 Architecture Analysis Summary" with metric evolution, principle signals, score drivers, Mermaid diagram, and A2A comparison. Baseline is still sourced from the last successful main push.

Breaking change

load_architecture() signature changed from -> Architecture to -> tuple[Architecture, dict[str, float]]. All internal callers updated.

Validation

  • 278 tests pass, 9 skipped (MCP optional deps)
  • Ruff clean
  • Smoke-tested locally: report renders correctly with all new sections

The drift report posted on PRs was too sparse to be actionable:
- All baseline metric columns showed '—' (metrics never persisted)
- Entity movements were a bare count with no details
- No component breakdown, no diagram, no smell guidance

Changes:
1. Baseline now persists metrics alongside architecture (backwards
   compatible — old baselines without metrics still load fine).
2. Metric table shows real Baseline→Current deltas with direction
   indicators (🟢 improved / �� degraded / ⚪ unchanged).
3. Entity movements list top-15 individual moves (entity → component).
4. Component breakdown table (name, entity count, responsibility).
5. Mermaid component dependency diagram for visual orientation.
6. Smell entries include actionable 💡 recommendations.

Breaking change: load_architecture() now returns (Architecture, dict)
tuple. All callers updated.
Copilot AI review requested due to automatic review settings July 23, 2026 14:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown
Contributor

Architecture Drift Report

Algorithm: PKG | Entities: 673 | Components: 7

Drift from Baseline

Metric Baseline Current Delta
Components 7 7 +0
Similarity 1.00
BalancedArchitectureScore 0.68
PrincipleAlignmentScore 0.84
RCI 0.99
TurboMQ 0.24
BasicMQ 0.24
IntraConnectivity 0.00
InterConnectivity 0.01
TwoWayPairRatio 0.00
DependencyHealth 0.99
ComponentBalance 0.23
HubBalance 0.67
BoundaryClarity 1.00
DependencyDistribution 0.57
SmellDiscipline 0.91

Changes

  • 3 entity movement(s) between components
    • src.arcade_agent.ci.arch_diff._direction_iconArcadeAgent
    • src.arcade_agent.ci.arch_diff._metric_rowArcadeAgent
    • tests.test_serialization.test_save_load_with_metricsArcadeAgent

Components

Component Entities Responsibility
ArcadeAgent 587 Entities in src.arcade_agent
Tests 40 Entities in tests
TestTools 15 Entities in tests.test_tools
Fixtures 12 Entities in tests.fixtures
TestParsers 11 Entities in tests.test_parsers
Scripts 4 Entities in scripts
TestAlgorithms 4 Entities in tests.test_algorithms

Architecture Diagram

graph LR
    Scripts["Scripts"]
    ArcadeAgent["ArcadeAgent"]
    Tests["Tests"]
    Fixtures["Fixtures"]
    TestAlgorithms["TestAlgorithms"]
    TestParsers["TestParsers"]
    TestTools["TestTools"]
    Scripts --> ArcadeAgent
    Scripts --> Fixtures
    Tests --> ArcadeAgent
    Tests --> Fixtures
Loading

Smells (1)

  • Concern Overload: Tests
    • 💡 Split the overloaded component into smaller, focused units. Consider extracting sub-packages or introducing an interface layer.

Generated by arcade-agent

@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

🤖 Architecture Analysis Summary

Powered by arcade-agent — automatic architectural self-analysis


📈 Metric Evolution

Baseline commit: 746fc56

Legend: 🟢 better · 🔴 worse · 🟡 low impact · ⚪ no change

Metric Baseline Current Change
BalancedArchitectureScore 0.6193 0.6193 → (no change)
📦 Components 9 9 → (no change)
🧩 Entities 115 115 → (no change)
🔗 Edges 115 115 → (no change)
🏷️ Classes 22 22 → (no change)
ƒ Functions 271 273 🟡 ↑ (+2.0000)
🔧 Methods 34 34 → (no change)
RCI 0.5652 0.5652 → (no change)
TurboMQ 0.2493 0.2493 → (no change)
BasicMQ 0.2493 0.2493 → (no change)
IntraConnectivity 0.0182 0.0182 → (no change)
InterConnectivity 0.0323 0.0323 → (no change)
TwoWayPairRatio 0.0000 0.0000 → (no change)
DependencyHealth 0.9822 0.9822 → (no change)
ComponentBalance 0.5304 0.5304 → (no change)
HubBalance 0.5000 0.5000 → (no change)
BoundaryClarity 0.9444 0.9444 → (no change)
DependencyDistribution 0.6263 0.6263 → (no change)
SmellDiscipline 0.9333 0.9333 → (no change)
PrincipleAlignmentScore 0.8553 0.8553 → (no change)

🏛️ Current Architecture

Metric Value
📦 Components 9
🧩 Entities 115
🔗 Edges 115
🏷️ Classes 22
ƒ Functions 273
🔧 Methods 34
Balanced Score 🟡 0.6193 (Fair)
Principle Alignment 0.8553
RCI 0.5652
TurboMQ 0.2493
BasicMQ 0.2493
IntraConnectivity 0.0182
InterConnectivity 0.0323
TwoWayPairRatio 0.0000
DependencyHealth 0.9822
ComponentBalance 0.5304
HubBalance 0.5000
BoundaryClarity 0.9444
DependencyDistribution 0.6263
SmellDiscipline 0.9333

🧭 Principle Signals

Signal Value
AcyclicDependencies 1.0000
LayeringHealth 0.9867
ResponsibilityFocus 0.9333
InterfaceSegregation 1.0000
ComponentBalance 0.5304
HubBalance 0.5000
BoundaryClarity 0.9444
DependencyDistribution 0.6263
SmellDiscipline 0.9333

🎯 Score Drivers

Biggest risks

  • HubBalance: gap=0.5000 (signal=0.5000)
  • ComponentBalance: gap=0.4696 (signal=0.5304)
  • DependencyDistribution: gap=0.3737 (signal=0.6263)

Strongest areas

  • AcyclicDependencies: gap=0.0000 (signal=1.0000)
  • InterfaceSegregation: gap=0.0000 (signal=1.0000)
  • LayeringHealth: gap=0.0133 (signal=0.9867)

🕸️ High-Level Design

graph TD
    Algorithms["Algorithms\n39 entities\n5 classes / 3 methods"]
    Budget["Budget\n3 entities\n0 classes / 0 methods"]
    Cache["Cache\n4 entities\n0 classes / 0 methods"]
    Ci["Ci\n5 entities\n0 classes / 0 methods"]
    Exporters["Exporters\n14 entities\n1 classes / 0 methods"]
    Incremental["Incremental\n2 entities\n1 classes / 2 methods"]
    Parsers["Parsers\n17 entities\n11 classes / 27 methods"]
    Serialization["Serialization\n8 entities\n0 classes / 0 methods"]
    Tools["Tools\n23 entities\n4 classes / 2 methods"]
    Cache --> Serialization
    Ci --> Algorithms
    Ci --> Exporters
    Ci --> Serialization
    Ci --> Tools
    Serialization --> Algorithms
    Serialization --> Parsers
    Tools --> Algorithms
    Tools --> Cache
    Tools --> Exporters
    Tools --> Parsers
Loading
🏗️ Components breakdown
Component Entities Classes Methods
Algorithms 39 5 3
Tools 23 4 2
Parsers 17 11 27
Exporters 14 1 0
Serialization 8 0 0
Ci 5 0 0
Cache 4 0 0
Budget 3 0 0
Incremental 2 1 2

🚨 Architectural Smells

Severity Type Affected Components
🟡 medium Concern Overload Tools

📈 Evolution vs Baseline

Baseline commit: 746fc56

Architecture-to-Architecture (A2A) Comparison

Metric Value
A2A Similarity 1.0000
Matched Components 9
Components Added 0
Components Removed 0
Component matching details

Matched:

Baseline Current Similarity
Algorithms Algorithms 1.0000
Budget Budget 1.0000
Cache Cache 1.0000
Ci Ci 1.0000
Exporters Exporters 1.0000
Incremental Incremental 1.0000
Parsers Parsers 1.0000
Serialization Serialization 1.0000
Tools Tools 1.0000
High-level component statistics
Status Baseline Current Similarity Entities Classes Methods
matched Algorithms Algorithms 1.0000 39 → 39 (0) 5 → 5 (0) 3 → 3 (0)
matched Budget Budget 1.0000 3 → 3 (0) 0 → 0 (0) 0 → 0 (0)
matched Cache Cache 1.0000 4 → 4 (0) 0 → 0 (0) 0 → 0 (0)
matched Ci Ci 1.0000 5 → 5 (0) 0 → 0 (0) 0 → 0 (0)
matched Exporters Exporters 1.0000 14 → 14 (0) 1 → 1 (0) 0 → 0 (0)
matched Incremental Incremental 1.0000 2 → 2 (0) 1 → 1 (0) 2 → 2 (0)
matched Parsers Parsers 1.0000 17 → 17 (0) 11 → 11 (0) 27 → 27 (0)
matched Serialization Serialization 1.0000 8 → 8 (0) 0 → 0 (0) 0 → 0 (0)
matched Tools Tools 1.0000 23 → 23 (0) 4 → 4 (0) 2 → 2 (0)
Before/After Mermaid diagrams

Baseline

graph TD
    Algorithms["Algorithms\n39 entities\n5 classes / 3 methods"]
    Budget["Budget\n3 entities\n0 classes / 0 methods"]
    Cache["Cache\n4 entities\n0 classes / 0 methods"]
    Ci["Ci\n5 entities\n0 classes / 0 methods"]
    Exporters["Exporters\n14 entities\n1 classes / 0 methods"]
    Incremental["Incremental\n2 entities\n1 classes / 2 methods"]
    Parsers["Parsers\n17 entities\n11 classes / 27 methods"]
    Serialization["Serialization\n8 entities\n0 classes / 0 methods"]
    Tools["Tools\n23 entities\n4 classes / 2 methods"]
    Cache --> Serialization
    Ci --> Algorithms
    Ci --> Exporters
    Ci --> Serialization
    Ci --> Tools
    Serialization --> Algorithms
    Serialization --> Parsers
    Tools --> Algorithms
    Tools --> Cache
    Tools --> Exporters
    Tools --> Parsers
Loading

Current

graph TD
    Algorithms["Algorithms\n39 entities\n5 classes / 3 methods"]
    Budget["Budget\n3 entities\n0 classes / 0 methods"]
    Cache["Cache\n4 entities\n0 classes / 0 methods"]
    Ci["Ci\n5 entities\n0 classes / 0 methods"]
    Exporters["Exporters\n14 entities\n1 classes / 0 methods"]
    Incremental["Incremental\n2 entities\n1 classes / 2 methods"]
    Parsers["Parsers\n17 entities\n11 classes / 27 methods"]
    Serialization["Serialization\n8 entities\n0 classes / 0 methods"]
    Tools["Tools\n23 entities\n4 classes / 2 methods"]
    Cache --> Serialization
    Ci --> Algorithms
    Ci --> Exporters
    Ci --> Serialization
    Ci --> Tools
    Serialization --> Algorithms
    Serialization --> Parsers
    Tools --> Algorithms
    Tools --> Cache
    Tools --> Exporters
    Tools --> Parsers
Loading
Component dependency delta
Status Source Target
matched No dependency delta -

💡 CI/CD Insights

  • Quality Score: 🟡 Fair (BalancedArchitectureScore=0.6193)
  • Principle Alignment: 0.8553 (higher means cleaner layering, focus, and boundaries)
  • Top Risk Driver: HubBalance (signal=0.5000)
  • Trend: ➡️ Stable architectural quality
  • Architecture Stability: 🟢 High (A2A=1.0000)
  • Smells: ⚠️ 1 smell(s) — review suggested

📄 View HTML reports and artifacts


This comment is auto-generated by the self-dogfooding CI job. It updates on every push to this PR.

@tuannx
tuannx requested a review from lemduc July 23, 2026 15:02
Previously the detailed '🤖 Architecture Analysis Summary' comment
(metric evolution, principle signals, score drivers, Mermaid diagram,
A2A comparison) only appeared on PRs targeting main because ci.yml
had 'pull_request: branches: [main]'.

PRs targeting feature branches (e.g. codex/rust-parser) only got the
sparse arch-drift.yml comment. Remove the branch filter so every PR
receives the full analysis. The baseline artifact is still sourced
from the last successful main push, so comparisons remain meaningful.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants