README vs-context7 chart: per-library matplotlib SVG (revises #46) - #51
Merged
Conversation
…otlib SVG (#50) Revises #46's aggregate ASCII block (6/6 vs 0/6) into a per-library scorecard — each of 6 fast-moving libs shows LiveDocs' current version (✓) beside context7's actual default-match version and how far behind (react → v18 a major behind; vite → 8.0.10 one minor; the rest version-less). Convincing through transparency (real, checkable versions), not spin. - chart.py: chart_rows (pure, a verbatim projection of the snapshot; reuses judge_row so chart and CLI can't disagree) + render_svg (matplotlib, deterministic — fixed svg.hashsalt, fonttype=none so version strings stay real <text> and greppable, Date metadata stripped at savefig). - compare_context7.py: --emit-chart --write/--check (SVG asset guard, reusing #46's argparse/atomic-write discipline); render_readme_block now embeds the image + a VISIBLE markdown caption carrying the honesty caveats + capture date. Removed the now-dead render_bar_chart (+ its 2 tests). - sample.json: enriched each context7 entry with structured default_version / default_status (restating the recorded prose) so chart_rows needs no fragile prose-parsing (file also reformatted to standard indented JSON). - Drift guard shifts to the DATA layer (measured, not asserted — #27/#29): a test asserts the committed SVG's version tokens == the snapshot, robust across matplotlib upgrades unlike a full byte-diff. matplotlib added to requirements; docs/assets/vs-context7-freshness.svg generated.
Reconcile after the chart change: test_chart.py +8, test_compare.py 22→20 (dropped the 2 dead render_bar_chart tests). Via scripts/sync-test-counts.sh. Milestone prose (0 → 50 Python) left frozen.
Opus ensemble (5 lenses + devil's-advocate) surfaced real holes in the guard + honesty framing — closed here: - HIGH (byte-drift): render_svg baked `Matplotlib v3.10.6` into the SVG (only Date was stripped), so `--emit-chart --check` false-drifted on any matplotlib bump. Now strips `Creator` too → the committed SVG carries no version string. - Guard strengthened byte→SEMANTIC + version-independent: test parses the SVG's per-library <text> cells POSITIONALLY and checks each with the harness's `version_matches` boundary matcher — catches a stale number, a false substring (1.0.2 vs 1.0.228), a COLUMN SWAP (proven by a new test), and a hand-tuned headline score. Dropped the matplotlib-fragile byte-exact pytest assertion. - Chart headline now derives from `tally()` (shared contested-set denominator with the README caption + CLI table) — the SVG can no longer contradict the caption. - Honesty: caption concedes context7 "often carries the current version in a lower-ranked entry" (the load-bearing per-library concession the snapshot keeps) and scopes the claim to the *default match*. - chart_rows keeps the None-able judge_row verdicts (faithful projection; feeds tally); render_svg validates captured_at via _require_iso_date (parity with the README path); token guard filters ground_truth + uses version_matches. Tests: test_chart.py stays 8 (−2 byte/token, +2 semantic+swap); 66 Python green.
R2 opus ensemble found the SUBTLER guard holes the R1 semantic guard missed: - HIGH: the guard read <text> content + document order only, blind to x/fill → a coordinate or colour swap made the chart visually lie (LiveDocs shows ✗, context7 shows ✓) while every <text> stayed individually true, and it passed. Now parses each cell's x + fill and asserts content EQUALITY + column order (ld x < c7 x) + fill colour + ANCHORED footer scores — so text / prose / coord / colour / footer- transpose tampering all fail loud, version-independently (x-order + fixed hex are stable where layout bytes aren't). New multi-vector test proves all 5 vectors. - ✓/✗ mark + colour are now DERIVED from the measured verdict (scorecard_cells, the single definition shared by build_figure + the guard) instead of hardcoded-by- column; fail-loud `n_rows == n` assert so a future non-contested row can't skew the drawn body vs the printed denominator. - Over-concession corrected (#27/#29): "often carries the current version in a lower-ranked entry" → "can carry (as with react here)" — true for 1/6, not "often" — in the README caption, the module docstring, AND the snapshot honesty_note. - Softened the render_svg docstrings: stripping Creator removed ONE volatile source, but text coords stay freetype-volatile, so the CI guard is semantic, not byte. Tests: test_chart.py stays 8 (single swap test → multi-vector visual-lie test); 66 Python green; committed SVG byte-identical (verdict-derived render is unchanged).
R3 (partial — DA/Codex blocked by the API session limit) confirmed the R2 visual-lie
HIGH closed + regression PASS, but 3 lens agents converged on one residual: the guard
checked the value cells but NOT the sub-captions, headers, or title — so a hand-edited
sub-caption lie ("no version in top match" → "context7 is actually current") or a header
swap survived. Convergent fix instead of whack-a-mole:
- _assert_svg_faithful now compares the COMPLETE ordered <text> content to a fresh render
of the snapshot (title + both headers + every library/ecosystem/cell/sub-caption +
footer + no-extras), version-independently (strings are stable; only coords drift), plus
the value cells' fill-colour + column order. Removed the now-unused _parse_svg_scorecard.
The visual-lie test adds sub-caption-lie + header-swap vectors — all 7 now fail loud.
- honesty_note LOW: "the other five have no current-version entry at all" over-claimed
(vite HAS an 8.0.10 entry, just behind) → "don't surface the current version at all —
their context7 entries are behind or version-less".
Tests: test_chart.py stays 8; 66 Python green; committed SVG + README byte-identical.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #50
Summary
Revises #46's aggregate ASCII block (
6/6 vs 0/6) into a per-library matplotlib SVG scorecard at the README top — each of 6 fast-moving libs shows LiveDocs' current version (✓) beside context7's actual default-match version and how far behind (react → v18 a major behind; vite → 8.0.10; the rest version-less). Convincing through transparency (real, checkable versions), not spin.Changes
chart.py—chart_rows(pure snapshot projection, reusesjudge_row) +render_svg(matplotlib, deterministic SVG)compare_context7.py—--emit-chart --write/--check;render_readme_blockembeds the image + a visible caption of honesty caveats; removed the dead ASCIIrender_bar_chartcompare_context7_sample.json— enriched each context7 entry with structureddefault_version/default_status(restating the recorded prose)docs/assets/vs-context7-freshness.svggenerated; matplotlib added to requirements⚠ Reviewer: confirm the SVG renders
The README embeds
docs/assets/vs-context7-freshness.svgvia a relative path. GitHub usually renders repo SVG but it's historically finicky — please confirm the chart displays at the top of the rendered README. Fallback if not: raw URL or PNG.Checklist
Refs #50)/idd-verify #50) + confirm SVG renders on github.com/idd-close🤖 /idd-implement · no close trailer — manual /idd-close after merge.