Skip to content

Commit 30da130

Browse files
claudejjohare
andcommitted
feat(dataset): corpus repair, transitive categories, bridging published
Resyncs the release to the repaired corpus and corrects figures this repo was publishing about itself. Dataset classes 7,457 -> 7,874 (417 pages created for concepts two or more pages already referenced) triples 252,974 -> 258,200 resolvable edges 96,377 -> 98,776 uncategorised 4,498 -> 3 domainless 4 -> 0 orphaned classes 471 -> 9 (the 9 are domain roots, correctly parented to owl:Thing) dangling refs 2,637 -> 1,568 (singletons deliberately left) validation 0 errors / 961 warnings -> 0 errors / 0 warnings / 1,402 info Three corrections behind those numbers 1. Category inheritance read direct parents only, so a class two hops below a category root fell to CATEGORY_NONE. That mislabelled 4,033 classes as uncategorised — 89.7% of the figure this repo published as a limitation. The resolver now walks ancestry breadth-first. 2. Overlap was published as a defect. 958 of the 961 "warnings" were MULTI_PARENT, and multiple inheritance here is deliberate: 1,401 classes are bridged, 454 across categories, 153 across domains. The corpus was clean and the pipeline was describing its own design as a defect list. MULTI_PARENT is now info severity, and the bridging is published rather than discarded: stats.json "bridging" counts bridges.json 542 entries, per-class category + domain membership overview.json 124 edges = 34 backbone + 90 weighted category<->category The NGG1 node record carries a single u16 category, so binary tiers keep the nearest one and full membership lives in JSON. Documented, not glossed. 3. Corpus repair by a 33-agent swarm, independently verified. 68% of added parents point at specific semantic classes rather than category roots. Dangling subClassOf held at 5 — the repair introduced no new breakage. Docs across README, both architecture docs, schema reference, CI/CD, reproduce guide, both methodology docs and ecosystem resynced to measured figures, with bridging documented as a design property. What remains is stated plainly: 3 uncategorised, 1,568 singleton dangling refs, 0 individuals, and the single-category NGG1 constraint. Co-Authored-By: jjohare <github@thedreamlab.uk>
1 parent e68419e commit 30da130

1,034 files changed

Lines changed: 84042 additions & 44969 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/build.yml

Lines changed: 14 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -30,11 +30,11 @@
3030
#
3131
# What this workflow proves, on every push and pull request:
3232
# 1. the corpus contains no credential-shaped strings (load-bearing: the
33-
# corpus is the published artefact — 7,457 markdown pages go out verbatim)
33+
# corpus is the published artefact — 7,874 markdown pages go out verbatim)
3434
# 2. the NGG1 binary writer still matches its 183-byte golden fixture
3535
# (pipeline/tests/test_emit_graph_tiers.py, pipeline/tests/fixtures/ngg1-3n2e.bin)
3636
# 3. the 7-stage pipeline runs end to end from ontology/pages/
37-
# 4. the build produced exactly 7457 OWL classes — a hard contract gate, not
37+
# 4. the build produced exactly 7874 OWL classes — a hard contract gate, not
3838
# a printed statistic
3939
# 5. pipeline.validate reports zero errors over the corpus
4040
#
@@ -61,7 +61,7 @@ env:
6161
# The corpus contract. Both dist-ci/data/graph/stats.json (`classes`) and
6262
# dist-ci/data/ontology.json (`class[]` length) must report this many OWL
6363
# classes or the build fails. Change it only alongside the corpus.
64-
EXPECTED_CLASSES: '7457'
64+
EXPECTED_CLASSES: '7874'
6565

6666
jobs:
6767
build:
@@ -81,12 +81,12 @@ jobs:
8181
# key on the internet, so this gate blocks the build rather than warning.
8282
#
8383
# Every pattern is anchored to a token-shaped suffix on purpose. A bare
84-
# `sk-` substring match hits 1097 of the 7457 corpus files (risk-, task-,
84+
# `sk-` substring match hits 1107 of the 7874 corpus files (risk-, task-,
8585
# disk-, desk-assistant …), which would make the gate useless noise; the
8686
# anchored form `sk-[A-Za-z0-9]{20,}` matches 0. Likewise the bare prefix
8787
# `[Bb]earer ` used below appears in 64 pages as HTTP prose, while
8888
# `[Bb]earer <20+ token chars>` matches 0. Verified against the corpus at
89-
# 7457 pages: 0 hits total.
89+
# 7874 pages: 0 hits total.
9090
- name: Secret scan (corpus must contain no credential-shaped strings)
9191
run: |
9292
set -uo pipefail
@@ -141,12 +141,12 @@ jobs:
141141
ls -la dist-ci/data/graph/
142142
143143
# ------------------------------------------------------------------
144-
# GATE 3 — corpus contract: the build must produce 7457 OWL classes
144+
# GATE 3 — corpus contract: the build must produce 7874 OWL classes
145145
# ------------------------------------------------------------------
146146
# Read from the artefact the pipeline actually wrote, not from stdout.
147147
# A silent parse regression (a dropped JSON-LD fence, a changed public
148148
# filter) shows up here as a number, not as a warning nobody reads.
149-
- name: Assert corpus contract (7457 classes)
149+
- name: Assert corpus contract (7874 classes)
150150
run: |
151151
set -euo pipefail
152152
python - <<'PY'
@@ -179,10 +179,13 @@ jobs:
179179
# ------------------------------------------------------------------
180180
# GATE 4 — validation must report zero errors
181181
# ------------------------------------------------------------------
182-
# pipeline.validate exits 1 on any error. Warnings do not fail the build:
183-
# the corpus currently carries 961 of them (957 MULTI_PARENT — a class
184-
# with more than one subClassOf — plus 4 INVALID_DOMAIN), and they are a
185-
# known, recorded property of the taxonomy rather than a defect.
182+
# pipeline.validate exits 1 on any error. Warnings do not fail the build,
183+
# and the corpus currently carries none: the report reads 0 errors, 0
184+
# warnings, 1401 info, all of the info entries MULTI_PARENT. That code was
185+
# a warning until it was reclassified — multiple inheritance is deliberate
186+
# here, so 957 of the old 961 warnings were the design, not a defect list
187+
# (docs/ci-cd/build-and-gates.md §3). The bridging it counts is published
188+
# in stats.json (`bridging`), bridges.json and overview.json.
186189
- name: Validate corpus (0 errors required)
187190
run: python -m pipeline.validate ontology/pages
188191

README.md

Lines changed: 123 additions & 71 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)