-
Notifications
You must be signed in to change notification settings - Fork 1
feat: enforce source semantic coverage evidence #702
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
seonghobae
wants to merge
60
commits into
main
Choose a base branch
from
feat/source-semantic-coverage-audit
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
60 commits
Select commit
Hold shift + click to select a range
766568b
feat: enforce source semantic coverage evidence
seonghobae e04249e
fix(ontology): make SHACL validation portable
8a23991
fix(test): mirror production migration execution
c57d4cd
Merge branch 'feat/source-semantic-coverage-audit' of https://github.…
e34e3a1
docs: track canonical orchestrator alias PR
seonghobae 3aeb8cf
docs: refresh exact open PR evidence
7db5bdf
fix(semantic): require probability sample evidence
7d4d7dc
Merge branch 'feat/source-semantic-coverage-audit' of https://github.…
b439c8e
fix(semantic): bind selected sample membership
e4c6140
fix(import): remove evidence length heuristics
c488ae8
docs(gaps): refresh exact protected queue
e8fd06c
docs: record disjoint time-stratified sample
seonghobae 02e453f
fix(ontology): align Unicode body availability
7d4c846
fix(audit): use internal orchestrator credential
seonghobae bee2cbb
fix(audit): validate probability stratum cardinality
5133184
docs(adr): remove semantic coverage identifier collisions
896430e
fix(import): preserve authoritative bodies on metadata refresh
2161343
fix(audit): bind inclusion probabilities to strata
039ad1a
fix(audit): reject unverifiable sample artifacts
0013b87
feat(ontology): add provenance-bound content semantics
seonghobae ebb1dfa
fix(import): require source classifications and trigger revisions
seonghobae 3c551c2
fix(audit): include PROV semantic alignments
seonghobae 77c45f7
fix(orchestrator): forward provider host allowlist
seonghobae 78a1441
docs(math): freeze local scoring owner boundary (#712)
seonghobae af4642f
fix(audit): distinguish schema gaps from instances
seonghobae 39ab844
fix(audit): verify Rust sampling design artifacts
4720085
Merge commit 'refs/pull/702/head' of https://github.com/ContextualWis…
73428f9
fix(audit): separate content classification from coverage
seonghobae 019ca26
fix(docs): remove ADR trailing whitespace
e50fd5a
docs(adr): remove trailing whitespace
seonghobae 0768e64
Merge commit 'refs/pull/702/head' of https://github.com/ContextualWis…
e75d564
fix(test): preserve audit import ordering
670d47d
Merge commit 'refs/pull/702/head' of https://github.com/ContextualWis…
66f064c
docs(audit): record governed-stratum sample
seonghobae f3add0f
test(orchestrator): verify provider host CLI pin
seonghobae b0c21d2
feat(audit): verify semantic document key coverage
seonghobae ae586b0
fix(audit): harden dynamic identifier boundary
seonghobae ff22ca2
feat(audit): distinguish direct evidence from PROV
seonghobae 3d9e359
fix(audit): include PROV derivation table
seonghobae bf229e1
feat(audit): bind exact Rust inclusion ratios
5de66ab
Merge commit 'refs/codex/pr702-latest' into HEAD
7de6af0
docs(audit): record unavailable artifact references
seonghobae 039b742
docs(audit): verify complete source artifact replay
seonghobae 5cff76f
docs(audit): distinguish PROV deployment from linkage
seonghobae eda7083
fix(audit): require governed PROV support profile
2803651
feat(audit): bind terminal semantic coverage provenance
seonghobae 9939eed
Merge remote-tracking branch 'origin/feat/source-semantic-coverage-au…
seonghobae 05bdd5b
fix(audit): require structured multi-agent coverage
seonghobae 8ffe11f
fix(audit): retain rejected attempt provenance
seonghobae 4201bf6
fix(provenance): close reviewed trust boundaries
seonghobae 9d17100
fix(audit): drain provider results before joining
9229422
fix(audit): correlate provider trace with attempt provenance
seonghobae 8b51c73
docs(gap): refresh stacked estimator delivery state
seonghobae a4b52b4
fix(audit): bind spawned telemetry to attempt provenance
seonghobae 9f247a6
fix(audit): bind terminal result to attempt provenance
seonghobae 023f32d
docs: distinguish rare-category sampling gap
4325941
fix(import): avoid reprocessing unavailable source bodies
seonghobae f058c6b
feat(ontology): carry derived export-source place and region
seonghobae ebee952
Merge remote-tracking branch 'origin/feat/source-semantic-coverage-au…
seonghobae 93e7b81
codex: address PR review feedback (#702)
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| # Export-source ontology coverage (ADR 0246) | ||
|
|
||
| - Analyzed an aggregate-only, authorized export source against the published | ||
| ontology; all content of its rows is expressible through governed | ||
| `postTypeScheme` types, content-semantic classes, PROV attribution, and | ||
| raw-code-as-instance lifecycle fields (ADR 0246). | ||
| - Closed the one derived-semantic gap: `:Location` now carries the place its | ||
| content names (`:locationName`) and an ISO 3166-1 country/region code | ||
| (`:countryCode`) as instance data, with a SHACL `:LocationShape` failing | ||
| closed on non-code country values. | ||
| - No new lookup category is seeded; raw ERP codes remain instance data until | ||
| a caller governs their code system (ADR 0145 / 0241 / 0246). | ||
| - Aggregate-only reporting and the derived-node (not column-projection) | ||
| discipline preserve ADR 0001 / 0207 boundaries; supporting evidence in | ||
| `docs/doctoring/export-source-ontology-coverage.md`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # Local scoring ownership contract | ||
|
|
||
| - Identified active Python lineage-channel, reconstruction-decision, and | ||
| corporate-entity similarity paths as migration debt. | ||
| - Defined the fail-closed owner envelopes required before those paths can be | ||
| removed, without assigning corporate identity or moving existing heuristics | ||
| to another repository by assumption. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # ADR 0240: Explicit missing-body import boundary | ||
|
|
||
| **Status:** Accepted | ||
| **Date:** 2026-08-26 | ||
| **Extends:** [ADR 0102](0102-semantic-source-unit-boundaries.md) | ||
|
|
||
| ## Context | ||
|
|
||
| An authorized source export can contain titles, lifecycle fields, customer and | ||
| project codes, lineage keys, actors, timestamps, and source-artifact provenance | ||
| while exposing no record body. Requiring a non-empty body makes every such row | ||
| unimportable. Copying the title into the body would instead manufacture body | ||
| evidence and falsely imply semantic-unit coverage. | ||
|
|
||
| ## Decision | ||
|
|
||
| 1. The PostgreSQL importer accepts exactly one of a mapped body column or | ||
| `--no-body-dimension-evidence` containing a non-blank operator statement. | ||
| The importer records that attestation but does not use an arbitrary text- | ||
| length threshold as a proxy for evidence quality. | ||
| 2. A missing body persists as the empty source representation. The title stays | ||
| `post_title`; it is never copied into `post_body` or emitted as a paragraph. | ||
| 3. Content-unit, embedding, summary, VISION, and body-search coverage remain | ||
| unavailable until an authoritative body/file source is connected. | ||
| 4. Structured source fields retain their existing raw provenance columns and | ||
| semantic-hint boundaries. Their presence does not prove an entity binding. | ||
| 5. The import result repeats the evidence statement so an operator can retain | ||
| it with private runtime evidence. Repository artifacts contain aggregates | ||
| only. | ||
| 6. `scripts/audit_source_semantic_coverage.py` reproduces availability counts | ||
| from caller-mapped columns and emits no source values. | ||
| 7. RDF `bodyAvailable` and its published SHACL constraint use the same | ||
| whitespace predicate as the Python projector, including Unicode separator, | ||
| next-line, and legacy information-separator characters. A body containing | ||
| only those characters is unavailable; validators must not reinterpret it as | ||
| semantic evidence. | ||
| 8. A no-body-dimension re-import preserves an already-populated target body | ||
| atomically in the source-post UPSERT. The preserved body is also the input | ||
| to revision and semantic-content persistence; an unavailable source | ||
| dimension must not erase evidence acquired from an authoritative body | ||
| source. | ||
|
|
||
| ## Consequences | ||
|
|
||
| Title-only structured records can participate in explicitly supported | ||
| lineage and source-metadata views without fabricated prose. Semantic body | ||
| coverage remains honestly incomplete and can be retried after the owning | ||
| source supplies bodies. | ||
|
|
||
| ## Evidence | ||
|
|
||
| A 2026-08-26 aggregate-only source inspection found 43,814 rows, 43,814 | ||
| non-empty titles, zero non-empty bodies, 40,001 customer-code rows, 4,490 | ||
| project-code rows, and complete process-unit, sales-pool, actor, and | ||
| source-artifact provenance fields. No source value, identifier, organization, | ||
| or artifact path was copied into this repository. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # ADR 0241: Source classification semantic hints | ||
|
|
||
| **Status:** Accepted | ||
| **Date:** 2026-08-26 | ||
| **Extends:** [ADR 0004](0004-knowledge-graph-ontology.md), | ||
| [ADR 0117](0117-catalog-backed-semantic-hints.md), | ||
| [ADR 0159](0159-published-ontology-pages.md), | ||
| [ADR 0207](0207-repository-case-ontology-namespace-canonical.md), and | ||
| [ADR 0222](0222-project-nodes-in-ontology-neighborhood.md) | ||
|
|
||
| ## Context | ||
|
|
||
| The importer preserves a governed VOC type plus caller-mapped source stage and | ||
| detail-state codes, but semantic extraction received neither classification. | ||
| An authorized source reference catalog currently provides examples, not a | ||
| complete code list or authoritative definitions for every observed value. | ||
| Dropping the fields loses source evidence; minting ontology concepts from | ||
| partial examples invents semantics. | ||
|
|
||
| ## Decision | ||
|
|
||
| 1. Pass `voc_type_code`, `source_stage_code`, and | ||
| `source_detail_state_code` to contextual-orchestrator as labeled raw source | ||
| hints with exact `source_post` column provenance. | ||
| 2. Raw codes are context only. They do not assert a lifecycle transition, | ||
| inspection outcome, quality grade, entity relationship, or classified | ||
| ontology concept. An RDF source-code literal asserts only the observed raw | ||
| value and its predicate, not the value's business meaning. | ||
| 3. RDF projects the governed five-value VOC type as `:hasPostType` to its | ||
| published SKOS concept. Stage and detail-state remain literal properties; | ||
| projecting a raw literal preserves evidence without minting a concept. | ||
| 4. Promoting stage/detail values to ontology concepts requires a complete | ||
| source-owned code catalog, stable definitions, mapping provenance, and | ||
| SHACL fixtures. Partial screen examples are insufficient. | ||
| 5. Missing codes remain `none`; no default classification is inferred. Every | ||
| RDF post projector therefore requires an explicit governed VOC type. | ||
|
coderabbitai[bot] marked this conversation as resolved.
|
||
|
|
||
| ## Consequences | ||
|
|
||
| Semantic extraction can consider classifications already preserved by the | ||
| import boundary without silently losing them or overclaiming their meaning. | ||
| The ontology remains intentionally incomplete for source-specific grades and | ||
| inspection states until their authority is available. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.