diff --git a/CHANGELOG.md b/CHANGELOG.md index fe429a8..0f40105 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,7 +24,7 @@ Documentation tooling now counts prose separately from fenced code and diagrams, The abbreviated license file has been replaced with the official Apache License 2.0 text. Project attribution is recorded separately in NOTICE, and citation metadata identifies the author as an independent Romanian researcher. The existing release tag is preserved. The citation's release version is not silently advanced to an unpublished development version merely because the Python package declares that version. -Two analysis-integrity issues have targeted corrections after the documentation baseline. Pathway enrichment now applies monotonic Benjamini–Hochberg adjustment over the full valid pathway family, including valid pathways with no observed overlap in the correction denominator. Multi-omics grouping now rejects duplicate sample-layer pairs and inconsistent participant mappings instead of silently overwriting earlier values. The frontend now exposes key provenance boundaries and has build-plus-browser smoke coverage, but it still needs broader accessibility, visual regression, and production deployment evidence. Publication origin classification is improved, but provider origin remains a retrieval-boundary label rather than scientific validation. No clinical validation, independent benchmark, or operational scientific review service is asserted by this documentation entry. +Analysis-integrity work now includes reproducible time handling for navigation scores, plus the two targeted corrections recorded after the documentation baseline. Pathway enrichment now applies monotonic Benjamini–Hochberg adjustment over the full valid pathway family, including valid pathways with no observed overlap in the correction denominator. Multi-omics grouping now rejects duplicate sample-layer pairs and inconsistent participant mappings instead of silently overwriting earlier values. The frontend now exposes key provenance boundaries and has build-plus-browser smoke coverage, but it still needs broader accessibility, visual regression, and production deployment evidence. Publication origin classification is improved, but provider origin remains a retrieval-boundary label rather than scientific validation. No clinical validation, independent benchmark, or operational scientific review service is asserted by this documentation entry. ## [0.2.0] - 2026-09-14 diff --git a/README.md b/README.md index 222a742..a477ce7 100644 --- a/README.md +++ b/README.md @@ -90,7 +90,7 @@ Only then consider a structured evidence observation. Define the study design, s The A–G taxonomy assigns systematic reviews to A, randomized trials to B, clinical studies to C, observational studies to D, animal studies to E, in-vitro studies to F, and computational work to G. This is the project's navigation convention. It is not a claim that every review is stronger than every experiment or a substitute for evaluating bias and relevance. The implementation also maps retracted records to G, so clients must retain original study design alongside publication status. -The numerical navigation score uses hand-selected design weights, confidence, replication metadata, sample size, and publication age. It is not a probability of truth, a treatment effect, or a validated evidence-certainty scale. Date-sensitive outputs can change with execution time. The [whitepaper](WHITEPAPER.md) now describes the actual arithmetic and its limitations. A research report should show the inputs and discuss whether conclusions depend on the ranking choices. +The numerical navigation score uses hand-selected design weights, confidence, replication metadata, sample size, and publication age. It is not a probability of truth, a treatment effect, or a validated evidence-certainty scale. Date-sensitive outputs can now be frozen by passing an explicit scoring time to the evidence engine; API summaries also expose `scoring_as_of` so exported interpretations can record the temporal basis. The [whitepaper](WHITEPAPER.md) describes the actual arithmetic and its limitations. A research report should show the inputs and discuss whether conclusions depend on the ranking choices. Experimental analysis utilities need similarly narrow interpretation. The biological-age routine is an ordinary-least-squares baseline. The survival utility produces Kaplan–Meier points without a complete inference framework. Multi-omics integration now rejects duplicate sample-layer pairs and inconsistent participant mappings, but it still does not perform feature harmonization, statistical batch correction, or biological interpretation. The pathway routine now applies a monotonic Benjamini–Hochberg correction across the valid pathway hypothesis family, while results still depend on the declared universe, pathway annotations, and upstream gene selection. These are reasons to inspect methods before drawing scientific conclusions, not details to hide behind a general research-use label. diff --git a/ROADMAP.md b/ROADMAP.md index 8dd01cf..c7e18da 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -38,7 +38,7 @@ Publication-origin classification has moved from a fixed false synthetic flag to Complete the documentation expansion with distinct, source-backed explanations rather than repeated filler. Every tracked Markdown file has a minimum prose target, but accuracy and implementation alignment remain separate requirements. The automated inventory should continue to expose short documents, missing attribution, and structural problems. Code examples need execution checks, and diagrams need to identify proposed components clearly. A passing word-count gate is not a substitute for editorial review. -Continue resolving analysis limitations through focused changes and reference cases. Pathway enrichment now defines the tested family as pathways with at least one member in the supplied universe and applies monotonic Benjamini–Hochberg adjustment while returning only observed overlaps. Multi-omics integration now rejects duplicate sample-layer pairs and inconsistent participant mappings instead of silently overwriting values. Remaining analysis priorities include a controllable or clearly recorded time basis for date-sensitive scoring, explicit manifests for preprocessing metadata, broader edge-case fixtures, and downstream methods that keep their scientific assumptions visible. Each change should state its effect on existing outputs and should not be promoted as clinical validation merely because a regression test passes. +Continue resolving analysis limitations through focused changes and reference cases. Pathway enrichment now defines the tested family as pathways with at least one member in the supplied universe and applies monotonic Benjamini–Hochberg adjustment while returning only observed overlaps. Multi-omics integration now rejects duplicate sample-layer pairs and inconsistent participant mappings instead of silently overwriting values. Evidence scoring now accepts an explicit `as_of` time and exposes the summary timestamp, reducing one reproducibility risk in date-sensitive navigation scores. Remaining analysis priorities include explicit manifests for preprocessing metadata, broader edge-case fixtures, and downstream methods that keep their scientific assumptions visible. Each change should state its effect on existing outputs and should not be promoted as clinical validation merely because a regression test passes. ## Publication-path acceptance diff --git a/docs/API.md b/docs/API.md index 0ceeb91..5feb1c5 100644 --- a/docs/API.md +++ b/docs/API.md @@ -85,7 +85,7 @@ The citation export route should not be described as a complete publication expo `GET /api/v1/evidence/{record_id}/review-events` lists stored review events for a fixture evidence record when the review repository is configured. The route returns audit events, not a full reviewer user interface. It is the persistence boundary for the human-review workflow: reviewer actions can be stored, inspected, and connected to citation-export eligibility, while user management and role delegation remain future work. -Evidence grades and scores require their methodological labels. The A–G mapping is a project taxonomy, and the numerical navigation score uses heuristic constants. Neither is a calibrated scientific certainty estimate. The score also depends on execution time when a publication date is present. The API's ability to serialize a number does not justify describing it as a treatment effect, probability of truth, or measure of human longevity benefit. +Evidence grades and scores require their methodological labels. The A–G mapping is a project taxonomy, and the numerical navigation score uses heuristic constants. Neither is a calibrated scientific certainty estimate. When a publication date is present, the score depends on the scoring time; evidence summaries now expose `scoring_as_of` so clients can record that temporal basis. The API's ability to serialize a number does not justify describing it as a treatment effect, probability of truth, or measure of human longevity benefit. Publication origin classification is now explicit for persisted publications, including synthetic seed rows and records saved through the PubMed ingestion path. Clients and operators must still avoid treating `synthetic: false` as proof of scientific reliability. It means the record was not classified as synthetic by the storage contract and entered through a provider-boundary path; it does not mean the publication is complete, unretracted, clinically relevant, or human reviewed. diff --git a/docs/audits/v0.3.0-baseline.md b/docs/audits/v0.3.0-baseline.md index 5d27fd4..76337c8 100644 --- a/docs/audits/v0.3.0-baseline.md +++ b/docs/audits/v0.3.0-baseline.md @@ -93,7 +93,7 @@ Two analysis utilities now have narrower and more defensible contracts than they The multi-omics grouping helper now rejects duplicate sample-layer pairs and rejects a sample identifier that maps to more than one participant identifier. This removes the previous silent-overwrite behavior. The utility still returns a compact layer map for each sample and preserves explicit missing feature values as `None`. It does not perform batch correction, feature harmonization, unit conversion, privacy review, or biological integration. Those remain downstream scientific tasks. -Verification for this follow-up added targeted regression cases for the pathway hypothesis family, monotonic adjusted p-values, genes outside the universe, missing omics values, duplicate omics layers, and inconsistent participant mappings. The local focused check reported `ruff check` passing and the analysis test module passing with seven tests. This is implementation evidence for the stated software contracts, not evidence that any pathway result or multi-omics association is biologically valid. +Verification for this follow-up added targeted regression cases for the pathway hypothesis family, monotonic adjusted p-values, genes outside the universe, missing omics values, duplicate omics layers, and inconsistent participant mappings. The local focused check reported `ruff check` passing and the analysis test module passing with seven tests. This is implementation evidence for the stated software contracts, not evidence that any pathway result or multi-omics association is biologically valid. A subsequent scoring reproducibility check adds an explicit `as_of` parameter for navigation-score age calculations and records `scoring_as_of` in evidence summaries, reducing drift caused by executing the same report on different dates. This audit records project responsibility under Ciprian Ștefan Pleșca, an independent Romanian researcher. It does not claim an external audit institution, university affiliation, scientific peer review, or security certification. Its contribution is a traceable account of what was inspected, what changed, what was actually checked, and what remains unresolved. Future updates should preserve that separation so that readers can evaluate progress without having to infer completion from presentation quality. diff --git a/src/openlongevity/evidence.py b/src/openlongevity/evidence.py index 55b9ad5..0ff4fc2 100644 --- a/src/openlongevity/evidence.py +++ b/src/openlongevity/evidence.py @@ -45,6 +45,17 @@ class EvidenceLevel(StrEnum): } +def _utc_datetime(value: datetime) -> datetime: + if value.tzinfo is None: + return value.replace(tzinfo=UTC) + return value.astimezone(UTC) + + +def _parse_publication_datetime(value: str) -> datetime: + parsed = datetime.fromisoformat(value.replace("Z", "+00:00")) + return _utc_datetime(parsed) + + class EvidenceEngine: """Grade and summarize records without implying clinical effectiveness.""" @@ -53,8 +64,13 @@ def grade(self, record: EvidenceRecord) -> EvidenceLevel: return EvidenceLevel.G return _LEVEL_BY_TYPE[record.study_type] - def score(self, record: EvidenceRecord) -> float: - """Return a transparent navigation score, not a validated effect estimate.""" + def score(self, record: EvidenceRecord, *, as_of: datetime | None = None) -> float: + """Return a transparent navigation score, not a validated effect estimate. + + ``as_of`` freezes the publication-age component for reproducible reports. When + omitted, the current UTC time preserves the historical runtime behavior. + """ + scoring_time = _utc_datetime(as_of) if as_of else datetime.now(UTC) score = _BASE_SCORE[record.study_type] * record.confidence if record.replication_status.casefold() in {"replicated", "independent"}: score *= 1.15 @@ -68,10 +84,7 @@ def score(self, record: EvidenceRecord) -> float: try: age_years = max( 0.0, - ( - datetime.now(UTC) - - datetime.fromisoformat(record.publication_date).replace(tzinfo=UTC) - ).days + (scoring_time - _parse_publication_datetime(record.publication_date)).days / 365.25, ) score *= max(0.75, 1.0 - age_years * 0.01) @@ -79,9 +92,12 @@ def score(self, record: EvidenceRecord) -> float: pass return round(min(1.0, max(0.0, score)), 4) - def summarize(self, records: Iterable[EvidenceRecord]) -> dict[str, object]: + def summarize( + self, records: Iterable[EvidenceRecord], *, as_of: datetime | None = None + ) -> dict[str, object]: records = tuple(records) active = tuple(r for r in records if r.retraction_status is not RetractionStatus.RETRACTED) + scoring_time = _utc_datetime(as_of) if as_of else datetime.now(UTC) grades = Counter(self.grade(r).value for r in active) mean = sum(r.confidence for r in active) / len(active) if active else 0.0 return { @@ -89,9 +105,12 @@ def summarize(self, records: Iterable[EvidenceRecord]) -> dict[str, object]: "active_records": len(active), "evidence_distribution": dict(sorted(grades.items())), "mean_confidence": round(mean, 4), - "mean_navigation_score": round(sum(self.score(r) for r in active) / len(active), 4) + "mean_navigation_score": round( + sum(self.score(r, as_of=scoring_time) for r in active) / len(active), 4 + ) if active else 0.0, + "scoring_as_of": scoring_time.isoformat(), "disclaimer": "Research use only. Not medical advice.", } diff --git a/tests/test_core.py b/tests/test_core.py index 96687ed..69ce346 100644 --- a/tests/test_core.py +++ b/tests/test_core.py @@ -1,3 +1,5 @@ +from datetime import UTC, datetime + from openlongevity.evidence import EvidenceEngine, EvidenceLevel from openlongevity.gaps import ResearchGapDetector from openlongevity.graph import EvidenceGraph @@ -28,6 +30,26 @@ def test_grading_and_retraction() -> None: ) +def test_navigation_score_accepts_explicit_scoring_time() -> None: + engine = EvidenceEngine() + dated = record( + "dated", + StudyType.RCT, + publication_date="2020-01-01", + replication_status="replicated", + sample_size=200, + ) + early = datetime(2021, 1, 1, tzinfo=UTC) + later = datetime(2031, 1, 1, tzinfo=UTC) + + assert engine.score(dated, as_of=early) == engine.score(dated, as_of=early) + assert engine.score(dated, as_of=early) > engine.score(dated, as_of=later) + + summary = engine.summarize([dated], as_of=early) + assert summary["scoring_as_of"] == "2021-01-01T00:00:00+00:00" + assert summary["mean_navigation_score"] == engine.score(dated, as_of=early) + + def test_gap_detector_flags_translation_gap() -> None: gaps = ResearchGapDetector().detect("senescence", [record("a", StudyType.ANIMAL)]) assert gaps[0].kind == "translational_gap"