Architecture

Architecture

Author: Ciprian Ștefan Pleșca

This page describes the structural decomposition of OpenLongevity: the modules that exist, the responsibility each one owns exclusively, and the boundaries the codebase enforces between them. The guiding architectural principle, stated in the project's own README, is that different parts of the system "should be assessed individually" — a source adapter retrieving metadata is not the same claim as a database preserving revisions, which is not the same claim as a graph illustrating relationships.

Layered decomposition

flowchart TD
    subgraph L1["Layer 1 — Ingress adapters (src/openlongevity/providers)"]
        base["base.py: Provider contracts, Publication, Provenance"]
        pubmed["pubmed.py: PubMedProvider"]
        epmc["europe_pmc.py"]
        openalex["openalex.py"]
        crossref["crossref.py"]
        ctgov["clinicaltrials.py"]
        httpmod["http.py: shared transport helpers"]
    end

    subgraph L2["Layer 2 — Domain core (src/openlongevity)"]
        models["models.py: EvidenceRecord, ResearchGap, Contradiction"]
        evidence["evidence.py: EvidenceEngine"]
        gaps["gaps.py: ResearchGapDetector"]
        evgraph["graph.py: EvidenceGraph"]
        constants["constants.py: DISCLAIMER"]
    end

    subgraph L3["Layer 3 — Persistence"]
        db["db.py: Database, ORM rows"]
        repo["repository.py: PublicationRepository"]
    end

    subgraph L4["Layer 4 — Interfaces"]
        api["api.py: FastAPI app"]
        cli["cli.py: argparse entry point"]
    end

    subgraph L5["Adjacent surfaces"]
        biomarkers["biomarkers/: catalog"]
        literature["literature/: adapter Protocol"]
        analysis["analysis/: statistical utilities"]
        rustcore["packages/rust/evidence-core"]
        webapp["apps/web: TypeScript shell"]
    end

    L1 -->|normalized Publication| L3
    L3 --> L4
    L2 -->|synthetic fixtures| L4
    L4 --> webapp
    analysis -.independent, no cross-imports.-> L4
    biomarkers -.reference catalog.-> L4

Loading

Module responsibilities

• providers/base.py defines the shared contract: SearchQuery (validated query, page, limit), Provenance (source provider, source identifier, checksum, normalization/parser versions), Publication (the normalized record shape), ClinicalTrial, and the LiteratureProvider Protocol. Every concrete provider must satisfy this protocol, which keeps ingestion behind a single narrow interface regardless of upstream API differences.
• providers/pubmed.py implements PubMedProvider against NCBI E-utilities, with bounded response size (5 MB), rate limiting (asyncio.Lock plus a fixed 0.35 s delay to stay under 3 requests/second per instance), retry with exponential backoff on 429/5xx, and defused XML parsing to avoid unsafe entity expansion.
• models.py is the single source of truth for domain enumerations (StudyType, RetractionStatus, FindingDirection, ReviewStatus) and the frozen dataclasses EvidenceRecord, ResearchGap, and Contradiction. Validation lives in __post_init__, so an invalid record cannot be constructed.
• evidence.py owns evidence grading and the navigation-score arithmetic (see Evidence-Grading-Engine). It has no knowledge of persistence or transport.
• gaps.py consumes EvidenceRecord sequences and the EvidenceEngine to detect five gap categories through explicit heuristics (see Research-Gap-Detection).
• graph.py is an intentionally minimal in-memory adjacency structure (EvidenceGraph) used only for illustrative relationship demonstrations — it does not encode causal biological claims.
• db.py declares the SQLAlchemy ORM models (PublicationRow, PublicationRevisionRow) and an async Database wrapper. Schema evolution is exclusively through Alembic migrations, never through ad hoc DDL in application code.
• repository.py implements PublicationRepository, the only component permitted to write to publications and publication_revisions. It computes a content hash excluding the volatile retrieved_at field, so identical re-ingestion does not fabricate a false revision.
• api.py is the composition root: it wires providers, the repository, the evidence engine, and the gap detector into a FastAPI application, and is the only place where fixture-backed and persistence-backed routes coexist — explicitly labeled by mode.
• cli.py is a thin, synchronous demonstration entry point; it does not touch the API, the database, or the provider layer, and always operates on a synthetic in-memory record.

Request lifecycle: publication ingestion

sequenceDiagram
    autonumber
    participant Op as Operator
    participant API as FastAPI (/api/v1/ingestion/pubmed)
    participant Prov as PubMedProvider
    participant NCBI as NCBI E-utilities
    participant Repo as PublicationRepository
    participant DB as PostgreSQL

    Op->>API: POST body {query, limit} + X-Ingestion-Key
    API->>API: secrets.compare_digest(key, configured key)
    alt key missing or invalid
        API-->>Op: 401 UNAUTHORIZED
    else key valid
        API->>Prov: search(SearchQuery)
        Prov->>NCBI: esearch.fcgi (rate-limited, retried)
        NCBI-->>Prov: id list (validated numeric)
        Prov->>NCBI: efetch.fcgi (bounded to 5 MB)
        NCBI-->>Prov: PubmedArticle XML (defused parse)
        Prov-->>API: list[Publication] with checksum + provenance
        loop each Publication
            API->>Repo: save(publication)
            Repo->>DB: INSERT ... ON CONFLICT DO NOTHING
            Repo->>DB: SELECT ... FOR UPDATE
            alt content changed
                Repo->>DB: revision += 1, INSERT revision row
            end
            Repo-->>API: serialized record (revision, synthetic=False)
        end
        API-->>Op: PublicationPage (mode = "persisted")
    end

Loading

Boundary rules enforced by design

1. Fixtures never reach the database. The EvidenceRecord fixtures instantiated in api.py are process-local Python objects; nothing in evidence.py or gaps.py imports repository.py.
2. The repository is the only writer. No other module constructs PublicationRow or PublicationRevisionRow directly.
3. Provenance is mandatory for persistence. PublicationRepository.save raises ValueError if a Publication lacks a Provenance with a checksum — a record with no traceable origin cannot be stored.
4. The ingestion key gates writes, not reads. Search and retrieval routes require no key; only the PubMed ingestion route, which triggers an outbound network call and a database write, is protected.
5. Analysis utilities are dependency-light and side-effect-free. analysis/biological_age.py, analysis/survival.py, and analysis/pathway.py operate purely on lists and sets passed by the caller — they do not import the API, the database, or the providers.

Next

Continue to Domain-Model for the typed core, or Evidence-Grading-Engine for the scoring arithmetic.