GroundTruth

Ask an operational question. Inspect the evidence before trusting the answer.

At a glance. Answers cite the exact document version they came from, and questions the sources cannot support are refused. Limits: no hosted deployment, and the answer-coverage gate is not yet demonstrated on the integrated main — no accuracy claim is made here; the verification record states what was measured.

GroundTruth combines scoped retrieval, immutable citations, document versions and human review in a FastAPI/Next.js application. The portable connected profile uses real SQLite persistence, pinned local CPU semantic embeddings and clearly labeled source-extractive answers. A separate visual sample tour runs without a backend. PostgreSQL/pgvector and Redis/Celery code is included; live infrastructure verification remains deferred while the owner continues without WSL.

Visual tour · Actual workflow · Original verification · Reader verification · Case study · Setup · Architecture

The latest fresh blind gate failed: 17/120 complete answers and80/80 correct negatives, versus69/120 and79/80 for the original baseline on default reranked retrieval. All103 positive failures were refusals; no execution, access, citation or span errors were observed. The complete failed comparison preserves both first reports, all four methods, frozen inputs and independent replay. This set is now exposed regression material, never another unseen test. Repair continues; the owner's review service has not been promoted.

The earlier source30 candidate now answers 48/48 development positives and32/32 negatives correctly, with zero observed execution, access, citation or span errors. Independent replay confirms the result and the predeclared margin grid selects NLI margin2. These are exposed development cases. Candidate de79a787 was frozen before a clean author and separate label reviewer prepared that fresh blind200. Development qualification did not generalize to the result above; no unseen pass is claimed. The complete development evidence includes every margin result and the selected configuration's keyword, vector, hybrid and reranked runs, each48/48 positives and32/32 negatives. Failed higher margin observations are preserved alongside the passing runs. The failed blind comparison is preserved; the current development cycle adds multi-part evidence selection and an explicitly approved larger local extractive-reader comparison. It is not a replacement unseen validation pass.

The active quality repair records the approved composite proposal profile, disclosed activity-role interpretation, mandatory support checks and fresh development/blind gates. These candidate contracts have not yet established a passing unseen result.

The first combined-profile development result improves to 45/48 complete answers and32/32 correct negatives, with zero recorded execution, authorization, citation or span errors. It still fails the 46/48 development prerequisite: three ordinary procedures were refused because of unrelated postponed-audit record handling. The complete failed report and frozen inputs remain separate from every earlier failure. No new blind set or review-instance promotion is claimed.

The source27 recheck, after saved-evidence integrity repairs, also records 45/48 positives and32/32 negatives. Its independently audited archive retains the full report, source, inputs and six actual neutral applicability decisions. This remains a failed development result; changing a certificate cannot establish answer coverage.

The earlier source29 development run preserves the same 45/48 answers and32/32 negatives after the approved context reading was enabled. Its independent audit isolates one neutral card-reference identity check in each of six notes; the other22 checks pass. A separately versioned typed-reference repair subsequently qualified source30 as recorded above. Fresh blind acceptance and review-instance promotion remain open.

The current engineering bundle retains full Python and frontend checks, isolated-package evidence, real procedure/version workflows, outage recovery, and the offline preview. Its screenshots and records are separate from answer-quality acceptance. The separate direct frontend capture records actual current TypeScript and ESLint exits with exact source bindings.

The separate combined-procedure engineering checkpoint records real connected uploads, source-copy inspection, version/history behavior, model-asset failure/recovery and the offline preview. Its portable evidence bundle includes captured responses and a pinned model-free replay. Answer-quality acceptance remains open; these engineering examples are separate from DEV/blind measurements.

The latest reviewed reader checkpoint records the repaired multi-document extraction path, exact development source, real connected browser verification and installed-wheel checks. Its exposed development scores do not replace the preserved failed blind results.

The new frozen200-case comparison also failed:69/120 answers and62/80 negatives, versus39/120 and75/80 for its baseline. The complete result and independent audit are retained. The reader candidate is not promoted to the owner's review instance.

Portfolio demo

Choose Start guided demo or explore freely. Persistent Interactive sample data labeling distinguishes the preview from actual execution. One scenario state drives Chat, Documents, Workflows and Admin, with complete reset.

cd apps/web
npm ci
npm run demo:ui

Open http://localhost:3505.

Click path:

1. Ask about staging rollback; open the highlighted current citation.
2. Compare an outdated runbook with the current version.
3. Inspect an unsupported-question refusal and a curated procedural conflict.
4. Follow a quarantined-file recovery, then inspect consistent review/audit views.
5. Open the actual workflow and engineering evidence.

The preview makes no API/provider/font request. Sample file selection does not transmit file contents. Keyboard-operable source details return focus to the citation; reduced motion and mobile/tablet layouts are supported. See the walkthrough.

Set NEXT_PUBLIC_DEMO_MODE=true in any Next.js build or dev command to enable forced demo mode without the demo:ui script.

Offline demo

The fastest proof path needs no credentials, database, Redis, model download, or sibling repository:

python -m pip install -e "apps/api[dev]"
make demo
# without make: python examples/run_demo.py

make demo runs the deterministic command-line grounded-answer and refusal demo. It does not start Docker or open the web interface.

Actual connected quickstart

python -m pip install -e "apps/api[dev,embeddings,reader,support]"
python -m app.model_setup --directory .tmp/models/minilm
python -m app.reader_setup --model roberta --directory .tmp/models/reader-roberta
python -m app.support_setup --directory .tmp/models/support
python scripts/portfolio_local.py migrate
python scripts/portfolio_local.py serve

• API: http://127.0.0.1:8400
• OpenAPI: http://127.0.0.1:8400/docs
• Production frontend: http://127.0.0.1:3505

Seed in a separate terminal with python scripts/portfolio_local.py seed, then enter the generated disposable API key through Connection. The complete setup includes isolated environments, CPU-only dependency installation, frontend build/start, private manifest handling and explicit deterministic mode. Connected failures remain errors; switching to sample mode is always explicit. Historical library defaults preserve the original offline golden fixture and are not the authenticated application profile.

The local reader is an optional, separately cached CPU extractive QA model. It copies source passages and highlights their exact offsets; it does not generate new factual prose. Missing assets produce an error. Explicit lexical mode remains available. Answer-coverage work preserves the first failed evaluation and defines the separate frozen blind acceptance gate; adding a reader does not by itself demonstrate that this gate passes. The earlier MiniLM candidate failed the original rollback browser journey at threshold -8; that result remains preserved. The earlier RoBERTa-only grid recorded47/48 answers and32/32 negatives. The current development launcher selects the complete measured RoBERTa QA4/NLI2/composite/activity/copy/context-v2 recipe, which records48/48 and 32/32 on each retrieval method. Fresh blind acceptance remains pending. Explicit --reader-model minilm retains the prior MiniLM profile and cache location. The existing review instance has not been promoted on development results alone. The first new 200-case comparison also failed: default reader 55/120 correct answers and 77/80 correct negatives, versus baseline 30/120 and 68/80. The target is 114/120 and 80/80. Both complete first reports, the concurrent-editor integrity qualification and a subsequently repaired diagnostic-access defect are recorded in reader verification. Multi-document support remains; no failed evidence was replaced.

Measured limits

The frozen corpus has 12 synthetic runbooks and 100 agent-authored cases; labels have no human/model-quality validation. After development-only tuning, the first held-out measurement passed 37/80 contracts per method: expected sources were retrieved, but conservative support checks refused42 answerable cases and missed one refusal. Zero unauthorized-source exposures or execution errors were observed. The first result is preserved; subsequent repairs are exposed regressions, not fresh held-out validation. Literal coverage is not entailment. See the case study before making quality claims.

Live PostgreSQL/Redis, large indexed performance, hosted CI, provider quality and hosted multi-user authentication remain unverified. No production-capacity or universal contradiction-detection claim is made.

What it does

1. Ingests TXT, PDF, Markdown, HTML, DOCX, CSV, and TSV by default. XLSX/PPTX, OCR, and model-backed embeddings/reranking are explicit extras.
2. Normalizes and deduplicates documents and chunks before embedding, retaining failed source files with quarantine metadata for reindexing.
3. Retrieves with vector and keyword search, reciprocal-rank fusion, and deterministic reranking fallbacks.
4. Answers or refuses using confidence and safety gates, with citations and a retrieval trace in normal and SSE flows.
5. Tracks operations with request/workspace context, structured audit events, cost attribution, API-key-aware rate limits, document version history, workflow status events, and a notification outbox.
6. Supports local evaluation through a dependency-free fixture adapter and a deterministic portfolio evidence bundle.

Architecture

flowchart LR
    WEB[Next.js web] -->|REST and SSE| API[FastAPI API]
    API --> ING[Ingestion]
    API --> RAG[Retrieve, rerank, refuse, generate, cite]
    API --> WF[Versions, workflows, audit, notifications]
    ING --> DB[(PostgreSQL and pgvector)]
    RAG --> DB
    WF --> DB
    WF -. optional .-> REDIS[(Redis and Celery)]
    RAG -. optional .-> PROVIDER[LLM and embedding providers]

Loading

The API owns a narrow compatibility subset vendored at apps/api/app/internal/vendor_core/. It is pinned to FishRaposo/operator-shared-core v1.3.0 at commit dbf276a7708da65b55e1f10b35af634b300d1f07, internally namespaced, MIT-licensed, and shipped in the wheel. No external shared_core install, Git URL, or sibling checkout is required. See migration provenance.

Verification

# Canonical installs
python -m pip install -e "apps/api[dev]"
cd apps/web && npm ci && cd ../..

# Complete local gate; Playwright Chromium must already be installed
make check

# Install the browser once when needed
cd apps/web && npx playwright install chromium

The gate covers API pytest, Ruff lint and format checks, Pyright, the forbidden external-dependency scan, wheel build/content/isolated-import verification, deterministic portfolio evidence, frontend Vitest, ESLint, production build, and Chromium Playwright smoke tests across three viewports plus axe WCAG scans. The CI workflow_dispatch integration input installs PostgreSQL and Redis extras before rerunning the SQLite/offline integration-shaped contracts; it does not claim live-service coverage.

The default gate (make test / CI) runs the apps/api/tests suite — 396 tests pass with 1 skip at the 0.2.0 audit — plus 18 repository-level verifier contracts under scripts/tests. The final local pass also recorded 72 passing frontend Vitest tests across 20 files, Ruff check/format, repository-wide Pyright with zero errors, a production Next.js build, and 21/21 Chromium browser checks (sample journey plus axe WCAG scans across desktop, mobile, and tablet viewports). The connected live and reader-diagnostic browser suites run against a real disposable FastAPI + production Next.js stack; they are exercised in the dedicated connected-live CI job rather than the default browser gate. The evidence bundle reproduced byte-for-byte twice with hash daaa900b228aa7820ead848bdbf51ae3a6b723b514c24588f25f1f554741e334. Live PostgreSQL/Redis integration and Docker runtime execution were not available on this host and are not implied pass claims.

Security and deployment boundaries

• API authentication and rate limiting are configurable; rate buckets are isolated by workspace and hashed API-key identity.
• Uploaded filenames are not trusted for storage paths.
• SMTP, webhooks, OCR, Office parsing, local model inference, PostgreSQL, Redis, and Docker are opt-in integrations.
• SAML/SSO, hosted/team workflows, hosted notification services, mandatory infrastructure, cloud object storage, and hosted scheduling are deliberately deferred. See Security and Roadmap.

Part of the stack

evalforge · llm-gateway · aria-agent · agenttrace · WCP-Compliance-Agent-V5

License and provenance

GroundTruth is MIT licensed. Third-party provenance and the vendored upstream license are recorded in THIRD_PARTY_NOTICES.md.

The post-seal scope repair fixes a specific source-title instruction parsing defect and verifies five exposed HTTP regressions, full engineering gates and real production browser workflows. It does not change the failed69/120+62/80 result or supply new unseen acceptance. The owner's review instance remains unchanged.