Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -482,8 +482,9 @@ unavailable, so that run is Failed rather than a fabricated score.
The home list is clickable: `GET /api/analysis-runs/{id}` fills a
labeled detail (cutoff, requested date, 12-character digest prefixes
with full digests on hover, counts, status history)
without exposing a DSN or raw record. Opening a cutoff title warns
that the live body may have changed after the run. Status history is detail-only
without exposing a DSN or raw record. Opening a cutoff title still
shows the live body; titles rewritten after the run are marked
updated after cutoff. Status history is detail-only
and uses lookup labels plus occurrence times; a failure event keeps
its machine `failure_code` rather than an invented caption. Failed
TEPP list rows add a next-action line (open the run, then connect the
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.d/0.87.3-analysis-run-live-write-clock.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# 0.87.3 Analysis-run live write clock

In-cutoff titles now say whether the live row was rewritten after the
run. Open Demo public post as the edited counter-example; Demo private
post still matches the January cutoff. Bodies stay live.
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,21 @@ All notable changes to this project are documented here. Format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versioning follows
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.87.3] - 2026-08-16

### Added

- Analysis-run detail now compares each in-cutoff title's live
`updated_at` with that run's knowledge cutoff. After `make seed`,
open the Demo Corp lineage run: Demo public post is marked
**Updated after cutoff**; Demo private post is not. Opening a
marked title still shows the live body -- cutoff body versioning
stays a later slice (ADR 0016). The list stays aggregates-only.
No TEPP theta is invented. The mark uses the `PostBadge` token
module (`cd frontend && pnpm run storybook`). A title or body
rewrite stamps `updated_at` unless the statement sets that clock
(migration 0022).

## [0.87.0] - 2026-08-16

### Added
Expand Down
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ mention TEPP. A failed period-report row rebuilds the report. A
pending TEPP row does not claim a calibrated measurement. A pending
lineage row says reconstruction has not started yet.
Digest prefixes stay audible; hover a prefix to read the full digest.
Opening a cutoff title shows the live post -- compare it with the
cutoff before treating the body as reconstructed evidence (ADR 0016).
Opening a cutoff title shows the live post. Titles marked updated
after cutoff were rewritten after the run; compare those bodies
before treating them as reconstructed evidence (ADR 0016).
`POST /api/analysis-runs` records Pending on an authorized
cutoff capture (ADR 0017) and does not reconstruct lineage.
46 changes: 39 additions & 7 deletions backend/app/analysis_run_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,22 @@ def _iso(value: Any) -> str:
return value.isoformat() if hasattr(value, "isoformat") else str(value)


def _as_utc(value: datetime) -> datetime:
"""Treat a naive clock as UTC so cutoff comparison stays timezone-aware."""
if value.tzinfo is None:
return value.replace(tzinfo=timezone.utc)
return value.astimezone(timezone.utc)


def live_write_after_cutoff(updated_at: datetime, knowledge_cutoff: datetime) -> bool:
"""True when the live row was rewritten after the run's analysis clock.

``created_at <= knowledge_cutoff`` admits the title. ``updated_at`` is
the live write clock (ADR 0016). Equal times stay in-cutoff evidence.
"""
return _as_utc(updated_at) > _as_utc(knowledge_cutoff)


async def _counts_by_run(
conn: asyncpg.Connection,
run_ids: list[str],
Expand Down Expand Up @@ -258,15 +274,21 @@ async def fetch_visible_scope_posts(
scope_key: str | None,
affiliated_entity_ids: list[str],
knowledge_cutoff: Any,
) -> list[dict[str, str]]:
) -> list[dict[str, Any]]:
"""ABAC-visible post titles known at the run cutoff -- never a hidden body.

``knowledge_cutoff`` is the analysis clock (W3C Time / ISO 8601-1:2019;
ADR 0013/0016). A later live post must not appear inside an earlier run.
``updated_at`` is compared separately so the operator can see which
in-cutoff titles were rewritten after that clock. The live body is
still not returned.
"""
columns = (
"post_id, post_title, visibility_code, corporate_entity_id, updated_at"
)
if scope_kind_code == "analysis_scope_corporate_entity" and corporate_entity_id:
rows = await conn.fetch(
"select post_id, post_title, visibility_code, corporate_entity_id "
f"select {columns} "
"from source_post where corporate_entity_id = $1 "
"and created_at <= $2 "
"order by created_at, post_title",
Expand All @@ -275,7 +297,7 @@ async def fetch_visible_scope_posts(
)
elif scope_kind_code == "analysis_scope_process_unit" and process_unit_id:
rows = await conn.fetch(
"select post_id, post_title, visibility_code, corporate_entity_id "
f"select {columns} "
"from source_post where process_unit_id = $1 "
"and created_at <= $2 "
"order by created_at, post_title",
Expand All @@ -284,7 +306,7 @@ async def fetch_visible_scope_posts(
)
elif scope_kind_code == "analysis_scope_thread_group" and scope_key:
rows = await conn.fetch(
"select post_id, post_title, visibility_code, corporate_entity_id "
f"select {columns} "
"from source_post where thread_group_key = $1 "
"and created_at <= $2 "
"order by created_at, post_title",
Expand All @@ -293,20 +315,30 @@ async def fetch_visible_scope_posts(
)
elif scope_kind_code == "analysis_scope_all_visible":
rows = await conn.fetch(
"select post_id, post_title, visibility_code, corporate_entity_id "
f"select {columns} "
"from source_post where created_at <= $1 "
"order by created_at, post_title",
knowledge_cutoff,
)
else:
return []
affiliated = {str(entity_id) for entity_id in affiliated_entity_ids}
posts: list[dict[str, str]] = []
posts: list[dict[str, Any]] = []
for row in rows:
visible = row["visibility_code"] == "public" or str(row["corporate_entity_id"]) in affiliated
if not visible:
continue
posts.append({"post_id": str(row["post_id"]), "post_title": row["post_title"]})
updated_at = row["updated_at"]
posts.append(
{
"post_id": str(row["post_id"]),
"post_title": row["post_title"],
"updated_at": _iso(updated_at),
"live_after_cutoff": live_write_after_cutoff(
updated_at, knowledge_cutoff
),
}
)
return posts


Expand Down
79 changes: 75 additions & 4 deletions backend/tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
_MIGRATION_PATH = Path(__file__).resolve().parents[2] / "migrations" / "0001_initial_schema.sql"
_REGISTRY_MIGRATION = Path(__file__).resolve().parents[2] / "migrations" / "0018_analysis_run_registry.sql"
_RETENTION_MIGRATION = Path(__file__).resolve().parents[2] / "migrations" / "0020_analysis_run_retention_purge.sql"
_WRITE_CLOCK_MIGRATION = Path(__file__).resolve().parents[2] / "migrations" / "0022_source_post_write_clock.sql"


def _postgres_available() -> bool:
Expand Down Expand Up @@ -117,6 +118,7 @@ def seeded_db(demo_analyst_token):
cur.execute(_MIGRATION_PATH.read_text())
cur.execute(_REGISTRY_MIGRATION.read_text())
cur.execute(_RETENTION_MIGRATION.read_text())
cur.execute(_WRITE_CLOCK_MIGRATION.read_text())
cur.execute(
"insert into common_lookup_value (lookup_category, lookup_code, lookup_label) values "
"('corporate_entity_level', 'group', 'Group'), "
Expand Down Expand Up @@ -305,11 +307,21 @@ def _insert_post(
visibility_code: str,
body: str = "body",
created_at: str = "2026-01-10T12:00:00Z",
updated_at: str | None = None,
) -> str:
written_at = updated_at if updated_at is not None else created_at
cur.execute(
"insert into source_post (author_account_id, corporate_entity_id, post_title, post_body, voc_type_code, visibility_code, created_at) "
"values (%s, %s, %s, %s, 'voc', %s, %s) returning post_id",
(account_id, corporate_entity_id, title, body, visibility_code, created_at),
"insert into source_post (author_account_id, corporate_entity_id, post_title, post_body, voc_type_code, visibility_code, created_at, updated_at) "
"values (%s, %s, %s, %s, 'voc', %s, %s, %s) returning post_id",
(
account_id,
corporate_entity_id,
title,
body,
visibility_code,
created_at,
written_at,
),
)
return str(cur.fetchone()[0])

Expand All @@ -329,6 +341,14 @@ def _insert_post(
"A follow-up written after the January 2026 run cutoff.",
created_at="2026-01-20T12:00:00Z",
)
_insert_post(
"Edited own-corp private post",
own_corp_id,
"private",
"A January post rewritten after the run cutoff.",
created_at="2026-01-10T12:00:00Z",
updated_at="2026-01-13T09:00:00Z",
)

cur.execute(
"insert into cataloged_person (person_name, person_side_code) values "
Expand Down Expand Up @@ -494,8 +514,14 @@ def test_analysis_runs_are_labeled_aggregates_and_hide_other_scopes(
assert all("failure_code" not in event for event in history)
titles = {post["post_title"] for post in body["visible_posts"]}
assert "Own-corp private post" in titles
assert "Edited own-corp private post" in titles
assert "Late own-corp private post" not in titles
assert "Other-corp private post" not in titles
posts_by_title = {post["post_title"]: post for post in body["visible_posts"]}
assert posts_by_title["Own-corp private post"]["live_after_cutoff"] is False
assert posts_by_title["Edited own-corp private post"]["live_after_cutoff"] is True
assert posts_by_title["Edited own-corp private post"]["updated_at"].startswith("2026-01-13")
assert "post_body" not in posts_by_title["Edited own-corp private post"]
assert "postgresql://" not in str(body)
assert "visible_posts" not in visible

Expand Down Expand Up @@ -587,12 +613,57 @@ def test_post_list_includes_public_and_own_corp_but_excludes_other_corp(client,
response = client.get("/api/posts", headers={"Authorization": f"Bearer {demo_analyst_token}"})
assert response.status_code == 200
titles = {post["post_title"] for post in response.json()}
assert titles == {"Public post", "Own-corp private post", "Late own-corp private post"}
assert titles == {
"Public post",
"Own-corp private post",
"Late own-corp private post",
"Edited own-corp private post",
}
public = next(post for post in response.json() if post["post_title"] == "Public post")
assert public["voc_type_label"] == "Voice of Customer"
assert public["visibility_label"] == "Public"


def test_body_rewrite_moves_the_write_clock_unless_the_statement_sets_it(
client, demo_analyst_token, seeded_db
) -> None:
"""A later body write is live-after-cutoff; an explicit clock stays put."""
conn = psycopg2.connect(seeded_db["dsn"])
try:
with conn.cursor() as cur:
cur.execute(
"update source_post set post_body = %s where post_id = %s",
("Rewritten after the January run.", seeded_db["own_private_post_id"]),
)
cur.execute(
"select post_id from source_post where post_title = %s",
("Edited own-corp private post",),
)
edited_id = str(cur.fetchone()[0])
cur.execute(
"update source_post set post_body = %s, updated_at = %s "
"where post_id = %s",
("Still the January rewrite.", "2026-01-13T09:00:00Z", edited_id),
)
cur.execute(
"update source_post set thread_group_key = %s where post_title = %s",
("thread-group-rewrite", "Edited own-corp private post"),
)
conn.commit()
finally:
conn.close()

detail = client.get(
f"/api/analysis-runs/{seeded_db['visible_run_id']}",
headers={"Authorization": f"Bearer {demo_analyst_token}"},
)
assert detail.status_code == 200
posts = {post["post_title"]: post for post in detail.json()["visible_posts"]}
assert posts["Own-corp private post"]["live_after_cutoff"] is True
assert posts["Edited own-corp private post"]["live_after_cutoff"] is True
assert posts["Edited own-corp private post"]["updated_at"].startswith("2026-01-13")


def test_post_detail_uses_lookup_labels_not_raw_codes(client, demo_analyst_token, seeded_db) -> None:
response = client.get(
f"/api/posts/{seeded_db['public_post_id']}",
Expand Down
1 change: 1 addition & 0 deletions docker/postgres-init/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ COPY migrations/0017_prov_o_standard_relations.sql /docker-entrypoint-initdb.d/1
COPY migrations/0018_analysis_run_registry.sql /docker-entrypoint-initdb.d/19-analysis-run-registry.sql
COPY migrations/0019_role_catalog_identity.sql /docker-entrypoint-initdb.d/20-role-catalog-identity.sql
COPY migrations/0020_analysis_run_retention_purge.sql /docker-entrypoint-initdb.d/21-analysis-run-retention-purge.sql
COPY migrations/0022_source_post_write_clock.sql /docker-entrypoint-initdb.d/22-source-post-write-clock.sql
# Official image already drops to this account at runtime; declare it so
# the Dockerfile itself satisfies DS-0002 (explicit non-root USER).
USER postgres
17 changes: 11 additions & 6 deletions docs/adr/0016-analysis-run-knowledge-cutoff-posts.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,11 @@ every scope branch (corporate entity, process unit, thread group, and
all-visible). ABAC visibility is applied after that temporal gate.
Click-through still opens the live post body -- post versioning is a
later slice -- but the run list itself must not advertise a post the
run was not allowed to know. The detail must say that next action
plainly: compare the opened body with this cutoff before treating it
as reconstructed evidence.
run was not allowed to know. Detail compares the live `updated_at`
write clock with `knowledge_cutoff` and marks titles rewritten after
the run. The next action is specific: only those marked titles need a
cutoff comparison before treating the live body as reconstructed
evidence.

Reproducibility digests on the same detail use a labeled group whose
accessible name does not replace the visible prefixes (W3C Accessible
Expand All @@ -44,11 +46,14 @@ run.
- After `make seed`, the Demo Corp lineage run lists Demo public post
and other in-cutoff Demo Corp titles. The later fixture account-review
post (2026-02-10) does not appear.
- Open the run, read the live-body warning, then open a listed post
and compare it with the cutoff date.
- Open the run: Demo public post is marked updated after cutoff
(`updated_at` 2026-01-13). Demo private post is not.
- Hover a digest prefix to read the full code or configuration digest
when you need to match the API payload.
- Post-body versioning at the cutoff remains future work.
- Post-body versioning at the cutoff remains future work. The write
clock is a projection, not a stored cutoff body. Migration 0022
stamps `updated_at` on title or body rewrites and honors an explicit
`updated_at` so `make seed` can keep Demo public post at 2026-01-13.
- Thread-group *run list* visibility now uses the same cutoff
(ADR 0018). A later public post cannot surface a previously hidden
thread-group run.
Expand Down
3 changes: 2 additions & 1 deletion docs/doctoring/ANALYSIS_RUN_REGISTRY_REFERENCES.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
| Source | Product implication | Implemented evidence |
|---|---|---|
| W3C PROV-DM and PROV-O | Preserve identifiable entities, activities, agents, generation/use, and derivation without flattening provenance into display-only edges. | `analysis_source_snapshot`, `analysis_run`, authenticated requester, append-only status events, immutable digests; later product bindings continue to use the separate `provenance_*` layer from ADR 0011. |
| W3C Time Ontology in OWL | Keep temporal concepts explicit and avoid collapsing distinct clocks. | Evidence availability and snapshot capture remain on `analysis_source_snapshot`; analysis knowledge cutoff and request time remain on `analysis_run`; status occurrence and database record time remain distinct. `GET /api/analysis-runs/{id}` visible posts apply `created_at <= knowledge_cutoff` (ADR 0016). Opening a listed title warns that the live body may have changed after that cutoff. |
| W3C Time Ontology in OWL | Keep temporal concepts explicit and avoid collapsing distinct clocks. | Evidence availability and snapshot capture remain on `analysis_source_snapshot`; analysis knowledge cutoff and request time remain on `analysis_run`; status occurrence and database record time remain distinct. `GET /api/analysis-runs/{id}` visible posts apply `created_at <= knowledge_cutoff` (ADR 0016). Detail compares live `updated_at` with that cutoff and marks titles rewritten after the run. |
| W3C Accessible Name and Description Computation 1.1 | Do not let `aria-label` replace visible text the operator must hear. | Analysis-run digest prefixes live in a labeled group; the prefixes remain the accessible contents and the full digest is on `title` for hover verification. |
| ISO 8601-1:2019 | Use unambiguous timestamp representation and timezone-aware persistence. | PostgreSQL `timestamptz` for availability, capture, cutoff, request, occurrence, and record clocks; tests use explicit `Z` offsets. |
| PostgreSQL 18 constraints and trigger contracts | Put integrity close to durable truth and use constraints for row shape while triggers enforce cross-row state and serialization. | Digest/check constraints, category allowlists, account-scoped uniqueness, shape constraints, immutable-row triggers, shared snapshot-row locking, and serialized status transitions. |
Expand Down Expand Up @@ -71,6 +71,7 @@ provenance, retention, and immutable evidence rather than blanket masking.
|---|---|
| One snapshot supports multiple analyses | Insert two runs over one snapshot with different valid cutoffs. |
| Future evidence is excluded | Reject a run whose cutoff precedes the snapshot's maximum availability time. A late own-corp post stays out of `visible_posts`. |
| Live write clock is distinct from admission | An in-cutoff title with `updated_at` after the cutoff is marked `live_after_cutoff`; equal clocks stay unmarked. After `make seed`, Demo public post is the 2026-01-13 counter-example. A later title or body rewrite stamps `updated_at` unless the statement sets that clock (migration 0022). |
| Evidence cannot change after derivation | Reject snapshot/count updates and count insert/delete after the first run. |
| Count/run race is serialized | Both paths acquire the snapshot row first; a later concurrency test must prove one legal winner and no lost freeze. |
| Request identity is stable | Reject analysis-run updates; scope and lifecycle live in their own relations. |
Expand Down
1 change: 1 addition & 0 deletions docs/storybook-inventory.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ buyer-facing control you can click before changing product CSS.
|---|---|---|
| `Evidence/CitationChip` | Click a cited title to open that source post. | `--color-chip-border`, `--radius-chip`, `CitationChip` |
| `Chrome/PopupCloseButton` | Close the evidence panel or post popup. | `--space-close-inset`, `--font-size-close`, `PopupCloseButton` |
| `AnalysisRun/PostBadge` | Open the marked title and compare the live body with this run. | `--font-size-badge`, `--opacity-badge`, `PostBadge` |

Repeated web objects must use `frontend/src/styles/tokens.css` and a module
under `frontend/src/components/`. Do not add a second Node package manager;
Expand Down
2 changes: 1 addition & 1 deletion frontend/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "frontend",
"private": true,
"version": "0.87.0",
"version": "0.87.3",
"type": "module",
"scripts": {
"dev": "vite",
Expand Down
Loading
Loading