Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@ All notable changes to this project are documented here. Format follows

### Added

- ADR 0210's Dashboard consumer now persists a normalized, exact-provenance
projection for TEPP temporal topics and fast-mlsirm case-deletion model
influence. The API authorizes the fitted analysis scope before returning
rows; the UI preserves ties, multiple membership, uncertainty, time states,
and source links, and otherwise names the missing producer contract without
calculating a local score.

- Event Lineage now persists each reconstructed connection's independent
channel scores, the normalized weights actually used, and their
contributions. The Event Lineage DAG discloses those exact values as inferred
Expand Down
283 changes: 281 additions & 2 deletions backend/app/operations_dashboard.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from __future__ import annotations

from datetime import date
import json
from typing import Any, Protocol

from backend.app.post_eligibility import SOURCE_POST_ELIGIBILITY_SQL
Expand Down Expand Up @@ -106,11 +107,11 @@ def _operations_case_jsonld(
class _Connection(Protocol):
async def fetchrow(self, query: str, *args: object) -> Any:
"""Fetch one projected row."""
pass
pass # pragma: no cover - structural Protocol member

async def fetch(self, query: str, *args: object) -> list[Any]:
"""Fetch projected rows."""
pass
pass # pragma: no cover - structural Protocol member


def _visible_period_sql(alias: str = "post") -> str:
Expand Down Expand Up @@ -250,6 +251,18 @@ async def fetch_operations_dashboard(
""",
*args,
)
topic_context = (
{
"status_code": "not_applicable",
"reason_code": "external_information_view",
"next_action": "전체 Dashboard로 전환해 Topic model influence를 확인하세요.",
"required_contracts": [],
"model_run": None,
"topics": [],
}
if external_only
else await _fetch_topic_context_dashboard(conn, visible, args)
)
facts: dict[tuple[str, str], list[dict[str, str]]] = {}
for row in fact_rows:
key = (str(row["post_id"]), row["case_kind_code"])
Expand Down Expand Up @@ -304,6 +317,7 @@ async def fetch_operations_dashboard(
}
for kind, label in CASE_KIND_LABELS.items()
],
"topic_context": topic_context,
"cases": [
{
"post_id": str(row["post_id"]),
Expand Down Expand Up @@ -331,6 +345,271 @@ async def fetch_operations_dashboard(
}


async def _fetch_topic_context_dashboard(
conn: _Connection,
visible_post_sql: str,
args: tuple[object, ...],
) -> dict[str, Any]:
"""Project exact accepted producer rows or an actionable unavailable state."""
authorized_model_scope = """
((scope.scope_kind_code = 'analysis_scope_corporate_entity'
and scope.corporate_entity_id::text = any($1::text[])
and cardinality($2::text[]) = 0)
or
(scope.scope_kind_code = 'analysis_scope_process_unit'
and scope.process_unit_id::text = any($2::text[])))
"""
readiness = await conn.fetchrow(
f"""
with visible_post as (
select post.post_id
from source_post post
where {visible_post_sql}
)
select exists (
select 1
from topic_context_membership membership
join topic_model_run model
on model.topic_model_run_id = membership.topic_model_run_id
join analysis_run analysis on analysis.analysis_run_id = model.analysis_run_id
join analysis_run_scope scope on scope.analysis_run_id = analysis.analysis_run_id
join visible_post on visible_post.post_id = membership.source_post_id
where {authorized_model_scope}
) as tepp_posterior_persisted,
exists (
select 1
from topic_post_context_influence influence
join topic_context_membership membership
on membership.topic_model_run_id = influence.topic_model_run_id
and membership.topic_context_membership_id = influence.topic_context_membership_id
join topic_model_run model
on model.topic_model_run_id = influence.topic_model_run_id
join analysis_run analysis on analysis.analysis_run_id = model.analysis_run_id
join analysis_run_scope scope on scope.analysis_run_id = analysis.analysis_run_id
join visible_post on visible_post.post_id = membership.source_post_id
where {authorized_model_scope}
) as fast_mlsirm_influence_persisted
""",
*args,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Dashboard crashes on every non-external request

_fetch_topic_context_dashboard passes all five query parameters (*args) to two SQL statements that reference only $1$4; the fifth (external_only) is unused. asyncpg rejects the extra argument, so every non-external dashboard request fails. The mock-connection tests never bind real parameters, so CI stays green.

Prompt for agents
In _fetch_topic_context_dashboard (backend/app/operations_dashboard.py), both conn.fetchrow (the readiness query around line 393) and conn.fetch (the rows query around line 490) are called with *args, where args is the 5-element tuple built in fetch_operations_dashboard: (corporate_entity_ids, process_unit_ids, period_start, period_end, external_only). However, these two SQL statements only reference positional parameters $1 through $4 (via _visible_period_sql and authorized_model_scope); they never reference $5. asyncpg requires the number of passed arguments to exactly match the highest parameter referenced in the query, so passing 5 args to a 4-parameter query raises InterfaceError and the dashboard endpoint fails for every non-external request. Fix by passing only the first four arguments (e.g. *args[:4]) to both topic-context queries, since external_only is already handled by the caller before this function runs.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

)
Comment on lines +362 to +394

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Readiness query runs even when unused

The readiness fetchrow executes on every non-external request, but its result is consumed only in the if not rows: branch. When accepted rows exist, it is a wasted round-trip; it could be deferred until rows is empty.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

rows = await conn.fetch(
f"""
with visible_post as (
select post.post_id,
coalesce(post.event_occurred_at, post.created_at) as occurred_at
from source_post post
where {visible_post_sql}
), eligible as (
select model.topic_model_run_id, model.tepp_run_id, model.tepp_snapshot_id,
model.tepp_schema_version, model.tepp_model_contract_version,
model.tepp_artifact_sha256, model.posterior_draw_set_id,
model.posterior_draw_count, model.topic_count,
snapshot.snapshot_sha256 as source_snapshot_sha256,
analysis.knowledge_cutoff,
influence_run.topic_influence_run_id,
influence_run.fast_mlsirm_schema_version,
influence_run.fast_mlsirm_version,
influence_run.fast_mlsirm_code_revision,
influence_run.fast_mlsirm_artifact_sha256,
influence_run.compute_backend_code,
influence_run.precision_code,
influence_run.membership_fingerprint_sha256,
influence.topic_index, activity.state_code,
activity.valid_from as activity_valid_from,
activity.valid_to as activity_valid_to,
membership.dimension_code, membership.context_id,
context.context_label, membership.membership_weight,
membership.evidence_sha256 as membership_evidence_sha256,
membership.source_post_id, visible_post.occurred_at,
influence.influence_value,
influence.uncertainty_method_code,
influence.uncertainty_lower_value,
influence.uncertainty_upper_value,
influence.diagnostic_status_code,
influence_run.accepted_at
from topic_post_context_influence influence
join topic_influence_run influence_run
on influence_run.topic_model_run_id = influence.topic_model_run_id
and influence_run.topic_influence_run_id = influence.topic_influence_run_id
join topic_model_run model
on model.topic_model_run_id = influence.topic_model_run_id
join analysis_run analysis on analysis.analysis_run_id = model.analysis_run_id
join analysis_run_scope scope on scope.analysis_run_id = analysis.analysis_run_id
join analysis_source_snapshot snapshot
on snapshot.analysis_source_snapshot_id = analysis.analysis_source_snapshot_id
join topic_context_membership membership
on membership.topic_model_run_id = influence.topic_model_run_id
and membership.topic_context_membership_id = influence.topic_context_membership_id
join topic_context_definition context
on context.topic_model_run_id = membership.topic_model_run_id
and context.dimension_code = membership.dimension_code
and context.context_id = membership.context_id
join visible_post on visible_post.post_id = membership.source_post_id
join topic_activity_interval activity
on activity.topic_model_run_id = influence.topic_model_run_id
and activity.topic_index = influence.topic_index
and visible_post.occurred_at >= activity.valid_from
and visible_post.occurred_at < activity.valid_to
Comment on lines +448 to +452

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Influence rows dropped when no activity interval covers the post

The eligible CTE inner-joins topic_activity_interval on the post's occurred_at falling within [valid_from, valid_to). A post whose event time lands in a gap between intervals is silently dropped, so an authorized influential post never appears. Confirm the producer guarantees gap-free interval coverage.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

where visible_post.occurred_at >= membership.valid_from
and visible_post.occurred_at < membership.valid_to
and {authorized_model_scope}
), selected as (
select topic_model_run_id, topic_influence_run_id
from eligible
order by accepted_at desc, topic_model_run_id, topic_influence_run_id
limit 1
)
select eligible.*,
coalesce((
select jsonb_agg(jsonb_build_object(
'event_code', relation.event_code,
'source_topic_index', relation.source_topic_index,
'target_topic_index', relation.target_topic_index,
'event_time', relation.event_time,
'evidence_sha256', relation.evidence_sha256
) order by relation.event_time, relation.relation_ordinal)
from topic_lineage_relation relation
where relation.topic_model_run_id = eligible.topic_model_run_id
and (relation.source_topic_index = eligible.topic_index
or relation.target_topic_index = eligible.topic_index)
), '[]'::jsonb) as lineage_events
from eligible
join selected using (topic_model_run_id, topic_influence_run_id)
order by eligible.topic_index,
case eligible.dimension_code
when 'business_unit' then 0
when 'process_unit' then 1
when 'team' then 2
else 3
end,
eligible.context_label,
eligible.influence_value desc,
eligible.occurred_at,
eligible.source_post_id
""",
*args,
)
if not rows:
tepp_ready = bool(readiness and readiness["tepp_posterior_persisted"])
return {
"status_code": "unavailable",
"reason_code": (
"fast_mlsirm_influence_not_persisted"
if tepp_ready
else "tepp_topic_posterior_not_persisted"
),
"next_action": (
"동일 TEPP run·snapshot·cutoff에 결합된 fast-mlsirm 결과를 완료하세요."
if tepp_ready
else "TEPP posterior topic 계약 결과를 먼저 완료하세요."
),
"required_contracts": [
{
"authority": "TEPP",
"schema_version": "tepp.topic_context_posterior.v1",
"state_code": "persisted" if tepp_ready else "not_persisted",
},
{
"authority": "fast-mlsirm",
"schema_version": "fast_mlsirm.topic_context_influence.v1",
"state_code": (
"persisted"
if readiness and readiness["fast_mlsirm_influence_persisted"]
else "not_persisted"
),
},
],
"model_run": None,
"topics": [],
}

first = rows[0]
topics: dict[int, dict[str, Any]] = {}
for row in rows:
topic_index = int(row["topic_index"])
raw_lineage_events = row["lineage_events"]
lineage_events = (
json.loads(raw_lineage_events)
if isinstance(raw_lineage_events, str)
else list(raw_lineage_events)
)
topic = topics.setdefault(
topic_index,
{
"topic_index": topic_index,
"activity_intervals": [],
"lineage_events": lineage_events,
"contexts": [],
},
)
interval = {
"state_code": row["state_code"],
"valid_from": row["activity_valid_from"].isoformat(),
"valid_to": row["activity_valid_to"].isoformat(),
}
if interval not in topic["activity_intervals"]:
topic["activity_intervals"].append(interval)
context_key = (row["dimension_code"], row["context_id"])
context = next(
(
item
for item in topic["contexts"]
if (item["dimension_code"], item["context_id"]) == context_key
),
None,
)
if context is None:
context = {
"dimension_code": row["dimension_code"],
"context_id": row["context_id"],
"context_label": row["context_label"],
"influences": [],
}
topic["contexts"].append(context)
context["influences"].append(
{
"post_id": str(row["source_post_id"]),
"occurred_at": row["occurred_at"].isoformat(),
"topic_state_code": row["state_code"],
"model_influence": float(row["influence_value"]),
"uncertainty_method_code": row["uncertainty_method_code"],
"uncertainty_lower_value": float(row["uncertainty_lower_value"]),
"uncertainty_upper_value": float(row["uncertainty_upper_value"]),
"diagnostic_status_code": row["diagnostic_status_code"],
"membership_weight": float(row["membership_weight"]),
"membership_evidence_sha256": row["membership_evidence_sha256"],
}
)

return {
"status_code": "accepted",
"reason_code": None,
"next_action": "Topic과 조직 수준을 선택해 model influence와 근거 글을 확인하세요.",
"required_contracts": [
{"authority": "TEPP", "schema_version": first["tepp_schema_version"], "state_code": "persisted"},
{"authority": "fast-mlsirm", "schema_version": first["fast_mlsirm_schema_version"], "state_code": "persisted"},
],
"model_run": {
"tepp_run_id": first["tepp_run_id"],
"tepp_snapshot_id": first["tepp_snapshot_id"],
"source_snapshot_sha256": first["source_snapshot_sha256"],
"knowledge_cutoff": first["knowledge_cutoff"].isoformat(),
"tepp_model_contract_version": first["tepp_model_contract_version"],
"tepp_artifact_sha256": first["tepp_artifact_sha256"],
"posterior_draw_set_id": first["posterior_draw_set_id"],
"posterior_draw_count": int(first["posterior_draw_count"]),
"topic_count": int(first["topic_count"]),
"fast_mlsirm_version": first["fast_mlsirm_version"],
"fast_mlsirm_code_revision": first["fast_mlsirm_code_revision"],
"fast_mlsirm_artifact_sha256": first["fast_mlsirm_artifact_sha256"],
"compute_backend_code": first["compute_backend_code"],
"precision_code": first["precision_code"],
"membership_fingerprint_sha256": first["membership_fingerprint_sha256"],
},
"topics": list(topics.values()),
}


def _period_label(period_start: date | None, period_end: date | None) -> str:
"""Format the exact event-time interval represented by the projection."""
if period_start and period_end:
Expand Down
11 changes: 9 additions & 2 deletions docs/adr/0210-temporal-topic-context-influence-dashboard.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# ADR 0210: TEPP temporal topics and fast-mlsirm context influence

- Status: Accepted
- Implementation maturity: producer-contract required; consumer projection not yet shipped
- Implementation maturity: consumer projection candidate; accepted producer result unavailable
- Date: 2026-08-25
- Depends on: ADR 0132 (TEPP topic-lineage boundary), ADR 0206 (operations Dashboard)
- Upstream authorities: TEPP ADR 0012; fast-mlsirm ADR 0002 and ADR 0007
Expand Down Expand Up @@ -116,7 +116,7 @@ another dimension.

### LineageWeave consumer and persistence

Use normalized objects such as `topic_model_run`, `topic_definition`,
Use normalized objects `topic_model_run`, `topic_definition`,
`topic_activity_interval`, `topic_lineage_relation`, `topic_post_coordinate`,
`topic_context_membership`, `topic_influence_run`, and
`topic_post_context_influence`. Large result tables are partitioned by tenant
Expand All @@ -129,6 +129,13 @@ renormalizing scores. The frontend renders an exact-value table alongside the
temporal topic view, uses text/pattern as well as color for topic state, and
supports keyboard, touch, reduced motion, narrow viewports, and screen readers.

The LineageWeave consumer projection is allowed to land before activation. In
that state, it reports which exact producer contract is not persisted and
returns no topic, influence, rank, or fallback value. An accepted result is
readable only when its analysis-run scope is wholly authorized for the caller;
filtering individual result rows after a broader fit is insufficient because
the fitted value would still include hidden observations.

```mermaid
sequenceDiagram
participant Source as Authorized source snapshot
Expand Down
Loading