Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -332,6 +332,9 @@ v0.88.0). Do not invent a theta.
Opening a cutoff-rewritten title shows **Body this run knew** from
`source_post_revision` beside the live rewrite (ADR 0025 / v2.1.0).
Do not invent the earlier sentence when no revision covers the cutoff.
Global Ask uses the same revision cover when `knowledge_cutoff` is set
(ADR 0216 / #271); omit the field to keep the live-query contract, and
never substitute a live body for a missing historical cover.

A corporate-entity similarity result has three outcomes: unique, miss,
or tie (ADR 0026). A tie is not a miss. Keep the organization name
Expand Down
4 changes: 3 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,7 +240,9 @@ Each direct edge includes `interval_relation_code` /
`interval_relation_label` computed from the posts' observed windows.
Global Ask merges cited threads from one post/edge fetch pair and
caps the payload at the landing node bound, keeping cited posts first
(ADR 0169). Open a cited post to read the focused thread.
(ADR 0169). Optional `knowledge_cutoff` on `POST /api/ask` selects the
covering `source_post_revision` and never substitutes a live body
(ADR 0216). Open a cited post to read the focused thread.
`POST /api/lineage/rebuild` (`post_admin`) re-runs `reconstruct()` over
every `source_post` and atomically rewrites edges, channel signals, and
Allen interval relations. Reconstruct grouping is
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.d/2.19.0-global-ask-knowledge-cutoff.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# 2.19.0 Global Ask knowledge cutoff

Ask Agent now accepts an optional UTC knowledge cutoff. Dated questions use
the retained source-post revision from that clock, never the live rewrite,
and say when a historical body was not kept. Leaving the cutoff blank keeps
the live-query contract.
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ All notable changes to this project are documented here. Format follows

### Added


- Expanded Voice-of-X post taxonomy (ADR 0246): the governed `voc_type`
scheme adds Voice of Supplier, Employee, Business, Regulator, Investor,
Society, and Process as source-post categories. Ontology SKOS concepts and
Expand Down Expand Up @@ -63,6 +62,11 @@ All notable changes to this project are documented here. Format follows
deterministic application read
model (`lineageweave.worker_function_taxonomy`) exposes fail-closed
lookups; ranks are scale positions and are never used as weights.
=======

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Stray merge conflict marker left in changelog

A bare ======= git conflict marker sits between two bullet entries in the Added section, with no surrounding conflict markers. It renders as a literal line in the published changelog.

Suggested change
=======
- Global Ask accepts an optional UTC `knowledge_cutoff`. Dated questions
Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

- Global Ask accepts an optional UTC `knowledge_cutoff`. Dated questions
retrieve only posts available by that clock, cite the retained
`source_post_revision`, and name when a historical body was not kept.
Omitting the cutoff keeps the live-query contract (ADR 0216 / #271).
- Persist explicit paragraph, list, table, MathML formula, and caller-parsed
conversation-turn semantic-unit kinds without inferring absent boundaries.
- Event Lineage now persists each reconstructed connection's independent
Expand Down
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ Opening a cutoff-rewritten title shows **Body this run knew** from
`source_post_revision` beside the live rewrite, with both clocks named.
Compare those two texts before treating the live body as reconstructed
evidence; do not invent an earlier sentence when no revision covers the
cutoff.
cutoff. Global Ask optional `knowledge_cutoff` uses the same cover
(ADR 0216).

## Where the rest lives

Expand Down
3 changes: 3 additions & 0 deletions backend/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -3161,6 +3161,9 @@ async def ask_agent(
polls ``GET /api/ask/jobs/{id}`` for the settled answer. Submission
still fails fast on the states that cannot ever succeed (blank
question, missing permission, unconfigured orchestrator).

Optional ``knowledge_cutoff`` selects retained evidence available at
that clock. Omitting it keeps the live-query contract (ADR 0216).
"""
return await submit_global_ask(
pool=pool,
Expand Down
3 changes: 2 additions & 1 deletion backend/app/post_chat_ingestion.py
Comment thread
seonghobae marked this conversation as resolved.
Comment thread
seonghobae marked this conversation as resolved.
Comment thread
seonghobae marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,6 @@ def _seoul_today() -> date:

_POST_CHAT_CANDIDATE_LIMIT = 32


def _source_hint_facts(row: Any) -> tuple[str, ...]:
"""Render raw source fields as explicitly weak, column-level evidence."""
facts: list[str] = []
Expand Down Expand Up @@ -834,6 +833,8 @@ async def gather_global_chat_sources(
# `find_linked_post_ids`'s `.direct` set used by the post-scoped chat
# flow. Only the top match is expanded so lower-ranked semantic candidates
# cannot each pull a separate lineage chain into the bounded context.
# Cutoff answers skip this expansion: reconstructed edges have no
# available-time contract and must not leak later neighbors (ADR 0216).
lineage_neighbor_ids: list[str] = []
lineage_anchor_id = candidate_ids[0] if candidate_ids else None
if lineage_anchor_id and knowledge_cutoff is None:
Comment thread
seonghobae marked this conversation as resolved.
Expand Down
2 changes: 1 addition & 1 deletion backend/app/source_post_revision.py
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ async def fetch_known_at_revisions(
) -> dict[str, dict[str, str]]:
"""Batch-load the retained revision covering ``as_of`` for each post.

Missing posts stay absent so callers can report an honest historical-body
Missing covers are omitted so callers can report an honest historical-body
limitation without substituting the live title or body.
"""

Expand Down
40 changes: 40 additions & 0 deletions frontend/src/App.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { fireEvent, render, screen, waitFor, within } from "@testing-library/rea
import userEvent from "@testing-library/user-event";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import App from "./App";
import { optionalKnowledgeCutoffIso } from "./api";
import { setLocale } from "./i18n";
import { OIDC_RETURN_URL_STORAGE_KEY } from "./oidcReturnUrl";

Expand All @@ -27,6 +28,16 @@ beforeEach(() => {
};
});

it("normalizes valid knowledge cutoffs and rejects invalid input", () => {
expect(optionalKnowledgeCutoffIso("")).toBeUndefined();
expect(optionalKnowledgeCutoffIso("2026-01-15T12:00")).toBe(
new Date("2026-01-15T12:00").toISOString(),
);
expect(() => optionalKnowledgeCutoffIso("not-a-date")).toThrow(
"invalid knowledge cutoff",
);
});

afterEach(() => {
vi.unstubAllGlobals();
window.history.replaceState({}, "", "/");
Expand Down Expand Up @@ -1971,6 +1982,35 @@ describe("App, authenticated", () => {
expect(screen.queryByText(/ontology_iri|contextual_orchestrator/i)).not.toBeInTheDocument();
});

it("converts the local knowledge cutoff to UTC for Global Ask", async () => {
const fetchMock = stubBackend();
render(<App />);
expect(await screen.findByRole("button", { name: "View post: Public post" })).toBeInTheDocument();
await userEvent.click(screen.getByRole("button", { name: "Ask Agent" }));
expect(screen.getByLabelText("Use evidence available by (optional)")).toBeInTheDocument();
expect(screen.getByText("Choose a time on this device, or leave blank to use the latest evidence.")).toBeInTheDocument();
await userEvent.type(screen.getByRole("textbox", { name: "Ask a question" }), "Phoenix?");
await userEvent.type(
screen.getByLabelText("Use evidence available by (optional)"),
"2026-01-15T12:00",
);
await userEvent.click(screen.getByRole("button", { name: "Ask" }));
expect(
await screen.findByText("The cited project is supported by the stored semantic evidence."),
).toBeInTheDocument();
const askCall = fetchMock.mock.calls.find(
([input, init]) => String(input).endsWith("/api/ask") && (init as RequestInit | undefined)?.method === "POST",
);
expect(askCall).toBeTruthy();
const askInit = askCall?.[1] as RequestInit | undefined;
expect(askInit).toBeDefined();
expect(JSON.parse(String(askInit?.body))).toEqual({
question: "Phoenix?",
verify_external: false,
knowledge_cutoff: new Date("2026-01-15T12:00").toISOString(),
});
Comment thread
seonghobae marked this conversation as resolved.
});

it("localizes Ask delivery copy instead of rendering Korean literals in English", async () => {
stubBackend({ askDelivery: true });
render(<App />);
Expand Down
22 changes: 18 additions & 4 deletions frontend/src/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ import { useAuth } from "react-oidc-context";
import {
askPostChat,
askAgent,
optionalKnowledgeCutoffIso,
BackendError,
createAnalysisRun,
startAnalysisRun,
Expand Down Expand Up @@ -4830,11 +4831,11 @@ export function AskAgentPanel({
onOpenPost: (postId: string) => void;
}) {
const [question, setQuestion] = useState("");
const [knowledgeCutoff, setKnowledgeCutoff] = useState("");
const [answer, setAnswer] = useState<AskAgentResponse | null>(null);
const [error, setError] = useState<string | null>(null);
const [asking, setAsking] = useState(false);
Comment thread
seonghobae marked this conversation as resolved.
const [verifyExternal, setVerifyExternal] = useState(false);
const [knowledgeCutoff, setKnowledgeCutoff] = useState("");
const [evidenceLayerPostId, setEvidenceLayerPostId] = useState<string | null>(null);
const now = new Date();
const localKnowledgeCutoffMax = new Date(
Expand All @@ -4844,6 +4845,14 @@ export function AskAgentPanel({
async function handleAsk() {
const normalized = question.trim();
if (!normalized) return;
let cutoff: string | undefined;
try {
cutoff = optionalKnowledgeCutoffIso(knowledgeCutoff);
} catch {
setAnswer(null);
setError(t("Enter a valid knowledge cutoff, then ask again."));
return;
}
setAsking(true);
setError(null);
try {
Comment thread
seonghobae marked this conversation as resolved.
Expand All @@ -4852,7 +4861,7 @@ export function AskAgentPanel({
accessToken,
normalized,
verifyExternal,
knowledgeCutoff ? new Date(knowledgeCutoff).toISOString() : undefined,
cutoff,
),
);
} catch (err) {
Expand Down Expand Up @@ -4888,13 +4897,15 @@ export function AskAgentPanel({
<span>{t("Check eligible public claims")}</span>
</label>
<label className="ask-agent-field">
<span>{t("Knowledge cutoff (optional)")}</span>
<span>{t("Use evidence available by (optional)")}</span>
<input
type="datetime-local"
aria-label={t("Use evidence available by (optional)")}
value={knowledgeCutoff}
max={localKnowledgeCutoffMax}
onChange={(event) => setKnowledgeCutoff(event.target.value)}
/>
<small>{t("Choose a time on this device, or leave blank to use the latest evidence.")}</small>
</label>
<button className="btn-primary" onClick={() => void handleAsk()} disabled={asking || !question.trim()}>
{asking ? t("Asking...") : t("Ask")}
Expand All @@ -4911,7 +4922,7 @@ export function AskAgentPanel({
{answer.grounding_status === "fully_cutoff_grounded"
? t("Fully cutoff-grounded")
: t("Partially cutoff-grounded")}
{` · ${answer.knowledge_cutoff}`}
{` · ${new Date(answer.knowledge_cutoff).toLocaleString()}`}
</p>
{answer.limitations?.length ? (
<p role="alert">
Expand Down Expand Up @@ -4952,6 +4963,9 @@ export function AskAgentPanel({
</span>
) : null}
</button>
{post.historical_body_unavailable ? (
<p className="post-meta">{t("Historical body unavailable")}</p>
) : null}
Comment thread
seonghobae marked this conversation as resolved.
Comment thread
seonghobae marked this conversation as resolved.
<button
type="button"
className="citation-chip"
Expand Down
5 changes: 4 additions & 1 deletion frontend/src/AskAgentCutoff.stories.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,10 @@ export const PartialHistoricalEvidence: Story = {
play: async ({ canvasElement }) => {
const canvas = within(canvasElement);
await userEvent.type(canvas.getByLabelText("Ask a question"), "What was known about Apollo?");
await userEvent.type(canvas.getByLabelText("Knowledge cutoff (optional)"), "2026-01-15T12:00");
await userEvent.type(
canvas.getByLabelText("Use evidence available by (optional)"),
"2026-01-15T12:00",
);
await userEvent.click(canvas.getByRole("button", { name: "Ask" }));
await expect(canvas.findByText(/Partially cutoff-grounded/)).resolves.toBeVisible();
await expect(canvas.getByRole("alert")).toHaveTextContent("Current-only semantic channels were excluded");
Expand Down
2 changes: 1 addition & 1 deletion frontend/src/AskAgentPanel.test.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ describe("AskAgentPanel public verification", () => {
screen.getByRole("checkbox", { name: "Check eligible public claims" }),
);
await userEvent.type(
screen.getByLabelText("Knowledge cutoff (optional)"),
screen.getByLabelText("Use evidence available by (optional)"),
"2026-01-15T12:00",
);
await userEvent.click(screen.getByRole("button", { name: "Ask" }));
Expand Down
10 changes: 10 additions & 0 deletions frontend/src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1200,6 +1200,16 @@ interface AskJobStatus {
failure_detail?: string | null;
}

export function optionalKnowledgeCutoffIso(value: string): string | undefined {
const input = value.trim();
if (!input) return undefined;
const parsed = new Date(input);
if (Number.isNaN(parsed.getTime())) {
throw new RangeError("invalid knowledge cutoff");
}
return parsed.toISOString();
}

/** Submit the question as an asynchronous job and poll it to completion.
* The signature and resolved value are unchanged from the old synchronous
* call, so callers (AskAgentPanel) keep their existing pending/complete
Expand Down
16 changes: 16 additions & 0 deletions frontend/src/i18n.ts
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,10 @@ const TRANSLATIONS: Partial<Record<Locale, Record<string, string>>> = {
"Ask a question": "질문 입력",
"Check eligible public claims": "검증 가능한 공개 주장을 확인",
"Knowledge cutoff (optional)": "지식 컷오프(선택)",
"Use evidence available by (optional)": "이 시점까지 사용 가능한 근거 사용(선택)",
"Choose a time on this device, or leave blank to use the latest evidence.": "이 기기의 시간을 선택하거나, 최신 근거를 사용하려면 비워 두세요.",
"Historical body unavailable": "해당 시점의 본문을 사용할 수 없습니다",
"Enter a valid knowledge cutoff, then ask again.": "올바른 지식 컷오프를 입력한 뒤 다시 질문하세요.",
"Knowledge-cutoff grounding": "지식 컷오프 근거 상태",
"Fully cutoff-grounded": "컷오프 시점 근거로 완전히 구성됨",
"Partially cutoff-grounded": "컷오프 시점 근거로 일부만 구성됨",
Expand Down Expand Up @@ -724,6 +728,10 @@ const TRANSLATIONS: Partial<Record<Locale, Record<string, string>>> = {
"Ask a question": "输入问题",
"Check eligible public claims": "核验符合条件的公开声明",
"Knowledge cutoff (optional)": "知识截止时间(可选)",
"Use evidence available by (optional)": "使用截至此时间可用的证据(可选)",
"Choose a time on this device, or leave blank to use the latest evidence.": "选择此设备上的时间,或留空以使用最新证据。",
"Historical body unavailable": "该时间点的正文不可用",
"Enter a valid knowledge cutoff, then ask again.": "请输入有效的知识截止时间,然后重新提问。",
"Knowledge-cutoff grounding": "知识截止依据状态",
"Fully cutoff-grounded": "完全基于截止时间证据",
"Partially cutoff-grounded": "部分基于截止时间证据",
Expand Down Expand Up @@ -1277,6 +1285,10 @@ const TRANSLATIONS: Partial<Record<Locale, Record<string, string>>> = {
"Ask a question": "質問を入力",
"Check eligible public claims": "対象となる公開主張を検証",
"Knowledge cutoff (optional)": "知識カットオフ(任意)",
"Use evidence available by (optional)": "この時点までに利用可能な証拠を使用(任意)",
"Choose a time on this device, or leave blank to use the latest evidence.": "この端末の時刻を選択するか、最新の証拠を使用する場合は空欄にしてください。",
"Historical body unavailable": "指定時点の本文は利用できません",
"Enter a valid knowledge cutoff, then ask again.": "有効な知識カットオフを入力して、もう一度質問してください。",
"Knowledge-cutoff grounding": "知識カットオフ根拠状態",
"Fully cutoff-grounded": "カットオフ時点の根拠で完全に構成",
"Partially cutoff-grounded": "カットオフ時点の根拠で部分的に構成",
Expand Down Expand Up @@ -1809,6 +1821,10 @@ const TRANSLATIONS: Partial<Record<Locale, Record<string, string>>> = {
"Ask a question": "Nhập câu hỏi",
"Check eligible public claims": "Kiểm tra các tuyên bố công khai đủ điều kiện",
"Knowledge cutoff (optional)": "Mốc cắt tri thức (tùy chọn)",
"Use evidence available by (optional)": "Dùng bằng chứng có sẵn đến thời điểm này (tùy chọn)",
"Choose a time on this device, or leave blank to use the latest evidence.": "Chọn thời gian trên thiết bị này, hoặc để trống để dùng bằng chứng mới nhất.",
"Historical body unavailable": "Nội dung tại thời điểm đó không khả dụng",
"Enter a valid knowledge cutoff, then ask again.": "Nhập mốc cắt tri thức hợp lệ rồi đặt câu hỏi lại.",
"Knowledge-cutoff grounding": "Trạng thái căn cứ theo mốc cắt tri thức",
"Fully cutoff-grounded": "Được căn cứ đầy đủ tại mốc cắt",
"Partially cutoff-grounded": "Được căn cứ một phần tại mốc cắt",
Expand Down
2 changes: 2 additions & 0 deletions lineageweave/post_chat.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ def cited_post_summaries(

The sliding evidence chip must show the source post's title, not a
truncated UUID -- a missing title is omitted, never invented.
Cutoff citations also name the retained revision and limitation flags.
"""
by_id = {source.post_id: source for source in sources}
citations: list[dict[str, str | bool | list[str] | None]] = []
Expand Down Expand Up @@ -157,6 +158,7 @@ def _buyer_evidence_kind(fact: str) -> str:
return "source_field"



def _buyer_evidence_text(fact: str) -> str:
cleaned = re.sub(r"\s*\|\s*(?:ontology_iri|extraction_method|confidence):\s*[^|\[]+", "", fact)
cleaned = re.sub(r"\s*\[provenance=[^]]+\]", "", cleaned)
Expand Down
2 changes: 2 additions & 0 deletions tests/test_global_ask_sources.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,7 @@ class FakeConnection:
async def fetch(self, query: str, *args):
if "unit_similarity" in query:
assert "authorized_evidence_candidates" in query
assert "$9::timestamptz" not in query
assert query.index("authorized_evidence_candidates") < query.rindex("limit $8")
assert args[8] == ["exclusive responsibility"]
return [
Expand Down Expand Up @@ -209,6 +210,7 @@ async def fetch(self, query: str, *args):

source_query, source_args = calls[-1]
assert "process_unit_id::text = any($2::text[])" in source_query
assert source_query.count("created_at <= $7") == 1
assert source_args[:2] == (["corp-demo"], ["process-demo"])


Expand Down
16 changes: 15 additions & 1 deletion tests/test_source_post_revision.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,11 @@
from datetime import datetime, timezone
from pathlib import Path

from backend.app.source_post_revision import parse_as_of_clock, revision_covers_clock
from backend.app.source_post_revision import (
fetch_known_at_revisions,
parse_as_of_clock,
revision_covers_clock,
)

_ROOT = Path(__file__).resolve().parents[1]
_MIGRATION = _ROOT / "migrations" / "0024_source_post_revision.sql"
Expand Down Expand Up @@ -59,3 +63,13 @@ def test_revision_migration_records_title_or_body_rewrites_only() -> None:
assert seed.index("0024_source_post_revision.sql") < seed.index(
"0025_role_person_catalog_identity.sql"
)


def test_batch_revision_lookup_omits_missing_covers() -> None:
import inspect

source = inspect.getsource(fetch_known_at_revisions)
assert "source_post_revision" in source
assert "written_at <= $2" in source
assert "superseded_at is null or superseded_at > $2" in source
assert "never a live body" in source.lower() or "Missing covers are omitted" in source
Loading