-
Notifications
You must be signed in to change notification settings - Fork 1
feat(ask): nominate persisted semantic evidence #672
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
seonghobae
wants to merge
26
commits into
main
Choose a base branch
from
feat/global-ask-semantic-public-current
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
5387acb
feat(ask): nominate persisted semantic evidence
6d4b0e3
docs(gaps): record current protected queue
6852439
docs(gap): reconcile semantic evidence queue
5468c04
docs: assign unique semantic nomination ADR number
fa24548
feat(ask): verify typed public semantic claims (#682)
seonghobae 9d1c5a1
test(ask): apply semantic search indexes in integration
f4cb992
fix(ask): show empty completed verification
3d743ff
fix(ask): pass claim client factory directly
91caeb0
style(ask): remove unused claim status import
ac521a2
fix(ask): isolate unexpected verification adapter failures
147f208
perf(ask): release pool before embedding calls
b7570c5
docs(adr): assign unique semantic decision ids
5aff98d
fix(semantic): isolate migration ids and late-bind verifier
99a0322
fix(schema): reserve unique public-verification migration id
eecbf88
fix(db): recover interrupted semantic indexes
213d661
fix(ask): classify embedding transport failures
672fdbf
fix(ask): defer optional verification client
748944a
fix(ask): preserve public verification boundaries
9460283
Merge remote-tracking branch 'origin/main' into restack/pr-672
seonghobae 3f76c18
fix(ask): connect semantic evidence safely
5d2626b
fix(ask): wire typed public claims
b464e88
docs: reconcile stacked evidence rows
b4169c0
docs: reconcile stacked audit records
62b2dba
Merge branch 'feat/global-ask-semantic-public-current' of https://git…
f78f036
docs(ask): state typed claim selection contract
a3e87a8
test: follow global Ask embedding contract
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,129 @@ | ||
| """Nominate Global Ask posts from persisted semantic and KG evidence.""" | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| from datetime import date | ||
|
|
||
| import asyncpg | ||
|
|
||
| from .post_eligibility import SOURCE_POST_ELIGIBILITY_SQL | ||
|
|
||
|
|
||
| async def semantic_candidate_post_ids( | ||
| conn: asyncpg.Connection, | ||
| question: str, | ||
| *, | ||
| maximum_candidates: int, | ||
| authorized_corporate_entity_ids: list[str], | ||
| authorized_process_unit_ids: list[str], | ||
| date_from: date | None, | ||
| date_to: date | None, | ||
| ) -> list[str]: | ||
| """Return bounded post IDs whose persisted semantic evidence matches. | ||
|
|
||
| Nomination grants no access and returns no evidence text. The caller must | ||
| apply the ordinary source-post RBAC/ABAC, eligibility, and time boundary | ||
| before reading any nominated row. | ||
| """ | ||
|
|
||
| if maximum_candidates <= 0 or not question.strip(): | ||
|
seonghobae marked this conversation as resolved.
|
||
| return [] | ||
| rows = await conn.fetch( # nosemgrep: python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli | ||
| f""" | ||
| with search_query as ( | ||
| select websearch_to_tsquery('simple', $1) as value | ||
| ), candidate_post as ( | ||
| select mention.post_id, post.created_at | ||
| from post_project_mention mention | ||
| join source_post post on post.post_id = mention.post_id | ||
| cross join search_query | ||
| where to_tsvector( | ||
| 'simple', | ||
| coalesce(mention.project_key, '') || ' ' || | ||
| coalesce(mention.project_name, '') || ' ' || | ||
| coalesce(mention.evidence_text, '') || ' ' || | ||
| coalesce(mention.ontology_iri, '') | ||
| ) @@ search_query.value | ||
| union all | ||
| select role.post_id, post.created_at | ||
| from post_summary_role role | ||
| join source_post post on post.post_id = role.post_id | ||
| cross join search_query | ||
| where to_tsvector( | ||
| 'simple', | ||
| coalesce(role.actor_name, '') || ' ' || | ||
| coalesce(role.responsibility, '') || ' ' || | ||
| coalesce(role.affiliated_organization_name, '') | ||
| ) @@ search_query.value | ||
| union all | ||
| select mention.post_id, post.created_at | ||
| from post_person_mention mention | ||
| join cataloged_person person on person.person_id = mention.person_id | ||
| join source_post post on post.post_id = mention.post_id | ||
| cross join search_query | ||
| where to_tsvector( | ||
| 'simple', coalesce(person.person_name, '') || ' ' || | ||
| coalesce(person.last_known_job_title, '') | ||
| ) @@ search_query.value | ||
| union all | ||
| select mention.post_id, post.created_at | ||
| from post_person_mention mention | ||
| join source_post post on post.post_id = mention.post_id | ||
| cross join search_query | ||
| where to_tsvector('simple', coalesce(mention.mention_context, '')) | ||
| @@ search_query.value | ||
| union all | ||
| select mention.post_id, post.created_at | ||
| from post_organization_mention mention | ||
| join corporate_entity entity | ||
| on entity.corporate_entity_id = mention.corporate_entity_id | ||
| join source_post post on post.post_id = mention.post_id | ||
| cross join search_query | ||
| where to_tsvector('simple', entity.entity_name) @@ search_query.value | ||
| union all | ||
| select mention.post_id, post.created_at | ||
| from post_team_mention mention | ||
| join cataloged_team team on team.team_id = mention.team_id | ||
| join source_post post on post.post_id = mention.post_id | ||
| cross join search_query | ||
| where to_tsvector( | ||
| 'simple', | ||
| coalesce(team.team_name, '') || ' ' || | ||
| coalesce(team.affiliated_organization_name, '') | ||
| ) @@ search_query.value | ||
| union all | ||
| select evidence.evidence_post_id, post.created_at | ||
| from knowledge_graph_edge edge | ||
| join knowledge_graph_edge_evidence evidence | ||
| on evidence.knowledge_graph_edge_id = edge.knowledge_graph_edge_id | ||
| join source_post post on post.post_id = evidence.evidence_post_id | ||
| cross join search_query | ||
| where to_tsvector( | ||
| 'simple', | ||
| replace(coalesce(edge.edge_type_code, '') || ' ' || | ||
| coalesce(edge.source_node_type_code, '') || ' ' || | ||
| coalesce(edge.target_node_type_code, ''), '_', ' ') | ||
| ) @@ search_query.value | ||
| ) | ||
| select candidate.post_id::text as post_id | ||
| from candidate_post candidate | ||
| join source_post post on post.post_id = candidate.post_id | ||
| where (post.visibility_code = 'public' | ||
| or (post.corporate_entity_id::text = any($3::text[]) | ||
| and (cardinality($4::text[]) = 0 | ||
| or post.process_unit_id::text = any($4::text[])))) | ||
| and {SOURCE_POST_ELIGIBILITY_SQL.format(alias='post')} | ||
| and ($5::date is null or (coalesce(post.event_occurred_at, post.created_at) at time zone 'Asia/Seoul')::date >= $5) | ||
| and ($6::date is null or (coalesce(post.event_occurred_at, post.created_at) at time zone 'Asia/Seoul')::date <= $6) | ||
|
seonghobae marked this conversation as resolved.
|
||
| group by candidate.post_id | ||
| order by max(candidate.created_at) desc, candidate.post_id desc | ||
| limit $2 | ||
|
seonghobae marked this conversation as resolved.
|
||
| """, | ||
| question, | ||
| maximum_candidates, | ||
| authorized_corporate_entity_ids, | ||
| authorized_process_unit_ids, | ||
| date_from, | ||
| date_to, | ||
| ) | ||
|
seonghobae marked this conversation as resolved.
seonghobae marked this conversation as resolved.
|
||
| return [str(row["post_id"]) for row in rows] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.