From eb6901749a4c4915dc33878bc84aeeb333ad58a7 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sat, 22 Aug 2026 20:13:42 +0900 Subject: [PATCH 1/3] docs: record the zcrht811_export_rows -> source_post import mapping Documents the column mapping for scripts/import_postgresql_posts.py against the goal's source DSN (postgresql://.../postgres public.zcrht811_export_rows), derived by inspecting real column values (not guessed from SAP field-name convention alone) and verified end to end: ran the unmodified importer against the real source table, targeting a throwaway database created and dropped in the same session. 9 of 10 sampled rows imported; the 1 skip was the correct publication-state exclusion (dtsts_field = 'W', this codebase's own WRITING_SOURCE_DETAIL_STATE_CODE concept). The one genuinely non-obvious finding: mdraft_field (the field whose name most directly suggests "draft") is empty on every row in the table -- confirmed by an aggregate group by across all ~43,814 rows, not a sample -- so it carries no usable signal. dtsts_field is mapped to both --detail-state-column and --draft-column/--exclude-draft-value W since it's the field that actually carries the source's real lifecycle signal. No row content (real customer/project/employee data) appears in this document or was written to any git-tracked location; the throwaway verification database was dropped immediately after the import assertion. --- docs/zcrht811-export-rows-import-mapping.md | 96 +++++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 docs/zcrht811-export-rows-import-mapping.md diff --git a/docs/zcrht811-export-rows-import-mapping.md b/docs/zcrht811-export-rows-import-mapping.md new file mode 100644 index 000000000..f07865870 --- /dev/null +++ b/docs/zcrht811-export-rows-import-mapping.md @@ -0,0 +1,96 @@ +# `zcrht811_export_rows` import mapping + +Column mapping for importing a SAP VOC (Voice of Customer) ticket export +into `source_post` via `scripts/import_postgresql_posts.py`. Derived by +inspecting real column values in the source table -- not guessed from SAP +field-name convention alone -- and verified end to end against a throwaway +target database (imported real rows through the real importer, confirmed +`source_post` row shape, dropped the database). No row content from the +source table appears in this document or in git history; only the schema +mapping does. + +## Field mapping + +| `zcrht811_export_rows` column | Importer flag | Notes | +|---|---|---| +| `guid_field` | `--post-id-column` | The stable source UUID. | +| `docnosub_field` | `--record-key-column` | Human-readable document number (date-based, e.g. `YYMMDD-NNNN-NN`). | +| `title_field` | `--title-column` | | +| `voccts_field` | `--body-column` | Rich HTML VOC content. `vocctsb_field` is a secondary/backup content field, observed empty on every sampled row -- not mapped. | +| `erdat_field` + `erzet_field` | `--created-at-column` | Two separate SAP date/time columns (`ERDAT`/`ERZET` convention); combine in the source query with `(erdat_field + erzet_field)`, not mappable as a single raw column. | +| `aedat_field` + `aezet_field` | `--updated-at-column` | Same combination pattern (`AEDAT`/`AEZET`). | +| `voctp_field` | `--voc-type-column` | Values observed: `VOC`, `VOP` (also expect `VOM`/`VOCC`/`VOCO` per the target vocabulary); maps directly, no translation needed. | +| `dtsts_field` | `--detail-state-column` **and** `--draft-column` (`--exclude-draft-value W`) | See "Publication-state gating" below -- this is the one non-obvious mapping. | +| `loevm_field` | `--deleted-column` (`--exclude-deleted-value X`) | Standard SAP deletion flag (`LOEVM`). Observed on ~0.7% of rows (295 of 43,814 at time of writing). | +| `kunnr_field` | `--source-customer-code-column` | Standard SAP customer number (`KUNNR`). | +| `zgbispjtno_field` | `--source-project-code-column` | Custom Z-field; project reference number, often empty (not every VOC record names a project). | +| `pucode_field` | `--source-business-unit-column` | Duplicated by `voc_pucode` in the source table; `pucode_field` was used since it reads as the more specific/authoritative of the two. | +| `bukrs_field` | `--source-company-code-column` | Standard SAP company code (`BUKRS`). | +| `userid_field` | `--source-author-code-column` | | +| `ernam_field` | `--source-author-name-column` | | + +Not mapped (no corresponding target concept, or redundant with a mapped +column): `mandt_field` (SAP client/tenant -- a single fixed value for this +export, not per-record data), `acthguid_field` (looks like a +parent/thread activity GUID; worth investigating for +`--thread-group-column` in a follow-up once its relationship to +`post_lineage_edge`-style grouping is understood -- not mapped here to +avoid guessing), `grade_field`, `inspt_field`, `duedt_field`, `vbeln_field`, +`posnr_field`, `ststs_field` (a second, coarser status code alongside +`dtsts_field`; not needed once `dtsts_field` gates publication state), +`land1_field`, `voc_pucode` (see `pucode_field` above), `admin_txt`, +`mdraft_field`/`mreg_field` (observed **always empty** across every row in +the table -- cannot serve as a draft signal; see below), `pgcode_field`, +`erdpt_field`/`erdlo_field`/`ertlo_field`/`ertcd_field`, +`aedlo_field`/`aetlo_field`/`aetcd_field`, `source_artifact_path` / +`source_artifact_sha256` (present on this table but not used by this +mapping; a body stored as an external artifact would need +`--body-artifact-path-column`/`--body-artifact-sha256-column` instead of +`--body-column` if `voccts_field` is ever empty for a real row -- not +observed in the rows sampled here, all of which had inline body content). + +## Publication-state gating + +The importer requires a `--draft-column` with at least one non-empty value +and at least one `--exclude-draft-value` (`_validate_publication_state` in +`scripts/import_postgresql_posts.py`). `mdraft_field` -- the field whose +name most directly suggests "draft" -- is empty on every row in this table +(confirmed by an aggregate `group by`, not a sample): it carries no signal +at all for this export. + +`dtsts_field` does carry a real lifecycle signal. Its observed value +distribution (of ~43,814 rows): + +| `dtsts_field` | Approximate share | +|---|---| +| `A` | ~94% | +| `W` | ~5% | +| `D`, `R` | remainder | + +`W` lines up with this codebase's own `WRITING_SOURCE_DETAIL_STATE_CODE` +concept (`backend/app/main.py`'s `_can_use_post_for_analysis` already +excludes posts in that state from every derived feature) -- a +`source_detail_state_code` of `"w"` is exactly what a genuinely +in-progress/not-yet-finalized record should carry. Mapping `dtsts_field` +to **both** `--detail-state-column` (so the raw state is preserved in +`source_post.source_detail_state_code`) and `--draft-column` with +`--exclude-draft-value W` (so an in-progress record is gated out of the +import the same way this codebase already gates it out of every derived +feature) is the mapping that best matches what the source data actually +means, not an arbitrary reuse of one column for two purposes. + +## Verified import (throwaway database only) + +Ran the unmodified `scripts/import_postgresql_posts.py` against +`postgresql://seonghobae@localhost/postgres` `public.zcrht811_export_rows` +(10 non-deleted rows with non-empty body content, `--allow-demo-corporate-entity` +with a clearly-marked `DEMO-SAP-VERIFY-*` scope), targeting a throwaway +database created and dropped in the same session: + +- 10 source rows read, 1 correctly skipped (`dtsts_field = 'W'`), 9 imported. +- All 9 imported rows had a non-empty `post_title`; 8 had a resolved + `source_customer_code`; 4 had a resolved `source_project_code` (expected + sparsity -- not every VOC record names a project). + +No row content was written to any git-tracked location; the throwaway +database was dropped immediately after the assertion above. From 7e497ef62b9466a702dd8cdf5f2393c62f56fa75 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Sat, 22 Aug 2026 20:32:58 +0900 Subject: [PATCH 2/3] fix(frontend): use OIDC return-url helpers and guard AdminPanel render Same shared-ancestor bug as #418/#415/#426/#427/#429: the login button built an unsanitized returnUrl inline instead of returnUrlFromLocation()/ rememberOidcReturnUrl(), and AdminPanel's accessToken (string, required) was rendered from a string | undefined at both call sites. --- frontend/src/App.test.tsx | 3 +++ frontend/src/App.tsx | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/frontend/src/App.test.tsx b/frontend/src/App.test.tsx index 7462abd2c..70eb27590 100644 --- a/frontend/src/App.test.tsx +++ b/frontend/src/App.test.tsx @@ -41,6 +41,9 @@ describe("App, unauthenticated", () => { state: expect.objectContaining({ returnUrl: expect.stringMatching(/^\//) }), }), ); + // Persisted as a fallback in case the OIDC state round-trip is dropped + // (see oidcReturnUrl.ts's restoreOidcReturnUrl, consumed in main.tsx). + expect(window.sessionStorage.getItem("lineageweave.oidc.returnUrl")).toMatch(/^\//); }); }); diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 6fba0dd41..6e52be55d 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -4610,7 +4610,8 @@ export default function App({ showLabPanels = false }: { showLabPanels?: boolean
- {destination === "admin" ? : null}