From 68f77a4aedcec428398db2aff8aaecc6fe9b5349 Mon Sep 17 00:00:00 2001 From: Manish Kumar Date: Sun, 30 Aug 2026 22:44:20 -0500 Subject: [PATCH] docs(audit): close SAT-2 producer rollout --- docs/AUDIT_TENANT_PRODUCER_ROLLOUT_SAT2.md | 130 +++++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 docs/AUDIT_TENANT_PRODUCER_ROLLOUT_SAT2.md diff --git a/docs/AUDIT_TENANT_PRODUCER_ROLLOUT_SAT2.md b/docs/AUDIT_TENANT_PRODUCER_ROLLOUT_SAT2.md new file mode 100644 index 0000000..96d3139 --- /dev/null +++ b/docs/AUDIT_TENANT_PRODUCER_ROLLOUT_SAT2.md @@ -0,0 +1,130 @@ +# Security Audit Tenant Producer Rollout — SAT-2 release closure + +## Baseline + +SAT-2 was reconciled against Security Audit main at merge +073e02377da5f05ba5aa2e9dc1b18e81ea09dcab. That baseline contains SAT-1's +first-class organization_id and tenant_scope +fields, signed serialized event payloads, durable persistence, the nullable +organization column, the unknown default, migration 0003_tenant_contract, +and the organization/timestamp/event index. + +## Classification rules + +- organization: only an organization ID obtained from an independently + verified IAM identity, authenticated server context, or trusted persisted + ownership record. +- global: only an explicitly platform-wide operation whose contract says it + is not tenant-owned. It is never a fallback for missing identity. +- unknown: no authoritative organization is available. Missing organization + data is represented as organization_id: null and tenant_scope: unknown. + +Context remains observability metadata, not tenant authority. No producer may +derive a tenant from URLs, headers that were not verified, usernames, resource +names, paths, workflow/run/trace IDs, scheduler handles, service names, or +environment variables. + +## Producer inventory and rollout + +| Producer | Authoritative source | SAT-2 status | Remaining gap | +|---|---|---|---| +| API Gateway | IAM-validated identity attached to request state | COMPLETE | Pre-auth and trace events remain UNKNOWN | +| TES | IAM-validated Identity.organization_id; persisted RunRecord ownership for timeout events | COMPLETE | No harmless live audited fixture | +| RAG | IAM-validated UserContext.org_id | COMPLETE | Pre-auth events remain UNKNOWN | +| Workflow Bundles | IAM-validated UserContext.org_id | COMPLETE | No harmless live audited fixture | +| LIMS | independently verified request IAM context via get_verified_org_id | COMPLETE | local-session/Django-admin and SSO paths remain UNKNOWN | +| Model Registry | no Security Audit stream emission path found | NOT_APPLICABLE | Integrate with Security Audit when an approved path exists | +| Security SDK | generic client exists but is not used by discovered Security Audit producers | NOT_APPLICABLE | Future contract hardening/adoption requires an owner | +| Auth-related ledger | separate IAM audit ledger, not Security Audit stream | NOT_APPLICABLE | Separate integration decision | +| Control Center | explicitly out of SAT-2 scope; no changes made | NOT_APPLICABLE | Review in its own authorized work | +| Workbench/local audit log | local application audit path, not a discovered Security Audit producer | NOT_APPLICABLE | Confirm during its own integration work | + +## Release closure matrix + +| Producer | Code | Tests | Live evidence | Release | +|---|---|---|---|---| +| Gateway | COMPLETE | PASS | CERTIFIED | READY | +| RAG | COMPLETE | PASS | CERTIFIED | READY | +| LIMS | COMPLETE | PASS | CERTIFIED | READY | +| TES | COMPLETE | PASS | FIXTURE LIMITED | REVIEWABLE | +| Workflow Bundles | COMPLETE | PASS | FIXTURE LIMITED | REVIEWABLE | +| Model Registry | N/A | N/A | N/A | OUT OF SCOPE | +| Security SDK | N/A | N/A | N/A | FUTURE | + +## Integrity and compatibility + +All changed producers place tenant fields in the event dictionary before +JSON serialization and HMAC signing. The exact serialized data string is +signed and published, so tenant fields are integrity-covered through Redis, +the consumer, and SQL persistence. Existing call sites can omit the optional +fields and safely produce null/unknown; no legacy event is rejected. + +No global event producer was added in SAT-2. Platform startup/configuration +events require an explicit contract decision before being classified global. + +## Live certification evidence + +The local deployment was inspected using its actual Compose service names. The +SAT-1 Security Audit API/worker was rebuilt from the merged SAT-1 source because +the previously running worker image loaded a pre-SAT-1 model; no database, +Redis, Auth, Control Center, or unrelated service was restarted. Only the five +SAT-2 producer services and then the Security Audit API/worker were recreated. + +The deployed database was verified read-only to contain `organization_id`, +non-null `tenant_scope` with the `unknown` default, migration `0003`, and +`ix_audit_events_org_timestamp_event`. + +With a disposable Auth-created organization (`589`) and a validated identity, +fresh durable rows with `integrity_status=valid` were observed for: + +- API Gateway: `iam_auth_success`, `request`, and `upstream_forward`, all + `organization_id=589`, `tenant_scope=organization`. +- RAG: a source-correct serving deployment durably persisted + `organization_id=590`, `tenant_scope=organization`, and + `integrity_status=valid`. The image copies `/app/ragbio` and installs the + checked-out project; certification must use that release image/source path, + not an ad hoc `PYTHONPATH` overlay. +- LIMS: `sample_exported`, `organization_id=589`, + `tenant_scope=organization`. + +TES list operations and Workflow Bundles list operations do not emit audited +events, and the live stores had no existing run/workflow fixture that could be +used for a harmless audited read. Their propagation is covered by focused +unit/contract tests, but not live durable certification in SAT-2.1. GLOBAL was +not exercised because no legitimate global producer event was found. An +unauthenticated Gateway request produced a valid `NULL/unknown` row. + +The Redis stream also showed tenant-bearing Gateway and LIMS payloads before +the rebuilt worker persisted them, confirming tenant fields were present in +the signed producer payload. No raw payloads, signatures, tokens, or keys were +exposed in the evidence collection. + +## Validation and follow-up + +Each changed producer has focused contract coverage for top-level tenant +fields, context non-authority, unknown defaults, and signing serialization. +Current focused results: Gateway 25 passed; TES signing 16 passed; RAG audit +9 passed; Workflow Bundles signing 15 passed; LIMS signed-audit 45 passed. +Gateway's stale pre-SAT-2 envelope assertion was updated to the merged +contract. TES route-isolation tests collected 15 environment-skipped cases. +Full TES/Workflow suites were not used as release gates because they enter +long-running backend/integration paths. Changed-file compilation and diff +checks pass when bytecode is redirected outside the read-only worktrees. +Ruff passes for RAG and LIMS; touched Gateway, TES, and Workflow files retain +pre-existing lint debt, with no broad cleanup performed. + +SAT-2.2 closed the RAG live defect in the certification deployment. The +source already passed the verified `UserContext.org_id` as the top-level +tenant field; the deployed `ragbio-server` had instead imported a stale +installed package because `/app` was absent from its serving-process import +path. Running the released package from `/app` produced and durably persisted +`organization_id=590`, `tenant_scope=organization`, and +`integrity_status=valid`. The production image/build must include the SAT-2 +source rather than rely on a source overlay. + +TES and Workflow Bundles remain live-unvalidated because their available list +operations emit no audit event and no harmless audited fixture was present. +This is a LIVE EVIDENCE LIMITATION, not an implementation blocker. SAT-3 may add +organization-scoped query authorization only after +the Security Audit contract and producer rollout are merged. SAT-4 freshness +and retention semantics remain separate work.