Repository navigation
fix: close PII gaps in anonymized responses (STABLE360-13580) + perf for large responses - #5
Merged
Merged
Conversation
- activity caller: apply embedded-user rules at every depth (EpilotAuth email, username claims, token) - activities stored without a caller (e.g. message API thread activities) are recognized and scrubbed - catch-all: emails/phones/IBANs in strings outside recognized shapes are scrubbed - relation_address attributes are anonymized as addresses ($relation_ref kept) - entity fields not in the schema (journey_data) get the non-schema field heuristics - message cc/bcc/reply_to names pseudonymized; message/thread _title redacted - file filename/_title pseudonymized Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CRtqyNKiBCCdu3JSwr2zGp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes PII that reached anonymized callers (MCP read profile, CLI/SDK with
anonymizetokens) through entity-api responses — STABLE360-13580. Also makesanonymizeResponse2.7–5.6× faster on large responses.PII fixes
caller.EpilotAuth(email,cognito:username, claims)calleris the request security context, i.e. nesteduserstrategy) at every depth; addusername/cognito:username/preferred_usernamegetThreadTimelineactor.emailPOST /v1/activitywithoutcaller→ notisActivityLike→ payload walked unscrubbedcallerno longer required (payloadsuffices); plus catch-all: emails/phones/IBANs in strings outside recognized shapes are scrubbed (not inside entities, sodata_classification: 'public'fields stay untouched)delivery_address/billing_addressrelation_addressnot classifiedrelation_address: 'address';$relation_refkept in address itemsjourney_datanames/streetanonymizeUnknownfield heuristics to non-schema object fields (only when schema is known)_title(= subject)message:_title,thread:_title→ redact (also coversthread.latest_message)cc[].nameMESSAGE_PARTY_KEYSwasfrom/tocc,bcc,reply_tofilenamefile:filename,file:_title→valuepseudonym (same input → same pseudonym; shape already recognized by epilot-mcp's mask detector)City / postal code stay visible by existing policy (
ADDRESS_KEEP_KEYS).Performance
Profiled on 0.2.3: ~30% in
mapEntries+ GC (entries/fromEntries arrays per node), ~20% inscrubText(4 regex passes on every string), 13% inloadSchemas(anawaitper node), ~12% re-classifying every attribute of every entity.loadSchemas: one synchronous pass collects distinct (org, slug), resolved concurrently once eachmapEntriesas a plain loop; walker / leaf mappers return unchanged subtrees as-is instead of copying (result may share objects with the input — read-only; input still never mutated)Setlookups for key listsscrubTextprefilter:@for emails, an exact-superset guard for the IBAN/phone patterns (same lookbehinds) — timestamps, numbers, labels skip all regexes. Fuzz test (20k strings) asserts the guard never skips a matchanonymizeUnknownmemoizes per-key rulesOutput is byte-identical to the unoptimized build of this branch on all three workloads.
Release as 0.3.0 (curated defaults changed); entity-api bump: https://gitlab.com/e-pilot/product/entities/entity-api/-/merge_requests/1830 (
^0.2.2won't pick it up).Test plan
npm test— 45 passed (stable360-13580.test.ts: ticket cases + public fields untouched;performance.test.ts: prefilter fuzz, schema resolved once, input not mutated)tsc --noEmitsrc/anonymizationtests (20) pass against this build; new ticket tests fail on 0.2.2🤖 Generated with Claude Code
https://claude.ai/code/session_01CRtqyNKiBCCdu3JSwr2zGp