Repository navigation
feat: value-level PII detection for unclassified fields (STABLE360-13580) - #6
Merged
Merged
Conversation
…580) The catch-all scrub of kept string values now also detects: - PII inside URLs: components are decoded and scrubbed; map links (Google, Apple, Bing, OpenStreetMap, short links) lose the place name, place ids and Google's data= blob, and coordinates are rounded to 2 decimals (~1 km). Covers custom attributes like opportunity link_gebaeudeansicht. - German street addresses (street word + 1-3 digit house number); postal code and city stay visible, as on address items. - IBANs are now mod-97 checked (shorter valid prefixes tried), cutting false positives on IBAN-shaped codes. Classification fixes: - internal_user attributes use the user strategy. - curated _title defaults for billing_account, calendar_event, contract, lead, request and ticket. - data_classification 'pii' honours curated / type strategies (a pii first_name stays a person pseudonym) and is never kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Field-name and type rules cannot see personal data inside values of unclassified custom attributes, e.g. opportunity
link_gebaeudeansichtholding a Google Maps link with the customer's street, house number and coordinates (found while verifying STABLE360-13580). The catch-all scrub of kept strings (scrubText) now also detects, replacing only the matched span:maps.google.*, Apple, Bing, OpenStreetMap,goo.gl/maps/maps.app.goo.gl): place name / address query →street_xxxxpseudonym + postal code and city; place ids (cid,ftid,place_id, short-link ids) pseudonymized; Google'sdata=segment (feature / knowledge-graph ids, precise pins) dropped; coordinates rounded to 2 decimals (~1 km)./030-1234567is a slug far more often), but they are in query values.straße/strasse/str./weg/platz/allee/gasse/ring/damm(compound or standalone after up to two capitalized words) + 1-3 digit house number (12a,12-14,10/1). Postal code and city stay, consistent with address items.DE89 … 00 BICstill masks the IBAN).@,/or a digit skip all detectors; street matching is behind a cheap suffix+digit prefilter.data_classification: 'public'and system keys stay untouched, as before.Classification fixes:
internal_user→userstrategy (templatecreated_by/updated_byembed user objects)._titledefaults (text) forbilling_account(= contact name),calendar_event,contract,lead,request,ticket.data_classification: 'pii'now honours curated and type strategies (piifirst_name→person,salutation→redact) instead of falling back tovalue, and apiiattribute is never kept via akeepoverride. This unblocks marking namespiiin the entity-api blueprints (entity-api MR !1733).False-positive tradeoffs
Arbeitsplatz 2,Lösungsweg 3→street_…). Years don't match (house numbers ≤ 3 digits).An der Kirche 3,Platz der Republik 1) and names in free text are still not detected.Test plan
npm test: 99 passed (53 new invalue-detection.test.ts: reallink_gebaeudeansichtvalue on an entity and in an activity payload/operations, all map providers, encoded URLs, streets, IBAN checksum, idempotency, false positives incl. product codes, OBIS, dates, prices, UUIDs, plain URLs)npm run lint(tsc) andnpm run build🤖 Generated with Claude Code