feat(memory-defense): detect international and Chinese PII

[oldnicke]

Fixes #3574.

Summary

• rebase the fix onto the latest upstream main
• add redaction patterns for email, Mainland China mobile numbers, international phone numbers, PRC identity numbers, and Mainland China bank cards
• add context-bound Chinese and English person-name and postal-address detection to limit false positives
• cover comma-delimited English and Chinese addresses without truncating persisted PII
• prevent invalid 16-digit 62 candidates from falling through to generic credit-card redaction
• validate PRC birth dates and checksums, validate bank cards with Luhn, and fully mask new PII hit previews
• synchronize all locale catalogs and generated coding-agent documentation

Testing

• uv run --no-sync pytest -p no:xdist -o addopts="" tests/test_memory_defense.py -q -k "redaction_samples or redaction_patterns or each_redaction or apply_redaction" (241 passed, 56 deselected)
• ./scripts/hooks/lint.sh
• pre-commit hooks
• node hindsight-docs/scripts/sync-coding-agents-doc.mjs --check

The DB-backed Memory Defense cases still require a local test database with the current entities.entity_kind migration.

[r266-tech]

Reviewed PR #3577 at head f9a5f89bd8ebe9627fb428fb478ef8538bd0f67a from snapshot snapshot_42730654edccdd2d9eaf5a3ca26f16e72; the supplied verification confirms checkout matches the authority-bound external PR head and git diff --check exited 0.

The PR description reports 233 focused Memory Defense tests passing plus lint and pre-commit checks; DB-backed Memory Defense cases were not run against a current test database.

High: The postal-address detector stops at the first comma/semicolon/sentence punctuation and caps the match at 120 characters, so Shipping address: 123 Main St, Apt 4, Springfield, IL 62704 is only partially redacted, leaving the apartment, city, state, and postal code in persisted content. Extend address matching to cover comma-delimited components while bounding unrelated prose, and add English and Chinese multi-component regression tests.

Medium: The contextual person-name and address alternatives have no leading label boundary and do not constrain the address value, so username: Alice can match the embedded name: substring and server address: localhost can be redacted as an address. Require a separator/word boundary before context labels, add stronger value/context constraints, and cover username, server address, and IP address with negative tests.

Medium: A 16-digit value prefixed with 62 that fails the new bank-card Luhn validator still falls through to the older generic credit_card pattern, which accepts 13-16 digits, so the invalid candidate is redacted anyway. Define precedence for invalid 62-prefixed 16-digit candidates and add a regression test for that shape.

Low: Only the English control-plane catalog was updated to report 52 Memory Defense patterns; the nine other actively loaded locale catalogs still report 45. Synchronize the 52-pattern text across all active locale catalogs or derive the displayed count from a single source.

[oldnicke]

@r266-tech Thanks for the detailed review. I addressed all of the requested changes in the latest commit (7ae13198b18fe491570dbfcd273ab91e1980bb4a).

Addressed items:

• Extended the postal-address detector to preserve comma-delimited English and Chinese address components while keeping length and statement-boundary limits.
• Added label boundaries and negative coverage for username: Alice, server address: localhost, and IP address: 192.0.2.1.
• Prevented invalid 16-digit 62-prefixed values from falling through to the generic credit_card detector.
• Updated all active locale catalogs from 45 to 52 Memory Defense patterns.
• Synchronized the generated coding-agent documentation.
• Applied _ascii_token_pattern to the locale-aware detectors where the generic ASCII boundary is safe. The email detector keeps a dedicated boundary expression because valid email local-parts may contain ., +, %, and related characters.

Validation:

• Memory Defense focused tests: 265 passed
• lint: passed
• pre-commit hooks: passed
• generated documentation check: passed
• all current CI checks: passed
• branch rebased onto the latest main

The DB-backed Memory Defense cases still require a current local test database and were not included in the focused local run. Could you please take another look and re-review the updated commit?

[oldnicke]

@r266-tech @nicoloboschi

Thanks for the review. I’ve addressed all of the requested changes in the latest commit. All tests, lint, pre-commit, documentation checks, and CI are passing, and the PR is mergeable.

Could you please re-review it when you have a chance? If everything looks good, would you mind merging it?

Thanks!