Skip to content
This repository was archived by the owner on May 13, 2026. It is now read-only.

feat(ingestion): surface schema-service 409 as typed conflict, file dev cleanup tooling - #1029

Merged
shiba4life merged 1 commit into
mainfrom
fix/schema-service-duplicate-descriptive-names
May 13, 2026
Merged

feat(ingestion): surface schema-service 409 as typed conflict, file dev cleanup tooling#1029
shiba4life merged 1 commit into
mainfrom
fix/schema-service-duplicate-descriptive-names

Conversation

@shiba4life

Copy link
Copy Markdown
Collaborator

Summary

Companion to schema_service PR #145 (https://github.com/EdgeVector/schema_service/pull/145), which stops two Approved schemas from ever sharing a descriptive_name by returning 409 with a typed DescriptiveNameConflict body instead of silently growing a duplicate canonical.

This fold_db_node-side PR makes that 409 reach the user cleanly:

  • New IngestionError::SchemaDescriptiveNameConflict { descriptive_name, existing_canonical, reason } variant, with a user_message() arm that says:

    "There's already a schema called '' on this node. Rename your new schema or reuse the existing one — two schemas can't share a descriptive name."

  • schema_creation::create_new_schema_with_node detects the conflict via the canonical message sentinel emitted by schema_service_client::add_schema and converts to the typed variant before propagating.
  • Sentinel-based detection is deliberate: the fold_db_node pinned rev of schema_service doesn't yet have the new 409 message text — the bump-cascade bot lands it on its 2h schedule. Until then, this code path is dormant and the existing SchemaCreationError wrapping is unchanged.
  • Once the cascade lands schema_service main, the typed AddSchemaOutcome::Conflict enum becomes available in the client; a follow-up PR will swap to typed detection.

Investigation findings (Phase 1 of kanban #49603)

Live dev snapshot of https://y0q3m6vk75.execute-api.us-west-2.amazonaws.com/v1/schemas/available (940 schemas, 2026-05-13) found 11 descriptive_name collision groups including Photography x3, Journal Entries x3, Vehicle x2 and 8 more — all active, none superseded_by. Root cause: state::add_schema had three sites (state.rs:1144 / 1230 / 1300 pre-patch) that detected a descriptive_name match, ran is_cross_schema_type_expansion, and silently fell through to persist a fresh canonical when the schema_types didn't match. PR #1028's 409-at-the-query-boundary papered over the symptom; schema_service PR #145 fixes the cause.

Dev cleanup

schema_service PR #145 also ships a new admin endpoint POST /v1/admin/dedupe-descriptive-names that walks the registry, keeps the schema with the largest field set as the survivor per descriptive_name group, and marks the rest superseded_by = survivor. Idempotent. Data preserved.

Operator runbook (run once on dev after schema_service PR #145 deploys):

SCHEMA_URL=$(./scripts/get-env-url.sh dev schema_service)
curl -sf -X POST "$SCHEMA_URL/v1/admin/dedupe-descriptive-names" | jq
curl -sf "$SCHEMA_URL/v1/schemas/available" | jq '[.schemas[] | .descriptive_name] | group_by(.) | map(select(length>1)) | length'
# expect: 0

Out of scope

  • Prod cleanup. Filed as a separate kanban task — needs a deliberate data-preservation plan that prod's live writes deserve.
  • Concurrent-Lambda race. Two Lambdas missing each other's writes can still produce a duplicate. Needs DynamoDB conditional writes / per-name lease — out of scope here; will file as its own kanban.
  • Typed-client wiring. Switches from sentinel string detection to AddSchemaOutcome::Conflict once the bump cascade lands the new schema_service rev.

Test plan

  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo fmt --all --check
  • cargo test --lib ingestion::error (3 new tests green: parse_schema_name_conflict_recognises_the_canonical_message, parse_schema_name_conflict_ignores_unrelated_errors, schema_descriptive_name_conflict_user_message_is_actionable)
  • After schema_service PR fix: poll_and_decrypt uses correct pseudonyms for message decryption #145 deploys + bump cascade lands: operator hits POST /v1/admin/dedupe-descriptive-names on dev once, confirms 0 collisions remain.
  • End-to-end: register a "Contacts" schema once → 200. Register a second "Contacts" with a different schema_type → 409 from schema_service surfaces as IngestionError::SchemaDescriptiveNameConflict with the rename prompt copy.

🤖 Generated with Claude Code

When the schema service refuses a proposed schema because its
`descriptive_name` already maps to a different Approved canonical (the
new 409 introduced upstream in schema_service PR #145 — kanban #49603),
the message needs to reach the user as "There's already a schema called
'Photography' on this node. Rename your new schema or reuse the existing
one." — not as "Schema creation error: Failed to create schema via
schema service: schema service refused duplicate descriptive_name …"
wrapped twice.

Adds `IngestionError::SchemaDescriptiveNameConflict { descriptive_name,
existing_canonical, reason }` with a `user_message()` arm that produces
the rename / reuse copy. `schema_creation::create_new_schema_with_node`
detects the conflict via the message sentinel emitted by
`schema_service_client::add_schema` and converts it to the typed variant
before propagating.

The detection is string-based on purpose: the fold_db_node pinned rev
of schema_service doesn't have the new error message yet (the bump
cascade lands it on its 2h schedule). On the current rev, the detection
is a no-op — the legacy "Schema service returned unexpected CONFLICT
(409)" message doesn't match — and the existing
`IngestionError::SchemaCreationError` path stays. Once the cascade
bumps the rev, the new message text starts arriving and the typed
variant fires automatically. A follow-up PR will swap to the typed
`AddSchemaOutcome::Conflict` enum once that's available.

Tests:
- `parse_schema_name_conflict_recognises_the_canonical_message` pins the
  sentinel + parsing against the exact message shape `add_schema`
  produces.
- `parse_schema_name_conflict_ignores_unrelated_errors` keeps transport
  / 5xx errors flowing through the existing path.
- `schema_descriptive_name_conflict_user_message_is_actionable` pins
  the UI-facing copy.

Filed kanban follow-ups: prod-cleanup (separate scope; needs deliberate
data preservation plan) and concurrent-Lambda race prevention (needs
backend-level atomicity — DynamoDB conditional writes / per-name lease).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@shiba4life
shiba4life enabled auto-merge May 13, 2026 03:59
@shiba4life
shiba4life added this pull request to the merge queue May 13, 2026
Merged via the queue into main with commit 849d19c May 13, 2026
12 checks passed
@shiba4life
shiba4life deleted the fix/schema-service-duplicate-descriptive-names branch May 13, 2026 04:09
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant