Add per-destination event filtering - #56
Merged
Merged
Conversation
Adds an optional filter: {column, values} block to destination configs. Records whose column value is not in values are skipped and committed instead of delivered, cutting receive-and-discard fan-out when many destinations share one topic.
Fail-open semantics: records missing the filter column, or whose value is not a string, are delivered — a config mistake degrades to the pre-filter behaviour, never to silent data loss. Destinations without a filter are untouched.
Includes projector/config/e2e test coverage, README config docs, and a CLAUDE.md with the dev-env setup and the verified macOS recipe for running the full DB-backed test suite locally.
rafaeelricco
approved these changes
Jul 16, 2026
Review feedback on PR #56: a fail-open (missing column, null or non-string value, non-object record) looked exactly like a normal match, so a typo'd filter column silently restored full delivery and the no-op filter could go unnoticed indefinitely. matchesFilter is now filterVerdict returning Matched / Skipped / FailedOpen with the reason. On the first FailedOpen a destination logs a warning naming the column and cause; subsequent fail-opens on that destination stay quiet so a permanently mismatched column cannot flood the log at full event rate (one warning per destination per emulator run). Delivery behaviour is unchanged — fail-open still delivers.
ctuncay
added a commit
that referenced
this pull request
Jul 16, 2026
Reverts #56 due to push back from Luis, will revisit the code change once Marcelo is back.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional per-destination
filterblock to the emulator config:Records whose
columnvalue is not invaluesare skipped and committed instead of delivered. Destinations without afilterare untouched.Motivation: consuming apps register many HTTP destinations over the same topic, and every destination receives every record just to discard the irrelevant ones. On Sumori's local stack (34 destinations), ~91% of order-event deliveries are receive-and-discard waste. Filtering at the emulator removes that fan-out; the consumer's own
accept()decoding remains the correctness gate — the filter is purely an optimization.Fail-open semantics: records missing the filter column, or whose value is not a string, are delivered. A config mistake degrades to the pre-filter behaviour, never to silent data loss. Empty
valueslists are rejected at config parse time.Compatibility — safe to release and roll out
filteris optional. Any existing config runs unchanged with identical delivery behaviour — the filtering code path is only reached when a destination declares a filter. Projects that don't use filtering are unaffected by upgrading the image.filterblocks still loads on an old image — it just delivers everything, which is the pre-filter behaviour. Config rollout and image rollout can therefore happen in either order.Testing
valuesrejection).CLAUDE.md).🤖 Generated with Claude Code