fix(paperless-ngx): widen email cron to */10 to reduce double-ingest#39
Open
ClaydeCode wants to merge 1 commit into
Open
fix(paperless-ngx): widen email cron to */10 to reduce double-ingest#39ClaydeCode wants to merge 1 commit into
ClaydeCode wants to merge 1 commit into
Conversation
The every-minute PAPERLESS_EMAIL_TASK_CRON (`* * * * *`) races with itself: Paperless's mail-check task has no overlap guard, and it writes the message to the ProcessedMail dedup table only after the attachment is consumed (post-OCR). So a second mail-check fired by the next cron tick can pass the dedup `.exists()` check and consume the same attachment again before the first pass has recorded it. The first import succeeds; the second is rejected by Paperless's document-hash duplicate detection and surfaces as a failure alert users find confusing. Widen the interval to `*/10` — Paperless upstream's default. Because the dedup record is written only after OCR completes, the race window can span minutes; 10 minutes gives a comfortable margin over a single mail-fetch+OCR pass while email ingestion stays acceptably responsive. This mitigates, not eliminates, the race: a genuine fix would need an upstream overlap lock, and the user-configured multi-rule case (two mail rules matching one message) is out of the template's control. Fixes #30 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fixes #30.
On Paperless NGX, an email attachment gets ingested twice. The first ingest succeeds; the second always fails and raises a failure alert in the Paperless UI — the document actually imported fine, so users are confused by an error that means nothing.
Root cause
apps/paperless-ngx/docker-compose.yml.templateoverrode the mail-check cron to every minute:Paperless's mail-check task has no overlap guard, and it writes the message to its
ProcessedMaildedup table only after the attachment is consumed (post-OCR). So a mail-check fired by the next cron tick can pass the dedup.exists()check and consume the same attachment again before the first pass recorded it. First import succeeds; the second is rejected by document-hash duplicate detection → the failure alert.Change
One line: widen the interval to
*/10 * * * *— Paperless upstream's default.Because the dedup record lands only after OCR completes, the race window can span minutes; 10 minutes gives a comfortable margin over a single fetch+OCR pass while email ingestion stays acceptably responsive. The prior every-minute value was chosen for responsiveness (per git history) but is the aggressive knob that triggers the race.
Scope note: this mitigates, not eliminates. A genuine fix needs an upstream overlap lock; and the user-configured multi-rule case (two mail rules matching one message, which upstream treats as not-a-bug) is out of the template's control.
Verification
python3 build_store_data.pyexits 0 (generated zip/metadata are untracked; only the template changed).Review panel
Adversarial reviewer (always-on) — no blocking findings. Advisories, all addressed:
*/5reduces but doesn't eliminate the race; dedup is written post-OCR so the window is minutes-long — prefer upstream default*/10for 2x margin." → Fixed: adopted*/10instead of the originally-committed*/5.No other specialist (test / DevEx / security / DB / API / UX) was dispatched — a one-line config value change touches none of those lenses.
🤖 Generated with Claude Code