Skip to content

fix(data): align PDF DOM upload budget with sidecar - #1427

Open
seonghobae wants to merge 8 commits into
developfrom
codex/pdf-dom-upload-64m
Open

fix(data): align PDF DOM upload budget with sidecar#1427
seonghobae wants to merge 8 commits into
developfrom
codex/pdf-dom-upload-64m

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Customer outcome

Manual PDF DOM uploads in the Data workspace now accept the same bounded 64 MiB payload as Naruon's signed email-import path and the NewsDOM sidecar. A larger request is rejected with 413 before storage or recognition work.

Root cause

The direct Data upload and pending-payload decoder retained an independent 20 MiB guard after the cross-service transport contract moved to 64 MiB.

Evidence

  • python -m pytest -q backend/tests/test_data_api.py -k 'pdf_dom_upload or pending_pdf_document_decoder' — 4 passed.
  • ruff check backend/api/data.py backend/tests/test_data_api.py — passed.
  • git diff --check — passed.
  • ADR-0005 and the doctoring record document the local consumer contract and APA 7th references.

Safety

Signed-session authorization, PDF signature validation, bounded read, base64 persistence, worker deferral, and fail-closed malformed-payload handling remain unchanged. The matching NewsDOM sidecar contract is delivered separately in ContextualWisdomLab/newsdom-api#665.

No customer or private reference data was read or committed. This is a backend contract change; no Figma design file is required.


Open in Devin Review

@coderabbitai

coderabbitai Bot commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Important

Approval pending

CodeRabbit has no unresolved comments, but it has not reviewed the latest commit.

Use the checkbox below to review the latest commit. CodeRabbit will approve the changes if it finds no blocking issues.

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae enabled auto-merge (squash) August 20, 2026 11:35

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review Please independently review exact current head 795bfd2c05d6003aacedad043105a15f194687af against base 81c105645ca6e680f5f8c15ba9c33b67eb63c48b. Verify the 64 MiB PDF DOM upload contract, attachment transport boundaries above 20 MiB, parser/image-prefix limits, and current Checks. Publish structured exact-head evidence only; do not reuse predecessor review or coverage state.

@seonghobae

seonghobae commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

Review finding withdrawn

Withdrawn after exact-base recheck. The earlier P1 relied on stale local evidence that treated protected develop@81c105645ca6e680f5f8c15ba9c33b67eb63c48b as a 20 MiB email-import base. The live protected blob at that exact SHA now contains MAX_IMPORT_UPLOAD_BYTES = 64 * 1024 * 1024 in backend/services/email_import_service.py:62, and the API imports that constant. The original finding is therefore not a source defect in PR #1427. It must not be used to block merge.

@seonghobae
seonghobae disabled auto-merge August 21, 2026 12:31
@seonghobae

seonghobae commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up withdrawn

The previous dependency disposition is withdrawn for the same reason: exact protected develop@81c105645ca6e680f5f8c15ba9c33b67eb63c48b already contains the 64 MiB email-import transport ceiling. PR #1353 remains independently relevant to HWP/HWPX parser recognition, but it is not required to establish the PDF/email transport ceiling claimed by PR #1427. Revalidation must use the exact live base blob, not the stale local checkout.

@seonghobae

Copy link
Copy Markdown
Contributor Author

The new docs/adr/0005-bounded-pdf-dom-upload-contract.md has no Figma File ID field. This is a backend/data-contract change with no new visual surface, so record Figma File ID: N/A — signed data upload contract; no visual surface near the ADR metadata. Keep the field explicit for traceability even when Figma is not applicable. This is documentation traceability; no force/bypass is eligible.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review

Current-head review request after the ADR traceability stack merged.

Exact head: 29be15e
Exact base: 81c1056

Read the complete current PDF DOM upload diff including ADR-0005 with its Figma File ID field. Verify 64 MiB alignment with the NewsDOM sidecar, signed-session/auth-before-data handling, PDF signature, first over-limit byte accounting, base64 persistence, pending decode, quota/cleanup, tests, and rollback implications. Produce a substantive current-head verdict with path:line evidence; do not reuse predecessor evidence.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head review finding: the live consumer dependency is not aligned as described. Exact Naruon head 29be15e sets _MAX_PDF_DOM_UPLOAD_BYTES to 64 MiB, but ContextualWisdomLab/newsdom-api develop currently defines MAX_PARSE_UPLOAD_BYTES = 20 * 1024 * 1024 in src/newsdom_api/main.py:44 and tests still assert the 20 MiB boundary. This is a substantive cross-repository contract gap, not a review or runner deadlock. I am opening the owning sidecar PR first; keep this PR blocked until the exact sidecar contract is merged and revalidated.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Owning sidecar contract PR opened: ContextualWisdomLab/newsdom-api#682. Its exact head 71671c6 raises MAX_PARSE_UPLOAD_BYTES from the live 20 MiB to 64 MiB with a regression test, ADR/doctoring record, and 100% local coverage. Naruon#1427 remains stacked logically on that merge; after #682 normal merge, revalidate #1427 against the new sidecar contract and current Naruon head.

@opencode-agent
opencode-agent Bot disabled auto-merge August 21, 2026 14:47
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

PR governance metadata gate is not ready for 37ab7bd5c1a2dbc3a52f360ef791e476dccd861a:

  • Current-head CodeRabbit issue comment has blocking warning/failure evidence on 37ab7bd.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Exact-head disposition for 29be15e4ec5e29dc1f62ac636928c9307a6f520f against develop@81c105645ca6e680f5f8c15ba9c33b67eb63c48b: the hosted Strix run 32487496869 / job 96787228260 failed before producing a vulnerability report with loginAsGuest failed after 10 attempts because the Strix Caido service could not connect to 127.0.0.1:48080. This is incomplete scanner infrastructure evidence, not an application source finding; the run reported zero vulnerabilities before the bootstrap failure. Central owner PR ContextualWisdomLab/.github#1153 contains the typed fail-closed classifier and exact regression fixture for this signal. Do not downgrade or bypass this required check. After #1153 is merged normally, rerun Strix for this exact Naruon head; retain the separate sidecar contract dependency on newsdom-api#682 before merging this 64 MiB upload contract.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@coderabbitai review\n\nPlease review only exact current head 29be15e; do not reuse predecessor-head evidence. The current metadata gate is stale until this exact head receives a fresh review.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head Checks RCA: the historical failure for this exact head was not a source finding. The run log shows Strix Caido bootstrap failing after 10 attempts because was unavailable; no scan report was produced and the gate failed closed. A normal was attempted, but GitHub returned 404 because the owning workflow is no longer available. The metadata-only gate therefore remains stale/infrastructure-blocked; no test or security check was weakened.

@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "Local checkout:"
git rev-parse HEAD

echo "GitHub PR `#1427` head:"
gh pr view 1427 --repo ContextualWisdomLab/naruon --json headRefOid --jq '.headRefOid'

Length of output: 448


@seonghobae: Verified. The local checkout and GitHub PR #1427 head both match 29be15e4ec5e29dc1f62ac636928c9307a6f520f. I will perform a fresh review for this exact head only.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Correction to the preceding audit comment: the historical strix failure for this exact head was not a source finding. The run log shows the Strix Caido bootstrap loginAsGuest failed after ten attempts because localhost port 48080 was unavailable; no scan report was produced and the gate failed closed. A normal Actions rerun was attempted, but GitHub returned 404 because the owning workflow is no longer available. The metadata-only gate therefore remains stale/infrastructure-blocked; no test or security check was weakened.

@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent @cwl-noema-review review only exact current head 29be15e4ec5e29dc1f62ac636928c9307a6f520f of PR #1427 against base develop@81c105645ca6e680f5f8c15ba9c33b67eb63c48b. Check the PDF DOM upload budget contract and full diff, then publish structured exact-head verdicts and current-head check binding. Do not merge, update branch, dismiss reviews, bypass protection, or reuse predecessor evidence.

@opencode-agent opencode-agent Bot added priority: medium Normal-priority or P2 work scope: product-gap Customer-visible product gap status: needs-review Open pull request requiring current-head review or checks labels Aug 22, 2026
@opencode-agent opencode-agent Bot added the type: bug Defect or incorrect behavior label Aug 22, 2026
@coderabbitai

coderabbitai Bot commented Aug 22, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

An error occurred during the review process. Please try again later.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae enabled auto-merge (squash) August 24, 2026 00:27
@seonghobae

Copy link
Copy Markdown
Contributor Author

Please independently review the complete exact current head 29be15e against protected develop at 81c1056. Re-read the full diff and current Checks, distinguish resolved predecessor findings from current source defects, and publish a current-head substantive verdict.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head restack (37ab7bd5c1a2dbc3a52f360ef791e476dccd861a) is now based on develop@e5e99b4e3bb081b92c602358878856536030e2ca and includes the independent #1468 PostgreSQL smoke fixture root fix as a stack predecessor. The merge-result tree passed PYTHONWARNINGS=error python -m pytest backend/tests -q (1834 passed, 2 skipped), the PDF/NewsDOM and email-import focus (65 passed), Ruff, and git diff --check. Hosted Checks and a qualifying current-head approval remain authoritative.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Open in Devin Review

Comment thread backend/scripts/bootstrap_db.py
Comment thread backend/api/data.py
@seonghobae

Copy link
Copy Markdown
Contributor Author

Current-head review disposition for 37ab7bd: both Devin threads are informational, not change requests. The legacy-index removal is required for the current schema, and the 64 MiB parser/base64 storage capacity concern is already recorded as the bounded-contract object-storage gap in ADR-0005 and the product baseline. No source change is required for either thread; hosted Checks and independent review remain the merge gates.

@seonghobae
seonghobae enabled auto-merge (squash) August 25, 2026 14:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority: medium Normal-priority or P2 work scope: product-gap Customer-visible product gap status: needs-review Open pull request requiring current-head review or checks type: bug Defect or incorrect behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant