fix(vault): make a large OpenCreds import possible at all - #26
Merged
Conversation
Importing an OpenCreds database wrote nothing and looked like a crash: the window vanished and the vault stayed empty. Three separate defects, all on the path between choosing a file and seeing the items. The import ran in the action popup. A popup is dismissed as soon as it loses focus, and both halves of the flow took focus away -- the file chooser, and then `window.prompt` for the export passphrase. Chrome suppresses JS dialogs in a popup, so the prompt returned null and the handler took its `passphrase === null` cancel path, silently and with no message, if the popup had not already closed under the file chooser. Not one item was ever sent. The flow moves to the options page, an ordinary extension tab, which survives a file chooser and can ask for the passphrase in the page. Items were then written one POST at a time, sequentially, with the caller awaiting all of them. A few thousand keys is a few thousand round trips -- minutes during which the page that started it is gone and the result goes nowhere. Writes now run eight at a time, the call returns once the work is scheduled, and progress is polled. A worker restart still loses the job, so re-running the same file resumes instead: every item carries its own id, and an id already stored comes back 409, which is now counted as already-present rather than as a failure. Finally the list call fetched one page. The server has always capped a response at 1000 items and there was no way to ask for the rest, so any vault larger than that was listed short with no error -- the items were stored and never shown. GET accepts an offset, orders by (updated_at, id) so paging is total across rows written in the same instant, and reports `paged` so a client can tell a paging server from one that ignores the parameter. Verified against a real 3,122-item database: all 3,122 parse, encrypt, store and read back with every value byte-identical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q79wCDBvrPetsms1HCcnoX
ThreatCrush Security Scan11 finding(s) MEDIUM: 7 | LOW: 4
Snippets are redacted; ThreatCrush never prints matched credential material. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Importing an OpenCreds database into the vault wrote nothing and looked like a crash: the window vanished and the vault stayed empty.
vault_itemsin prod confirms it — zero rows for every user, so not one POST was ever sent.Three separate defects sit between choosing a file and seeing the items.
1. The import ran in the action popup, where it cannot work
A popup is dismissed as soon as it loses focus, and both halves of the flow took focus away: the file chooser, and then
window.promptfor the export passphrase. Chrome suppresses JS dialogs in an action popup, so the prompt returnednulland the handler took itspassphrase === nullcancel path — silently, with no message — if the popup had not already closed under the file chooser.Moved to the options page, an ordinary extension tab: it survives a file chooser, asks for the passphrase in the page, and stays open long enough to show progress. The popup's Import control now opens it. No
window.prompt/window.confirmon this path.2. Writes were sequential, and the caller awaited all of them
The API creates one item per request, so a few thousand keys is a few thousand round trips — minutes, during which the page that started it is long gone and the result goes nowhere.
Writes now run 8 at a time; the call returns once the work is scheduled and progress is polled, so closing the page no longer abandons the import. A service-worker restart still loses the job, so re-running the same file resumes instead: every item carries its own id, an id already stored comes back 409, and that is counted as already-present rather than as a failure. Re-running the same file is therefore safe and does not duplicate.
Item data is deliberately kept out of any persisted progress record — nothing that would put plaintext secrets somewhere to survive a restart.
3. The list call fetched one page
The server has always capped a response at
MAX_PAGE_SIZE(1000) and there was no way to ask for the rest, so any vault larger than that was listed short with no error anywhere — the items were stored and simply never shown.GET /api/vault/itemsacceptsoffset, orders by(updated_at, id)so paging is total across the thousands of rows an import writes in the same instant, and reportspaged: true. The client pages until a short response. The flag matters: without it a client cannot tell a paging server from one that ignores the parameter and would loop on the same page forever — so this is safe to ship before the web deploy.Verification
apps/web/__tests__/extension-auth.test.jsare pre-existing onmasterand unrelated.offset.Note
The extension change works against the current production API; a vault larger than 1000 items will simply list the first 1000 until
apps/webis deployed. Nothing is lost in the meantime — the items are stored and appear once it ships.🤖 Generated with Claude Code
https://claude.ai/code/session_01Q79wCDBvrPetsms1HCcnoX