Lock in the duplicate (id, ts) guard (#304) - #574
Merged
Merged
Conversation
#304 is a different bug from #294 despite the comment there: not two statements in one query, but two rows with the same (id, ts) in one INSERT - the same datapoint logged from two sources that landed on the same millisecond - which aborted the whole batch with a primary key violation. It is already fixed. MySQL carries `ON DUPLICATE KEY UPDATE id=id`, PostgreSQL and SQLite `ON CONFLICT DO NOTHING`. MS SQL has nothing, and that is correct: init() creates a plain `CREATE INDEX i_id on ...(id, ts)` rather than a primary key, so no uniqueness conflict can arise there. Nothing tested any of it. test/testInsertStatements.js now asserts the guard per dialect, including the deliberate absence for MS SQL, so that someone levelling the dialects does not add one there. It also pins DO NOTHING against DO UPDATE: the duplicate sits *inside* one statement, which DO NOTHING handles but DO UPDATE answers with "cannot affect row a second time" - the obvious way to make the last value win would bring the bug straight back. Since the issue was reported on MariaDB, that dialect gets an executable check rather than a string match. test/testMySQLInsertDuplicates.js talks to the server directly - no js-controller, no adapter instance - creates its own throwaway database, and covers three things: the reported batch inserts cleanly, the same batch without the guard still fails with ER_DUP_ENTRY '4-1681102802872', and a genuinely broken statement still raises ER_BAD_FIELD_ERROR. That last one is why the guard is not INSERT IGNORE, as proposed in #407: IGNORE would swallow it. It is picked up by the MySQL job's existing `test/testMySQL*.js` glob and sorts after testMySQLExistingNoNulls.js, so the documented order of the existing-data tests is unaffected. Without a reachable server the cases skip with the reason, so the glob stays usable locally. Verified against MariaDB 10.11.14: 3 passing with a server, 3 pending without. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Oct 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #573. #304 is already fixed too — this adds the tests that were missing, and the executable proof on the engine it was reported against.
#304 is not #294
The comment on #294 says they are the same report. They are not:
INSERTin one query(id, ts)in one INSERTIn #304 the same datapoint is logged from two sources that land on the same millisecond —
ts 1681102802872appears twice for id 4, once from source 2 and once from source 3.Current state
ON DUPLICATE KEY UPDATE id=idON CONFLICT DO NOTHINGON CONFLICT DO NOTHINGThe MS SQL exception is not an oversight:
init()createsCREATE INDEX i_id on ...(id, ts), a plain non-unique index rather than a primary key, so no uniqueness conflict can arise. Nothing tested any of this.What the tests pin
test/testInsertStatements.js:DO NOTHINGagainstDO UPDATE. The duplicate sits inside one statement.DO NOTHINGcopes;DO UPDATEanswers with "cannot affect row a second time". Making the last value win is the obvious improvement here, and it would bring the bug straight back;PRIMARY KEY(id, ts).Executable proof on MariaDB
Since #304 was reported on MariaDB, that dialect gets a real check rather than a string match.
test/testMySQLInsertDuplicates.jstalks to the server directly — no js-controller, no adapter instance, about a second — in its own throwaway database:ER_DUP_ENTRY: Duplicate entry '4-1681102802872' for key 'PRIMARY'— the exact error from the issue;ER_BAD_FIELD_ERROR.Case 3 is the answer to #407, which proposes
INSERT IGNORE: that would swallow it. The code already says so in a comment; now a test holds it.Verified against MariaDB 10.11.14 — 3 passing with a server, 3 pending without, so
npx mocha 'test/testMySQL*.js'stays usable on a machine with none.Wiring
No workflow change needed: the MySQL job's existing
test/testMySQL*.jsglob picks it up, and it sorts aftertestMySQLExistingNoNulls.js, so the documented ordering of the existing-data tests is untouched. It requires onlybuild/lib/mysql, neverbuild/main.js— the constraint from #567.184 unit tests pass,
prettier --checkclean.#294, #304 and #407 can all be closed.
🤖 Generated with Claude Code