Skip to content

fix(Database): Use real idle-timer to prevent lastInsertId being reset on MariaDB/MySQL - #62697

Merged
DerDreschner merged 1 commit into
masterfrom
fix/db-connectivity-check-idle-timer
Jul 30, 2026
Merged

fix(Database): Use real idle-timer to prevent lastInsertId being reset on MariaDB/MySQL#62697
DerDreschner merged 1 commit into
masterfrom
fix/db-connectivity-check-idle-timer

Conversation

@DerDreschner

@DerDreschner DerDreschner commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

The current implementation of the idle timer is based on a strict 30 second interval. This generates issues with the getLastInsertId() on MariaDB and MySQL database back ends, as the timer fires a dummy SELECT statement to keep the connection open. The documentation of MariaDB clearly states:

If the last query wasn't an INSERT or UPDATE statement or if the modified table does not have a column with the AUTO_INCREMENT attribute and LAST_INSERT_ID was not used, this function will return zero.

Source: https://mariadb.com/docs/connectors/mariadb-connector-c/api-functions/mysql_insert_id

In case the idle timer runs between a INSERT or UPDATE statement and a getLastInsertId() call (mostly occ commands or CI runs), the return value will be reset to 0 which then messes with the further code paths. The resulting errors are diverse and hard to trace back to this issue.

To illustrate how diverse the resulting errors are, I let Claude identify some recent failed CI runs that are affected by this mechanism:

Run Date Failing test Observed failure Where the id comes from Confidence
30460658032 2026‑07‑29 Test\Files\Cache\ScannerTest::testFolder Failed asserting that 0 matches expected 705. — children written with parent = 0, DB row had the real id Cache::insert()getLastInsertId(), threaded as $parentId Proven (PR touched only MemcachedFactory; sibling matrix job green; deterministic repro exists)
30526516607 2026‑07‑30 Test\Share20\DefaultShareProviderTest::testDeleteSingleShareLazy ShareNotFound from getShareById() (DefaultShareProvider.php:785) DefaultShareProvider::create()getLastInsertId() Mechanism‑verified
30469262511 2026‑07‑29 Test\HelperStorageTest::testGetStorageInfoExcludingExtStorage Failed asserting that 0 matches expected 5. — root folder size never written scanner root id → Scanner::scanChildren() persists size via Cache::update($folderId, …); id 0 → WHERE fileid = 0 no‑op, size stays -1 Mechanism‑verified
30330027896 2026‑07‑28 Test\AppFramework\Db\QBMapperDBTest::testUpdateDateTime + DefaultShareProviderTest::testGetAccessListCurrentAccessRequired update‑by‑id without effect / wrong access‑list size QBMapper::insert() sets entity id from getLastInsertId() Mechanism‑verified (dependabot branch — unrelated code)
30489961329 2026‑07‑29 Test\Files\ObjectStore\ObjectStoreStorageTest::testMove file content reads back false objects addressed as urn:oid:{fileId} (ObjectStoreStorage.php) Mechanism‑verified
30351634642 2026‑07‑28 OCA\ShareByMail\Tests\ShareByMailProviderTest::testCreateMailShare / ::testAddShareToDB one‑off single‑test failures ShareByMailProvidergetLastInsertId() Mechanism‑verified
30378281124 2026‑07‑28 OCA\Federation\Tests\DbHandlerTest::testAddServer id‑based assertion mismatch DbHandler::addServer()getLastInsertId() Mechanism‑verified
#54890 open issue UserGlobalStoragesServiceTest::testGetUniqueStorages Failed asserting that an array has the key 0. — the corrupted id is in the message DBConfigService.phpreturn $query->getLastInsertId(); Mechanism‑verified

A possible next step is to remove the idle timer entirely and react on the ConnectionLost exception instead (see doctrine/dbal#6903) or use a 3rd-party library like https://github.com/facile-it/doctrine-mysql-come-back. Such an approach has additional benefits, as it also works for lost connections during running queries, which might be the reason why the DBAL developers dropped the Connection::ping() method in DBAL 3 (which was affected by this bug as well, fun fact).

To make the fix easy to backport, I've focused on a in-place fix for now.

Checklist

AI (if applicable)

  • The content of this PR was partly or fully generated using AI

@DerDreschner DerDreschner self-assigned this Jul 30, 2026
@DerDreschner
DerDreschner requested a review from a team as a code owner July 30, 2026 13:50
@DerDreschner
DerDreschner requested review from salmart-dev and removed request for a team July 30, 2026 13:50
@DerDreschner DerDreschner added 3. to review Waiting for reviews tests Related to tests CI feature: database Database related DB AI assisted labels Jul 30, 2026
@DerDreschner
DerDreschner force-pushed the fix/db-connectivity-check-idle-timer branch from 25918f0 to cc69c33 Compare July 30, 2026 13:53
@DerDreschner

Copy link
Copy Markdown
Contributor Author

/backport to stable34

@DerDreschner

Copy link
Copy Markdown
Contributor Author

/backport to stable33

@DerDreschner

Copy link
Copy Markdown
Contributor Author

/backport to stable32

@DerDreschner

Copy link
Copy Markdown
Contributor Author

/backport to stable31

@DerDreschner

Copy link
Copy Markdown
Contributor Author

/backport to stable30

@DerDreschner

Copy link
Copy Markdown
Contributor Author

/backport to stable29

@DerDreschner

DerDreschner commented Jul 30, 2026

Copy link
Copy Markdown
Contributor Author

Backport down to stable29 happens to stabilize CI runs on GitHub. Previous branches don't have the phpunit-mysql and phpunit-mariadb tests using GitHub Actions. Backport further down isn't necessary, as the mechanism was removed in stable22 and re-implemented in stable29.

@DerDreschner
DerDreschner enabled auto-merge July 30, 2026 13:59
@DerDreschner
DerDreschner force-pushed the fix/db-connectivity-check-idle-timer branch from cc69c33 to 44692a2 Compare July 30, 2026 14:29
…set on MariaDB/MySQL

The previous implementation of the idle timer runs on a strict 30 second interval and sends a dummy `SELECT` statement to keep the connection open.

This generates issues with the `lastInsertId` on long-running tasks (like our CI pipeline), as the MariaDB documentation clearly states:

> If the last query wasn't an INSERT or UPDATE statement or if the modified table does not have a column with the AUTO_INCREMENT attribute and LAST_INSERT_ID was not used, this function will return zero.

Source: https://mariadb.com/docs/connectors/mariadb-connector-c/api-functions/mysql_insert_id

To mitigate that, this commit now uses a real idle-timer per connection instead.

Assisted-by: ClaudeCode:claude-fable-5
Signed-off-by: David Dreschner <david.dreschner@nextcloud.com>
@DerDreschner
DerDreschner force-pushed the fix/db-connectivity-check-idle-timer branch from 44692a2 to acf233c Compare July 30, 2026 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

3. to review Waiting for reviews AI assisted bug CI feature: database Database related DB tests Related to tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fix flaky tests

3 participants