Fix Database Locking Issues#607
Open
fk0815 wants to merge 4 commits into
Open
Conversation
added 4 commits
April 15, 2026 20:22
The get_db() function was creating a new database connection on every call due to incorrect variable scoping. This caused connection leaks that led to 'Database busy. database is locked' errors when reloading pages. Changed from using a global variable (which wasn't working due to missing global keyword) to a static variable that maintains state within the function scope. This ensures only one connection is created per request, preventing connection accumulation and permanent locks.
…writes WAL (Write-Ahead Logging) mode allows SQLite readers to continue reading data even while a write transaction is in progress. This is critical for applications like BirdNET-Pi where the analysis script frequently writes detections while the web interface reads from the database. This change enables WAL mode and sets synchronous=NORMAL for better performance while maintaining data safety.
Instead of creating a separate transaction for each detection, this change batches all detections from a single audio file into one transaction. This significantly reduces the number of write operations and exclusive locks on the database, allowing the web interface to read more reliably during analysis. The batch function write_detections_to_db() handles multiple detections in a single BEGIN/COMMIT transaction, minimizing lock duration.
The 1-second busy timeout was too short for scenarios where the Python analysis script holds write locks while processing multiple detections. Increasing to 5 seconds gives the web interface more time to wait for write operations to complete before showing 'database is busy' errors. This change affects all PHP scripts that access the database: - common.php (get_db and ImageProvider) - history.php - overview.php - play.php - species_tools.php - todays_detections.php - weekly_report.php
|
Can someone review this? I can't comment on the PHP, especially the seemingly wrong implementation of the DB connection singleton. On the Python side of things, it does make sense to batch detection insertion, and it's looking fine. I've been actually running this for a few hours now, stress testing the UI in parallel, and can't reproduce a locked database. To test it, add the remote to your local git repo and run the update script: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix Database Locking Issues
Related Discussion: #323
This pull request addresses the "Database busy. database is locked" errors that occur when reloading pages while detections are being processed in the background.
Changes Included
• Fixed get_db() function that was creating new connections on every call
• Changed from broken global variable to static variable
• Ensures only one connection per PHP request
• Added PRAGMA journal_mode=WAL to allow concurrent reads during writes
• Set PRAGMA synchronous=NORMAL for better performance
• Critical for applications with frequent writes and reads
• Added write_detections_to_db() function for batch transactions
• Groups all detections from a file into single transaction
• Reduces lock duration and improves concurrency
• Increased timeout from 1s to 5s in all PHP scripts
• Gives web interface more time to wait for write operations
• Reduces "database is busy" errors
This PR has been crated with the help of mistral vibe AI.