Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 55 additions & 14 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,74 @@
# Changelog

All notable changes to the SQLiteQ project will be documented in this file.
All notable changes to the sqliteq project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Added
### ⚠️ Breaking Changes

- `Reject(ackID, reason)` on `Queue` and `PriorityQueue`, required by varmq's `IAcknowledgeable` since varmq#173: finishes an item that failed for good, never requeued. Deleted like an ack, or with `WithRemoveOnComplete(false)` kept with `status` `failed`. Same group commit as `Acknowledge`. The reason is the job's outcome error.
- Job outcome (varmq spec 6.13): `WatchOutcome` (`contract.OutcomeWatcher`) and `AcknowledgeResult` (`contract.ResultAcknowledgeable`) on both queues. A job with a Job ID (`EnqueueOnce` id, else the item's JSON `id`) gets a record in `<name>__sqliteq_jobs` when it ends, same transaction as the finish. Watches poll it by seq every 50 ms (new records only), from any queue object on the file, any process. Retention: completed 1000 / 24 h, failed and cancelled 10000 / 7 d; the count limit spares records under 1 min old. A polling queue object also sweeps retention and expired marks once a minute.
- Job cancel (varmq spec 6.14): `CancelJob`, `OnCancel`, `AcknowledgeCancelled` (`contract.JobCanceller`, `CancelNotifier`, `CancelAcknowledgeable`). Marks in `<name>__sqliteq_cancelled`, 24 h each. Every pop drops a marked waiting job (outcome `cancelled`, at most 1000 per transaction; a single-item pop goes on past that, a batch pop reports an item left); `OnCancel` polls new marks. `ErrJobEnded` when the record shows it ended.
- Queue inspection (`contract.QueueInspector`) on both queues, so varmq's REST API and vartop list sqliteq queues. `QueueID` is `sqliteq:<h>:fifo|prio:<name>`, `<h>` the first 8 hex of the SHA-256 of the file's resolved path, `<name>` in lower case. `QueueStats`: counts per row status in one statement (cancelled under failed; completed and failed only with `WithRemoveOnComplete(false)`), totals submitted / completed / failed from new table `<name>__sqliteq_meta` (an insert trigger counts adds; a finish counts in its transaction; an older table starts from its rows, so jobs deleted before the upgrade are missing; `Purge` resets them), and the pause. `Jobs`: pending in hand-out order, processing by ack id, kept completed, failed (with cancelled) and cancelled rows newest first, payload the envelope's `data` verbatim, errors from the outcome record. Attempts are not counted (0 pending, 1 after).
- Durable queue pause (varmq spec 6.7, 6.8): `Pause`, `Resume`, `Paused`, `PauseRefusals` (`varmq.QueuePauser`) and `OnPauseChange` (`varmq.PauseNotifier`). The flag is a row of `<name>__sqliteq_meta`; every pop statement selects nothing while it exists, in every process on the file; it outlives restarts and `Purge`. A refused pop sets `Paused`; while set, the queue object polls the row every 50 ms to see a `Resume` from another process.
- `EnqueueOnce` that stores an id drops its old record and mark (trigger). `Purge` clears records and marks too. `Queue.Close` drops watches and hooks.
- Requires Go 1.27 (was 1.24): `go.mod` requires VarMQ, which needs it. (#22)
- Queue name must not contain `__sqliteq_`, in both layouts. (#22)
- Reason: a queue's indexes, side tables and triggers use that part in their names. (#22)

### Breaking
### Added

- Requires Go 1.27 (was 1.24), for the varmq dependency.
- A queue name must not contain `__sqliteq_`, in both layouts: a queue's indexes, sidecar tables and trigger use it.
- `Reject(ackID, reason)` on `Queue` and `PriorityQueue`, for VarMQ's `IAcknowledgeable` (varmq#173). (#21)
- `Reject` finishes an item that failed for good: never requeued. (#21)
- Rejected item deleted like an ack; with `WithRemoveOnComplete(false)` kept with `status` `failed`. (#21)
- `Reject` uses the same group commit as `Acknowledge`. (#21)
- `Reject` reason stored as the job's outcome error. (#22)
- Job outcome (VarMQ spec 6.13): `WatchOutcome` (`contract.OutcomeWatcher`) on both queues. (#22)
- `AcknowledgeResult` (`contract.ResultAcknowledgeable`) on both queues. (#22)
- Job ID = `EnqueueOnce` id, else the item's JSON `id`. (#22)
- Job with a job ID gets a record in `<name>__sqliteq_jobs` when it ends, in the finish's transaction. (#22)
- Watches poll records by seq every 50 ms, from any queue object on the file, any process. (#22)
- Retention: completed 1000 / 24 h; failed and cancelled 10000 / 7 d. (#22)
- Count limit spares records under 1 min old. (#22)
- Polling queue object sweeps retention and expired marks once a minute. (#22)
- Job cancel (VarMQ spec 6.14): `CancelJob` (`contract.JobCanceller`) on both queues. (#22)
- `OnCancel` (`contract.CancelNotifier`) polls new marks every 50 ms. (#22)
- `AcknowledgeCancelled` (`contract.CancelAcknowledgeable`) finishes a running job that stopped for a cancel. (#22)
- Cancel marks in `<name>__sqliteq_cancelled`, 24 h each. (#22)
- Every pop drops a marked waiting job, outcome `cancelled`, at most 1000 per transaction. (#22)
- At the drop cap: single-item pop goes on; batch pop reports an item left. (#22)
- `CancelJob` returns `contract.ErrJobEnded` when the record shows the job ended. (#22)
- `EnqueueOnce` that stores an id drops its old record and mark (trigger). (#22)
- `Purge` clears records and marks too. (#22)
- `Queue.Close` drops watches and hooks. (#22)
- Queue inspection (`contract.QueueInspector`) on both queues: VarMQ's REST API and vartop list sqliteq queues. (#23)
- `QueueID`: `sqliteq:<h>:fifo|prio:<name>`, `<name>` in lower case. (#23)
- `<h>`: first 8 hex of the SHA-256 of the file's resolved path. (#23)
- `QueueStats` counts: per row status, one statement; cancelled under failed. (#23)
- Completed and failed counts only with `WithRemoveOnComplete(false)`. (#23)
- `QueueStats` totals: submitted, completed, failed, in new table `<name>__sqliteq_meta`. (#23)
- Insert trigger counts adds. (#23)
- Finish counts in its transaction. (#23)
- `Purge` resets totals. (#23)
- Older table: totals start from its rows, so jobs deleted before the upgrade are missing. (#23)
- `QueueStats` reports the pause. (#23)
- `Jobs`: pending in hand-out order, processing by ack ID. (#23)
- `Jobs`: kept completed, failed (with cancelled) and cancelled rows, newest first. (#23)
- `Jobs` payload: the envelope's `data` verbatim; errors from the outcome record. (#23)
- Attempts not counted: 0 pending, 1 after. (#23)
- Durable queue pause (VarMQ spec 6.7, 6.8): `Pause`, `Resume`, `Paused`, `PauseRefusals` (`varmq.QueuePauser`). (#23)
- `OnPauseChange` (`varmq.PauseNotifier`). (#23)
- Pause flag: a row of `<name>__sqliteq_meta`. (#23)
- Every pop selects nothing while the flag exists, in every process on the file. (#23)
- Pause outlives restarts and `Purge`. (#23)
- Refused pop sets `Paused`; while set, the queue object polls the row every 50 ms for a remote `Resume`. (#23)

### Changed

- Transactions begin `IMMEDIATE`. A finish (`Acknowledge`, `Reject`) always runs in a transaction; a pop runs in one while any cancel mark is live.
- Transactions begin `IMMEDIATE`. (#22)
- Finish (`Acknowledge`, `Reject`) always runs in a transaction. (#22)
- Pop runs in a transaction while any cancel mark is live. (#22)

## [0.3.0] - 2026-10-23

### Breaking
### ⚠️ Breaking Changes

- `New(path, opts ...StoreOption)` returns `(Queues, error)` instead of panicking. `NewQueue` and `NewPriorityQueue` keep their signatures.
- `path` is now a folder, created if missing, holding one `<name>.db` file per queue with `<name>` in lower case (`NewQueue("Emails")` uses `emails.db`). Pass `WithSingleFile()` to keep the v0.2 layout and open an existing v0.2 file.
Expand Down Expand Up @@ -103,7 +144,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Initial release of SQLiteQ
- Initial release of sqliteq
- Basic queue functionality with SQLite backend
- Support for acknowledgment-based processing
- Priority queue implementation with configurable priority levels
Expand Down
Loading