Skip to content

feat: workflow counts + task times (contract v0.9.0) - #26

Merged
fahimfaisaal merged 4 commits into
mainfrom
feat/workflow-counts
Nov 10, 2026
Merged

fahimfaisaal merged 4 commits into
mainfrom
feat/workflow-counts

Conversation

@fahimfaisaal

Copy link
Copy Markdown
Member

Summary

Workflow counts + task times. Implements contract.WorkflowCounter, contract.HostLocal and TaskRecord times (contract v0.9.0, goptics/varmq#191).

 __sqliteq_wf_tasks
+  enqueued_at, started_at, finished_at                  INTEGER DEFAULT 0
+  undo_enqueued_at, undo_started_at, undo_finished_at   INTEGER DEFAULT 0
+INDEX __sqliteq_wf_workflows_state (state, created_at DESC, id)
WorkflowCounts
  SELECT state, COUNT(*) ... GROUP BY state     # same rows as WorkflowKeys

Runnable
  lock-free read -> answer                      # foreign write lock: no error
  true + task still pending -> stamp started_at # failed stamp: log, answer stays true
  unknown workflow -> ErrWorkflowNotFound, no row made

Purge
  workflow tables present, store not ready -> keep jobs, log cause
  • HostLocal marker: varmq adds host name to the store ID in GET /workflow-counts; same path on two hosts = two stores.
  • Columns added at open to older tables; old rows read 0.
  • go.mod: contract v0.8.0 -> v0.9.0; varmq -> 8057d16 (merge commit of feat: workflow counts + task times (contract v0.9.0) varmq#191, pseudo-version; no varmq release tag yet).

Evidence

  • Before: main: no WorkflowCounts, records carry no times; Runnable under a foreign write lock gives an error.
    After: GOWORK=off go build ./... && go vet ./... && go test -race -count=1 ./...
    ok  	github.com/goptics/sqliteq	13.748s
    ?   	github.com/goptics/sqliteq/example	[no test files]
    
    New tests appended to queue_test.go, migration_test.go: TestWorkflowCounts, TestWorkflowTaskTimes, TestRunnableUnderForeignLock, TestRunnableStampGuard, TestPurgeKeepsJobsWithoutStore, TestMigrateWorkflowTaskTimes.

Merge Danger

Door: one-way (schema)

ALTER TABLE ... ADD COLUMN on open. Older sqliteq reads the upgraded file fine (extra columns ignored); columns stay after a revert.

Blast Radius: workflow store

  • First open of a big old file: six ADD COLUMN + one CREATE INDEX.
  • Two processes that open the same old file at once can race on ADD COLUMN (not covered by a test).
  • go.mod pins varmq by pseudo-version: re-pin when varmq tags its next release.

- WorkflowCounts (contract.WorkflowCounter): GROUP BY state, same rows as WorkflowKeys
- index __sqliteq_wf_workflows_state (state, created_at DESC, id)
- task times on TaskRecord + Undo: EnqueuedAt at release, StartedAt in Runnable on true, FinishedAt at end/cancel/drop/fail-fast
- Runnable: one write tx, check then stamp; unknown workflow makes no row
- __sqliteq_wf_tasks: six INTEGER DEFAULT 0 time columns, guarded ALTER at open
- on-demand purge store runs init: old tables migrate before Purge loads them
- tests: counts per state incl. purge + sweep, task times, v0.8 schema migration
- Runnable: answer from lock-free read; yes -> stamp started_at in own
  write, guarded on pending state. Stamp error logged, answer stays true.
  Foreign write lock no longer gives "database is locked" (worker ran task
  on that error).
- purgeStore returns err when workflow tables may exist but store cannot
  be prepared (check, open, init fail); Purge logs, deletes nothing.
- HostLocal marker; guarded ALTER for task time columns (concurrent open).
- Doc: enqueued_at = release time, kept while push waits in outbox.
- Tests: Runnable under foreign lock, stamp guard, count query plan uses
  state index, Purge keeps jobs on corrupt workflow file.
@codecov

codecov Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.39785% with 8 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
workflow.go 92.30% 5 Missing ⚠️
workflow_finish.go 84.21% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

@fahimfaisaal
fahimfaisaal merged commit a83ae91 into main Nov 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant