Skip to content

feat: workflow store with outbox finish (contract v0.8.0) - #25

Merged
fahimfaisaal merged 8 commits into
mainfrom
feat/workflow-store
Nov 9, 2026
Merged

fahimfaisaal merged 8 commits into
mainfrom
feat/workflow-store

Conversation

@fahimfaisaal

Copy link
Copy Markdown
Member

Summary

VarMQ workflows on SQLite. Store is its own object, found by scope (varmq #187, contract v0.8.0).

qs, _    := sqliteq.New(dir)                 // queues: ScopeID() = "sqliteq:<abs root>"
store, _ := sqliteq.NewWorkflowStore(qs)     // registers in contract by scope; Close unregisters
wf := varmq.NewWorkflow(store)               // workers find store via contract.LookupWorkflowStore(q.ScopeID())

Folder layout: own file __sqliteq_workflows.db, outbox finish:

FinishTask(queueID, ackID, end)
  tx(workflow db): record end + write released tasks to outbox
  push each outbox row: EnqueueOnce("wf:<id>:<task>[:undo]"), delete row
  ack job in its queue file
recovery loop: push leftover outbox rows at open + every 1 s
redelivered job / repeated push: Runnable pre-check skips ended task

WithSingleFile: end + pushes + ack in one transaction.

Also: reserved wf: job-ID prefix for plain jobs, purge ends workflow tasks cancelled (stored intent applied only after job row gone), cancel mark never drops valid workflow item, case-fold JSON keys = plain job, WithWorkflowResultLimit (64 KiB) / WithWorkflowRetention (7 d).

Evidence

  • go build ./... && go vet ./... && go test -race -count=1 ./... pass against contract v0.8.0 + varmq c75ed52.
  • Tests: crash between outbox steps (simulated), e2e workflow with varmq workers, both file layouts.
  • Integration run (varmq + sqliteq branches): folder and single-file workflow with undo ends compensated.

Merge Danger

Door: two-way

New API, unreleased. Workflow tables created only by NewWorkflowStore.

Blast Radius: sqliteq users on upgrade

Requires contract v0.8.0; varmq pinned to pseudo-version of c75ed52 until v1.11.0 tag.

The workflow write holds the file mutex, so FinishTask must not take
queues.mu: byID is now a copy-on-write atomic map. The folder root is
resolved, so queue and store scopes agree through symlinks.
Outbox row for unreachable push targets in both layouts, queue lookup across
clients of a scope, purge without a registered store (one transaction in a
single file, stored intents in folders), job ID guard for wf: IDs, json-style
key folding, scope checks, deps check, expired resubmit, sweep throttle.
@codecov

codecov Bot commented Nov 9, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.78327% with 118 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
workflow.go 90.14% 62 Missing ⚠️
workflow_finish.go 81.08% 28 Missing ⚠️
queue.go 83.48% 18 Missing ⚠️
wfmachine.go 92.85% 10 Missing ⚠️

📢 Thoughts on this report? Let us know!

@fahimfaisaal
fahimfaisaal merged commit d5be329 into main Nov 9, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant