Skip to content

feat(opencode): read-only SQLite adapter with v2.0.x inline schema support - #25

Merged
HarryHeYu merged 4 commits into
mainfrom
fix/opencode-v20x-inline-schema
Nov 12, 2026
Merged

HarryHeYu merged 4 commits into
mainfrom
fix/opencode-v20x-inline-schema

Conversation

@HarryHeYu

Copy link
Copy Markdown
Owner

What this adds

The OpenCode adapter: read-only SQLite indexing for OpenCode sessions, plus a
provider capability-matrix entry, isolation coverage, and an anonymised fixture
suite.

It supports three shapes, detected structurally (never by a product version
string):

shape session table message content parts table
v2 (v2.2.x) session_v2 session_message session_part
v2-inline (v2.0.x) session_v2 inline in session_message.data none
legacy session message part

v2-inline is discriminated by requiring session_v2 + session_message and
forbidding session_part, so a v2.2.x database (which also has
session_message) still resolves to v2.

Why the inline shape matters (this was a real failure, not a hypothetical)

The first Phase 3 cut only knew the v2.2.x parts schema. On a real
OpenCode v2.0.29 install the adapter returned unknown-schema, discovered
no databases, and indexed 0 of 8 real sessions — while the 27 synthetic
tests were green, because their fixtures model v2.2.x. That gap is the reason
synthetic results are reported as UNIT_VERIFIED and never as LIVE_VERIFIED.

Real structure observed (no user text): session_message.data holds content
inline — user.text is a string; assistant.content is an array of
text / reasoning / tool items; a tool's exit code is in
state.metadata.exit, not state.exit.

Behaviour details

  • Read-only: file:...?mode=ro + PRAGMA query_only=ON; never
    immutable=1 (that would hide committed WAL pages). -wal / -shm stay in
    the discovered source set so a running agent's new turns trigger a rescan.
  • Mapping: session title/cwd/repo/model/tokens/timestamps; user and
    assistant content; reasoning; tool calls and results from the same record's
    state; seq is a stable per-session counter; ms → s.
  • Provenance from structure only: origin comes from the message type
    column. system / synthetic / compaction / idle / model-switched
    rows become meta with provider_system, never a human turn. A subagent
    session (parent_id) marks its user turns provider_synthetic.
  • No invention: a running tool gets no fabricated result; a missing exit
    code stays None (0 is preserved as a real code); empty user text emits no
    user event; an unknown message type is skipped rather than guessed.
  • Discovery priority: an explicit VOYAGER_OPENCODE_DB / OPENCODE_DB is
    now authoritative — the default data-directory database and channel builds
    are not probed when an override is set. This prevents an operator-named
    (or test-named) database from silently causing a read of a real HOME
    database.
  • No native resume: can_resume = can_fork = False, resume_cmd = None;
    the capability matrix says NOT_FOUND.

Tests

Directed (tests/test_opencode.py, 36 tests): both v2 shapes, legacy, unknown
schema, corrupt / absent database, tool call/result pairing, running tool with
no invented result, error without a fake exit code, synthetic/system not
promoted to a human turn, subagent sessions, Unicode (CJK/Japanese/Korean/emoji),
malformed JSON, empty sessions, read-only (DB/WAL bytes and row counts
unchanged), WAL discovery and WAL-driven incremental rescan through the real
Store + run_scan, idempotence, streaming equivalence, 2-char CJK search,
SOURCE_MISSING retention, a 5000-message session, and the override-priority
regression.

tests/test_opencode.py tests/test_docs.py tests/test_isolation_guard.py tests/test_consistency.py
=> 105 passed, 3 skipped        (skips: Windows case-insensitive, no symlink, skills-ref absent)

scripts/check_consistency.py --strict  => all checks passed
python -m compileall voyager tests     => clean

Real-install gate (one v2.0.29 install, isolated copy)

The real .db / -wal / -shm were copied byte-for-byte into scratch; every
SQLite connection targeted the copy; HOME, USERPROFILE and XDG_DATA_HOME
were all redirected into scratch and the run asserts no candidate lies outside
it. The real directory was only hashed.

Gate A — schema + query results: PASS

probe          -> ('ok', 'v2-inline')
discover       -> opencode.db, opencode.db-wal, opencode.db-shm
sessions       -> 8 / 8 indexed (oracle: 8)
oracle totals  user 68 | text 206 | reasoning 2053 | tool 2617 | tool_result 2617
adapter totals user 68 | assistant 206 | reasoning 2053 | tool_call 2617 | tool_result 2617
mismatches     -> NONE

The oracle is plain stdlib sqlite3 on the same copy — it does not import the
adapter, so agreement is independent.

Gate B — no modification of user data: PASS

check result
real .db SHA-256 before/after identical
real -wal SHA-256 before/after identical
real -shm SHA-256 before/after identical
row counts, all 20 tables identical
new files in isolated dir none

Gate C — capability ledger: no cell promoted. provider_state stays
NOT_FOUND_IN_CURRENT_AUDIT in the isolated run; the adapter does not claim
LIVE_VERIFIED.

Report: docs/audits/2026-11-opencode-v20x-real-gate.md (no user text, no
paths, no hashes).

Known limits (explicitly not claimed)

  • One install, one platform, one version (v2.0.29). A future shape change fails
    loudly as unknown-schema rather than being misparsed.
  • Native resume / startup hook / transcript writer: NOT_FOUND.
  • Cross-agent continuation: NOT_VERIFIED (no live agent run).
  • File attachments in user.files on the inline shape: NOT_VERIFIED (empty in
    the observed data).
  • Multi-channel real install: NOT_VERIFIED.
  • message_count counts user + assistant content events, not raw
    session_message rows (existing contract, unchanged).

Scope

Additive: one provider in enabled_adapters / PROVIDERS. No change to
Store, the continuity engine, the bridge schema, any existing CLI contract,
or any other adapter. No database migration, no new runtime dependency (stdlib
sqlite3 only), no version bump, no tag or release change.

Tracking: sessionFlow#23 (fix(opencode): support v2.0.x inline session_message schema). Not merged by the author — left for maintainer review.

…pport

Adds the OpenCode adapter (read-only SQLite: session_v2 / session_message / session_part / project), a provider capability matrix entry, isolation coverage, and an anonymised fixture suite.

Three shapes are detected structurally, never by a product version string:
- v2 (v2.2.x): parts in session_part;
- v2-inline (v2.0.x): content inline in session_message.data, no session_part;
- legacy: session / message / part.

The inline shape maps user.text, assistant.content[] (text / reasoning / tool), reads a tool exit code from state.metadata.exit, keeps provider-generated rows as meta, never invents a tool result, and orders events by seq with a deterministic tiebreak. An explicit VOYAGER_OPENCODE_DB / OPENCODE_DB is authoritative, so an operator-named database cannot trigger an unrequested read of the default HOME database.

Real-install gate (one v2.0.29 install, isolated copy): probe -> ok/v2-inline, 8/8 sessions indexed, per-kind counts exactly match an independent read-only sqlite3 oracle, and the real .db / -wal / -shm hashes and all row counts are unchanged. Synthetic results stay UNIT_VERIFIED; native resume and cross-agent continuation remain NOT_FOUND / NOT_VERIFIED.
Copilot AI balanced review requested due to automatic review settings November 12, 2026 14:05

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

The opencode block still said 'no real install was observed' and described
only v2/legacy shapes, which contradicts this PR's own gate report (one
real v2.0.29 install, three shapes). States stay UNIT_VERIFIED: one
install/version/platform does not justify promoting a cell, and nothing is
LIVE_VERIFIED.

@HarryHeYu HarryHeYu left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final technical review — APPROVE (no blocking issues)

Reviewed at 9603164 (was 9471608; one fix added, see "Found and fixed").
Independent evidence, not just "CI is green": I re-derived each claim on
synthetic fixtures and re-checked the real-install gate.

1. Schema identification and priority — verified

case result
v2.2.x (has session_part) ('ok', 'v2')
v2.0.x (inline, no session_part) ('ok', 'v2-inline')
pre-v2 ('ok', 'legacy')
readable non-OpenCode DB ('unknown-schema', None)
truncated file ('unreadable', None) — not absent
DB with both a parts table and inline content ('ok', 'v2')

Priority is structural: SCHEMA_FORBID = {"v2-inline": ("session_part",)} means
a database that carries session_part can never be misread as inline, even if
SCHEMAS order changes later. No product version string is consulted.

2. session_message.data parsing — verified

Inline shape: user.text → user event; assistant.content[] → text /
reasoning / tool events. Tool exit code is read from
state.metadata.exit and 0 is preserved as a real code (not treated as
missing). 27/27 checks passed, including CJK text, one-tool-call/one-result
pairing, and a running tool producing no fabricated result.

3. Event order and counts — verified

seq is a strictly increasing 1..N counter per session; a second scan yields an
identical (seq, kind, content) stream (idempotent). Counts match the fixture
oracle exactly: user 1, assistant 2, reasoning 1, tool_call 1, tool_result 1,
meta 2 (system + compaction), with provider rows staying meta
(origin=provider_system).

4. Tool call / result pairing — verified

tool_call and tool_result share tool_call_id. A result is emitted only
when upstream recorded completed + output, or an error; running produces the
call alone. No exit code is invented for an error state.

5. WAL consistency, incremental indexing, idempotence — verified

A live writer appending one turn through -wal is seen on the next scan
(1 → 2 user events); repeated scans are byte-stable in event terms. -wal and
-shm are in the discovered source set, which is what makes a running agent's
new turns visible before a checkpoint.

6. Read-only behaviour and the -shm side effect — verified

Connections are mode=ro with PRAGMA query_only=1; an INSERT on that
connection is refused; the .db bytes are unchanged after a scan. The one real
side effect — SQLite may rewrite the -shm shared-memory index when opening a
WAL database read-only — is documented in the audit report rather than hidden.
.db and -wal bytes and all row counts are unchanged.

7. Explicit override isolation — verified

With VOYAGER_OPENCODE_DB set, _candidate_dbs() returns only the named
database, and discover() does not include the default data-dir database. This
closes the earlier finding that an operator-named file could cause an
unrequested read of the real HOME database.

8. Error handling — verified

Unknown message types are skipped (no invented content, no crash); a malformed
JSON row does not abort the scan; unknown schema / corrupt file are named
(unusable_sources / unrecognised_sources) rather than reported as "0
sessions" or retired as SOURCE_MISSING.

9. Capability matrix — no overclaim

No cell is LIVE_VERIFIED; native_resume and every startup/injection
dimension are NOT_FOUND_IN_CURRENT_AUDIT; discovery/indexing stay
UNIT_VERIFIED.

Found and fixed in review (P1, now in 9603164)

The opencode block still said "no real install was observed" and described
only v2/legacy, contradicting this PR's own gate report. Those strings are
user-visible in voyager verify --matrix, so I corrected them to state: three
shapes exercised synthetically, one real v2.0.29 install observed, states kept
at UNIT_VERIFIED because one install/version/platform does not justify
promotion. Re-ran the capability/doctor/verification/dashboard suites: 148
passed; test_opencode.py + test_docs.py + test_capability_consistency.py:
61 passed / 1 skipped.

Real-install gate — scope is narrow, and stated as such

Gate A compared the adapter against an independent read-only sqlite3 oracle on
a byte-identical copy: 8/8 sessions, user 68 / assistant 206 / reasoning 2053 /
tool_call 2617 / tool_result 2617, zero mismatches. Gate B: .db / -wal /
-shm SHA-256 and all 20 table row counts unchanged. Gate C: nothing promoted.

This is one install, one version (v2.0.29), one platform (Windows). It does
not generalise to other v2.0.x builds, to macOS/Linux, or to multi-channel
installs. Synthetic results are not a substitute for it, and native resume /
cross-agent continuation are not supported — the adapter denies resume.

Residual risks (non-blocking, documented)

  1. A future shape change fails loudly as unknown-schema (good) but will need
    another adapter round — detection is by table/column shape, not version.
  2. message_count counts user+assistant content events, not raw
    session_message rows (existing contract).
  3. user.files attachments on the inline shape were empty in the observed data:
    NOT_VERIFIED.
  4. Multi-channel and non-Windows installs: NOT_VERIFIED.

Merge readiness

No blocking issues. After CI is green on 9603164 this is mergeable as an
additive provider (no Store/continuity/bridge-schema change, no migration, no
new dependency, no version or tag change). I am not merging it in this round
— the merge is a separate, explicit step, and no release is made here.

The previous commit was created through the Git Data API from the Windows
working tree, so it uploaded CRLF and rewrote every line. Normalise to LF to
match the repository; the content is unchanged.
@HarryHeYu
HarryHeYu merged commit ee9fd9e into main Nov 12, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants