Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docker/omnigraph-server.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ ARG OMNIGRAPH_VERSION=0.10.0
# Kept separate because on a moving tag the two differ — see
# witan_core/omnigraph_install.py :: _OMNIGRAPH_RELEASE_TAG.
ARG OMNIGRAPH_RELEASE_TAG=edge
ARG OMNIGRAPH_SHA256_X86_64=68099e33941cc5c252f36d4c2a26f1dfff6b28e4eb627ba1f7bf098856d34349
ARG OMNIGRAPH_SHA256_ARM64=d0e42176625584370a26c8a6ca43bc24447ea822483c46d971807475a5a5782a
ARG OMNIGRAPH_SHA256_X86_64=6b53f3eff3793d4012fd6b5c151c5589390699ce42661e3ce2ff1ed18b995f4c
ARG OMNIGRAPH_SHA256_ARM64=11c06689fcffa6e5fc9fbe9f632ba8e493f1f42997bbd10f09440f5e1d582aab

# ── Fetch + checksum-verify the release, extract both binaries ────────────────
FROM debian:trixie-slim@sha256:3a39a0592364683e6bab97937b72cad5a8fa6dcbbee90edb3bb48c7f8e94f258 AS fetch
Expand Down
4 changes: 2 additions & 2 deletions docker/witan.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ ARG OMNIGRAPH_VERSION=0.10.0
# Kept separate because on a moving tag the two differ — see
# witan_core/omnigraph_install.py :: _OMNIGRAPH_RELEASE_TAG.
ARG OMNIGRAPH_RELEASE_TAG=edge
ARG OMNIGRAPH_SHA256_X86_64=68099e33941cc5c252f36d4c2a26f1dfff6b28e4eb627ba1f7bf098856d34349
ARG OMNIGRAPH_SHA256_ARM64=d0e42176625584370a26c8a6ca43bc24447ea822483c46d971807475a5a5782a
ARG OMNIGRAPH_SHA256_X86_64=6b53f3eff3793d4012fd6b5c151c5589390699ce42661e3ce2ff1ed18b995f4c
ARG OMNIGRAPH_SHA256_ARM64=11c06689fcffa6e5fc9fbe9f632ba8e493f1f42997bbd10f09440f5e1d582aab
# Keep in lockstep with witan-council's version (mcp/servers/witan/pyproject.toml
# [project].version / [tool.bumpversion]); it labels the built image.
ARG WITAN_VERSION=0.8.0
Expand Down
4 changes: 4 additions & 0 deletions docs/_data/environment.toml
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,10 @@ desc = "How long before nominal expiry a cached token is treated as already expi
default = "~/.config/witan/tokens.json"
desc = "Where both CLIs cache OIDC tokens. Shared on purpose, next to the shared config file."

[section.vars.WITAN_MERGE_WATERMARKS]
default = "~/.config/witan/merge-watermarks.json"
desc = "Where `witan migrate merge` records what each pair of stores looked like when they last agreed, so the next merge can name the nodes both sides have written since. Beside the token cache, and per-machine — losing it costs one merge's divergence report."

[section.vars.WITAN_REMOTE_CALL_BUDGET_SECONDS]
default = "0"
desc = "Deadline for a single remote graph call, used to decide whether to honour a server's retry hint or give up. `0` means no client-side deadline — obey the server's hints."
Expand Down
56 changes: 55 additions & 1 deletion docs/guides/migration-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,11 @@ rather than in your environment. Read the decisions: `added` should be roughly
the row count of your store, and `updated` should be small. A large `updated`
on a first migration means slugs are colliding that shouldn't — stop there.

This run has no watermark to compare against, so it cannot tell you whether any
collision is a divergence — and being a dry run it records none either, so
step 4 is equally blind. The first run that can report divergence is a merge
*after* step 4 has succeeded. See [Divergence](#divergence).

**4. Run it:**

```bash
Expand Down Expand Up @@ -224,7 +229,7 @@ witan migrate merge [SOURCE] [--from <name>] [--to <name>] [--target <uri>] [--d
| `--from <name>` | A `[targets.<name>]` block's `server`, in place of `SOURCE`. A target with only a `remote_url` is refused — nothing local to export. |
| `--to <name>` | A `[targets.<name>]` block as the destination: through its deployment if it has a `remote_url`, into its `server` store if not. |
| `--target <uri>` | A destination store URI. Defaults to your configured store. Mutually exclusive with `--to`; `.jsonl` is refused (a target is a graph, not a snapshot). |
| `--dry-run` | Print the per-slug decisions, write nothing. |
| `--dry-run` | Print the per-slug decisions, write nothing. Reports divergence; records no watermark. |

Notes:

Expand All @@ -243,6 +248,55 @@ Notes:
- Reconciliation covers nodes only. Edge rows (`Tagged`, `ParentOf`, …) have no
slug and pass through unreconciled, same as raw `--mode merge`.

## Divergence

Newest-record-wins is a whole-**record** decision, and several witan fields are
append-only logs rather than values — `WorkflowProject.description`, which
accretes status blocks, most of all. When both stores have written the same node
since they last agreed, keeping the newer record does not resolve a stale value;
it deletes the other side's text.

Every merge therefore records a **watermark** for the pair of stores: the newest
timestamp in the source, and the newest that will be in the target once this
merge's winners land. The next merge uses it to name the nodes both sides have
written since:

```
2 node(s) changed on BOTH sides since the last merge (2026-08-19T19:46:00Z).
Newest-record-wins keeps one side and drops the other's edit …
WorkflowProject wp-witan-multi-user-service-deployment-dcf6ee
source 2026-08-19T19:49:00Z target 2026-08-19T22:06:00Z -> kept target
```

Nothing is merged for you. Reconcile the named slugs by hand — read both sides,
write the combined value to whichever store you want to win, and re-run — then
the merge resolves them on its own rule.

- **The first merge of a pair has no watermark and says so.** That is "cannot
tell", not "nothing diverged"; until one is recorded, diff the projects you
care about yourself.
- `--dry-run` reports divergence but records no watermark: the mark describes a
target with this merge's winners in it, and a dry run wrote none of them.
- Marks live in `~/.config/witan/merge-watermarks.json`
(`$WITAN_MERGE_WATERMARKS`), beside the token cache, keyed by source store and
destination. Per-machine, and losing the file costs one merge's reporting.
Local paths are keyed by their resolved absolute path, so `graph.omni`,
`../graph.omni` and `file:///…/graph.omni` share one mark rather than three.
- **A merge that fails part-way leaves no mark.** The standing one is retired
before the first batch commits and a fresh one installed only on success,
because batches commit independently: rows from a half-finished merge are
already in the target, and a mark that predates them would read those rows as
an independent target edit. The next run says it cannot tell, which is true.
Re-run the merge to get back to a marked state.
- Each side is compared against its own mark, which keeps the source's clock
out of the target's threshold and vice versa — a laptop and a cluster do not
agree closely enough for a cross-clock comparison. One documented exception:
the rows a merge loads carry their source timestamps into the target, so the
target mark is raised to cover them (otherwise every row a merge added would
come back as a target edit). Under a source clock running ahead, that leaves
a blind window the width of the skew in which a genuine target edit is not
reported.

## Fallback: in-cluster merge (operator)

Use when the MCP tier is unavailable, or to merge on someone else's behalf.
Expand Down
6 changes: 6 additions & 0 deletions docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1056,6 +1056,12 @@ entirely. Rows only in ``source`` are always added; rows only in the
target are left untouched. Repeatable — re-running against an
already-merged target loads nothing new.

Each merge records a per-side watermark for the pair of stores, so the next
one can name the nodes BOTH sides have written since — the case where
newest-record-wins is not resolving a stale value but discarding somebody's
edit. Nothing is auto-merged; the divergent slugs are reported for you to
reconcile. The first merge of a pair has no watermark and says so.

**Parameters**:

* `SOURCE, --source`: Store URI to merge from (local path, ``s3://``, ``file://``, or an
Expand Down
1 change: 1 addition & 0 deletions docs/reference/environment.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ token; both `witan` and `witan code` share one cache, so you log in once.

| Variable | Default | Description |
| --- | --- | --- |
| `WITAN_MERGE_WATERMARKS` | `~/.config/witan/merge-watermarks.json` | Where `witan migrate merge` records what each pair of stores looked like when they last agreed, so the next merge can name the nodes both sides have written since. Beside the token cache, and per-machine — losing it costs one merge's divergence report. |
| `WITAN_OIDC_AUDIENCE` | — | Audience/resource to request, matching the deployment's own `WITAN_OIDC_AUDIENCE`. Sent on the device-auth and token requests so an issuer with an audience mapper stamps the right `aud` claim. |
| `WITAN_OIDC_CLIENT_ID` | — | OIDC client id presented during the device grant. |
| `WITAN_OIDC_EXPIRY_SKEW_SECONDS` | `90` | How long before nominal expiry a cached token is treated as already expired and refreshed. Sized so a refresh happens before a long write starts rather than partway through one. |
Expand Down
2 changes: 2 additions & 0 deletions docs/reference/mcp-tools/memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -301,6 +301,8 @@ carry no slug and pass through additively, exactly as they do there.
| `rows` | list[object] | **required** | One batch of ``omnigraph export`` records — ``{"type": Node, "data":<br>{…}}`` for a node, ``{"edge": Edge, "from": …, "to": …}`` for an edge. |
| `dry_run` | bool | `False` | Reconcile and return the per-row ``decisions`` **without writing<br>anything**. Run the whole migration this way first: it is the only way<br>to see which side wins each ``(type, slug)`` before the graph changes. |
| `claim_from_author` | str? | `null` | The identity the *source* store wrote, when that store is your own.<br>Rows authored by exactly this name are restamped to the calling actor<br>before they are written; every other row keeps its author untouched.<br>Pass your local ``cfg.author`` here — the server cannot derive it,<br>having neither the caller's config nor their git checkout.<br>Without it, a migrated row keeps a name that no deployed identity can<br>ever match, and ``memory_delete`` refuses its own author forever<br>(#267). With it, the rows you migrate end up owned by the same identity<br>that owns everything you write afterwards. See ``_claim_authorship``<br>for why this matches rather than stamping unconditionally. |
| `since` | object? | `null` | The watermark recorded by the caller's *previous* merge into this graph<br>(``{"source_ts", "target_ts"}``). Supplied, a collision both sides have<br>written since then is marked ``diverged`` in the decisions — the merge<br>rule is unchanged, the losing edit is simply no longer discarded in<br>silence. Same value on every batch of one merge. |
| `watermark` | object? | `null` | The running watermark returned by the *previous batch of this same<br>merge*, folded into this batch's. The source is split across batches, so<br>no single batch sees its newest row; carrying the running value is what<br>lets the client end up with a mark covering the whole merge without<br>having to compare exported timestamps itself. ``None`` on the first<br>batch. |

## `claim_authorship`

Expand Down
41 changes: 41 additions & 0 deletions mcp/servers/witan/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,47 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/) (pre-1.0:
a MINOR bump may include breaking changes).

## [0.29.0] - 2026-08-24

### Fixed

- **`witan migrate merge` no longer discards a divergent edit in silence.**
Reconciliation is newest-record-wins per node, which is right when one side is
simply stale. It is not the same thing when BOTH stores have written the same
node since they last agreed: there the loser's edit is deleted, and several
witan fields are append-only logs rather than values —
`WorkflowProject.description`, which accretes status blocks, above all. Hit
for real on 2026-08-19, where the local store held the fourth update to a
project description and the deployment held the fifth, and merging either way
would have dropped one permanently.

The loss was invisible: a discarded divergence counted as `kept`, the same
bucket as the 3000-odd nodes that genuinely needed no action, and `--dry-run`
printed `kept-target` for it exactly as it did for them.

A merge now records a watermark for the pair of stores — the newest timestamp
in the source, and the newest that will be in the target once its winners land
— in `~/.config/witan/merge-watermarks.json` (`$WITAN_MERGE_WATERMARKS`). The
next merge uses it to name every node both sides have written since, with both
timestamps and which side was kept. The merge rule is unchanged and nothing is
auto-merged; what changes is that the slugs to reconcile by hand are now
nameable. The decisions carry a `diverged` flag and the counts a `diverged`
total, which cuts across `updated`/`kept_target` rather than partitioning with
them.

Each side is only ever compared against its own mark — a source is a laptop's
clock and a deployed target is a cluster's, and comparing across them would
read skew as divergence. The first merge of a pair has no watermark and says
so, which is "cannot tell", not "nothing diverged". `--dry-run` reports
divergence but records nothing, since it wrote none of the winners its mark
would describe.

Over the deployment the watermark crosses as a `store_merge` parameter and is
threaded batch to batch, so the mark the client records covers the whole merge
rather than its last batch. A deployment too old to return one yields no mark
at all rather than a partial one, and the CLI says the next merge will be
blind.

## [0.28.0] - 2026-08-21

### Fixed
Expand Down
56 changes: 55 additions & 1 deletion mcp/servers/witan/docs/migration-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,11 @@ rather than in your environment. Read the decisions: `added` should be roughly
the row count of your store, and `updated` should be small. A large `updated`
on a first migration means slugs are colliding that shouldn't — stop there.

This run has no watermark to compare against, so it cannot tell you whether any
collision is a divergence — and being a dry run it records none either, so
step 4 is equally blind. The first run that can report divergence is a merge
*after* step 4 has succeeded. See [Divergence](#divergence).

**4. Run it:**

```bash
Expand Down Expand Up @@ -214,7 +219,7 @@ witan migrate merge [SOURCE] [--from <name>] [--to <name>] [--target <uri>] [--d
| `--from <name>` | A `[targets.<name>]` block's `server`, in place of `SOURCE`. A target with only a `remote_url` is refused — nothing local to export. |
| `--to <name>` | A `[targets.<name>]` block as the destination: through its deployment if it has a `remote_url`, into its `server` store if not. |
| `--target <uri>` | A destination store URI. Defaults to your configured store. Mutually exclusive with `--to`; `.jsonl` is refused (a target is a graph, not a snapshot). |
| `--dry-run` | Print the per-slug decisions, write nothing. |
| `--dry-run` | Print the per-slug decisions, write nothing. Reports divergence; records no watermark. |

Notes:

Expand All @@ -233,6 +238,55 @@ Notes:
- Reconciliation covers nodes only. Edge rows (`Tagged`, `ParentOf`, …) have no
slug and pass through unreconciled, same as raw `--mode merge`.

## Divergence

Newest-record-wins is a whole-**record** decision, and several witan fields are
append-only logs rather than values — `WorkflowProject.description`, which
accretes status blocks, most of all. When both stores have written the same node
since they last agreed, keeping the newer record does not resolve a stale value;
it deletes the other side's text.

Every merge therefore records a **watermark** for the pair of stores: the newest
timestamp in the source, and the newest that will be in the target once this
merge's winners land. The next merge uses it to name the nodes both sides have
written since:

```
2 node(s) changed on BOTH sides since the last merge (2026-08-19T19:46:00Z).
Newest-record-wins keeps one side and drops the other's edit …
WorkflowProject wp-witan-multi-user-service-deployment-dcf6ee
source 2026-08-19T19:49:00Z target 2026-08-19T22:06:00Z -> kept target
```

Nothing is merged for you. Reconcile the named slugs by hand — read both sides,
write the combined value to whichever store you want to win, and re-run — then
the merge resolves them on its own rule.

- **The first merge of a pair has no watermark and says so.** That is "cannot
tell", not "nothing diverged"; until one is recorded, diff the projects you
care about yourself.
- `--dry-run` reports divergence but records no watermark: the mark describes a
target with this merge's winners in it, and a dry run wrote none of them.
- Marks live in `~/.config/witan/merge-watermarks.json`
(`$WITAN_MERGE_WATERMARKS`), beside the token cache, keyed by source store and
destination. Per-machine, and losing the file costs one merge's reporting.
Local paths are keyed by their resolved absolute path, so `graph.omni`,
`../graph.omni` and `file:///…/graph.omni` share one mark rather than three.
- **A merge that fails part-way leaves no mark.** The standing one is retired
before the first batch commits and a fresh one installed only on success,
because batches commit independently: rows from a half-finished merge are
already in the target, and a mark that predates them would read those rows as
an independent target edit. The next run says it cannot tell, which is true.
Re-run the merge to get back to a marked state.
- Each side is compared against its own mark, which keeps the source's clock
out of the target's threshold and vice versa — a laptop and a cluster do not
agree closely enough for a cross-clock comparison. One documented exception:
the rows a merge loads carry their source timestamps into the target, so the
target mark is raised to cover them (otherwise every row a merge added would
come back as a target edit). Under a source clock running ahead, that leaves
a blind window the width of the skew in which a genuine target edit is not
reported.

## Fallback: in-cluster merge (operator)

Use when the MCP tier is unavailable, or to merge on someone else's behalf.
Expand Down
4 changes: 2 additions & 2 deletions mcp/servers/witan/pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "witan-council"
version = "0.28.0"
version = "0.29.0"
description = "witan — agent memory, planning, and collaboration graph (work-coordination layer + umbrella CLI)"
readme = "README.md"
license = "BSD-3-Clause"
Expand Down Expand Up @@ -213,7 +213,7 @@ packages = ["witan"]
"schema" = "schema"

[tool.bumpversion]
current_version = "0.28.0"
current_version = "0.29.0"
allow_dirty = true

[[tool.bumpversion.files]]
Expand Down
Loading
Loading