Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,13 @@ The project publishes 0.x prerelease versions; a stable release line is not yet

### Added

- Workspace-scoped Markdown Vault sync HTTP API (`#234`): discover Vaults, pull bounded
snapshots and atomically commit logical-note deltas with whole-Vault CAS.
Independent empty notes, immutable revisions and deletion tombstones preserve
identity and concurrent edits; workspace, read/write and token-path boundaries
reuse canonical authorization. Additive migration 0027 leaves existing files
unchanged. See `docs/VAULT_SYNC.md` for limits, downgrade data loss and the
explicit boundary from current workspace-bundle export/import formats.
- Per-agent memory namespace filter on lexical recall (`#227`): `POST /v1/context`
and `mem_context` accept `agent_id` plus optional `extra_agent_ids`. When set,
structured-memory recall hard-filters `producer_agent` before ranking so
Expand All @@ -17,6 +24,17 @@ The project publishes 0.x prerelease versions; a stable release line is not yet
Design: `docs/agent-memory-namespace.md`.
- `mem put <path> --watch` one-way foreground directory watch: poll `--interval` (default 30s), ingest new files after one quiet interval, report changed files without re-ingesting, and never propagate local deletes. Refs #110.

### Fixed

- Restore Linux/Windows Web dependency audit gates by removing the unpatched
braces build-tool chain with official Tailwind/PostCSS 4 and typescript-eslint
8, while retaining existing theme utility behavior and browser acceptance.
Apply compatible transitive security updates without relaxing audit thresholds.
Validation and public advisory references: `docs/acceptance/VAULT_SYNC_VALIDATION.md`.
- Build isolated lifecycle-test MinIO/mc images from checksum-pinned official
sources when former public images are no longer anonymously pullable. Keep
production assets and all HTTP/CLI/MCP assertions unchanged. Refs #234.

### Changed

- The live recall producer (`python3 -m benchmarks.recall produce`) reads engine, profile, provider and embedding dimension from the running memd instead of CLI labels, scores the multilingual v1 set (English and Chinese, including image-description queries), and stays on-demand rather than a CI gate. Refs #175.
Expand Down
22 changes: 22 additions & 0 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,28 @@ workspace import 的数据库 commit 若无法确认结果,服务端不得删

---

### F9A · Workspace Markdown Vault 同步(HTTP v1)

需求:[mem #234](https://github.com/bytefolk/mem/issues/234)。

`GET /v1/vault/list`、`GET /v1/vault/snapshot?vaultId=<UUID>` 与
`POST /v1/vault/commit` 提供工作区内的逻辑笔记同步底座。每份笔记使用独立
`noteId`,相同正文和空笔记不会因 blob 去重而合并。commit 传递
`{vaultId,baseRevision,title?,entries:[{noteId,path,content,deleted?,properties?}]}`
增量;整个 Vault 的 head 在单个事务内执行 CAS,过期 head 返回 `409`
及 `currentRevision`,不覆盖其他客户端。删除保留 tombstone 与不可原地更新的
正文 revision;重命名保留逻辑 ID。只读需 `read`,提交需 `read` + `write`,
复用现有 workspace membership 和 `/Vaults/<vaultId>` token path 边界。
properties 仅为有界数据,不能授予工具或身份权限。图片等原件继续使用 files API。

`vault-snapshot.v1` 是独立的当前状态可迁移合同,详见
[API、限制与错误码](docs/VAULT_SYNC.md) 和
[JSON Schema](docs/schemas/vault-snapshot.v1.schema.json)。CLI/MCP/Web 本次不新增
Vault 命令或独立笔记编辑器。现有 workspace bundle v1/v2 不包含 Vault 表和历史;
完整历史仍需数据库备份。服务不运行桌面文件监听或自动合并客户端冲突;客户端必须
拉取 head、三方合并并保留冲突副本。迁移 0027 不改变 files/auth/memory 表;降级
到 0026 会丢弃 Vault 数据,操作前需备份。

## 4. 非功能需求

| 维度 | 目标 |
Expand Down
17 changes: 14 additions & 3 deletions docs/TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,8 +274,10 @@ Required tests:
- `TestManagedAISettlementOutboxPostgres`
- `TestReleasedFileStageRetryPostgres`
- `TestDurableContextPostgres`
- `TestVaultSyncPostgres` (two-client CAS, distinct empty notes, auth/workspace,
immutable history, tombstones, path swap/title and migration 27 down/up)

Expected result: migrations reach the declared current head (currently `20`),
Expected result: migrations reach the declared current head (currently `27`),
all rollback-state assertions pass, every named test prints `PASS`, all commands
exit `0`, and the race run reports no data race.

Expand All @@ -298,6 +300,15 @@ Run the isolated, model-free process test:
make test-acceptance
```

The script builds temporary MinIO/mc binaries from checksum-pinned official
source archives because Community Edition is now source-only. It uses the
existing Go toolchain, `tar` and `sha256sum`, then builds a uniquely named test
image on a pinned official Alpine runtime. The temporary Compose override,
image and run directory are removed by its ownership-checked cleanup. This
does not replace images or credentials in a production deployment. Network
access to the fixed source archives and locked Go module dependencies is
required; missing artifacts or checksum mismatches fail the gate.

The script owns a per-run Compose project, random loopback PostgreSQL/MinIO
ports, an ephemeral database/store and a new memd process on an atomically
locked, twice-preflighted loopback port. It builds the current CLI, MCP adapter
Expand Down Expand Up @@ -449,8 +460,8 @@ Use this table in pull requests and add implementation-specific scenarios:
| V2 | Worker processing regressions remain hermetic | `make test-worker` | Exit `0`; real-model gate explicitly skipped |
| V3 | Localization, theme, enrichment, memory, transfer and managed-embedding control surfaces work in a browser | `make test-web` | Typecheck/lint/build, the localization audit, all browser acceptance suites and managed status mapping pass |
| V4 | High-risk Go paths are race-free | `make test-race` | Exit `0`; no data-race warning |
| V5 | Fresh schema, rollback and PostgreSQL semantics hold | `make test-integration` | Migration head and eighteen named tests pass, none skipped |
| V6 | DB concurrency paths are race-free | `make test-integration-race` | The same eighteen tests pass under `-race` |
| V5 | Fresh schema, rollback and PostgreSQL semantics hold | `make test-integration` | Migration head and all required named tests pass, none skipped |
| V6 | DB concurrency paths are race-free | `make test-integration-race` | The same required tests pass under `-race` |
| V7 | Real service boundaries agree | `make test-acceptance` | HTTP, CLI and MCP share one isolated service; memory citation/provenance, bounded checkpoint listing, full checkpoint get, lifecycle and forget redaction pass |
| V8 | Five config shapes and the real adapter preserve the host-neutral MCP contract | `MEM_MCP_CERT_BINARY=... make test-agent-certification` | All fixtures and current-adapter scenarios pass with no skip |
| V9 | Multilingual visual quality meets the chosen checkpoint | Opt-in command in section 7 | All fixed ranking assertions pass |
Expand Down
109 changes: 109 additions & 0 deletions docs/VAULT_SYNC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Workspace Vault sync v1

Implements [mem #234](https://github.com/bytefolk/mem/issues/234).

This additive text-note service keeps logical identity independent of the
existing `/v1/files` content-deduplication model. It never changes that model,
invokes AI, or treats note properties as permissions.

All endpoints use existing Bearer authentication and workspace membership.
`X-Workspace-ID`/a bound token selects the workspace. A token must also cover
the canonical path `/Vaults/<vaultId>`.

* `GET /v1/vault/list` (`read`) returns
`{ "vaults": [{ "vaultId", "title", "revision", "updatedAt" }] }`.
Out-of-path Vault IDs are omitted, without revealing their count.
* `GET /v1/vault/snapshot?vaultId=<UUID>` (`read`) returns a full snapshot.
A new Vault returns revision `0`, empty title, `updatedAt: null`, and `[]`
without creating a row.
* `POST /v1/vault/commit` (`read` + `write`) accepts
`{ "vaultId", "baseRevision", "title"?, "entries": [...] }`.
Entries are **deltas**, not a replacement of the full Vault. An entry is
`{ "noteId", "path", "content", "deleted"?, "properties"? }`.
Unlisted notes remain unchanged. `deleted: true` requires empty content.
Omitted properties retain existing metadata; a new note defaults to `{}`.
Empty entries are allowed only with a title update.

Both snapshot and successful commit return:

```json
{
"schemaVersion": "vault-snapshot.v1",
"vaultId": "aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa",
"title": "Knowledge",
"revision": 1,
"updatedAt": "2026-10-07T12:00:00Z",
"entries": [
{
"noteId": "bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb",
"path": "岗位/空白笔记.md",
"content": "",
"revision": 1,
"deleted": false,
"properties": {}
}
]
}
```

IDs are nonzero UUIDs. Vault IDs in requests use canonical lowercase form.
Paths are canonical relative POSIX paths up to 512 UTF-8 bytes, preserving
Chinese and spaces. Absolute paths, traversal, backslashes, control characters,
Windows-forbidden filename characters and trailing dots/spaces are rejected.
Two active notes cannot share a case-folded path. Logical notes with identical
content, including empty content, retain separate IDs. Text is UTF-8; only
newline, carriage return and tab control characters are accepted. Attachments
continue to use the existing mem file API and references rather than binary
data embedded in this text protocol.

One transaction locks and compares the whole Vault head. Successful commits
advance it exactly once and append content revisions; mutable head pointers
select current versions. Old entry revisions cannot be updated in place.
Delete creates a retained tombstone; it does not physically erase history.
Rename/swap changes logical paths while preserving note IDs.

* Stale head: `409 { "error": "vault_revision_conflict", "currentRevision": N }`.
* Active path collision: `409`, `error: vault_path_conflict`.
* Invalid body/path/identity: `400`, `error: vault_request_invalid`.
* Capacity/body/history limit: `413`, `error: vault_limit_exceeded`.
* Unavailable storage: `503`, `error: vault_unavailable`; success is not assumed.

Clients keep their own base snapshot and local manifest. On 409 they pull the
current head, merge nonoverlapping edits, and preserve conflicts as explicit
copies. A lost response is reconciled through snapshot readback, never a blind
overwrite. The service does not modify local files or promise a sync daemon.

Bounds: 4 MiB HTTP commit body, 1 MiB per note, 16 KiB per properties object,
256 UTF-8 bytes per title, 512 delta entries, 1000 heads **including tombstones**,
16 MiB current serialized content/path/properties, 64 MiB retained entry-history
content/path/properties, 10,000 commits and 64 Vaults per workspace. Limits
refuse atomically; snapshots are never silently truncated. A full snapshot may
require roughly 32 MiB on the wire due to JSON escaping; callers must use a
bounded Vault-specific response budget rather than an 8 MiB file-card budget.

Validation: unit/API tests are hermetic. `TestVaultSyncPostgres` requires an
explicit `MEM_TEST_DB` whose database name ends in `_test`; it verifies real
two-client races, distinct empty notes, workspace/auth isolation, tombstones,
immutable history, swaps, title CAS and additive migration down/up.

## Compatibility and portability boundary

The HTTP API is the canonical implementation. This initial additive surface
adds no CLI, MCP or standalone mem Web Vault editor; existing adapters and file
operations retain their contracts. Clients may synchronize ordinary Markdown
files and upload attachments through the existing file API.

The current workspace-bundle v1/v2 export/import formats do not contain Vault
heads or revision history. A Vault snapshot is a separate portable current-state
contract (docs/schemas/vault-snapshot.v1.schema.json). It has no auth tokens,
workspace membership or actor audit fields, and importing its properties never
grants permissions. Restoring a snapshot into an authorized target Vault uses
a normal CAS commit with the target head and a client-selected logical Vault
ID. Full history remains in PostgreSQL and requires an operator database backup;
this API does not claim to transport retained history between deployments.

Migration 0027 only creates new Vault tables and indexes. Downgrading to 0026
drops all Vault state and history, so preserve a database backup or current
snapshot before an operator downgrade. Existing files, auth and memory tables
are untouched. No login recovery, deployment credentials or model dependency
is introduced.
Loading
Loading