Skip to content

docs: add weaviate server upgrade path guide - #940

Open
dudanogueira wants to merge 3 commits into
langgenius:mainfrom
dudanogueira:docs/weaviate-server-upgrade-path
Open

docs: add weaviate server upgrade path guide#940
dudanogueira wants to merge 3 commits into
langgenius:mainfrom
dudanogueira:docs/weaviate-server-upgrade-path

Conversation

@dudanogueira

@dudanogueira dudanogueira commented Aug 12, 2026

Copy link
Copy Markdown

Summary

Adds self-host/deploy/troubleshooting/weaviate-server-migration-path, a sibling to the existing Weaviate v4 migration guide, covering how to move a self-hosted Weaviate server across minor versions.

The existing page ends at server 1.27 (its compatibility matrix reads "tested up to 1.33.1"). langgenius/dify#38214 moves Dify's bundled image to 1.39.0, and existing deployments cannot cross that gap in one step — Weaviate does not test or support skipping minor versions, and any release may carry an on-disk migration that expects the previous version to have run at least once. Without this page, a self-hosted user who pulls that upgrade has no documented path forward.

What the Page Covers

  • The one-minor-at-a-time ladder from 1.27 to 1.39, with the latest patch of each minor.
  • Graceful shutdown, which is the detail most likely to cost data. A hard kill leaves the HNSW commit log incomplete, and the resulting failure is quiet: object counts stay correct and fetch-by-ID returns exact vectors, but near_vector silently recalls only a subset until the index rebuilds. In testing, a hard kill between versions left a 10-object collection returning 3 results while the count still read 10.
  • The post-restart index-mount race, where /v1/meta answers before per-collection indexes finish mounting and queries fail with tried to browse non-existing index for Vector_index_<id>_Node.
  • Backup, per-rung verification, and rollback, plus what does not change (collection layout is identical, so no re-indexing; client 4.22.0 works across the whole ladder).

Notes for Review

  • Written English-first, then zh and ja in the same pass per AGENTS.md. Nav mirrored across all three docs.json sections.
  • Cross-linked both directions with weaviate-v4-migration so readers land on the right one: that page covers 1.19→1.27 and the schema migration, this one continues from 1.27.
  • The ladder was validated end-to-end on a seeded 1.27.0 volume through 1.38.2 — object count, byte-exact vectors, near_vector recall and BM25 checked at every rung, plus the hard-kill test that produced the recall failure described above. That run predates 1.39.0, whose release notes list no breaking changes.
  • Version numbers in the ladder are the latest patch of each minor as of 2026-08-12; the page tells readers to check for newer patches rather than treating the table as fixed.

Also Fixes the Stale Compatibility Matrix

The v4 page's matrix listed v4.17.0 for every release from 1.9.2 onward and capped the server column at "tested up to 1.33.1". Both were out of date, so the second commit corrects them against the actual dependency history:

Dify Version Client Landed in
1.9.2 - 1.13.0 v4.17.0 #25447
1.13.1 v4.20.4 #33502
1.14.0 - 1.16.x v4.20.5 #35021

4.22.0 landed on main in #39911 but is not in a tagged release yet, so it is deliberately not in the table.

The "tested up to" note is replaced rather than re-pinned: it was a moving target that goes stale on its own, so the page now explains why the server column has no upper bound and sends readers with an external Weaviate to the upgrade path.

The same commit also replaces a hardcoded pip install weaviate-client==4.17.0 in the troubleshooting section with uv sync --project api --group vdb-weaviate, so source installs get whatever version their checkout pins.

The existing v4 migration guide stops at server 1.27, but Dify now bundles
1.39 (langgenius/dify#38214). Existing deployments cannot jump that gap in
one step: Weaviate does not test or support skipping minor versions.

Adds a sibling page with the one-minor-at-a-time ladder and the two
operational details that decide whether vectors survive it — graceful
shutdown, which a hard kill silently costs you in HNSW recall, and the
post-restart index-mount race. Cross-links both directions with the v4 page.

en, zh and ja, with nav mirrored across all three sections.
@dudanogueira
dudanogueira requested a review from RiskeyL as a code owner August 12, 2026 22:22
Copilot AI lite review requested due to automatic review settings August 12, 2026 22:22
@dosubot dosubot Bot added size:XL This PR changes 500-999 lines, ignoring generated files. documentation Improvements or additions to documentation labels Aug 12, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new troubleshooting guide documenting the supported “one minor at a time” upgrade ladder for self-hosted Weaviate server deployments, bridging the gap from the previously documented 1.27-era guidance to Dify’s newer bundled Weaviate (1.39.0). It also cross-links this new guide from the existing Weaviate v4 migration guide and updates navigation across all three languages.

Changes:

  • Adds a new “Weaviate Server Upgrade Path” page in en/, plus zh/ and ja/ translations.
  • Cross-links the existing Weaviate v4 migration guide to route 1.27+ users to the new server upgrade ladder.
  • Updates docs.json navigation for en/zh/ja troubleshooting sections to include the new page.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
en/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx Adds an Info callout linking to the new server upgrade path guide.
en/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx New English guide describing the minor-by-minor upgrade ladder and operational pitfalls.
zh/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx Adds an Info callout linking to the new server upgrade path guide (zh).
zh/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx New Chinese translation of the server upgrade path guide.
ja/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx Adds an Info callout linking to the new server upgrade path guide (ja).
ja/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx New Japanese translation of the server upgrade path guide.
docs.json Adds the new page to the Troubleshooting nav for en/zh/ja.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread en/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx Outdated
Comment thread en/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx Outdated
Comment thread zh/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx Outdated
The matrix listed v4.17.0 for every release from 1.9.2 onward and capped
server coverage at "tested up to 1.33.1". Both are stale: the client moved
to 4.20.4 in 1.13.1 and 4.20.5 in 1.14.0, and the server bound was a moving
target that went out of date rather than a real limit.

Splits the client rows at their actual boundaries, replaces the tested-up-to
note with the reason the server column has no upper bound, and points readers
at the server upgrade path for external instances.

Also replaces the hardcoded `pip install weaviate-client==4.17.0` fix with
`uv sync`, so source installs get whatever version their checkout pins
instead of being pinned to a version Dify stopped shipping.
Copilot AI review requested due to automatic review settings August 12, 2026 22:29
Review feedback on langgenius#940:

- The warning said "twelve-step ladder" and the blast-radius note said
  "twelve-version jump" while the table lists thirteen rungs. Both counts
  are now phrased so they stay correct if the ladder changes.
- zh used 回合到 for backporting, which does not read as Chinese. Rewritten
  as 合入.

Applied to ja as well, where the same counts appeared as 10 段階 and
10 以上のバージョン.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (11)

en/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:58

  • This sentence refers to “bisecting a twelve-version jump”, but the exact number of rungs here is not 12 (the table includes a 1.27 patch step plus minors). Consider removing the hard-coded count to keep the wording accurate as the ladder changes.
Stepping one minor at a time also limits your blast radius. When a rung misbehaves, you roll back a single version and know exactly which release caused it, instead of bisecting a jump that crossed every minor at once.

en/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:57

  • The compatibility matrix uses v4.20.x, but Dify pins exact patch versions, and the PR description notes a patch-level change between 1.13.1 and 1.14.0+. Using x makes the table less precise than the surrounding rows (e.g., v4.17.0) and can mislead source installs trying to match the pinned client.
| Dify Version   | Weaviate-client Version | Compatible Weaviate Server Versions |
| -------------- | ----------------------- | ----------------------------------- |
| ≤ 1.9.1        | v3.x                    | 1.19.0 - 1.26.x                     |
| 1.9.2 - 1.13.0 | v4.17.0                 | 1.27.0 or higher                    |
| 1.13.1 - 1.16.x | v4.20.x                | 1.27.0 or higher                    |

en/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:14

  • In this warning, the text says “twelve-step ladder”, but the ladder below lists steps 0–13 (and may vary depending on a user's start version). Using a fixed number will become inaccurate as the table changes.

This issue also appears on line 58 of the same file.

<Warning>
  Back up your Weaviate volume before you start. An upgrade that goes wrong partway through the ladder is only recoverable if you have a snapshot to return to.
</Warning>

en/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:163

  • The phrase “Version 4.22.0, which current Dify ships” is ambiguous (and conflicts with the earlier matrix that stops at 1.16.x). To avoid going stale or implying a tagged release pin, it’s clearer to refer to “the version pinned by your Dify checkout” instead of a specific version number.
The Python `weaviate-client` also needs no attention. Version `4.22.0`, which current Dify ships, works against every server version on the ladder, so you can upgrade the server independently.

zh/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:59

  • 兼容性矩阵里用 v4.20.x 会掩盖 Dify 实际固定的是具体补丁版本(且 1.13.1 与 1.14.0+ 的补丁版本不同)。表格其它行使用了精确版本号(如 v4.17.0),建议这里也写到补丁级别以便源码安装对齐依赖。
| Dify 版本    | Weaviate-client 版本 | 兼容的 Weaviate 服务器版本        |
| ------------ | ----------------------- | ----------------------------------- |
| ≤ 1.9.1     | v3.x                | 1.19.0 - 1.26.x                |
| 1.9.2 - 1.13.0 | v4.17.0          | 1.27.0 或更高版本              |
| 1.13.1 - 1.16.x | v4.20.x         | 1.27.0 或更高版本              |

ja/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:59

  • 互換性マトリックスが v4.20.x になっていますが、Dify は実際にはパッチまで固定しており(1.13.1 と 1.14.0+ でパッチが異なる)、他の行(例: v4.17.0)は厳密な表記です。ソースインストールで依存を合わせやすいよう、ここもパッチまで明記した方が正確です。
| Dify バージョン | Weaviate-client バージョン | 互換性のある Weaviate サーバーバージョン |
| ------------ | ----------------------- | ----------------------------------- |
| ≤ 1.9.1      | v3.x                    | 1.19.0 - 1.26.x                     |
| 1.9.2 - 1.13.0 | v4.17.0               | 1.27.0 以降                         |
| 1.13.1 - 1.16.x | v4.20.x              | 1.27.0 以降                         |

zh/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:165

  • “当前 Dify 附带的 4.22.0” 这一表述容易过期,也可能被理解为“已发布版本”固定了该版本。为避免与前文兼容矩阵(截至 1.16.x)产生歧义,建议改为“使用当前 Dify 代码固定的版本”。
Python `weaviate-client` 同样无需处理。当前 Dify 附带的 `4.22.0` 可与阶梯上的所有服务器版本配合工作,因此服务器可以单独升级。

ja/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:165

  • 「現行の Dify が同梱する 4.22.0」は将来すぐに古くなる表現で、タグ付きリリースがそのバージョンを固定しているかのように読めてしまいます。前ページの互換表(〜1.16.x)とも整合しづらいので、「チェックアウトが固定しているバージョン」を参照する表現にすると誤解を減らせます。
Python の `weaviate-client` についても対応は不要です。現行の Dify が同梱する `4.22.0` は一覧のすべてのサーバーバージョンで動作するため、サーバーだけを個別にアップグレードできます。

en/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:636

  • The troubleshooting snippet runs Docker Compose and then uv sync --project api in the same code block, but it doesn't clarify the working directory. If a reader is in docker/ for the Compose commands (as elsewhere in this guide), --project api will resolve to docker/api and fail. Make the directory context explicit so copy/paste works.
docker compose down
docker compose up -d

# For source installations, install the version your Dify checkout pins
uv sync --project api --group vdb-weaviate

zh/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:637

  • 这段排障命令同时包含 Docker Compose 和 uv sync --project api,但没有说明工作目录。如果读者按常见做法在 docker/ 下运行 Compose,--project api 会解析为 docker/api 并失败。建议把目录切换写清楚,确保复制粘贴即可执行。
docker compose down
docker compose up -d

# 对于源码安装,安装当前 Dify 代码固定的版本
uv sync --project api --group vdb-weaviate

ja/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:637

  • このコードブロックは Docker Compose と uv sync --project api を並べていますが、実行ディレクトリが明示されていません。ガイド内の他箇所と同様に docker/ で Compose を実行した場合、--project apidocker/api を指して失敗します。ディレクトリ移動を明記してコピペで動くようにすると安全です。
docker compose down
docker compose up -d

# ソースインストールの場合、Dify のコードが固定しているバージョンをインストールします
uv sync --project api --group vdb-weaviate

Copilot AI review requested due to automatic review settings August 12, 2026 22:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (9)

en/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:126

  • The verification block also uses the redacted Authorization: ****** placeholder without stating what it represents. Add a one-line hint after defining KEY so users know to replace ****** with Bearer $KEY.
KEY="<WEAVIATE_API_KEY>"

zh/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:128

  • 这里定义了 KEY,但下面的 header 仍是 Authorization: ******,且未解释其含义。建议在 KEY 后补充一行注释,提示把 ****** 替换为 Bearer $KEY
KEY="<WEAVIATE_API_KEY>"

ja/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:128

  • KEY を定義していますが、下のヘッダーは Authorization: ****** のままで、何を入れるべきかが明記されていません。******Bearer $KEY に置き換える注釈を 1 行追加してください。
KEY="<WEAVIATE_API_KEY>"

en/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:636

  • uv sync --project api --group vdb-weaviate depends on running from the Dify repo root (so api/ resolves). Add an explicit cd /path/to/dify (or similar) so readers don’t run this from the wrong directory.
# For source installations, install the version your Dify checkout pins
uv sync --project api --group vdb-weaviate

zh/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:637

  • uv sync --project api --group vdb-weaviate 需要在 Dify 仓库根目录执行(否则找不到相对路径 api/)。建议在命令前明确 cd /path/to/dify,避免读者在错误目录下运行导致失败。
# 对于源码安装,安装当前 Dify 代码固定的版本
uv sync --project api --group vdb-weaviate

ja/self-host/deploy/troubleshooting/weaviate-v4-migration.mdx:637

  • uv sync --project api --group vdb-weaviate は Dify リポジトリのルートから実行しないと(相対パスの api/ が解決できず)失敗します。実行ディレクトリを明示する cd /path/to/dify を追加してください。
# ソースインストールの場合、Dify のコードが固定しているバージョンをインストールします
uv sync --project api --group vdb-weaviate

en/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:19

  • The curl snippets use a redacted Authorization: ****** value, but don’t explain what it should be replaced with. Add an explicit note inside the code block so readers know to supply a Bearer <WEAVIATE_API_KEY> value.

This issue also appears on line 125 of the same file.

```bash
curl -s -H "Authorization: Bearer <WEAVIATE_API_KEY>" \

zh/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:21

  • 代码块里使用了脱敏后的 Authorization: ******,但没有说明 ****** 需要替换成什么。建议在命令前加一行注释,明确应填 Bearer <WEAVIATE_API_KEY>,避免读者照抄失败。

This issue also appears on line 127 of the same file.

```bash
curl -s -H "Authorization: Bearer <WEAVIATE_API_KEY>" \

ja/self-host/deploy/troubleshooting/weaviate-server-migration-path.mdx:21

  • コードブロック内の Authorization: ****** が伏せ字のままなので、何に置き換えるべきかが分かりません。Bearer <WEAVIATE_API_KEY> を入れる旨の注釈を 1 行追加して、コピペで迷わないようにしてください。

This issue also appears on line 127 of the same file.

```bash
curl -s -H "Authorization: Bearer <WEAVIATE_API_KEY>" \

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size:XL This PR changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants