feat: implement Cloudreve to AsterDrive migration#1
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
📝 WalkthroughWalkthrough新增 Cloudreve 到 AsterDrive 的数据库迁移引擎、CLI 命令、分阶段迁移、检查点恢复、结果验证、SQLite 集成测试及使用文档，并调整实体映射与依赖声明。 Changes数据库迁移工具
Estimated code review effort: 4 (Complex) | ~75 minutes Sequence Diagram(s)sequenceDiagram
participant CLI
participant MigrationEngine
participant CloudreveDB
participant AsterDriveDB
CLI->>MigrationEngine: 执行 check 或 migrate
MigrationEngine->>CloudreveDB: 加载源数据并检查兼容性
MigrationEngine->>AsterDriveDB: 校验 schema 与目标状态
MigrationEngine->>AsterDriveDB: 分阶段写入并保存 checkpoint
MigrationEngine-->>CLI: 输出 MigrationReport
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
src/migration/mod.rs (1)
254-263: 🚀 Performance & Scalability | 🔵 Trivial全量入内存 + 单事务写入，大库要留个心眼。
load对每张源表都.all(db)一次性拉进内存，随后所有阶段在同一个目标事务里写入。对小型实例没问题，但面对大规模 Cloudreve 部署，内存占用与目标库的长事务/锁持有时间都会成为瓶颈。若目标用户群里存在这种规模，建议按批分页读取与提交，或至少在文档里标注内存需求。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/migration/mod.rs` around lines 254 - 263, Update SourceData::load and the migration flow that consumes it to avoid loading every source table with unbounded .all(db) calls and keeping all writes in one long transaction. Read and migrate records in bounded batches with periodic commits, preserving migration ordering and include_deleted behavior; alternatively, document the required memory and transaction-size limitations if batching cannot be introduced.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/cloudreve-entities/src/entities.rs`:
- Line 7: 为 entities 实体补充 PostgreSQL schema 绑定，更新 `cr::entities::Entity` 上的
SeaORM 表属性以显式指定 Cloudreve 使用的 schema，避免依赖连接默认 schema；同时保持 `src/migration/mod.rs`
中通过 `Database::connect` 查询该实体时能正确解析非 public schema，并补充对应的 PostgreSQL 集成测试。
In `@crates/cloudreve-entities/src/files.rs`:
- Around line 7-8: 保留 files::Model 中的 deleted_at 字段，并在文件迁移查询流程中对 files 应用
filter_deleted，与其他实体保持一致；确保 include_deleted 为 false 时排除已删除文件，避免其迁移到目标库。
---
Nitpick comments:
In `@src/migration/mod.rs`:
- Around line 254-263: Update SourceData::load and the migration flow that
consumes it to avoid loading every source table with unbounded .all(db) calls
and keeping all writes in one long transaction. Read and migrate records in
bounded batches with periodic commits, preserving migration ordering and
include_deleted behavior; alternatively, document the required memory and
transaction-size limitations if batching cannot be introduced.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 6e369702-3c4b-4b9f-a6fa-4469913354bc
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (23)
Cargo.tomlREADME.mdcrates/cloudreve-entities/src/dav_accounts.rscrates/cloudreve-entities/src/direct_links.rscrates/cloudreve-entities/src/entities.rscrates/cloudreve-entities/src/file_entities.rscrates/cloudreve-entities/src/files.rscrates/cloudreve-entities/src/fs_events.rscrates/cloudreve-entities/src/groups.rscrates/cloudreve-entities/src/metadata.rscrates/cloudreve-entities/src/nodes.rscrates/cloudreve-entities/src/oauth_clients.rscrates/cloudreve-entities/src/oauth_grants.rscrates/cloudreve-entities/src/passkeys.rscrates/cloudreve-entities/src/settings.rscrates/cloudreve-entities/src/shares.rscrates/cloudreve-entities/src/storage_policies.rscrates/cloudreve-entities/src/tasks.rscrates/cloudreve-entities/src/users.rssrc/lib.rssrc/main.rssrc/migration/mod.rssrc/migration/phases.rs
| #[sea_orm::model] | ||
| #[derive(Clone, Debug, PartialEq, Eq, DeriveEntityModel)] | ||
| #[sea_orm(schema_name = "public", table_name = "entities")] | ||
| #[sea_orm(table_name = "entities")] |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -n 'search_path|schema_name|CLOUDREVE_DATABASE_URL' src crates Cargo.tomlRepository: AsterCommunity/AsterDriveMigration
Length of output: 236
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '\n== entities.rs ==\n'
cat -n crates/cloudreve-entities/src/entities.rs
printf '\n== migration/mod.rs ==\n'
cat -n src/migration/mod.rs
printf '\n== sea_orm / schema / search_path / database url references ==\n'
rg -n 'search_path|schema_name|Schema::|Database::connect|CLOUDREVE_DATABASE_URL|set_schema|with_schema|schema' src crates Cargo.tomlRepository: AsterCommunity/AsterDriveMigration
Length of output: 35564
🌐 Web query:
SeaORM #[sea_orm(schema_name = "...")] PostgreSQL default schema behavior table_name search_path
💡 Result:
In SeaORM, the #[sea_orm(schema_name = "...", table_name = "...")] attribute explicitly binds an entity to a specific PostgreSQL schema [1][2]. When schema_name is provided, SeaORM generates SQL queries that qualify the table name with the specified schema (e.g., "schema_name"."table_name") [3][4]. The interaction between schema_name and the PostgreSQL search_path is as follows: 1. Explicit Schema Binding: By using the schema_name attribute, you are telling SeaORM to always use that specific schema for the entity, regardless of the active search_path [3][4]. 2. PostgreSQL search_path: The search_path (configured via ConnectOptions::set_schema_search_path [5] or the database connection string [6]) is a PostgreSQL-level mechanism that tells the database which schemas to look into when an identifier is unqualified [6][7]. 3. Behavior: If you define an entity with a schema_name, SeaORM treats it as a fully qualified identifier. Consequently, the search_path has no effect on these specific entities because the query already includes the schema prefix [3]. If you do not specify a schema_name, SeaORM will not prefix the table, and PostgreSQL will rely on the search_path to resolve the table location [6]. Note that there are known edge cases, particularly with schema synchronization tools, where some generated SQL (such as for indexes or constraints) may not always consistently use the qualified schema name unless explicitly handled or configured [3]. For managing PostgreSQL schema configuration in SeaORM, you can use: - ConnectOptions: opt.set_schema_search_path("my_schema".into()) [5] - Connection String: postgres://user:pass@host/db?options[search_path]=my_schema [6]
Citations:
- 1: https://www.sea-ql.org/SeaORM/docs/generate-entity/entity-format/
- 2: https://www.sea-ql.org/SeaORM/docs/1.1.x/generate-entity/entity-structure/
- 3: schema-sync creates indexes against unqualified table names for entities in non-default PostgreSQL schemas SeaQL/sea-orm#3084
- 4: https://github.com/SeaQL/sea-orm/blob/63e6e491/sea-orm-macros/src/derives/entity_model.rs
- 5: https://github.com/SeaQL/sea-orm/blob/master/src/database/mod.rs
- 6: Use the
search_path=schema_namesuffix in PG connection string SeaQL/sea-orm#1272 - 7: https://www.sea-ql.org/sea-orm-cookbook/016-search-path.html
🌐 Web query:
SeaORM schema_name PostgreSQL default search_path public unqualified table name entity model
💡 Result:
In SeaORM, managing PostgreSQL schemas and unqualified table names is handled primarily through the schema_name attribute in the entity model and connection configuration [1][2][3]. To map an entity to a specific PostgreSQL schema, you use the schema_name attribute within the #[sea_orm(...)] derive macro [1][2]. This ensures the generated SQL appropriately qualifies the table name with the schema (e.g., "schema_name"."table_name") [4][2]. For connection-level handling, specifically when you want to avoid manual schema qualification or rely on the PostgreSQL search_path, you have two main options: 1. Connection Options: You can set the default PostgreSQL schema search path programmatically when establishing the database connection [5][6]. let mut opt = ConnectOptions::new("postgres://user:pass@localhost/db".to_owned); opt.set_schema_search_path("my_schema".into); let db = Database::connect(opt).await?; 2. Connection String: You can append the options[search_path] parameter to your connection string, which allows SQLx and SeaORM to honor the search path without requiring explicit schema qualification in your code [6]. postgres://user:pass@localhost/db?options[search_path]=my_schema Note that while schema_name explicitly defines the schema for an entity, relying on search_path can sometimes lead to issues with certain automated tools or generated queries (such as schema-sync creating indexes with unqualified table names if not handled correctly) [7][4]. If you explicitly define schema_name in your entity model, SeaORM typically generates schema-qualified identifiers [4][2]. If you require specific migration behaviors for non-public schemas, you can pass the -s <schema_name> flag to sea-orm-cli or set the DATABASE_SCHEMA environment variable [5][8].
Citations:
- 1: https://www.sea-ql.org/SeaORM/docs/generate-entity/entity-format/
- 2: https://github.com/SeaQL/sea-orm/blob/63e6e491/src/entity/base_entity.rs
- 3: https://www.sea-ql.org/SeaORM/docs/1.1.x/generate-entity/entity-format/
- 4: schema-sync creates indexes against unqualified table names for entities in non-default PostgreSQL schemas SeaQL/sea-orm#3084
- 5: Run migrations on PostgreSQL schema SeaQL/sea-orm#1056
- 6: Use the
search_path=schema_namesuffix in PG connection string SeaQL/sea-orm#1272 - 7: https://www.sea-ql.org/sea-orm-cookbook/016-search-path.html
- 8: [Migration] Create table with PostgreSQL schema name SeaQL/sea-orm#1029
补上 PostgreSQL schema 绑定
crates/cloudreve-entities/src/entities.rs:7 现在只有 table_name = "entities"；src/migration/mod.rs 里又是直接 Database::connect(url) 后跑 cr::entities::Entity::find().all(db)，没有任何 schema_name / search_path 配置。这样源表解析全靠连接默认 schema，非 public 部署会读错表或直接失败。要么显式固定 Cloudreve schema，要么把 search_path 配进连接串，再补 PostgreSQL 集成测试。
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/cloudreve-entities/src/entities.rs` at line 7, 为 entities 实体补充
PostgreSQL schema 绑定，更新 `cr::entities::Entity` 上的 SeaORM 表属性以显式指定 Cloudreve 使用的
schema，避免依赖连接默认 schema；同时保持 `src/migration/mod.rs` 中通过 `Database::connect`
查询该实体时能正确解析非 public schema，并补充对应的 PostgreSQL 集成测试。
| #[sea_orm(table_name = "files")] | ||
| pub struct Model { |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
不要移除 files.deleted_at，否则删除文件也会被迁移。
src/migration/mod.rs:235-285 对其他实体调用了 filter_deleted，但 files 直接使用查询结果；移除该字段后，include_deleted = false 无法过滤已删除文件，目标库会出现本应排除的记录。请保留字段，并对 files 同样应用 filter_deleted。
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/cloudreve-entities/src/files.rs` around lines 7 - 8, 保留 files::Model
中的 deleted_at 字段，并在文件迁移查询流程中对 files 应用 filter_deleted，与其他实体保持一致；确保
include_deleted 为 false 时排除已删除文件，避免其迁移到目标库。
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/migration/phases.rs`:
- Around line 455-491: 在迁移流程中更新标签关联逻辑：除现有的 tags 去重外，围绕
entity_properties::ActiveModel 插入增加按 (entity_type, entity_id, tag_id)
去重，确保同一实体复用同一 tag_id 时不会重复插入 system.tags 关联。仅首次出现该组合时插入并递增 migrated_properties 和
migrated_tag_assignments，重复关联应直接跳过。
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: d9f558bf-e792-4885-a64d-5de52a8434af
⛔ Files ignored due to path filters (1)
Cargo.lockis excluded by!**/*.lock
📒 Files selected for processing (7)
Cargo.tomlREADME.mddocs/cloudreve-to-asterdrive-field-mapping.mddocs/migration-architecture.mdsrc/main.rssrc/migration/mod.rssrc/migration/phases.rs
🚧 Files skipped from review as they are similar to previous changes (2)
- src/main.rs
- Cargo.toml
| let tag_id = match tags.get(&(owner_user_id, normalized_name.clone())) { | ||
| Some(tag_id) => *tag_id, | ||
| None => { | ||
| let tag = ad::tags::ActiveModel { | ||
| scope_type: Set("personal".to_string()), | ||
| owner_user_id: Set(Some(owner_user_id)), | ||
| team_id: Set(None), | ||
| name: Set(name), | ||
| normalized_name: Set(normalized_name.clone()), | ||
| color: Set(target_tag_color(&metadata.value)), | ||
| sort_order: Set(0), | ||
| created_at: Set(metadata.created_at), | ||
| updated_at: Set(metadata.updated_at), | ||
| ..Default::default() | ||
| } | ||
| .insert(transaction) | ||
| .await | ||
| .wrap_err_with(|| format!("migrate tag metadata {}", metadata.id))?; | ||
| tags.insert((owner_user_id, normalized_name), tag.id); | ||
| report.migrated_tags += 1; | ||
| tag.id | ||
| } | ||
| }; | ||
| ad::entity_properties::ActiveModel { | ||
| entity_type: Set(entity_type.to_string()), | ||
| entity_id: Set(entity_id), | ||
| namespace: Set("system.tags".to_string()), | ||
| name: Set(tag_id.to_string()), | ||
| value: Set(None), | ||
| ..Default::default() | ||
| } | ||
| .insert(transaction) | ||
| .await | ||
| .wrap_err_with(|| format!("attach migrated tag for metadata {}", metadata.id))?; | ||
| report.migrated_properties += 1; | ||
| report.migrated_tag_assignments += 1; | ||
| continue; |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# 确认 AD entity_properties 是否对 (entity_type, entity_id, namespace, name) 建唯一约束/索引
fd -t f . crates/asterdrive-entities/src | xargs rg -nP 'entity_properties|unique|index' -l
rg -nP -C3 'unique|index_name|primary_key' $(fd 'entity_propert' crates/asterdrive-entities/src)Repository: AsterCommunity/AsterDriveMigration
Length of output: 2670
标签关联也要去重，不然大小写变体会把迁移撞回去
tags 只去重了标签行，system.tags 这条 entity_properties 还是每条 metadata 都在插。entity_properties 对 (entity_type, entity_id, namespace, name) 有唯一键，同一文件上的 tag:Important 和 tag:important 会复用同一个 tag_id，第二次写入就直接撞唯一约束，整笔迁移回滚。这里得再按 (entity_type, entity_id, tag_id) 去重。
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/migration/phases.rs` around lines 455 - 491, 在迁移流程中更新标签关联逻辑：除现有的 tags
去重外，围绕 entity_properties::ActiveModel 插入增加按 (entity_type, entity_id, tag_id)
去重，确保同一实体复用同一 tag_id 时不会重复插入 system.tags 关联。仅首次出现该组合时插入并递增 migrated_properties 和
migrated_tag_assignments，重复关联应直接跳过。
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/main.rs (1)
24-26: 🔒 Security & Privacy | 🟠 Major | ⚡ Quick win给
source_url/target_url/default_password加上hide_env_values = true
src/main.rs:24-26,38-39这三个env参数会把已设置的值暴露到--help；direct_link_secret已经隐藏了，这里也该统一处理，避免泄露数据库连接串和默认密码。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/main.rs` around lines 24 - 26, Update the argument definitions for source_url, target_url, and default_password in the CLI arguments struct to include hide_env_values = true alongside their existing env settings. Match the existing direct_link_secret configuration and leave unrelated arguments unchanged.
🧹 Nitpick comments (2)
README.md (1)
59-59: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win补清
--allow-non-empty-target的边界说明。 README 现在只说它会跳过空库保护，但没说清楚它不是合并模式：正常迁移还是一堆insert，已有记录不会被覆盖，撞上主键/唯一约束就会失败。--resume也只是同一run_id的中断续跑，不是“对非空目标重复执行”的通用语义。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@README.md` at line 59, 完善 README 中 --allow-non-empty-target 的说明，明确它仅跳过目标核心表必须为空的保护，不是合并或覆盖模式；迁移仍执行插入，已有记录不会被覆盖，遇到主键或唯一约束冲突可能失败。同时说明 --resume 仅用于续跑同一 run_id 的中断迁移，不支持对非空目标重复执行。src/main.rs (1)
86-94: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
emit_report建议先打印再写文件，并修正 bail 消息。当前顺序是先写文件再打印，若
write_json_report失败，用户在 stdout 上什么都看不到。建议调换顺序，让用户始终能看到报告内容。此外，行 92 的 bail 消息说 "inspect the JSON report"，但如果用户未提供--report-path，根本没有 JSON 文件可查，消息具有误导性。♻️ 建议修复：调换顺序并修正消息
fn emit_report(report_path: Option<&Path>, report: &MigrationReport) -> Result<()> { - if let Some(path) = report_path { - write_json_report(path, report)?; - } println!("{report}"); + if let Some(path) = report_path { + write_json_report(path, report)?; + } if report.validation.performed && !report.validation.passed { - bail!("migration committed but post-migration validation failed; inspect the JSON report"); + bail!("migration committed but post-migration validation failed; review the report above{}", + report_path.map(|_| "").unwrap_or(" (re-run with --report-path to save a JSON report)")); } Ok(()) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/main.rs` around lines 86 - 94, Update emit_report to print the MigrationReport before calling write_json_report, ensuring report output appears even when file writing fails. Also revise the validation-failure bail message so it does not imply a JSON report exists when no report_path was provided, while retaining the existing failure behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/cloudreve-to-asterdrive-field-mapping.md`:
- Around line 520-538: 更新“JSON 报告字段”表，补充 MigrationReport 中缺失的
schema_version、generated_at、dry_run、warnings 和 skipped 字段说明；保留现有 skipped_by_type
与 skipped_objects 行，并确保表格完整覆盖报告结构体中的字段。
---
Outside diff comments:
In `@src/main.rs`:
- Around line 24-26: Update the argument definitions for source_url, target_url,
and default_password in the CLI arguments struct to include hide_env_values =
true alongside their existing env settings. Match the existing
direct_link_secret configuration and leave unrelated arguments unchanged.
---
Nitpick comments:
In `@README.md`:
- Line 59: 完善 README 中 --allow-non-empty-target
的说明，明确它仅跳过目标核心表必须为空的保护，不是合并或覆盖模式；迁移仍执行插入，已有记录不会被覆盖，遇到主键或唯一约束冲突可能失败。同时说明 --resume
仅用于续跑同一 run_id 的中断迁移，不支持对非空目标重复执行。
In `@src/main.rs`:
- Around line 86-94: Update emit_report to print the MigrationReport before
calling write_json_report, ensuring report output appears even when file writing
fails. Also revise the validation-failure bail message so it does not imply a
JSON report exists when no report_path was provided, while retaining the
existing failure behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3dfb2848-af0d-46c8-860a-eb1d697fc4f2
📒 Files selected for processing (7)
README.mddocs/cloudreve-to-asterdrive-field-mapping.mddocs/migration-architecture.mdsrc/main.rssrc/migration/checkpoint.rssrc/migration/mod.rssrc/migration/phases.rs
🚧 Files skipped from review as they are similar to previous changes (3)
- docs/migration-architecture.md
- src/migration/mod.rs
- src/migration/phases.rs
| ## 16. JSON 报告字段 | ||
|
|
||
| 通过 `--report-path <path>` 可为 `check` 或 `migrate` 输出 `schema_version=1` 的 JSON 报告。 | ||
|
|
||
| | 报告字段 | 内容 | | ||
| |---|---| | ||
| | `source_*` | Cloudreve 各类源对象数量 | | ||
| | `migrated_*` | 本次写入 AD 的各类对象数量 | | ||
| | `skipped_by_type` | 按 `file`、`blob`、`share`、`direct_link` 等类型聚合的跳过数量 | | ||
| | `skipped_objects` | 每条跳过记录的对象类型、Cloudreve source ID 和明确原因 | | ||
| | `mappings` | policy、policy group、user、folder、blob、file、share、task 的排序 source ID -> target ID | | ||
| | `tag_assignments` | Cloudreve metadata ID、源 file/folder ID、AD entity ID、AD tag ID 和标签名 | | ||
| | `direct_links` | Cloudreve direct-link/file ID、AD file ID、新 URL、原名称、下载次数和限速 | | ||
| | `validation` | 是否执行/通过，以及每项检查的 expected、actual 和失败信息 | | ||
| | `run_id` | checkpoint run ID；未指定时由迁移工具生成 UUID | | ||
| | `resumed` | 本次执行是否从已有 checkpoint 恢复 | | ||
| | `completed_stages` | 已原子提交完成的迁移阶段列表 | | ||
|
|
||
| 当前提交后校验覆盖核心表增量数量、导入任务是否全为终态且无 lease、`system.tags` 关联是否存在、`cloudreve.direct_links` 属性中的 URL 是否与报告一致。报告不会保存数据库密码、存储密钥或 Cloudreve task private state，但会包含新 direct-link URL，因此必须限制报告文件访问权限。 |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
JSON 报告字段表漏了五个字段。
对照 MigrationReport 结构体，schema_version（正文提到但未列入表格）、generated_at、dry_run、warnings、skipped（总跳过数，表里只列了 skipped_by_type 和 skipped_objects）都没出现在表格中。用户照着文档解析报告会漏读这些字段。
📝 建议补充的表格行
| 报告字段 | 内容 |
|---|---|
+| `schema_version` | 报告 schema 版本，当前为 `1` |
+| `generated_at` | 报告生成时间（UTC） |
| `source_*` | Cloudreve 各类源对象数量 |
| `migrated_*` | 本次写入 AD 的各类对象数量 |
+| `skipped` | 跳过对象总数 |
+| `dry_run` | 是否为 dry-run 模式 |
+| `warnings` | 迁移过程中产生的警告信息列表 |
| `skipped_by_type` | 按 `file`、`blob`、`share`、`direct_link` 等类型聚合的跳过数量 |
| `skipped_objects` | 每条跳过记录的对象类型、Cloudreve source ID 和明确原因 |
| `mappings` | policy、policy group、user、folder、blob、file、share、task 的排序 source ID -> target ID |
| `tag_assignments` | Cloudreve metadata ID、源 file/folder ID、AD entity ID、AD tag ID 和标签名 |
| `direct_links` | Cloudreve direct-link/file ID、AD file ID、新 URL、原名称、下载次数和限速 |
| `validation` | 是否执行/通过，以及每项检查的 expected、actual 和失败信息 |
| `run_id` | checkpoint run ID；未指定时由迁移工具生成 UUID |
| `resumed` | 本次执行是否从已有 checkpoint 恢复 |
| `completed_stages` | 已原子提交完成的迁移阶段列表 |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| ## 16. JSON 报告字段 | |
| 通过 `--report-path <path>` 可为 `check` 或 `migrate` 输出 `schema_version=1` 的 JSON 报告。 | |
| | 报告字段 | 内容 | | |
| |---|---| | |
| | `source_*` | Cloudreve 各类源对象数量 | | |
| | `migrated_*` | 本次写入 AD 的各类对象数量 | | |
| | `skipped_by_type` | 按 `file`、`blob`、`share`、`direct_link` 等类型聚合的跳过数量 | | |
| | `skipped_objects` | 每条跳过记录的对象类型、Cloudreve source ID 和明确原因 | | |
| | `mappings` | policy、policy group、user、folder、blob、file、share、task 的排序 source ID -> target ID | | |
| | `tag_assignments` | Cloudreve metadata ID、源 file/folder ID、AD entity ID、AD tag ID 和标签名 | | |
| | `direct_links` | Cloudreve direct-link/file ID、AD file ID、新 URL、原名称、下载次数和限速 | | |
| | `validation` | 是否执行/通过，以及每项检查的 expected、actual 和失败信息 | | |
| | `run_id` | checkpoint run ID；未指定时由迁移工具生成 UUID | | |
| | `resumed` | 本次执行是否从已有 checkpoint 恢复 | | |
| | `completed_stages` | 已原子提交完成的迁移阶段列表 | | |
| 当前提交后校验覆盖核心表增量数量、导入任务是否全为终态且无 lease、`system.tags` 关联是否存在、`cloudreve.direct_links` 属性中的 URL 是否与报告一致。报告不会保存数据库密码、存储密钥或 Cloudreve task private state，但会包含新 direct-link URL，因此必须限制报告文件访问权限。 | |
| ## 16. JSON 报告字段 | |
| 通过 `--report-path <path>` 可为 `check` 或 `migrate` 输出 `schema_version=1` 的 JSON 报告。 | |
| | 报告字段 | 内容 | | |
| |---|---| | |
| | `schema_version` | 报告 schema 版本，当前为 `1` | | |
| | `generated_at` | 报告生成时间（UTC） | | |
| | `source_*` | Cloudreve 各类源对象数量 | | |
| | `migrated_*` | 本次写入 AD 的各类对象数量 | | |
| | `skipped` | 跳过对象总数 | | |
| | `dry_run` | 是否为 dry-run 模式 | | |
| | `warnings` | 迁移过程中产生的警告信息列表 | | |
| | `skipped_by_type` | 按 `file`、`blob`、`share`、`direct_link` 等类型聚合的跳过数量 | | |
| | `skipped_objects` | 每条跳过记录的对象类型、Cloudreve source ID 和明确原因 | | |
| | `mappings` | policy、policy group、user、folder、blob、file、share、task 的排序 source ID -> target ID | | |
| | `tag_assignments` | Cloudreve metadata ID、源 file/folder ID、AD entity ID、AD tag ID 和标签名 | | |
| | `direct_links` | Cloudreve direct-link/file ID、AD file ID、新 URL、原名称、下载次数和限速 | | |
| | `validation` | 是否执行/通过，以及每项检查的 expected、actual 和失败信息 | | |
| | `run_id` | checkpoint run ID；未指定时由迁移工具生成 UUID | | |
| | `resumed` | 本次执行是否从已有 checkpoint 恢复 | | |
| | `completed_stages` | 已原子提交完成的迁移阶段列表 | | |
| 当前提交后校验覆盖核心表增量数量、导入任务是否全为终态且无 lease、`system.tags` 关联是否存在、`cloudreve.direct_links` 属性中的 URL 是否与报告一致。报告不会保存数据库密码、存储密钥或 Cloudreve task private state，但会包含新 direct-link URL，因此必须限制报告文件访问权限。 |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/cloudreve-to-asterdrive-field-mapping.md` around lines 520 - 538,
更新“JSON 报告字段”表，补充 MigrationReport 中缺失的
schema_version、generated_at、dry_run、warnings 和 skipped 字段说明；保留现有 skipped_by_type
与 skipped_objects 行，并确保表格完整覆盖报告结构体中的字段。
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
src/migration/phases.rs (1)
300-314: 🚀 Performance & Scalability | 🔵 Trivial | 🏗️ Heavy lift文件迁移阶段全量加载 entities 和 file_entities 到内存
migrate_files一次性把源库的entities和file_entities全部拉进内存。blob 阶段已经做了分批处理，文件阶段却没有——大型 Cloudreve 实例（百万级文件 + 千万级 entities）跑下来直接 OOM 不是开玩笑的。建议参照
migrate_blobs_batched的分批策略，按 file ID 区间分批加载关联数据，避免单次查询撑爆内存。🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/migration/phases.rs` around lines 300 - 314, Update migrate_files to avoid loading all entities and file_entities at once: mirror migrate_blobs_batched’s batching strategy and query source records in file-ID ranges, processing and releasing each batch before fetching the next. Preserve the include_deleted filtering and existing migration/report behavior while ensuring memory usage remains bounded for large databases.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/migration/phases.rs`:
- Around line 352-355: 在计算 primary_entity_id 的回退逻辑中，保留 file.primary_entity 存在但不在
blob_mappings 时回退到 version_entities.last() 的行为，并在该回退发生时记录一条 warning，明确说明原
primary entity 无对应 blob 且当前版本已被替换。修改 primary_entity_id 附近的处理，避免无效或不存在的
primary_entity 触发无关警告。
---
Nitpick comments:
In `@src/migration/phases.rs`:
- Around line 300-314: Update migrate_files to avoid loading all entities and
file_entities at once: mirror migrate_blobs_batched’s batching strategy and
query source records in file-ID ranges, processing and releasing each batch
before fetching the next. Preserve the include_deleted filtering and existing
migration/report behavior while ensuring memory usage remains bounded for large
databases.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 92195ceb-a99d-4b32-88b6-ae1c23925b36
📒 Files selected for processing (7)
README.mddocs/cloudreve-to-asterdrive-field-mapping.mddocs/migration-architecture.mdsrc/main.rssrc/migration/checkpoint.rssrc/migration/mod.rssrc/migration/phases.rs
🚧 Files skipped from review as they are similar to previous changes (5)
- src/main.rs
- docs/cloudreve-to-asterdrive-field-mapping.md
- docs/migration-architecture.md
- README.md
- src/migration/mod.rs
| let primary_entity_id = file | ||
| .primary_entity | ||
| .filter(|id| blob_mappings.contains_key(id)) | ||
| .or_else(|| version_entities.last().map(|entity| entity.id)); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
primary entity 回退时静默吞掉了信息
file.primary_entity 存在但不在 blob_mappings 中时，代码直接回退到 version_entities.last()，没记录任何 warning。用户根本不知道这个文件的"当前版本"被换了——版本语义可能就此错乱。
🔧 建议在回退时补一条 warning
- let primary_entity_id = file
- .primary_entity
- .filter(|id| blob_mappings.contains_key(id))
- .or_else(|| version_entities.last().map(|entity| entity.id));
+ let primary_entity_id = match file.primary_entity {
+ Some(id) if blob_mappings.contains_key(&id) => Some(id),
+ Some(id) => {
+ report.warnings.push(format!(
+ "file {} primary entity {} was not migrated, falling back to latest version",
+ file.id, id
+ ));
+ version_entities.last().map(|entity| entity.id)
+ }
+ None => version_entities.last().map(|entity| entity.id),
+ };📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| let primary_entity_id = file | |
| .primary_entity | |
| .filter(|id| blob_mappings.contains_key(id)) | |
| .or_else(|| version_entities.last().map(|entity| entity.id)); | |
| let primary_entity_id = match file.primary_entity { | |
| Some(id) if blob_mappings.contains_key(&id) => Some(id), | |
| Some(id) => { | |
| report.warnings.push(format!( | |
| "file {} primary entity {} was not migrated, falling back to latest version", | |
| file.id, id | |
| )); | |
| version_entities.last().map(|entity| entity.id) | |
| } | |
| None => version_entities.last().map(|entity| entity.id), | |
| }; |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/migration/phases.rs` around lines 352 - 355, 在计算 primary_entity_id
的回退逻辑中，保留 file.primary_entity 存在但不在 blob_mappings 时回退到 version_entities.last()
的行为，并在该回退发生时记录一条 warning，明确说明原 primary entity 无对应 blob 且当前版本已被替换。修改
primary_entity_id 附近的处理，避免无效或不存在的 primary_entity 触发无关警告。
Summary by CodeRabbit
check输出兼容性预检报告；migrate分阶段迁移，并支持--report-path、--run-id、--resume、可选包含已删除数据、非空目标策略与跳过不支持的存储策略。