Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions .agents/docs/2026-10-04-add-taskflow-spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Taskflow 4.1.0 模块接入

## 范围与消费方式

以最小必要改动接入上游 CPU 模块,包名为 `taskflow.taskflow@4.1.0`,入口保持 `import tf;`。不提供独立 compat 包、CUDA feature 或自行维护的 API wrapper。

```toml
[dependencies.taskflow]
taskflow = "4.1.0"
```

消费者使用 `import std; import tf;`。模块只提供上游实际导出的 API,预处理宏不随模块导出,版本查询使用 `tf::version()`。模块与文本头文件混用不在本次验收范围内。

## 上游与包结构

- 上游:[taskflow/taskflow v4.1.0](https://github.com/taskflow/taskflow/releases/tag/v4.1.0),MIT。
- Tag commit:`45366fe5bc4f2f8ec9aa590b40c504e296886865`。
- 归档:`https://github.com/taskflow/taskflow/archive/refs/tags/v4.1.0.tar.gz`。
- SHA-256:`2107f90e315e48a676922010b036357ff2b0c6b9160ce17fa9396e5860b1d715`,独立下载摘要一致。
- 归档根目录为 `taskflow-4.1.0/`,没有符号链接;Linux、macOS、Windows 使用相同归档及校验值。
- 描述符为内联 Form B,采用 C++23,`import_std = false`,include 根为归档根目录。
- 显式编译 `tf.core.cppm`、`tf.algorithm.cppm`、`tf.utility.cppm`、`tf.cppm`;不通配 modules 目录,因为 `tf_with_cuda.cppm` 同样提供主模块 `tf`。
- 未配置 CN 镜像,使用纯字符串上游 URL;需要镜像时应上传相同归档字节。

该形态参考 `khronos.vulkan-hpp` 的上游模块复用方式,安装期适配参考 `compat.muduo`,Linux 线程链接配置参考 `chriskohlhoff.asio`。

## 必要适配

安装钩子逐项检查目标文件可读且替换位置恰好出现一次,不匹配立即失败。下载归档保持上游原样,适配只发生在解包后的三个模块文件中。

| 文件 | 适配 | 验证依据 |
|---|---|---|
| `modules/tf.core.cppm` | 移除 `HasGraph`、`ProfileData` 两条失效导出 | 4.1.0 头文件中不存在这两个名字,原模块在 Clang 下直接编译失败 |
| `modules/tf.core.cppm`、`modules/tf.cppm` | 将总头文件包含及版本函数导出移至 core,主接口先导入 core 再导入 algorithm | GCC 16.1.0 消费原模块时找不到 `tf::Executor`、`tf::Taskflow`;宿主 GCC 16.2.1 独立复现,组合调整后通过 |
| `modules/tf.utility.cppm` | 显式包含 `<algorithm>` | libc++ 下 `iterator.hpp` 使用的 `std::max` 缺少声明 |

逐文件摘要比较确认其余上游文件未改变,包括所有头文件及调度实现。没有独立 fork,也未修改 mcpp 引擎或现有 CI 配置。

Linux 最终链接使用 `-pthread`。不单独给依赖模块的编译命令增加该参数,否则 LLVM 消费者与 PCM 的线程配置不一致。实际构建命令确认链接参数到达消费者。

## 测试与索引登记

`tests/examples/taskflow-module` 通过单条 `taskflow` 索引重定向消费当前 checkout。一个测试 main 与一个辅助 TU 共同链接,覆盖:

- `import std; import tf;` 的多 TU 消费。
- 两个 worker 执行两条 DAG 分支并汇合,检查结果为 42。
- 1024 个元素的 `reduce`,检查结果为 1024。
- `next_pow2` 与 `tf::version()`。

断言失败返回非零退出码,运行时间由 mcpp 测试超时约束。成员已登记到根 workspace,中英文描述符目录已有对应条目。

## 验证结果

使用 CI 固定的 mcpp `2026.10.1.2`:

```sh
mcpp test -p taskflow-module --cache off --timeout 30
mcpp test -p taskflow-module --toolchain llvm@22.1.8 --cache off --timeout 30
mcpp xpkg parse --all-os pkgs/t/taskflow.taskflow.lua
```

- Linux GCC 16.1.0、LLVM 22.1.8/libc++:各 `1 passed; 0 failed`,GCC 增量运行通过。
- 冷安装、Lua 语法、三平台描述符解析通过。
- 镜像 URL、包身份、保留 namespace、全仓跨包引用、平台版本一致性、重复版本检查通过。
- workspace 与描述符变更均能由现有 CI 选择规则命中 `taskflow-module`。
- macOS、Windows 尚待 PR CI 构建和运行验证;描述符解析不代表运行验收。
1 change: 1 addition & 0 deletions docs/descriptor-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,4 @@ in the [root README](../README.md#reference-examples).
| Module layer over a compat source build (external Form-A repo) | [`godotengine.godot-cpp-m`](../pkgs/g/godotengine.godot-cpp-m.lua) (two versions tracking upstream: `10.0.0-rc1` = Godot 4.6, `4.5.0` = Godot 4.5. `import godot_cpp;` re-exports the whole `godot` namespace, ~1800 names GENERATED from the headers rather than curated; the 1022-TU build stays in `compat.godot-cpp`, so the index carries only this descriptor. Macros — `GDCLASS`, `GDREGISTER_CLASS`, `memnew`, `ERR_*` — are the one thing a named module cannot export, so the package ships a side header to include next to the import. It also ships a generated `hashfuncs.hpp` shim — upstream's header minus `static` on two functions whose bodies declare an unnamed union — without which GCC refuses the module interface outright, a hard error no `-W` flag reaches) |
| C++23 module wrapper | [`nlohmann.json`](../pkgs/n/nlohmann.json.lua) · [`marzer.tomlplusplus`](../pkgs/m/marzer.tomlplusplus.lua) · [`neargye.magic_enum`](../pkgs/n/neargye.magic_enum.lua) · [`boost-ext.ut`](../pkgs/b/boost-ext.ut.lua) (upstream's own `include/boost/ut.cppm` reproduced verbatim but for one `__argc`/`__argv` shim that Clang-on-MSVC needs; namespace `boost-ext` since it is NOT an official Boost library) |
| C++23 module, upstream's own unit | [`khronos.vulkan-hpp`](../pkgs/k/khronos.vulkan-hpp.lua) (Vulkan-Hpp 1.4.357.0 — Khronos generates `vulkan.cppm` / `vulkan_video.cppm` into every Vulkan-Headers release, so `sources` names the two units and NOTHING is authored here; the payload is the same tarball, URL and sha256 as `compat.vulkan-headers`, which is what makes the module and the headers it includes impossible to skew. `import_std = true` is forced by the unit's own unconditional `export import std;`. No `include_dirs`: the headers arrive with the `compat.vulkan` dependency, which is also what satisfies the STATIC dispatcher's direct calls at link — depending on headers alone gives a package that compiles and then fails at every consumer's link. Module names stay upstream's `vulkan` / `vulkan_video`, never `khronos.vulkan`. The second unit `import`s the first and mcpp orders the pair from the scan, which `tests/examples/vulkan-hpp-module/tests/video.cpp` is the regression for) |
| C++23 module, upstream's CPU partitions | [`taskflow.taskflow`](../pkgs/t/taskflow.taskflow.lua) (Taskflow 4.1.0, `import tf;`; reuses the four upstream CPU module units and omits the competing CUDA entry point. The checked install hook removes two nonexistent exports, moves the umbrella include/version export into core and imports core first for GCC, and supplies `<algorithm>` to utility for libc++. Only three module files are patched; headers and scheduler implementation stay upstream. No fork; consumers use `tf::version()` because macros are not exported) |
1 change: 1 addition & 0 deletions docs/zh/descriptor-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,4 @@
| 模块层叠在 compat 源码构建之上(外部 Form-A 仓) | [`godotengine.godot-cpp-m`](../../pkgs/g/godotengine.godot-cpp-m.lua)(两个版本与上游对齐:`10.0.0-rc1` 对应 Godot 4.6,`4.5.0` 对应 Godot 4.5。`import godot_cpp;` 重导出整个 `godot` 命名空间,约 1800 个名字由头文件**生成**而非手工罗列;1022 个 TU 的构建留在 `compat.godot-cpp`,索引侧只留这一个描述符。宏 —— `GDCLASS`、`GDREGISTER_CLASS`、`memnew`、`ERR_*` —— 是具名模块唯一带不走的东西,故包内附一个与 import 并排包含的侧头文件。另外还带一份生成的 `hashfuncs.hpp` 遮蔽头 —— 上游那个头去掉两个函数的 `static`(它们体内声明了匿名 union)—— 否则 GCC 直接拒绝该模块接口,且是任何 `-W` 开关都够不到的硬错误) |
| C++23 module wrapper | [`nlohmann.json`](../../pkgs/n/nlohmann.json.lua) · [`marzer.tomlplusplus`](../../pkgs/m/marzer.tomlplusplus.lua) · [`neargye.magic_enum`](../../pkgs/n/neargye.magic_enum.lua) · [`boost-ext.ut`](../../pkgs/b/boost-ext.ut.lua)(逐字复用上游自带的 `include/boost/ut.cppm`,仅加一处 Clang-on-MSVC 需要的 `__argc`/`__argv` shim;命名空间取 `boost-ext`,因其并非 boost 官方库) |
| C++23 module,上游自带单元 | [`khronos.vulkan-hpp`](../../pkgs/k/khronos.vulkan-hpp.lua)(Vulkan-Hpp 1.4.357.0 —— Khronos 把 `vulkan.cppm` / `vulkan_video.cppm` 生成进每个 Vulkan-Headers release,所以 `sources` 点名这两个单元即可,本仓**不写一行**包装体;载荷与 `compat.vulkan-headers` 是同一份 tarball、同一个 URL 与 sha256,这让模块与它 include 的头不可能错配。`import_std = true` 由单元自身无条件的 `export import std;` 决定。不声明 `include_dirs`:头随 `compat.vulkan` 依赖到达,而该依赖同时满足静态 dispatcher 在链接期的直接调用 —— 只依赖头会得到一个「能编译、每个消费者都链接失败」的包。模块名保持上游的 `vulkan` / `vulkan_video`,绝不写成 `khronos.vulkan`。第二个单元 `import` 第一个,顺序由 mcpp 扫描决定,`tests/examples/vulkan-hpp-module/tests/video.cpp` 就是这条的回归)|
| C++23 module,上游 CPU 分区 | [`taskflow.taskflow`](../../pkgs/t/taskflow.taskflow.lua)(Taskflow 4.1.0,`import tf;`;复用上游四个 CPU 模块单元,排除提供同名主模块的 CUDA 入口。安装钩子逐项校验匹配次数:移除两个不存在的导出,将总头文件及版本导出移至 core 并优先导入 core 以兼容 GCC,为 utility 补 `<algorithm>` 以兼容 libc++。仅适配三个模块文件,头文件和调度实现保持上游原样,无独立 fork;宏不随模块导出,版本查询使用 `tf::version()`) |
1 change: 1 addition & 0 deletions mcpp.toml
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ members = [
"tests/examples/spdlog-compiled",
"tests/examples/sqlite3",
"tests/examples/sqlitecpp",
"tests/examples/taskflow-module",
"tests/examples/tinyhttps",
"tests/examples/usockets",
"tests/examples/opencl",
Expand Down
86 changes: 86 additions & 0 deletions pkgs/t/taskflow.taskflow.lua
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
package = {
spec = "1",
namespace = "taskflow",
name = "taskflow",
description = "Taskflow CPU task graphs through the upstream tf C++ module",
licenses = { "MIT" },
repo = "https://github.com/taskflow/taskflow",
type = "package",

xpm = {
linux = {
["4.1.0"] = {
url = "https://github.com/taskflow/taskflow/archive/refs/tags/v4.1.0.tar.gz",
sha256 = "2107f90e315e48a676922010b036357ff2b0c6b9160ce17fa9396e5860b1d715",
},
},
macosx = {
["4.1.0"] = {
url = "https://github.com/taskflow/taskflow/archive/refs/tags/v4.1.0.tar.gz",
sha256 = "2107f90e315e48a676922010b036357ff2b0c6b9160ce17fa9396e5860b1d715",
},
},
windows = {
["4.1.0"] = {
url = "https://github.com/taskflow/taskflow/archive/refs/tags/v4.1.0.tar.gz",
sha256 = "2107f90e315e48a676922010b036357ff2b0c6b9160ce17fa9396e5860b1d715",
},
},
},

mcpp = {
language = "c++23",
import_std = false,
modules = { "tf" },
include_dirs = { "*" },
-- CUDA 入口同样提供 tf 主模块,与 CPU 入口不能同时编译
sources = {
"*/modules/tf.core.cppm",
"*/modules/tf.algorithm.cppm",
"*/modules/tf.utility.cppm",
"*/modules/tf.cppm",
},
targets = { ["taskflow"] = { kind = "lib" } },
deps = {},
linux = {
-- 模块编译选项须与消费者一致,线程依赖在最终链接时满足
ldflags = { "-pthread" },
},
},
}

import("xim.libxpkg.pkginfo")

function install()
local wrap = "taskflow-" .. pkginfo.version()
local function patch(relative, before, after)
local source = path.join(wrap, relative)
local content = assert(io.readfile(source), "taskflow.taskflow: cannot read " .. source)
local pattern = before:gsub("(%W)", "%%%1")
local patched, count = content:gsub(pattern, function() return after end)
assert(count == 1, "taskflow.taskflow: expected exactly one match in " .. source)
io.writefile(source, patched)
end

-- 4.1.0 的模块导出表包含两个在该版本头文件中不存在的名字
for _, name in ipairs({ "HasGraph", "ProfileData" }) do
patch("modules/tf.core.cppm", " using tf::" .. name .. ";\n", "")
end

-- 主接口的重复头声明会遮蔽 GCC 从分区导出的类型,版本声明随 core 导出
patch("modules/tf.core.cppm", "module;\n", "module;\n#include <taskflow/taskflow.hpp>\n")
patch("modules/tf.core.cppm", "export namespace tf {\n",
"export namespace tf {\n using tf::version;\n")
patch("modules/tf.cppm", "#include <taskflow/taskflow.hpp>\n", "")
patch("modules/tf.cppm", "export import :algorithm;\nexport import :core;",
"export import :core;\nexport import :algorithm;")

-- iterator.hpp 使用 std::max,但 libc++ 的间接包含不提供它
patch("modules/tf.utility.cppm", "module;\n", "module;\n#include <algorithm>\n")

local prefix = pkginfo.install_dir()
os.tryrm(prefix)
os.mkdir(prefix)
os.mv(wrap, path.join(prefix, wrap))
return true
end
10 changes: 10 additions & 0 deletions tests/examples/taskflow-module/mcpp.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# 本成员通过本地索引验证 taskflow.taskflow 的模块消费
[package]
name = "taskflow-module-tests"
version = "0.1.0"

[indices]
taskflow = { path = "../../.." }

[dependencies.taskflow]
taskflow = "4.1.0"
7 changes: 7 additions & 0 deletions tests/examples/taskflow-module/src/utility.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
import std;
import tf;

bool taskflow_utility_ok() {
return std::string_view(tf::version()) == "4.1.0"
&& tf::next_pow2(std::uint64_t{17}) == 32;
}
26 changes: 26 additions & 0 deletions tests/examples/taskflow-module/tests/taskflow.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import std;
import tf;

bool taskflow_utility_ok();

int main() {
tf::Executor executor(2);
tf::Taskflow flow;
int left = 0;
int right = 0;
int joined = 0;
auto a = flow.emplace([&] { left = 20; });
auto b = flow.emplace([&] { right = 22; });
auto c = flow.emplace([&] { joined = left + right; });
a.precede(c);
b.precede(c);
executor.run(flow).get();
if (joined != 42) return 1;

std::vector<int> values(1024, 1);
int sum = 0;
tf::Taskflow reduction;
reduction.reduce(values.begin(), values.end(), sum, std::plus<int>{});
executor.run(reduction).get();
return sum == 1024 && taskflow_utility_ok() ? 0 : 2;
}
Loading