From 813c47648b585c006bd612695559544583dfcd99 Mon Sep 17 00:00:00 2001 From: Songhao Jia Date: Thu, 24 Sep 2026 01:58:11 -0700 Subject: [PATCH] Ship linkable libraries in the Windows wheel The Windows wheel shipped one fused Python extension and no linkable libraries, so a C++ application got nothing from it: the headers and the CMake package were there, but every component a consumer asked for resolved to nothing. Linux and macOS already ship the runtime, the kernels, the delegates and the thread pool as separate libraries that a C++ application links directly (#21610, #21771). This does the same for Windows. Before (Windows x64, CPython 3.12): extension/pybindings/_C.cp312-win_amd64.pyd 8.27 MB <- everything fused in here lib/ does not exist After: lib/executorch_kernels_optimized.dll 5.02 MB lib/executorch_backend_xnnpack.dll 2.47 MB lib/executorch.dll 0.38 MB lib/executorch_kernels_quantized.dll 0.22 MB lib/executorch_threadpool.dll 0.21 MB lib/executorch_etdump.dll 0.05 MB (loaded by _C; not offered to C++) lib/*.lib <- import libraries a consumer links extension/pybindings/_C.cp312-win_amd64.pyd 0.63 MB <- just the bindings now ## How The three mechanisms that made the shared layout Linux and macOS only now have Windows equivalents: what ELF / Mach-O Windows exporting the runtime API default visibility WINDOWS_EXPORT_ALL_SYMBOLS keeping a registration-only lib --no-as-needed / named dylib /INCLUDE of a per-DLL anchor finding sibling libraries $ORIGIN / @loader_path os.add_dll_directory, and a consumer copies the DLLs - Exports. The runtime has no dllexport annotations, which is why CMakeLists.txt refused a shared Windows build. Each shipped component DLL now exports all of its symbols, unless its target already exports through its own annotations (WINDOWS_EXPORT_ALL_SYMBOLS OFF), which it keeps. CMake builds that list from a target's own objects and cannot read an archive, so the runtime DLL takes its components as objects rather than through /WHOLEARCHIVE. The caller says which archives are part of a DLL's exports (the helper's EXPORTED option) instead of the helper inferring it from a property another call sets, so the result does not depend on call order. That uses $, new in CMake 3.27, so a shared Windows build asks for 3.27 with a clear message, the Windows build requirement is raised to match, and executorch_shared publishes the cxx_std_20 its headers need on Windows. The PAL source is built with its C functions strong there: the export list skips weak symbols, so the DLL exported none of the et_pal_* functions and clock.h's inline ticks_to_ns() failed to link against it. - Retention. A PE import survives only if some symbol from it is referenced, so each shipped DLL exports `executorch_anchor_` and the imported targets carry `/INCLUDE` of it, the counterpart of the Linux `--no-as-needed`. The name follows the shipped file name without any configuration postfix, and is resolved at the end of configure and written out literally, so it survives `cmake --install` (a generator expression would be evaluated against the imported target, which has no OUTPUT_NAME) and is the same in every configuration of a multi-config build. - One registry. A consumer names the runtime's import library ahead of any archive, as on Linux, so nothing resolves the registry from a private static copy. Without this the optimized kernels registered into their own table: 28 operators visible instead of 242. - Loading. A DLL records no search path. The Python entry points that load a DLL depending on executorch/lib register that directory first (portable_lib, kernels.quantized, codegen.tools, llm.custom_ops.op_tile_crop_aot), computed without resolving symlinks so an editable install, where those directories are links, registers its own lib too. A C++ consumer copies the DLLs beside its executable with `$`, which the C++ guide now shows, including in the quick start. - Debug. The DLLs use the C++ library of the configuration the wheel was built in, and a consumer in the other configuration would mix the two and corrupt memory. The package defines ET_PREBUILT_RELEASE_CRT, or ET_PREBUILT_DEBUG_CRT for a DEBUG=1 build, and the runtime headers refuse the mismatched configuration at compile time with a message naming the right one, for both single-config (-DCMAKE_BUILD_TYPE) and multi-config (--config) generators. naming the right one. (A /FAILIFMISMATCH link option would not do: link.exe ignores it on the command line and honours it only inside an object file.) - cpuinfo. The thread pool DLL exports pthreadpool, so the process has one pool, but keeps cpuinfo private. cpuinfo's feature checks (cpuinfo_has_x86_avx2 and the rest) are inline reads of cpuinfo_isa, which cpuinfo.h does not declare dllimport, so a DLL can only read its own copy; with cpuinfo exported, XNNPACK initialized the thread pool's copy and read its own, all zeros, and ran baseline kernels. Each DLL that uses cpuinfo now carries all of it. Measured on an x64 machine with AVX2: an XNNPACK-delegated 2 x Linear(1024) model at batch 256 went from 2.05-2.14 ms to 1.47-1.52 ms per run, with the same results. CUDA is not part of this. A Windows build with CUDA on keeps the static layout it had, since the Windows CUDA delegate has not been built as a DLL; a follow-up stacked on this ships it. Two bugs only the shared layout exposes on Windows are fixed here: - The process hung at exit about half the time. Windows terminates worker threads before running a DLL's static destructors, so destroying the global thread pool there waited on a lock a terminated worker could hold (`LdrShutdownProcess -> pthreadpool_destroy -> mtx_lock`). The pool is now deliberately not destroyed on Windows; Linux and macOS are unchanged. - `quantized_ops_aot_lib.dll` failed to load, which `executorch.kernels.quantized` swallows, so quantized export silently lost its out variants. The Windows wheel is built without the event tracer (unchanged), so the `etdump` component is not offered to C++ there rather than handing a consumer a profiler that records nothing. The delegates are unchanged too: the Windows wheel ships XNNPACK, as it did before this change. QNN, OpenVINO and TorchAO stay Linux (and for TorchAO, aarch64) wheel components, and Core ML and MLX macOS ones. QNN in particular builds on Windows (build-qnn-windows-x64 and -arm64 pass), but the wheel enables it only where pre_build_script.sh downloads the SDK and the pybind preset's Linux branch turns it on; bringing it to the Windows wheel means the SDK download on the Windows builder, its import library and anchor, and tests, which is a separate change. The pkg-config file the Linux and macOS wheels ship (#23169) is not generated on Windows either, since its link flags are GNU-style and a DLL records no search path for them; the test checks that it is absent. The pre-3.28 variables route links the import libraries and states C++20, which the Windows runtime headers need. ## Tests The two wheel suites now run on Windows, as they do on macOS since #21771, wired into test_windows.py. Their Windows forms ask the platform's own questions: - symbols: dumpbin /exports matched against MSVC decorated names, and for code a DLL takes from an archive, which the export list cannot name, the import direction: the owner must import `register_kernels` / `register_backend` from executorch.dll; - dependencies: dumpbin /dependents and /imports in place of readelf / otool; - loading: LoadLibrary per binary in its own process, which binds every import as ldd -r does, from the package and from a relocated copy; - paths: every recorded dependency is a bare DLL name; - platform tag: the PE machine field of every binary against win_amd64; - C++ consumers: built with --config Release, DLLs copied beside the executable, and the installed package removed from PATH so nothing resolves it for them; - package entry points: the extensions are imported the way a user reaches them (the pybindings through portable_lib), with nothing added to the DLL search path, and `import executorch.kernels.quantized` has to register the quantized out variants, so the entry points' own registration is what is tested; - a consumer in the wrong configuration has to be refused; an application using the thread pool has to exit five runs in a row; no DLL may import cpuinfo from another (the split state has no numeric symptom, only slower kernels); every program the suites build and every Python probe or export they start has a timeout, so a hang fails as a named timeout rather than the whole job timing out. Compiler, CMake and binary tool invocations are not bounded. ## Test plan On Windows 11 x64, VS 2022 BuildTools with ClangCL, CPython 3.12, from a wheel built with `python setup.py bdist_wheel` and installed into a clean environment: - `.ci/scripts/wheel/test_windows.py`: passes, including every check in test_shared_libraries.py and test_cpp_sdk.py (single owner of each component, _C contains no component, every DLL loads in place and relocated, custom op registers, platform tag; version request, 121 of 123 headers compile, documented example builds, runtime alone has no kernels, kernels / quantized / XNNPACK consumers match eager PyTorch, delegate fails without its component, pre-3.28 route on CMake 3.24, relocation, one shared registry) and MobileNetV3 through XNNPACK matching eager. - The exit hang: 8 of 20 and 10 of 20 runs of two consumers hung before the thread pool fix, 0 of 20 after. - Linux and macOS: the CMake changes are behind WIN32 / MSVC and the Python changes behind sys.platform == "win32"; relying on their wheel rows for confirmation. --- .ci/scripts/wheel/test_cpp_sdk.py | 381 +++++++++++-- .ci/scripts/wheel/test_shared_libraries.py | 550 ++++++++++++++++++- .ci/scripts/wheel/test_windows.py | 14 + CMakeLists.txt | 62 ++- README-wheel.md | 4 + codegen/tools/__init__.py | 20 + docs/source/using-executorch-cpp.md | 62 ++- extension/llm/custom_ops/op_tile_crop_aot.py | 16 + extension/pybindings/portable_lib.py | 5 + extension/threadpool/CMakeLists.txt | 11 +- extension/threadpool/threadpool.cpp | 8 + kernels/quantized/__init__.py | 15 + pyproject.toml | 3 +- requirements-dev.txt | 3 +- runtime/platform/compiler.h | 14 + runtime/platform/default/posix.cpp | 14 +- setup.py | 95 +++- tools/cmake/Utils.cmake | 114 +++- tools/cmake/executorch-wheel-config.cmake | 92 +++- tools/cmake/preset/default.cmake | 3 +- tools/cmake/preset/pybind.cmake | 11 +- 21 files changed, 1377 insertions(+), 120 deletions(-) diff --git a/.ci/scripts/wheel/test_cpp_sdk.py b/.ci/scripts/wheel/test_cpp_sdk.py index 02110ee70d1..176b1aedb5d 100644 --- a/.ci/scripts/wheel/test_cpp_sdk.py +++ b/.ci/scripts/wheel/test_cpp_sdk.py @@ -25,6 +25,28 @@ import tempfile from pathlib import Path +_WINDOWS = sys.platform == "win32" + +# Long enough for an honest run to load a model, short enough that an application which +# never exits fails as a named timeout inside the job's limit rather than hanging the job. +_RUN_TIMEOUT = 300 +# An export traces and lowers a model, which is slower than running one. +_EXPORT_TIMEOUT = 900 + +# Windows has no runtime search path, so an application finds the shipped DLLs only +# beside itself. This is the copy step the documentation tells a Windows consumer to add, +# appended to each consumer project so the checks exercise the documented setup. +_WINDOWS_DLL_COPY = """ +if(WIN32) + add_custom_command( + TARGET consumer POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different $ + $ + COMMAND_EXPAND_LISTS + ) +endif() +""" + # Exports the model to a .pte and prints the reference outputs, so the C++ side can # be compared against eager PyTorch rather than merely checked for not crashing. # @@ -71,6 +93,12 @@ def forward(self, x, image): import executorch as _executorch _root = Path(list(_executorch.__path__)[0]) / "kernels" / "quantized" + if sys.platform == "win32": + # Loaded directly, so register the shipped DLLs' directory the way + # executorch.kernels.quantized does before its own load. + import os + + os.add_dll_directory(str(_root.parents[1] / "lib")) _libs = sorted(_root.glob("*quantized_ops_aot_lib.*")) assert len(_libs) == 1, f"expected one ahead-of-time library, found {_libs}" torch.ops.load_library(str(_libs[0])) @@ -262,12 +290,15 @@ def _consumer_cmake(components) -> str: f"target_link_libraries(consumer PRIVATE executorch::{name})" for name in components ) - return f"""cmake_minimum_required(VERSION 3.28) + return ( + f"""cmake_minimum_required(VERSION 3.28) project(consumer CXX) find_package(executorch REQUIRED COMPONENTS {requested}) add_executable(consumer consumer.cpp) {links} """ + + _WINDOWS_DLL_COPY + ) def _mach_o_runtime_paths(binary) -> list: @@ -295,11 +326,16 @@ def _mach_o_runtime_paths(binary) -> list: def _dynamic_lib_suffix() -> str: """The loadable library suffix on this platform, including the dot.""" + if _WINDOWS: + return ".dll" return ".dylib" if sys.platform == "darwin" else ".so" def _library_file_name(base_name: str) -> str: """The file name a library has on this platform.""" + if _WINDOWS: + # PE names carry no lib prefix. + base_name = re.sub(r"^lib", "", base_name) return f"{base_name}{_dynamic_lib_suffix()}" @@ -308,9 +344,13 @@ def _recorded_dependencies(binary) -> str: readelf prints the ELF dynamic section, otool -l the Mach-O load commands. Both carry the same facts: a dependency entry and a runtime search path entry, named - NEEDED and RUNPATH on ELF, LC_LOAD_DYLIB and LC_RPATH on Mach-O. + NEEDED and RUNPATH on ELF, LC_LOAD_DYLIB and LC_RPATH on Mach-O. A PE binary + records only the DLLs it imports, which dumpbin lists. """ - if sys.platform == "darwin": + if _WINDOWS: + tool, args = _visual_studio_tool("dumpbin.exe"), ["/dependents"] + needed = "dumpbin" + elif sys.platform == "darwin": tool, args = _tool("otool"), ["-l"] needed = "otool" else: @@ -343,6 +383,65 @@ def _tool(name: str) -> str: return str(beside) if beside.is_file() else name +def _visual_studio_tool(tool_name: str) -> str: + """Locate dumpbin.exe or clang++.exe from Visual Studio, whether or not its environment is set. + + These checks run from a plain interpreter, so vcvars has not put dumpbin or the + bundled LLVM on PATH. vswhere is what Visual Studio installs to answer exactly this. + Only those two tools are known here; add a search pattern for any other. + """ + found = shutil.which(tool_name) + if found: + return found + vswhere = ( + Path(os.environ.get("ProgramFiles(x86)", r"C:\Program Files (x86)")) + / "Microsoft Visual Studio" + / "Installer" + / "vswhere.exe" + ) + assert vswhere.is_file(), f"Visual Studio is needed to find {tool_name}" + pattern = { + "dumpbin.exe": r"VC\Tools\MSVC\**\bin\Hostx64\x64\dumpbin.exe", + "clang++.exe": r"VC\Tools\Llvm\x64\bin\clang++.exe", + }[tool_name] + matches = subprocess.run( + [str(vswhere), "-latest", "-products", "*", "-find", pattern], + capture_output=True, + text=True, + check=True, + ).stdout.splitlines() + matches = [line.strip() for line in matches if line.strip()] + assert matches, f"the installed Visual Studio has no {tool_name}" + return matches[-1] + + +def _cxx() -> str: + """The compiler driver the direct header and link probes use. + + On Windows the clang++ that ships with Visual Studio, which accepts the same flags + as elsewhere and finds the MSVC headers and libraries on its own. + """ + return _visual_studio_tool("clang++.exe") if _WINDOWS else _tool("c++") + + +def _cmake_build(cmake: str, build_dir: Path, config: str = "Release") -> list: + """The build command for a consumer project. + + The configuration is named for the multi-config Visual Studio generator, which + otherwise builds Debug. A Debug consumer uses the debug C++ library, whose types are + laid out differently from the Release one the shipped DLLs use, which the runtime + target's headers refuse at compile time. Single-config generators ignore it. + """ + return [cmake, "--build", str(build_dir), "--config", config] + + +def _executable(build_dir: Path, name: str) -> Path: + """Where a consumer build leaves its executable.""" + if _WINDOWS: + return build_dir / "Release" / f"{name}.exe" + return build_dir / name + + def _loader_clean_environment() -> dict: """The environment with every loader override removed. @@ -359,7 +458,19 @@ def _loader_clean_environment() -> dict: "DYLD_FALLBACK_LIBRARY_PATH", "DYLD_INSERT_LIBRARIES", ) - return {key: value for key, value in os.environ.items() if key not in overrides} + environment = { + key: value for key, value in os.environ.items() if key not in overrides + } + if _WINDOWS: + # The Windows loader also searches PATH, so an entry reaching the installed + # package would find the DLLs the application is supposed to bring itself. + package = str(_installed_package_dir()).lower() + environment["PATH"] = os.pathsep.join( + entry + for entry in environment.get("PATH", "").split(os.pathsep) + if not entry.lower().startswith(package) + ) + return environment def _installed_package_dir() -> Path: @@ -398,6 +509,7 @@ def _export(work_dir: Path, mode: str) -> tuple: capture_output=True, text=True, check=False, + timeout=_EXPORT_TIMEOUT, ) assert result.returncode == 0, ( f"exporting the {mode} model failed, so the C++ side cannot be checked " @@ -438,7 +550,7 @@ def _build_consumer(work_dir: Path, name: str, components) -> Path: f"installed package:\n{configured.stdout[-2000:]}\n{configured.stderr[-2000:]}" ) built = subprocess.run( - [_tool("cmake"), "--build", str(build_dir)], + _cmake_build(_tool("cmake"), build_dir), capture_output=True, text=True, check=False, @@ -447,7 +559,7 @@ def _build_consumer(work_dir: Path, name: str, components) -> Path: f"a consumer requesting {list(components)} compiled against the shipped " f"headers but did not link:\n{built.stdout[-3000:]}\n{built.stderr[-3000:]}" ) - consumer = build_dir / "consumer" + consumer = _executable(build_dir, "consumer") assert consumer.is_file(), f"the build produced no {consumer}" return consumer @@ -484,6 +596,7 @@ def _run_consumer( text=True, check=False, env=environment, + timeout=_RUN_TIMEOUT, ) assert result.returncode == 0, ( "the C++ application built against the installed wheel did not run " @@ -528,6 +641,7 @@ def test_runtime_alone_links_but_cannot_compute(work_dir: Path) -> None: text=True, check=False, env=environment, + timeout=_RUN_TIMEOUT, ) combined = result.stdout + result.stderr assert result.returncode != 0, ( @@ -552,6 +666,87 @@ def test_kernels_component_runs_a_model(work_dir: Path) -> None: print(f"✓ a C++ app linking executorch::kernels_optimized runs a model ({output})") +def test_thread_pool_consumer_exits(work_dir: Path) -> None: + """An application using the thread pool exits, run after run. + + On Windows the pool used to be torn down at exit after the process had already + terminated its workers, waiting on a lock a dead worker held, and about half the runs + never exited. Five runs catch that almost every time, and each run is bounded by the + timeout in _run_consumer, so a regression fails as a named timeout here. + """ + model, reference = _export(work_dir, "plain") + consumer = _build_consumer( + work_dir, "with-thread-pool", ["runtime", "kernels_optimized", "threadpool"] + ) + runs = 5 + for _ in range(runs): + _run_consumer(consumer, model, reference, work_dir) + print(f"✓ an application using the thread pool exited cleanly {runs} runs in a row") + + +def test_mismatched_configuration_is_refused_on_windows(work_dir: Path) -> None: + """A consumer built in the other configuration from the DLLs fails to compile. + + The DLLs use the C++ library of the configuration the wheel was built in, and a + program in the other configuration uses the other library, whose types are laid out + differently, so the two mixed would corrupt memory. The package states which one it + has, and the runtime headers refuse the mismatch; this builds the opposite of what the + installed package states and expects that refusal. + """ + package_dir = _installed_package_dir() + config_text = ( + package_dir / "share" / "cmake" / "executorch-config.cmake" + ).read_text() + debug_package = "ET_PREBUILT_DEBUG_CRT" in config_text + assert debug_package or "ET_PREBUILT_RELEASE_CRT" in config_text, ( + "the installed package config states neither C++ library, so a consumer built in " + "the wrong configuration would not be caught" + ) + wrong, needed = ("Release", "Debug") if debug_package else ("Debug", "Release") + source_dir = work_dir / "mismatched-consumer" + build_dir = work_dir / "mismatched-consumer-build" + source_dir.mkdir(parents=True, exist_ok=True) + (source_dir / "consumer.cpp").write_text(_CONSUMER_SOURCE) + (source_dir / "CMakeLists.txt").write_text( + _consumer_cmake(["runtime", "kernels_optimized"]) + ) + configured = subprocess.run( + [ + _tool("cmake"), + "-S", + str(source_dir), + "-B", + str(build_dir), + f"-DCMAKE_PREFIX_PATH={package_dir / 'share' / 'cmake'}", + ], + capture_output=True, + text=True, + check=False, + ) + assert ( + configured.returncode == 0 + ), f"the {wrong} consumer could not configure:\n{configured.stdout[-2000:]}" + built = subprocess.run( + _cmake_build(_tool("cmake"), build_dir, wrong), + capture_output=True, + text=True, + check=False, + ) + output = built.stdout + built.stderr + assert built.returncode != 0, ( + f"a {wrong} consumer built against {needed} DLLs, so the debug and release C++ " + "libraries would meet in one process and corrupt memory instead of failing here" + ) + assert f"build this program as {needed}" in output, ( + f"the {wrong} consumer failed, but not on the C++ library check the runtime " + f"headers are meant to raise:\n{output[-2000:]}" + ) + print( + f"✓ a {wrong} consumer of {needed} DLLs is refused at compile time instead of " + "mixing C++ libraries" + ) + + def test_delegated_model_needs_the_delegate_component(work_dir: Path) -> None: """A delegated model runs when the delegate is linked, and fails when it is not. @@ -606,6 +801,7 @@ def test_delegated_model_needs_the_delegate_component(work_dir: Path) -> None: text=True, check=False, env=environment, + timeout=_RUN_TIMEOUT, ) assert result.returncode != 0, ( "a delegated program ran in an application that never linked the delegate, so " @@ -626,6 +822,32 @@ def test_delegated_model_needs_the_delegate_component(work_dir: Path) -> None: ) +def test_consumer_is_relocatable_on_windows(work_dir: Path) -> None: + """The Windows form of the relocation check. + + A PE binary records no search path and no absolute location for a DLL, only its + name, and the loader looks beside the executable first. So the whole check is that + the application runs from somewhere else with the DLLs next to it and the installed + package out of reach. + """ + model, reference = _export(work_dir, "plain") + consumer = _build_consumer(work_dir, "relocate", ["runtime", "kernels_optimized"]) + dynamic = _recorded_dependencies(consumer) + assert _library_file_name("libexecutorch") in dynamic, ( + "the application records no dependency on the shipped runtime, so it is not " + f"linking what the wheel ships:\n{dynamic}" + ) + package_dir = _installed_package_dir() + deployed = work_dir / "deployed" + deployed.mkdir(parents=True, exist_ok=True) + moved = deployed / consumer.name + shutil.copy2(consumer, moved) + for library in sorted((package_dir / "lib").glob("*.dll")): + shutil.copy2(library, deployed / library.name) + output = _run_consumer(moved, model, reference, work_dir) + print(f"✓ the application still runs deployed away from the wheel ({output})") + + def test_consumer_is_relocatable(work_dir: Path) -> None: """The application still runs after being moved away from the wheel. @@ -766,20 +988,23 @@ def backends_seen(name, components) -> int: # variable under test is how many further component libraries are linked, not # whether the program executes. lean = backends_seen("registry-lean", ["runtime", "kernels_optimized"]) - full = backends_seen( - "registry-full", - ["runtime", "kernels_optimized", "threadpool", "etdump", "backend_xnnpack"], - ) + # The profiler is not offered on Windows, so the full set there is one smaller. + full_components = ["runtime", "kernels_optimized", "threadpool", "backend_xnnpack"] + if not _WINDOWS: + full_components.insert(3, "etdump") + full = backends_seen("registry-full", full_components) # The delegate genuinely adds one backend, so the counts differ by exactly that. # What must not happen is the count resetting or doubling, which is what a second # registry in the process looks like. assert full == lean + 1, ( f"an application linking two components sees {lean} registered backends while " - f"one linking five, of which exactly one registers a backend, sees {full}. A " - "component is carrying its own registry rather than resolving the shared one." + f"one linking {len(full_components)}, of which exactly one registers a backend, " + f"sees {full}. A component is carrying its own registry rather than resolving " + "the shared one." ) print( - f"✓ one shared registry: {lean} backends with two components, {full} with five" + f"✓ one shared registry: {lean} backends with two components, {full} with " + f"{len(full_components)}" ) @@ -895,6 +1120,49 @@ def test_profiler_component_is_usable(work_dir: Path) -> None: requested and linked but not used. """ package_dir = _installed_package_dir() + if _WINDOWS: + # The Windows wheel is built without the event tracer, so the component is + # deliberately not offered. Requiring it has to fail at configure time rather + # than hand a consumer a profiler that records nothing. The same project without + # etdump has to configure, and the failure has to name etdump, so this passes + # only for the reason it states rather than for any configure failure. + config_dir = package_dir / "share" / "cmake" + + def configure(name, components): + source_dir = work_dir / name + source_dir.mkdir(parents=True, exist_ok=True) + (source_dir / "consumer.cpp").write_text("int main() { return 0; }\n") + (source_dir / "CMakeLists.txt").write_text(_consumer_cmake(components)) + return subprocess.run( + [ + _tool("cmake"), + "-S", + str(source_dir), + "-B", + str(work_dir / f"{name}-build"), + f"-DCMAKE_PREFIX_PATH={config_dir}", + ], + capture_output=True, + text=True, + check=False, + ) + + without = configure("without-etdump", ["runtime"]) + assert without.returncode == 0, ( + "the control project without etdump did not configure, so the etdump refusal " + f"below would say nothing:\n{(without.stdout + without.stderr)[-1500:]}" + ) + configured = configure("with-etdump", ["runtime", "etdump"]) + assert configured.returncode != 0, ( + "a consumer requiring executorch::etdump configured on Windows, where the " + "wheel is built without the event tracer and the profiler records nothing" + ) + assert "etdump" in configured.stdout + configured.stderr, ( + "requiring etdump failed to configure, but not with a message about etdump:\n" + f"{(configured.stdout + configured.stderr)[-1500:]}" + ) + print("✓ executorch::etdump is not offered on Windows") + return # Globbed, not an exact name: the library carries a version suffix outside a wheel build, and an exact # match would silently skip this check there. The profiler is required elsewhere in this suite, so its # absence is a fault rather than a reason to skip. @@ -937,7 +1205,7 @@ def test_profiler_component_is_usable(work_dir: Path) -> None: f"{configured.stdout[-1500:]}\n{configured.stderr[-1500:]}" ) built = subprocess.run( - [_tool("cmake"), "--build", str(build_dir)], + _cmake_build(_tool("cmake"), build_dir), capture_output=True, text=True, check=False, @@ -987,7 +1255,7 @@ def test_every_shipped_header_compiles(work_dir: Path) -> None: # These ship because other shipped headers include them, so they cannot be left out, and they do # not compile on their own: each needs a third-party library the wheel links but publishes no # headers for, or a platform other than the one being built for. - "mman_windows.h", # a Windows compatibility shim, needs the MinGW headers + *(() if _WINDOWS else ("mman_windows.h",)), # a Windows compatibility shim # These say in their own text that they must not be included directly, and name the header to # include instead. Including one anyway is a use error rather than a packaging defect. "c10/util/complex_math.h", @@ -1006,7 +1274,7 @@ def test_every_shipped_header_compiles(work_dir: Path) -> None: f"#include <{relative.as_posix()}>\nint main() {{ return 0; }}\n" ) result = subprocess.run( - [_tool("c++"), "-std=c++20", *includes, "-fsyntax-only", str(source)], + [_cxx(), "-std=c++20", *includes, "-fsyntax-only", str(source)], capture_output=True, text=True, check=False, @@ -1078,6 +1346,15 @@ def test_shipped_headers_have_implementations(work_dir: Path) -> None: "int main() { float data[4] = {}; auto tensor = make_tensor_ptr({2, 2}, data); " "return tensor->numel() == 4 ? 0 : 1; }\n" ), + # The C platform layer. clock.h's inline ticks_to_ns() calls et_pal_ticks_to_ns_multiplier(), + # which the runtime defines weakly so an application can override it. A weak definition was left + # out of the Windows DLL's export list, so this linked on Linux and macOS and not on Windows. + "runtime/platform/clock.h": ( + "#include \n" + "#include \n" + "int main() { return executorch::runtime::ticks_to_ns(et_pal_current_ticks()) >= 0 " + "? 0 : 1; }\n" + ), # The profiler, which lives in its own library rather than in the runtime. Included because a # dead declaration shipped on this class for a while and the probes above could not reach it: # they link the runtime only, so no etdump symbol was ever resolved here. @@ -1109,6 +1386,15 @@ def test_shipped_headers_have_implementations(work_dir: Path) -> None: "-DC10_USING_CUSTOM_GENERATED_MACROS", ] library_dir = package / "lib" + if _WINDOWS: + # The profiler is not offered on Windows and ships no import library there. + del probes["devtools/etdump/etdump_flatcc.h"] + + def shipped(name: str) -> bool: + if _WINDOWS: + return (library_dir / f"{name}.lib").is_file() + return (library_dir / (f"lib{name}" + _dynamic_lib_suffix())).is_file() + unresolved = [] for header, program in probes.items(): assert ( @@ -1118,12 +1404,13 @@ def test_shipped_headers_have_implementations(work_dir: Path) -> None: source.write_text(program) result = subprocess.run( [ - _tool("c++"), - "-std=c++17", + _cxx(), + # The Windows runtime headers need C++20, as the imported targets state. + "-std=c++20" if _WINDOWS else "-std=c++17", *includes, str(source), "-o", - str(work_dir / "link_probe"), + str(work_dir / ("link_probe.exe" if _WINDOWS else "link_probe")), f"-L{library_dir}", "-lexecutorch", # The component libraries too, not only the runtime. Linking the runtime alone left @@ -1136,16 +1423,20 @@ def test_shipped_headers_have_implementations(work_dir: Path) -> None: "executorch_kernels_optimized", "executorch_threadpool", ) - if (library_dir / (f"lib{name}" + _dynamic_lib_suffix())).is_file() + if shipped(name) ], - f"-Wl,-rpath,{library_dir}", + # PE records no search path; nothing here runs the probe anyway. + *([] if _WINDOWS else [f"-Wl,-rpath,{library_dir}"]), ], capture_output=True, text=True, check=False, ) if result.returncode != 0: - missing = re.findall(r"undefined reference to `([^']+)'", result.stderr) + missing = re.findall( + r"undefined (?:reference to `([^']+)'|symbol: (.+))", result.stderr + ) + missing = [first or second for first, second in missing] unresolved.append(f"{header}: {sorted(set(missing))[:3] or 'did not link'}") assert not unresolved, ( @@ -1215,7 +1506,7 @@ def test_documented_example_compiles(work_dir: Path) -> None: f"{configured.stdout[-1500:]}{configured.stderr[-1500:]}" ) built = subprocess.run( - [_tool("cmake"), "--build", str(build_dir)], + _cmake_build(_tool("cmake"), build_dir), capture_output=True, text=True, check=False, @@ -1241,7 +1532,8 @@ def _provision_pre_328_cmake(work_dir: Path) -> str: return override venv_dir = work_dir / "pre-328-cmake" - binary = venv_dir / "bin" / "cmake" + scripts = venv_dir / ("Scripts" if _WINDOWS else "bin") + binary = scripts / ("cmake.exe" if _WINDOWS else "cmake") if not binary.is_file(): try: subprocess.run( @@ -1251,7 +1543,7 @@ def _provision_pre_328_cmake(work_dir: Path) -> str: ) subprocess.run( [ - str(venv_dir / "bin" / "pip"), + str(scripts / ("pip.exe" if _WINDOWS else "pip")), "install", "--quiet", "cmake==3.24.*", @@ -1327,6 +1619,13 @@ def test_pre_3_28_route_builds_a_consumer_through_variables(work_dir: Path) -> N "target_compile_definitions(consumer PRIVATE ${EXECUTORCH_COMPILE_DEFINITIONS})\n" "target_link_libraries(consumer PRIVATE ${EXECUTORCH_LIBRARIES})\n" "set_target_properties(consumer PROPERTIES CXX_STANDARD ${EXECUTORCH_CXX_STANDARD})\n" + # No imported targets on this route, so no TARGET_RUNTIME_DLLS either. A Windows + # consumer copies the DLLs from the directory the package reports. + "if(WIN32)\n" + ' file(GLOB _dlls "${EXECUTORCH_RUNTIME_LIBRARY_DIR}/*.dll")\n' + " add_custom_command(TARGET consumer POST_BUILD COMMAND ${CMAKE_COMMAND} -E " + "copy_if_different ${_dlls} $)\n" + "endif()\n" ) build_dir = work_dir / "pre-328-build" for command in ( @@ -1338,7 +1637,7 @@ def test_pre_3_28_route_builds_a_consumer_through_variables(work_dir: Path) -> N str(build_dir), f"-DCMAKE_PREFIX_PATH={config.parent}", ], - [old_cmake, "--build", str(build_dir)], + _cmake_build(old_cmake, build_dir), ): result = subprocess.run(command, capture_output=True, text=True, check=False) assert result.returncode == 0, ( @@ -1347,7 +1646,7 @@ def test_pre_3_28_route_builds_a_consumer_through_variables(work_dir: Path) -> N f"{result.stdout[-2000:]}\n{result.stderr[-2000:]}" ) - consumer = build_dir / "consumer" + consumer = _executable(build_dir, "consumer") assert consumer.is_file(), f"the build produced no {consumer}" # Run it. Linking proves the variables name the right files; only executing proves # they also leave the program able to find them at run time, which is the half of @@ -1361,7 +1660,7 @@ def test_pre_3_28_route_builds_a_consumer_through_variables(work_dir: Path) -> N "a consumer built through EXECUTORCH_LIBRARIES on pre-3.28 CMake does not " f"depend on the runtime:\n{dependencies}" ) - assert "libexecutorch_kernels_optimized" in dependencies, ( + assert _library_file_name("libexecutorch_kernels_optimized") in dependencies, ( "the pre-3.28 aggregate does not carry the CPU kernels, so a consumer built " "through it would fail at run time with operators reported missing" ) @@ -1468,7 +1767,7 @@ def test_aggregate_variable_excludes_the_quantized_kernels(work_dir: Path) -> No str(build_dir), f"-DCMAKE_PREFIX_PATH={config.parent}", ], - [_tool("cmake"), "--build", str(build_dir)], + _cmake_build(_tool("cmake"), build_dir), ): result = subprocess.run(command, capture_output=True, text=True, check=False) assert result.returncode == 0, ( @@ -1476,16 +1775,16 @@ def test_aggregate_variable_excludes_the_quantized_kernels(work_dir: Path) -> No f"{result.stdout[-2000:]}\n{result.stderr[-2000:]}" ) - consumer = build_dir / "consumer" + consumer = _executable(build_dir, "consumer") dependencies = _recorded_dependencies(consumer) - assert "libexecutorch_kernels_quantized" not in dependencies, ( + assert _library_file_name("libexecutorch_kernels_quantized") not in dependencies, ( "an application that linked only ${EXECUTORCH_LIBRARIES} depends on the " "quantized kernels. That library collides with the export-time plugin, so it " "has to be opted into by name rather than handed to every consumer." ) # The rest of the aggregate still has to be there, or this would pass by shipping # nothing at all. - assert "libexecutorch_kernels_optimized" in dependencies, ( + assert _library_file_name("libexecutorch_kernels_optimized") in dependencies, ( "the aggregate no longer carries the CPU kernels, so an application linking it " "would fail at run time with the operators reported missing" ) @@ -1537,6 +1836,16 @@ def test_pkg_config_builds_a_consumer(work_dir: Path) -> None: the check does not depend on one the build machine happens to have. """ pc_dir = _installed_package_dir() / "lib" / "pkgconfig" + if _WINDOWS: + # Not shipped on Windows: the file's link flags are GNU-style and a DLL records no + # search path for them to point at. Asserted rather than skipped, so a Windows wheel + # that starts shipping a file no Windows build system can use fails here. + assert not (pc_dir / "executorch.pc").exists(), ( + f"the Windows wheel ships {pc_dir / 'executorch.pc'}, whose GNU-style link flags " + "no Windows consumer can use" + ) + print("✓ no pkg-config file is shipped on Windows") + return assert (pc_dir / "executorch.pc").is_file(), ( f"the wheel ships no pkg-config file in {pc_dir}, so a build system that reads " "pkg-config cannot find the runtime" @@ -1617,11 +1926,17 @@ def run_tests(work_dir: Path) -> None: test_runtime_alone_links_but_cannot_compute(work_dir) test_kernels_component_runs_a_model(work_dir) test_pkg_config_builds_a_consumer(work_dir) + test_thread_pool_consumer_exits(work_dir) + if _WINDOWS: + test_mismatched_configuration_is_refused_on_windows(work_dir) test_pre_3_28_route_builds_a_consumer_through_variables(work_dir) test_quantized_kernels_component_runs_a_model(work_dir) test_aggregate_variable_excludes_the_quantized_kernels(work_dir) test_delegated_model_needs_the_delegate_component(work_dir) - test_consumer_is_relocatable(work_dir) + if _WINDOWS: + test_consumer_is_relocatable_on_windows(work_dir) + else: + test_consumer_is_relocatable(work_dir) test_one_registry_in_the_cpp_process(work_dir) diff --git a/.ci/scripts/wheel/test_shared_libraries.py b/.ci/scripts/wheel/test_shared_libraries.py index e23aabb4f6a..4764e3813a4 100644 --- a/.ci/scripts/wheel/test_shared_libraries.py +++ b/.ci/scripts/wheel/test_shared_libraries.py @@ -140,6 +140,11 @@ # platform. Both are checked elsewhere. if sys.platform == "darwin": _BUNDLED_THREADPOOL_SYMBOLS = ("cpuinfo_initialize",) +elif sys.platform == "win32": + # cpuinfo is private to each DLL on Windows, where its inline feature checks can only + # read the calling DLL's own copy; test_cpuinfo_state_is_not_split_on_windows checks + # that no DLL shares it. The pool is still shared, so pthreadpool keeps its row. + _BUNDLED_THREADPOOL_SYMBOLS = ("pthreadpool_create",) else: _BUNDLED_THREADPOOL_SYMBOLS = ("pthreadpool_create", "cpuinfo_initialize") # The delegate's own entry points. A second definer means the delegate is compiled @@ -191,6 +196,141 @@ # Symbol kinds that mean the object owns the code or storage. _OWNING_KINDS = frozenset("TtBbDdGgSsRrWV") +_WINDOWS = sys.platform == "win32" + +# Components whose code a Windows DLL takes from a static archive, so the automatic export list, +# which covers only the DLL's own objects, never names their symbols and an export count sees +# nothing. What is observable is the direction of the link: the owner must import the registry +# entry point from the runtime DLL, which it cannot do while carrying a private registry of its own. +_WINDOWS_IMPORT_WITNESSES = { + "set of CPU kernels": ("executorch.dll", "executorch::runtime::register_kernels"), + "set of quantized kernels": ( + "executorch.dll", + "executorch::runtime::register_kernels", + ), + # Registration is what a second copy of the delegate's runtime would break, and the one thing a + # PE file records about it. + "bundled XNNPACK runtime": ( + "executorch.dll", + "executorch::runtime::register_backend", + ), +} + +_PE_REPORTS: dict = {} + + +def _dumpbin() -> str: + """dumpbin from the installed Visual Studio, found whether or not vcvars ran.""" + found = shutil.which("dumpbin") + if found: + return found + vswhere = ( + Path(os.environ.get("ProgramFiles(x86)", r"C:\Program Files (x86)")) + / "Microsoft Visual Studio" + / "Installer" + / "vswhere.exe" + ) + assert vswhere.is_file(), "Visual Studio is required to inspect the wheel's DLLs" + matches = [ + line.strip() + for line in subprocess.run( + [ + str(vswhere), + "-latest", + "-products", + "*", + "-find", + r"VC\Tools\MSVC\**\bin\Hostx64\x64\dumpbin.exe", + ], + capture_output=True, + text=True, + check=True, + ).stdout.splitlines() + if line.strip() + ] + assert matches, "the installed Visual Studio has no dumpbin" + return matches[-1] + + +def _pe_report(library: Path, option: str) -> str: + """dumpbin's output for one file, cached because several checks read the same one.""" + key = (str(library), option) + if key not in _PE_REPORTS: + result = subprocess.run( + [_dumpbin(), "/nologo", option, str(library)], + capture_output=True, + text=True, + check=False, + ) + assert result.returncode == 0, ( + f"dumpbin could not read {library.name}, which is a shipped binary, so the checks " + f"cannot be trusted: {(result.stderr or result.stdout).strip()[:200]}" + ) + _PE_REPORTS[key] = result.stdout + return _PE_REPORTS[key] + + +def _pe_exports(library: Path) -> list: + """The names a DLL exports, as the linker recorded them.""" + return [ + match.group(1) + for match in re.finditer( + r"^\s+\d+\s+[0-9A-F]+\s+[0-9A-F]{8}\s+(\S+)", + _pe_report(library, "/exports"), + re.M, + ) + ] + + +def _pe_imports(library: Path) -> dict: + """The names a binary imports, keyed by the lowercased DLL each comes from.""" + imports: dict = {} + current = None + for line in _pe_report(library, "/imports").splitlines(): + header = re.match(r"^ (\S+\.dll)$", line, re.I) + if header: + current = imports.setdefault(header.group(1).lower(), []) + continue + entry = re.match(r"^\s+[0-9A-F]+\s+(\S+)$", line) + if entry and current is not None: + current.append(entry.group(1)) + return imports + + +def _pe_dependents(library: Path) -> set: + """The DLL names a binary records a dependency on.""" + report = _pe_report(library, "/dependents") + return { + line.strip() + for line in report.splitlines() + if re.fullmatch(r"\s+\S+\.dll", line, re.I) + } + + +def _decorated_prefix(symbol: str): + """The leading part of the MSVC decorated name for a qualified C++ name, or None for a C name. + + A C++ name is decorated innermost first, so a::b::f begins ?f@b@a@@ and a constructor + a::B::B begins ??0B@a@@. The trailing @@ closes the scope, so a longer name that merely + starts with the same text cannot match, which is the anchoring the ELF reader gets from + matching demangled names whole. + """ + symbol = symbol.removesuffix("()") + if "::" not in symbol: + return None + parts = symbol.split("::") + if parts[-1] == parts[-2]: + return "??0" + "@".join(reversed(parts[:-1])) + "@@" + return "?" + "@".join(reversed(parts)) + "@@" + + +def _pe_names_include(names, symbol: str) -> bool: + """Whether a list of decorated names contains `symbol`.""" + prefix = _decorated_prefix(symbol) + if prefix is None: + return symbol in names + return any(name.startswith(prefix) for name in names) + def _declared_requirements() -> set: """Import names the installed wheel declares a requirement for. @@ -332,7 +472,19 @@ def _loader_clean_environment() -> dict: "DYLD_FALLBACK_LIBRARY_PATH", "DYLD_INSERT_LIBRARIES", ) - return {key: value for key, value in os.environ.items() if key not in overrides} + environment = { + key: value for key, value in os.environ.items() if key not in overrides + } + if _WINDOWS: + # The Windows loader also searches PATH, so an entry reaching the installed package + # would supply the DLLs a shipped binary is supposed to find on its own. + package = str(_installed_package_dir()).lower() + environment["PATH"] = os.pathsep.join( + entry + for entry in environment.get("PATH", "").split(os.pathsep) + if not entry.lower().startswith(package) + ) + return environment def _dynamic_section(library) -> str | None: @@ -392,6 +544,8 @@ def _recorded_dependencies(library) -> set: result apart from one this could not read, and the second is a passing check that examined nothing. """ + if _WINDOWS: + return _pe_dependents(Path(library)) if sys.platform == "darwin": tool = _tool("otool") assert tool is not None, "otool is required to inspect the wheel" @@ -421,6 +575,8 @@ def _raw_recorded_identity(library) -> str | None: Separate from _recorded_identity because that function's basename reduction is correct for comparing names but hides whether the recorded string is absolute, which is a different fault. """ + if _WINDOWS: + return _recorded_identity(library) section = _dynamic_section(library) if section is None: return None @@ -449,8 +605,14 @@ def _recorded_identity(library) -> str | None: ELF calls it the soname. Mach-O calls it the install name and spells it as a path, usually relative to the consumer's runtime search path, so the two compare only by - the last component. + the last component. A DLL records the name it was linked as in its export directory, which + is what an import library, and so a consumer, carries. """ + if _WINDOWS: + match = re.search( + r"following exports for (\S+)", _pe_report(Path(library), "/exports") + ) + return match.group(1) if match else None if sys.platform == "darwin": tool = _tool("otool") assert tool is not None, "otool is required to inspect the wheel" @@ -605,13 +767,17 @@ def _shipped_object_patterns() -> list[str]: Both suffixes are listed because a Mach-O Python extension is a .so by convention, so a dylib-only pattern silently drops the extension on macOS, which is the one artifact these - checks exist to verify. + checks exist to verify. Windows names a library .dll and a Python extension .pyd. """ + if _WINDOWS: + return ["*.dll", "*.pyd"] return ["*.dylib*", "*.so*"] def _dynamic_lib_suffix() -> str: """The loadable library suffix on this platform, including the dot.""" + if _WINDOWS: + return ".dll" return ".dylib" if sys.platform == "darwin" else ".so" @@ -619,8 +785,10 @@ def _library_file_name(base_name: str) -> str: """The file name a component's library has on this platform. The component table names libraries without a suffix so one table serves both - platforms. + platforms. A PE name carries no lib prefix either. """ + if _WINDOWS: + base_name = re.sub(r"^lib", "", base_name) return f"{base_name}{_dynamic_lib_suffix()}" @@ -677,7 +845,13 @@ def _shipped_runtime_libraries(package_dir: Path): return [] return [ path - for path in sorted(lib_dir.glob(f"lib*{_dynamic_lib_suffix()}*")) + for path in sorted( + lib_dir.glob( + f"*{_dynamic_lib_suffix()}" + if _WINDOWS + else f"lib*{_dynamic_lib_suffix()}*" + ) + ) if path.is_file() and not path.is_symlink() ] @@ -696,6 +870,8 @@ def _defines_symbol(library: Path, symbol: str) -> bool: compiled with hidden visibility is invisible to it. Catching that needs a running process, which counts what actually registered rather than what is visible. """ + if _WINDOWS: + return _pe_names_include(_pe_exports(library), symbol) result = subprocess.run( [_tool("nm"), *_nm_defined_args(), str(library)], capture_output=True, @@ -777,6 +953,12 @@ def _is_export_only(library: Path) -> bool: return False if library.name.endswith(f"_aot_lib{_dynamic_lib_suffix()}"): return True + if _WINDOWS: + # Compared as whole names: executorch.dll contains the text torch.dll. + return bool( + {name.lower() for name in _pe_dependents(library)} + & {"torch.dll", "torch_cpu.dll"} + ) dynamic = _dynamic_section(library) if dynamic is None: return False @@ -808,7 +990,7 @@ def _assert_single_definer( these libraries defined the backend registry symbols in one released wheel and not in the release before it, so the duplication this catches does happen. """ - assert _tool("nm") is not None, "nm is required to inspect the wheel" + assert _WINDOWS or _tool("nm") is not None, "nm is required to inspect the wheel" package_dir = _installed_package_dir() libraries = [ @@ -827,6 +1009,14 @@ def _assert_single_definer( # A component is either wholly present or wholly absent. Some symbols defined and # others not means a partial build, which is neither of those and is a fault. present = {symbol for symbol, definers in found.items() if definers} + if ( + not present + and _WINDOWS + and owner is not None + and what in _WINDOWS_IMPORT_WITNESSES + ): + _assert_windows_import_witness(what, owner, libraries) + return if not present: # When the caller has already established that the owner library ships, finding none of its # symbols is a fault rather than an absence. Returning success here made this a no-op the moment @@ -862,6 +1052,34 @@ def _assert_single_definer( print(f"✓ single {what}{where} across {len(libraries)} shipped libraries") +def _assert_windows_import_witness(what: str, owner: str, libraries) -> None: + """The Windows form of the single-owner check for code a DLL takes from an archive. + + The export list does not name that code, so the check asks the question the other way + round: the owner imports the registry entry point from the runtime DLL rather than + carrying it, and no other shipped binary defines that entry point. + """ + runtime_dll, entry_point = _WINDOWS_IMPORT_WITNESSES[what] + owners = [library for library in libraries if library.name == owner] + assert owners, f"the wheel ships no {owner}, which owns the {what}" + imported = _pe_imports(owners[0]).get(runtime_dll, []) + assert _pe_names_include(imported, entry_point), ( + f"{owner} does not import {entry_point} from {runtime_dll}, so the {what} it carries " + "registers into a registry of its own rather than the one the runtime owns" + ) + definers = [ + library.name + for library in libraries + if _pe_names_include(_pe_exports(library), entry_point) + ] + assert definers == [ + runtime_dll + ], f"expected only {runtime_dll} to define {entry_point}, found {definers}" + print( + f"✓ single {what} owned by {owner}, which imports {entry_point} from {runtime_dll}" + ) + + def _wheel_cuda_train() -> str: """The CUDA train the installed wheel was built for, or "" for a CPU wheel. @@ -1121,6 +1339,73 @@ def test_each_component_has_one_owner() -> None: ) +def _depends_on_torch(library: Path) -> bool: + """Whether a Windows binary imports torch's DLLs, which resolve once torch is imported.""" + return any( + name.lower().startswith(("torch", "c10")) for name in _pe_dependents(library) + ) + + +def _import_probe(package_dir: Path, module: str) -> str: + """The code a clean interpreter runs to import `module` the way a user would. + + Off Windows that is the import itself. On Windows nothing is added to the DLL search + path here: the package's own entry points have to register executorch/lib, so a probe + that registered it would pass whether or not they do. The pybindings extensions are + reached through portable_lib, the public module that loads them; everything else is + imported directly, which runs its package's initializer. Torch comes first only for + an extension that links it, as the Linux check lets the loader decide. + """ + if not _WINDOWS: + return f"import {module}" + relative = Path(*module.split(".")[1:]) + extension = next(package_dir.glob(f"{relative}.cp3*.pyd")) + prelude = "import torch\n" if _depends_on_torch(extension) else "" + if module.startswith("executorch.extension.pybindings."): + prelude += "import executorch.extension.pybindings.portable_lib\n" + return f"{prelude}import {module}" + + +def test_package_entry_points_load_their_libraries() -> None: + """The package modules that load a shipped library find its dependencies themselves. + + executorch.kernels.quantized loads the export-time quantized operators and swallows a + load failure, so a library that cannot find executorch/lib shows up only as quantized + export missing its out variants. Checked through the package in a clean interpreter, + the way a user reaches it, and by the operators that have to arrive, so the check fails + whenever the load does. + """ + package_dir = _installed_package_dir() + if not list(package_dir.glob("kernels/quantized/*quantized_ops_aot_lib.*")): + print("- this wheel ships no quantized export library, nothing to check") + return + if importlib.util.find_spec("torch") is None: + print("- torch is not installed, skipping the package entry point check") + return + result = subprocess.run( + [ + sys.executable, + "-c", + "import torch\n" + "import executorch.kernels.quantized\n" + "print(torch.ops.quantized_decomposed.quantize_per_tensor.overloads())\n", + ], + capture_output=True, + text=True, + check=False, + env=_loader_clean_environment(), + timeout=_PROBE_TIMEOUT, + ) + assert result.returncode == 0 and "'out'" in result.stdout, ( + "importing executorch.kernels.quantized did not register the quantized out " + "variants, so its library failed to load and the failure was swallowed: " + f"{(result.stdout + result.stderr).strip()[-800:]}" + ) + print( + "✓ executorch.kernels.quantized loads its library and registers the out variants" + ) + + def test_python_extensions_import() -> None: """Every shipped Python extension must import from a clean environment. @@ -1136,10 +1421,12 @@ def test_python_extensions_import() -> None: """ package_dir = _installed_package_dir() modules = [] - for extension in sorted(package_dir.rglob("*.so")): + extension_pattern = "*.pyd" if _WINDOWS else "*.so" + interpreter_marker = ".cp3" if _WINDOWS else ".cpython-" + for extension in sorted(package_dir.rglob(extension_pattern)): # Only Python extensions, which carry the interpreter's suffix. The plain # shared libraries under lib/ are checked by the load test instead. - if ".cpython-" not in extension.name: + if interpreter_marker not in extension.name: continue relative = extension.relative_to(package_dir).parent module = extension.name.split(".", 1)[0] @@ -1155,11 +1442,12 @@ def test_python_extensions_import() -> None: environment = _loader_clean_environment() for module in modules: result = subprocess.run( - [sys.executable, "-c", f"import {module}"], + [sys.executable, "-c", _import_probe(package_dir, module)], capture_output=True, text=True, check=False, env=environment, + timeout=_PROBE_TIMEOUT, ) if result.returncode == 0: print(f"✓ {module} imports from a clean environment") @@ -1311,6 +1599,7 @@ def _dyld_load_failure(library: Path, *, with_torch: bool) -> str: # Any loader override in the build environment would paper over a runtime # search path the shipped library is actually missing. env=_loader_clean_environment(), + timeout=_PROBE_TIMEOUT, ) if result.returncode == 0: return "" @@ -1493,6 +1782,94 @@ def _assert_shipped_libraries_relocate_with_dyld() -> None: print("✓ every shipped library resolves without the build tree") +_WINDOWS_LOAD_PROBE = """ +import ctypes +import os +import sys + +# Torch's own DLLs are the documented exception, as on Linux: they resolve once the package +# that owns them is imported, which is how every binary that links them is used. Only for +# those binaries, so the rest load without torch's directories already in the process. +if sys.argv[3] == "torch": + import torch # noqa: F401 + +# executorch/lib, which a C++ program copies beside itself and the package's entry points +# register. Whether they do is test_package_entry_points_load_their_libraries. +os.add_dll_directory(sys.argv[2]) +ctypes.WinDLL(sys.argv[1]) +""" + +# Long enough for the slowest honest run, a Python probe importing torch or a consumer loading +# a model, and short enough that a process which never exits fails as a named timeout well +# inside the job's limit rather than as the whole job timing out. +_PROBE_TIMEOUT = 300 + + +def _assert_shipped_libraries_load_on_windows(root: Path, package_dir: Path) -> None: + """Load every shipped binary under `root`, each in its own process. + + `root` is where the binaries are loaded from and `package_dir` the installed package + that lists them; the relocation check passes a copy of the package as `root`. + + LoadLibrary resolves every imported DLL and every imported symbol before it returns, so + a successful load answers what ldd -r answers on Linux. A process per binary, because a + DLL already in the process satisfies a later request by name and would stand in for a + dependency the binary under test cannot find itself. + """ + lib_dir = root / "lib" + broken = {} + for library in _shipped_shared_objects(package_dir): + target = root / library.relative_to(package_dir) + torch_needed = "torch" if _depends_on_torch(library) else "-" + result = subprocess.run( + [ + sys.executable, + "-c", + _WINDOWS_LOAD_PROBE, + str(target), + str(lib_dir), + torch_needed, + ], + capture_output=True, + text=True, + check=False, + env=_loader_clean_environment(), + timeout=_PROBE_TIMEOUT, + ) + if result.returncode != 0: + broken[str(target.relative_to(root))] = result.stderr.strip()[-300:] + assert not broken, ( + "shipped binaries fail to load from the package, so they need a DLL or a symbol " + f"nothing in the wheel or in torch provides: {broken}" + ) + + +def test_cpuinfo_state_is_not_split_on_windows() -> None: + """No shipped DLL imports cpuinfo functions from another DLL. + + cpuinfo's feature checks, cpuinfo_has_x86_avx2 and the rest, are inline reads of + cpuinfo_isa. cpuinfo.h does not declare that data dllimport, so a DLL can only read + its own copy. A DLL that imports cpuinfo_initialize from the thread pool therefore + fills in the thread pool's copy and reads its own, all zeros: XNNPACK saw no AVX2 and + ran baseline kernels, with correct results and nothing failing. A DLL that uses + cpuinfo has to carry all of it. + """ + package_dir = _installed_package_dir() + split = {} + for library in _shipped_shared_objects(package_dir): + for dll, symbols in _pe_imports(library).items(): + imported = sorted(s for s in symbols if s.startswith("cpuinfo_")) + if imported: + split[str(library.relative_to(package_dir))] = {dll: imported} + assert not split, ( + "shipped DLLs import cpuinfo functions from another DLL while their inline feature " + f"checks read a copy of cpuinfo_isa that nothing initializes: {split}" + ) + print( + "✓ no shipped DLL imports cpuinfo, so each one's feature checks read what it initialized" + ) + + def test_shipped_libraries_load() -> None: """Every shipped library must depend only on things that exist. @@ -1514,6 +1891,11 @@ def test_shipped_libraries_load() -> None: if sys.platform == "darwin": _assert_shipped_libraries_load_with_dyld() return + if _WINDOWS: + package_dir = _installed_package_dir() + _assert_shipped_libraries_load_on_windows(package_dir, package_dir) + print("✓ every shipped library loads in an environment with torch present") + return if _tool("ldd") is None: print("- ldd not available, skipping the load check") return @@ -1627,6 +2009,19 @@ def test_shipped_libraries_resolve_without_build_tree() -> None: if sys.platform == "darwin": _assert_shipped_libraries_relocate_with_dyld() return + if _WINDOWS: + # A DLL records no search path, so relocating is copying the package somewhere else + # and loading every binary from the copy with the original out of reach. + package_dir = _installed_package_dir() + with tempfile.TemporaryDirectory() as work_dir: + root = Path(work_dir) / package_dir.name + for library in _shipped_shared_objects(package_dir): + target = root / library.relative_to(package_dir) + target.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(library, target) + _assert_shipped_libraries_load_on_windows(root, package_dir) + print("✓ every shipped library loads from a relocated copy of the package") + return if _tool("ldd") is None or _tool("patchelf") is None: print("- ldd or patchelf unavailable, skipping the relocated load check") return @@ -1764,7 +2159,9 @@ def test_custom_op_compiles(work_dir: Path) -> None: ) compiled = subprocess.run( - [_tool("cmake"), "--build", str(build_dir)], + # Release named for the multi-config Visual Studio generator, whose default is Debug + # and so a different C++ library from the shipped DLLs. Single-config generators ignore it. + [_tool("cmake"), "--build", str(build_dir), "--config", "Release"], capture_output=True, text=True, check=False, @@ -1814,6 +2211,7 @@ def test_custom_op_compiles(work_dir: Path) -> None: text=True, check=False, env=_loader_clean_environment(), + timeout=_PROBE_TIMEOUT, ) assert loaded.returncode == 0, ( "a custom operator built against the shipped extension cannot be loaded, or it " @@ -1942,6 +2340,9 @@ def test_wheel_platform_tag() -> None: if sys.platform == "darwin": _assert_mach_o_architecture_matches(wheels[-1]) return + if _WINDOWS: + _assert_pe_architecture_matches(wheels[-1]) + return if importlib.util.find_spec("auditwheel") is None: # Installed here rather than skipped, because auditwheel is not in any CI @@ -2101,6 +2502,65 @@ def _unusable_runtime_path_kind(entry: str, library_name: str) -> str | None: return "an absolute directory the wheel has a relative route to" +def _assert_no_recorded_paths_on_windows() -> None: + """The Windows form of the absolute path check. + + A PE file has no runtime search path; it records each dependency by name, and the loader + searches for it. So the property to hold is that every recorded dependency and every + DLL's own recorded name is a bare file name, never a path from the build machine. + """ + package_dir = _installed_package_dir() + offenders = {} + libraries = _shipped_shared_objects(package_dir) + for library in libraries: + recorded = set(_pe_dependents(library)) + if library.suffix.lower() == ".dll": + identity = _recorded_identity(library) + if identity: + recorded.add(identity) + with_path = sorted(name for name in recorded if re.search(r"[\\/:]", name)) + if with_path: + offenders[str(library.relative_to(package_dir))] = with_path + assert not offenders, ( + "shipped binaries record a dependency or name as a path from the machine that built " + f"them, which exists nowhere else: {offenders}" + ) + print( + f"✓ none of the {len(libraries)} shipped binaries records a path, only bare DLL names" + ) + + +# The PE machine field for each architecture a Windows wheel tag names. +_PE_MACHINES = {"win_amd64": 0x8664, "win_arm64": 0xAA64, "win32": 0x14C} + + +def _assert_pe_architecture_matches(wheel: Path) -> None: + """Every binary in a Windows wheel must be built for the architecture its tag names.""" + claimed = wheel.name.split("-")[-1].removesuffix(".whl") + assert ( + claimed in _PE_MACHINES + ), f"could not read a Windows architecture from the tag {claimed}" + wrong = {} + inspected = 0 + with zipfile.ZipFile(wheel) as archive: + for name in archive.namelist(): + if not name.lower().endswith((".dll", ".pyd")): + continue + data = archive.read(name) + header = int.from_bytes(data[0x3C:0x40], "little") + assert data[header : header + 4] == b"PE\0\0", f"{name} is not a PE file" + machine = int.from_bytes(data[header + 4 : header + 6], "little") + inspected += 1 + if machine != _PE_MACHINES[claimed]: + wrong[name] = hex(machine) + assert inspected, f"{wheel.name} contains no DLL or extension to compare" + assert not wrong, ( + f"the wheel is tagged {claimed} but these binaries are built for another " + f"architecture, so it would install where it cannot run: {wrong}" + ) + print(f"✓ the wheel is tagged for the architecture it contains ({claimed})") + + def test_no_absolute_runtime_paths() -> None: """No shipped library may search a directory a user does not have. @@ -2126,6 +2586,9 @@ def test_no_absolute_runtime_paths() -> None: # build machine's directories would ship looking correct. # Mach-O keeps its search path in load commands that otool reads, and otool comes # with the developer tools, so only the ELF side needs an install step. + if _WINDOWS: + _assert_no_recorded_paths_on_windows() + return if sys.platform != "darwin" and _tool("patchelf") is None: print("- patchelf not present, installing it so this check can run") subprocess.run( @@ -2182,6 +2645,33 @@ def test_no_absolute_runtime_paths() -> None: ) +def _extension_symbol_tables(extension: Path): + """Predicates for whether the extension imports, and whether it defines, a symbol. + + Module scope so the enclosing test stays inside the complexity limit lintrunner enforces. + """ + if _WINDOWS: + imported = [name for names in _pe_imports(extension).values() for name in names] + exported = _pe_exports(extension) + return ( + lambda symbol: _pe_names_include(imported, symbol), + lambda symbol: _pe_names_include(exported, symbol), + ) + undefined = subprocess.run( + [_tool("nm"), *_nm_undefined_args(), str(extension)], + capture_output=True, + text=True, + check=False, + ).stdout + defined = subprocess.run( + [_tool("nm"), *_nm_defined_args(), str(extension)], + capture_output=True, + text=True, + check=False, + ).stdout + return (lambda symbol: symbol in undefined, lambda symbol: symbol in defined) + + def test_extension_contains_no_component() -> None: """The Python extension must link the components, not contain them. @@ -2190,10 +2680,14 @@ def test_extension_contains_no_component() -> None: inside the extension. The direct statement is that the extension defines none of what the shipped libraries own, and records a dependency on each instead. """ - assert _tool("nm") is not None, "nm is required to inspect the wheel" + assert _WINDOWS or _tool("nm") is not None, "nm is required to inspect the wheel" package_dir = _installed_package_dir() - extensions = sorted((package_dir / "extension" / "pybindings").glob("_C.*.so")) + extensions = sorted( + (package_dir / "extension" / "pybindings").glob( + "_C.*.pyd" if _WINDOWS else "_C.*.so" + ) + ) assert len(extensions) == 1, f"expected one _C, found {extensions}" extension = extensions[0] @@ -2283,25 +2777,17 @@ def test_extension_contains_no_component() -> None: # appear in the dynamic symbol table at all, so "defines nothing" on its own is # satisfiable by an extension that still carries its own private runtime. An # UNDEFINED reference cannot be faked that way: it says the definition is not - # here and has to come from a dependency. - undefined = subprocess.run( - [_tool("nm"), *_nm_undefined_args(), str(extension)], - capture_output=True, - text=True, - check=False, - ).stdout - defined = subprocess.run( - [_tool("nm"), *_nm_defined_args(), str(extension)], - capture_output=True, - text=True, - check=False, - ).stdout + # here and has to come from a dependency. On Windows the import table says the same thing, + # naming the DLL each definition comes from. + imports_symbol, defines_symbol = _extension_symbol_tables(extension) candidates = (*_REGISTRY_SYMBOLS, *_THREADPOOL_SYMBOLS) # A symbol the extension defines itself is the hidden copy this check exists to catch. A # symbol it neither imports nor defines is simply unused, which happens for the backend # registry when the wheel is built with the optional delegates off. carried = [ - symbol for symbol in candidates if symbol not in undefined and symbol in defined + symbol + for symbol in candidates + if not imports_symbol(symbol) and defines_symbol(symbol) ] assert not carried, ( f"{extension.name} defines {carried} itself rather than importing it, so it carries a " @@ -2311,7 +2797,7 @@ def test_extension_contains_no_component() -> None: # worst version of this: an extension that whole-archived a private runtime with # hidden visibility and kept the shipped one as a dependency it never uses. What it # calls has to be imported, and these it calls. - unimported = [symbol for symbol in _EXTENSION_IMPORTS if symbol not in undefined] + unimported = [symbol for symbol in _EXTENSION_IMPORTS if not imports_symbol(symbol)] assert not unimported, ( f"{extension.name} calls {unimported} but imports none of them, so the definition it " "reaches is inside itself and the process holds a second registry the shipped runtime " @@ -2352,7 +2838,7 @@ def test_shipped_library_names_are_expected() -> None: # a real file, so it is still caught. shipped = sorted( p - for p in lib_dir.glob(f"*{_dynamic_lib_suffix()}*") + for p in lib_dir.glob(f"*{_dynamic_lib_suffix()}" + ("" if _WINDOWS else "*")) if p.is_file() and not p.is_symlink() ) assert shipped, f"the wheel ships a lib directory with no libraries: {lib_dir}" @@ -2388,7 +2874,8 @@ def test_shipped_library_names_are_expected() -> None: # Unversioned, because the wheel build does not version these. A trailing # . would also be a name packaging did not produce here. These are # libraries, so the suffix follows the platform and macOS spells them .dylib. - permitted = re.compile(rf"(?:{'|'.join(known)})\{_dynamic_lib_suffix()}") + names = [re.sub(r"^lib", "", name) for name in known] if _WINDOWS else known + permitted = re.compile(rf"(?:{'|'.join(names)})\{_dynamic_lib_suffix()}") unknown = sorted(p.name for p in shipped if not permitted.fullmatch(p.name)) assert not unknown, ( f"the wheel ships {unknown} under lib/, which packaging does not produce. " @@ -2545,6 +3032,8 @@ def test_model_matches_eager_pytorch(work_dir: Path) -> None: text=True, check=False, cwd=str(work_dir), + # It exports two models before running them, which is slower than a probe. + timeout=_PROBE_TIMEOUT * 3, ) assert result.returncode == 0, ( f"the {mode} model does not export, run, and match eager PyTorch: " @@ -2673,6 +3162,9 @@ def run_tests(work_dir: Path) -> None: # hide a strong one. test_each_component_has_one_owner() test_python_extensions_import() + test_package_entry_points_load_their_libraries() + if _WINDOWS: + test_cpuinfo_state_is_not_split_on_windows() test_declared_dependencies_match_the_wheel_tag() test_extension_contains_no_component() test_shipped_library_names_are_expected() diff --git a/.ci/scripts/wheel/test_windows.py b/.ci/scripts/wheel/test_windows.py index 879a9ed9e75..8600b676f1f 100644 --- a/.ci/scripts/wheel/test_windows.py +++ b/.ci/scripts/wheel/test_windows.py @@ -13,6 +13,8 @@ import test_base import test_clean_install +import test_cpp_sdk +import test_shared_libraries import torch from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner from executorch.examples.models import Backend, Model, MODEL_NAME_TO_MODEL @@ -86,6 +88,18 @@ def run_tests(model_tests: List[ModelTest]) -> None: test_base.test_cmsis_nn_install() + # The wheel ships the runtime, the kernels, the delegate and the thread pool as + # separate DLLs here too, so check that each has exactly one owner and that all of + # them load. + with tempfile.TemporaryDirectory() as work_dir: + test_shared_libraries.run_tests(Path(work_dir)) + + # And that a C++ application outside the wheel can actually use them. Nothing else + # covers this: the Python extension links those DLLs itself, so it passes whether or + # not the package config names them or the shipped headers are complete. + with tempfile.TemporaryDirectory() as work_dir: + test_cpp_sdk.run_tests(Path(work_dir)) + run_tests( model_tests=[ ModelTest( diff --git a/CMakeLists.txt b/CMakeLists.txt index 737133db121..b70d8243cd9 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -228,17 +228,30 @@ if(EXECUTORCH_BUILD_SHARED) ) endif() # Said here rather than left to fail somewhere downstream, where packaging - # looked for a library the build never emitted. Windows is still refused: - # there the runtime carries no export annotations, so a DLL would link against - # nothing, which is a missing capability rather than a different spelling of - # one. - if(NOT CMAKE_SYSTEM_NAME STREQUAL "Linux" AND NOT APPLE) + # looked for a library the build never emitted. The runtime carries no export + # annotations, so on Windows each shipped component DLL exports all of its + # symbols instead; see executorch_target_shipped_runtime_path. + if(NOT CMAKE_SYSTEM_NAME STREQUAL "Linux" + AND NOT APPLE + AND NOT (WIN32 AND MSVC) + ) message( FATAL_ERROR - "EXECUTORCH_BUILD_SHARED is supported on Linux and macOS only, not " + "EXECUTORCH_BUILD_SHARED is supported on Linux, macOS and Windows (MSVC) only, not " "${CMAKE_SYSTEM_NAME}." ) endif() + # The Windows DLLs take their components as objects through $, + # which CMake added in 3.27. Older versions fail at generate time with an + # expression error that does not name the cause. + if(MSVC AND CMAKE_VERSION VERSION_LESS 3.27) + message( + FATAL_ERROR + "EXECUTORCH_BUILD_SHARED on Windows needs CMake 3.27 or newer, found " + "${CMAKE_VERSION}. Upgrade CMake (pip install 'cmake>=3.27,<4') or " + "build with -DEXECUTORCH_BUILD_SHARED=OFF." + ) + endif() set(CMAKE_POSITION_INDEPENDENT_CODE ON) endif() @@ -1066,14 +1079,35 @@ if(EXECUTORCH_BUILD_SHARED) executorch_shared PUBLIC C10_USING_CUSTOM_GENERATED_MACROS ) - # Link executorch without WHOLE_ARCHIVE because its INTERFACE link options - # (from executorch_target_link_options_shared_lib) already force + # Outside MSVC, link executorch without WHOLE_ARCHIVE because its INTERFACE + # link options (from executorch_target_link_options_shared_lib) already force # whole-archive. Everything else is pulled in through link options rather than # the WHOLE_ARCHIVE link feature, because these archives also reference each # other plainly and CMake before 3.29 refuses to mix a feature with a plain # reference to the same item. - target_link_libraries(executorch_shared PRIVATE executorch) - set(_executorch_shared_whole_archive executorch_core) + if(MSVC) + # Bundled as objects like the rest; linking it would bring its interface + # /WHOLEARCHIVE, which lld-link processes ahead of the bundled objects. + set(_executorch_shared_whole_archive executorch executorch_core) + # The objects come through $, which does not pass on + # executorch_core's cxx_std_20. The public headers need it on Windows (the + # vendored c10 reaches std::countl_zero on the MSVC branch), so a consumer + # linking this target directly has to be told. + target_compile_features(executorch_shared PUBLIC cxx_std_20) + # A DLL resolves its own references at link time, so a weak PAL cannot be + # overridden through it anyway, and the export list + # WINDOWS_EXPORT_ALL_SYMBOLS generates skips weak symbols: without this the + # runtime DLL exports none of the et_pal_* functions, and the public + # clock.h, whose inline ticks_to_ns() calls et_pal_ticks_to_ns_multiplier(), + # fails to link against it. + set_source_files_properties( + ${EXECUTORCH_PAL_DEFAULT_FILE_PATH} PROPERTIES COMPILE_DEFINITIONS + ET_PAL_STRONG_SYMBOLS + ) + else() + target_link_libraries(executorch_shared PRIVATE executorch) + set(_executorch_shared_whole_archive executorch_core) + endif() foreach(_ext_target extension_data_loader extension_flat_tensor extension_named_data_map extension_module_static extension_tensor @@ -1083,7 +1117,8 @@ if(EXECUTORCH_BUILD_SHARED) endif() endforeach() foreach(_whole_target ${_executorch_shared_whole_archive}) - executorch_target_whole_archive(executorch_shared ${_whole_target}) + # EXPORTED: this is the runtime API every other component links. + executorch_target_whole_archive(executorch_shared ${_whole_target} EXPORTED) endforeach() configure_file( tools/cmake/executorch.pc.in ${CMAKE_CURRENT_BINARY_DIR}/executorch.pc @@ -1093,8 +1128,9 @@ if(EXECUTORCH_BUILD_SHARED) DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig ) # pip decides where the wheel lands, so the wheel's copy of this file locates - # everything relative to its own location. - if(EXECUTORCH_BUILD_WHEEL_DO_NOT_USE) + # everything relative to its own location. Not on Windows: its link flags are + # GNU-style (-Wl,-rpath) and a DLL records no search path to point them at. + if(EXECUTORCH_BUILD_WHEEL_DO_NOT_USE AND NOT WIN32) set(_executorch_pc_definitions "-DC10_USING_CUSTOM_GENERATED_MACROS") if(EXECUTORCH_ENABLE_EVENT_TRACER) string(APPEND _executorch_pc_definitions " -DET_EVENT_TRACER_ENABLED") diff --git a/README-wheel.md b/README-wheel.md index 5f9b8819288..9bd359645d8 100644 --- a/README-wheel.md +++ b/README-wheel.md @@ -38,6 +38,10 @@ to run ExecuTorch `.pte` files, with some restrictions: prebuilt module. OpenVINO requires the runtime to be installed separately: `pip install executorch[openvino]` +On Linux, macOS and Windows the runtime, kernels and backends also ship as shared libraries +with headers and a CMake package, so a C++ application can link them without building +ExecuTorch; see [Using ExecuTorch with C++](docs/source/using-executorch-cpp.md). + Please visit the [ExecuTorch website](https://pytorch.org/executorch) for tutorials and documentation. Here are some starting points: * [Getting Started](https://pytorch.org/executorch/main/getting-started-setup) diff --git a/codegen/tools/__init__.py b/codegen/tools/__init__.py index e69de29bb2d..96207c9f06f 100644 --- a/codegen/tools/__init__.py +++ b/codegen/tools/__init__.py @@ -0,0 +1,20 @@ +# Copyright (c) Meta Platforms, Inc. and affiliates. +# All rights reserved. +# +# This source code is licensed under the BSD-style license found in the +# LICENSE file in the root directory of this source tree. + +import os +import sys + +# selective_build links the shipped runtime DLL in executorch/lib, and Windows +# records no search path in a DLL. Not resolved: in an editable install this +# directory is a symlink, and executorch/lib sits beside the link. +_lib_dir = os.path.abspath( + os.path.join( + os.path.dirname(os.path.abspath(__file__)), os.pardir, os.pardir, "lib" + ) +) +if sys.platform == "win32" and os.path.isdir(_lib_dir): + os.add_dll_directory(_lib_dir) +del os, sys, _lib_dir diff --git a/docs/source/using-executorch-cpp.md b/docs/source/using-executorch-cpp.md index 517a79f1655..e175789225a 100644 --- a/docs/source/using-executorch-cpp.md +++ b/docs/source/using-executorch-cpp.md @@ -47,7 +47,7 @@ cover. ### Using the prebuilt libraries from the pip package -On Linux and macOS, current main/nightly wheels ship the runtime as prebuilt +On Linux, macOS and Windows, current main/nightly wheels ship the runtime as prebuilt shared libraries together with the headers and a CMake package. Stable releases from before this packaging was introduced do not contain the namespaced CMake targets used below; use the documentation for your installed release. @@ -159,6 +159,21 @@ That is the two input arrays added together. The long `python -c` part just prin CMake package, so CMake can find it. Run `./build/app` from the folder holding `model.pte`, because the path in `main.cpp` is relative. +On Windows, three things differ. Add the DLL copy step from [the Windows section](#on-windows) to +`CMakeLists.txt`, since a Windows program finds the DLLs only next to itself. Build Release, because +the DLLs use the release C++ library and a Debug build is refused at compile time. And with Visual +Studio the program ends up in a `Release` folder: + +``` +cmake -S . -B build -DCMAKE_PREFIX_PATH="..." -DCMAKE_BUILD_TYPE=Release +cmake --build build --config Release +.\build\Release\app.exe +``` + +`-DCMAKE_BUILD_TYPE` is what a single-configuration generator such as Ninja, the default in VS Code +and CLion, reads; it ignores `--config` and leaves the program in `build\app.exe`. Visual Studio +reads `--config` instead. Giving both covers either. + #### Adding kernels and backends Add a component to both lines to get more. Nothing else in the program changes. @@ -175,12 +190,12 @@ These are the components the package provides: | Component | What it gives you | Where | | --- | --- | --- | -| `runtime` | The engine. Always needed. | Linux, macOS | -| `kernels_optimized` | Fast CPU operators. The usual choice. | Linux, macOS | -| `backend_xnnpack` | The XNNPACK backend, for models exported with it. | Linux, macOS | -| `threadpool` | Multi-threaded execution. | Linux, macOS | +| `runtime` | The engine. Always needed. | Linux, macOS, Windows | +| `kernels_optimized` | Fast CPU operators. The usual choice. | Linux, macOS, Windows | +| `backend_xnnpack` | The XNNPACK backend, for models exported with it. | Linux, macOS, Windows | +| `threadpool` | Multi-threaded execution. | Linux, macOS, Windows | | `etdump` | Profiling, to record what ran and how long it took. | Linux, macOS | -| `kernels_quantized` | The quantized operator kernels | Linux, macOS | +| `kernels_quantized` | The quantized operator kernels | Linux, macOS, Windows | | `kernels_torchao` | The TorchAO low-bit quantized kernels | Linux and macOS, aarch64 only | | `backend_cuda` | The CUDA delegate | Linux | | `extension_cuda` | The CUDA stream extension | Linux | @@ -263,8 +278,9 @@ message(STATUS "Metal kernels: ${MLX_METALLIB_PATH}") #### Using pkg-config -Build systems such as Meson and Autotools read pkg-config files. The wheel ships one for the -runtime. It covers the engine and the thread pool. Name the kernel libraries yourself, the same +Build systems such as Meson and Autotools read pkg-config files. The Linux and macOS wheels ship +one for the runtime; the Windows wheel does not, so use the CMake package there. It covers the +engine and the thread pool. Name the kernel libraries yourself, the same way you add CMake components. Point pkg-config at the file with an absolute path, because the library search path it gives the linker is built from that path: @@ -347,8 +363,9 @@ way, because they sit next to the runtime library. target_link_libraries(app PRIVATE ${EXECUTORCH_QUANTIZED_KERNELS_LIBRARY}) ``` -You should not need `LD_LIBRARY_PATH`. The shipped libraries record where their neighbours live, so -they find each other once the program links against the installed package. +On Linux and macOS you should not need `LD_LIBRARY_PATH`. The shipped libraries record where their +neighbours live, so they find each other once the program links against the installed package. +Windows has no such record; see below. On Linux, linking the runtime asks the linker for `DT_RUNPATH` rather than the older `DT_RPATH`. That is deliberate: `DT_RPATH` is searched before `LD_LIBRARY_PATH` and applies to a dependency's @@ -368,6 +385,31 @@ target_link_libraries(app PRIVATE executorch::runtime prefer_rpath) The order matters. A target's own link options are emitted before those of its dependencies, and the last of the two settings decides the tag for every entry in the link. +#### On Windows + +A Windows DLL records no search path, so the loader finds the shipped DLLs only next to your +program (or on `PATH`). Copy them there after each build; the imported targets name every DLL your +program links: + +```cmake +add_custom_command( + TARGET app POST_BUILD + COMMAND ${CMAKE_COMMAND} -E copy_if_different $ $ + COMMAND_EXPAND_LISTS +) +``` + +Build Release: configure with `-DCMAKE_BUILD_TYPE=Release` for a single-configuration generator such +as Ninja, or build with `cmake --build build --config Release` under Visual Studio. The DLLs are built +against the release C++ library, and a Debug program uses the debug one, whose types are laid out +differently, so mixing the two would corrupt memory. The runtime headers therefore refuse a Debug +build at compile time, with an error asking for Release. On CMake older than 3.28, copy the DLLs from +`${EXECUTORCH_RUNTIME_LIBRARY_DIR}` instead, and apply `${EXECUTORCH_COMPILE_DEFINITIONS}`, which +carries that check. + +The `etdump` component is not offered on Windows, because the Windows wheel is built without the +event tracer. + ### Running on a GPU with the CUDA package The CUDA build is a separate package. Releases cover CUDA 13.2 and 13.4, so pick the index diff --git a/extension/llm/custom_ops/op_tile_crop_aot.py b/extension/llm/custom_ops/op_tile_crop_aot.py index e03a979ffc5..ca4243c3918 100644 --- a/extension/llm/custom_ops/op_tile_crop_aot.py +++ b/extension/llm/custom_ops/op_tile_crop_aot.py @@ -5,6 +5,8 @@ # LICENSE file in the root directory of this source tree. import logging +import os +import sys from pathlib import Path import torch @@ -13,6 +15,20 @@ tile_crop = torch.ops.preprocess.tile_crop.default assert tile_crop is not None except: + # The library depends on the shipped runtime DLLs in executorch/lib, and + # Windows records no search path in a DLL. Not resolved: in an editable + # install this directory is a symlink, and executorch/lib sits beside it. + _lib_dir = os.path.abspath( + os.path.join( + os.path.dirname(os.path.abspath(__file__)), + os.pardir, + os.pardir, + os.pardir, + "lib", + ) + ) + if sys.platform == "win32" and os.path.isdir(_lib_dir): + os.add_dll_directory(_lib_dir) libs = list(Path(__file__).parent.resolve().glob("*custom_ops_aot_lib.*")) assert len(libs) == 1, f"Expected 1 library but got {len(libs)}" logging.info(f"Loading custom ops library: {libs[0]}") diff --git a/extension/pybindings/portable_lib.py b/extension/pybindings/portable_lib.py index f8731b77343..9811cad76fa 100644 --- a/extension/pybindings/portable_lib.py +++ b/extension/pybindings/portable_lib.py @@ -66,6 +66,11 @@ # The extension DLL should be in the same directory as this file. pybindings_dir = os.path.dirname(os.path.abspath(__file__)) os.add_dll_directory(pybindings_dir) + # The shared runtime and its components ship in executorch/lib. Windows + # records no search path in a DLL, so the directory is registered here. + _lib_dir = os.path.join(pybindings_dir, os.pardir, os.pardir, "lib") + if os.path.isdir(_lib_dir): + os.add_dll_directory(os.path.abspath(_lib_dir)) except Exception as e: logger.error( "Failed to add the pybinding extension DLL to the search path. " diff --git a/extension/threadpool/CMakeLists.txt b/extension/threadpool/CMakeLists.txt index ed16cb169ac..42e9fc43ef7 100644 --- a/extension/threadpool/CMakeLists.txt +++ b/extension/threadpool/CMakeLists.txt @@ -53,9 +53,16 @@ if(EXECUTORCH_BUILD_SHARED) # Ships beside libexecutorch.so in the wheel's lib/ directory. executorch_target_shipped_runtime_path(extension_threadpool) # cpuinfo and pthreadpool are forced static, so bundle them inside this - # library instead of making every consumer supply them. + # library instead of making every consumer supply them. pthreadpool is + # EXPORTED, because the other components run their work on this one pool. + # cpuinfo is not: on Windows its feature checks (cpuinfo_has_x86_avx2 and the + # rest) are inline reads of cpuinfo_isa, and cpuinfo.h does not declare that + # data dllimport, so another DLL can only ever read its own copy. Exporting + # the functions let XNNPACK initialize this library's copy and read its own, + # all zeros, and run baseline kernels. Kept private, each DLL that uses + # cpuinfo carries all of it. Elsewhere the whole archive is exported anyway. executorch_target_whole_archive(extension_threadpool cpuinfo) - executorch_target_whole_archive(extension_threadpool pthreadpool) + executorch_target_whole_archive(extension_threadpool pthreadpool EXPORTED) target_link_libraries(extension_threadpool PUBLIC executorch_shared) else() target_link_libraries( diff --git a/extension/threadpool/threadpool.cpp b/extension/threadpool/threadpool.cpp index 2845b4a5473..2bf13b380be 100644 --- a/extension/threadpool/threadpool.cpp +++ b/extension/threadpool/threadpool.cpp @@ -153,7 +153,15 @@ ThreadPool* get_threadpool() { return std::min(result, tsan_thread_limit); })(); +#if defined(_WIN32) + // Never destroyed. At process exit Windows terminates the worker threads + // before running a DLL's static destructors, so tearing the pool down there + // waits on a lock a terminated worker may hold and the process never exits. + static auto& threadpool = *new std::unique_ptr( + std::make_unique(num_threads)); +#else static auto threadpool = std::make_unique(num_threads); +#endif // Inheriting from old threadpool to get around segfault issue // commented above at child_atfork diff --git a/kernels/quantized/__init__.py b/kernels/quantized/__init__.py index 388363047f2..4ab706ed53b 100644 --- a/kernels/quantized/__init__.py +++ b/kernels/quantized/__init__.py @@ -5,8 +5,23 @@ # LICENSE file in the root directory of this source tree. try: + import os + import sys from pathlib import Path + # The library depends on the shipped runtime and quantized kernels DLLs in + # executorch/lib, and Windows records no search path in a DLL. Not resolved: + # in an editable install this directory is a symlink, and executorch/lib + # sits beside the link. + _lib_dir = os.path.abspath( + os.path.join( + os.path.dirname(os.path.abspath(__file__)), os.pardir, os.pardir, "lib" + ) + ) + if sys.platform == "win32" and os.path.isdir(_lib_dir): + os.add_dll_directory(_lib_dir) + del os, sys, _lib_dir + libs = list(Path(__file__).parent.resolve().glob("**/*quantized_ops_aot_lib.*")) del Path assert len(libs) == 1, f"Expected 1 library but got {len(libs)}" diff --git a/pyproject.toml b/pyproject.toml index f7cf98679fb..b632a81628a 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,7 @@ [build-system] requires = [ - "cmake>=3.26,<4.0.0", # For building binary targets in the wheel. 4.0.0 breaks third-party CMake build so temporarily pin the version. + "cmake>=3.26,<4.0.0; sys_platform != 'win32'", # For building binary targets in the wheel. 4.0.0 breaks third-party CMake build so temporarily pin the version. + "cmake>=3.27,<4.0.0; sys_platform == 'win32'", # The shared Windows build uses $, new in 3.27. "packaging>=24.2", # Lower bound required by setuptools "patchelf; sys_platform == 'linux'", # Writes the runtime search paths that let the shipped libraries find each other. "pip>=23", # For building the pip package. diff --git a/requirements-dev.txt b/requirements-dev.txt index 1220547aeb5..dfbfbee1e22 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -1,6 +1,7 @@ # Pip packages needed to build from source. Mainly for development of ExecuTorch. -cmake>=3.26, <4.0.0 # For building binary targets in the wheel. +cmake>=3.26, <4.0.0; sys_platform != 'win32' # For building binary targets in the wheel. +cmake>=3.27, <4.0.0; sys_platform == 'win32' # The shared Windows build uses $, new in 3.27. packaging>=24.2 # Lower bound required by setuptools patchelf; sys_platform == 'linux' # Writes the runtime search paths that let the shipped libraries find each other. pip>=23 # For building the pip package. diff --git a/runtime/platform/compiler.h b/runtime/platform/compiler.h index 692d590f44c..e609af70474 100644 --- a/runtime/platform/compiler.h +++ b/runtime/platform/compiler.h @@ -41,6 +41,20 @@ "Macro clash with min and max -- define NOMINMAX when compiling your program on Windows" #endif +// The prebuilt Windows package defines ET_PREBUILT_RELEASE_CRT or +// ET_PREBUILT_DEBUG_CRT, after the C++ library its DLLs were built with. A +// program built with the other one (/MDd and /MTd define _DEBUG, /MD and /MT do +// not) sees differently laid out types: the two mixed in one process corrupt +// memory instead of failing to link. +#if defined(ET_PREBUILT_RELEASE_CRT) && defined(_DEBUG) +#error \ + "The prebuilt ExecuTorch DLLs use the release C++ library; build this program as Release (configure with -DCMAKE_BUILD_TYPE=Release, or build with --config Release under a multi-config generator such as Visual Studio)" +#endif +#if defined(ET_PREBUILT_DEBUG_CRT) && !defined(_DEBUG) +#error \ + "The prebuilt ExecuTorch DLLs use the debug C++ library; build this program as Debug (configure with -DCMAKE_BUILD_TYPE=Debug, or build with --config Debug under a multi-config generator such as Visual Studio)" +#endif + /* * Define annotations aliasing C++ declaration attributes. * See all C++ declaration attributes here: diff --git a/runtime/platform/default/posix.cpp b/runtime/platform/default/posix.cpp index 837ffd02833..5f5c8f26140 100644 --- a/runtime/platform/default/posix.cpp +++ b/runtime/platform/default/posix.cpp @@ -75,7 +75,7 @@ static bool initialized = false; * This function should be called before any other function provided by the PAL * to initialize any global state. Typically overridden by PAL implementer. */ -#ifdef _MSC_VER +#if defined(_MSC_VER) && !defined(ET_PAL_STRONG_SYMBOLS) #pragma weak et_pal_init #endif // _MSC_VER void et_pal_init(void) { @@ -91,7 +91,7 @@ void et_pal_init(void) { * Immediately abort execution, setting the device into an error state, if * available. */ -#ifdef _MSC_VER +#if defined(_MSC_VER) && !defined(ET_PAL_STRONG_SYMBOLS) #pragma weak et_pal_abort #endif // _MSC_VER ET_NORETURN void et_pal_abort(void) { @@ -103,7 +103,7 @@ ET_NORETURN void et_pal_abort(void) { * * @retval Timestamp value in system ticks. */ -#ifdef _MSC_VER +#if defined(_MSC_VER) && !defined(ET_PAL_STRONG_SYMBOLS) #pragma weak et_pal_current_ticks #endif // _MSC_VER et_timestamp_t et_pal_current_ticks(void) { @@ -122,7 +122,7 @@ et_timestamp_t et_pal_current_ticks(void) { * * @retval The ratio of nanoseconds to system ticks. */ -#ifdef _MSC_VER +#if defined(_MSC_VER) && !defined(ET_PAL_STRONG_SYMBOLS) #pragma weak et_pal_ticks_to_ns_multiplier #endif // _MSC_VER et_tick_ratio_t et_pal_ticks_to_ns_multiplier(void) { @@ -142,7 +142,7 @@ et_tick_ratio_t et_pal_ticks_to_ns_multiplier(void) { * @param[in] message Message string to log. * @param[in] length Message string length. */ -#ifdef _MSC_VER +#if defined(_MSC_VER) && !defined(ET_PAL_STRONG_SYMBOLS) #pragma weak et_pal_emit_log_message #endif // _MSC_VER void et_pal_emit_log_message( @@ -196,7 +196,7 @@ void et_pal_emit_log_message( * @returns the allocated memory, or nullptr on failure. Must be freed using * et_pal_free(). */ -#ifdef _MSC_VER +#if defined(_MSC_VER) && !defined(ET_PAL_STRONG_SYMBOLS) #pragma weak et_pal_allocate #endif // _MSC_VER void* et_pal_allocate(size_t size) { @@ -208,7 +208,7 @@ void* et_pal_allocate(size_t size) { * * @param[in] ptr Pointer to memory to free. May be nullptr. */ -#ifdef _MSC_VER +#if defined(_MSC_VER) && !defined(ET_PAL_STRONG_SYMBOLS) #pragma weak et_pal_free #endif // _MSC_VER void et_pal_free(void* ptr) { diff --git a/setup.py b/setup.py index c0a23cb0195..f6eb6deda74 100644 --- a/setup.py +++ b/setup.py @@ -1335,6 +1335,52 @@ def get_executable_name(name: str) -> str: return name +# Visual Studio is a multi-config generator and writes into a per-config subdirectory. +_CFG = "%BUILD_TYPE%/" if _is_windows() else "" + + +def _windows_import_libraries() -> List["BuiltFile"]: + """The import library beside each DLL offered to C++ consumers, which they link against.""" + if not _is_windows(): + return [] + entries = [ + ("", "executorch_shared", "executorch", []), + ( + "extension/threadpool/", + "executorch_threadpool", + None, + ["EXECUTORCH_BUILD_PTHREADPOOL", "EXECUTORCH_BUILD_CPUINFO"], + ), + ( + "configurations/", + "executorch_kernels_optimized", + None, + ["EXECUTORCH_BUILD_KERNELS_OPTIMIZED"], + ), + ( + "kernels/quantized/", + "executorch_kernels_quantized", + None, + ["EXECUTORCH_BUILD_KERNELS_QUANTIZED"], + ), + ( + "backends/xnnpack/", + "executorch_backend_xnnpack", + None, + ["EXECUTORCH_BUILD_XNNPACK"], + ), + ] + return [ + BuiltFile( + src_dir=f"%CMAKE_CACHE_DIR%/{subdir}{_CFG}", + src_name=f"{built}.lib", + dst=f"executorch/lib/{shipped or built}.lib", + dependent_cmake_flags=["EXECUTORCH_BUILD_SHARED", *flags], + ) + for subdir, built, shipped, flags in entries + ] + + class _BaseExtension(Extension): """A base class that maps an abstract source to an abstract destination.""" @@ -2465,6 +2511,17 @@ def __exit__(self, *args, **kwargs): # https://setuptools.pypa.io/en/latest/userguide/extension.html#setuptools.command.build.SubCommand.get_output_mapping +def _crt_definition(build_type: str) -> str: + """The define naming the C++ library the Windows DLLs were built with. + + A Debug build links the debug C++ library, and a consumer has to match it, so the + runtime headers check this against the consumer's own configuration. + """ + if build_type.lower() == "debug": + return "ET_PREBUILT_DEBUG_CRT" + return "ET_PREBUILT_RELEASE_CRT" + + def _substitute_tracer_definition(path: str, cmake_cache_dir: str) -> None: """Fill in the tracer placeholder in an installed CMake configuration file. @@ -2481,6 +2538,8 @@ def _substitute_tracer_definition(path: str, cmake_cache_dir: str) -> None: enabled = CMakeCache(cache_path=cache_path).is_enabled( "EXECUTORCH_ENABLE_EVENT_TRACER" ) + build_type = CMakeCache(cache_path=cache_path).get("CMAKE_BUILD_TYPE") + crt = _crt_definition(build_type.value if build_type else get_build_type()) with open(path) as handle: contents = handle.read() with open(path, "w") as handle: @@ -2488,7 +2547,7 @@ def _substitute_tracer_definition(path: str, cmake_cache_dir: str) -> None: contents.replace( "@EXECUTORCH_TRACER_DEFINITION@", "ET_EVENT_TRACER_ENABLED" if enabled else "", - ) + ).replace("@EXECUTORCH_CRT_DEFINITION@", crt) ) @@ -2523,7 +2582,7 @@ def _substitute_tracer_definition_from_args(destination) -> None: text.replace( "@EXECUTORCH_TRACER_DEFINITION@", "ET_EVENT_TRACER_ENABLED" if enabled else "", - ) + ).replace("@EXECUTORCH_CRT_DEFINITION@", _crt_definition(get_build_type())) ) @@ -2892,7 +2951,7 @@ def iter_distribution_names(self): # only useful where something upgrades the library independently of # what links it, which never happens inside a wheel. BuiltFile( - src_dir="%CMAKE_CACHE_DIR%/", + src_dir="%CMAKE_CACHE_DIR%/" + _CFG, src_name=get_dynamic_lib_name("executorch"), dst="executorch/lib/" + get_dynamic_lib_name("executorch"), dependent_cmake_flags=["EXECUTORCH_BUILD_SHARED"], @@ -2901,7 +2960,7 @@ def iter_distribution_names(self): # code fused into the Python extension, so a process has one copy of # it however many consumers load. BuiltFile( - src_dir="%CMAKE_CACHE_DIR%/devtools/etdump/", + src_dir="%CMAKE_CACHE_DIR%/devtools/etdump/" + _CFG, src_name=get_dynamic_lib_name("executorch_etdump"), dst="executorch/lib/" + get_dynamic_lib_name("executorch_etdump"), # Not gated on EXECUTORCH_BUILD_DEVTOOLS. The shared build adds @@ -2915,7 +2974,7 @@ def iter_distribution_names(self): # library so that a process has one pool rather than one per # component that uses it. BuiltFile( - src_dir="%CMAKE_CACHE_DIR%/extension/threadpool/", + src_dir="%CMAKE_CACHE_DIR%/extension/threadpool/" + _CFG, src_name=get_dynamic_lib_name("executorch_threadpool"), dst="executorch/lib/" + get_dynamic_lib_name("executorch_threadpool"), @@ -2932,7 +2991,7 @@ def iter_distribution_names(self): # Install the merged CPU kernels beside them, so the operators are # registered once per process rather than once per component. BuiltFile( - src_dir="%CMAKE_CACHE_DIR%/configurations/", + src_dir="%CMAKE_CACHE_DIR%/configurations/" + _CFG, src_name=get_dynamic_lib_name("executorch_kernels_optimized"), dst="executorch/lib/" + get_dynamic_lib_name("executorch_kernels_optimized"), @@ -2945,18 +3004,25 @@ def iter_distribution_names(self): ], ), # For build systems that read pkg-config rather than CMake packages. - BuiltFile( - src_dir="%CMAKE_CACHE_DIR%/", - src_name="executorch-wheel.pc", - dst="executorch/lib/pkgconfig/executorch.pc", - dependent_cmake_flags=["EXECUTORCH_BUILD_SHARED"], + # Not on Windows, where CMakeLists.txt does not generate it. + *( + [] + if _is_windows() + else [ + BuiltFile( + src_dir="%CMAKE_CACHE_DIR%/", + src_name="executorch-wheel.pc", + dst="executorch/lib/pkgconfig/executorch.pc", + dependent_cmake_flags=["EXECUTORCH_BUILD_SHARED"], + ), + ] ), # The CUDA delegate and the process-wide CUDA stream helper, for a # wheel built from a CUDA index. Only present when the build asks for # CUDA, so packaging requires that rather than looking for files a # CPU-only build never produced. BuiltFile( - src_dir="%CMAKE_CACHE_DIR%/backends/cuda/", + src_dir="%CMAKE_CACHE_DIR%/backends/cuda/" + _CFG, src_name=get_dynamic_lib_name("executorch_backend_cuda"), dst="executorch/lib/" + get_dynamic_lib_name("executorch_backend_cuda"), @@ -2990,7 +3056,7 @@ def iter_distribution_names(self): # A C++ application running a quantized model could not link # them before. BuiltFile( - src_dir="%CMAKE_CACHE_DIR%/kernels/quantized/", + src_dir="%CMAKE_CACHE_DIR%/kernels/quantized/" + _CFG, src_name=get_dynamic_lib_name("executorch_kernels_quantized"), dst="executorch/lib/" + get_dynamic_lib_name("executorch_kernels_quantized"), @@ -3028,7 +3094,7 @@ def iter_distribution_names(self): # Install the XNNPACK delegate beside them, so a process has one # copy of it instead of one per component that uses it. BuiltFile( - src_dir="%CMAKE_CACHE_DIR%/backends/xnnpack/", + src_dir="%CMAKE_CACHE_DIR%/backends/xnnpack/" + _CFG, src_name=get_dynamic_lib_name("executorch_backend_xnnpack"), dst="executorch/lib/" + get_dynamic_lib_name("executorch_backend_xnnpack"), @@ -3063,6 +3129,7 @@ def iter_distribution_names(self): "EXECUTORCH_COREML_DELEGATE_LIBRARY_BUILT", ], ), + *_windows_import_libraries(), # Install the prebuilt pybindings extension wrapper for the runtime, # portable kernels, and a selection of backends. This lets users # load and execute .pte files from python. diff --git a/tools/cmake/Utils.cmake b/tools/cmake/Utils.cmake index 9696a340b7c..c6db858bcf8 100644 --- a/tools/cmake/Utils.cmake +++ b/tools/cmake/Utils.cmake @@ -82,7 +82,16 @@ endfunction() # options are also emitted before the ordered link libraries, which keeps a # bundled archive ahead of anything that would otherwise satisfy the same # symbols. +# +# EXPORTED says the archive's symbols are part of what the target exports. On +# Windows that bundles the archive as objects, because the export list +# WINDOWS_EXPORT_ALL_SYMBOLS builds covers only the target's own objects; +# without it the archive is bundled with /WHOLEARCHIVE and stays private to the +# DLL. Stated by the caller rather than inferred from that property, so the +# result does not depend on the order of calls. Ignored elsewhere, where the +# whole archive is exported either way. function(executorch_target_whole_archive target_name archive_target) + cmake_parse_arguments(ARG "EXPORTED" "" "" ${ARGN}) # One self-contained option per archive. The path sits inside the option so # its text is unique, which matters because CMake removes a duplicate option # and that would leave every archive after the first outside the scope, @@ -104,6 +113,25 @@ function(executorch_target_whole_archive target_name archive_target) ${target_name} PRIVATE "SHELL:LINKER:-force_load,$" ) + elseif(MSVC AND ARG_EXPORTED) + # Objects rather than /WHOLEARCHIVE: WINDOWS_EXPORT_ALL_SYMBOLS builds the + # .def from the target's own objects and cannot read an archive, so a + # whole-archived runtime would link but export nothing. COMPILE_ONLY takes + # the usage requirements without the archive or its interface link options: + # a /WHOLEARCHIVE among them is processed ahead of these objects and pulls + # the same members out of the archive again, which is a duplicate symbol. + # The ordinary link below is not needed: the objects are direct inputs of + # this link, so a rebuilt archive cannot leave stale contents behind. + target_sources(${target_name} PRIVATE $) + target_link_libraries( + ${target_name} PRIVATE $ + ) + return() + elseif(MSVC) + target_link_options( + ${target_name} PRIVATE + "LINKER:/WHOLEARCHIVE:$" + ) else() target_link_options( ${target_name} @@ -127,11 +155,17 @@ function(executorch_target_link_options_shared_lib target_name) # constructor. Export scoped --no-as-needed retention instead, which is what # actually keeps a registration-only shared library on the link line. get_target_property(_target_type ${target_name} TYPE) + # A Windows DLL is retained by its anchor, which + # executorch_target_shipped_runtime_path adds together with the /INCLUDE its + # consumers need; a DLL without one has nothing to reference. + if(_target_type STREQUAL "SHARED_LIBRARY" AND MSVC) + return() + endif() # A shared library is never an archive, so the archive handling below does not # apply to one on any platform. On Apple it actively harms: -force_load on a # shared library makes every consumer absorb a copy of its contents, which put # a second operator registry inside the runtime library. - if(_target_type STREQUAL "SHARED_LIBRARY" AND NOT MSVC) + if(_target_type STREQUAL "SHARED_LIBRARY") # Mach-O keeps a library named on the link line whether or not anything # references it, so there is nothing to counter and ld rejects the GNU # flags. @@ -401,6 +435,33 @@ function(executorch_target_retain_shared_library target_name library_target) target_link_options( ${target_name} PRIVATE "$" ) + elseif(MSVC) + # The Visual Studio generator emits link options after the libraries, so the + # import library is prepended to LINK_LIBRARIES instead, as a plain path so + # it carries no usage requirements: the runtime's + # C10_USING_CUSTOM_GENERATED_MACROS would strip torch's dllimport from a + # target compiling against ATen. First on the line, it resolves the runtime + # ahead of any static core that arrives transitively. The anchor keeps a + # registration-only DLL in the import table. + get_target_property(_existing_link_libraries ${target_name} LINK_LIBRARIES) + if(NOT _existing_link_libraries) + set(_existing_link_libraries "") + endif() + set_property( + TARGET ${target_name} + PROPERTY LINK_LIBRARIES "$" + ${_existing_link_libraries} + ) + target_link_options( + ${target_name} PRIVATE + "LINKER:/INCLUDE:$" + ) + add_dependencies(${target_name} ${library_target}) + target_include_directories( + ${target_name} + PRIVATE $ + ) + return() else() target_link_options( ${target_name} @@ -420,8 +481,57 @@ endfunction() # records, and in the build tree these libraries are NOT siblings: the runtime # sits at the top while the others are in their own subdirectories. Those # recorded directories are what resolves them there, and packaging strips them -# so nothing absolute ships. +# so nothing absolute ships. A registration-only Windows DLL stays in a +# consumer's import table only if the consumer references one of its symbols, so +# each shipped DLL exports an empty C function its consumers force-reference +# with /INCLUDE, the counterpart of --no-as-needed. Named after the shipped +# file, without any configuration postfix, so the name is the same in every +# configuration. Written out rather than left as a generator expression, because +# the link option is exported and an installed imported target has no +# OUTPUT_NAME to evaluate it against. Scheduled by +# executorch_target_shipped_runtime_path for the end of configure. +function(_executorch_add_windows_anchor target_name binary_dir) + get_target_property(_name ${target_name} OUTPUT_NAME) + if(NOT _name) + set(_name ${target_name}) + endif() + set(_anchor "executorch_anchor_${_name}") + set(_anchor_source "${binary_dir}/${target_name}_anchor.cpp") + # Written now rather than with file(GENERATE): this runs in the top-level + # directory, and a generated file is only known as such in the directory that + # generates it, so the target's own directory would look for it on disk. + file(CONFIGURE OUTPUT "${_anchor_source}" CONTENT + "extern \"C\" __declspec(dllexport) void ${_anchor}(void) {}\n" + ) + target_sources(${target_name} PRIVATE "${_anchor_source}") + set_target_properties( + ${target_name} PROPERTIES EXECUTORCH_ANCHOR "${_anchor}" + ) + target_link_options(${target_name} INTERFACE "LINKER:/INCLUDE:${_anchor}") +endfunction() + function(executorch_target_shipped_runtime_path target_name) + if(WIN32) + # No export annotations in the runtime, so each shipped component DLL + # exports all of its symbols, plus a C anchor consumers force-reference to + # keep it loaded. A target that set WINDOWS_EXPORT_ALL_SYMBOLS OFF exports + # through its own annotations and keeps that choice. + get_target_property(_export_all ${target_name} WINDOWS_EXPORT_ALL_SYMBOLS) + if(NOT DEFINED _export_all OR _export_all STREQUAL "_export_all-NOTFOUND") + set_target_properties( + ${target_name} PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON + ) + endif() + # The anchor is named after the shipped file, which some targets name only + # after this call (the optimized kernels are renamed once gen_operators_lib + # returns), so it is added when configuration ends. + cmake_language( + EVAL + CODE + "cmake_language(DEFER DIRECTORY [[${CMAKE_SOURCE_DIR}]] CALL _executorch_add_windows_anchor [[${target_name}]] [[${CMAKE_CURRENT_BINARY_DIR}]])" + ) + return() + endif() # Mach-O spells the same idea @loader_path. if(APPLE) set(_origin "@loader_path") diff --git a/tools/cmake/executorch-wheel-config.cmake b/tools/cmake/executorch-wheel-config.cmake index 0cfdf80b370..3f6ffc9268a 100644 --- a/tools/cmake/executorch-wheel-config.cmake +++ b/tools/cmake/executorch-wheel-config.cmake @@ -84,7 +84,7 @@ # OpenVINO runtime by name, which a C++ program # installs and points OPENVINO_LIB_PATH at. # executorch::threadpool The shared thread pool. -# executorch::etdump The profiler. +# executorch::etdump The profiler. Not on Windows. # ~~~ # # EXECUTORCH_LIBRARIES carries every component except the quantized kernels, @@ -219,6 +219,18 @@ set(EXECUTORCH_COMPILE_DEFINITIONS C10_USING_CUSTOM_GENERATED_MACROS # targets and would otherwise compile these headers with whatever its compiler # defaults to. set(EXECUTORCH_CXX_STANDARD 17) +if(WIN32) + # The vendored c10 headers reach std::countl_zero on the MSVC branch, which is + # why executorch_core states cxx_std_20 on Windows. + set(EXECUTORCH_CXX_STANDARD 20) + # The DLLs use the C++ library of the configuration the wheel was built in, + # release for a normal build and debug for DEBUG=1, which packaging fills in + # here. A program built in the other configuration uses the other library, + # whose types are laid out differently, so the two mixed would corrupt memory + # rather than fail to link. This makes the runtime headers refuse that build + # at compile time; see runtime/platform/compiler.h. + list(APPEND EXECUTORCH_COMPILE_DEFINITIONS "@EXECUTORCH_CRT_DEFINITION@") +endif() foreach(_required_include ${EXECUTORCH_INCLUDE_DIRS}) if(NOT EXISTS "${_required_include}") message( @@ -266,6 +278,10 @@ function(_executorch_find_library _output _base_name) file(GLOB _matches "${_executorch_package_root}/lib/${_base_name}.dylib" "${_executorch_package_root}/lib/${_base_name}.*.dylib" ) + elseif(WIN32) + # PE names carry no lib prefix and no version. + string(REGEX REPLACE "^lib" "" _windows_name "${_base_name}") + file(GLOB _matches "${_executorch_package_root}/lib/${_windows_name}.dll") else() file(GLOB _matches "${_executorch_package_root}/lib/${_base_name}.so" "${_executorch_package_root}/lib/${_base_name}.so.*" @@ -321,7 +337,16 @@ if(_executorch_runtime_library AND NOT _executorch_targets_supported) # imported targets express the same thing through link options, which this # older-CMake route cannot use. set(EXECUTORCH_FOUND ON) - list(APPEND EXECUTORCH_LIBRARIES "${_executorch_runtime_library}") + if(WIN32) + # A consumer links the import library beside a DLL, not the DLL itself. + string(REGEX REPLACE "\\.dll$" ".lib" _executorch_runtime_link_file + "${_executorch_runtime_library}" + ) + list(APPEND EXECUTORCH_LIBRARIES "${_executorch_runtime_link_file}") + unset(_executorch_runtime_link_file) + else() + list(APPEND EXECUTORCH_LIBRARIES "${_executorch_runtime_library}") + endif() # Every shipped library, not only the kernels. A delegate registers itself # from a static initializer, so leaving one out gave a clean configure and # then a load failure saying the backend is not registered, which reads as a @@ -347,6 +372,10 @@ if(_executorch_runtime_library AND NOT _executorch_targets_supported) libexecutorch_threadpool libexecutorch_etdump ) + # Not offered on Windows, for the reason given at the component definition. + if(WIN32 AND _executorch_component STREQUAL "libexecutorch_etdump") + continue() + endif() _executorch_find_library( _executorch_component_library "${_executorch_component}" ) @@ -361,6 +390,18 @@ if(_executorch_runtime_library AND NOT _executorch_targets_supported) EXECUTORCH_LIBRARIES "-Wl,--push-state,--no-as-needed,${_executorch_component_library},--pop-state" ) + elseif(WIN32) + # The import library, plus a reference to the DLL's anchor so the linker + # keeps a registration-only DLL in the import table. + string(REGEX REPLACE "\\.dll$" ".lib" _executorch_component_link_file + "${_executorch_component_library}" + ) + string(REGEX REPLACE "^lib" "" _executorch_component_base + "${_executorch_component}" + ) + list(APPEND EXECUTORCH_LIBRARIES "${_executorch_component_link_file}" + "-INCLUDE:executorch_anchor_${_executorch_component_base}" + ) else() list(APPEND EXECUTORCH_LIBRARIES "${_executorch_component_library}") endif() @@ -391,6 +432,14 @@ if(_executorch_runtime_library AND NOT _executorch_targets_supported) set(EXECUTORCH_QUANTIZED_KERNELS_LIBRARY "-Wl,--push-state,--no-as-needed,${EXECUTORCH_QUANTIZED_KERNELS_LIBRARY},--pop-state" ) + elseif(EXECUTORCH_QUANTIZED_KERNELS_LIBRARY AND WIN32) + string(REGEX REPLACE "\\.dll$" ".lib" EXECUTORCH_QUANTIZED_KERNELS_LIBRARY + "${EXECUTORCH_QUANTIZED_KERNELS_LIBRARY}" + ) + set(EXECUTORCH_QUANTIZED_KERNELS_LIBRARY + "${EXECUTORCH_QUANTIZED_KERNELS_LIBRARY}" + "-INCLUDE:executorch_anchor_executorch_kernels_quantized" + ) endif() message( STATUS @@ -433,6 +482,22 @@ elseif(_executorch_runtime_library) INTERFACE_COMPILE_DEFINITIONS "C10_USING_CUSTOM_GENERATED_MACROS;@EXECUTORCH_TRACER_DEFINITION@" ) + if(WIN32) + string(REGEX REPLACE "\\.dll$" ".lib" _executorch_runtime_implib + "${_executorch_runtime_library}" + ) + # The release C++ library check, see EXECUTORCH_COMPILE_DEFINITIONS. + set_target_properties( + executorch::runtime + PROPERTIES IMPORTED_IMPLIB "${_executorch_runtime_implib}" + INTERFACE_COMPILE_FEATURES cxx_std_20 + ) + set_property( + TARGET executorch::runtime + APPEND + PROPERTY INTERFACE_COMPILE_DEFINITIONS "@EXECUTORCH_CRT_DEFINITION@" + ) + endif() # $ORIGIN comes first so an application deployed beside its own copy of the # runtime finds that copy. The loader takes the first match, so leading with # the install directory would send a relocated application back to the @@ -581,6 +646,21 @@ function(_executorch_define_component _suffix _library_name) "LINKER:-rpath,@loader_path/../lib" "LINKER:-rpath,${_executorch_package_root}/lib" ) + elseif(WIN32) + # No search path to record; the DLL has to sit beside the application. The + # anchor reference is what keeps a registration-only DLL in the import + # table. + string(REGEX REPLACE "\\.dll$" ".lib" _implib "${_library}") + set_target_properties( + ${_target} PROPERTIES IMPORTED_IMPLIB "${_implib}" + INTERFACE_COMPILE_FEATURES cxx_std_20 + ) + set_property( + TARGET ${_target} + APPEND + PROPERTY INTERFACE_LINK_OPTIONS + "LINKER:/INCLUDE:executorch_anchor_${_library_name}" + ) endif() if(NOT _component_OPT_IN) set(EXECUTORCH_LIBRARIES @@ -622,8 +702,12 @@ if(TARGET executorch::kernels_quantized) endif() # The profiler. A C++ application could not record timing data from an installed # package before, because the implementation shipped only inside the Python -# extension. -_executorch_define_component(etdump executorch_etdump) +# extension. Not offered on Windows, where the wheel is built without the event +# tracer and the library would record nothing; it ships there only because the +# Python extension links it. +if(NOT WIN32) + _executorch_define_component(etdump executorch_etdump) +endif() # The switch a source build sets, on the runtime rather than on the thread pool # target. The guarded declaration lives in a runtime header that every component diff --git a/tools/cmake/preset/default.cmake b/tools/cmake/preset/default.cmake index 89e679aec7c..2e6330837c4 100644 --- a/tools/cmake/preset/default.cmake +++ b/tools/cmake/preset/default.cmake @@ -240,7 +240,8 @@ define_overridable_option( ) define_overridable_option( EXECUTORCH_BUILD_SHARED - "Build a consolidated ExecuTorch shared library (Linux and macOS)" BOOL OFF + "Build a consolidated ExecuTorch shared library (Linux, macOS and Windows)" + BOOL OFF ) # Threadpool size options. At most one can be specified. Note that the default diff --git a/tools/cmake/preset/pybind.cmake b/tools/cmake/preset/pybind.cmake index d1a8df20bef..d3a28344a3f 100644 --- a/tools/cmake/preset/pybind.cmake +++ b/tools/cmake/preset/pybind.cmake @@ -132,13 +132,18 @@ elseif(CMAKE_SYSTEM_NAME STREQUAL "Linux") endif() set_overridable_option(EXECUTORCH_BUILD_OPENVINO OFF) # Ship one shared runtime that both the pybind extension and standalone C++ - # consumers link, so a process has a single backend registry. Not set on - # Windows, where the runtime has no export annotations for a DLL. + # consumers link, so a process has a single backend registry. set_overridable_option(EXECUTORCH_BUILD_SHARED ON) elseif(CMAKE_SYSTEM_NAME STREQUAL "Windows" OR CMAKE_SYSTEM_NAME STREQUAL "WIN32" ) - # Windows or other OS-specific code here + # One shared runtime, as on Linux and macOS. The event tracer stays off here, + # so the profiler library ships only because the Python extension links it. + # Not with CUDA: the Windows CUDA delegate has not been built as a DLL, so a + # CUDA build keeps the static layout it had. + if(NOT EXECUTORCH_BUILD_CUDA) + set_overridable_option(EXECUTORCH_BUILD_SHARED ON) + endif() else() message( FATAL_ERROR "Unsupported CMAKE_SYSTEM_NAME for pybind: ${CMAKE_SYSTEM_NAME}"