[LionelColaso]

Adds native Windows support end to end: LocalAI now releases a windows/amd64 server binary, and the llama-cpp backend is built natively for windows/amd64 under MSYS2 UCRT64 and packaged as an OCI image tar that LocalAI installs and runs as a native process - no docker daemon or WSL required on the host.

Server side

• goreleaser: add windows (amd64/arm64) to the release targets

• Makefile: download the win64 protoc zip and rename protoc.exe to protoc, resolve code-gen plugins via --plugin instead of PATH, force SHELL=sh and name the binary local-ai.exe on Windows, ignore protoc.exe

• build-test.yaml: add a native windows-latest build gate that installs GNU make via Chocolatey, adds Git for Windows' usr/bin to PATH and builds with CGO_ENABLED=0

• pkg/downloader: close the write handle before removing or renaming a partial download so Windows file locks do not break the resume and error paths; guard the POSIX-permission and symlink tests on non-Windows

• tests: Windows guards and path fixes for core/gallery, video_internal, loader and the testcontainers database setup

Backend side

• scripts/build/llama-cpp-windows.sh: builds gRPC from source (pinned v1.59.0, with mingw-w64 fixes for c-ares, boringssl and zlib), then the three llama.cpp variants (cpu-all with GGML_CPU_ALL_VARIANTS + Vulkan, rpc, and the AVX-off fallback), bundles the mingw runtime DLLs and ships an OCI tar via local-ai util create-oci-image. Re-runnable and auto-dispatches into MSYS2 when launched from Git for Windows' bash. JOBS override supported for memory-limited hosts.

• core/gallery: backends install skips the OCI-registry digest lookup for ocifile:// streams (a local tarball is not a registry reference), so the local-build install step no longer prints a confusing could not parse reference digest warning

• backend/cpp/llama-cpp/run.ps1: PowerShell launcher (mirrors run.sh) that pkg/model/process.go starts on Windows

• backend/index.yaml: windows/amd64 backend entry and variants

• pkg/system/capabilities.go: windows engine preference rules so the gallery picks the native build on Windows hosts

• .github/backend-matrix.yml + backend_build_windows.yml: windows matrix entries and a reusable windows build workflow; backend.yml and backend_pr.yml wire the windows backend jobs (build on PR, publish on master)

• docs/content/getting-started/windows.md plus related page updates (GPU-acceleration, install)

Notes for Reviewers

• All 5 review items from mudler-agent's Oct 27 review are resolved:

  1. ✅ Missing path/filepath import in store_unix.go — added
  2. ✅ Windows backend catalog targets — added windows-llama-cpp and windows-llama-cpp-development entries to gallery/backends/index.yaml
  3. ✅ Windows store locks — replaced process-local maps with LockFileEx OS-backed locks in policy, artifacts, ownership, and selection stores
  4. ✅ Store close lock leak — resolved; OS-backed lock releases on handle close, no map entry to clean up
  5. ✅ Process-launch race — refactored behind processLauncher interface; Windows uses CREATE_SUSPENDED → assign job tree → ResumeThread, so children can't escape before assignment

• Additional fixes:

  • Windows blob rename — blob.go and fs.go now tolerate Windows "Access is denied" when os.Rename targets an existing file
  • Default addon host policy on Windows — defaults to empty policy to suppress first-launch warning
  • Deduplicated directory-sync tolerance — pkg/atomicfile.SyncDirTolerant() shared by journal.go, artifacts/store.go, resources/selection.go, jsonstore/store.go, blob/fs.go
  • Launcher interface — processLauncherWindows/processLauncherOther; process.go has no runtime.GOOS branch
  • Unit tests — process_launcher_windows_test.go, process_launcher_other_test.go
  • Developer guide — .agents/windows-backends.md covering all 9 steps to add a Windows backend
  • Fixed malformed build tags — tests/addons/foundation/cluster/*.go //go:build !windows// → proper newline

• The Windows backend image ships three llama-cpp executables picked by the run.ps1 launcher: llama-cpp-cpu-all.exe (all ggml CPU variants, auto-detects a Vulkan device at runtime and falls back to CPU), llama-cpp-grpc.exe (gRPC-RPC build, selected when LLAMACPP_GRPC_SERVERS is set) and llama-cpp-fallback.exe (static, AVX-off fallback). The mingw runtime DLLs are bundled in the image, so no MSYS2 install is needed on the host.

• macOS is intentionally unaffected; the Darwin matrix entries are unchanged.

• Review follow-ups (richiejp): backend processes now run inside a Windows Job Object with JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE (pkg/model/windows_job_windows.go), so model unloads, graceful shutdown, and an abrupt local-ai.exe exit all reap the wrapper + backend tree; it degrades to a logged warning when the host already nests the process in a non-breakaway job.

• Windows smoke suite (tests/e2e/windows): builds and boots the real local-ai.exe with the mock-backend laid out exactly as a gallery install ships it (run.sh for discovery, run.ps1 to launch) and asserts a chat completion via run.ps1 plus job-object tree reaping after hard-killing local-ai.exe. Self-skips on non-Windows hosts; wired as make test-windows-smoke and a windows-latest job in tests-e2e.yml. 2/2 specs pass locally on a Windows host and on the tests-windows-smoke CI job.

• backend_build_windows.yml's publish job now signs the windows images keyless with cosign (same flow as backend_merge.yml); the gallery verification: block stays unpopulated until it can cover every published variant.

• Commits are DCO-signed and carry an Assisted-by: trailer per .agents/ai-coding-assistants.md.

Signed commits

• Yes, I signed my commits.

• Documentation updated (docs/content/) for user-facing changes, or not applicable

Closes #2368

[localai-org-maint-bot]

@mudler the Windows launcher concerns are resolved on the rebased head: the unused run.cmd is removed, the native run.exe path is now explained, and backends/llama-cpp-windows is serialized in .NOTPARALLEL. I reviewed the range-diff from the prior head; git diff --check, the launcher build, pkg/system, and all 48 backend-filter tests pass. Good from my side once the newly restarted Windows/backend CI completes.

[localai-org-maint-bot]

@mudler the requested launcher change is addressed on the rebased head: the compiled run.exe helper is gone, run.ps1 mirrors the binary selection and PATH setup, and pkg/model/process.go invokes it through Windows PowerShell. I isolated the patch from the rebase and verified git diff --check, all 48 backend-filter tests, and pkg/system; the contributor also reports native Windows launcher coverage for selection, argument forwarding, paths with spaces, and exit propagation. Good from my side; only DCO is currently reported, so the Windows/backend workflow result is still pending.

[localai-org-maint-bot]

Found a runtime blocker: pkg/model/process.go sets process.WithName("powershell.exe") (a bare filename) together with process.WithWorkDir(workDir). On Windows, Go's syscall.StartProcess resolves the application name relative to attr.Dir when both are set — so it looks for <workDir>\powershell.exe instead of searching PATH, and CreateProcess with an explicit lpApplicationName does no PATH lookup. Every Windows backend launch will fail with ERROR_FILE_NOT_FOUND.

Fix: resolve the full path before calling WithName, e.g. exec.LookPath("powershell.exe") (which searches PATH and finds System32), or hardcode C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe.

The rest of the PR looks sound — capability detection ordering, docs, CI workflows. Once this is fixed, it should be good to merge.

[mudler]

we are almost there :) thanks for the patience @LionelColaso. Last review pass and then should be good to go

[mudler-agent]

Two findings from the Windows support review. Rechecked against the current head; the affected code is unchanged.

[LionelColaso]

Just small code nits on my side, but otherwise the direction looks good. Unfortunately, I do not have windows to test this, so we will have to see via CI and user reports how this behaves in real hardware.

I use it and tested it both (local-ai.exe and llama-cpp backend) everything seems to be working and been using it.
also seems to pass all checks and tests on mirror pr here for my testing LionelColaso#2
if i find any bugs or any bugs get reported in the issues section i will try fixing it and creating a pull request for it
may support more backends if any one wants it but i belive most will be content with using llama-cpp backend alone
if u have any further changes to be done let me know.

[LionelColaso]

we are almost there :) thanks for the patience @LionelColaso. Last review pass and then should be good to go

No problem I understand its not easy it was good experience and thanks for having patience as well now everything should be working as per my local testing

Hopefully all well now Thanks

[mudler-agent]

Thanks for the work on this. It needs a rebase because it conflicts with master. Two items from the earlier automated review that I could not confirm are fixed on the current head, could you check both?

• scripts/build/llama-cpp-windows.sh (around line 683): the gRPC staging directory is not created before prepare.sh, so a fresh build fails.
• pkg/model/process.go (around line 340): the Windows job object is assigned after PowerShell starts, so children can escape it and orphan the backend and its GPU memory.

[mudler]

Thanks @LionelColaso !

[mudler-agent]

Thanks for continuing to work on this! I reviewed the current head, 68406cba13af3ae3c3c6b4818fd2a83fed040ffd. The earlier missing grpc-server staging-directory issue is fixed. I still see these issues to address before merging:

1. Unix build failure — missing import. core/services/addons/artifacts/store_unix.go:11 calls filepath.Join but does not import path/filepath. Current-head UI CI reports undefined: filepath. Please add the import and recheck the Unix build.

2. Windows backend catalog targets are missing. gallery/backends/index.yaml:27,4034 maps the release and development aliases to windows-llama-cpp and windows-llama-cpp-development, but neither concrete entry exists. The generated compatibility index has the same gap. Normal Windows backend installation fails during meta-backend resolution, before downloading the image. The backend-library CI check reports both missing targets. Please add the concrete Windows entries through the catalog workflow and regenerate the compatibility index.

3. Windows store locks need interprocess exclusion. core/services/addons/policy/store_windows.go:21–29, and the corresponding artifacts, ownership and selection implementations, use process-local maps instead of OS locks. Two LocalAI processes sharing a state directory can both acquire the lock and modify durable state concurrently. Please use an OS-backed exclusive lock, such as LockFileEx, and test contention between separate processes.

4. Closing a Windows store leaves its map lock occupied. The map entry is removed only by unlockFile, whereas policy.Store.Close() (policy/store.go:345) only closes the file. Failed-open cleanup does the same. Closing and reopening a store in the same process therefore leaves it reported as locked; the artifacts and selection stores have the same lifetime mismatch. An OS-backed lock released with its handle would also address this. Please cover close/reopen and failed-open/retry paths.

5. The production process-launch ordering still has the previously reported race. pkg/model/process.go:329–340 calls grpcControlProcess.Run() before assigning the Windows job. PowerShell can spawn the backend before assignment, and existing children are not retroactively enrolled. Abrupt LocalAI termination can then leave the backend alive. The new startProcessWindows helper has no callers, so it does not change production ordering. Also, its ResumeThread(windows.Handle(cmd.Process.Pid)) passes a PID rather than a thread handle; please do not wire it in unchanged. The actual launch path needs suspend/assign/resume or an equivalent startup handshake, with a regression test exercising that path rather than a test-only child-creation gate.

Validation: all 51 backend-filter tests, pkg/system tests, shell syntax and diff whitespace checks pass. The compile and catalog issues are confirmed by current-head CI. The lock and launch findings are based on source analysis; I have not reproduced them on a Windows host. Native Windows/MSYS2 packaging, DLL closure, Vulkan initialization and real llama.cpp inference still need execution evidence—the mock-backend smoke suite does not establish those.

[LionelColaso]

I need to add some fixes for issue related to addons which causes build to fail on current updated source so till then its not merge ready