#2328: Use Starlark rules#2429
Conversation
97c351e to
0ea4978
Compare
|
Looks like this is broken in CI, and I don't have a clue where to begin to fix it. I'll take another look on Monday. |
|
Thank you, that looks quite good. It just seems there is a problem with running the example and the start script regarding rules_java. This is because you only added rules_java to the WORKSPACE file for rules_haskell, which only has an effect on this project (all tests are green for rules_haskell) but does not have any effect on other workspaces (such as the example or in the project created by the You should add a similar BTW, usually we try to be quite conservative with module updates in MODULE.bazel files for the main module and only bump bazel_deps in the test module. Due to the MVS algorithm we ensure that things work for newer versions but don't force anyone else using rules_haskell to use the latest version of our dependencies. This is in contrast to WORKSPACE mode where one is free to use a different version whatsoever. See https://github.com/tweag/renovate-presets#why-are-updates-to-the-root-modulebazel-ignored for the rationale. Having said that, if things work with the current version bumps, let's keep them this way as Bazel 6 is EOL already and we really ought to move on to Bazel 7 and 8... One thing that failed after the version bump of protobuf is the Windows compilation, see https://github.com/tweag/rules_haskell/actions/runs/26277294428/job/77344953463?pr=2429 -- so that should probably be reverted. |
|
Thanks @avdv, that makes a lot of sense, I'll see if I can get things working. |
|
That looks much better now, only a few failing jobs left... All the failing jobs use bzlmod (the One error looks familiar to me: https://github.com/tweag/rules_haskell/actions/runs/26503196184/job/78048731156#step:10:1466 It is the same error we see when trying to upgrade rules_cc to version 0.2.0 in this PR (job: https://github.com/tweag/rules_haskell/actions/runs/23786889871/job/69312225140) Is bumping the rules_cc module really necessary? Maybe it works with lower versions... However, if we can fix the problem, that would be even better. |
20795fb to
b9a1411
Compare
|
FYI, I tried rules_cc 0.0.15 because it seemed to work on my machine, but actually was not the case. |
3745a09 to
21b7669
Compare
7567617 to
cb5d04f
Compare
|
Unfortunately, it still did not work even with rules_cc version 0.1.1. I took a deeper look and compared the cc toolchain of the old and new versions and I realized that the problem is the same we saw after upgrading Bazel -- the rules_cc toolchain does no longer produce shared libraries on macOS for cc_library targets. But the failing test depends on a shared library to work... So, I added a cc_shared_library explicitly and this resolves the problem. 🎉 There are now these issues left:
For 1) I already added a patch to protoc which we also applied to rules_haskell_test. But it seems that's not enough. \edit: I upgraded abseil-cpp and that fixed it. For 3), this seems to be caused by using a nixpkgs based C/C++ compiler which does not work when the nix specific variables are not set (e.g. inside of isolated build actions). It seems the example does not register a nixpkgs based cc toolchain for bzlmod. \edit: adding a nixpkgs based cc toolchain fixed this |
e1d9f12 to
2dba7fd
Compare
|
Some debugging output: # "bazel-out/x64_windows-opt-exec-2B5CBBC6/bin/external/rules_haskell~1.0/haskell/cc_wrapper-bash.exe" "-fstack-check" "-o" "bazel-out/x64_windows-fastbuild/bin/tests/library-external-workspace/bin-deps.exe" "-lm" "-no-pie" "-Wl,--enable-auto-import" "-Wl,--gc-sections" "bazel-out\x64_windows-fastbuild\bin\tests\library-external-workspace\_obj\bin-deps\Main.o" "C:\msys64\tmp\ghc716_0\ghc_5.o" "-Lbazel-out/x64_windows-fastbuild/bin/tests/library-external-workspace/" "-Lbazel-out/x64_windows-fastbuild/bin/tests/library-external-workspace/../../external/_main~non_module_deps~library_repo/" "-LC:\_bzl\33jjwvv5\external\rules_haskell~1.0~haskell_toolchains~bindist_rules_haskell_tests_1.0_windows_amd64\lib\x86_64-windows-ghc-9.4.8\base-4.17.2.1" "-LC:\_bzl\33jjwvv5\external\rules_haskell~1.0~haskell_toolchains~bindist_rules_haskell_tests_1.0_windows_amd64\lib\x86_64-windows-ghc-9.4.8\ghc-bignum-1.3" "-LC:\_bzl\33jjwvv5\external\rules_haskell~1.0~haskell_toolchains~bindist_rules_haskell_tests_1.0_windows_amd64\lib\x86_64-windows-ghc-9.4.8\ghc-prim-0.9.1" "-LC:\_bzl\33jjwvv5\external\rules_haskell~1.0~haskell_toolchains~bindist_rules_haskell_tests_1.0_windows_amd64\lib\x86_64-windows-ghc-9.4.8\rts-1.0.2" "C:\msys64\tmp\ghc716_0\ghc_2.o" "-Wl,-u,hs_atomic_add64" "-Wl,-u,hs_atomic_sub64" "-Wl,-u,hs_atomic_and64" "-Wl,-u,hs_atomic_nand64" "-Wl,-u,hs_atomic_or64" "-Wl,-u,hs_atomic_xor64" "-Wl,-u,hs_atomicread64" "-Wl,-u,hs_atomicwrite64" "-Wl,-u,base_GHCziTopHandler_runIO_closure" "-Wl,-u,ba
se_GHCziTopHandler_runNonIO_closure" "-Wl,-u,ghczmprim_GHCziTuple_Z0T_closure" "-Wl,-u,ghczmprim_GHCziTypes_True_closure" "-Wl,-u,ghczmprim_GHCziTypes_False_closure" "-Wl,-u,base_GHCziPack
_unpackCString_closure" "-Wl,-u,base_GHCziWeak_runFinalizzerBatch_closure" "-Wl,-u,base_GHCziIOziException_stackOverflow_closure" "-Wl,-u,base_GHCziIOziException_heapOverflow_closure" "-Wl
,-u,base_GHCziIOziException_allocationLimitExceeded_closure" "-Wl,-u,base_GHCziIOziException_blockedIndefinitelyOnMVar_closure" "-Wl,-u,base_GHCziIOziException_blockedIndefinitelyOnSTM_clo
sure" "-Wl,-u,base_GHCziIOziException_cannotCompactFunction_closure" "-Wl,-u,base_GHCziIOziException_cannotCompactPinned_closure" "-Wl,-u,base_GHCziIOziException_cannotCompactMutable_closu
re" "-Wl,-u,base_GHCziIOPort_doubleReadException_closure" "-Wl,-u,base_ControlziExceptionziBase_nonTermination_closure" "-Wl,-u,base_ControlziExceptionziBase_nestedAtomically_closure" "-Wl,-u,base_GHCziEventziThread_blockedOnBadFD_closure" "-Wl,-u,base_GHCziConcziSync_runSparks_closure" "-Wl,-u,base_GHCziConcziIO_ensureIOManagerIsRunning_closure" "-Wl,-u,base_GHCziConcziIO_interruptIOManager_closure" "-Wl,-u,base_GHCziConcziIO_ioManagerCapabilitiesChanged_closure" "-Wl,-u,base_GHCziConcziSignal_runHandlersPtr_closure" "-Wl,-u,base_GHCziTopHandler_flushStdHandles_closure" "-Wl,-u,base_GHCziTopHandler_runMainIO_closure" "-Wl,-u,ghczmprim_GHCziTypes_Czh_con_info" "-Wl,-u,ghczmprim_GHCziTypes_Izh_con_info" "-Wl,-u,ghczmprim_GHCziTypes_Fzh_con_info" "-Wl,-u,ghczmprim_GHCziTypes_Dzh_con_info" "-Wl,-u,ghczmprim_GHCziTypes_Wzh_con_info" "-Wl,-u,base_GHCziPtr_Ptr_con_info" "-Wl,-u,base_GHCziPtr_FunPtr_con_info" "-Wl,-u,base_GHCziInt_I8zh_con_info" "-Wl,-u,base_GHCziInt_I16zh_con_info" "-Wl,-u,base_GHCziInt_I32zh_con_info" "-Wl,-u,base_GHCziInt_I64zh_con_info" "-Wl,-u,base_GHCziWord_W8zh_con_info" "-Wl,-u,base_GHCziWord_W16zh_con_info" "-Wl,-u,base_GHCziWord_W32zh_con_info" "-Wl,-u,base_GHCziWord_W64zh_con_info" "-Wl,-u,base_GHCziStable_StablePtr_con_info" "-Wl,-u,hs_atomic_add8" "-Wl,-u,hs_atomic_add16" "-Wl,-u,hs_atomic_add32" "-Wl,-u,hs_atomic_sub8" "-Wl,-u,hs_atomic_sub16" "-Wl,-u,hs_atomic_sub32" "-Wl,-u,hs_atomic_and8" "-Wl,-u,hs_atomic_and16" "-Wl,-u,hs_atomic_and32" "-Wl,-u,hs_atomic_nand8" "-Wl,-u,hs_atomic_nand16" "-Wl,-u,hs_atomic_nand32" "-Wl,-u,hs_atomic_or8" "-Wl,-u,hs_atomic_or16" "-Wl,-u,hs_atomic_or32" "-Wl,-u,hs_atomic_xor8" "-Wl,-u,hs_atomic_xor16" "-Wl,-u,hs_atomic_xor32" "-Wl,-u,hs_cmpxchg8" "-Wl,-u,hs_cmpxchg16" "-Wl,-u,hs_cmpxchg32" "-Wl,-u,hs_cmpxchg64" "-Wl,-u,hs_xchg8" "-Wl,-u,hs_xchg16" "-Wl,-u,hs_xchg32" "-Wl,-u,hs_xchg64" "-Wl,-u
,hs_atomicread8" "-Wl,-u,hs_atomicread16" "-Wl,-u,hs_atomicread32" "-Wl,-u,hs_atomicwrite8" "-Wl,-u,hs_atomicwrite16" "-Wl,-u,hs_atomicwrite32" "-Wl,-u,base_GHCziStackziCloneStack_StackSna
pshot_closure" "-Wl,-u,base_GHCziEventziWindows_processRemoteCompletion_closure" "-lHStestsZSlibrary-external-workspaceZSTestLib" "-lHSexternalZSZUmainz7eUnonZUmoduleZUdepsz7eUlibraryZUrep
oZSTestLib" "-lHSbase-4.17.2.1" "-lHSghc-bignum-1.3" "-lHSghc-prim-0.9.1" "-lHSrts-1.0.2_thr" "-lCffi-6" "-lwsock32" "-luser32" "-lshell32" "-lmingw32" "-lkernel32" "-ladvapi32" "-lmingwex
" "-lws2_32" "-lshlwapi" "-lole32" "-lrpcrt4" "-lntdll" "-luser32" "-lmingw32" "-lmingwex" "-lucrt" "-lm" "-lwsock32" "-lgdi32" "-lwinmm" "-ldbghelp" "-lpsapi" "-lpthread" "-lstdc++" "-Wl,
-verbose" "-pthread" "-Wno-unused-command-line-argument" -v
clang version 14.0.6
Target: x86_64-w64-windows-gnu
Thread model: posix
InstalledDir: C:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/bin
"C:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/bin/ld.lld" -m i386pep -Bdynamic -o bazel-out/x64_windows-fastbuild/bin/tests/library-external-workspace/bin-deps.exe C:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/x86_64-w64-mingw32/lib/crt2.o C:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/x86_64-w64-mingw32/lib/crtbegin.o -Lbazel-out/x64_windows-fastbuild/bin/tests/library-external-workspace/ -L/c/_bzl/33jjwvv5/execroot/_main/bazel-out/x64_windows-fastbuild/bin/external/_main~non_module_deps~library_repo -L../../external/rules_haskell~1.0~haskell_toolchains~bindist_rules_haskell_tests_1.0_windows_amd64/lib/x86_64-windows-ghc-9.4.8/base-4.17.2.1 -L../../external/rules_haskell~1.0~haskell_toolchains~bindist_rules_haskell_tests_1.0_windows_amd64/lib/x86_64-windows-ghc-9.4.8/ghc-bignum-1.3 -L../../external/rules_haskell~1.0~haskell_toolchains~bindist_rules_haskell_tests_1.0_windows_amd64/lib/x86_64-windows-ghc-9.4.8/ghc-prim-0.9.1 -L../../external/rules_haskell~1.0~haskell_toolchains~bindist_rules_haskell_tests_1.0_windows_amd64/lib/x86_64-windows-ghc-9.4.8/rts-1.0.2 -LC:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/x86_64-w64-mingw32/lib -LC:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/lib -LC:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/x86_64-w64-mingw32/sys-
root/mingw/lib -LC:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/lib/clang/14.0.6/lib/windows -lm --enable-auto-import -
-gc-sections "bazel-out\\x64_windows-fastbuild\\bin\\tests\\library-external-workspace\\_obj\\bin-deps\\Main.o" "C:\\msys64\\tmp\\ghc716_0\\ghc_5.o" "C:\\msys64\\tmp\\ghc716_0\\ghc_2.o" -u
hs_atomic_add64 -u hs_atomic_sub64 -u hs_atomic_and64 -u hs_atomic_nand64 -u hs_atomic_or64 -u hs_atomic_xor64 -u hs_atomicread64 -u hs_atomicwrite64 -u base_GHCziTopHandler_runIO_closure
-u base_GHCziTopHandler_runNonIO_closure -u ghczmprim_GHCziTuple_Z0T_closure -u ghczmprim_GHCziTypes_True_closure -u ghczmprim_GHCziTypes_False_closure -u base_GHCziPack_unpackCString_closure -u base_GHCziWeak_runFinalizzerBatch_closure -u base_GHCziIOziException_stackOverflow_closure -u base_GHCziIOziException_heapOverflow_closure -u base_GHCziIOziException_allocationLimitExceeded_closure -u base_GHCziIOziException_blockedIndefinitelyOnMVar_closure -u base_GHCziIOziException_blockedIndefinitelyOnSTM_closure -u base_GHCziIOziException_cannotCompactFunction_closure -u base_GHCziIOziException_cannotCompactPinned_closure -u base_GHCziIOziException_cannotCompactMutable_closure -u base_GHCziIOPort_doubleReadException_closure -u base_ControlziExceptionziBase_nonTermination_closure -u base_ControlziExceptionziBase_nestedAtomically_closure -u base_GHCziEventziThread_blockedOnBadFD_closure -u base_GHCziConcziSync_runSparks_closure -u base_GHCziConcziIO_ensureIOManagerIsRunning_closure -u base_GHCziConcziIO_interruptIOManager_closure -u base_GHCziConcziIO_ioManagerCapabilitiesChanged_closure -u base_GHCziConcziSignal_runHandlersPtr_closure -u base_GHCziTopHandler_flushStdHandles_closure -u base_GHCziTopHandler_runMainIO_closure -u ghczmprim_GHCziTypes_Czh_con_info -u ghczmprim_GHCziTypes_Izh_con_info -u ghczmprim_GHCziTypes_Fzh_con_info -u ghczmprim_GHCziTypes_Dzh_con_info -u ghczmprim_GHCziTypes_Wzh_con_info -u base_GHCziPtr_Ptr_con_info -u base_GHCziPtr_FunPtr_con_info -u base_GHCziInt_I8zh_con_info -u base_GHCziInt_I16zh_con_info -u base_GHCziInt_I32zh_con_info -u base_GHCziInt_I64zh_con_info -u base_GHCziWord_W8zh_con_info -u base_GHCziWord_W16zh_con_info -u base_GHCziWord_W32zh_con_info -u base_GHCziWord_W64zh_con_info -u base_GHCziStable_StablePtr_con_info -u hs_atomic_add8 -u hs_atomic_add16 -u hs_atomic_add32 -u hs_atomic_sub8 -u hs_atomic_sub16 -u hs_atomic_sub32 -u hs_atomic_and8 -u hs_atomic_and16 -u hs_atomic_and32 -u hs_atomic_nand8 -u hs_atomic_nand16 -u hs_atomic_nand32 -u hs_atomic_or8 -u hs_atomic_or16 -u hs_atomic_or32 -u hs_atomic_xor8 -u hs_atomic_xor16 -u hs_atomic_xor32 -u hs_cmpxchg8 -u hs_cmpxchg16 -u hs_cmpxchg32 -u hs_cmpxchg64 -u hs_xchg8 -u hs_xchg16 -u hs_xchg32 -u hs_xchg64 -u hs_atomicread8 -u hs_atomicread16 -u hs_atomicread32 -u hs_atomicwrite8 -u hs_atomicwrite16 -u hs_atomicwrite32 -u base_GHCziStackziCloneStack_StackSnapshot_closure -u base_GHCziEventziWindows_processRemoteCompletion_closure -lHStestsZSlibrary-external-workspaceZSTestLib -lHSexternalZSZUmainz7eUnonZUmoduleZUdepsz7eUlibraryZUrepoZSTestLib -lHSbase-4.17.2.1 -lHSghc-bignum-1.3 -lHSghc-prim-0.9.1
-lHSrts-1.0.2_thr -lCffi-6 -lwsock32 -luser32 -lshell32 -lmingw32 -lkernel32 -ladvapi32 -lmingwex -lws2_32 -lshlwapi -lole32 -lrpcrt4 -lntdll -luser32 -lmingw32 -lmingwex -lucrt -lm -lwso
ck32 -lgdi32 -lwinmm -ldbghelp -lpsapi -lpthread -lc++ -verbose -lmingw32 C:/_bzl/33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/ming
w/lib/clang/14.0.6/lib/windows/libclang_rt.builtins-x86_64.a -lunwind -lmoldname -lmingwex -lpthread -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 C:/_bzl/33jjwvv5/execroot/_main/exte
rnal/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/lib/clang/14.0.6/lib/windows/libclang_rt.builtins-x86_64.a -lunwind -lmoldname -lmingwex -lkernel32 C:/_bzl/
33jjwvv5/execroot/_main/external/rules_haskell~1.0~haskell_toolchains~rules_haskell_ghc_windows_amd64/mingw/x86_64-w64-mingw32/lib/crtend.o
lld: error: unable to find library -lHSexternalZSZUmainz7eUnonZUmoduleZUdepsz7eUlibraryZUrepoZSTestLib
clang: error: linker command failed with exit code 1 (use -v to see invocation)Notice the After manually normalizing the argument to |
The local cc toolchain created by the updated rules_cc module is different: It does not support creating shared libraries on macOS anymore (the toolchain is missing the `supports_dynamic_linker` feature). Instead, we have to use a cc_shared_library explictly. See bazelbuild/apple_support#326 (comment)
This is actually the oldest version that is compatible with Bazel 6 and also provides cc_shared_library at the expected place. But it also requires protobuf 27.0, which is broken on Windows using the MinGW compiler. We have to add a patch (which only applies to this module) in order to fix this. Also, bump versions of modules which are newer in the dependency graph.
On Windows, we try to work around the path limit by normalizing paths. This stopped working because $PWD was set to a Windows-style path (C:/...) instead of a MSYS-style one. So, relativizing the shortened path did not work and the MSYS style path was handed to the linker which did not understand it and could not find the libraries in there.
This repository is already instantiated by calling `rules_haskell_dependencies()`.
741240e to
103b5b0
Compare
Merge Queue Status
This pull request spent 20 seconds in the queue, including 3 seconds running CI. Required conditions to merge
|
An attempt to fix #2328, which removes the use of the native rules_cc, rules_java and protobuf rules.
bazel build [--enable_bzlmod] //...andbazel build [--enable_bzlmod] //...all build and pass on my machine so hopefully this has actually worked...Updates deps:
0.0.9->0.0.17(latest 0.0.X version)6.1.0->6.5.2(latest 6.X.Y version)21.7->28.3(matchinghaskell/repositories.bzl)Each commit should probably be reviewed separately, most changes were from
buildifier, with the final commit implementing some optional changes suggested by the LSP including documenting some undocumented arguments as far as I could understand them.