You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I originally filed this as an issue at #22483, but it might be better as a discussion until some rough consensus can be reached.
The basic problem I want to address is the use of complex language-specific BUILD file generators in rulesets. Currently these tools need to execute on the host, which means a native binary for the host platform must either (1) exist as a pre-compiled download, or (2) be built as part of repository rule execution. Both approaches have significant drawbacks:
Pre-compiled downloads for ruleset tools only exist for the most popular platforms. For example the rules_rust ruleset provides the cargo-bazel tool compiled for Linux, macOS, and Windows on x86-64 and aarch64 -- while this does cover most users, it means that I can't easily test the Bazel builds for my Rust projects on FreeBSD or RISC-V.
The set of host platforms supported by Bazel is large. It would be unreasonable to expect ruleset authors to provide pre-compiled binaries for every platform that Bazel can be coaxed into run on.
Logic within a repository rule to discover the host platform is fussy and fragile, and usually fails to distinguish platform variants (such as -gnu vs -musl on Linux).
Compiling the tool just-in-time as part of the repository rule is slow, runs without sandboxing, and is un-cacheable. Lack of sandboxing can cause hermeticity errors, such as builds that depend on dotfiles in the user's homedir, or accidental mutation of lock files in the source tree.
At a previous employer, modifying rules_go to use precompiled gazelle binaries provided a significant improvement in CI performance but made the process of updating rules_go much more difficult.
Over the past few years, support for WASM as both a compilation target and embedded interpreter has become widespread. Many languages (including Rust and Go) can be compiled to standalone WASM binaries, and the Chicory runtime implements a WASM interpreter in Java with no native code dependencies.
If it were possible to execute a WASM binary within a repository rule, then ruleset authors could provide a single pre-compiled .wasm file and Bazel could execute it on every platform with identical behavior and full sandboxing. The API does not need to be complex -- since Bazel can handle the file I/O using existing repository_ctx functions, the WASM blob only needs to provide pure functions and the API can be written in terms of plain byte buffers.
I've put together a proof-of-concept to verify that this does in fact work as expected, using WASM modules written in Rust and Go (modified from existing ruleset tools). The resulting repository rules work without modification on all the host platforms I have tried, and they no longer need to execute uname or poke around in /lib to detect host properties.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
I originally filed this as an issue at #22483, but it might be better as a discussion until some rough consensus can be reached.
The basic problem I want to address is the use of complex language-specific BUILD file generators in rulesets. Currently these tools need to execute on the host, which means a native binary for the host platform must either (1) exist as a pre-compiled download, or (2) be built as part of repository rule execution. Both approaches have significant drawbacks:
rules_rust
ruleset provides thecargo-bazel
tool compiled for Linux, macOS, and Windows on x86-64 and aarch64 -- while this does cover most users, it means that I can't easily test the Bazel builds for my Rust projects on FreeBSD or RISC-V.-gnu
vs-musl
on Linux).gazelle
binaries provided a significant improvement in CI performance but made the process of updating rules_go much more difficult.Over the past few years, support for WASM as both a compilation target and embedded interpreter has become widespread. Many languages (including Rust and Go) can be compiled to standalone WASM binaries, and the Chicory runtime implements a WASM interpreter in Java with no native code dependencies.
If it were possible to execute a WASM binary within a repository rule, then ruleset authors could provide a single pre-compiled
.wasm
file and Bazel could execute it on every platform with identical behavior and full sandboxing. The API does not need to be complex -- since Bazel can handle the file I/O using existingrepository_ctx
functions, the WASM blob only needs to provide pure functions and the API can be written in terms of plain byte buffers.I've put together a proof-of-concept to verify that this does in fact work as expected, using WASM modules written in Rust and Go (modified from existing ruleset tools). The resulting repository rules work without modification on all the host platforms I have tried, and they no longer need to execute
uname
or poke around in/lib
to detect host properties.Beta Was this translation helpful? Give feedback.
All reactions