Add NixOS support

[jaroeichler]

Description

Use nix-ld to run Bazel.

Fixes #1229

Type of change

Please delete options that aren't relevant.

• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How Has This Been Tested?

Used a pretty minimal system configuration and Bazel test ... passes 51 of 52 tests, integration_tests/running_actions_manager_test fails.

Checklist

• Updated documentation if needed
• Tests added/amended
• bazel test //... passes locally
• PR is contained in a single commit, using git amend see some docs

---

This change is 

[CLAassistant]

All committers have signed the CLA.

[jaroeichler]

services.envfs.enable is only needed for some scripts that need /bin/bash. Is there a better way?

@SpamDoodler can you please check if this works on your system? I've made a few changes to what we've tried a few days ago. In particular, it works without NIX_CC_USE_RESPONSE_FILE=0 on a pretty minimal system configuration.

[aaronmondal]

Reviewed 5 of 5 files at r1, all commit messages.
Reviewable status: 0 of 2 LGTMs obtained, and all files reviewed, and pending CI: Remote / large-ubuntu-22.04, and 3 discussions need to be resolved

---

flake.nix line 80 at r1 (raw file):

          "/run/current-system/sw/bin"
          "${binutils.bintools}/bin"
          "${coreutils}/bin"

nit: For kicks, consider using uutils-coreutils-noprefix as pure rust implementation of the coreutils 💃

---

flake.nix line 83 at r1 (raw file):

          "${customClang}/bin"
          "${git}/bin"
          "${python3Full}/bin"

Interesting. This looks like it exposes several nonhermeticities in the devShell. Most of this should probably be in the main devShell as well.

Regarding the python dependency, why do we need the Full variant? What's the closure size difference here?

---

flake.nix line 490 at r1 (raw file):

                ln -fs ${nixosBazelrc} nixos.bazelrc
                export CC=customClang
              fi

This should be a flake module similar to the config.pre-commit.installationScript and config.local-remote-execution.installationScript. This way users can use it as a generic "NixOS Bazel compatibility" module and are not bound to our choice of clang/binutils/python and may import it in arbitrary Bazel+Nix projects.

See:

• nativelink/flake.nix

  Line 44 in 1b38a64

  ./local-remote-execution/flake-module.nix

• (unused in our flake, but makes it importable)

  nativelink/flake.nix

  Line 479 in 1b38a64

  flakeModule = ./local-remote-execution/flake-module.nix;

• https://github.com/TraceMachina/nativelink/blob/main/local-remote-execution/flake-module.nix
• https://github.com/TraceMachina/nativelink/blob/main/local-remote-execution/modules/lre.nix

Also note that the symlink here is too naive and should be done similar to what we have in

nativelink/local-remote-execution/modules/lre.nix

Line 135 in 1b38a64

config = {

[jaroeichler]

Reviewable status: 0 of 2 LGTMs obtained, and 4 of 7 files reviewed, and pending CI: Bazel Dev / ubuntu-22.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Installation / macos-13, Installation / macos-14, Installation / ubuntu-22.04, Publish image, Publish nativelink-worker-init, Publish nativelink-worker-lre-cc, docker-compose-compiles-nativelink (22.04), macos-13, ubuntu-20.04 / stable, ubuntu-22.04 / stable, windows-2022 / stable, and 3 discussions need to be resolved

---

flake.nix line 80 at r1 (raw file):

Previously, aaronmondal (Aaron Siddhartha Mondal) wrote…

nit: For kicks, consider using uutils-coreutils-noprefix as pure rust implementation of the coreutils 💃

Let's go with that.

---

flake.nix line 83 at r1 (raw file):

Previously, aaronmondal (Aaron Siddhartha Mondal) wrote…

Interesting. This looks like it exposes several nonhermeticities in the devShell. Most of this should probably be in the main devShell as well.

Regarding the python dependency, why do we need the Full variant? What's the closure size difference here?

There's around 20M closure size difference. The only reason I went for that is that it's used somewhere else in the flake for some image. I've replaced it with python3

---

flake.nix line 490 at r1 (raw file):

Previously, aaronmondal (Aaron Siddhartha Mondal) wrote…

This should be a flake module similar to the config.pre-commit.installationScript and config.local-remote-execution.installationScript. This way users can use it as a generic "NixOS Bazel compatibility" module and are not bound to our choice of clang/binutils/python and may import it in arbitrary Bazel+Nix projects.

See:

• nativelink/flake.nix

  Line 44 in 1b38a64

  ./local-remote-execution/flake-module.nix

• (unused in our flake, but makes it importable)

  nativelink/flake.nix

  Line 479 in 1b38a64

  flakeModule = ./local-remote-execution/flake-module.nix;

• https://github.com/TraceMachina/nativelink/blob/main/local-remote-execution/flake-module.nix
• https://github.com/TraceMachina/nativelink/blob/main/local-remote-execution/modules/lre.nix

Also note that the symlink here is too naive and should be done similar to what we have in

nativelink/local-remote-execution/modules/lre.nix

Line 135 in 1b38a64

config = {

I've pretty much copied your setup. An empty default of paths in nixos.settings.path is somehow intentional since I don't want to pass customClang to the module and just replacing it with pkgs.clang leads to iostream not found. Side note, lib.mdDoc is deprecated.

[aaronmondal]

Nice work!

Reviewed 3 of 3 files at r2, all commit messages.
Reviewable status: 1 of 2 LGTMs obtained, and all files reviewed

[aaronmondal]

+@adam-singer

Reviewable status: 1 of 2 LGTMs obtained, and all files reviewed (waiting on @adam-singer)