This fixes a long standing issue where rust hooks behave differently when used inside buildRustPackage vs inside mkDerivation, which lead to surprising behavior, like for example the package being built for the wrong paltform or the linker not being found especially in cross compilation scenarios.
The reason for this inconsitency was, that buildRustPackage consumed the hooks in a non-spliced form, via [this inherit statement](4506ece030/pkgs/development/compilers/rust/make-rust-platform.nix (L60)), and therefore the usual platform shift on the hooks introduced by putting them in `nativeBuildInputs` was not applied here.
Thoug whenever the hook was used inside other builders like `mkDerivation` the platform shift did apply correctly as the hook was consumed via the spliced package set, introducing the inconsitecy.
Because of the wrong (non-spliced) use in buildRustPackage, most rust hooks have been designed with the wrong build/host/target shift in mind which is fixed by this change.
Due to the inconsitent behavior between different builders, workarounds like `rust.envVars`, which were previously introduced, likely become obsolete by this change.
This likely fixes a bunch of cross compilation issues for rust packages that are not based on `buildRustPackage` but instead consume the hooks directly.
Done:
- ensure that `buildRustPackage` consumes spliced hooks by using makeScopeWithSplicing' in make-rust-platform.nix.
- refactor hooks to make them refer to correct build/host/target packages.
- remove `rust.envVars` workaround from all rust hooks
- implement tests for most rust hooks in /pkgs/test/rut-hooks
The newly added tests can be executed for native as well as cross compilation via:
```
nix-build -A tests.rust-hooks -A pkgsCross.riscv64.tests.rust-hooks
```
The sysroot derivation used by buildRustPackage hasn't built since
c263e0b33b ("cargo,clippy,rustc,rustfmt: 1.81.0 -> 1.82.0"). That
means that it's never built on 24.11, so there can't be anybody
relying on it working on any supported version Nixpkgs. In the
meantime, it's causing maintenance friction — I've spent a lot of time
trying to figure out how to build it as part of the transition away
from fetchCargoTarball, before realising that it doesn't work anyway
and must be unused. It can come back in the future if somebody wants
to maintain it.
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.
Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.
A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.
This commit was automatically created and can be verified using
nix-build https://github.com/infinisil/treewide-nixpkgs-reformat-script/archive/a08b3a4d199c6124ac5b36a889d9099b4383463f.tar.gz \
--argstr baseRev 78e9caf153
result/bin/apply-formatting $NIXPKGS_PATH
Previously, it wasn't possible to access the list of platforms we can
build Rust programs for outside of buildRustPackage. This was a
problem for packages that have optional Rust components, like
gstreamer or Meson, as there was no way to only build the Rust parts
for supported platforms. Now it's possible to get that information
from rustc's passthru.
We should constrain the set of supported platforms to the platforms
supported by the compiler. Otherwise we run into the unfortunate
situation where setting meta.platforms in a Rust package to
platforms.unix or platforms.linux will reintroduce CPU architectures
unsupported by the compiler.
Currently we cannot debug rust binaries on darwin (via lldb).
The debug symbols seem to be provided by default in a number of files
in `target/debug/deps/*.rcgu.o`. As far as I can tell these have
hardcoded paths referring to the ephemeral build directory. However,
`split-debuginfo=packed` conveniently produces a `.dSYM` file that can
be copied to `$out/bin/` and immediately provide debugging information.
Fixes https://github.com/NixOS/nixpkgs/issues/262131
We need this stuff to be available in lib so make-derivation.nix can
access it to construct the Meson cross file.
This has a couple of other advantages:
- It makes Rust less special. Now figuring out what Rust calls a
platform is the same as figuring out what Linux or QEMU call it.
- We can unify the schema used to define Rust targets, and the schema
used to access those values later. Just like you can set "config"
or "system" in a platform definition, and then access those same
keys on the elaborated platform, you can now set "rustcTarget" in
your crossSystem, and then access "stdenv.hostPlatform.rustcTarget"
in your code.
"rustcTarget", "rustcTargetSpec", "cargoShortTarget", and
"cargoEnvVarTarget" have the "rustc" and "cargo" prefixes because
these are not exposed to code by the compiler, and are not
standardized. The arch/os/etc. variables are all named to match the
forms in the Rust target spec JSON.
The new rust.target-family only takes a list, since we don't need to
worry about backwards compatibility when that name is used.
The old APIs are all still functional with no warning for now, so that
it's possible for external code to use a single API on both 23.05 and
23.11. We can introduce the warnings once 23.05 is EOL, and make them
hard errors when 23.11 is EOL.
Rust is not yet able to target the n32 ABI on mips64.
Let's add `isMips64n32` to the `meta.badPlatforms` of all
derivations created by buildRustPackage.
I use this to automatically detect which packages on my system can
be built for n32 (almost all of them) and build those using n32, and
the few packages (mainly those that depend on boost or rust) that
can't for n64.
This upgrade unfortunately removes MIPS support, as it has been
dropped to Tier 3[1] and so bootstrap tarballs are no longer provided.
It looks like it was dropped due to multiple codegen bugs, and lack of
maintenance, so bringing it back would probably involve engaging with
Rust/LLVM upstream on those.
[1]: https://github.com/rust-lang/compiler-team/issues/648
> If using a target spec JSON file, the <triple> value is the filename
> stem. For example --target foo/bar.json would match [target.bar].
- https://doc.rust-lang.org/cargo/reference/config.html#target
I've also exposed toRustTargetSpecShort as a public function, because
it's useful to be able to know what the target subdirectory will be.
armv6l-linux was incorrectly added to the list of platforms without host
tools in #227987. arm-unknown-linux-gnueabihf is present in the list of
Tier 2 targets with host tools, and this target corresponds to our
armv6l-linux platform.
rustc supports way more platforms than Linux and Darwin. We might not
be able to build it for every platform at the moment, but that's what
meta.broken is for.
There are other platforms that rustc can produce binaries for, but
can't run on itself, so those are listed in the defaults for
buildRustPackage.
Sometimes it's more ergonomic to set up the build environment in
hooks, to add to the default behaviour rather than replacing it. It's
very surprising that the fetcher works fine with a custom unpackPhase,
but not with custom preUnpack or postUnpack.
Packages that use preUnpack or postUnpack and Cargo FODs seem to be
very rare. I searched Nixpkgs for files containing one of
"cargoHash", "cargoDeps", and "cargoSha256", and one of "preUnpack" or
"postUnpack", and only found two such packages:
python3.pkgs.tokenizers and rustdesk. Neither of their Cargo FOD
hashes are affected by this change. So if that's any indication,
we're unlikely to be breaking many out-of-tree hashes with these
changes either.
`cargoDeps` is already passed as `mkDerivation` arguments, and should
not be `passthru`ed again. This fixes the mismatch of `drv.cargoDeps`
and the actual dependency when the original derivation is overriden.
This reverts commit b6fc00b8f4.
Rust 1.66.0 contains a fix for libiconv being linked unconditionally on macOS, but this only applies to packages that don't depend on older versions of `libc`.
For now, let's go back to including libiconv in `buildInputs` by default for packages that use `buildRustPackage`. As packages bump their `libc` versions, we can eventually stop including it by default, and manually add it where needed.
