this equates to -fno-strict-aliasing
this is not a particularly new flag and is supported by all our
gcc and clang versions.
add to pkgsExtraHardening's defaultHardeningFlags.
Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:
nix-build ci -A fmt.check
This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).
This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).
Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).
If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
Darwin includes its SDK in `extraBuildInputs`, which isn’t needed in
`stdenvNoCC` because it depends on a compiler. Including it greatly
increases the size of the `stdenvNoCC` closure (by ~1 GiB).
This wasn’t an issue in the past when darwin included CoreFoundation
because CoreFoundation on its own is very small. Now that it includes
the whole SDK and links some bintools, it pulls in 300~400 MiB for the
SDK plus another ~445 MiB for LLVM.
Fixes#371465
When using pkgsCross with a system that ends up the same as the
localSystem, then modifications for package sets like pksgMusl need to
be done for **both** localSystem and crossSystem. Consider the following
on x86_64-linux:
pkgsCross.gnu64.pkgsMusl
Before this change, this would result in a musl buildPlatform, but a gnu
hostPlatform. This breaks the promise of "stacking" package sets on top
of each other.
After this change, it results in a musl buildPlatform and a musl
hostPlatform. This works better.
One could expect this to result in the same as pkgsCross.musl64, i.e. a
gnu buildPlatform and a musl hostPlatform, however I couldn't get this
to work without increasing memory usage for ci/eval by many, many GB.
This is caused by usage of pkgsi686Linux inside the main package set,
which follows the same hybrid pattern.
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.
There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:
- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.
- Once pkgsCross is used any subsequent package sets should affect the
**host platform** and not the build platform. Examples:
- pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
- pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64
- Modifications from an earlier layer should not be lost, unless
explicitly overwritten. Examples:
- pkgsStatic.pkgsMusl should still be static.
- pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.
Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
same flags over and over again.
Supersedes #136549Resolves#114510Resolves#212494Resolves#281596
No change, just move appendOverlays and extend to the bottom, since they
will be changed much less often. This makes it easier to compare the
other package sets side-by-side.
pkgsLLVM introduced an unwanted target offset, as can be seen with the
following commands.
Before this commit:
```
$ nix eval --system x86_64-linux -f . pkgsCross.aarch64-multiplatform.buildPackages.stdenv.hostPlatform.config
"x86_64-unknown-linux-gnu"
$ nix eval --system x86_64-linux -f . pkgsCross.aarch64-multiplatform.buildPackages.pkgsLLVM.stdenv.hostPlatform.config
"aarch64-unknown-linux-gnu"
Explaination: These two should be the same, since the only difference
between them is the addition of "pkgsLLVM". The buildPackages of the
pkgsCross set should have a hostPlatform which is the native platform.
```
After this commit:
```
$ nix eval --system x86_64-linux -f . pkgsCross.aarch64-multiplatform.buildPackages.stdenv.hostPlatform.config
"x86_64-unknown-linux-gnu"
$ nix eval --system x86_64-linux -f . pkgsCross.aarch64-multiplatform.buildPackages.pkgsLLVM.stdenv.hostPlatform.config
"x86_64-unknown-linux-gnu"
```
When elaborating a system with both "config" and "system" arguments
given, they might not match the parsed results. Example:
elaborate {
config = "i686-unknown-linux-gnu";
system = "x86_64-linux";
}
This would result in a parsed system for i686, because the config
argument is preferred. But since "// args //" comes after system has
been inferred from parsed, it is overwritten again. This results in
config and parsed all pointing to i686, while system still tells the
story of x86_64.
Inconsistent arguments can also be given when passing "parsed" directly.
This happened in stage.nix for the various package sets.
The solution is simple: One of the three arguments needs to be treated
as the ultimate source of truth. "system" can already be losslessly
extracted from "parsed". However, "config" currently can not, for
example for various -mingw32 cases. Thus everything must be derived
from "config".
To do so, "system" and "parsed" arguments are made non-overrideable for
systems.elaborate. This means, that "system" will be used to parse when
"config" is not given - and "parsed" will be ignored entirely.
The systemToAttrs helper is exposed on lib.systems, because it's useful
to deal with top-level localSystem / crossSystem arguments elsewhere.
In preparation for the deprecation of `stdenv.isX`.
These shorthands are not conducive to cross-compilation because they
hide the platforms.
Darwin might get cross-compilation for which the continued usage of `stdenv.isDarwin` will get in the way
One example of why this is bad and especially affects compiler packages
https://www.github.com/NixOS/nixpkgs/pull/343059
There are too many files to go through manually but a treewide should
get users thinking when they see a `hostPlatform.isX` in a place where it
doesn't make sense.
```
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv.is" "stdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv'.is" "stdenv'.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "clangStdenv.is" "clangStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "gccStdenv.is" "gccStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenvNoCC.is" "stdenvNoCC.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "inherit (stdenv) is" "inherit (stdenv.hostPlatform) is"
fd --type f "\.nix" | xargs sd --fixed-strings "buildStdenv.is" "buildStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "effectiveStdenv.is" "effectiveStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "originalStdenv.is" "originalStdenv.hostPlatform.is"
```
For a long time, we've had `crossLibcStdenv`, `*Cross` libc attributes,
and `*bsdCross` pre-libc package sets. This was always bad because
having "cross" things is "not declarative": the naming doesn't reflect
what packages *need* but rather how we *provide* something. This is
ugly, and creates needless friction between cross and native building.
Now, almost all of these `*Cross` attributes are gone: just these are
kept:
- Glibc's and Musl's are kept, because those packages are widely used
and I didn't want to risk changing the native builds of those at this
time.
- generic `libcCross`, `theadsCross`, and friends, because these relate
to the convolulted GCC bootstrap which still needs to be redone.
The BSD and obscure Linux or freestnanding libcs have conversely all
been made to use a new `stdenvNoLibc`, which is like the old
`crossLibcStdenv` except:
1. It usable for native and cross alike
2. It named according to what it *is* ("a standard environment without
libc but with a C compiler"), rather than some non-compositional
jargon ("the stdenv used for building libc when cross compiling",
yuck).
I should have done this change long ago, but I was stymied because of
"infinite recursions". The problem was that in too many cases we are
overriding `stdenv` to *remove* things we don't need, and this risks
cyles since those more minimal stdenvs are used to build things in the
more maximal stdenvs.
The solution is to pass `stage.nix` `stdenvNoCC`, so we can override to
*build up* rather than *tear down*. For now, the full `stdenv` is also
passed, so I don't need to change the native bootstraps, but I can see
this changing as we make things more uniform and clean those up.
(adapted from commit 51f1ecaa59)
(adapted from commit 1743662e55)
Most Linux distributions are enabling this these days and it does
protect against real world vulnerabilities as demonstrated by
CVE-2018-16864 and CVE-2018-16865.
Fix#53753.
Information on llvm version support gleaned from
6609892a2d68e07da3e5092507a730
Information on gcc version support a lot harder to gather,
but both 32bit and 64bit arm do appear to be supported
based on the test suite.
To build the security wrappers[1] the pkgsStatic stdenv is used, so the
binaries are static. However, the hostPlatform may have gcc attributes
that are *required* to build binaries so they can run on the host
platform. In particular, this is the case when using gcc.arch, which
ends up injecting -march=... in the gcc wrapper. Those attributes are
not contained in hostPlatform.parsed.
This change sets the same gcc attributes found in the hostPlatform for
the pkgsStatic cross system, so it can build binaries with the same gcc
flags.
[1]: nixos/modules/security/wrappers/default.nix
* Extend libc
Include non-libc core libraries in the libc package. Many of these
mirror libraries present in glibc on linux, such as libgcc, libraries
used for iconv, and libraries used for reading kernel info (libkvm,
libprocstat, libmemstat).
Without this many packages outside the freebsd tree would need to be
modified to include standard dependencies which would already be on
the system for other packages.
* Mark FreeBSD as using LLVM
* Update default LLVM version FreeBSD
* Use patch monolith
The patchesRoot system combined with the fact that each derivation
will Request specific names of patches makes it very annoying to use
other FreeBSD source trees with nixpkgs. This new system allows
providing one Or more entire trees of patches whose contents will be
dynamically Parsed and only the relevant patches will be applied for
any one Derivation.
With this commit, the following knobs are available for specifying the
FreeBSD source:
- overriding `freebsd.versionInfo`, for picking another official
supported FreeBSD release.
- overriding `freebsd.source` for specifying a specific unpatched
FreeBSD source tree.
- overriding `freebsd.patches`, for specifying the patches to apply.
Co-Authored-by: Audrey Dutcher <audrey@rhelmot.io>
Co-Authored-by: John Ericson <John.Ericson@Obsidian.Systems>
Replace writeReferencesToFile with writeClosure.
Make writeClosure accept a list of paths instead of a path.
Re-implement with JSON-based exportReferencesGraph interface provided by
__structuredAttrs = true.
Reword the documentation.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Someone Serge <sergei.kozlukov@aalto.fi>
Closes#193336Closes#261694
Related to #108984
The goal here was to get the following flake to build and run on
`aarch64-darwin`:
```nix
{ inputs.nixpkgs.url = <this branch>;
outputs = { nixpkgs, ... }: {
checks.aarch64-darwin.default =
nixpkgs.legacyPackages.aarch64-darwin.nixosTest {
name = "test";
nodes.machine = { };
testScript = "";
};
};
}
```
… and after this change it does. There's no longer a need for the
user to set `nodes.*.nixpkgs.pkgs` or
`nodes.*.virtualisation.host.pkgs` as the correct values are inferred
from the host system.
this equates to -ftrivial-auto-var-init=pattern
clang has removed support for -ftrivial-auto-var-init=zero and
are unlikely to re-add it, so use -ftrivial-auto-var-init=pattern
on both compilers if only to make behaviour more consistent
between the two.
add to pkgsExtraHardening's defaultHardeningFlags.
This introduces the `pkgs/by-name` directory as proposed by RFC 140.
Included are:
- The implementation to add packages defined in that directory to the
top-level package scope
- Contributer documentation on how to add packages to it
- A GitHub Actions workflow to check the structure of it on all PRs
