movefasta/nixpkgs
1
0
Fork 0
mirror of https://github.com/NixOS/nixpkgs.git synced 2025-06-10 03:23:29 +03:00
Code Issues Projects Releases Packages Wiki Activity

Merge remote-tracking branch 'upstream/staging-next' into staging

Browse source 
Conflicts:
	pkgs/development/python-modules/apsw/default.nix
	pkgs/development/python-modules/deal/default.nix
	pkgs/development/python-modules/kubernetes/default.nix
This commit is contained in:
annalee 2024-03-28 13:02:34 +00:00
commit 675fb1648e
No known key found for this signature in database
302 changed files with 13153 additions and 3193 deletions
Download patch file Download diff file Expand all files Collapse all files

116
doc/languages-frameworks/python.section.md
View file

@ -254,17 +254,19 @@ The next example shows a non trivial overriding of the `blas` implementation to
be used through out all of the Python package set:
```nix
python3MyBlas = pkgs.python3.override {
packageOverrides = self: super: {
# We need toPythonModule for the package set to evaluate this
blas = super.toPythonModule(super.pkgs.blas.override {
blasProvider = super.pkgs.mkl;
});
lapack = super.toPythonModule(super.pkgs.lapack.override {
lapackProvider = super.pkgs.mkl;
});
{
python3MyBlas = pkgs.python3.override {
packageOverrides = self: super: {
# We need toPythonModule for the package set to evaluate this
blas = super.toPythonModule(super.pkgs.blas.override {
blasProvider = super.pkgs.mkl;
});
lapack = super.toPythonModule(super.pkgs.lapack.override {
lapackProvider = super.pkgs.mkl;
});
};
};
};
}
```
This is particularly useful for numpy and scipy users who want to gain speed with other blas implementations.
@ -322,7 +324,9 @@ python3Packages.buildPythonApplication rec {
This is then added to `all-packages.nix` just as any other application would be.
```nix
luigi = callPackage ../applications/networking/cluster/luigi { };
{
luigi = callPackage ../applications/networking/cluster/luigi { };
}
```
Since the package is an application, a consumer doesn't need to care about
@ -342,7 +346,9 @@ the attribute in `python-packages.nix`, and the `toPythonApplication` shall be
applied to the reference:
```nix
youtube-dl = with python3Packages; toPythonApplication youtube-dl;
{
youtube-dl = with python3Packages; toPythonApplication youtube-dl;
}
```
#### `toPythonModule` function {#topythonmodule-function}
@ -354,10 +360,12 @@ bindings should be made available from `python-packages.nix`. The
modifications.
```nix
opencv = toPythonModule (pkgs.opencv.override {
enablePython = true;
pythonPackages = self;
});
{
opencv = toPythonModule (pkgs.opencv.override {
enablePython = true;
pythonPackages = self;
});
}
```
Do pay attention to passing in the right Python version!
@ -1162,7 +1170,8 @@ a good indication that the package is not in a valid state.
Pytest is the most common test runner for python repositories. A trivial
test run would be:
```
```nix
{
nativeCheckInputs = [ pytest ];
checkPhase = ''
runHook preCheck
@ -1171,6 +1180,7 @@ test run would be:
runHook postCheck
'';
}
```
However, many repositories' test suites do not translate well to nix's build
@ -1178,7 +1188,8 @@ sandbox, and will generally need many tests to be disabled.
To filter tests using pytest, one can do the following:
```
```nix
{
nativeCheckInputs = [ pytest ];
# avoid tests which need additional data or touch network
checkPhase = ''
@ -1188,6 +1199,7 @@ To filter tests using pytest, one can do the following:
runHook postCheck
'';
}
```
`--ignore` will tell pytest to ignore that file or directory from being
@ -1213,7 +1225,8 @@ when a package may need many items disabled to run the test suite.
Using the example above, the analogous `pytestCheckHook` usage would be:
```
```nix
{
nativeCheckInputs = [
pytestCheckHook
];
@ -1233,12 +1246,14 @@ Using the example above, the analogous `pytestCheckHook` usage would be:
disabledTestPaths = [
"tests/test_failing.py"
];
}
```
This is especially useful when tests need to be conditionally disabled,
for example:
```
```nix
{
disabledTests = [
# touches network
"download"
@ -1250,6 +1265,7 @@ for example:
# can fail when building with other packages
"socket"
];
}
```
Trying to concatenate the related strings to disable tests in a regular
@ -1263,20 +1279,24 @@ all packages have test suites that can be run easily, and some have none at all.
To help ensure the package still works, [`pythonImportsCheck`](#using-pythonimportscheck) can attempt to import
the listed modules.
```
```nix
{
pythonImportsCheck = [
"requests"
"urllib"
];
}
```
roughly translates to:
```
```nix
{
postCheck = ''
PYTHONPATH=$out/${python.sitePackages}:$PYTHONPATH
python -c "import requests; import urllib"
'';
}
```
However, this is done in its own phase, and not dependent on whether [`doCheck = true;`](#var-stdenv-doCheck).
@ -1307,7 +1327,8 @@ pkg3>=1.0,<=2.0
we can do:
```
```nix
{
nativeBuildInputs = [
pythonRelaxDepsHook
];
@ -1318,6 +1339,7 @@ we can do:
pythonRemoveDeps = [
"pkg2"
];
}
```
which would result in the following `requirements.txt` file:
@ -1330,9 +1352,11 @@ pkg3
Another option is to pass `true`, that will relax/remove all dependencies, for
example:
```
```nix
{
nativeBuildInputs = [ pythonRelaxDepsHook ];
pythonRelaxDeps = true;
}
```
which would result in the following `requirements.txt` file:
@ -1357,7 +1381,8 @@ work with any of the [existing hooks](#setup-hooks).
`unittestCheckHook` is a hook which will substitute the setuptools `test` command for a [`checkPhase`](#ssec-check-phase) which runs `python -m unittest discover`:
```
```nix
{
nativeCheckInputs = [
unittestCheckHook
];
@ -1365,6 +1390,7 @@ work with any of the [existing hooks](#setup-hooks).
unittestFlagsArray = [
"-s" "tests" "-v"
];
}
```
#### Using sphinxHook {#using-sphinxhook}
@ -1374,7 +1400,8 @@ using the popular Sphinx documentation generator.
It is setup to automatically find common documentation source paths and
render them using the default `html` style.
```
```nix
{
outputs = [
"out"
"doc"
@ -1383,13 +1410,15 @@ render them using the default `html` style.
nativeBuildInputs = [
sphinxHook
];
}
```
The hook will automatically build and install the artifact into the
`doc` output, if it exists. It also provides an automatic diversion
for the artifacts of the `man` builder into the `man` target.
```
```nix
{
outputs = [
"out"
"doc"
@ -1401,14 +1430,17 @@ for the artifacts of the `man` builder into the `man` target.
"singlehtml"
"man"
];
}
```
Overwrite `sphinxRoot` when the hook is unable to find your
documentation source root.
```
```nix
{
# Configure sphinxRoot for uncommon paths
sphinxRoot = "weird/docs/path";
}
```
The hook is also available to packages outside the python ecosystem by
@ -1753,6 +1785,7 @@ folder and not downloaded again.
If you need to change a package's attribute(s) from `configuration.nix` you could do:
```nix
{
nixpkgs.config.packageOverrides = super: {
python3 = super.python3.override {
packageOverrides = python-self: python-super: {
@ -1767,6 +1800,7 @@ If you need to change a package's attribute(s) from `configuration.nix` you coul
};
};
};
}
```
`python3Packages.twisted` is now globally overridden.
@ -1779,11 +1813,13 @@ To modify only a Python package set instead of a whole Python derivation, use
this snippet:
```nix
{
myPythonPackages = python3Packages.override {
overrides = self: super: {
twisted = ...;
twisted = <...>;
};
}
};
}
```
### How to override a Python package using overlays? {#how-to-override-a-python-package-using-overlays}
@ -1819,7 +1855,7 @@ final: prev: {
(
python-final: python-prev: {
foo = python-prev.foo.overridePythonAttrs (oldAttrs: {
...
# ...
});
}
)
@ -1846,7 +1882,7 @@ The Python interpreters are by default not built with optimizations enabled, bec
the builds are in that case not reproducible. To enable optimizations, override the
interpreter of interest, e.g using
```
```nix
let
pkgs = import ./. {};
mypython = pkgs.python3.override {
@ -1864,17 +1900,21 @@ Some packages define optional dependencies for additional features. With
`extras-require`, while PEP 621 calls these `optional-dependencies`.
```nix
optional-dependencies = {
complete = [ distributed ];
};
{
optional-dependencies = {
complete = [ distributed ];
};
}
```
and letting the package requiring the extra add the list to its dependencies
```nix
dependencies = [
...
] ++ dask.optional-dependencies.complete;
{
dependencies = [
# ...
] ++ dask.optional-dependencies.complete;
}
```
This method is using `passthru`, meaning that changing `optional-dependencies` of a package won't cause it to rebuild.