Since v12, PostgreSQL doesn't support recovery.conf anymore and fails to
start up when this option is set:
FATAL: using recovery command file "recovery.conf" is not supported
This is documented at:
https://www.postgresql.org/docs/current/recovery-config.html
Closes#386804
The issue with coercing to `types.str` is that it's not mergeable, so
any declarations will result in an eval error like this:
error: The option `interactive.nodes.tmp.services.postgresql.settings.shared_preload_libraries' has conflicting definition values:
- In `/home/ma27/Projects/nixpkgs-hack/tmp.nix@node-tmp': "foo"
- In `/home/ma27/Projects/nixpkgs-hack/tmp.nix@node-tmp': "bar2"
Use `lib.mkForce value` or `lib.mkDefault value` to change the priority on any of these definitions.
Using a mergeable type (`types.comma`, i.e. a string, where all declarations
get joined with a comma as delimiter) fixes the problem.
Closes#385603
The problem described is that `wal-g` requires syscalls from `@resources`.
However, we don't have support for it in the module now and I don't
think it's reasonable to only support hardening adjustments for things
support by this module. Also, list is a bad datatype here since it
doesn't allow the level of customizations we need.
This is only for the syscall filterset since it's the option that's hard
to customize otherwise. For downstream configs, it's recommended to
adjust the hardening as needed in other cases.
Hence I decided to implement `services.postgresql.systemCallFilter` with
the following semantics:
* `systemCallFilter."~@resources" = true` adds `~@resources` to the
filterset.
* Setting this to `false` (e.g. in a downstream configuration using
`wal-g`) removes the entry `~@resources` from the filterset. In this
case it's sufficient since `@system-service` implies `@resources` and
the `~@resources` declaration after that discards that.
I decided to not implement logic about negations in here, but to keep
it rather simple by only allowing to set/unset entries.
As described in `systemd.exec(5)`, the ordering matters: e.g.
`@system-service` implies `@resources`, but `~@resources` _after_ that
reverts that. By default, the ordering of the keys is as follows:
* syscall groups (starting with `@`) come at first.
* negations of syscall groups (starting with `~@`) come after that.
* anything else at the end.
If further ordering is needed, it can be done like this:
```
{
services.postgresql.systemCallFilter."~@resources" = {
enable = true; # whether or not it's part of the final SystemCallFilter
priority = 23; # ordering priority in the filterset.
};
}
```
The lower the priority, the higher up the entry will be in the final
filterset.
In the case that the user wants to provide a custom data directory, we
need to grant `ReadWritePaths` for that directory. Previously this would
not happen when `/var/lib/postgresql` was used, because the condition
was not in fact checking for the default data directory, creating a gap
in then if-else scenario.
Fixes: #371680
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.
Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.
A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.
This commit was automatically created and can be verified using
nix-build a08b3a4d19.tar.gz \
--argstr baseRev b32a094368
result/bin/apply-formatting $NIXPKGS_PATH
This will be EOL at the end of November, so there's little reason to
keep it in 24.11[1]. As discussed, we'd like to keep it for as long as
possible to make sure there's a state in nixpkgs that has the latest
minor of postgresql_12 available with the most recent CVEs fixed for
people who cannot upgrade[2].
This aspect has been made explicit in the manual now for the next .11
release.
During the discussions it has been brought up that if people just do
`services.postgresql.enable = true;` and let the code decide the
postgresql version based on `system.stateVersion`, there's a chance that
such EOL dates will be missed. To make this harder, a warning will now
be raised when using the stateVersion-condition and the oldest still
available major is selected.
Additionally regrouped the postgresql things in the release notes to
make sure these are all shown consecutively. Otherwise it's a little
hard to keep track of all the changes made to postgresql in 24.11.
[1] https://endoflife.date/postgresql
[2] https://github.com/NixOS/nixpkgs/pull/353158#issuecomment-2453056692
The test breaks like this otherwise:
machine # WARNING: error during JITing: Permission denied
machine # [ 14.012280] postgres[913]: [913] WARNING: error during JITing: Permission denied
machine # ERROR: failed to look up symbol "evalexpr_0_1": Failed to materialize symbols: { (main, { evalexpr_0_1, evalexpr_0_0 }) }
The issue was that the old test-case used `/tmp` to share data. Using
`JoinsNamespaceOf=` wasn't a real workaround since the private `/tmp` is
recreated when a service gets stopped/started which is the case here, so
the wals were still lost.
To keep the test building with `PrivateTmp=yes`, create a dedicated
directory in `/var/cache` with tmpfiles and allow the hardened
`postgresql.service` to access it via `ReadWritePaths`.
Upgrade default postgresql for stateVersion >=24.11.
This also rebuilds all packages linking against `libpq.so` to use
postgresql 16.
After re-reading https://www.postgresql.org/docs/16/release-16.html
I don't see any major risks about doing that.
these changes were generated with nixq 0.0.2, by running
nixq ">> lib.mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> mdDoc[remove] Argument[keep]" --batchmode nixos/**.nix
nixq ">> Inherit >> mdDoc[remove]" --batchmode nixos/**.nix
two mentions of the mdDoc function remain in nixos/, both of which
are inside of comments.
Since lib.mdDoc is already defined as just id, this commit is a no-op as
far as Nix (and the built manual) is concerned.
The enableJIT = true case was fixed in #221851 or
e2fb651752 respectively.
However this did not take the case into consideration, when doing this:
services.postgresql = {
enable = true;
enableJIT = false;
package = pkgs.postgresql_15_jit;
};
If enableJIT is treated as the source of truth, then this should indeed
cause JIT to be disabled, which this commit does.
The main idea behind that was to be able to do more sophisticated
merging for stuff that goes into `postgresql.conf`:
`shared_preload_libraries` is a comma-separated list in a `types.str`
and thus not mergeable. With this change, the option accepts both a
comma-separated string xor a list of strings.
This can be implemented rather quick using `coercedTo` +
freeform modules. The interface still behaves equally, but it allows to
merge declarations for this option together.
One side-effect was that I had to change the `attrsOf (oneOf ...)` part into
a submodule to allow declaring options for certain things. While at it,
I decided to move `log_line_prefix` and `port` into this structure as
well.
This makes it less error-prone to use the llvm package in extensions, because
it will always match the package used by the postgresql derivation itself.
Previously, you could've accidentally used llvm instead of postgresql.llvm
with a different result.
...effectively what was planned already in #266270, but it was too late
because the branches were restricted and didn't allow any breaking
changes anymore.
It also suffers from the same issue that we already had when discussing
this the last time[1] when `ensureDBOwnership` was ultimately introduced
as band-aid fix: newly created users don't get CREATE permission on
the `public` schema anymore (since psql 15), even with `ALL PRIVILEGES`.
If one's use-case is more sophisticated than having a single owner, it's
questionable anyways if this module is the correct tool since
permissions aren't dropped on a change to this option or a removal which
is pretty surprising in the context of NixOS.
[1] https://github.com/NixOS/nixpkgs/pull/266270
As it is technically a breaking change, we should at least make a strong deprecation
of `ensurePermissions` and leave it in the broken state it is, for out of tree users.
We give them a 6 months notice to migrate away by doing so, which is honest.
In the meantime, we forbid usage of `ensurePermissions` inside of nixpkgs.
Closes#216989
First of all, a bit of context: in PostgreSQL, newly created users don't
have the CREATE privilege on the public schema of a database even with
`ALL PRIVILEGES` granted via `ensurePermissions` which is how most of
the DB users are currently set up "declaratively"[1]. This means e.g. a
freshly deployed Nextcloud service will break early because Nextcloud
itself cannot CREATE any tables in the public schema anymore.
The other issue here is that `ensurePermissions` is a mere hack. It's
effectively a mixture of SQL code (e.g. `DATABASE foo` is relying on how
a value is substituted in a query. You'd have to parse a subset of SQL
to actually know which object are permissions granted to for a user).
After analyzing the existing modules I realized that in every case with
a single exception[2] the UNIX system user is equal to the db user is
equal to the db name and I don't see a compelling reason why people
would change that in 99% of the cases. In fact, some modules would even
break if you'd change that because the declarations of the system user &
the db user are mixed up[3].
So I decided to go with something new which restricts the ways to use
`ensure*` options rather than expanding those[4]. Effectively this means
that
* The DB user _must_ be equal to the DB name.
* Permissions are granted via `ensureDBOwnerhip` for an attribute-set in
`ensureUsers`. That way, the user is actually the owner and can
perform `CREATE`.
* For such a postgres user, a database must be declared in
`ensureDatabases`.
For anything else, a custom state management should be implemented. This
can either be `initialScript`, doing it manual, outside of the module or
by implementing proper state management for postgresql[5], but the
current state of `ensure*` isn't even declarative, but a convergent tool
which is what Nix actually claims to _not_ do.
Regarding existing setups: there are effectively two options:
* Leave everything as-is (assuming that system user == db user == db
name): then the DB user will automatically become the DB owner and
everything else stays the same.
* Drop the `createDatabase = true;` declarations: nothing will change
because a removal of `ensure*` statements is ignored, so it doesn't
matter at all whether this option is kept after the first deploy (and
later on you'd usually restore from backups anyways).
The DB user isn't the owner of the DB then, but for an existing setup
this is irrelevant because CREATE on the public schema isn't revoked
from existing users (only not granted for new users).
[1] not really declarative though because removals of these statements
are simply ignored for instance: https://github.com/NixOS/nixpkgs/issues/206467
[2] `services.invidious`: I removed the `ensure*` part temporarily
because it IMHO falls into the category "manage the state on your
own" (see the commit message). See also
https://github.com/NixOS/nixpkgs/pull/265857
[3] e.g. roundcube had `"DATABASE ${cfg.database.username}" = "ALL PRIVILEGES";`
[4] As opposed to other changes that are considered a potential fix, but
also add more things like collation for DBs or passwords that are
_never_ touched again when changing those.
[5] As suggested in e.g. https://github.com/NixOS/nixpkgs/issues/206467
As described in the release lifecycle docs from postgresql[1], v11 will
stop receiving fixes as of Nov 9 2023. This means it's EOL throughout
the entire lifetime of 23.11, so let's drop it now.
A lot of examples are also referencing postgresql_11. Where it's
sensible, use postgresql_15 as example now to avoid confusion.
This is also handy because the LLVM 16 fix for postgresql is not
available for postgresql 11 ;-)
[1] https://www.postgresql.org/support/versioning/
This was causing the following warning before when building the manual:
warning: literalExample is deprecated, use literalExpression instead, or use literalMD for a non-Nix description.
Rather than using `literalExpression`, nothing is used. This option
expects a string and the example is a string, no special handling
required. Both `literalExample` from the docbook ages and
`literalExpression` now are only required if the example is
a Nix expression rather than a value of the option's type.
Make sure that JIT is actually available when using
services.postgresql = {
enable = true;
enableJIT = true;
package = pkgs.postgresql_15;
};
The current behavior is counter-intuitive because the docs state that
`enableJIT = true;` is sufficient even though it wasn't in that case
because the declared package doesn't have the LLVM dependency.
Fixed by using `package.withJIT` if `enableJIT = true;` and
`package.jitSupport` is `false`.
Also updated the postgresql-jit test to test for that case.
This is useful if your postgresql version is dependant on
`system.stateVersion` and not pinned down manually. Then it's not
necessary to find out which version exactly is in use and define
`package` manually, but just stay with what NixOS provides as default:
$ nix-instantiate -A postgresql
/nix/store/82fzmb77mz2b787dgj7mn4a8i4f6l6sn-postgresql-14.7.drv
$ nix-instantiate -A postgresql_jit
/nix/store/qsjkb72fcrrfpsszrwbsi9q9wgp39m50-postgresql-14.7.drv
$ nix-instantiate -A postgresql.withJIT
/nix/store/qsjkb72fcrrfpsszrwbsi9q9wgp39m50-postgresql-14.7.drv
$ nix-instantiate -A postgresql.withJIT.withoutJIT
/nix/store/82fzmb77mz2b787dgj7mn4a8i4f6l6sn-postgresql-14.7.drv
I.e. you can use postgresql with JIT (for complex queries only[1]) like
this:
services.postgresql = {
enable = true;
enableJIT = true;
};
Performing a new override instead of re-using the `_jit`-variants for
that has the nice property that overlays for the original package apply
to the JIT-enabled variant, i.e.
with import ./. {
overlays = [
(self: super: {
postgresql = super.postgresql.overrideAttrs (_: { fnord = "snens"; });
})
];
};
postgresql.withJIT.fnord
still gives the string `snens` whereas `postgresql_jit` doesn't have the
attribute `fnord` in its derivation.
[1] https://www.postgresql.org/docs/current/runtime-config-query.html#GUC-JIT-ABOVE-COST
this converts meta.doc into an md pointer, not an xml pointer. since we
no longer need xml for manual chapters we can also remove support for
manual chapters from md-to-db.sh
since pandoc converts smart quotes to docbook quote elements and our
nixos-render-docs does not we lose this distinction in the rendered
output. that's probably not that bad, our stylesheet didn't make use of
this anyway (and pre-23.05 versions of the chapters didn't use quote
elements either).
also updates the nixpkgs manual to clarify that option docs support all
extensions (although it doesn't support headings at all, so heading
anchors don't work by extension).
