All workflows where it's remotely useful now trigger on a pull_request
event when the workflow file itself is changed. This gives us basic
sanity testing of changes to workflow files itself and reduces the need
for manual tests in forks.
Previously, the attrs step consisted of:
- 7s queue time
- 1m 15s run time
Only 25s of this were spent preparing the attr paths. A bit more than a
minute was just spent for queuing, checking out the repo, downloading
nix, downloading dependencies, uploading the artifacts - and then
downloading them again in the next step. All of that can be avoided if
we collect the attrs as part of the outpaths job.
By running the attrs step as part of each outpaths step the attrpaths
will be collected 4x, but:
- We save a minute for each eval run to complete.
- We save a full job, giving us more free runners and *possibly* less
queue times for other jobs in the repo.
- We reduce complexity in the workflow file.
We can use the same "hack" as in labeler-no-sync.yml, because OWNERS is
the only file without extension in that directory structure.
This makes it easier to search for CI related PRs via label.
This adds the minimum nix version and the latest lix version to the
matrix of parse checks. Especially the minimum nix version is relevant,
because parsing routinely breaks because of introduction of newer
syntax.
Adding lix just completes the picture.
The nix-parse workflow can now be run locally the same way as in CI.
To do this, the CI's workflow was slightly adjusted. Instead of testing
only the changed files, we're now testing all files in the repository.
This is possible in two ways:
1. By calling nix-instantiate once with all files as arguments. This
will be rather fast, but only the first error is shown before it errors
out.
2. By calling nix-instantiate once for each file. This will be much
slower, but has the advantage that we see all errors at once.
To avoid running the long variant every time, we first do a quick check
with the fast version. If that fails, we run the slower one to report
the errors. This gives us the best of both.
ARM runners are supposed to be more energy efficient than x86. Also,
from limited testing, they appear to be faster for the eval jobs as
well. Average run time for the "Outpaths (x86_64-linux)" job was 4m 27s,
so far. In the first run, this job came in at 3m 9s. This effect did not
show for other jobs, yet.
The following two exceptions are made right now:
- nixpkgs-lib-tests currently fails on the ARM runner building Nix 2.3
- nixpkgs-vet is currently pinned to a x86_64-linux only binary release
I added a lint-action.sh script in .github/workflows a while ago while
fixing some warnings. But I haven't run it myself ever since. This needs
to be part of CI to make any use of it.
The eval-aliases job is independent of all the other eval jobs. By
splitting it into a separate workflow, we avoid running it in two cases:
1. When turning a PR "ready to review". In this case, the main eval
workflow needs to run to be able to tag reviewers - but not
eval-aliases.
2. On branches like master, staging, etc. We only need to run eval there
to have a result to compare against in PRs. eval-aliases doesn't
contribute to that.
Thus, this will avoid wasting resources.
We intend to use the edited event to react to base branch changes - but
before this change, we also ran those jobs on simple edits like title or
description.
While this works for some of the quicker jobs, it will not be
sustainable for all evaluation-related jobs. But evaluation needs to be
re-triggered on a base branch change as well, thus this change.
The workflow has been disabled for 9 months. Except for the Eval
workflow, this is the most complex, yet unused, workflow. As discussed
in #332695, this needs a proper wrapper first. Chances are high, that
once a good CLI tool is available, the workflow would be implemented
entirely different: We could easily run it via treefmt as well, so that
we get the same results locally as in CI.
We already have treefmt running for nixfmt, so it's easy to just add
another formatter to it. This gives a much better UX, because all
formatting errors are reported through the same channel.
It also saves us one CI job, which takes most of the time to just set up
the machine, clone the repo and download Nix - while doing a minimum of
actual work.
Total execution time for treefmt is ~10% slower:
- 38s only nixfmt
- 43s nixfmt + editorconfig-checker
We have a pinned nixpkgs revision for all our CI tools, but we're not
making use of it for editorconfig-checker. Instead, it has it's own,
unmaintained, pin.
Let's fix that. On the way, we're upgrading the tool to 3.2.1, which
requires adjusting our .editorconfig file slightly to ignore
auto-generated package-lock.json files.
We already have a generic keep-sorted workflow, so we don't need to roll
our own for the maintainers list.
This will result in a slightly different order, because the nix script
ignored "-" and "_", while keep-sorted sorts them first. But, the order
is consistent and that's what matters.
Because of the branches filter, the job would **not** re-run when only
the base-branch was changed, thus leaving the contributor with a failed
check.
With this change, the job should be triggered and then skipped in this
case, so the check in the list should be replaced with a skip.
Note, this also means that this step will show up in every PR as
skipped, even when it was never failed before.
