render all manual chapters to docbook from scratch every time the manual
is built. nixos-render-docs is quick enough at this to not worry about
the cost (needing only about a second), and it means we can remove
md-to-db.sh in the next commit.
no changes to the rendered html manual except for replacements and smartquotes.
we'll soon add another docbook converter that does not emit a section as
a collection of chapters, but sections or chapters on their own. this
should clarify naming a bit before there can be any confusion.
mdoc is just too slow to render on groff, and semantic markup doesn't
help us any for generated pages.
this produces a lot of changes to configuration.nix.5, but only few
rendering changes. most of those seem to be place losing a space where
docbook emitted roff code that did not faithfully represent the input
text, though a few places also gained space where docbook dropped them.
notably we also don't need the compatibility code docbook-xsl emitted
because that problem was fixed over a decade ago.
this will handle block quotes, which the docbook stylesheets turned into
a mess of roff requests that ended up showing up in the output instead
of being processed.
since we want to move away from docbook and having docbook manpages
around is going to block further progress we have to translate the
current nixos manpages from docbook it *something* else. mdoc seems the
most appropriate choice since markdown can't represent all the things
manpages can differentiate with even more extensions. if we ever need
html manpages we can render them using troff, like many of the online
manpage viewers, or rewrite them again. since we haven't had html
manpages for any of these in many years that seems unlikely to happen.
unlike most of the recent markdown conversions this comes with a lot of
minor rendering changes, mostly in spacing. docbook-xslt creates manual
pages in a very different dialect than mdoc (which is used here).
this converts meta.doc into an md pointer, not an xml pointer. since we
no longer need xml for manual chapters we can also remove support for
manual chapters from md-to-db.sh
since pandoc converts smart quotes to docbook quote elements and our
nixos-render-docs does not we lose this distinction in the rendered
output. that's probably not that bad, our stylesheet didn't make use of
this anyway (and pre-23.05 versions of the chapters didn't use quote
elements either).
also updates the nixpkgs manual to clarify that option docs support all
extensions (although it doesn't support headings at all, so heading
anchors don't work by extension).
as far as we can tell nixos has only ever had a total of one olink, and
currently has no olinks at all. we can't currently represent olinks in
markdown docs, and if we re-add support for cross-document links they
will take a different form (and not use docbook, which will have to be
phased out before we re-add anything).
the olinkdb is thus unused and takes 10 seconds on our machine to build,
holding up the rest of the manual for no benefit.
following the plan in https://github.com/NixOS/nixpkgs/pull/189318#discussion_r961764451
also adds an activation script to print the warning during activation
instead of during build, otherwise folks using the new CLI that hides
build logs by default might never see the warning.
deprecate literalDocBook by adding a warning (that will not fire yet) to
its uses and other docbook literal strings by adding optional warning
message to mergeJSON.
By writing the unchecked outputs before checking them, they will
be written to a store path, which appears in the log, and which
sticks around even if the build fails. Eventually it is GCed, but
until then, you can open the file.
If you run it in a terminal+editor combination like VSCode, the
failure location is just one Ctrl+click away.
a few things should've used buildPackages/nativeBuildInputs to not not require
the host architecture for building docs. tested by building aarch64-linux docs
on x86_64-linux, and the result looks good.
this partially solves the problem of "missing description" warnings of the
options doc build being lost by nix build, at the cost of failing builds that
previously ran. an option to disable this behaviour is provided.
most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.
in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
Motivation is to support other repositories containing nixos
modules that would like to generate options documentation:
- nix-darwin
- private repos
- arion
- ??
Because when I see "config.system.build.manual.manual" after I forgot
what it means I ask "Why do I need that second `.manual` there again?".
Doesn't happen with `config.system.build.manual.manualHTML`.
What annoyed me for a long time was the fact, that in order to break
into a new paragraph, you need to insert </para><para> in the
description attribute of an option.
Now we will automatically create <para/> elements for every block that
is separated by two consecutive newlines.
I first tried to do this within options-to-docbook.xsl, but it turns
out[1] that this isn't directly possible with XSLT 1.0, so I added
another XSLT file that postprocesses the option descriptions that are
now enclosed in <nixos:option-description/> by options-to-docbook.xsl.
The splitting itself is a bit more involved, because we can't simply
split on every \n\n because we'd also split text nodes of elements, for
example:
<screen><![CDATA[
one line
another one
]]></screen>
This would create one <para/> element for "one line" and another for
"another line", which we obviously don't want because <screen/> is used
to display verbatim contents of what a user is seeing on the screen.
So what we do instead is splitting *only* the top-level text nodes
within the outermost <para/> and leave all elements as-is. If there are
more than one <para/> elements at the top-level, we simply don't process
it at all, because the description then already contains </para><para>.
https://www.mhonarc.org/archive/html/xsl-list/2012-09/msg00319.html
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edolstra, @domenkozar
This makes the command ‘nix-env -qa -f. --arg config '{skipAliases =
true;}'’ work in Nixpkgs.
Misc...
- qtikz: use libsForQt5.callPackage
This ensures we get the right poppler.
- rewrites:
docbook5_xsl -> docbook_xsl_ns
docbook_xml_xslt -> docbook_xsl
diffpdf: fixup
- Rectifies diverging CSS by combining
nixos/nixpkgs docs CSS
- Moves our custom Highlight.js loader in to
the hljs package
- Switches the nixos docs to use SVG
callouts too
This allows one to specify "related packages" in NixOS that get rendered into
the configuration.nix(5) man page. The interface philosophy is pretty much
stolen from TeX bibliography.
See the next several commits for examples.
Among other things, this will allow *2nix tools to output plain data
while still being composable with the traditional
callPackage/.override interfaces.
This allows one to specify "related packages" in NixOS that get rendered into
the configuration.nix(5) man page. The interface philosophy is pretty much
stolen from TeX bibliography.
Previously only the line numbers of a giant, internally generated XML file were
printed, without any kind of debuggability.
Now at least the mentioned lines are printed with a little bit of surrounding
context (to have something to grep for).
```
manual-combined.xml:4863: element para: Relax-NG validity error : Did not expect element para there
4859 <chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xi="http://www.w3.org/2001/XInclude" version="5.0" xml:id="sec-writing-modules">
4860
4861 <title>Writing NixOS Modules</title>
4862
4863 <para>NixOS has a modular system for declarative configuration. This
4864 system combines multiple <emphasis>modules</emphasis> to produce the
4865 full system configuration. One of the modules that constitute the
```
