Merge branch 'containers'

Fixes #2105.

maintainers/scripts/nixpkgs-lint.pl

@@ -31,8 +31,7 @@ GetOptions("package|p=s" => \$filter,
            "maintainer|m=s" => \$maintainer,
            "file|f=s" => \$path,
            "help" => sub { showHelp() }
-    )
+    ) or exit 1;
-    or die("syntax: $0 ...\n");
 
 # Evaluate Nixpkgs into an XML representation.
 my $xml = `nix-env -f '$path' -qa '$filter' --xml --meta --drv-path`;

nixos/doc/manual/containers.xml

@@ -0,0 +1,242 @@
+<chapter xmlns="http://docbook.org/ns/docbook"
+         xmlns:xlink="http://www.w3.org/1999/xlink"
+         xml:id="ch-containers">
+
+<title>Containers</title>
+
+<para>NixOS allows you to easily run other NixOS instances as
+<emphasis>containers</emphasis>. Containers are a light-weight
+approach to virtualisation that runs software in the container at the
+same speed as in the host system. NixOS containers share the Nix store
+of the host, making container creation very efficient.</para>
+
+<warning><para>Currently, NixOS containers are not perfectly isolated
+from the host system. This means that a user with root access to the
+container can do things that affect the host. So you should not give
+container root access to untrusted users.</para></warning>
+
+<para>NixOS containers can be created in two ways: imperatively, using
+the command <command>nixos-container</command>, and declaratively, by
+specifying them in your <filename>configuration.nix</filename>. The
+declarative approach implies that containers get upgraded along with
+your host system when you run <command>nixos-rebuild</command>, which
+is often not what you want. By contrast, in the imperative approach,
+containers are configured and updated independently from the host
+system.</para>
+
+
+<section><title>Imperative container management</title>
+
+<para>We’ll cover imperative container management using
+<command>nixos-container</command> first. You create a container with
+identifier <literal>foo</literal> as follows:
+
+<screen>
+$ nixos-container create foo
+</screen>
+
+This creates the container’s root directory in
+<filename>/var/lib/containers/foo</filename> and a small configuration
+file in <filename>/etc/containers/foo.conf</filename>. It also builds
+the container’s initial system configuration and stores it in
+<filename>/nix/var/nix/profiles/per-container/foo/system</filename>. You
+can modify the initial configuration of the container on the command
+line. For instance, to create a container that has
+<command>sshd</command> running, with the given public key for
+<literal>root</literal>:
+
+<screen>
+$ nixos-container create foo --config 'services.openssh.enable = true; \
+  users.extraUsers.root.openssh.authorizedKeys.keys = ["ssh-dss AAAAB3N…"];'
+</screen>
+
+</para>
+
+<para>Creating a container does not start it. To start the container,
+run:
+
+<screen>
+$ nixos-container start foo
+</screen>
+
+This command will return as soon as the container has booted and has
+reached <literal>multi-user.target</literal>. On the host, the
+container runs within a systemd unit called
+<literal>container@<replaceable>container-name</replaceable>.service</literal>.
+Thus, if something went wrong, you can get status info using
+<command>systemctl</command>:
+
+<screen>
+$ systemctl status container@foo
+</screen>
+
+</para>
+
+<para>If the container has started succesfully, you can log in as
+root using the <command>root-login</command> operation:
+
+<screen>
+$ nixos-container root-login foo
+[root@foo:~]#
+</screen>
+
+Note that only root on the host can do this (since there is no
+authentication).  You can also get a regular login prompt using the
+<command>login</command> operation, which is available to all users on
+the host:
+
+<screen>
+$ nixos-container login foo
+foo login: alice
+Password: ***
+</screen>
+
+With <command>nixos-container run</command>, you can execute arbitrary
+commands in the container:
+
+<screen>
+$ nixos-container run foo -- uname -a
+Linux foo 3.4.82 #1-NixOS SMP Thu Mar 20 14:44:05 UTC 2014 x86_64 GNU/Linux
+</screen>
+
+</para>
+
+<para>There are several ways to change the configuration of the
+container. First, on the host, you can edit
+<literal>/var/lib/container/<replaceable>name</replaceable>/etc/nixos/configuration.nix</literal>,
+and run
+
+<screen>
+$ nixos-container update foo
+</screen>
+
+This will build and activate the new configuration. You can also
+specify a new configuration on the command line:
+
+<screen>
+$ nixos-container update foo --config 'services.httpd.enable = true; \
+  services.httpd.adminAddr = "foo@example.org";'
+
+$ curl http://$(nixos-container show-ip foo)/
+&lt;!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">…
+</screen>
+
+However, note that this will overwrite the container’s
+<filename>/etc/nixos/configuration.nix</filename>.</para>
+
+<para>Alternatively, you can change the configuration from within the
+container itself by running <command>nixos-rebuild switch</command>
+inside the container. Note that the container by default does not have
+a copy of the NixOS channel, so you should run <command>nix-channel
+--update</command> first.</para>
+
+<para>Containers can be stopped and started using
+<literal>nixos-container stop</literal> and <literal>nixos-container
+start</literal>, respectively, or by using
+<command>systemctl</command> on the container’s service unit. To
+destroy a container, including its file system, do
+
+<screen>
+$ nixos-container destroy foo
+</screen>
+
+</para>
+
+</section>
+
+
+<section><title>Declarative container specification</title>
+
+<para>You can also specify containers and their configuration in the
+host’s <filename>configuration.nix</filename>.  For example, the
+following specifies that there shall be a container named
+<literal>database</literal> running PostgreSQL:
+
+<programlisting>
+containers.database =
+  { config =
+      { config, pkgs, ... }:
+      { services.postgresql.enable = true;
+        services.postgresql.package = pkgs.postgresql92;
+      };
+  };
+</programlisting>
+
+If you run <literal>nixos-rebuild switch</literal>, the container will
+be built and started. If the container was already running, it will be
+updated in place, without rebooting.</para>
+
+<para>By default, declarative containers share the network namespace
+of the host, meaning that they can listen on (privileged)
+ports. However, they cannot change the network configuration. You can
+give a container its own network as follows:
+
+<programlisting>
+containers.database =
+  { privateNetwork = true;
+    hostAddress = "192.168.100.10";
+    localAddress = "192.168.100.11";
+  };
+</programlisting>
+
+This gives the container a private virtual Ethernet interface with IP
+address <literal>192.168.100.11</literal>, which is hooked up to a
+virtual Ethernet interface on the host with IP address
+<literal>192.168.100.10</literal>.  (See the next section for details
+on container networking.)</para>
+
+<para>To disable the container, just remove it from
+<filename>configuration.nix</filename> and run <literal>nixos-rebuild
+switch</literal>. Note that this will not delete the root directory of
+the container in <literal>/var/lib/containers</literal>.</para>
+
+</section>
+
+
+<section><title>Networking</title>
+
+<para>When you create a container using <literal>nixos-container
+create</literal>, it gets it own private IPv4 address in the range
+<literal>10.233.0.0/16</literal>. You can get the container’s IPv4
+address as follows:
+
+<screen>
+$ nixos-container show-ip foo
+10.233.4.2
+
+$ ping -c1 10.233.4.2
+64 bytes from 10.233.4.2: icmp_seq=1 ttl=64 time=0.106 ms
+</screen>
+
+</para>
+
+<para>Networking is implemented using a pair of virtual Ethernet
+devices. The network interface in the container is called
+<literal>eth0</literal>, while the matching interface in the host is
+called <literal>c-<replaceable>container-name</replaceable></literal>
+(e.g., <literal>c-foo</literal>).  The container has its own network
+namespace and the <literal>CAP_NET_ADMIN</literal> capability, so it
+can perform arbitrary network configuration such as setting up
+firewall rules, without affecting or having access to the host’s
+network.</para>
+
+<para>By default, containers cannot talk to the outside network. If
+you want that, you should set up Network Address Translation (NAT)
+rules on the host to rewrite container traffic to use your external
+IP address. This can be accomplished using the following configuration
+on the host:
+
+<programlisting>
+networking.nat.enable = true;
+networking.nat.internalInterfaces = ["c-+"];
+networking.nat.externalInterface = "eth0";
+</programlisting>
+where <literal>eth0</literal> should be replaced with the desired
+external interface. Note that <literal>c-+</literal> is a wildcard
+that matches all container interfaces.</para>
+
+</section>
+
+
+</chapter>
+

nixos/doc/manual/manual.xml

@@ -54,6 +54,7 @@
   <xi:include href="running.xml" />
   <!-- <xi:include href="userconfiguration.xml" /> -->
   <xi:include href="troubleshooting.xml" />
+  <xi:include href="containers.xml" />
   <xi:include href="development.xml" />
 
   <xi:include href="release-notes.xml" />

nixos/modules/installer/cd-dvd/channel.nix

@@ -28,7 +28,7 @@ in
 {
   # Provide the NixOS/Nixpkgs sources in /etc/nixos.  This is required
   # for nixos-install.
-  boot.postBootCommands =
+  boot.postBootCommands = mkAfter
     ''
       if ! [ -e /var/lib/nixos/did-channel-init ]; then
         echo "unpacking the NixOS/Nixpkgs sources..."

nixos/modules/module-list.nix

@@ -307,6 +307,7 @@
   ./tasks/scsi-link-power-management.nix
   ./tasks/swraid.nix
   ./testing/service-runner.nix
+  ./virtualisation/container-config.nix
   ./virtualisation/containers.nix
   ./virtualisation/libvirtd.nix
   #./virtualisation/nova.nix

nixos/modules/services/networking/dhcpcd.nix

@@ -34,8 +34,9 @@ let
 
       # Ignore peth* devices; on Xen, they're renamed physical
       # Ethernet cards used for bridging.  Likewise for vif* and tap*
-      # (Xen) and virbr* and vnet* (libvirt).
+      # (Xen) and virbr* and vnet* (libvirt) and c-* and ctmp-* (NixOS
-      denyinterfaces ${toString ignoredInterfaces} peth* vif* tap* tun* virbr* vnet* vboxnet*
+      # containers).
+      denyinterfaces ${toString ignoredInterfaces} peth* vif* tap* tun* virbr* vnet* vboxnet* c-* ctmp-*
 
       ${config.networking.dhcpcd.extraConfig}
     '';

nixos/modules/services/networking/nat.nix

@@ -10,6 +10,8 @@ let
 
   cfg = config.networking.nat;
 
+  dest = if cfg.externalIP == null then "-j MASQUERADE" else "-j SNAT --to-source ${cfg.externalIP}";
+
 in
 
 {
@@ -27,14 +29,27 @@ in
         '';
     };
 
+    networking.nat.internalInterfaces = mkOption {
+      type = types.listOf types.str;
+      default = [];
+      example = [ "eth0" ];
+      description =
+        ''
+          The interfaces for which to perform NAT. Packets coming from
+          these interface and destined for the external interface will
+          be rewritten.
+        '';
+    };
+
     networking.nat.internalIPs = mkOption {
       type = types.listOf types.str;
-      example = [ "192.168.1.0/24" ] ;
+      default = [];
+      example = [ "192.168.1.0/24" ];
       description =
         ''
           The IP address ranges for which to perform NAT.  Packets
-          coming from these networks and destined for the external
+          coming from these addresses (on any interface) and destined
-          interface will be rewritten.
+          for the external interface will be rewritten.
         '';
     };
 
@@ -80,25 +95,37 @@ in
 
         preStart =
           ''
+            iptables -t nat -F PREROUTING
             iptables -t nat -F POSTROUTING
             iptables -t nat -X
-          ''
+
-          + (concatMapStrings (network:
+            # We can't match on incoming interface in POSTROUTING, so
-            ''
+            # mark packets coming from the external interfaces.
-            iptables -t nat -A POSTROUTING \
+            ${concatMapStrings (iface: ''
-              -s ${network} -o ${cfg.externalInterface} \
+              iptables -t nat -A PREROUTING \
-              ${if cfg.externalIP == null
+                -i '${iface}' -j MARK --set-mark 1
-                then "-j MASQUERADE"
+            '') cfg.internalInterfaces}
-                else "-j SNAT --to-source ${cfg.externalIP}"}
+
-            ''
+            # NAT the marked packets.
-          ) cfg.internalIPs) +
+            ${optionalString (cfg.internalInterfaces != []) ''
-          ''
+              iptables -t nat -A POSTROUTING -m mark --mark 1 \
+                -o ${cfg.externalInterface} ${dest}
+            ''}
+
+            # NAT packets coming from the internal IPs.
+            ${concatMapStrings (range: ''
+              iptables -t nat -A POSTROUTING \
+                -s '${range}' -o ${cfg.externalInterface} ${dest}}
+            '') cfg.internalIPs}
+
             echo 1 > /proc/sys/net/ipv4/ip_forward
           '';
 
         postStop =
           ''
+            iptables -t nat -F PREROUTING
             iptables -t nat -F POSTROUTING
+            iptables -t nat -X
           '';
       };
   };

nixos/modules/services/web-servers/apache-httpd/default.nix

@@ -621,7 +621,7 @@ in
       { description = "Apache HTTPD";
 
         wantedBy = [ "multi-user.target" ];
-        requires = [ "keys.target" ];
+        wants = [ "keys.target" ];
         after = [ "network.target" "fs.target" "postgresql.service" "keys.target" ];
 
         path =

nixos/modules/system/activation/switch-to-configuration.pl

@@ -26,7 +26,10 @@ EOF
     exit 1;
 }
 
-die "This is not a NixOS installation (/etc/NIXOS is missing)!\n" unless -f "/etc/NIXOS";
+# This is a NixOS installation if it has /etc/NIXOS or a proper
+# /etc/os-release.
+die "This is not a NixOS installation!\n" unless
+    -f "/etc/NIXOS" || (read_file("/etc/os-release", err_mode => 'quiet') // "") =~ /ID=nixos/s;
 
 openlog("nixos", "", LOG_USER);
 
@@ -173,7 +176,10 @@ while (my ($unit, $state) = each %{$activePrev}) {
                 # FIXME: do something?
             } else {
                 my $unitInfo = parseUnit($newUnitFile);
-                if (!boolIsTrue($unitInfo->{'X-RestartIfChanged'} // "yes")) {
+                if (boolIsTrue($unitInfo->{'X-ReloadIfChanged'} // "no")) {
+                    write_file($reloadListFile, { append => 1 }, "$unit\n");
+                }
+                elsif (!boolIsTrue($unitInfo->{'X-RestartIfChanged'} // "yes")) {
                     push @unitsToSkip, $unit;
                 } else {
                     # If this unit is socket-activated, then stop the
@@ -321,7 +327,7 @@ if (scalar @restart > 0) {
 # that are symlinks to other units.  We shouldn't start both at the
 # same time because we'll get a "Failed to add path to set" error from
 # systemd.
-my @start = unique("default.target", "timers.target", split('\n', read_file($startListFile, err_mode => 'quiet') // ""));
+my @start = unique("default.target", "timers.target", "sockets.target", split('\n', read_file($startListFile, err_mode => 'quiet') // ""));
 print STDERR "starting the following units: ", join(", ", sort(@start)), "\n";
 system("@systemd@/bin/systemctl", "start", "--", @start) == 0 or $res = 4;
 unlink($startListFile);

nixos/modules/system/boot/systemd-unit-options.nix

@@ -243,6 +243,17 @@ in rec {
       '';
     };
 
+    reloadIfChanged = mkOption {
+      type = types.bool;
+      default = false;
+      description = ''
+        Whether the service should be reloaded during a NixOS
+        configuration switch if its definition has changed.  If
+        enabled, the value of <option>restartIfChanged</option> is
+        ignored.
+      '';
+    };
+
     stopIfChanged = mkOption {
       type = types.bool;
       default = true;

nixos/modules/system/boot/systemd.nix

@@ -279,7 +279,11 @@ let
           [Service]
           ${let env = cfg.globalEnvironment // def.environment;
             in concatMapStrings (n: "Environment=\"${n}=${getAttr n env}\"\n") (attrNames env)}
-          ${optionalString (!def.restartIfChanged) "X-RestartIfChanged=false"}
+          ${if def.reloadIfChanged then ''
+            X-ReloadIfChanged=true
+          '' else if !def.restartIfChanged then ''
+            X-RestartIfChanged=false
+          '' else ""}
           ${optionalString (!def.stopIfChanged) "X-StopIfChanged=false"}
           ${attrsToSection def.serviceConfig}
         '';

nixos/modules/virtualisation/container-config.nix

@@ -0,0 +1,103 @@
+{ config, pkgs, lib, ... }:
+
+with lib;
+
+{
+
+  config = mkIf config.boot.isContainer {
+
+    # Provide a login prompt on /var/lib/login.socket.  On the host,
+    # you can connect to it by running ‘socat
+    # unix:<path-to-container>/var/lib/login.socket -,echo=0,raw’.
+    systemd.sockets.login =
+      { description = "Login Socket";
+        wantedBy = [ "sockets.target" ];
+        socketConfig =
+          { ListenStream = "/var/lib/login.socket";
+            SocketMode = "0666";
+            Accept = true;
+          };
+      };
+
+    systemd.services."login@" =
+      { description = "Login %i";
+        environment.TERM = "linux";
+        serviceConfig =
+          { Type = "simple";
+            StandardInput = "socket";
+            ExecStart = "${pkgs.socat}/bin/socat -t0 - exec:${pkgs.shadow}/bin/login,pty,setsid,setpgid,stderr,ctty";
+            TimeoutStopSec = 1; # FIXME
+          };
+      };
+
+    # Also provide a root login prompt on /var/lib/root-login.socket
+    # that doesn't ask for a password. This socket can only be used by
+    # root on the host.
+    systemd.sockets.root-login =
+      { description = "Root Login Socket";
+        wantedBy = [ "sockets.target" ];
+        socketConfig =
+          { ListenStream = "/var/lib/root-login.socket";
+            SocketMode = "0600";
+            Accept = true;
+          };
+      };
+
+    systemd.services."root-login@" =
+      { description = "Root Login %i";
+        environment.TERM = "linux";
+        serviceConfig =
+          { Type = "simple";
+            StandardInput = "socket";
+            ExecStart = "${pkgs.socat}/bin/socat -t0 - \"exec:${pkgs.shadow}/bin/login -f root,pty,setsid,setpgid,stderr,ctty\"";
+            TimeoutStopSec = 1; # FIXME
+          };
+      };
+
+    # Provide a daemon on /var/lib/run-command.socket that reads a
+    # command from stdin and executes it.
+    systemd.sockets.run-command =
+      { description = "Run Command Socket";
+        wantedBy = [ "sockets.target" ];
+        socketConfig =
+          { ListenStream = "/var/lib/run-command.socket";
+            SocketMode = "0600";  # only root can connect
+            Accept = true;
+          };
+      };
+
+    systemd.services."run-command@" =
+      { description = "Run Command %i";
+        environment.TERM = "linux";
+        serviceConfig =
+          { Type = "simple";
+            StandardInput = "socket";
+            TimeoutStopSec = 1; # FIXME
+          };
+        script =
+          ''
+            #! ${pkgs.stdenv.shell} -e
+            source /etc/bashrc
+            read c
+            eval "command=($c)"
+            exec "''${command[@]}"
+          '';
+      };
+
+    systemd.services.container-startup-done =
+      { description = "Container Startup Notification";
+        wantedBy = [ "multi-user.target" ];
+        after = [ "multi-user.target" ];
+        script =
+          ''
+            if [ -p /var/lib/startup-done ]; then
+              echo done > /var/lib/startup-done
+            fi
+          '';
+        serviceConfig.Type = "oneshot";
+        serviceConfig.RemainAfterExit = true;
+      };
+
+  };
+
+}

nixos/modules/virtualisation/containers.nix

@@ -2,6 +2,29 @@
 
 with pkgs.lib;
 
+let
+
+  runInNetns = pkgs.stdenv.mkDerivation {
+    name = "run-in-netns";
+    unpackPhase = "true";
+    buildPhase = ''
+      mkdir -p $out/bin
+      gcc ${./run-in-netns.c} -o $out/bin/run-in-netns
+    '';
+    installPhase = "true";
+  };
+
+  nixos-container = pkgs.substituteAll {
+    name = "nixos-container";
+    dir = "bin";
+    isExecutable = true;
+    src = ./nixos-container.pl;
+    perl = "${pkgs.perl}/bin/perl -I${pkgs.perlPackages.FileSlurp}/lib/perl5/site_perl";
+    inherit (pkgs) socat;
+  };
+
+in
+
 {
   options = {
 
@@ -14,19 +37,12 @@ with pkgs.lib;
       '';
     };
 
-    systemd.containers = mkOption {
+    containers = mkOption {
       type = types.attrsOf (types.submodule (
         { config, options, name, ... }:
         {
           options = {
 
-            root = mkOption {
-              type = types.path;
-              description = ''
-                The root directory of the container.
-              '';
-            };
-
             config = mkOption {
               description = ''
                 A specification of the desired configuration of this
@@ -45,21 +61,53 @@ with pkgs.lib;
               '';
             };
 
+            privateNetwork = mkOption {
+              type = types.bool;
+              default = false;
+              description = ''
+                Whether to give the container its own private virtual
+                Ethernet interface.  The interface is called
+                <literal>eth0</literal>, and is hooked up to the interface
+                <literal>c-<replaceable>container-name</replaceable></literal>
+                on the host.  If this option is not set, then the
+                container shares the network interfaces of the host,
+                and can bind to any port on any interface.
+              '';
+            };
+
+            hostAddress = mkOption {
+              type = types.nullOr types.string;
+              default = null;
+              example = "10.231.136.1";
+              description = ''
+                The IPv4 address assigned to the host interface.
+              '';
+            };
+
+            localAddress = mkOption {
+              type = types.nullOr types.string;
+              default = null;
+              example = "10.231.136.2";
+              description = ''
+                The IPv4 address assigned to <literal>eth0</literal>
+                in the container.
+              '';
+            };
+
           };
 
           config = mkMerge
-            [ { root = mkDefault "/var/lib/containers/${name}";
+            [ (mkIf options.config.isDefined {
-              }
-              (mkIf options.config.isDefined {
                 path = (import ../../lib/eval-config.nix {
                   modules =
                     let extraConfig =
                       { boot.isContainer = true;
                         security.initialRootPassword = mkDefault "!";
                         networking.hostName = mkDefault name;
+                        networking.useDHCP = false;
                       };
                     in [ extraConfig config.config ];
-                  prefix = [ "systemd" "containers" name ];
+                  prefix = [ "containers" name ];
                 }).config.system.build.toplevel;
               })
             ];
@@ -69,12 +117,10 @@ with pkgs.lib;
       example = literalExample
         ''
           { webserver =
-              { root = "/containers/webserver";
+              { path = "/nix/var/nix/profiles/webserver";
-                path = "/nix/var/nix/profiles/webserver";
               };
             database =
-              { root = "/containers/database";
+              { config =
-                config =
                   { config, pkgs, ... }:
                   { services.postgresql.enable = true;
                     services.postgresql.package = pkgs.postgresql92;
@@ -94,29 +140,96 @@ with pkgs.lib;
   };
 
 
-  config = {
+  config = mkIf (!config.boot.isContainer) {
 
-    systemd.services = mapAttrs' (name: container: nameValuePair "container-${name}"
+    systemd.services."container@" =
-      { description = "Container '${name}'";
+      { description = "Container '%i'";
 
-        wantedBy = [ "multi-user.target" ];
+        unitConfig.RequiresMountsFor = [ "/var/lib/containers/%i" ];
 
-        unitConfig.RequiresMountsFor = [ container.root ];
+        path = [ pkgs.iproute ];
+
+        environment.INSTANCE = "%i";
+        environment.root = "/var/lib/containers/%i";
 
         preStart =
           ''
-            mkdir -p -m 0755 ${container.root}/etc
+            mkdir -p -m 0755 $root/var/lib
-            if ! [ -e ${container.root}/etc/os-release ]; then
+
-              touch ${container.root}/etc/os-release
+            # Create a named pipe to get a signal when the container
+            # has finished booting.
+            rm -f $root/var/lib/startup-done
+            mkfifo -m 0600 $root/var/lib/startup-done
+         '';
+
+        script =
+          ''
+            mkdir -p -m 0755 "$root/etc" "$root/var/lib"
+            if ! [ -e "$root/etc/os-release" ]; then
+              touch "$root/etc/os-release"
             fi
+
+            mkdir -p -m 0755 \
+              "/nix/var/nix/profiles/per-container/$INSTANCE" \
+              "/nix/var/nix/gcroots/per-container/$INSTANCE"
+
+            SYSTEM_PATH=/nix/var/nix/profiles/system
+            if [ -f "/etc/containers/$INSTANCE.conf" ]; then
+              . "/etc/containers/$INSTANCE.conf"
+            fi
+
+            # Cleanup from last time.
+            ifaceHost=c-$INSTANCE
+            ifaceCont=ctmp-$INSTANCE
+            ns=net-$INSTANCE
+            ip netns del $ns 2> /dev/null || true
+            ip link del $ifaceHost 2> /dev/null || true
+            ip link del $ifaceCont 2> /dev/null || true
+
+            if [ "$PRIVATE_NETWORK" = 1 ]; then
+              # Create a pair of virtual ethernet devices.  On the host,
+              # we get ‘c-<container-name’, and on the guest, we get
+              # ‘eth0’.
+              ip link add $ifaceHost type veth peer name $ifaceCont
+              ip netns add $ns
+              ip link set $ifaceCont netns $ns
+              ip netns exec $ns ip link set $ifaceCont name eth0
+              ip netns exec $ns ip link set dev eth0 up
+              ip link set dev $ifaceHost up
+              if [ -n "$HOST_ADDRESS" ]; then
+                ip addr add $HOST_ADDRESS dev $ifaceHost
+                ip netns exec $ns ip route add $HOST_ADDRESS dev eth0
+                ip netns exec $ns ip route add default via $HOST_ADDRESS
+              fi
+              if [ -n "$LOCAL_ADDRESS" ]; then
+                ip netns exec $ns ip addr add $LOCAL_ADDRESS dev eth0
+                ip route add $LOCAL_ADDRESS dev $ifaceHost
+              fi
+              runInNetNs="${runInNetns}/bin/run-in-netns $ns"
+              extraFlags="--capability=CAP_NET_ADMIN"
+            fi
+
+            exec $runInNetNs ${config.systemd.package}/bin/systemd-nspawn \
+              -M "$INSTANCE" -D "/var/lib/containers/$INSTANCE" $extraFlags \
+              --bind-ro=/nix/store \
+              --bind-ro=/nix/var/nix/db \
+              --bind-ro=/nix/var/nix/daemon-socket \
+              --bind="/nix/var/nix/profiles/per-container/$INSTANCE:/nix/var/nix/profiles" \
+              --bind="/nix/var/nix/gcroots/per-container/$INSTANCE:/nix/var/nix/gcroots" \
+              "$SYSTEM_PATH/init"
           '';
 
-        serviceConfig.ExecStart =
+        postStart =
-          "${config.systemd.package}/bin/systemd-nspawn -M ${name} -D ${container.root} --bind-ro=/nix ${container.path}/init";
+          ''
+            # This blocks until the container-startup-done service
+            # writes something to this pipe.  FIXME: it also hangs
+            # until the start timeout expires if systemd-nspawn exits.
+            read x < $root/var/lib/startup-done
+          '';
 
         preStop =
           ''
-            pid="$(cat /sys/fs/cgroup/systemd/machine/${name}.nspawn/system/tasks 2> /dev/null)"
+            pid="$(cat /sys/fs/cgroup/systemd/machine/$INSTANCE.nspawn/system/tasks 2> /dev/null)"
             if [ -n "$pid" ]; then
               # Send the RTMIN+3 signal, which causes the container
               # systemd to start halt.target.
@@ -131,7 +244,52 @@ with pkgs.lib;
               done
             fi
           '';
-      }) config.systemd.containers;
+
+        restartIfChanged = false;
+        #reloadIfChanged = true; # FIXME
+
+        serviceConfig.ExecReload = pkgs.writeScript "reload-container"
+          ''
+            #! ${pkgs.stdenv.shell} -e
+            SYSTEM_PATH=/nix/var/nix/profiles/system
+            if [ -f "/etc/containers/$INSTANCE.conf" ]; then
+              . "/etc/containers/$INSTANCE.conf"
+            fi
+            echo $SYSTEM_PATH/bin/switch-to-configuration test | \
+              ${pkgs.socat}/bin/socat unix:$root/var/lib/run-command.socket -
+          '';
+
+        serviceConfig.SyslogIdentifier = "container %i";
+      };
+
+    # Generate a configuration file in /etc/containers for each
+    # container so that container@.target can get the container
+    # configuration.
+    environment.etc = mapAttrs' (name: cfg: nameValuePair "containers/${name}.conf"
+      { text =
+          ''
+            SYSTEM_PATH=${cfg.path}
+            ${optionalString cfg.privateNetwork ''
+              PRIVATE_NETWORK=1
+              ${optionalString (cfg.hostAddress != null) ''
+                HOST_ADDRESS=${cfg.hostAddress}
+              ''}
+              ${optionalString (cfg.localAddress != null) ''
+                LOCAL_ADDRESS=${cfg.localAddress}
+              ''}
+            ''}
+          '';
+      }) config.containers;
+
+    # FIXME: auto-start containers.
+
+    # Generate /etc/hosts entries for the containers.
+    networking.extraHosts = concatStrings (mapAttrsToList (name: cfg: optionalString (cfg.localAddress != null)
+      ''
+        ${cfg.localAddress} ${name}.containers
+      '') config.containers);
+
+    environment.systemPackages = [ nixos-container ];
 
   };
 }

nixos/modules/virtualisation/nixos-container.pl

@@ -0,0 +1,238 @@
+#! @perl@
+
+use strict;
+use POSIX;
+use File::Path;
+use File::Slurp;
+use Fcntl ':flock';
+use Getopt::Long qw(:config gnu_getopt);
+
+my $socat = '@socat@/bin/socat';
+
+# Parse the command line.
+
+sub showHelp {
+    print <<EOF;
+Usage: nixos-container list
+       nixos-container create <container-name> [--config <string>] [--ensure-unique-name]
+       nixos-container destroy <container-name>
+       nixos-container start <container-name>
+       nixos-container stop <container-name>
+       nixos-container login <container-name>
+       nixos-container root-login <container-name>
+       nixos-container run <container-name> -- args...
+       nixos-container set-root-password <container-name> <password>
+       nixos-container show-ip <container-name>
+EOF
+    exit 0;
+}
+
+my $ensureUniqueName = 0;
+my $extraConfig = "";
+
+GetOptions(
+    "help" => sub { showHelp() },
+    "ensure-unique-name" => \$ensureUniqueName,
+    "config=s" => \$extraConfig
+    ) or exit 1;
+
+my $action = $ARGV[0] or die "$0: no action specified\n";
+
+
+# Execute the selected action.
+
+mkpath("/etc/containers", 0, 0755);
+mkpath("/var/lib/containers", 0, 0700);
+
+if ($action eq "list") {
+    foreach my $confFile (glob "/etc/containers/*.conf") {
+        $confFile =~ /\/([^\/]+).conf$/ or next;
+        print "$1\n";
+    }
+    exit 0;
+}
+
+my $containerName = $ARGV[1] or die "$0: no container name specified\n";
+$containerName =~ /^[a-zA-Z0-9\-]+$/ or die "$0: invalid container name\n";
+
+sub writeNixOSConfig {
+    my ($nixosConfigFile) = @_;
+
+    my $nixosConfig = <<EOF;
+{ config, pkgs, ... }:
+
+with pkgs.lib;
+
+{ boot.isContainer = true;
+  security.initialRootPassword = mkDefault "!";
+  networking.hostName = mkDefault "$containerName";
+  networking.useDHCP = false;
+  $extraConfig
+}
+EOF
+
+    write_file($nixosConfigFile, $nixosConfig);
+}
+
+if ($action eq "create") {
+    # Acquire an exclusive lock to prevent races with other
+    # invocations of ‘nixos-container create’.
+    my $lockFN = "/run/lock/nixos-container";
+    open(my $lock, '>>', $lockFN) or die "$0: opening $lockFN: $!";
+    flock($lock, LOCK_EX) or die "$0: could not lock $lockFN: $!";
+
+    my $confFile = "/etc/containers/$containerName.conf";
+    my $root = "/var/lib/containers/$containerName";
+
+    # Maybe generate a unique name.
+    if ($ensureUniqueName) {
+        my $base = $containerName;
+        for (my $nr = 0; ; $nr++) {
+            $containerName = "$base-$nr";
+            $confFile = "/etc/containers/$containerName.conf";
+            $root = "/var/lib/containers/$containerName";
+            last unless -e $confFile || -e $root;
+        }
+    }
+
+    die "$0: container ‘$containerName’ already exists\n" if -e $confFile;
+
+    # Get an unused IP address.
+    my %usedIPs;
+    foreach my $confFile2 (glob "/etc/containers/*.conf") {
+        my $s = read_file($confFile2) or die;
+        $usedIPs{$1} = 1 if $s =~ /^HOST_ADDRESS=([0-9\.]+)$/m;
+        $usedIPs{$1} = 1 if $s =~ /^LOCAL_ADDRESS=([0-9\.]+)$/m;
+    }
+
+    my ($ipPrefix, $hostAddress, $localAddress);
+    for (my $nr = 1; $nr < 255; $nr++) {
+        $ipPrefix = "10.233.$nr";
+        $hostAddress = "$ipPrefix.1";
+        $localAddress = "$ipPrefix.2";
+        last unless $usedIPs{$hostAddress} || $usedIPs{$localAddress};
+        $ipPrefix = undef;
+    }
+
+    die "$0: out of IP addresses\n" unless defined $ipPrefix;
+
+    my @conf;
+    push @conf, "PRIVATE_NETWORK=1\n";
+    push @conf, "HOST_ADDRESS=$hostAddress\n";
+    push @conf, "LOCAL_ADDRESS=$localAddress\n";
+    write_file($confFile, \@conf);
+
+    close($lock);
+
+    print STDERR "host IP is $hostAddress, container IP is $localAddress\n";
+
+    mkpath("$root/etc/nixos", 0, 0755);
+
+    my $nixosConfigFile = "$root/etc/nixos/configuration.nix";
+    writeNixOSConfig $nixosConfigFile;
+
+    # The per-container directory is restricted to prevent users on
+    # the host from messing with guest users who happen to have the
+    # same uid.
+    my $profileDir = "/nix/var/nix/profiles/per-container";
+    mkpath($profileDir, 0, 0700);
+    $profileDir = "$profileDir/$containerName";
+    mkpath($profileDir, 0, 0755);
+
+    system("nix-env", "-p", "$profileDir/system",
+           "-I", "nixos-config=$nixosConfigFile", "-f", "<nixpkgs/nixos>",
+           "--set", "-A", "system") == 0
+        or die "$0: failed to build initial container configuration\n";
+
+    print "$containerName\n" if $ensureUniqueName;
+    exit 0;
+}
+
+my $root = "/var/lib/containers/$containerName";
+my $profileDir = "/nix/var/nix/profiles/per-container/$containerName";
+my $confFile = "/etc/containers/$containerName.conf";
+die "$0: container ‘$containerName’ does not exist\n" if !-e $confFile;
+
+sub isContainerRunning {
+    my $status = `systemctl show 'container\@$containerName'`;
+    return $status =~ /ActiveState=active/;
+}
+
+sub stopContainer {
+    system("systemctl", "stop", "container\@$containerName") == 0
+        or die "$0: failed to stop container\n";
+}
+
+if ($action eq "destroy") {
+    die "$0: cannot destroy declarative container (remove it from your configuration.nix instead)\n"
+        unless POSIX::access($confFile, &POSIX::W_OK);
+
+    stopContainer if isContainerRunning;
+
+    rmtree($profileDir) if -e $profileDir;
+    rmtree($root) if -e $root;
+    unlink($confFile) or die;
+}
+
+elsif ($action eq "start") {
+    system("systemctl", "start", "container\@$containerName") == 0
+        or die "$0: failed to start container\n";
+}
+
+elsif ($action eq "stop") {
+    stopContainer;
+}
+
+elsif ($action eq "update") {
+    my $nixosConfigFile = "$root/etc/nixos/configuration.nix";
+
+    # FIXME: may want to be more careful about clobbering the existing
+    # configuration.nix.
+    writeNixOSConfig $nixosConfigFile if defined $extraConfig;
+
+    system("nix-env", "-p", "$profileDir/system",
+           "-I", "nixos-config=$nixosConfigFile", "-f", "<nixpkgs/nixos>",
+           "--set", "-A", "system") == 0
+        or die "$0: failed to build container configuration\n";
+
+    if (isContainerRunning) {
+        print STDERR "reloading container...\n";
+        system("systemctl", "reload", "container\@$containerName") == 0
+            or die "$0: failed to reload container\n";
+    }
+}
+
+elsif ($action eq "login") {
+    exec($socat, "unix:$root/var/lib/login.socket", "-,echo=0,raw");
+}
+
+elsif ($action eq "root-login") {
+    exec($socat, "unix:$root/var/lib/root-login.socket", "-,echo=0,raw");
+}
+
+elsif ($action eq "run") {
+    shift @ARGV; shift @ARGV;
+    open(SOCAT, "|-", $socat, "unix:$root/var/lib/run-command.socket", "-");
+    print SOCAT join(' ', map { "'$_'" } @ARGV), "\n";
+    close(SOCAT);
+}
+
+elsif ($action eq "set-root-password") {
+    # FIXME: don't get password from the command line.
+    my $password = $ARGV[2] or die "$0: no password given\n";
+    open(SOCAT, "|-", $socat, "unix:$root/var/lib/run-command.socket", "-");
+    print SOCAT "passwd\n";
+    print SOCAT "$password\n";
+    print SOCAT "$password\n";
+    close(SOCAT);
+}
+
+elsif ($action eq "show-ip") {
+    my $s = read_file($confFile) or die;
+    $s =~ /^LOCAL_ADDRESS=([0-9\.]+)$/m or die "$0: cannot get IP address\n";
+    print "$1\n";
+}
+
+else {
+    die "$0: unknown action ‘$action’\n";
+}

nixos/modules/virtualisation/run-in-netns.c

@@ -0,0 +1,50 @@
+#define _GNU_SOURCE
+
+#include <stdio.h>
+#include <string.h>
+#include <errno.h>
+
+#include <unistd.h>
+#include <sched.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/mount.h>
+#include <fcntl.h>
+#include <linux/limits.h>
+
+int main(int argc, char * * argv)
+{
+    if (argc < 3) {
+        fprintf(stderr, "%s: missing arguments\n", argv[0]);
+        return 1;
+    }
+
+    char nsPath[PATH_MAX];
+
+    sprintf(nsPath, "/run/netns/%s", argv[1]);
+
+    int fd = open(nsPath, O_RDONLY);
+    if (fd == -1) {
+        fprintf(stderr, "%s: opening network namespace: %s\n", argv[0], strerror(errno));
+        return 1;
+    }
+
+    if (setns(fd, CLONE_NEWNET) == -1) {
+        fprintf(stderr, "%s: setting network namespace: %s\n", argv[0], strerror(errno));
+        return 1;
+    }
+
+    umount2(nsPath, MNT_DETACH);
+    if (unlink(nsPath) == -1) {
+        fprintf(stderr, "%s: unlinking network namespace: %s\n", argv[0], strerror(errno));
+        return 1;
+    }
+
+    /* FIXME: Remount /sys so that /sys/class/net reflects the
+       interfaces visible in the network namespace. This requires
+       bind-mounting /sys/fs/cgroups etc. */
+
+    execv(argv[2], argv + 2);
+    fprintf(stderr, "%s: running command: %s\n", argv[0], strerror(errno));
+    return 1;
+}

nixos/tests/containers.nix

@@ -0,0 +1,79 @@
+# Test for NixOS' container support.
+
+{ pkgs, ... }:
+
+{
+
+  machine =
+    { config, pkgs, ... }:
+    { imports = [ ../modules/installer/cd-dvd/channel.nix ];
+      virtualisation.writableStore = true;
+      virtualisation.memorySize = 768;
+
+      containers.webserver =
+        { privateNetwork = true;
+          hostAddress = "10.231.136.1";
+          localAddress = "10.231.136.2";
+          config =
+            { services.httpd.enable = true;
+              services.httpd.adminAddr = "foo@example.org";
+            };
+        };
+
+      virtualisation.pathsInNixDB = [ pkgs.stdenv ];
+    };
+
+  testScript =
+    ''
+      $machine->succeed("nixos-container list") =~ /webserver/;
+
+      # Start the webserver container.
+      $machine->succeed("nixos-container start webserver");
+
+      # Since "start" returns after the container has reached
+      # multi-user.target, we should now be able to access it.
+      my $ip = $machine->succeed("nixos-container show-ip webserver");
+      chomp $ip;
+      $machine->succeed("ping -c1 $ip");
+      $machine->succeed("curl --fail http://$ip/ > /dev/null");
+
+      # Stop the container.
+      $machine->succeed("nixos-container stop webserver");
+      $machine->fail("curl --fail --connect-timeout 2 http://$ip/ > /dev/null");
+
+      # Make sure we have a NixOS tree (required by ‘nixos-container create’).
+      $machine->succeed("nix-env -qa -A nixos.pkgs.hello >&2");
+
+      # Create some containers imperatively.
+      my $id1 = $machine->succeed("nixos-container create foo --ensure-unique-name");
+      chomp $id1;
+      $machine->log("created container $id1");
+
+      my $id2 = $machine->succeed("nixos-container create foo --ensure-unique-name");
+      chomp $id2;
+      $machine->log("created container $id2");
+
+      die if $id1 eq $id2;
+
+      my $ip1 = $machine->succeed("nixos-container show-ip $id1");
+      chomp $ip1;
+      my $ip2 = $machine->succeed("nixos-container show-ip $id2");
+      chomp $ip2;
+      die if $ip1 eq $ip2;
+
+      # Start one of them.
+      $machine->succeed("nixos-container start $id1");
+
+      # Execute commands via the root shell.
+      $machine->succeed("echo uname | nixos-container root-shell $id1") =~ /Linux/;
+      $machine->succeed("nixos-container set-root-password $id1 foobar");
+
+      # Destroy the containers.
+      $machine->succeed("nixos-container destroy $id1");
+      $machine->succeed("nixos-container destroy $id2");
+
+      # Destroying a declarative container should fail.
+      $machine->fail("nixos-container destroy webserver");
+    '';
+
+}

nixos/tests/default.nix

@@ -8,6 +8,7 @@ with import ../lib/testing.nix { inherit system minimal; };
 {
   avahi = makeTest (import ./avahi.nix);
   bittorrent = makeTest (import ./bittorrent.nix);
+  containers = makeTest (import ./containers.nix);
   firefox = makeTest (import ./firefox.nix);
   firewall = makeTest (import ./firewall.nix);
   installer = makeTests (import ./installer.nix);