Merge branch 'master' into improved-make-overridable

.github/CODEOWNERS

@@ -7,17 +7,43 @@
 # For documentation on this file, see https://help.github.com/articles/about-codeowners/
 # Mentioned users will get code review requests.
 
-# Python-related code and docs
-pkgs/top-level/python-packages.nix	@FRidh
-pkgs/development/interpreters/python/*	@FRidh
-pkgs/development/python-modules/*	@FRidh
-doc/languages-frameworks/python.md	@FRidh
+# This file
+.github/CODEOWNERS @edolstra
 
 # Boostraping and core infra
-pkgs/stdenv/	@Ericson2314
-pkgs/build-support/cc-wrapper/	@Ericson2314
+pkgs/stdenv/    @edolstra
+pkgs/build-support/cc-wrapper/  @edolstra
+
+# Libraries
+lib/  @edolstra
+
+# Python-related code and docs
+pkgs/top-level/python-packages.nix      @FRidh
+pkgs/development/interpreters/python/*  @FRidh
+pkgs/development/python-modules/*       @FRidh
+doc/languages-frameworks/python.md      @FRidh
+
+# Haskell
+pkgs/development/compilers/ghc                       @peti
+pkgs/development/haskell-modules                     @peti
+pkgs/development/haskell-modules/default.nix         @Profpatsch @peti
+pkgs/development/haskell-modules/generic-builder.nix @Profpatsch @peti
+pkgs/development/haskell-modules/hoogle.nix          @Profpatsch @peti
+
+# R
+pkgs/applications/science/math/R @peti
+pkgs/development/r-modules       @peti
 
 # Darwin-related
-pkgs/stdenv/darwin/*	@copumpkin @LnL7
-pkgs/os-specific/darwin/*	@LnL7
-pkgs/os-specific/darwin/apple-source-releases/*	@copumpkin
+pkgs/stdenv/darwin/*    @copumpkin @LnL7
+pkgs/os-specific/darwin/*       @LnL7
+pkgs/os-specific/darwin/apple-source-releases/* @copumpkin
+
+# Beam-related (Erlang, Elixir, LFE, etc)
+pkgs/development/beam-modules/*	@gleber
+pkgs/development/interpreters/erlang/*	@gleber
+pkgs/development/interpreters/lfe/*	@gleber
+pkgs/development/interpreters/elixir/*	@gleber
+pkgs/development/tools/build-managers/rebar/*	@gleber
+pkgs/development/tools/build-managers/rebar3/*	@gleber
+pkgs/development/tools/erlang/*	@gleber

.github/CONTRIBUTING.md

@@ -23,7 +23,7 @@ under the terms of [COPYING](../COPYING), which is an MIT-like license.
   Examples:
 
   * nginx: init at 2.0.1
-  * firefox: 3.0 -> 3.1.1
+  * firefox: 54.0.1 -> 55.0
   * nixos/hydra: add bazBaz option
 
     Dual baz behavior is needed to do foo.

.github/PULL_REQUEST_TEMPLATE.md

@@ -5,10 +5,7 @@
 
 <!-- Please check what applies. Note that these are not hard requirements but merely serve as information for reviewers. -->
 
-- [ ] Tested using sandboxing
-  ([nix.useSandbox](http://nixos.org/nixos/manual/options.html#opt-nix.useSandbox) on NixOS,
-    or option `build-use-sandbox` in [`nix.conf`](http://nixos.org/nix/manual/#sec-conf-file)
-    on non-NixOS)
+- [ ] Tested using sandboxing ([nix.useSandbox](http://nixos.org/nixos/manual/options.html#opt-nix.useSandbox) on NixOS, or option `build-use-sandbox` in [`nix.conf`](http://nixos.org/nix/manual/#sec-conf-file) on non-NixOS)
 - Built on platform(s)
    - [ ] NixOS
    - [ ] macOS

.version

@@ -1 +1 @@
-17.09
+18.03

doc/coding-conventions.xml

@@ -254,7 +254,7 @@ bound to the variable name <varname>e2fsprogs</varname> in
   dash) — e.g., <literal>"hello-0.3.1rc2"</literal>.</para></listitem>
 
   <listitem><para>If a package is not a release but a commit from a repository, then
-  the version part of the name <emphasis>must</emphasis> be the date of that 
+  the version part of the name <emphasis>must</emphasis> be the date of that
   (fetched) commit. The date must be in <literal>"YYYY-MM-DD"</literal> format.
   Also append <literal>"unstable"</literal> to the name - e.g.,
   <literal>"pkgname-unstable-2014-09-23"</literal>.</para></listitem>
@@ -365,7 +365,7 @@ splitting up an existing category.</para>
   <varlistentry>
     <term>If it’s a (set of) <emphasis>tool(s)</emphasis>:</term>
     <listitem>
-      <para>(A tool is a relatively small program, especially one intented
+      <para>(A tool is a relatively small program, especially one intended
       to be used non-interactively.)</para>
       <variablelist>
         <varlistentry>
@@ -456,7 +456,7 @@ splitting up an existing category.</para>
   <varlistentry>
     <term>If it’s a <emphasis>window manager</emphasis>:</term>
     <listitem>
-      <para><filename>applications/window-managers</filename> (e.g. <filename>awesome</filename>, <filename>compiz</filename>, <filename>stumpwm</filename>)</para>
+      <para><filename>applications/window-managers</filename> (e.g. <filename>awesome</filename>, <filename>stumpwm</filename>)</para>
     </listitem>
   </varlistentry>
   <varlistentry>
@@ -608,7 +608,7 @@ evaluate correctly.</para>
 </section>
 <section xml:id="sec-sources"><title>Fetching Sources</title>
   <para>There are multiple ways to fetch a package source in nixpkgs. The
-    general guidline is that you should package sources with a high degree of
+    general guideline is that you should package sources with a high degree of
     availability. Right now there is only one fetcher which has mirroring
     support and that is <literal>fetchurl</literal>. Note that you should also
     prefer protocols which have a corresponding proxy environment variable.
@@ -661,9 +661,9 @@ src = fetchFromGitHub {
 </section>
 
 <section xml:id="sec-patches"><title>Patches</title>
-  <para>Only patches that are unique to <literal>nixpkgs</literal> should be 
+  <para>Only patches that are unique to <literal>nixpkgs</literal> should be
     included in <literal>nixpkgs</literal> source.</para>
-  <para>Patches available online should be retrieved using 
+  <para>Patches available online should be retrieved using
     <literal>fetchpatch</literal>.</para>
   <para>
 <programlisting>

doc/languages-frameworks/haskell.md

@@ -867,6 +867,67 @@ use the following to get the `scientific` package build with `integer-simple`:
 nix-build -A haskell.packages.integer-simple.ghc802.scientific
 ```
 
+### Quality assurance
+
+The `haskell.lib` library includes a number of functions for checking for
+various imperfections in Haskell packages. It's useful to apply these functions
+to your own Haskell packages and integrate that in a Continuous Integration
+server like [hydra](https://nixos.org/hydra/) to assure your packages maintain a
+minimum level of quality. This section discusses some of these functions.
+
+#### failOnAllWarnings
+
+Applying `haskell.lib.failOnAllWarnings` to a Haskell package enables the
+`-Wall` and `-Werror` GHC options to turn all warnings into build failures.
+
+#### buildStrictly
+
+Applying `haskell.lib.buildStrictly` to a Haskell package calls
+`failOnAllWarnings` on the given package to turn all warnings into build
+failures. Additionally the source of your package is gotten from first invoking
+`cabal sdist` to ensure all needed files are listed in the Cabal file.
+
+#### checkUnusedPackages
+
+Applying `haskell.lib.checkUnusedPackages` to a Haskell package invokes
+the [packunused](http://hackage.haskell.org/package/packunused) tool on the
+package. `packunused` complains when it finds packages listed as build-depends
+in the Cabal file which are redundant. For example:
+
+```
+$ nix-build -E 'let pkgs = import <nixpkgs> {}; in pkgs.haskell.lib.checkUnusedPackages {} pkgs.haskellPackages.scientific'
+these derivations will be built:
+  /nix/store/3lc51cxj2j57y3zfpq5i69qbzjpvyci1-scientific-0.3.5.1.drv
+...
+detected package components
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ - library
+ - testsuite(s): test-scientific
+ - benchmark(s): bench-scientific*
+
+(component names suffixed with '*' are not configured to be built)
+
+library
+~~~~~~~
+
+The following package dependencies seem redundant:
+
+ - ghc-prim-0.5.0.0
+
+testsuite(test-scientific)
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+no redundant packages dependencies found
+
+builder for ‘/nix/store/3lc51cxj2j57y3zfpq5i69qbzjpvyci1-scientific-0.3.5.1.drv’ failed with exit code 1
+error: build of ‘/nix/store/3lc51cxj2j57y3zfpq5i69qbzjpvyci1-scientific-0.3.5.1.drv’ failed
+```
+
+As you can see, `packunused` finds out that although the testsuite component has
+no redundant dependencies the library component of `scientific-0.3.5.1` depends
+on `ghc-prim` which is unused in the library.
+
 ## Other resources
 
   - The Youtube video [Nix Loves Haskell](https://www.youtube.com/watch?v=BsBhi_r-OeE)

doc/languages-frameworks/python.md

@@ -590,7 +590,7 @@ By default tests are run because `doCheck = true`. Test dependencies, like
 e.g. the test runner, should be added to `buildInputs`.
 
 By default `meta.platforms` is set to the same value
-as the interpreter unless overriden otherwise.
+as the interpreter unless overridden otherwise.
 
 ##### `buildPythonPackage` parameters
 
@@ -774,6 +774,21 @@ The `buildPythonPackage` function sets `DETERMINISTIC_BUILD=1` and
 Both are also exported in `nix-shell`.
 
 
+### Automatic tests
+
+It is recommended to test packages as part of the build process.
+Source distributions (`sdist`) often include test files, but not always.
+
+By default the command `python setup.py test` is run as part of the
+`checkPhase`, but often it is necessary to pass a custom `checkPhase`. An
+example of such a situation is when `py.test` is used.
+
+#### Common issues
+
+- Non-working tests can often be deselected. In the case of `py.test`: `py.test -k 'not function_name and not other_function'`.
+- Unicode issues can typically be fixed by including `glibcLocales` in `buildInputs` and exporting `LC_ALL=en_US.utf-8`.
+- Tests that attempt to access `$HOME` can be fixed by using the following work-around before running tests (e.g. `preCheck`): `export HOME=$(mktemp -d)`
+
 ## FAQ
 
 ### How to solve circular dependencies?
@@ -985,8 +1000,9 @@ rec {
 
 Following rules are desired to be respected:
 
-* Python libraries are supposed to be called from `python-packages.nix` and packaged with `buildPythonPackage`. The expression of a library should be in `pkgs/development/python-modules/<name>/default.nix`. Libraries in `pkgs/top-level/python-packages.nix` are sorted quasi-alphabetically to avoid merge conflicts.
+* Python libraries are called from `python-packages.nix` and packaged with `buildPythonPackage`. The expression of a library should be in `pkgs/development/python-modules/<name>/default.nix`. Libraries in `pkgs/top-level/python-packages.nix` are sorted quasi-alphabetically to avoid merge conflicts.
 * Python applications live outside of `python-packages.nix` and are packaged with `buildPythonApplication`.
 * Make sure libraries build for all Python interpreters.
 * By default we enable tests. Make sure the tests are found and, in the case of libraries, are passing for all interpreters. If certain tests fail they can be disabled individually. Try to avoid disabling the tests altogether. In any case, when you disable tests, leave a comment explaining why.
-* Commit names of Python libraries should include `pythonPackages`, for example `pythonPackages.numpy: 1.11 -> 1.12`.
+* Commit names of Python libraries should reflect that they are Python libraries, so write for example `pythonPackages.numpy: 1.11 -> 1.12`.
+

doc/languages-frameworks/rust.md

@@ -17,7 +17,7 @@ into the `environment.systemPackages` or bring them into scope with
 `nix-shell -p rustStable.rustc -p rustStable.cargo`.
 
 There are also `rustBeta` and `rustNightly` package sets available.
-These are not updated very regulary. For daily builds use either rustup from
+These are not updated very regularly. For daily builds use either rustup from
 nixpkgs or use the [Rust nightlies overlay](#using-the-rust-nightlies-overlay).
 
 ## Packaging Rust applications

doc/package-notes.xml

@@ -101,7 +101,7 @@ modulesTree = [kernel]
 $ nix-env -i ncurses
 $ export NIX_CFLAGS_LINK=-lncurses
 $ make menuconfig ARCH=<replaceable>arch</replaceable></screen>
-          
+
           </para>
         </listitem>
 
@@ -111,9 +111,9 @@ $ make menuconfig ARCH=<replaceable>arch</replaceable></screen>
         </listitem>
 
       </orderedlist>
-    
+
     </para>
-    
+
   </listitem>
 
   <listitem>
@@ -409,24 +409,24 @@ it. Place the resulting <filename>package.nix</filename> file into
 <title>Steam in Nix</title>
 
 <para>
-  Steam is distributed as a <filename>.deb</filename> file, for now only 
-  as an i686 package (the amd64 package only has documentation). 
-  When unpacked, it has a script called <filename>steam</filename> that 
+  Steam is distributed as a <filename>.deb</filename> file, for now only
+  as an i686 package (the amd64 package only has documentation).
+  When unpacked, it has a script called <filename>steam</filename> that
   in ubuntu (their target distro) would go to <filename>/usr/bin
-  </filename>. When run for the first time, this script copies some 
-  files to the user's home, which include another script that is the 
-  ultimate responsible for launching the steam binary, which is also 
+  </filename>. When run for the first time, this script copies some
+  files to the user's home, which include another script that is the
+  ultimate responsible for launching the steam binary, which is also
   in $HOME.
 </para>
 <para>
   Nix problems and constraints:
 <itemizedlist>
-  <listitem><para>We don't have <filename>/bin/bash</filename> and many 
+  <listitem><para>We don't have <filename>/bin/bash</filename> and many
   scripts point there. Similarly for <filename>/usr/bin/python</filename>
   .</para></listitem>
   <listitem><para>We don't have the dynamic loader in <filename>/lib
   </filename>.</para></listitem>
-  <listitem><para>The <filename>steam.sh</filename> script in $HOME can 
+  <listitem><para>The <filename>steam.sh</filename> script in $HOME can
   not be patched, as it is checked and rewritten by steam.</para></listitem>
   <listitem><para>The steam binary cannot be patched, it's also checked.</para></listitem>
 </itemizedlist>
@@ -446,10 +446,10 @@ it. Place the resulting <filename>package.nix</filename> file into
 <title>How to play</title>
 
 <para>
-  For 64-bit systems it's important to have 
-  <programlisting>hardware.opengl.driSupport32Bit = true;</programlisting> 
-  in your <filename>/etc/nixos/configuration.nix</filename>. You'll also need 
-  <programlisting>hardware.pulseaudio.support32Bit = true;</programlisting> 
+  For 64-bit systems it's important to have
+  <programlisting>hardware.opengl.driSupport32Bit = true;</programlisting>
+  in your <filename>/etc/nixos/configuration.nix</filename>. You'll also need
+  <programlisting>hardware.pulseaudio.support32Bit = true;</programlisting>
   if you are using PulseAudio - this will enable 32bit ALSA apps integration.
   To use the Steam controller, you need to add
   <programlisting>services.udev.extraRules = ''
@@ -470,23 +470,31 @@ it. Place the resulting <filename>package.nix</filename> file into
 
   <varlistentry>
     <term>Steam fails to start. What do I do?</term>
-    <listitem><para>Try to run 
+    <listitem><para>Try to run
     <programlisting>strace steam</programlisting>
     to see what is causing steam to fail.</para></listitem>
   </varlistentry>
 
   <varlistentry>
-  <term>Using the FOSS Radeon drivers</term>
+  <term>Using the FOSS Radeon or nouveau (nvidia) drivers</term>
   <listitem><itemizedlist><listitem><para>
-  The open source radeon drivers need a newer libc++ than is provided 
-  by the default runtime, which leads to a crash on launch. Use
-  <programlisting>environment.systemPackages = [(pkgs.steam.override { newStdcpp = true; })];</programlisting>
-  in your config if you get an error like
+  Both the open source radeon drivers as well as the nouveau drivers (nvidia)
+  need a newer libc++ than is provided by the default runtime, which leads to a
+  crash on launch. Use <programlisting>environment.systemPackages =
+  [(pkgs.steam.override { newStdcpp = true; })];</programlisting> in your config
+  if you get an error like
   <programlisting>
 libGL error: unable to load driver: radeonsi_dri.so
 libGL error: driver pointer missing
 libGL error: failed to load driver: radeonsi
 libGL error: unable to load driver: swrast_dri.so
+libGL error: failed to load driver: swrast</programlisting>
+  or
+  <programlisting>
+libGL error: unable to load driver: nouveau_dri.so
+libGL error: driver pointer missing
+libGL error: failed to load driver: nouveau
+libGL error: unable to load driver: swrast_dri.so
 libGL error: failed to load driver: swrast</programlisting></para></listitem>
   <listitem><para>
   Steam ships statically linked with a version of libcrypto that
@@ -504,7 +512,7 @@ libGL error: failed to load driver: swrast</programlisting></para></listitem>
   <listitem><para>
   There is no java in steam chrootenv by default. If you get a message like
   <programlisting>/home/foo/.local/share/Steam/SteamApps/common/towns/towns.sh: line 1: java: command not found</programlisting>
-  You need to add  
+  You need to add
   <programlisting> steam.override { withJava = true; };</programlisting>
   to your configuration.
   </para></listitem>
@@ -519,14 +527,14 @@ libGL error: failed to load driver: swrast</programlisting></para></listitem>
 
 <title>steam-run</title>
 <para>
-The FHS-compatible chroot used for steam can also be used to run 
+The FHS-compatible chroot used for steam can also be used to run
 other linux games that expect a FHS environment.
-To do it, add 
+To do it, add
 <programlisting>pkgs.(steam.override {
           nativeOnly = true;
           newStdcpp = true;
         }).run</programlisting>
-to your configuration, rebuild, and run the game with 
+to your configuration, rebuild, and run the game with
 <programlisting>steam-run ./foo</programlisting>
 </para>