Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

doc/meta: Mention --version as a good usecase for installCheckPhase #315616

Merged
merged 5 commits into from
Jun 7, 2024
Merged

doc/meta: Mention --version as a good usecase for installCheckPhase #315616

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
2093ef2
doc/meta: mention how Hydra and other tools don't run passthru.tests
doronbehar May 29, 2024
f4e7ce7
doc/meta: small link target shortenning
doronbehar May 29, 2024
f4e6f41
doc/meta: better explain an advantage or passthru.tests
doronbehar May 29, 2024
444c2b6
doc/meta: Mention --version as a good usecase for installCheckPhase
doronbehar May 29, 2024
0aa9043
doc/meta: still mention testVersion near the installCheckPhase recomm…
doronbehar May 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 28 additions & 4 deletions doc/stdenv/meta.chapter.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,17 +125,41 @@ $ cd path/to/nixpkgs
$ nix-build -A your-package.tests
```

Note that Hydra and [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) don't build these derivations by default, and that ([`@ofborg`](https://github.com/NixOS/ofborg)) only builds them when evaluating PRs for that particular package (or when manually instructed).

#### Package tests {#var-meta-tests-packages}

Tests that are part of the source package are often executed in the `installCheckPhase`.
Tests that are part of the source package are often executed in the `installCheckPhase`. This phase is also suitable for performing a `--version` test for packages that support such flag. Here's an example:

```nix
# Say the package is git
stdenv.mkDerivation(finalAttrs: {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this style seems less common than mkDerivation rec {, is there a reason we'd want to show finalAttrs here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this style seems less common than mkDerivation rec {, is there a reason we'd want to show finalAttrs here?

using finalAttrs is the modern way to mkDerivation nowadays, as it makes overriding the package much easier when the package references it self, and especially if you want to make it easy for the user reading this to maybe add a passthru.tests.comprehensive that will use finalAttrs.finalPackage which is not possible with the old mkDerivation rec { pattern.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using finalAttrs is the modern way to mkDerivation nowadays

Do you have a reference for that? https://nixos.org/manual/nixpkgs/unstable/#sec-using-stdenv is still showing rec

it makes overriding the package much easier when the package references it self, and especially if you want to make it easy for the user reading this to maybe add a passthru.tests.comprehensive that will use finalAttrs.finalPackage which is not possible with the old mkDerivation rec { pattern

TIL, very cool

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a reference for that? nixos.org/manual/nixpkgs/unstable/#sec-using-stdenv is still showing rec

I assumed that TBH :) The current docs for that don't mention it as a superior way to use mkDerivation, but personally I think it is. As for the docs at hand though, I still think it is better to suggest using a function, as it is especially relevant for passthru.tests when you want to use finalAttrs.finalPackage.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more ongoing discussion on this in #315337

pname = "git";
version = "...";
# ...

doInstallCheck = true;
installCheckPhase = ''
doronbehar marked this conversation as resolved.
Show resolved Hide resolved
runHook preInstallCheck

Prefer `passthru.tests` for tests that are introduced in nixpkgs because:
echo checking if 'git --version' mentions ${finalAttrs.version}
$out/bin/git --version | grep ${finalAttrs.version}
Comment on lines +145 to +146
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be great if this could simply be a hook which would be configured using a similar interface as the tester.

Copy link
Member

@Artturin Artturin Jun 15, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't show the whole output on failure, nor is it separated from the build environment, therefore it is inferior to testVersion, a hook which contains the testVersion code and clears the env and switches the dir to an empty directory will be needed.


runHook postInstallCheck
'';
# ...
})
```

Most programs distributed by Nixpkgs support such a `--version` flag, and it can help give confidence that the package at least got compiled properly. However, tests that are slightly non trivial will better fit into `passthru.tests`, because:

* `passthru.tests` tests the 'real' package, independently from the environment in which it was built
* we can run `passthru.tests` independently
* We can run and debug a `passthru.tests` independently, after the package was built (useful if it takes a long time).
* `installCheckPhase` adds overhead to each build

For more on how to write and run package tests, see [](#sec-package-tests).
It is also possible to still use `passthru.tests` to test the version, with [testVersion](#tester-testVersion).

For more on how to write and run package tests, see [`pkgs/README.md`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests).

#### NixOS tests {#var-meta-tests-nixos}

Expand Down