doc: add stdenv passthru chapter #315909

abathur · 2024-05-30T14:28:24Z

Reviewer on another PR queried about documenting a passthru attribute it adds, and looking into it led me to see how thin and scattered existing passthru docs are.

I'm hoping to keep this fairly narrow (reworking the skeleton and moving existing coverage) to focus review on what I see as the big/fundamental questions here:

whether we should have such a section
whether that section should list common passthru attrs
what razors we should apply to decide when a given passthru attr deserves mention here as opposed to a more-specific part of the manual, an in-repo README, code comments, etc.

@NixOS/documentation-team and @infinisil (given ~ownership of repo-internal READMEs, where a large fraction of the more in-depth passthru.tests coverage was moved last year).

Add a 👍 reaction to pull requests you find important.

abathur · 2024-05-30T14:30:37Z

doc/stdenv/passthru.chapter.md

+- These values are not passed to the builder, so you can change them without triggering a rebuild.
+
+One example of the passthru's power is how it enables us to associate the package for an interpreter Python or Ruby with a set of modules built for that specific interpreter (e.g., `python.pkgs` and `ruby.gems`) and to build runtime environments with a collection of those modules (e.g., (`python.withPackages`)[#python.withpackages-function] and `ruby.withPackages`).
+<!-- python.pkgs, ruby.gems, and ruby.withPackages are mentioned but don't have dedicated sections to link -->


Comment here is just a reminder for why I only linked one of these :)

Is this trying to say that python.pkgs and python.withPackages are defined in python.passthru? If so it should be explicit.

@kjeremy Can you clarify how that could be more explicit? (Perhaps in the form of a rephrasing? I feel like the text is already explicitly stating that the passthru is where these are defined.)

abathur · 2024-05-30T14:32:54Z

doc/stdenv/passthru.chapter.md

+<!-- 
+I'm extracting this statement from the old text because I feel like it isn't very clearly-phrased and doesn't help connect with something most people know already.
+I still think we should try to make sure the pattern it models is represented in the list of common attributes, though:
+
+convey a specific dependency of your derivation which contains a program with plugins support. 
+Later, others who make derivations with plugins can use passed-through dependency to ensure that their plugin would be binary-compatible with built program. 
+-->


Still undecided on this. Roughly a TODO, but I didn't want to go off searching for a real example of this and writing it up if all of this is off-base in the first place.

abathur · 2024-05-30T14:35:17Z

doc/stdenv/passthru.chapter.md

+Many `passthru` attributes are situational, so this section only attempts to list recurring attributes and patterns. 
+Very broadly, passthru attributes listed here fall in one of these categories:
+<!-- 
+Very broadly, an attribute is worth documenting here if it fits either description below.
+Attributes that apply only to a single builder or language ecosystem are best documented alongside the builder or related docs.
+-->
+
+- Attributes that are (at least in theory) globally relevant--you could find them on any derivation in nixpkgs (generally because they don't entail any special support be built into the derivation they're added to). 
+  Common examples of this type are `passthru.tests` and `passthru.updateScript`.
+- Common patterns for adding extra functionality to a derivation (which do tend to entail building support into the derivation or its passthru). 
+  Common examples of this type are `passthru.optional-dependencies`, `passthru.withPlugins`, and `passthru.withPackages`.


I've waffled a little about whether the razor is something the text should try to qualify, or just something we can leave in a comment.

I did a bit of both to force that discussion :)

@fricklerhandwerk Could use guidance on this implementation of the razor and whether you think it belongs in the primary text or in a comment.

I think the general structure and semantics are fine. I'd do another pass to simplify the language and make it a bit less verbose when you're happy with the current state, but other than that it's a great addition and clarification overall. (A strong indicator of that is that I learned something that was not fully obvious to me beforehand...)

AndersonTorres · 2024-05-30T14:36:58Z

This text contains too much excitement for a technical document. It looks like an anime protagonist screaming すごい after each minute.

pkgs/development/mobile/androidenv/README.md

fricklerhandwerk · 2024-05-31T14:13:06Z

whether we should have such a section

Absolutely

whether that section should list common passthru attrs

Yes

what razors we should apply to decide when a given passthru attr deserves mention here as opposed to a more-specific part of the manual, an in-repo README, code comments, etc.

If it's commonly used, it should be in the common section. Otherwise the specific build helpers should document their passthrus and refer back to the common section.

Broad strokes: - create the chapter - move existing stdenv passthru coverage into it - move out-of-place coverage of passthru.tests from the stdenv meta chapter into it - (try to) apply 1-sentence-per-line to text I've touched - add legacy anchors for everything moved - update existing links to the new anchors - add tentative motivating text

abathur · 2024-06-07T15:24:41Z

Force-push to incorporate changes that landed a few hours ago via:

doc/meta: Mention --version as a good usecase for installCheckPhase #315616

I did reline the changes there and make some small phrasing/consistency edits.

AndersonTorres · 2024-06-07T16:22:11Z

What is needed to merge this?

fricklerhandwerk · 2024-06-07T16:28:13Z

I will properly review and merge on Monday or Tuesday.

infinisil · 2024-06-07T20:14:09Z

doc/stdenv/passthru.chapter.md

+Later, others who make derivations with plugins can use passed-through dependency to ensure that their plugin would be binary-compatible with built program.
+-->
+
+## Common passthru-attributes {#sec-common-passthru-attributes}


With the new convention, I think this entire section should go into pkgs/README.md instead, because it's only useful to Nixpkgs contributors. Though one could also argue that the attributes are exposed to users, so they could use it, but that's really not the intention behind these.

And it seems like this would combine well with https://github.com/NixOS/nixpkgs/tree/master/pkgs#package-tests. I never got around to doing this in the original PR, but it would be great to continue going that route!

Hm, I see what you mean but I'm not convinced that passthru attributes are purely contributor-facing. As you say those are part of the external interface and I definitely ran into them as a consumer. There may be something to say about how to work with them on the contributor side, but that's a different issue for another time.

This PR is roughly intended to force this sort of decision. (My primary goal here is that we find clarity on whether and to what extent the passthru should be documented in the manual and/or repo-internal docs and end up with a razor sharp enough to look at a given attr and decide whether/where to document it).

As long as we leave with that razor in hand, I won't be put out if the result is that we close this PR and do something else. (I'm not sure if it will make sense to have a full passthru chapter if, in practice, we think no or very few attrs would be appropriately documented there instead of in-repo.)

CC: @roberth, since this PR results from discussion with him.

Make nixpkgs-internal links relative/branchless

razor: if it is only ever needed by contributors, which is likely if links refer to the latest revision of the source code, then it's for the contributor guide

fricklerhandwerk · 2024-06-11T08:48:43Z

@abathur here's a sharper razor: if it is only ever needed by contributors, which is likely if links refer to the latest revision of the source code, then it's for the contributor guide. if examples won't go stale if the underlying code changes or moves, it's for the user's manual.

doc/stdenv/passthru.chapter.md

fricklerhandwerk · 2024-06-07T16:23:06Z

doc/stdenv/passthru.chapter.md

+Many `passthru` attributes are situational, so this section only attempts to list recurring attributes and patterns. 
+Very broadly, passthru attributes listed here fall in one of these categories:
+<!-- 
+Very broadly, an attribute is worth documenting here if it fits either description below.
+Attributes that apply only to a single builder or language ecosystem are best documented alongside the builder or related docs.
+-->
+
+- Attributes that are (at least in theory) globally relevant--you could find them on any derivation in nixpkgs (generally because they don't entail any special support be built into the derivation they're added to). 
+  Common examples of this type are `passthru.tests` and `passthru.updateScript`.
+- Common patterns for adding extra functionality to a derivation (which do tend to entail building support into the derivation or its passthru). 
+  Common examples of this type are `passthru.optional-dependencies`, `passthru.withPlugins`, and `passthru.withPackages`.


I think the general structure and semantics are fine. I'd do another pass to simplify the language and make it a bit less verbose when you're happy with the current state, but other than that it's a great addition and clarification overall. (A strong indicator of that is that I learned something that was not fully obvious to me beforehand...)

fricklerhandwerk · 2024-06-07T20:47:13Z

doc/stdenv/passthru.chapter.md

+Later, others who make derivations with plugins can use passed-through dependency to ensure that their plugin would be binary-compatible with built program.
+-->
+
+## Common passthru-attributes {#sec-common-passthru-attributes}


Hm, I see what you mean but I'm not convinced that passthru attributes are purely contributor-facing. As you say those are part of the external interface and I definitely ran into them as a consumer. There may be something to say about how to work with them on the contributor side, but that's a different issue for another time.

doc/build-helpers/images/dockertools.section.md

doc/build-helpers/fetchers.chapter.md

nixos-discourse · 2024-06-18T08:42:05Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-06-14-documentation-team-meeting-notes-132/47182/1

abathur requested a review from roberth as a code owner May 30, 2024 14:28

github-actions bot added the 8.has: documentation label May 30, 2024

abathur commented May 30, 2024

View reviewed changes

abathur force-pushed the manual_passthru_chapter branch from 3eecb94 to 1a13301 Compare May 30, 2024 14:37

ofborg bot added 10.rebuild-darwin: 0 10.rebuild-linux: 0 labels May 30, 2024

hsjobeki reviewed May 31, 2024

View reviewed changes

pkgs/development/mobile/androidenv/README.md Outdated Show resolved Hide resolved

abathur requested review from infinisil and fricklerhandwerk May 31, 2024 13:46

abathur force-pushed the manual_passthru_chapter branch from 1a13301 to e2698d7 Compare June 7, 2024 15:18

infinisil reviewed Jun 7, 2024

View reviewed changes

abathur and others added 2 commits June 7, 2024 16:48

Update pkgs/development/mobile/androidenv/README.md

549a7c7

Make nixpkgs-internal links relative/branchless

review pass, move details on auto-updates to contributor guide

03aaabd

razor: if it is only ever needed by contributors, which is likely if links refer to the latest revision of the source code, then it's for the contributor guide

fricklerhandwerk approved these changes Jun 11, 2024

View reviewed changes

more backticks

ff28f01

fricklerhandwerk merged commit 9ff9bbd into NixOS:master Jun 11, 2024
7 of 8 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

doc: add stdenv passthru chapter #315909

doc: add stdenv passthru chapter #315909

abathur commented May 30, 2024

abathur May 30, 2024

kjeremy May 31, 2024

abathur May 31, 2024

abathur May 30, 2024

abathur May 30, 2024

abathur Jun 7, 2024

fricklerhandwerk Jun 7, 2024

AndersonTorres commented May 30, 2024

fricklerhandwerk commented May 31, 2024

abathur commented Jun 7, 2024 •

edited

Loading

AndersonTorres commented Jun 7, 2024

fricklerhandwerk commented Jun 7, 2024

infinisil Jun 7, 2024

fricklerhandwerk Jun 7, 2024

abathur Jun 7, 2024 •

edited

Loading

fricklerhandwerk commented Jun 11, 2024

fricklerhandwerk Jun 7, 2024

fricklerhandwerk Jun 7, 2024

nixos-discourse commented Jun 18, 2024

doc: add stdenv passthru chapter #315909

doc: add stdenv passthru chapter #315909

Conversation

abathur commented May 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AndersonTorres commented May 30, 2024

fricklerhandwerk commented May 31, 2024

abathur commented Jun 7, 2024 • edited Loading

AndersonTorres commented Jun 7, 2024

fricklerhandwerk commented Jun 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abathur Jun 7, 2024 • edited Loading

Choose a reason for hiding this comment

fricklerhandwerk commented Jun 11, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nixos-discourse commented Jun 18, 2024

abathur commented Jun 7, 2024 •

edited

Loading

abathur Jun 7, 2024 •

edited

Loading