[RFC 0032] Phase running changes for better nix-shell use #32

dezgeg · 2018-08-29T23:24:59Z

Making some minor tweaks to how phases are run in the stdenv to improve the UX of nix-shell.

Rendered view: https://github.com/dezgeg/rfcs/blob/phase-changes/rfcs/0032-run-phase-changes-for-better-nix-shell-use.md

samueldr · 2018-08-30T00:15:32Z

given that the issue being fixed is a pure UX issue, not fixing it wouldn't prevent any other work from happening.

Good UX is important in everything. Anything that is clumsy will be fumbled with, and possibly be hated by the end-user, that's not a Nix-specific statement! Don't sell the desire for improvement short.

Is there anything preventing your runPhase alternative to be implemented right now? If it were implemented, (I think) it could be also made compatible with the implemented RFC. This would already be a net increase in UX over running eval .... The only drawback I see is that using runPhase would clobber the runPhase name if one would implement a runPhase as a phase.

As for the RFC itself, other than the generic baseline deprecation concerns, the idea seems sound enough. In reality the "new way" isn't that much of a hassle compared to the status quo. This mostly affects the implementation for a default phase; the user overriding a phase won't need to bother, right?.

As a new~ish user and developer running hooks also makes more sense than clobbering the pre parts by setting a custom phase. That is, in addition to making it easier to run phases properly.

dezgeg · 2018-08-30T09:48:59Z

Is there anything preventing your runPhase alternative to be implemented right now?

No, it could be added straight away. But I am kind of hopeful it won't be needed.

In reality the "new way" isn't that much of a hassle compared to the status quo. This mostly affects the implementation for a default phase; the user overriding a phase won't need to bother, right?.

Correct. You can see the changes needed in this commit: dezgeg/nixpkgs@87959f3

timokau · 2018-08-30T14:33:22Z

Good idea! I was annoyed by this multiple times and the downsides to his seem tiny.

As more future work (out of scope of this RFC, which is good as-is):

make it easier to emulate the build process. For example:
1. provide a command that will print out a sort of shell script that emulates what nix-build would do. That would eliminate the need to figure out which phases to run and in which order.
2. provide something like progressUntil installPhase which will execute everything up to but not including installPhase
add a debugging flag to nix-build (--shell-on-failure) that drops the user into a nix-shell in the current build directory if the build fails
figure out a way to make installPhase work in a nix-shell, maybe by mounting some tmpdir to $out

Out of all of those 1.i. should be pretty easy to implement and a great help already.

dezgeg · 2018-08-30T14:55:36Z

As more future work (out of scope of this RFC, which is good as-is):

I actually experimented with implementing such tool at some point: https://github.com/dezgeg/nix-debug-shell but I am not particularly good with things like documentation or finishing stuff that I've started, so it's in a kind of forgotten state and potentially bitrotted by now. Maybe it could be moved to https://github.com/nix-community/ if there are people willing to work on it.

Ericson2314 · 2018-08-30T17:34:57Z

Nice proposal. I definitely want this.

https://github.com/ryantrinkle/nix-build-inplace is another such debug build helper.

FRidh · 2018-08-30T17:59:03Z

For Python (mk-python-derivation.nix) we need this

    buildPhase = ''
        buildPhase
        (cd foo; buildPhase)
        (cd bar; buildPhase)
    '';

I'm glad to see this is covered.

This RFC looks good to me, though I think the Summary can be improved. Running the hooks independent of the phases has nothing to do with the UX of nix-shell usage and so that change should already be mentioned there.

dezgeg · 2018-08-30T18:52:11Z

For Python (mk-python-derivation.nix) we need this

@FRidh do you have a pointer on where is there is such code in Python infra that need changes? I have found (and fixed) some such cases in my branch but none in Python.

[...] I think the Summary can be improved. Running the hooks independent of the phases has nothing to do with the UX of nix-shell usage and so that change should already be mentioned there.

Yeah, the hook changes are an afterthought that came up after I started writing the description... I will reword it a bit.

Ericson2314 · 2018-08-30T20:25:41Z

This is probably off topic, but I'd like if we made the findInputs dependency stuff in a phase, which like preHook is run by nix-shell by default for backwards compat. It seems more consistent to do all "real work" in phases.

FRidh · 2018-08-31T05:48:26Z

@dezgeg fixupPhase in pkgs/development/interpreters/python/mk-python-derivation.nix

Actually, reading it again it is a bit different.

dezgeg · 2018-08-31T15:35:57Z

Actually, reading it again it is a bit different.

Yep, it's a different case. The sort of Nix-level code in mk-python-derivation.nix doesn't require any changes. Changes are only needed if inside a fixupPhase = '' ... ''; block (or equivalently, postFixup) there is a call to the Bash function fixupPhase.

FRidh · 2018-09-01T10:01:31Z

@dezgeg do you have a branch with this change? I would like to try it out, especially because of NixOS/nixpkgs@9333c1b

dezgeg · 2018-09-01T10:58:57Z

The branch is mentioned under related-issues: https://github.com/dezgeg/nixpkgs/tree/better-run-phases

timokau · 2018-10-05T13:35:02Z

Since there is nobody to decide and there hasn't been any opposition, should we merge this?

zimbatm · 2018-10-05T20:10:43Z

It seems like a good target for 19.03. How much pain will it create for out-of-tree nix code?

lheckemann · 2018-12-14T14:09:47Z

I usually use runHook buildPhase, is there anything wrong with that?

shlevy · 2019-01-17T13:13:28Z

@dezgeg Are you ready for us to open nominations for a shepherd team as per the new RFC process?

shlevy · 2019-01-24T11:44:46Z

Closing for now. Feel free to reopen when this is ready for general review and shepherding.

Mic92 · 2020-06-25T10:59:32Z

@lheckemann Do you still plan to make a meeting for this?

For example, for building the Nix flake, you would do: $ nix develop --configure $ nix develop --install $ nix develop --installcheck

nixos-discourse · 2020-10-05T15:50:31Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixpkgs-phases-which-shell-language-how-to-overwrite-phases/9307/3

nixos-discourse · 2020-10-05T19:39:13Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-to-create-package-with-multiple-sources/9308/3

lheckemann · 2020-11-09T20:02:21Z

So, I've finally arranged the meeting with @Ericson2314 and @samueldr. Results:

This RFC looks good generally
Unresolved question "This needs some testing to see if this works with the ancient Darwin Bash." — this should, as far as we know, not be a problem: the darwin bootstrap uses a more recent bash from the bootstrap tarball, not the macOS system bash; and nix-shell uses bashInteractive from <nixpkgs>, which is also recent enough for this not to be a concern.
Unresolved question regarding elegance of the way that backwards compatibility is ensured: we consider this an implementation detail which should not be an issue for the RFC in general.
Some general discussion on backwards-incompatible changes to stdenv. We decided not to bike-shed this RFC on that subject :)
We're not sure who will implement this. Both @dezgeg and @globin haven't been very active recently and may not have the capacity/time currently.
The last concern notwithstanding, we decided to call the Final Comment Period on this RFC, with disposition to accept. Speak within the next 10 days, or forever hold your peace!
Implementation need not follow immediately, but we believe accepting this RFC is helpful to indicate the community's general acceptance of the idea.

nixos-discourse · 2020-11-09T20:04:33Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rfc32-phase-running-changes-for-better-nix-shell-use-fcp/9918/1

roberth · 2023-01-25T18:42:31Z

rfcs/0032-run-phase-changes-for-better-nix-shell-use.md

+-buildPhase() {
+defaultBuildPhase() {
+-    runHook preBuild
+-
+     # set to empty if unset
+     : ${makeFlags=}
+
+@@ -1008,14 +1104,14 @@ buildPhase() {
+         make ${makefile:+-f $makefile} "${flagsArray[@]}"
+         unset flagsArray
+     fi
+-
+-    runHook postBuild
+ }
+
+buildPhase() {
+    runHook preBuild
+
+    if [ -n "$buildPhase" ]; then
+        eval "$buildPhase"
+    else
+        defaultBuildPhase
+    fi
+
+    runHook postBuild
+}
+


Has this not been implemented yet, or implemented differently?

Haven't seen @dezgeg in a while, don't think this ever got implemented.

Fwiw, NixOS/nixpkgs#175605 suggests a perhaps less breaking solution that's opt-in. Opting in is in itself messy though, so that's a reason to prefer this RFC if feasible.

…uments Some default.nix arguments

Co-authored-by: Linus Heckemann <git@sphalerite.org>

[RFC0032] Phase running changes for better nix-shell use

1479cad

dezgeg force-pushed the phase-changes branch from f3bfa1f to 1479cad Compare August 29, 2018 23:25

dezgeg changed the title ~~[RFCxxxx] Phase running changes for better nix-shell use~~ [RFC0032] Phase running changes for better nix-shell use Aug 29, 2018

dezgeg changed the title ~~[RFC0032] Phase running changes for better nix-shell use~~ [RFC 0032] Phase running changes for better nix-shell use Aug 29, 2018

This comment has been minimized.

Sign in to view

timokau approved these changes Aug 30, 2018

View reviewed changes

dezgeg mentioned this pull request Aug 30, 2018

Call for Content: 2018/08 NixOS/nixos-weekly#63

Merged

dezgeg added 2 commits August 30, 2018 21:56

Add syntax highlighting to code blocks

a16dadf

[RFC0032] Reword summary to mention the 2nd goal

928b72b

FRidh mentioned this pull request Sep 1, 2018

pytest: do not store .pytest_cache in $out NixOS/nixpkgs#40273

Closed

FRidh mentioned this pull request Sep 1, 2018

pythonPackages.pytest: fix .pytest_cache, again NixOS/nixpkgs#45898

Merged

9 tasks

teto mentioned this pull request Sep 8, 2018

consistent type for genericBuild phases NixOS/nixpkgs#31682

Open

veprbl mentioned this pull request Oct 20, 2018

MCFM: init at 8.2 NixOS/nixpkgs#48674

Closed

9 tasks

spacekookie mentioned this pull request Apr 30, 2020

RFC meeting 2020-04-30 NixOS/rfc-steering-committee#36

Closed

11 tasks

[0032] Make @globin primary author

6e48c1e

lheckemann mentioned this pull request May 28, 2020

RFC meeting 2020-05-28 NixOS/rfc-steering-committee#37

Closed

10 tasks

edolstra mentioned this pull request Jun 11, 2020

RFC meeting 2020-06-11 NixOS/rfc-steering-committee#38

Closed

9 tasks

Mic92 mentioned this pull request Jun 25, 2020

RFC meeting 2020-06-25 NixOS/rfc-steering-committee#39

Closed

12 tasks

lheckemann mentioned this pull request Jul 16, 2020

RFC meeting 2020-07-16 NixOS/rfc-steering-committee#42

Closed

12 tasks

spacekookie mentioned this pull request Jul 30, 2020

Meeting 2020-07-30 NixOS/rfc-steering-committee#43

Closed

12 tasks

edolstra mentioned this pull request Aug 13, 2020

Meeting 2020-08-13 NixOS/rfc-steering-committee#44

Closed

12 tasks

Mic92 mentioned this pull request Aug 27, 2020

Meeting 2020-08-27 NixOS/rfc-steering-committee#45

Closed

13 tasks

Ericson2314 referenced this pull request in NixOS/nix Aug 29, 2020

nix develop: Add convenience flags for running specific phases

f156513

For example, for building the Nix flake, you would do: $ nix develop --configure $ nix develop --install $ nix develop --installcheck

spacekookie mentioned this pull request Sep 10, 2020

Meeting 2020-09-10 NixOS/rfc-steering-committee#46

Closed

12 tasks

lheckemann mentioned this pull request Sep 24, 2020

RFC meeting 2020-09-24 NixOS/rfc-steering-committee#47

Closed

11 tasks

Mic92 mentioned this pull request Nov 5, 2020

RFC meeting 2020-10-22 NixOS/rfc-steering-committee#48

Closed

13 tasks

lheckemann mentioned this pull request Nov 5, 2020

RFC meeting 2020-11-05 NixOS/rfc-steering-committee#49

Closed

13 tasks

lheckemann added status: FCP in Final Comment Period and removed status: in discussion labels Nov 9, 2020

RFC32: no more unresolved questions

4a109d5

Mic92 merged commit eee6599 into NixOS:master Nov 19, 2020

Mic92 added status: accepted and removed status: FCP in Final Comment Period labels Nov 19, 2020

spacekookie mentioned this pull request Nov 19, 2020

Meeting 2020-11-19 NixOS/rfc-steering-committee#50

Closed

9 tasks

Artturin mentioned this pull request Jan 25, 2023

Fix the problem of forgotten runHook {pre,post}* calls NixOS/nixpkgs#175605

Open

roberth reviewed Jan 25, 2023

View reviewed changes

infinisil referenced this pull request in nixpkgs-architecture/rfc-140 Jan 30, 2023

Merge pull request #32 from nixpkgs-architecture/some-default.nix-arg…

3be7452

…uments Some default.nix arguments

KAction pushed a commit to KAction/rfcs that referenced this pull request Apr 13, 2024

[RFC 0032] Phase running changes for better nix-shell use (NixOS#32)

a640967

Co-authored-by: Linus Heckemann <git@sphalerite.org>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RFC 0032] Phase running changes for better nix-shell use #32

[RFC 0032] Phase running changes for better nix-shell use #32

dezgeg commented Aug 29, 2018 •

edited

Loading

This comment has been minimized.

samueldr commented Aug 30, 2018

dezgeg commented Aug 30, 2018

timokau commented Aug 30, 2018

dezgeg commented Aug 30, 2018

Ericson2314 commented Aug 30, 2018

FRidh commented Aug 30, 2018

dezgeg commented Aug 30, 2018

Ericson2314 commented Aug 30, 2018 •

edited

Loading

FRidh commented Aug 31, 2018 •

edited

Loading

dezgeg commented Aug 31, 2018

FRidh commented Sep 1, 2018

dezgeg commented Sep 1, 2018

timokau commented Oct 5, 2018

zimbatm commented Oct 5, 2018

lheckemann commented Dec 14, 2018

shlevy commented Jan 17, 2019

shlevy commented Jan 24, 2019

Mic92 commented Jun 25, 2020

nixos-discourse commented Oct 5, 2020

nixos-discourse commented Oct 5, 2020

lheckemann commented Nov 9, 2020

nixos-discourse commented Nov 9, 2020

roberth Jan 25, 2023

Ericson2314 Jan 25, 2023

roberth Jan 25, 2023

[RFC 0032] Phase running changes for better nix-shell use #32

[RFC 0032] Phase running changes for better nix-shell use #32

Conversation

dezgeg commented Aug 29, 2018 • edited Loading

This comment has been minimized.

samueldr commented Aug 30, 2018

dezgeg commented Aug 30, 2018

timokau commented Aug 30, 2018

dezgeg commented Aug 30, 2018

Ericson2314 commented Aug 30, 2018

FRidh commented Aug 30, 2018

dezgeg commented Aug 30, 2018

Ericson2314 commented Aug 30, 2018 • edited Loading

FRidh commented Aug 31, 2018 • edited Loading

dezgeg commented Aug 31, 2018

FRidh commented Sep 1, 2018

dezgeg commented Sep 1, 2018

timokau commented Oct 5, 2018

zimbatm commented Oct 5, 2018

lheckemann commented Dec 14, 2018

shlevy commented Jan 17, 2019

shlevy commented Jan 24, 2019

Mic92 commented Jun 25, 2020

nixos-discourse commented Oct 5, 2020

nixos-discourse commented Oct 5, 2020

lheckemann commented Nov 9, 2020

nixos-discourse commented Nov 9, 2020

roberth Jan 25, 2023

Choose a reason for hiding this comment

Ericson2314 Jan 25, 2023

Choose a reason for hiding this comment

roberth Jan 25, 2023

Choose a reason for hiding this comment

dezgeg commented Aug 29, 2018 •

edited

Loading

Ericson2314 commented Aug 30, 2018 •

edited

Loading

FRidh commented Aug 31, 2018 •

edited

Loading