Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rustPlatform: improve composability when the other builder defines a phase #334476

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

jian-lin
Copy link
Contributor

@jian-lin jian-lin commented Aug 13, 2024

Description of changes

Motivation: improve composability between the rust builder and Emacs lisp builders

Previously, if the other builder defines a phase, Rust hooks do
nothing, which is not desirable. With this patch applied, if the
other builder defines a phase, Rust hooks will use pre hoooks instead.

Review notes

  • emacsPackages.lspce is an Emacs lisp package with a dynamic module written in Rust. The commit rewriting it is an example of the improvement this PR brings.
  • emacsPackages.lspce builds.
  • The effect on Rust packages have not been tested, so this is a draft PR.

TODO

  • Check the impact of this PR on Rust packages.
  • Maybe also change cargoNextestHook and maturinBuildHook. I am not familiar with them though, so help is appreciated.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

Previously, if the other builder defines a phase, Rust hooks do
nothing, which is not desirable.  With this patch applied, if the
other builder defines a phase, Rust hooks will use pre hoooks instead.

TODO: maybe also change the other 3 hooks
Previously, this package is implemented as two derivations, which is
inconvenient for overriding.  This patch uses hooks from rustPlatform
to implement it as a single derivation, which can be overridden like
any other Rust packages.
archive="$NIX_BUILD_TOP/packages/$ename-$melpaVersion.tar"
fi

echo "add the dynamic module to the package tarball because it is needed for compilation"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo "add the dynamic module to the package tarball because it is needed for compilation"
# add the dynamic module to the package tarball because it is needed for compilation

I can't see a reason to echo this info by default.
If there is some reason to do this, it should use some logging functionality instead.

Copy link
Contributor Author

@jian-lin jian-lin Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I slightly prefer logging this info because it is a bit "unusual".

logging functionality

You mean things like nixInfoLog, right? It is very weird that those functions are not used outside of pkgs/stdenv/generic/setup.sh.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They were merged into Staging a week ago: #331560

And were injected on Master yesterday: #332764

jian-lin added a commit to linj-fork/nixpkgs that referenced this pull request Aug 14, 2024
It is better to not change the working directory.

One example showing the benefit is the rewriting[1] of lspce, where
the Rust build hook assumes the working directory is unchanged.

[1]: NixOS#334476
jian-lin added a commit to linj-fork/nixpkgs that referenced this pull request Aug 14, 2024
It is better to not change the working directory.

One example showing the benefit is the rewriting[1] of lspce, where
the Rust build hook assumes the working directory is unchanged.

[1]: NixOS#334476
jian-lin added a commit to linj-fork/nixpkgs that referenced this pull request Aug 14, 2024
It is better to not change the working directory.

One example showing the benefit is the rewriting[1] of lspce, where
the Rust build hook assumes the working directory is unchanged.

[1]: NixOS#334476
@AndersonTorres
Copy link
Member

Well, from my part I approve this PR because it improves things.

On the other hand, I think the solution is not clean. It looks like the unnamed Monster of Frankenstein to me.
It is more a structural problem than any other thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thread for #334476 (comment):

I think the solution is not clean. It looks like the unnamed Monster of Frankenstein to me. It is more a structural problem than any other thing.

@AndersonTorres Could you elaborate why you think it is not clean and what the structural problem is?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One problem I see is that the current design of builders does not allow us to move phases. An example is that elisp packages with a Rust module want to do the elisp buildPhase after the Rust buildPhase and installPhase.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is too subjective and strongly tied to a very particular perception I have.

If I was in a more barebones, Slackware-like distro, I would just manually install the dependencies (Rust compiler, Emacs, libs etc.) and write a shell script that executes a sequence of commands in the pristine, upstream source code.

I would design such a build script in a pipeline fashion.
A rough and not very accurate sketch would be something like this:

  1. split the pristine source code in a Rust-only part and an Emacs-only part
  2. run the Rust compiler pipeline to transform the Rust part in object code
    • or, more accurately, something suitable be processed by the future Emacs pipeline
  3. merge the object code with the Emacs code
  4. run the Emacs pipeline to transform this Emacs+object-code to a final Elisp package

On the other hand, when I look to your solution above, it is something similar to having two pipelines, one made for Emacs and other for Rust, disassemble both and amending specific parts of each so that they look externally similar to a Emacs pipeline.

Indeed, stretching the analogy a little bit, you are effectively debasing and polishing the pipes from the Rust pipeline so that they can be more easily amended to other pipes.

Copy link
Contributor Author

@jian-lin jian-lin Aug 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are effectively debasing and polishing the pipes from the Rust pipeline so that they can be more easily amended to other pipes

Yes, this is exactly what I am doing here, following the line of work making the Rust builder use setup hooks by @ danieldk. We share the same motivation: #112438.

I would say this is a way to make builders composable with minimal changes.

kira-bruneau pushed a commit to kira-bruneau/nur-packages that referenced this pull request Sep 2, 2024
It is better to not change the working directory.

One example showing the benefit is the rewriting[1] of lspce, where
the Rust build hook assumes the working directory is unchanged.

[1]: NixOS/nixpkgs#334476
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants