Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert to nix-community/mavenix in nix build #3749

Merged
merged 2 commits into from
Oct 24, 2023
Merged

Conversation

goodlyrottenapple
Copy link
Contributor

@goodlyrottenapple goodlyrottenapple commented Oct 24, 2023

This reverts the change to mavenix from #2673 which introduced a custom version of the mavenix build tool. The original mavenix builds a "repo" derivation which symlinks to all the pom and jar files maven needs to build k. The way mavenix assembles this repo is to first copy each pom/jar file into the nix store individually and then symlink them into a single derivation. Once we build using maven, mavenix copies any required runtime jars to the final k derivation. This obviously creates some redundancy, because there are now two copies of some jar/pom files in the store. For the cachix cache, this means unnecessary duplication each time we push to it. The solution in the aforementioned PR was to symlink to the already present jars/poms instead of copying them.

However, with the new binary cache, the focus is on speed of downloading the pre-compiled binaries and having lot of symlinked paths from the k derivation to each individual pom/jar file introduces a serious bottleneck when we call nix copy to download the k binary (see NixOS/nix#2559). Until such time as this issue is resolved, we should revert to the old mavenix which reduces the k closure size (i.e. the number of files the k derivation depends on in the nix store, not necessarily the total size in mb) drastically at the expense of some added duplication of jar files with each build.

@goodlyrottenapple goodlyrottenapple changed the title Do not copy jars Revert to nix-community/mavenix in nix build Oct 24, 2023
@goodlyrottenapple goodlyrottenapple marked this pull request as ready for review October 24, 2023 12:44
@rv-jenkins rv-jenkins merged commit 18c190f into develop Oct 24, 2023
@rv-jenkins rv-jenkins deleted the sam/nix-no-copy-jars branch October 24, 2023 14:43
@Baltoli Baltoli mentioned this pull request Dec 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants