Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

avoid multiple executions of makepkg for the same PKGBUILD #2340

Closed
linuxmetel opened this issue Jan 8, 2024 · 2 comments · Fixed by #2534
Closed

avoid multiple executions of makepkg for the same PKGBUILD #2340

linuxmetel opened this issue Jan 8, 2024 · 2 comments · Fixed by #2534
Labels
help wanted Status: Confirmed Bug has been verified Type: Bug

Comments

@linuxmetel
Copy link

linuxmetel commented Jan 8, 2024 •
edited
Loading

Is your feature request related to a problem? Please describe.

I wanted to install some themes for GRUB, so I tried installing packages in grub-themes-git from AUR.

$ yay -S `cat grub-themes-git-package-list`

However, this command seems to do like this for the 57 package (I translated from Japanese so there may be different from the original version):

==> Making package: grub-themes-git 2022.10.30.r35.g000171d-1 (09/01/2024 01:03:04)
==> Checking the runtime's dependencies...
==> Checking the dependencies in the building time...
==> Retrieving the source...
  -> Syncing grub2-themes git repo...
==> Validating sources file with sha256sums...
    grub2-themes ... skipped
==> Deleting the existing $srcdir/ directory...
==> Expanding the source...
  -> Copying grub2-themes git for building...
Cloning into 'grub2-themes'...
done.
==> Starting pkgver()...
==> Prepared the source
 -> grub-themes-git-2022.10.30.r35.g000171d-1 has already existed -- Skipping to build

Each process took a lot, and the sum reached hours on my machine.

Describe the solution you'd like

I think that yay command should skip preparing and building process for the packages whose PKGBUILD has been executed (in the same yay command's execution). In other words, I think that one PKGBUILD should be executed just once in single yay command's execution.

And also it may be a radical solution to implement the functionality for installing by pkgbase.
@linuxmetel linuxmetel added Status: Discussion Open Topic is being considered and open for discussion Type: Feature Request labels Jan 8, 2024
@linuxmetel linuxmetel changed the title avoid multiple execution of makepkg for the same pkgbase avoid multiple executions of makepkg for the same PKGBUILD Jan 8, 2024
@Jguer Jguer added Type: Bug help wanted Status: Confirmed Bug has been verified and removed Type: Feature Request Status: Discussion Open Topic is being considered and open for discussion labels Jan 23, 2024
@Ferdi265
Copy link
Contributor

I'm currently hitting this with ttf-ms-win11-auto. updating takes very long since each subpackage is built every time.

Since I've contributed to yay before, I can try to patch yay to fix this and see if there is an easy/obvious solution to this.

Ferdi265 added a commit to Ferdi265/yay that referenced this issue Nov 18, 2024
@Ferdi265
fix(installer): Fixes the same pkgbase being built multiple times
6b0ac30 
When building a PKGBUILD pkgbase with multiple pkgnames,
installAURPackages() invokes buildPkg() multiple times for the same
pkgbase. This causes prepare() to be run multiple times for the same
pkgbase, since detection of already built packages happens after
prepare().

Additionally, detection of already built pacakges can fail if the split
debug packages are enabled and the package does not contain any
binaries, causing no -debug package to be created by makepkg even though
it is listed by makepkg --packagelist.

This commit fixes this by keeping track of the pkgdests built by
buildPkg() and avoiding rebuilds of the same pkgbase in the same yay
invocation.

Fixes Jguer#2340.

Signed-off-by: Ferdinand Bachmann <ferdinand.bachmann@yrlf.at>
Ferdi265 added a commit to Ferdi265/yay that referenced this issue Nov 18, 2024
@Ferdi265
fix(installer): Fixes the same pkgbase being built multiple times
9b10910 
When building a PKGBUILD pkgbase with multiple pkgnames,
installAURPackages() invokes buildPkg() multiple times for the same
pkgbase. This causes prepare() to be run multiple times for the same
pkgbase, since detection of already built packages happens after
prepare().

Additionally, detection of already built pacakges can fail if the split
debug packages are enabled and the package does not contain any
binaries, causing no -debug package to be created by makepkg even though
it is listed by makepkg --packagelist.

This commit fixes this by keeping track of the pkgdests built by
buildPkg() and avoiding rebuilds of the same pkgbase in the same yay
invocation.

Fixes Jguer#2340.

Signed-off-by: Ferdinand Bachmann <ferdinand.bachmann@yrlf.at>
Ferdi265 added a commit to Ferdi265/yay that referenced this issue Nov 18, 2024
@Ferdi265
fix(installer): Fixes the same pkgbase being built multiple times
d2837e1 
When building a PKGBUILD pkgbase with multiple pkgnames,
installAURPackages() invokes buildPkg() multiple times for the same
pkgbase. This causes prepare() to be run multiple times for the same
pkgbase, since detection of already built packages happens after
prepare().

Additionally, detection of already built packages can fail if the split
debug packages are enabled and the package does not contain any
binaries, causing no -debug package to be created by makepkg even though
it is listed by makepkg --packagelist.

This commit fixes this by keeping track of the pkgdests built by
buildPkg() and avoiding rebuilds of the same pkgbase in the same yay
invocation.

Fixes Jguer#2340.

Signed-off-by: Ferdinand Bachmann <ferdinand.bachmann@yrlf.at>
@Ferdi265 Ferdi265 mentioned this issue Nov 18, 2024
Merged
Ferdi265 added a commit to Ferdi265/yay that referenced this issue Nov 18, 2024
@Ferdi265
fix(installer): Fixes the same pkgbase being built multiple times
d9e17f9 
When building a PKGBUILD pkgbase with multiple pkgnames,
installAURPackages() invokes buildPkg() multiple times for the same
pkgbase. This causes prepare() to be run multiple times for the same
pkgbase, since detection of already built packages happens after
prepare().

Additionally, detection of already built packages can fail if the split
debug packages are enabled and the package does not contain any
binaries, causing no -debug package to be created by makepkg even though
it is listed by makepkg --packagelist.

This commit fixes this by keeping track of the pkgdests built by
buildPkg() and avoiding rebuilds of the same pkgbase in the same yay
invocation.

Fixes Jguer#2340.

Signed-off-by: Ferdinand Bachmann <ferdinand.bachmann@yrlf.at>
@Ferdi265
Copy link
Contributor

Ferdi265 commented Nov 18, 2024 •
edited
Loading

My fix PR seems to work pretty well with ttf-ms-win11-auto, build times are reduced by a lot. I also tested the package @linuxmetel had the original issue with:

$ yay -S `cat grub-themes-git-package-list`

This runs in under 9 minutes on my machine now.

Jguer pushed a commit that referenced this issue Nov 19, 2024
@Ferdi265
fix(installer): Fixes the same pkgbase being built multiple times (#2534
ec837c8 
)

* fix(installer): Fixes the same pkgbase being built multiple times

When building a PKGBUILD pkgbase with multiple pkgnames,
installAURPackages() invokes buildPkg() multiple times for the same
pkgbase. This causes prepare() to be run multiple times for the same
pkgbase, since detection of already built packages happens after
prepare().

Additionally, detection of already built packages can fail if the split
debug packages are enabled and the package does not contain any
binaries, causing no -debug package to be created by makepkg even though
it is listed by makepkg --packagelist.

This commit fixes this by keeping track of the pkgdests built by
buildPkg() and avoiding rebuilds of the same pkgbase in the same yay
invocation.

Fixes #2340.

Signed-off-by: Ferdinand Bachmann <ferdinand.bachmann@yrlf.at>

* fix(installer): Fixes buildPkg() isTarget param being order-dependent

Previously, the buildPkg invocation for a pkgbase only considered
whether the current pkgname is part of installer.origTargets. This made
the decision whether to rebuild the package order-dependent.

This commit fixes this by keeping track of which pkgbases are part of
installer.origTargets and rebuilding the pkgbase if any of its pkgnames
is part of origTargets.

* fix(tests): Test that installing split packages avoids rebuilds

The previous two commits changed how split packages (packages with the same
pkgbase) are built, ensuring that those packages aren't built multiple
times.

This commit updates the lists of commands that the tests expect to be
run so that `makepkg` isn't run multiple times per pkgbase.

---------

Signed-off-by: Ferdinand Bachmann <ferdinand.bachmann@yrlf.at>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
help wanted Status: Confirmed Bug has been verified Type: Bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(installer): Fixes the same pkgbase being built multiple times Ferdi265/yay
3 participants
@Ferdi265 @Jguer @linuxmetel