Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve performance of the advanced copy task for structured content #870

Closed
hstct opened this issue Aug 24, 2023 · 2 comments · Fixed by #891
Closed

Improve performance of the advanced copy task for structured content #870

hstct opened this issue Aug 24, 2023 · 2 comments · Fixed by #891
Labels
.bugfix CHANGES/<issue_number>.bugfix Katello For bugs and issues known to affect Katello prio-list

Comments

@hstct
Copy link
Contributor

hstct commented Aug 24, 2023

Currently the way pulp_deb handles the advanced copy for structured content is inefficient. It fetches all the content and filters out what we don't want. As a side effect it will always include all architectures from the source RepositoryVersion despite which ones the packages are actually using.

It even creates a new RepositoryVersion on the target with just the ReleaseArchitectures if no packages are selected for the content (it should not copy in anything in that case nor create a new RepositoryVersion).

It should be possible to improve the performance by better querying the database.
@quba42
Copy link
Collaborator

quba42 commented Aug 24, 2023

I recall now why we chose to pull all the architectures along. ReleaseArchitectures actually represent the Architectures: field from the Release file which is explicitly allowed to advertise architectures that don't have any associated packages. That being said, just because it is allowed does not mean it is the right behaviour for the advanced copy. I think we should have a look what effect this has on the Katello incremental content view usecase as a thought experiment and then decide exactly what it is we want here.

@quba42
Copy link
Collaborator

quba42 commented Aug 28, 2023

Internal tracker: OR-3153

@quba42 quba42 added .bugfix CHANGES/<issue_number>.bugfix Katello For bugs and issues known to affect Katello prio-list and removed Triage-Needed labels Aug 28, 2023
quba42 added a commit to ATIX-AG/pulp_deb that referenced this issue Sep 6, 2023
@quba42
WIP: Improve advanced copy performance
7a957bb 
closes pulp#870
quba42 added a commit to ATIX-AG/pulp_deb that referenced this issue Sep 11, 2023
@quba42
Improve advanced copy performance
b94334d 
closes pulp#870
@quba42 quba42 linked a pull request Sep 11, 2023 that will close this issue
Improve advanced copy performance #891
Merged
quba42 added a commit to ATIX-AG/pulp_deb that referenced this issue Sep 13, 2023
@quba42
Improve advanced copy performance
4a8e0c7 
closes pulp#870

Also fix a minor bug where the structured copy would always copy all
ReleaseArchitecture content from the source repo, regardless of whether
it was needed.
daviddavis pushed a commit to daviddavis/pulp_deb that referenced this issue Nov 20, 2023
@quba42 @daviddavis
Improve advanced copy performance
09cfe27 
closes pulp#870

Also fix a minor bug where the structured copy would always copy all
ReleaseArchitecture content from the source repo, regardless of whether
it was needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
.bugfix CHANGES/<issue_number>.bugfix Katello For bugs and issues known to affect Katello prio-list
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Improve advanced copy performance ATIX-AG/pulp_deb
2 participants
@quba42 @hstct