Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added /prune/ endpoint to removing "old" RPMs from a Repository. #3536

Merged
merged 1 commit into from
Jun 12, 2024

Conversation

ggainey
Copy link
Contributor

@ggainey ggainey commented May 14, 2024

closes #2909.

@ggainey ggainey force-pushed the 2909_prune branch 2 times, most recently from 4d8ad7c to edf4989 Compare May 14, 2024 01:32
@ggainey ggainey marked this pull request as draft May 14, 2024 04:28
@ggainey
Copy link
Contributor Author

ggainey commented May 14, 2024

Draft until I figure out what is up w/ permissions - can't reproduce locally?!?

@ggainey ggainey marked this pull request as ready for review May 14, 2024 16:23
@ggainey ggainey requested review from dralley and ipanova May 14, 2024 17:44
@ggainey
Copy link
Contributor Author

ggainey commented May 14, 2024

Some notes on performance:

Some unscientific performance results follow. The repositories are a selection of "real" repos, of small (zoo), medium (baseos-9) and large (appstream-9, epel) amounts of content.

All runs used concurrency==1, oci-env, on my workstation (Intel® Core™ i7-6700K × 8), info-logging only.
Repositories are --no-autopublish.

Repository List:

name: zoo
size: 35
dups: 4

name: rhel_9_x86_64_appstream
size: 17524
dups: 11834

name: rhel_9_x86_64_baseos
size: 6185
dups: 4993

name: epel9
size: 21236
dups: 0

How "prune" was run

$ TGROUP=`http -b POST :5001/pulp/api/v3/rpm/prune/ repo_hrefs:='["*"]' repo_concurrency:=1 keep_days=0 dry_run=True | jq -r .task_group`

How timings were evaluated

for t in `http ":5001${TGROUP}" | jq -r .tasks[].pulp_href`
do 
  pulp task show --href "${t}" | jq '.started_at, .finished_at, .progress_reports[0].message, .progress_reports[0].total' 
done

Performance Runs

  • dry-run, keep=0 (max deletes discovered only)

    • zoo : 46ms
    • epel : 850ms
    • base : 380ms
    • apps : 912ms
  • prune, keep=1000 (no deletes discovered only)

    • zoo : 49ms
    • epel : 849ms
    • base : 392ms
    • apps : 929ms
  • prune, keep=0 (max actual deletes)

    • zoo : 163ms
    • epel : 2214ms
    • base : 1823ms
    • apps : 4698ms
  • prune, keep=0, post-max-prune (0 actual deletes)

    • zoo : 206ms
    • epel : 1884ms
    • base : 628ms
    • apps : 1365ms

staging_docs/user/guides/06-prune.md Outdated Show resolved Hide resolved
pulp_rpm/app/viewsets/prune.py Show resolved Hide resolved
pulp_rpm/app/tasks/prune.py Outdated Show resolved Hide resolved
pulp_rpm/app/serializers/prune.py Outdated Show resolved Hide resolved
pulp_rpm/app/serializers/prune.py Show resolved Hide resolved
pulp_rpm/app/serializers/prune.py Outdated Show resolved Hide resolved
staging_docs/user/guides/06-prune.md Show resolved Hide resolved
pulp_rpm/app/tasks/prune.py Outdated Show resolved Hide resolved
@ggainey ggainey marked this pull request as draft June 5, 2024 17:36
@ggainey ggainey marked this pull request as draft June 5, 2024 17:36
@ggainey
Copy link
Contributor Author

ggainey commented Jun 5, 2024

Drafting temporarily while I think on some suggestions.

@github-actions github-actions bot added the multi-commit Added when a PR consists of more than one commit label Jun 7, 2024
@ggainey ggainey marked this pull request as ready for review June 7, 2024 16:47
@ggainey ggainey requested review from dralley and ipanova June 7, 2024 21:39
CHANGES/2909.feature Outdated Show resolved Hide resolved
staging_docs/user/guides/06-prune.md Outdated Show resolved Hide resolved
Copy link
Member

@ipanova ipanova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!Thanks

@github-actions github-actions bot removed the multi-commit Added when a PR consists of more than one commit label Jun 11, 2024
@ggainey ggainey merged commit 10a703a into pulp:main Jun 12, 2024
13 of 16 checks passed
@xsuchy
Copy link

xsuchy commented Jun 12, 2024

Thank you.

@ggainey ggainey deleted the 2909_prune branch June 21, 2024 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

As an administrator I want to set prune policy based on datetime
4 participants