Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The action gets stuck checking existing stale PRs without getting to new ones #792

Closed
mezz opened this issue Aug 11, 2022 · 17 comments · Fixed by #1033
Closed

The action gets stuck checking existing stale PRs without getting to new ones #792

mezz opened this issue Aug 11, 2022 · 17 comments · Fixed by #1033
Assignees
Labels
bug Something isn't working keep

Comments

@mezz
Copy link

mezz commented Aug 11, 2022

Describe your issue

I have a long "stale" time of 90 days, and then 30 days to close.
The stale bot uses up all my API actions on iterating over existing stale commits and checking them for comments, and it never gets to apply the stale label to everything that needs it.
I am left with a handful of issues marked stale, but many more need the label.

Your stale action configuration

https://github.com/mezz/JustEnoughItems/blob/def1bfdcba3af124de6fe78fd23ed39c7f1d41a6/.github/workflows/stale.yml#L14-L28

Further context

I think the bot needs a strategy that lets it apply stale labels first, and then check random stale commits for their comments. If the action can persist some data between runs, it would be nice for it to track which issue it left off on so that it can start from there on the next run.

@mezz mezz added the bug Something isn't working label Aug 11, 2022
@steveluscher
Copy link

Oh no. And that's with only ~200 issues and ~20 pull requests? I'm definitely going to run into this problem then, having ~1000 issues and ~300 pull requests.

@dsame
Copy link
Contributor

dsame commented Feb 7, 2023

Hello @mezz , i thought about different strategies and none of them seems perfect to me. But what do you think about just descending sorting the issues by updated_at every time? This way the issues that were not be labeled go first and are updated to be checked next time last. Do you see any disadvantages of this approaches?

@mezz
Copy link
Author

mezz commented Feb 7, 2023

Hello @dsame ,
It is a bit difficult to predict but I think that may at least get things labeled.
I can imagine that the oldest stale issues will not get closed with this strategy.
If you can create a sorting option, I am happy to try it out and let you know if things improve though.

Are you able to store any state between runs, so that it can "resume" when it runs out of actions?

@jklymak
Copy link

jklymak commented Feb 7, 2023

We are discussing trying this in Matplotlib (>1600 open issues and > 300 open PRs). It's a little unclear what "operations-per-run" refers to - will we be making >1900 operations just to get the state of each Issue/PR? Or is it 20 operations in batches of 100? After that, is each label/close an operation? Sorry if this should be clear - not super experienced with what GH Actions consider a call or not.... Note that https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limits-for-requests-from-github-actions claims Actions get 1000/h, so maybe we don't need to limit to anything as low as 30... Thanks for any help; hopefully these clarifications are not hijacking this issue.

@dsame
Copy link
Contributor

dsame commented Feb 8, 2023

@mezz in theory it is possible to use the same storage that is used for caches to store the state but it will require extra API calls just to "get into the previous state" - that i want to avoid.

@mezz
Copy link
Author

mezz commented Feb 8, 2023

I believe the first priority for the system design should be correctness, and from there we can optimize if we can.
Unless there is a better solution that comes to mind, I am not sure how else this bot can work correctly.
My stale bot has maxed out its API calls for months, unfortunately wasting them without making much progress, so requiring extra API calls seems like a low price.

@worldomonation
Copy link

worldomonation commented May 26, 2023

Chiming in with the repo where I mainly work in for my company is again running into this issue. This problem arises when folks use this to mark issues as stale, but not necessarily let the action close it.

I asked a similar question back in early 2021 here: #329.

Numbers:
We have ~4600 active issues, of which 1000 or so are legitimate targets for this action (after taking into account label exemptions).

In each day's action run, it starts with the oldest issue (which has been marked stale for 1+ year now), gradually consuming API calls checking the same issues each time, then halts at around issue 32000. The newest issue is now at ~77xxx, so at the pace we're closing issues (which is manually, not using the action's auto-close feature) the bot will never catch up with the issues being filed.

I'm beginning to close stale issues as it's marked by this action (after a manual review) so it would be interesting to see if that would help the action go further.

@dsame dsame mentioned this issue Jun 22, 2023
2 tasks
@IvanZosimov IvanZosimov linked a pull request Jul 4, 2023 that will close this issue
2 tasks
@dsame
Copy link
Contributor

dsame commented Jul 13, 2023

Hello @mezz , can you please try to run the action on the main branch to make sure it solves the problem?

- uses: actions/stale@main

@dsame dsame reopened this Jul 13, 2023
@mezz
Copy link
Author

mezz commented Jul 14, 2023

Hi @dsame, here are the results of my run with actions/stale@main: https://github.com/mezz/JustEnoughItems/actions/runs/5558711782
It seems to be working correctly, thank you. I can let you know if I run into rate limits with it again.

What has changed in the newer version?

@mezz
Copy link
Author

mezz commented Jul 14, 2023

I see a strange error actually, I think things may not be quite right:
https://github.com/mezz/JustEnoughItems/actions/runs/5558711782/jobs/10154121794#step:2:1252

  [#2380] Marking this pull request as stale
  Error: [#2380] Error when adding a label: Resource not accessible by integration
  [#2380] This pull request is now stale
  [#2380] This pull request is already stale

@dsame
Copy link
Contributor

dsame commented Jul 20, 2023

Hello @mezz , has the problem happened again? The error message says about the server-side problem, usually with the permissions, but it does not seem to be a case. Was not it sporadic server issue?

@mezz
Copy link
Author

mezz commented Jul 20, 2023

I have this set up to run every hour automatically.
A list of all runs is available here: https://github.com/mezz/JustEnoughItems/actions/workflows/stale.yml

You can see the most recent run here:
https://github.com/mezz/JustEnoughItems/actions/runs/5613747199/job/15210436590
It looks like it still has the same issue present:

  [#2380] This pull request should be stale based on the last update date the 21-05-2022 (2022-05-21T06:22:54Z)
  [#2380] This pull request should be marked as stale based on the option days-before-stale (​[https://github.com/actions/stale#days-before-stale​)](https://github.com/actions/stale#days-before-stale%E2%80%8B)) (90)
  [#2380] Marking this pull request as stale
  Error: [#2380] Error when adding a label: Resource not accessible by integration
  [#2380] This pull request is now stale
  [#2380] This pull request is already stale

@dsame
Copy link
Contributor

dsame commented Jul 24, 2023

@mezz thank you for the quick response. Now i see the workflow lacks pull-requests: write permission, this why it can not modify PR.

Recommended permissions are:

permissions:
  contents: write # only for delete-branch option
  issues: write
  pull-requests: write

Did it help?

@dsame
Copy link
Contributor

dsame commented Aug 4, 2023

Hello @mezz, i am closing the issue due to inactivity, but feel free to reopen it or create the new one in case if the problem still exists.

@dsame dsame closed this as completed Aug 4, 2023
@timhirsh
Copy link

@dsame, this is awesome, thank you for implementing! I would love to start leveraging statefulness. When do you expect the next release to be cut?

@kamaradclimber
Copy link

That would indeed be great to get a new release 🙏

Alexander01998 added a commit to Wurst-Imperium/Wurst7 that referenced this issue Oct 19, 2023
@95-martin-orion
Copy link

Adding my support to the requests for a fresh release. The top-level README advertises this feature but it is not present in the latest release, which caused issues when I tried to use it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working keep
Projects
None yet
Development

Successfully merging a pull request may close this issue.

9 participants