-
-
Notifications
You must be signed in to change notification settings - Fork 4.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cronjob blocking update (forever) #9992
Comments
This was added on purpose to not have a running background job while updating. This is not really a regression, but a symptom of a previously broken behavior that was just ignored. Why do you have jobs that where reserved but never finished? Which jobs are still reserved? |
ref #9900 |
I have no idea – it’s on my local dev instance, and these were the ones with a reserved_at value of over 0:
|
Anything in the logs which says "cron" as app? |
@skjnldsv did you have anything like that? I need to check back tomorrow, am mobile only. |
I had an error about a missing index 0 (was on the level array a few lines above the waitforcron function) |
Ping me once you have this again and do some screensharing/pair-debugging |
@MorrisJobke The pending Job I had was a backgroundTest coming after running a phpunit test iirc |
Sure - I can recreate by manipulate the DB but that’s not the point. I want to know why the job was started and never finished. |
Errored here as well just now: But neither app is enabled, so when they are tried to be executed, the class does not exist => error => catched => ignored and continued The cron.php ignores those for 12 hours, to save time and not try them again and again, because that caused a lot of troubles for non system crons in the past. |
Any idea how to check then for proper cron execution? Wouldn't it help to put the "last checked" to 12 h in the future? |
Weird - this case should be catched by the server/lib/private/BackgroundJob/JobList.php Lines 253 to 277 in 18e9631
|
At least I can reproduce it and will now debug it locally.
|
Reservation is done before build: server/lib/private/BackgroundJob/JobList.php Lines 205 to 214 in 18e9631
It should also be like this, otherwise two could try this in parallel... |
…continous re-check * fixes issue where cronjobs of a not-loaded app are marked as "still running" because there is a "reserved_at" value stored * fixed #9992 Signed-off-by: Morris Jobke <hey@morrisjobke.de>
Fix is in #10010 |
* see #9992 Signed-off-by: Morris Jobke <hey@morrisjobke.de>
For the record, just happened again and the last log entries are:
Will test the PR fix :) |
…continous re-check * fixes issue where cronjobs of a not-loaded app are marked as "still running" because there is a "reserved_at" value stored * fixed #9992 Signed-off-by: Morris Jobke <hey@morrisjobke.de>
The thing is, if you where on an old branch sometime in between, that could have bricked the database again. |
Ah ... yes, that's true :/ |
According to the log it seems to be related to the UpdateCalendarResourcesRoomsBackgroundJob which throws a TypeError if no backend for the resource booking is available. |
Just wanted to upgrade from beta4 to RC1 and have the same issue again.... |
Which cron job wasn't finished? and did cron still run? |
Not sure. How to find out? The cron job itself was'nt running....the log even told me it was unable to run because nextcloud is in maintenance mode when i was trying to run it manually. |
I am having the same problem with infinate wait for pending cron jobs to finish. I tried both via Im not sure how to proceed on debugging, but ill be happy to provide details if needed |
Mine has a zero last run in
|
It checks only if "reserved_at" is set to a number that is bigger than "CURRENT UNIX TIMESTAMP" - 43200 (reserved within the past 12h). So if the reserved_at is 0 all should be fine. What DB are you using? |
You could press "Ctrl +C" and start it again. Because I guess SQLite could cause some trouble here :/ For other DBs this is not a problem usually. |
Argh, actually i'm using Postgres and the output above was from the "old" MySQL backup.... This is the real one:
|
Tried again, whatever it was, it's gone and the updater went through without any further issues. |
hey guys, just updating to new production release 14.0.3 and after the update, going back to Nextcoud startpage it says:
waiting for cron to finish... |
thank, this works. |
Btw @MorrisJobke I keep running into this issue. The command in the OP Are you saying that next time I run into it I should run |
Correct.
This basically not good, because it means that one background job crashed or it is actually still running. I guess we should remove the check again. I just thought it may be a good idea to do this. :( |
Just happened again, and the output of the select command is:
So it seems the profile data verification service. cc @schiessle @juliushaertl or who knows about this? And again, again verification:
|
Fix in #12188 |
Both @skjnldsv and me had this issue earlier, when an update gave us this message:
And that just kept coming again and again. Setting maintenance mode to false also does not work.
It can be fixed using
UPDATE oc_jobs SET reserved_at=0;
(thanks @nickvergessen), but should probably not happen.cc @MorrisJobke @nickvergessen
The text was updated successfully, but these errors were encountered: