server: Workaround for "finish file present too long" client bug #3300
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes second scenario of #3017 and improves #3019
Description of the Change
This patch deals with "finish file problem" completely, bringing tasks affected by this bug "back to life" on the server side.
As was noted in #3017 (comment) , this problem may also appear when client shutting down, and this is not fixed yet (I'll explain this scenario below). In #3019 was mentioned that "it should be treated as successful regardless of whether it exits, but right now we don't have a mechanism for killing a job and marking it as success."
This patch marks job as successful on the server side, so it'll fix all problems with all client versions.
Cause of the problem
There are two causes. First reason is a when an application really exits too slow due to high system load and swap trashing. It was fixed in #3019 by increasing exit timeout up to 5 minutes.
But in real life, on PrimeGrid and my own Private GFN Server, I've encountered only second type of problem. This is a race condition with following scenario:
Alternate Designs
Of course, it would be nice if this problem was also fixed in client itself (check for boinc_finish before starting programs in slot directories for a first time after reboot). But it will not help to users with old clients.
Also, a wrapper or native Boinc application may check for boinc_finish itself and do exit(0) immediately. But it's not possible to update existing wrappers and applications.
So this server patch seems to be lowest of evils. First it was implemented on my Private GFN Server and works great. Every day, few tasks are silently recovered.
Release Notes
N/A