Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix that a HTTP 504 - Gateway Timeout causes local files to be deleted when using --download-only & --cleanup-local-files mode #2342

Merged
merged 4 commits into from
Mar 18, 2023
Merged

Fix that a HTTP 504 - Gateway Timeout causes local files to be deleted when using --download-only & --cleanup-local-files mode #2342

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a670063
Add further debugging for Issue 2338
abraunegg Mar 17, 2023
23b326a
Merge branch 'master' into fix-issue-2338
abraunegg Mar 17, 2023
ea9483b
Update sync.d
abraunegg Mar 17, 2023
af3ed17
Update sync.d
abraunegg Mar 18, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions src/sync.d
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1798,20 +1798,49 @@ final class SyncEngine
log.vdebug("Retrying Query: changesAvailable = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLinkAvailable)");
changesAvailable = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLinkAvailable);
log.vdebug("Query 'changesAvailable = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLinkAvailable)' performed successfully on re-try");
if (changesAvailable.type() == JSONType.object) {
// are there any delta changes?
if (("value" in changesAvailable) != null) {
deltaChanges = count(changesAvailable["value"].array);
log.log("changesAvailable query reports that there are " , deltaChanges , " changes that need processing on OneDrive");
}
}
} catch (OneDriveException e) {
// display what the error is
log.vdebug("Query Error: changesAvailable = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLinkAvailable) on re-try after delay");
if (e.httpStatusCode == 504) {
log.log("OneDrive returned a 'HTTP 504 - Gateway Timeout' when attempting to query for changes - retrying applicable request");
log.vdebug("changesAvailable = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLinkAvailable) previously threw an error - retrying with empty deltaLinkAvailable");
// Increase delay and wait again before retry
log.vdebug("Thread sleeping for 90 seconds as the server did not receive a timely response from the upstream server it needed to access in attempting to complete the request");
Thread.sleep(dur!"seconds"(90));
log.vdebug("Retrying Query - using a null deltaLinkAvailable after delay");
try {
// try query with empty deltaLinkAvailable value
deltaLinkAvailable = null;
changesAvailable = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLinkAvailable);
log.vdebug("Query 'changesAvailable = onedrive.viewChangesByItemId(driveId, idToQuery, deltaLinkAvailable)' performed successfully on re-try");
if (changesAvailable.type() == JSONType.object) {
// are there any delta changes?
if (("value" in changesAvailable) != null) {
deltaChanges = count(changesAvailable["value"].array);
log.vdebug("changesAvailable query reports that there are " , deltaChanges , " changes that need processing on OneDrive when using a null deltaLink value");
}
}
} catch (OneDriveException e) {
// Tried 3 times, give up
displayOneDriveErrorMessage(e.msg, getFunctionName!({}));

// OK .. if this was a 504, and running with --download-only & --cleanup-local-files
// need to exit to preserve local data, otherwise potential files will be deleted that should not be deleted
// leading to undesirable potential data loss scenarios
if ((e.httpStatusCode == 504) && (cleanupLocalFiles)) {
// log why we are exiting
log.log("Exiting application due to OneDrive API Gateway Timeout & --download-only & --cleanup-local-files configured to preserve local data");
// Must exit here
onedrive.shutdown();
exit(-1);
}
return;
}
} else {
Expand Down