-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Warn about missing permissions before attempting to back up to a repository #1128
Comments
@Cookie04DE Thanks for opening this issue! I have a question: How should rustic check if the repository is writable? The point is, currently writing fails after rustic tries to write something to the repository, so we must write something to repository in order to check if it is writable. Now, this poses several problems:
When I read your issue I had two point in mind:
|
If compatible with current constraints, I would suggest to make rustic open the necessary files in write mode at the very start of the backup produce without writing anything to them yet. If that fails, it can fail fast; otherwise proceed, gather the backup data and actually attempt to write it. Otherwise, rustic could query the file permissions of the repo and cross-check them with the current uid/gid (don't know how compatible that approach is with different backends; or if false positives could occur). I'd be glad to provide you with the requested information. How can I obtain it? Is it just the regular output, or do I need to specify some environment variables or command line flags? However, to provide actually realistic values I would wait until the next regular scheduled date (next Saturday probably), if that's ok with you. Regarding your second point: Since I have now added my mounts to the fstab, I don't expect this to be a problem anymore. This having been a regular annoyance of mine can be chalked up to my laziness up to this point and I don't think I actually need a pre-backup hook or other custom checks for this specific case. Apart from that, I want to express my gratitude towards you and the other maintainers of this project for donating their time. I use it quite often, and it's very useful to me. Thanks! |
I fear this is totally incompatible with rustic's architecture and fundamentals. About checking permissions: First, this would be a local-Backend-only thing or at least need very backend-specific implementations, which we don't like to have as we focus on backup and have very few self-written code in rustic_backend (in fact moving most work to supporting libraries like opendal). So I'd strictly vote against that for the unified-backend reason. But there is IMO another issue: if we do whatever check we do, there might be other cases where writing still fails. Just think about full disc or additional ACLs provided by some specific file system. We would get more and more requests about pre-checking things before we write - while the writing still could fail for another reason we are not yet checking. Moreover, we would have to be very precise as false positives in those checks would be even more annoying than a late and correct write-error. In my opinion such extra checks are widely out of scope for rustic. TL;DR: Better report true errors instead of trying to guess which errors could occur. I'm still very interested in your reasons for the long time until rustic tries to write something! |
I understand the issue and also what @aawsome is saying, I do wonder, though, if it could be sensible to use a "throw-away" file as an indicator if something is writable. E.g. touch |
@simonsan See my first answer. I would strongly vote against such a "throw-away" or garbage file. However we could add an an optional feature to mimic restic's locking (create a lock file in the beginning and removes it in the end) - which would also increase compatibility for people who want to use restic and rustic on the same repository. That said, if you look at the restic forum, there have been tons of issues just because of not correctly removed lock files. I am actually very happy that we don't have to mess with these things in rustic, currently. |
Ok, I got the output. Repo on HDD 1:
Repo on HDD 2 (bad mount):
The bulk of the time was spent on the line after Repo on HDD 2 (good mount):
Repo on Flash drive (bad mount):
Again most of the time spent on the process described previously. Repo on Flash drive (good mount):
|
@Cookie04DE Thanks for your results. So, yes, you have many new or changed files which explains why pretty much data is written. And I guess you have quite a large pack files size which is also good in principle, but of course rustic waits until a pack file is full or the backup ends before writing to the repository. So, altogether your behavior can be explained, but I fear an early write-test is only possible once someone implements optional locking for rustic. Currently, I can only suggest to run the backup (much) more often. For only few changed or new files, the backup is much faster and you'll get faster feedback. |
I am frequently using rustic to back up to different types of file systems: ext4 and exFAT for example.
I have until now been to lazy to add the devices to the fstab, so I usually mount the devices manually.
With ext4 that causes no issues, but for file systems without permission data, if you forget to include
uid=<your user id here>,gid=<your group id here>
in the mount options the file system will get mounted with all files belonging to the root user.Now this is typically caught pretty early. In the case of rustic however, the missing permission error occurs at the very end of the backup procedure. Thus, it has already spent quite a bit of time to gather all the data to be committed to the repository before it attempts to write them and fails with a permission denied error.
This is frustrating to the user, because it means they have to redo the whole thing, this time with the correct mount options and wait again until the backup procedure has gathered all the relevant data.
I propose to add a check at the very start of the backup procedure, that checks if writing to the repo is even possible.
The text was updated successfully, but these errors were encountered: