Quick feedback on Windows rustic version

[deajan]

Hello,

Just happened to pop by your new 0.5.0 release of rustic. Thanks for the effort you put into it.
I was investigating if I could use it as an alternative backend for my project at https://github.com/netinvnet/npbackup which is basically a restic CLI / GUI with prometheus support.

So I gave it a quick test run on my windows environment:

Local backups comparaison:

• restic

SET RESTIC_REPOSITORY=c:\restic
SET RESTIC_PASSWORD=password
restic init --repository-version 2 --compression auto
restic backup c:\git\npbackup

repository e8085438 opened (version 2, compression level auto)
created new cache in C:\Users\Orsiris\AppData\Local\restic
no parent snapshot found, will read all files

Files:       16897 new,     0 changed,     0 unmodified
Dirs:         1183 new,     0 changed,     0 unmodified
Added to the repository: 1.155 GiB (620.145 MiB stored)

processed 16897 files, 1.876 GiB in 0:04
snapshot 2f8fb843 saved

• rustic

SET RUSTIC_REPOSITORY=c:\rustic
SET RUSTIC_PASSWORD=password
rustic init --set-version 2
rustic backup c:\git\npbackup
using no config file (.\rustic.toml doesn't exist)
[INFO] repository c:\rtoto: password is correct.
[INFO] using cache at C:\Users\SomeUser\AppData\Local\rustic\9ed2a3773ffa433ee92f420dbe3175cacc0887b79453c171f23f499cc79ebde8
[00:00:00] reading index...               ████████████████████████████████████████          0/0                                                                                                                                              [INFO] starting to backup "C:\\GIT\\npbackup"...
[00:00:00] getting latest snapshot...     ████████████████████████████████████████          0/0                                                                                                                                              [INFO] using no parent
[00:00:22] backing up...                  ████████████████████████████████████████   1.88 GiB/1.88 GiB   85.13 MiB/s  (ETA 0s)                                                                                                               Files:       16897 new, 0 changed, 0 unchanged
Dirs:        1182 new, 1 changed, 0 unchanged
Added to the repo: 612.7 MiB (raw: 1.1 GiB)
processed 16897 files, 1.9 GiB
snapshot bf01e4ac successfully saved.
[INFO] backup of "C:\\GIT\\npbackup" done.

So basically size is the same, but:

• restic uses approx 450MB of RAM whereas rustic uses up to 90MB
• restic is currently 5x faster than restic

I've a also tried to read a rest-server remote repository.
While I can read it with restic, I get the following results when trying rustic:

SET RUSTIC_REPOSITORY=rest:https://USER---COMPUTER:LongPassword@my.remote.server.tld/USER---COMPUTER
SET RUSTIC_PASSWORD=SomeReallyLongPasswordIUseWithRestic

rustic ls latest
using no config file (.\rustic.toml doesn't exist)
[WARN] Error error sending request for url (https://my.remote.server.tld/USER---COMPUTER/config): connection error: An existing connection was forcibly closed by the remote host. (os error 10054) at 539.476337ms, retrying

I've disabled IPv6 and got quite similar results, but with much more "connection closed" messages.

rustic ls latest
using no config file (.\rustic.toml doesn't exist)
[WARN] Error error sending request for url (https://my.remote.server.tld/USER---COMPUTER/config): connection closed before message completed at 737.243103ms, retrying
[WARN] Error error sending request for url (https://my.remote.server.tld/USER---COMPUTER/config): connection closed before message completed at 1.039009181s, retrying
[WARN] Error error sending request for url (https://my.remote.server.tld/USER---COMPUTER/config): connection closed before message completed at 1.149513485s, retrying
[WARN] Error error sending request for url (https://my.remote.server.tld/USER---COMPUTER/config): connection error: An existing connection was forcibly closed by the remote host.. (os error 10054) at 2.033619738s, retrying
[WARN] Error error sending request for url (https://my.remote.server.tld/USER---COMPUTER/config): connection closed before message completed at 1.741580498s, retrying

rest-server doesn't show any special messages in log file.
Does rustic support rest-server (0.11.0) as of version 0.5.0 ?

[aawsome]

Thanks a lot for your feedback @deajan

About memory and speed: Rustic is much more memory optimized, so the lower memory usage is expected. rustic however doesn't read and process new files in parallel as restic does. I need to improve this. However, if you do a second backup with many files unchanged, rustic should be at least equal fast as restic.

About your rest backend: Rest backends are of course supported.
Judging from the error message this seems to be a ssl connection problem - at least this is what google returned from searching the error code.
Can you try:

• connect without ssl, i.e. using http:// instead of https://?
• connect with a linux/macos version of rustic to the same rest-server? I would like to know if this is a windows-specific problem...

[deajan]

Thanks for your input. Indeed, I've found the culprit.
rustic does not send a user-agent, whereas I filter out any SSL request that doesn't have a user-agent set ;)
I've temporarily disabled that rule in my proxy, now rustic behaves correctly with my rest repo.

Would you mind adding a "rustic-rs" or so user-agent to your requests ? This helps knowing who knocks on the door.

Btw, when listing snapshots, rustic will output the username and password, which would look bad in log files I think.
Eg:

rustic list snapshots
[INFO] repository rest:https://USERNAME:PASSWORD@my.remote.computer.tld/USERNAME: password is correct.

I think this should definitly be filtered out like restic wihch returns

restic list snapshots
repository xxxxxx opened (version2, copression level auto)

[deajan]

I have redone the same (local) backup tries as above, but did a second backup without adding any file.

rustic is still slow for the file checksumming (understood, this is not a big culprit), but says it processed all files as new:

Output of 2nd run with rustic

using no config file (.\rustic.toml doesn't exist)
[INFO] repository c:\rustic: password is correct.
[INFO] using cache at C:\Users\Orsiris\AppData\Local\rustic\4e105828ba3534b5ef179b90aed7ad1b8a9f0dbf453955f2fdeffd55e3c63edf
[00:00:00] reading index...               ████████████████████████████████████████          2/2                                                                                                                                              [INFO] starting to backup "C:\\GIT\\npbackup"...
[00:00:00] getting latest snapshot...     ████████████████████████████████████████          2/2                                                                                                                                              [INFO] using parent 9f6d5ed8
[00:00:16] backing up...                  ████████████████████████████████████████   1.99 GiB/1.99 GiB   123.16 MiB/s (ETA 0s)                                                                                                               Files:       19590 new, 0 changed, 0 unchanged
Dirs:        1508 new, 1 changed, 0 unchanged
Added to the repo: 0 B (raw: 0 B)
processed 19590 files, 2.0 GiB
snapshot a0ec9c09 successfully saved.
[INFO] backup of "C:\\GIT\\npbackup" done.`

Output of 2nd run with restic

repository 7e097298 opened (version 2, compression level auto)
using parent snapshot ab08a7a7

Files:           0 new,     0 changed, 19590 unmodified
Dirs:            0 new,     0 changed,  1509 unmodified
Added to the repository: 0 B   (0 B   stored)

processed 19590 files, 1.992 GiB in 0:01
snapshot d24b4cf5 saved

In the end, rustic said it added 0B to the repository, but still said of the 19590 files that it processd, 19590 are new.

[aawsome]

Thanks @deajan for the second backup test. There is in fact something wrong here on windows. Maybe the parent processing needs some adaption on windows. However, this behavior explains why rustic is also pretty slow on the second run.

[aawsome]

Btw, when listing snapshots, rustic will output the username and password, which would look bad in log files I think.
Eg:

I opened #534 for this.

[deajan]

I opened #534 for this.

Yes, sorry for having piggybacked this issue with multiple problems.
Is it still possible to add a user-agent to rustic ?

[aawsome]

Thanks @deajan for the second backup test. There is in fact something wrong here on windows. Maybe the parent processing needs some adaption on windows. However, this behavior explains why rustic is also pretty slow on the second run.

Can you try a second backup run using the --ignore-ctime option?

[deajan]

Re-ran same backup with --ignore-ctime option, result is identical, all files and dirs are marked new.

[aawsome]

Thanks a lot for the continued testing.
I think this has something to do with the path processing. Can you run a rustic ls latest and give the first couple of lines which are given?
Also would you mind retesting with relative paths, i.e. chdir to the root (cd C:\GIT) and backup the dir without any leading .s or \ (rustic backup npbackup) and do this twice -> It should recognize that all files are unchanged now.

[deajan]

Here's rustic ls latest output:

C:\GIT\npbackup\RESTIC_SOURCE_FILES>rustic_0.15.1_windows_amd64.exe ls latest
using no config file (.\rustic.toml doesn't exist)
[INFO] repository c:\rustic: password is correct.
[INFO] using cache at C:\Users\Orsiris\AppData\Local\rustic\4e105828ba3534b5ef179b90aed7ad1b8a9f0dbf453955f2fdeffd55e3c63edf
[00:00:00] getting latest snapshot...     ████████████████████████████████████████          5/5                                                                                                                                              [00:00:00] reading index...               ████████████████████████████████████████          3/3                                                                                                                                              "GIT"
"GIT\\npbackup"
"GIT\\npbackup\\.eggs"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\EGG-INFO"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\EGG-INFO\\LICENSE"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\EGG-INFO\\PKG-INFO"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\EGG-INFO\\RECORD"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\EGG-INFO\\WHEEL"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\EGG-INFO\\top_level.txt"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\_yaml"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\_yaml\\__init__.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\__init__.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\_yaml.cp310-win_amd64.pyd"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\composer.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\constructor.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\cyaml.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\dumper.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\emitter.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\error.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\events.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\loader.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\nodes.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\parser.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\reader.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\representer.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\resolver.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\scanner.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\serializer.py"
"GIT\\npbackup\\.eggs\\PyYAML-6.0-py3.10-win-amd64.egg\\yaml\\tokens.py"
"GIT\\npbackup\\.eggs\\README.txt"
"GIT\\npbackup\\.eggs\\python_i18n-0.3.9-py3.10.egg"
[...]

Indeed, the drive letter part is missing.
I've run rustic with relative paths twice, and on second run rustic detected the files properly:

C:\GIT>rustic backup npbackup
using no config file (.\rustic.toml doesn't exist)
[INFO] repository c:\rustic: password is correct.
[INFO] using cache at C:\Users\User\AppData\Local\rustic\4e105828ba3534b5ef179b90aed7ad1b8a9f0dbf453955f2fdeffd55e3c63edf
[00:00:00] reading index...               ████████████████████████████████████████          4/4                                                                                                                                              [INFO] starting to backup "npbackup"...
[00:00:00] getting latest snapshot...     ████████████████████████████████████████          6/6                                                                                                                                              [INFO] using parent 17ea4df1
[00:00:01] backing up...                  ████████████████████████████████████████   2.04 GiB/2.04 GiB   1.29 GiB/s   (ETA 0s)                                                                                                               Files:       0 new, 0 changed, 17670 unchanged
Dirs:        0 new, 1 changed, 2801 unchanged
Added to the repo: 0 B (raw: 0 B)
processed 17670 files, 2.0 GiB
snapshot 3903ac8e successfully saved.
[INFO] backup of "npbackup" done.

[deajan]

I'm considering adding rustic to my backup benchmark system at https://github.com/deajan/backup-bench
Would you mind filling the in depth comparison table with rustic ?

Also, I'm investigating to use rustic as alternative backend for my backup program at https://github.com/netinvent/npbackup
So far, I've done a couple of successful tests, and the GAP to use rustic isn't that large.
I'm trying to make my program a first class citizen in windows too by:

• Providing Authenticode signed pre-built executables
• Adding a end user GUI
• Handling cloud files (reparse points)
• Having big exclusion lists for windows directories
• etc

So far, the only thing that keep me from integrating rustic is the lack of VSS support. Is this planned ?

[aawsome]

Indeed, the drive letter part is missing.

Ok, this seems to be a bug in the windows version and most likely explains why the parent dirs are not detected and not used to speed up the second backup. I'll need to investigate this...

[aawsome]

Ok, this seems to be a bug in the windows version and most likely explains why the parent dirs are not detected and not used to speed up the second backup. I'll need to investigate this...

This should be now fixed with #542. Is already available in the beta builds.

[aawsome]

So far, the only thing that keep me from integrating rustic is the lack of VSS support. Is this planned ?

I added this to the open points in rustic-rs/rustic_core#19.

[deajan]

#542 confirmed to work ;)
Thanks for the link to rustic-rs/rustic_core#19, I'll be looking forward to use rustic windows ;)

[simonsan]

I extracted the request for filling out the in-depth comparison table. With #542 confirmed to work, I think this can be closed, feel free to reopen in case anything is still not fixed.