How to set compression on backup

[KamikazeePL]

In restic I can use --compression option to set compression level when I'm making backup files.

How to set compression level in rustic when I make backup?

[aawsome]

In rustic, compression level is a repository option, i.e. it is saved in the repository config and needs to be set only once (or when you want to change it).

You can already set it when initializing a repository and change it using the command rustic config --set-compression.

[Schweber]

I can set

[repository]
set-compression = 9

in the config and rustic doesn't throw an error. Is this setting respected by rustic or silently disregarded?

As it's possible to set

[backup]
init = true

i would like to set the compression-level in the config as well.

[aawsome]

@Schweber this doesn't work. Use rustic config --set-compression.

[Schweber]

Ok, thanks

[aawsome]

Also note, that there is no max compression level, but you can use the compression levels used by zstd, see rustic config --help for more information.

[zxcat]

is it possible to view repo config?

• rustic config --help only shows --set-* flags, not --get-*
• and restic repoinfo doesn't show any config info

[aawsome]

Yes, you can use rustic cat config. This will, however, only show the raw json of the config file and not the default values used if an item is missing...
If anyone is up with a PR to also show the values, we'll happily merge that!

[LeoniePhiline]

For people looking to pick the right compression level:

At zstd compression levels > 3, throughput becomes too slow to be worth the little gain in compression ratio in most cases.

Borg uses zstd compression level 3 by default.

[simonsan]

@aawsome that's something we should adapt as well, I guess. Or at least have something in the docs about it

[aawsome]

@LeoniePhiline Thanks for the information!
I must say I did not make any measurements with different compression levels. I think it also depends about what you are trying to optimize for.

So if you optimize for backup speed, with a remote repository the backup may depend a lot on the connection to your storage backend. If you have a slow connection, but a fast CPU, higher compression rates can therefore lead to higher backup speed. For fast connections or backups to local storage, you may be better of with a low compression.

Besides this, people might optimize for other things like total cost where a higher compression helps, of course.

[LeoniePhiline]

Yes. I don't have links handy right now, but you can find benchmark charts online which show only a slight increase of compression ratio paired with a much larger increase in processing time for levels above 3.
I would expect people who want a small repo, but not a really really really super small repo to go with about lvl 3. Since disk space is cheap nowadays, and speed makes a difference, many might like to lower the level for worse compression but faster processing.

I had just wanted to mention this because zstd goes up to lvl 22, but most of these levels are not useful for everyday cases - which is not intuitive at all. Without charts (plotting compression-ratio-to-processing-time per each available level), the levels mean nothing to users.

[KamikazeePL]

https://peazip.github.io/fast-compression-benchmark-brotli-zstandard.html

[LeoniePhiline]

https://peazip.github.io/fast-compression-benchmark-brotli-zstandard.html

Note that these zstd compression levels are prefixed - as if they were arguments, not as if they were negative numbers. The levels discussed here are all > 0, even though they show up in the table (see below) looking like negative numbers.

The fastest (< 0 / negative level number) zstd compression levels, which are faster and provide lower compression ratio than level 1, were introduced later.

[aawsome]

@aawsome that's something we should adapt as well, I guess. Or at least have something in the docs about it

Actually the default compression level (i.e. if not explicitely set by the user) is taken from the zstd crate. Had a quick look:
https://docs.rs/zstd-sys/latest/zstd_sys/constant.ZSTD_CLEVEL_DEFAULT.html

The zstd crate (and therefore also rustic) uses 3 as default level.