Release 1.3.0a1 - alpha quality, for testing only

[ThomasWaldmann]

borgbackup 1.3.0a1 alpha release - do not use this on production backup repositories.

Long changelog:
https://borgbackup.readthedocs.io/en/1.3.0a1.post3/changes.html#version-1-3-0a1-2022-04-15

If you use pip to install this, use: pip install "borgbackup==1.3.0a1"

Docs: https://borgbackup.readthedocs.io/en/1.3.0a1.post3/

---

This discussion was created from the release Release 1.3.0a1 - alpha quality, for testing only.

[ThomasWaldmann]

Note: there was an issue with the docs build, this is why the docs are tagged ".post3".

[FabioPedretti]

I tried borg info with a 1.1 created repo and got an error:

# ./borg-1.3.0a1 info
/mnt/sshfs/borg-repo is not a valid repository. Check repo config.

1.2.0 is fine:

# ./borg-1.2.0 info
Repository ID: XXXX
Location: /mnt/sshfs/borg-repo
Encrypted: Yes (repokey BLAKE2b)
Cache: /root/borg-cache/XXXX
Security dir: /root/.config/borg/security/XXXX
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
All archives:              693.50 TB            501.69 TB              2.57 TB

                       Unique chunks         Total chunks
Chunk index:                 5354981           1172346072

Is this expected?

[FabioPedretti]

OK, no problem, and thanks for the work on the new release!
(I'll test the speed with a new repo (with source on NFS) in the following days.)

[ThomasWaldmann]

For speed tests, try the borg benchmark cpu first and evaluate its output before doing borg init. If you have the repo via ssh: also run the benchmark on the repo server for the crc32 speed.

[FabioPedretti]

I'll run it with -e none for a fair comparison, given that source file read on NFS (my use case) is likely order of magnitude slower than everything else (#4498).

[FabioPedretti]

With 1.3.0a1 I get similar performance of 1.2.0 with source on NFS, no surprise I think.

[ThomasWaldmann]

Most changes were in the crypto parts, so yes, if you disable encryption, no change expected.

[FabioPedretti]

borg benchmark cpu on a 10 years old Intel(R) Xeon(R) CPU E7- 4830 @ 2.13GHz running under a VM (along other 15 VM on the same host...):

# ./borg-1.3.0a1 benchmark cpu
Chunkers =======================================================
buzhash,19,23,21,4095    1GB        2.660s
fixed,1048576            1GB        0.195s
Non-cryptographic checksums / hashes ===========================
crc32 (libdeflate, used) 1GB        0.539s
crc32 (zlib)             1GB        1.519s
xxh64                    1GB        0.219s
Cryptographic hashes / MACs ====================================
hmac-sha256              1GB        5.970s
blake2b-256              1GB        3.477s
Encryption =====================================================
aes-256-ctr-hmac-sha256  1GB        8.052s
aes-256-ctr-blake2b      1GB        8.132s
aes-256-ocb              1GB        1.503s
chacha20-poly1305        1GB        2.937s
KDFs (slow is GOOD, use argon2!) ===============================
pbkdf2                   5          0.635s
argon2                   5          0.785s
Compression ====================================================
lz4          0.1GB      0.099s
zstd,1       0.1GB      0.216s
zstd,3       0.1GB      0.254s
zstd,22      0.1GB      31.751s
zlib,0       0.1GB      0.200s
zlib,6       0.1GB      6.112s
zlib,9       0.1GB      6.388s
lzma,0       0.1GB      31.070s
lzma,6       0.1GB      66.998s
lzma,9       0.1GB      54.765s
msgpack ========================================================
msgpack      100k Items 0.316s

[FabioPedretti]

Maybe adding some more zstd test levels between 3 and 22?

[FabioPedretti]

I run the test multiple times, it is reproducible, just some minor fluctuations. Strange that lzma,9 is faster than lzma,6...

[ThomasWaldmann]

More levels: yeah, could be done. what's there is: min, default, max.

[FabioPedretti]

It could be useful given that 22 is > 100x slower than 3. Maybe adding at least a couple more like 6 and 12 or something like that.

[ThomasWaldmann]

https://github.com/borgbackup/borg/pull/6610/files

[FabioPedretti]

Minor formatting issue, the second line of the table (the one starting with |mode) is not rigth aligned correctly:

./borg-linux64 help init
[...]
+---------------------------------+---------------+---------------+------------------+-------+
|mode (* = keyfile or repokey)|ID-Hash    |Encryption |Authentication|V>=|
+---------------------------------+---------------+---------------+------------------+-------+
| *-blake2-chacha20-poly1305      | BLAKE2b       | CHACHA20      | POLY1305         | 1.3   |
+---------------------------------+---------------+---------------+------------------+-------+
[...]

[ThomasWaldmann]

#6609

[FabioPedretti]

Any reason not to use BLAKE3 rather than BLAKE2b on the new 1.3 modes?

[ThomasWaldmann]

I'm still waiting for @oconnor663 to review and merge some of my PRs, so that py-blake3 (C implementation) works on macOS m1 and python < 3.10 also. Well, and it would be good to also have the C implementation on PyPI, so that one does not need rust to build it.

But then I guess nothing is holding us back. :-)

[ThomasWaldmann]

https://github.com/oconnor663/blake3-py/pulls

[ThomasWaldmann]

oconnor663/blake3-py#30

[FabioPedretti]

Which OCM mode is used? OCB1, OCB2 or OCB3?

[ThomasWaldmann]

I tried to find openssl docs about this, but couldn't.

I suspect it is OCB3 though, because everything else would be stupid / unsecure.