Add support for Offline HIBP password checker

[c4rlo]

The new keepassxc-cli hibpanalyze subcommand can check all passwords in the given database against a given list of SHA-1 password hashes (when --hibp is given). Such lists are available from the "Have I Been Pwned" (HIBP) project at https://haveibeenpwned.com/Passwords.

Example usage:

$ curl -Lo hashes.7z https://downloads.pwnedpasswords.com/passwords/pwned-passwords-sha1-ordered-by-hash-v4.7z
$ keepassxc-cli analyze --hibp <(7z x -so hashes.7z) passwords.kdbx
Insert password to unlock passwords.kdbx:
Password for entry 'foo' has been leaked 123 times!
Password for entry 'mygroup > bar' has been leaked 789 times!
Finished.

Type of change

• ✅ New feature (non-breaking change which adds functionality)

Description and Context

Have I Been Pwned is a widely used collection of leaked credentials. It is used by several other password managers.

There are several other variations of the functionality implemented in this PR that may be worth considering:

• Ability to access the functionality implemented in this PR from the GUI.
• Ability to access this functionality without having to download the entire password hash database (9.8 GiB compressed currently) first. This is possible using the HIBP pwned passwords API. To use it, one must send the first 20 bits of the hash up to the server, getting back a list of all matching hashes (generally a few hundred). This is a compromise that leaks a small-ish amount of information about the password to the HIBP server, while keeping data transfer volumes low.
• Ability to check all new or changed passwords against the HIBP API.

Testing strategy

The core logic (implemented in core/HibpOffline.cpp) has unit tests in tests/TestHibp.cpp.

What's still mostly missing is tests for the CLI-specific part.

Checklist:

• ✅ I have read the CONTRIBUTING document. [REQUIRED]
• ✅ My code follows the code style of this project. [REQUIRED]
• ✅ All new and existing tests passed. [REQUIRED]

[louib]

@c4rlo thanks for the contribution.

For the record, I opened a PR for that a few months ago, which I decided to not push through completion. Review comments from that PR might still be relevant (in particular the presence of the SHA1 algo in the standard crypto class).

Your PR would also address #1083.

One of the popular features currently requested is an integrated password analyzer, that would display information about duplicate passwords, expired passwords, passwords with low entropy, etc. Considering this, we could rename the command so that it allows us to add other metrics / stats. Something like keepassxc-cli analizer --hibp ?? Here's the password analyzer issue.

[c4rlo]

Thanks for the context, @louib -- I should have guessed this came up before :)

I'm happy to move the SHA-1 impl out of CryptoHash if that will help get this accepted.

As for the name of the subcommand, I'd be happy to go with analyze --hibp, especially if we think there will be further analysis modes added in future.

[droidmonkey]

Recommend not searching the Recycle Bin for passwords.

The actual implementation of the HIBP interface should not be contained in the CLI code. That should be added to Core. The interface class should have a meaningful name such as "HIBPInterface".

[c4rlo]

I believe I've addressed most comments now; see individual commits. Still left to do:

• Look into removing class Sha1Hash with its uchar m_data[20] and just using QByteArray instead. I might run some benchmarks first.
• Improve CLI unit tests (in addition to the lower-level unit tests we now have for the underlying logic).
• Add documentation.

[c4rlo]

Completed above items. This PR is ready for another round of review.

Btw TeamCity seems to have some issues...

[droidmonkey]

I completely missed that this requires to download the entire 9.8 GB password hash database. I find this to be completely untenable for the average user. I would much rather implement the online version instead of this.

[c4rlo]

I definitely think that supporting an online check would be very nice as well -- certainly more convenient in most cases. But the offline check has a few advantages also:

• Does not require Internet access, so works for people running keepassxc on an airgapped computer.
• Does not leak any information about the passwords. I don't really want to get into a discussion of whether leaking a few bits of a SHA-1 to HIBP is a problem or not, but some people may certainly feel uncomfortable with it.

As regards having to download a 9.8 GB file, I don't see that as a big problem as it's not something that would have to be done often.

[droidmonkey]

Ok, I'll do a deep dive on this pr once I'm satisfied with 2.4.1

[t4777sd]

For me personally I would like to have the option to download the data and run it offline.

[droidmonkey]

I made the necessary updates to merge this PR. Works great, but the HIBP file is HUGE (24 GB uncompressed) and takes about 5 minutes to crunch through on my laptop.

obsolete