[Client Tooling] Keybase (issue #455)

[h5law]

Description

This is an implementation of a Keybase for the pocket V1 client.

It uses BadgerDB as the local database for the persistent storage of keys. The current Keybase interface has the following methods

These are covered with the following tests

The tests use an in-memory database created with NewKeybaseInMemory() whereas when exposed to the user the CLI should call NewKeybase() instead with a default path/user-supplied path. The CLI integration has not been implemented as part of this PR however.

How it works

The keybase heavily relies upon shared/crypto/ed25519.go and the PublicKey and PrivateKey interfaces it exposes.

The keys are created using the methods from this library. Keys are then encrypted and armoured in the same fashion as in V0 and stored in the DB as a KeyPair struct (encoded to a []byte).

This struct contains the public key and the encrypted JSON encoded private key string. This struct has the following methods:

• GetAddressBytes() []byte -> returns []byte of the public key address
• GetAddressString() string -> returns the hex string of the public key address
• Unarmour(passphrase string) (crypto.PrivateKey, error) -> returns the unencrypted unarmoured private key if passphrase is correct
• ExportString(passphrase string) (string, error) -> returns raw private key string is passphrase is correct
• ExportJSON(passphrase string) (string, error) -> returns json armoured private key string if the passphrase is correct

Keys are stored in the database in the following manner:

• The keys for the BadgerDB key-value storage are the []byte value returned by KeyPair.GetAddressBytes() aka the []byte address of the public key
• The values are the []byte encoded KeyPair structs of the key

Key encryption/decryption works with or without a passphrase (when no passphrase is given "" must still be passed to the function as the passphrase argument)

For ease of use the hex string returned by KeyPair.GetAddressString() is used to access the keys in storage.

Importing and exporting keys in either JSON string format or the private key hex string is fully interoperable between V1 and V0

Signing and Verification of messages works - and is covered with a Tx in the tests. I would need to look into the use cases for this more (for example multisig) as the current implementation is very rudimentary and in theory should work for producing a signature on any []byte message but I would like to look into this more.

Issue

Fixes #455

Type of change

Please mark the relevant option(s):

• New feature, functionality or library
• Bug fix
• Code health or cleanup
• Major breaking change
• Documentation
• Other

List of changes

• Implement a keybase with BadgerDB local DB
• Add functionality to Import/Export keys, Get/List keys, Create new keys, Delete Keys, Sign Messages, Verify Signatures
• Add make test_app entry point in Makefile
• Add unit tests to cover Keybase use cases
• Add documentation for Keybase

Testing

• make develop_test
• LocalNet w/ all of the steps outlined in the README

Required Checklist

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have tested my changes using the available tooling
• I have updated the corresponding CHANGELOG

If Applicable Checklist

• I have updated the corresponding README(s); local and/or global
• I have added tests that prove my fix is effective or that my feature works
• I have added, or updated, mermaid.js diagrams in the corresponding README(s)
• I have added, or updated, documentation and mermaid.js diagrams in shared/docs/* if I updated shared/*README(s)

[h5law]

Thanks to @jessicadaugherty for giving me a hint to what might be the issue - HEX ENCODING!. V0's ed25519 library has 2 extra functions for the PrivateKey type RawBytes() and RawString() I previously ignored these as they are not in the V1 library of the same name but they were the key to this issue.

Import/Export JSON is fully interoperable between V0 and V1 now:

When encrypting the private key now convert the hex encoded private key string to []byte

This is instead of directly getting the result of PrivateKey.Bytes() to match how V0 does the same task - although this extra conversion step does nothing for security it changes the length of the cipher text in the JSON encoded string.

Now when decrypting we must do another conversion/decoding step:

And the private key is then created from bz. When originally implementing this I thought this was weird and instead used the more direct approach (in my opinion) however this made the encrypted cipher text much shorter (despite representing the same values). This is now fully interoperable with V0 and is covered in a unit test with an account generated by wallet.pokt.network (manually testing the reverse works by t.Log(jsonStr) in the ExportJSON test works too)

[Olshansk]

This is really impressive work, in terms of quality, speed and impact.

Regarding your comment that fixes the V0 <-> V1 conversion. Tbh, I'm not a fan of all the data transformations, and I'm sure there's some redundancy since we ported some of the crypto library logic in v0.

If you see opportunities to clean it up in the future, just go for it.

I will follow up in #455 regarding some next steps on this & other tickets.

[h5law]

@Olshansk I have left a reply to the tx.Delete() comment you made - personally I think it is better to stick to using the Update() transaction wrapper provided by BadgerDB as 1) It fits the pattern 2) it is less lines of code (it handles commit for us). I can change this if you really want the explicit tx.Delete() outside of a wrapper but this is why I didn't change it first time.

[Olshansk]

@h5law Almost over the finish line.

Left a couple small comments but also took the liberty to make some updates to the README while reading through it.

Take a look and fill free to update anything I missed or misrepresented.

Should be good to go afterwards.

[Olshansk]

One small comment but otherwise LGTM 🔥