Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/nar 1595 vault root key generation derivation #268

Merged
merged 18 commits into from
May 28, 2024

Conversation

Ptroger
Copy link
Contributor

@Ptroger Ptroger commented May 22, 2024

No description provided.

Copy link

linear bot commented May 22, 2024

NAR-1595 Vault Root Key Generation & Derivation

Each Vault can have any number of Mnemonics, Root Keys or Private Keys within it. If a Root Key is generated within the vault, it should contain a backup package.

Add Backup Encryption Key to Vault Tenant

A tenant can (optionally) have a backupPublicKeyprovided at provision; this key will be used to encrypt any Vault-generated keys for backup.

  • Add backupPublicKey to the Vault Client Create (/clients) method & store w/ client config details
  • Add a allowKeyExport: boolean flag to the Client Create method & store w/ client config details.
    • This is used to allow future exports.

Key Generation

Endpoint: POST /generate-key

Body params

  • curve: optional, only supports secp256k1
  • alg: optional, only supports ecdsa
  • keyId: optional, unique identifier for this key

Response body

  • keyId: string (the root seed keyId)
  • wallet: object, the first derived wallet at m/44'/60'/0'/0/0
    • resourceId: string (the derived-wallet resourceId)
    • publicKey: hex
    • address: hex
    • derivationPath: string path of this wallet
  • backup: (optional) string. encrypted backup of the root seed phrase

Requirements

  • Generate a new Mnemonic
  • Derive the first eth address on it (m/44'/60'/0'/0/0)
  • Format the resourceId, account, and publicKey for the wallet.
  • Store the mnemonic
  • Store the derived wallet pk
  • If the Client has a backupPublicKey, Encrypt the mnemonic with that key & return.
    • Store the encrypted backup in the database, in a new table, not in the KV store.
      • This is not encrypted w/ the vault's encryption key, it's "plaintext" since it's encrypted by the Client's encryption key. This allows us to store/export this encrypted backup even if the vault's encryption gets lost.
  • See below for KV data models for wallet/seed, and prisma schema for backup table

Future work

  • a Set Backup Key endpoint, usable if no backupPublicKey was provided at provision.
  • a Export Key endpoint, usable if allowKeyExport is true; encrypts a seed phrase w/ the backupPublicKey & returns it
    • These together allow a Vault to provision without knowing the backup key, and later the client can add the key & export/backup their stuff. It speeds up onboarding while not creating an extra export vulnerability.
    • If allowKeyExport: false, a generated key can never be exported.

Key Derivation

Endpoint: POST /derive-wallet

Body params

  • keyId: ID of the root seed to derive from
  • derivationPath: string path of the account to derive. Default to "next". Accept an array to derive multiple.

Response body

  • wallet: object | array of objects
    • resourceId: string (the derived-wallet resourceId)
    • publicKey: hex
    • address: hex
    • derivationPath: string path of this wallet

Requirements

  • Derive 1 or more accounts at the given path on the given seed
  • Store the wallets
  • If already derived, no-op; don't error on duplicate derivations, it's fine.

Import Seed

Endpoint: POST /import/seed

body params

  • keyId: (optional) string unique id for this seed phrase
  • encryptedSeed: string encrypted seed phrase, must be encrypted w/ the wrapping key from /import/encryption-key

response body

  • keyId: string (the root seed keyId)
  • wallet: object, the first derived wallet at m/44'/60'/0'/0/0
    • resourceId: string (the derived-wallet resourceId)
    • publicKey: hex
    • address: hex
    • derivationPath: string path of this wallet
  • backup: (optional) string. encrypted backup of the root seed phrase

Requirements

  • Derive the first eth address on it (m/44'/60'/0'/0/0)
  • Format the resourceId, account, and publicKey for the wallet.
  • Store the mnemonic, include an imported flag so we know it was not generated
  • Store the derived wallet pk

We DO return a backup of this seed simply so it follows the same rules & they can recover a full vault from one place.

Note: the Vault can have multiple generated OR imported root seeds, as well as imported PKs (non-root). Signing calls will reference individual wallet pks not root keys. In the future, we may extend to allow signing to take root keyId + derivation path so you do not have to derive first.

Data Models

key-value data schemas

(update wallet.schema.ts)

export const walletSchema = z.object({
  id: z.string().min(1),
  privateKey: z
    .string()
    .regex(/^(0x)?([A-Fa-f0-9]{64})$/)
    .transform((val: string): Hex => val as Hex),
  publicKey: z
    .string()
    .regex(/^(0x)?([A-Fa-f0-9]{64})$/)
    .transform((val: string): Hex => val as Hex),
  address: z
    .string()
    .regex(/^0x([A-Fa-f0-9]{40})$/)
    .transform((val: string): Hex => val as Hex),
  // root seed key id
  keyId: z.string().min(1).optional(),
  // If this is derived from a root seed key, this is the derivation path
  derivationPath: z.string().min(1).optional()
})

export const rootKeySchema = z.object({
  keyId: z.string().min(1),
  mnemonic: z.string().min(1),
})

Prisma

model Backup {
  id                  String   id default(uuid())
  clientId            String   map("client_id")
  backupPublicKeyHash String   map("backup_public_key_hash") // sha256 of the publicKey that was used to encrypt this backup
  keyId               String   map("key_id") // Root Key Id that is backed up
  data                String // Encrypted backup data, as a jwe string
  createdAt           DateTime default(now()) map("created_at")

  map("backup")
}

}
if (
rsaPublicKeySchema.safeParse(backupPublicKey).success === false &&
rsaPrivateKeySchema.safeParse(backupPublicKey).success === false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to support a Secp256k1 key as a backup key ? This code is because we currently only support RSA encryption in signature lib:
see slack message

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No; we'll add more later I'm sure, but today we can just do RSA.

secp256k1 isn't commonly used for encryption, it's good for signing.

VaultService,
WalletRepository,
PrismaService,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to do this because I'm using PrismaService directly in BackupRepository to write on the newly created table 'Backup'.
Although, it feels it's not great to do that, because we have a 'PrismaModule' and it seems to be breaking boundaries.
I'm unsure on what should be the correct path here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TLDR: you're correct to use PrismaService directly in your repositories! Just import PersistenceModule.

Pretty sure you can/should actually add PersistenceModule into the imports section above. That's the only change.

You're supposed to use PrismaService directly within your repository; the purpose of the PersistenceModule is to encapsulate all the setup & stuff, not to have a single module where we put all our data access logic.
So when you want to use it, you tell this module that it needs to import stuff from the Prisma Module. You don't import the individual services from that module direclty, just the entire thing.

That also has the TestPrismaService set up which helps in testing.
The client.module.ts in Armory might be a good reference -- it's doing all that, writing to individual tables (not the KV store).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @mattschoch said, import the persistence module, and it'll make everything marked as "exported" in that module available here.

You don't have to import the exported providers in your module.

import { ApiPropertyOptional } from '@nestjs/swagger'
import { IsNumber, IsOptional, IsString } from 'class-validator'
import { IsBoolean, IsNumber, IsOptional, IsString } from 'class-validator'

export class CreateClientDto {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you can use Zod to generate the DTO from a schema. Here's an example.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ptroger You didn't change this one to a zod schema

return path || `m/44'/60'/${accountIndex}'/${changeIndex}/${addressIndex}`
}

export const hdKeyToKid = (key: HDKey): string => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To whoever is wondering what HD means: hierarchical deterministic

import { ApiProperty } from '@nestjs/swagger'
import { IsOptional, IsString } from 'class-validator'

export class GenerateKeyDto {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my other comment about using Zod.

We deprecated class-validator and new DTOs must always use a Zod schema.

Comment on lines 21 to 28
await this.prismaService.backup.create({
data: {
clientId,
backupPublicKeyHash,
keyId,
data
}
})
Copy link
Collaborator

@wcalderipe wcalderipe May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving apart my other comment about using key-value.

Let's test your design skills: how can you improve this implementation?

export class MnemonicRepository {
private KEY_PREFIX = 'mnemonic'

constructor(private keyValueService: EncryptKeyValueService) {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I'm confused, why have you used the kv service here but not for the backup? 🤔

Copy link
Collaborator

@wcalderipe wcalderipe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job, especially adding E2E tests. As more you do it, the better you get at it :)

I left a blocker comment about using a specific table instead of the kv. However, after finishing the review and reading another repository using the kv, I'm unsure if that was intentional and why.

@Ptroger Ptroger force-pushed the feature/nar-1595-vault-root-key-generation-derivation branch from 200ed29 to 811293e Compare May 24, 2024 11:59
.set('authorization', `GNAP ${accessToken}`)
.send({
keyId,
derivationPaths: ['next', 'next']
Copy link
Collaborator

@wcalderipe wcalderipe May 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that a common pattern or did you come up with? I have to admit it smells.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you find bothering there ?
It wasn't my finding, but part of the requirement. The way I understand this:
One requirement is to be able to derive 'next' wallet.
Another requirement is to be able to derive multiple wallets on the same API call
Consequence is that you need a value that tells vault to use 'next' path for derivation.

Copy link
Collaborator

@wcalderipe wcalderipe May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[IMPORTANT]

If the requirement is to derive the next wallet, why can't we just do that without asking the user to pass next? If you need N wallets in a single call, why can't we use a number?

Example, we use derive as the number of wallets to derive as part of the body:

  • If derive is undefined, generate the next available index
  • If derive is a number bigger than 0, generate N wallets from the last known index
  • If derive is a number smaller or equal than 0, throw an error.

Isn't that what you want?

Copy link
Contributor Author

@Ptroger Ptroger May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requirement is you can derive 'next' wallet or ${customPath} wallet.
So having just a number won't work.

Say you want to derive wallet at path m/44'/60'/0'/0/2 AND next wallet. You will pass:
['m/44'/60'/0'/0/2', 'next'];

Following you logic of having a number of wallets, what you could have, is {
derive: number
derivationPaths?: string[]
}

But that doesn't removed the need for the array of strings. I think having only the array is more verbose, and you also don't need derive because its === derivationPaths.length value

As a note, if you are wondering what is the purpose of passing custom derivation paths, it could allow things like:

  • organize your HD wallet as you please
  • retrieve the PK of a wallet where you still have the rootKey and derivation path

Copy link
Contributor Author

@Ptroger Ptroger May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, currently, deriving a custom path will never update the 'nextAddrIndex' value. I'm not sure about that. Reason why I did not implemented it is because I don't think the current solution should last (nextAddrIndex should not be stored but computed.) I also think the interface for the post request should look like this:

paths: {
  accountIndex: number
  changeIndex: number
  addressIndex: number
} | DerivationPath[]

But in order to do this I need to really understand how to leverage querying capabilities with KV storage. Thus my message to you @wcalderipe this morning

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. Maybe I don't understand the problem well enough nor our existent needs, and that's why I think it's overcomplicated.

I still think asking the user to input next is weird when one just wants a single wallet 🤷‍♂️

Btw, currently, deriving a custom path will never update the 'nextAddrIndex' value. I'm not sure about that. Reason why I did not implemented it is because I don't think the current solution should last (nextAddrIndex should not be stored but computed.) I also think the interface for the post request should look like this:

Could you please explain what you have in mind for "not stored but computed"?

Comment on lines 32 to +36
useValue: {}
},
{
provide: KeyGenerationService,
useValue: {}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol... put something in the dependency injection container, otherwise that'll blow with shady errors.

Suggested change
useValue: {}
},
{
provide: KeyGenerationService,
useValue: {}
useValue: mock<ImportRepository>()
},
{
provide: KeyGenerationService,
useValue: mock<KeyGenerationService>()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah so I did that to 'fix' the test module as it requires a KeyGenService to be passed. But I did not unit test it, nor do I expect any values from it. What would you consider a valid 'not being used in the test yet needs to be here in order for the test module to run' state ? I thought empty made sense...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would you consider a valid 'not being used in the test yet needs to be here in order for the test module to run' state ?

Both an empty object and a mock are valid answers. Personally, I'd prefer an empty mock, even for a test that's not yet using it, because it lays the foundation for future changes. With an empty mock, anyone coming in later just needs to add the desired behavior. With an empty object, they'd have to write the whole thing from scratch.

},
{
provide: ClientService,
useValue: {
Copy link
Collaborator

@wcalderipe wcalderipe May 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can use a mock here to mock the whole implementation plus a custom behavior for the findById. Moreover, it'll enforce types on the mocked values which reduces the chance of adding a liability in the test due to a wrong interpretation of how the method works.

Suggested change
useValue: {
useValue: mock<ClientService>().findById.mockResolvedValue({})

Comment on lines 85 to 128
const module: TestingModule = await Test.createTestingModule({
providers: [
KeyGenerationService,
{
provide: WalletRepository,
useValue: {
// mock the methods of WalletRepository that are used in keyGenerationService
// for example:
save: jest.fn().mockResolvedValue({
id: 'walletId',
address: '0x2c4895215973CbBd778C32c456C074b99daF8Bf1',
privateKey: PRIVATE_KEY
})
}
},
{
provide: ImportRepository,
useValue: {}
},
{
provide: ClientService,
useValue: {
findById: jest.fn().mockResolvedValue({
backupPublicKey: rsaBackupKey
})
}
},
{
provide: MnemonicRepository,
useValue: {
save: jest.fn().mockResolvedValue({
mnemonic,
keyId: 'keyId'
})
}
},
{
provide: BackupRepository,
useValue: {
save: jest.fn().mockResolvedValue({})
}
}
]
}).compile()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you generating the whole application again? Tell me the problem you were having and I'll try to help you.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to modify the return values from mock services and their methods

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a better and more idiomatic way to solve that. Move the mocks to let variables that you define before compiling the module, then, when you need custom behavior, change the mock in the describe or it block.

This way you don't have to compile the whole application again.

let clientServiceMock: MockProxy<ClientService>
let encryptionServiceMock: MockProxy<EncryptionService>

clientServiceMock = mock<ClientService>()
clientServiceMock.findAll.mockResolvedValue([clientOne, clientTwo])
encryptionServiceMock = mock<EncryptionService>()
encryptionServiceMock.getKeyring.mockReturnValue(getTestRawAesKeyring())

encryptionServiceMock.getKeyring.mockImplementation(() => {
throw new EncryptionException('Something went wrong')
})

@@ -46,4 +48,13 @@ export class ImportController {

return response
}

@Post('/seed')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
@Post('/seed')
@Post('/seeds')

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why ? You want to import one seed

Copy link
Collaborator

@wcalderipe wcalderipe May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't matter the number of things you want to do. It's the arbitrary standard we have been trying to follow. It's annoying, but it creates consistency on our APIs.

  • Use nouns for endpoint
  • Always in the plural

Further reading: https://github.com/stickfigure/blog/wiki/How-to-%28and-how-not-to%29-design-REST-APIs#rule-1-do-use-plural-nouns-for-collections

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. Was unaware of this standard since the other endpoints were singular when I went there. Modifying them aswell

}

async save(clientId: string, key: Backup): Promise<Backup> {
await this.keyValueService.set(this.getKey(clientId, key.keyId.toLowerCase()), coerce.encode(Backup, key))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hint: instead of calling toLowerCase everywhere, add it to the getKey:

`${this.KEY_PREFIX}:${clientId}:${id}`.toLowerCase()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I copy-pasted from another repository and did not changed logic, just the objects and names. I'm gonna change it everywhere in another PR then

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up

Comment on lines 40 to 42
DerivationPathStart.refine((val): val is DerivationPathStart => val.startsWith(`m/44'/60'/`), {
message: "Derivation path must start with m/44'/60'/"
})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 💪

Comment on lines 35 to 36
const DerivationPathStart = z.string().startsWith(`m/44'/60'/`)
type DerivationPathStart = `m/44'/60'/${string}`
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can use custom here like we do for the hex.

const DERIVATION_PATH_PREFIX = "m/44'/60'/"

export const DerivationPath = z.custom<`${typeof DERIVATION_PATH_PREFIX}${string}`>(
  (value) => {
    const result = z.string().startsWith(DERIVATION_PATH_PREFIX).safeParse(value)

    if (result.success) {
      return value
    }

    return false
  },
  {
    message: `Derivation path must start with ${DERIVATION_PATH_PREFIX}`
  }
)

export type DerivationPath = z.infer<typeof DerivationPath>

image

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

magic

})
export type Wallet = z.infer<typeof Wallet>

export const UserFacingWallet = z.object({
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The term user facing is weird.

What do you think of private/public or internal/external?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

way better

Comment on lines 66 to 67
IMPORTED: 'imported',
GENERATED: 'generated'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A quick suggestion, make constant values upper case so they are very clear to the reader.

Suggested change
IMPORTED: 'imported',
GENERATED: 'generated'
IMPORTED: 'IMPORTED',
GENERATED: 'GENERATED'

export const RootKey = z.object({
keyId: z.string().min(1),
mnemonic: z.string().min(1),
origin: z.union([z.literal('imported'), z.literal('generated')]),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
origin: z.union([z.literal('imported'), z.literal('generated')]),
origin: z.union([z.literal(SeedOrigin.IMPORTED), z.literal(SeedOrigin.GENERATED)]),

Comment on lines 85 to 128
const module: TestingModule = await Test.createTestingModule({
providers: [
KeyGenerationService,
{
provide: WalletRepository,
useValue: {
// mock the methods of WalletRepository that are used in keyGenerationService
// for example:
save: jest.fn().mockResolvedValue({
id: 'walletId',
address: '0x2c4895215973CbBd778C32c456C074b99daF8Bf1',
privateKey: PRIVATE_KEY
})
}
},
{
provide: ImportRepository,
useValue: {}
},
{
provide: ClientService,
useValue: {
findById: jest.fn().mockResolvedValue({
backupPublicKey: rsaBackupKey
})
}
},
{
provide: MnemonicRepository,
useValue: {
save: jest.fn().mockResolvedValue({
mnemonic,
keyId: 'keyId'
})
}
},
{
provide: BackupRepository,
useValue: {
save: jest.fn().mockResolvedValue({})
}
}
]
}).compile()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a better and more idiomatic way to solve that. Move the mocks to let variables that you define before compiling the module, then, when you need custom behavior, change the mock in the describe or it block.

This way you don't have to compile the whole application again.

let clientServiceMock: MockProxy<ClientService>
let encryptionServiceMock: MockProxy<EncryptionService>

clientServiceMock = mock<ClientService>()
clientServiceMock.findAll.mockResolvedValue([clientOne, clientTwo])
encryptionServiceMock = mock<EncryptionService>()
encryptionServiceMock.getKeyring.mockReturnValue(getTestRawAesKeyring())

encryptionServiceMock.getKeyring.mockImplementation(() => {
throw new EncryptionException('Something went wrong')
})

}> {
const { keyId, encryptedSeed } = body

// const mnemonic = await this.decryptSeed(clientId, keyId, encryptedSeed)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// const mnemonic = await this.decryptSeed(clientId, keyId, encryptedSeed)

Comment on lines +71 to +72
const client = await this.clientService.findById(clientId)
const backup = await this.#maybeEncryptAndSaveBackup(clientId, kid, mnemonic, client?.backupPublicKey)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[IMPORTANT]

The synergy between maybeEncryptAndSaveBackup and saveMnemonic is difficult to wrap my head around.

  • What happens if maybeEncryptAndSaveBackup returns undefined?
  • What happens if a user passes a backup key, but it's invalid (see lines 36 and 37)?

Copy link
Contributor Author

@Ptroger Ptroger May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if maybeEncryptAndSaveBackup returns undefined?

nothing happens. Backup is optional. If the user didn't already provided a backup public key you can encrypt with, then you... don't backup the mnemonic.

What happens if a user passes a backup key, but it's invalid (see lines 36 and 37)?

Nothing happens aswell. That means that the backup key provided was not an RSA key, and we currently only support RSA encryption/decryption. I was unsure about this: #268 (comment)
So it still imports the mnemonic, and warn logs us that a user have a wrongly setuped backup key.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, let me see if I understood correctly.

  • The user adds a backup because they want to feel safe and have the ability to recover their data.
  • Unfortunately, they entered an invalid key.
  • The application indicated that everything was ok.
  • The user moves on with their life, thinking, "I'm safe," and stores the backup key.
  • Months pass by, and some issue occurs. The user comes back to recover the data.
  • The user can't recover the data because they never had a valid backup key in the first place.

Is that correct?

Copy link
Contributor Author

@Ptroger Ptroger May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The user adds a backup because they want to feel safe and have the ability to recover their data.

This is not implemented.

So most likely when it does,

The user adds a backup because they want to feel safe and have the ability to recover their data.
Unfortunately, they entered an invalid key.
The application indicated that everything was ok.

This won't be true. And endpoint will not return that everything was ok. As stated in the thread I gave you, I thought this would need to be 'provide any public key' and not just 'provide an rsa key' for backup.
So problem there is with the model I stored that is publicKeySchema, and not rsaPublicKeySchema. Thus this defensive code that shouldn't be here. Got it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't follow your response. What do you pretend to do?

To be transparent, I'm concerned about the behavior here. If the user provides the application with the wrong key, it MUST fail the operation with a clear error message, so the user can fix it and try again.

Copy link
Contributor Author

@Ptroger Ptroger May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. So here, the user does not provides a backupPublicKey. If the key was already provided before, vault uses it to produce a backup by encrypting the mnemonic with it.

Where you are correct is that this bit of code shouldn't be responsible to validade the key, as it was provided before. So it shouldn't validate the key is an RSA public key. The key should have been defined as a specific RSA Public Key, not any Public Key. So that there is no need to validate that the key after it was already accepted and inserted in DB.

Commit that fixes it:
bcb66e7

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope it clarifies the issue, lmk

@@ -46,4 +48,13 @@ export class ImportController {

return response
}

@Post('/seed')
Copy link
Collaborator

@wcalderipe wcalderipe May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't matter the number of things you want to do. It's the arbitrary standard we have been trying to follow. It's annoying, but it creates consistency on our APIs.

  • Use nouns for endpoint
  • Always in the plural

Further reading: https://github.com/stickfigure/blog/wiki/How-to-%28and-how-not-to%29-design-REST-APIs#rule-1-do-use-plural-nouns-for-collections

}

async save(clientId: string, key: Backup): Promise<Backup> {
await this.keyValueService.set(this.getKey(clientId, key.keyId.toLowerCase()), coerce.encode(Backup, key))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Up

Comment on lines 35 to 41
if (
rsaPublicKeySchema.safeParse(backupPublicKey).success === false &&
rsaPrivateKeySchema.safeParse(backupPublicKey).success === false
) {
this.logger.warn('Invalid backup public key provided. Need an RSA key', { clientId })
return
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this not throw? Or at least log an error?
It appears this scenario is

  • A backup key exists
  • The backup key is corrupt
  • Therefore, we will think there is a backup of these keys, but in reality nothing will be backed up, so in a disaster scenario you lose all your keys while thinking you had backups....

Note: throwing likely should not completely end the execution flow though, because if you've already generated & saved the key then you don't want to crash and say unsuccessful while there is a new key already.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit that fixes it:
bcb66e7

import { ApiPropertyOptional } from '@nestjs/swagger'
import { IsNumber, IsOptional, IsString } from 'class-validator'
import { IsBoolean, IsNumber, IsOptional, IsString } from 'class-validator'

export class CreateClientDto {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Ptroger You didn't change this one to a zod schema

Comment on lines +37 to +54
export const DerivationPath = z.union([
z.custom<`${typeof DERIVATION_PATH_PREFIX}${string}`>(
(value) => {
const result = z.string().startsWith(DERIVATION_PATH_PREFIX).safeParse(value)

if (result.success) {
return value
}

return false
},
{
message: `Derivation path must start with ${DERIVATION_PATH_PREFIX}`
}
),
z.literal('next')
])
export type DerivationPath = z.infer<typeof DerivationPath>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why require a specific prefix?
It's completely valid to derive a key from a different path.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see viem uses the typing with the prefix. I question enforcing their constraint but okay with it to start -- but why do we need this whole custom zod parser?

Copy link
Contributor Author

@Ptroger Ptroger May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you enforce that a wallet was not already derived without a specific standard for paths ?
Ofc you could generate a random string and have 99% chance that it was not already derived. Still doesn't feel very clean.
Viem partially followed bip44, I followed viem.
Meaning, part of the prefix should instead be programmatic (60 is ethereum slip44 in the prefix).

I think if we wish to integrate people that may already have derived wallet and want to import a seed that was already used, it would be better if we were following bip44 when deriving a new wallet from this seed.

Comment on lines 112 to 131
const header = decodeProtectedHeader(encryptedSeed)
const kid = header.kid

if (!kid) {
throw new ApplicationException({
message: 'Missing kid in JWE header',
suggestedHttpStatusCode: HttpStatus.BAD_REQUEST
})
}

const encryptionPrivateKey = await this.importRepository.findById(clientId, kid)

if (!encryptionPrivateKey) {
throw new ApplicationException({
message: 'Encryption Key Not Found',
suggestedHttpStatusCode: HttpStatus.NOT_FOUND
})
}

const mnemonic = await rsaDecrypt(encryptedSeed, encryptionPrivateKey.jwk)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all identical to the importEncryptedPrivateKey method, might be nice just use the same exact code, so both just do
const mnemnoic = await this.#decrypt(clientId, encryptedSeed)
or
const pk = await this.#decrypt(clientId, encryptedPrivateKey)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it

return wallets.length === 1 ? wallets[0] : wallets
}

async storeRootKeyAndFirstWallet(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this function. The word And is a signal that it should be 2 separate functions instead.
Have 1 function that saves the mnemonic.
Have another function that derives a wallet.
If you want to save the root key and then derive the first wallet, you call both functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it

Comment on lines 18 to 21
constructor(wallets: PrivateWallet[] | PrivateWallet) {
super()
this.wallets = Array.isArray(wallets) ? wallets.map(privateToPublicWallet) : privateToPublicWallet(wallets)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah so that's how you add a constructor to the zod dto!

FYI if you don't actually need to do anything "different" then you don't need to add a constructor, you can just use DeriveWalletResponseDto.create() which will effectively parse the values into the zod type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't knew that, I think I can remove one or two useless constructors thens

@Injectable()
export class BackupRepository {
constructor(private keyValueService: KeyValueService) {}
private KEY_PREFIX = 'mnemonic'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem like the right prefix for the backup

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy pasting is dangerous indeed

Comment on lines +94 to +99
if (path === 'next') {
wallet = await deriveWallet(mnemonicToRootKey(mnemonic), { rootKeyId: opts.keyId, addressIndex: curr })
curr++
} else {
wallet = await deriveWallet(mnemonicToRootKey(mnemonic), { rootKeyId: opts.keyId, path })
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remove the value next? so instead if a derivationPath is passed, use it, otherwise calculate the next one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do

keyId: z.string().min(1),
mnemonic: z.string().min(1),
origin: z.union([z.literal(SeedOrigin.GENERATED), z.literal(SeedOrigin.IMPORTED)]),
nextAddrIndex: z.number().min(0).default(0)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get rid of this? Storing the next-derivation index feels like an antipattern.
Instead, at runtime for key derivation, query for the wallets & look at the last one.
I suggest using a "max" approach. You always derive the next address after the latest one, so if a specific account is derived at a specific path & skips some, then that's the new "current" index. This will avoid the most weird scenarios IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't planned for this to stay, was a hack before I could query wallets and find the last one.
Btw, this capability is exactly why we need a deterministic derivation path with a prefix and indexes

export const RootKey = z.object({
keyId: z.string().min(1),
mnemonic: z.string().min(1),
origin: z.union([z.literal(SeedOrigin.GENERATED), z.literal(SeedOrigin.IMPORTED)]),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

origin is a good idea, but I worry it's going to be misleading.

My suggestion is that you need to add this field into the wallet schema as well.

Here's the scenario

  • Import Seed A, which is stored as imported
  • Generate Seed B, which is stored as generated
  • Derive Private Key R from Seed A
  • Derive Private Key X from Seed B
  • Import Private Key Q

Which wallets are imported?

  • Key R can be called imported by looking up seed A and seeing that it was imported
  • Key X can be called generated by looking up seed B and seeing that it was generated
  • Key Q maybe could be inferred that a null seed means it must have been imported?

wdyt?

Copy link
Contributor Author

@Ptroger Ptroger May 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand your comment.

You are saying this:

Key R can be called imported by looking up seed A and seeing that it was imported
Key X can be called generated by looking up seed B and seeing that it was generated
Key Q maybe could be inferred that a null seed means it must have been imported?

Is not enough, and origin should be explicitly added to wallet schema ?
I think it makes sense to add it there. I wrote this because i know that origin exists on rootkey. But for someone else that comes in the codebase, it can be handy to explicitly see the origin in the wallet

@Ptroger Ptroger merged commit cfe4407 into main May 28, 2024
2 checks passed
mattschoch pushed a commit that referenced this pull request Jun 19, 2024
* mnemonic generation and key derivation from mnemonic

* backup stored mnemonics

* added e2e test

* logs

* repared failing test after Wallet type modification

* fixed linting

* revised unit test

* Unit tests on util, switched table storage for unencrypted KV storage

* switch dto declaration to zod

* schema and naming changes

* switched url to plural

* import persistence module and not prisma service

* backup has to be rsa key on schema

* create-client-dto use zod, use PublicWallet.parse instead of custom func

* removed  function

* removed unnecessary constructor for DTO

* remove code duplication in importService

* added origin to wallet schema
wcalderipe pushed a commit that referenced this pull request Jul 5, 2024
* mnemonic generation and key derivation from mnemonic

* backup stored mnemonics

* added e2e test

* logs

* repared failing test after Wallet type modification

* fixed linting

* revised unit test

* Unit tests on util, switched table storage for unencrypted KV storage

* switch dto declaration to zod

* schema and naming changes

* switched url to plural

* import persistence module and not prisma service

* backup has to be rsa key on schema

* create-client-dto use zod, use PublicWallet.parse instead of custom func

* removed  function

* removed unnecessary constructor for DTO

* remove code duplication in importService

* added origin to wallet schema
@wcalderipe wcalderipe deleted the feature/nar-1595-vault-root-key-generation-derivation branch July 8, 2024 13:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants