Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

216 build keychainprogram #217

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 28 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

<br>
<p align="center">
<img width="140" src="./docs/peerbit-logo.png" alt="Peerbit icon Icon">
Expand Down Expand Up @@ -26,48 +25,67 @@ Your database schema can remain very simple but still utilize P2P networks, auto

## Optimized for performance

Peerbit is performant, so performant in fact you can use it for [streaming video](https://stream.dao.xyz) by having peers subscribing to database updates. In a low latency setting, you can achieve around 1000 replications a second and have a thoughput of 100 MB/s.
Peerbit is performant, so performant in fact you can use it for [streaming video](https://stream.dao.xyz) by having peers subscribing to database updates. In a low latency setting, you can achieve around 1000 replications a second and have a thoughput of 100 MB/s.

![Dogestream](/docs/videostream.gif)


## Other examples

### [Chat room](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/one-chat-room/)

[<img src="https://github.com/dao-xyz/peerbit-examples/blob/master/packages/one-chat-room/demo.gif" width="600" />](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/one-chat-room/)

### [Lobby + chat rooms](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/many-chat-rooms/)

[<img src="https://github.com/dao-xyz/peerbit-examples/blob/master/packages/many-chat-rooms/demo.gif" width="600" />](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/many-chat-rooms/)

### [Sync files](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/file-share/)

#### [React app](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/file-share/)

[<img src="https://github.com/dao-xyz/peerbit-examples/blob/master/packages/file-share/demo-frontend.gif" width="600" />](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/file-share/)

#### [CLI](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/file-share/)
[<img src="https://github.com/dao-xyz/peerbit-examples/blob/master/packages/file-share/demo-cli.gif" width="600" />](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/file-share/)


[<img src="https://github.com/dao-xyz/peerbit-examples/blob/master/packages/file-share/demo-cli.gif" width="600" />](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/file-share/)

### [Collaborative machine learning](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/collaborative-learning/)
[<img src="https://github.com/dao-xyz/peerbit-examples/blob/master/packages/collaborative-learning/demo.gif" width="600" />](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/collaborative-learning/)


[<img src="https://github.com/dao-xyz/peerbit-examples/blob/master/packages/collaborative-learning/demo.gif" width="600" />](https://github.com/dao-xyz/peerbit-examples/tree/master/packages/collaborative-learning/)

## Get Started

1. Install Peerbit by following the simple setup instructions in our [Installation Guide](https://peerbit.org/#/getting-started).

2. Dive into our comprehensive [Documentation](https://peerbit.org/#/modules/client/) or checkout the [Example repository](https://github.com/dao-xyz/peerbit-examples) to explore the powerful features and learn how to leverage Peerbit to its fullest potential.
2. Dive into our comprehensive [Documentation](https://peerbit.org/#/modules/client/) or checkout the [Example repository](https://github.com/dao-xyz/peerbit-examples) to explore the powerful features and learn how to leverage Peerbit to its fullest potential.

3. Join us on [Matrix](https://matrix.to/#/#peerbit:matrix.org) to connect, share ideas, and collaborate with like-minded individuals.

## Contribute

Peerbit is an open-source project, and we welcome contributions from developers like you! Feel free to contribute code, report issues, and submit feature requests. Together, let's shape the future of Peerbit.

IMPORTANT: Peerbit uses yarn.

1. Check yarn version: `yarn -v` should print 1.something
2. Install: `yarn`
3. Build: `yarn build`
4. Run tests: `yarn test`

You might possibly need to CMD + Shift + P and then enter to restart the typescript server after the build step.

To create a new package, follow the following steps:

1. Clone the time folder within /packages/utils/time to the desired destination and rename it
2. Update the package.json `name`, `description`, `version` fields
3. Possibly add other depencencies to the package.json `dependencies` field (like `@peerbit/crypto`)
4. Delete contents in CHANGELOG.md
5. Update the root package.json `workspaces.packages` field
6. Update root lerna.json `workspaces.packages` field
7. run yarn once in root

We recommend running tests with the VS Code integration though: https://marketplace.visualstudio.com/items?itemName=firsttris.vscode-jest-runner

## Let's Get Coding!

[peerbit.org](https://peerbit.org)


3 changes: 2 additions & 1 deletion lerna.json
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@
"packages/utils/uint8arrays",
"packages/utils/cache",
"packages/utils/time",
"packages/utils/logger"
"packages/utils/logger",
"packages/utils/keychain"
],
"version": "independent",
"npmClient": "yarn"
Expand Down
3 changes: 2 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,8 @@
"packages/utils/uint8arrays",
"packages/utils/time",
"packages/utils/cache",
"packages/utils/logger"
"packages/utils/logger",
"packages/utils/keychain"
]
},
"engines": {
Expand Down
1 change: 1 addition & 0 deletions packages/programs/program/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,4 @@ export {
export * from "./client.js";
export * from "./program.js";
export * from "./address.js";
export * from "./keychain.js";
219 changes: 219 additions & 0 deletions packages/programs/program/src/keychain.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,219 @@
import { AbstractType, field, option, variant, vec } from "@dao-xyz/borsh";
import { Program } from "./program";
import {
And,
BoolQuery,
ByteMatchQuery,
Documents,
MissingField,
Or,
PutOperation,
Query,
SearchRequest,
Sort,
SortDirection,
StateFieldQuery,
StringMatch
} from "../../data/document/src";
import {
EncryptedThing,
EncryptedSecp256k1Keypair,
EncryptedX25519Keypair,
EncryptedEd25519Keypair,
EncryptedKeypair,
DecryptedThing,
createLocalEncryptProvider,
X25519Keypair,
X25519PublicKey,
createDecrypterFromKeyResolver,
KeyResolverReturnType,
Ed25519PublicKey,
Secp256k1PublicKey,
Identity,
EncryptProvide,
KeyExchangeOptions,
DecryptProvider
} from "@peerbit/crypto";
import { compare } from "@peerbit/uint8arrays";

@variant(0)
class EncryptedExtendedKey {
// TODO: The owner of the key determined by wallet address or public sign key?
@field({ type: vec("u8") })
owner: Uint8Array;
// Is this key revoked? HAS TO BE TRUE if recipient is set.
@field({ type: "bool" })
revoked: boolean;
// Provide a single-use key for someone else
@field({ type: option(vec("u8")) })
recipient?: Uint8Array;
// TODO: add the actual key here
@field({ type: EncryptedKeypair })
keypair:
| EncryptedX25519Keypair
| EncryptedEd25519Keypair
| EncryptedSecp256k1Keypair;

constructor(parameters: {
owner: Uint8Array;
keypair:
| EncryptedX25519Keypair
| EncryptedEd25519Keypair
| EncryptedSecp256k1Keypair;
revoked?: boolean;
}) {
this.keypair = parameters.keypair;
this.owner = parameters.owner;
this.revoked = parameters.revoked ?? false;
}
}

export class KeychainProgram extends Program {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implements Keychain maybe? from '@peerbit/keychain'

@field({ type: Documents<EncryptedExtendedKey> })
keys: Documents<EncryptedExtendedKey>;
Copy link
Member

@marcus-pousette marcus-pousette Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When PR #215 is merged it should in practice just be possible to just do

@field({ type: Documents<Ed25519Keypair> })
keys: Documents<Ed25519Keypair>; // or something else secret

and later just do

sharedKeychain.keys.put(keypair, { encryption : { hash: ` the hash of the secret key you want to use` } })

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or, well it depends if you want to have a a public and private part of the document of course. Then you need the EncryptedExtendedKey type

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, but isn't the problem that the encryption property on the put method encrypts the whole key? How would I then find the public key of someone else that I want to encrypt something for, if the public key is encrypted as well?

Copy link
Collaborator Author

@benjaminpreiss benjaminpreiss Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

___ I edited this message by mistake ___ // marcus

Copy link
Collaborator Author

@benjaminpreiss benjaminpreiss Oct 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And I have also been thinking - sometimes, a user I want to encrypt something for has not generated a keypair yet. So at that time of speaking, I want to generate a keypair for him that is only single use. And that might have to be marked as such?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

property on the put method encrypts the whole key? How would I then find the public key of someone else that I want to encrypt something for, if the public key is encrypted as well?

Ah yee for this to work the decryption has to be performed by keys that exist in a lower level keychain. When you use the put method there PublicKeyEnvelope (in the crypto lib) will leak the publickeys of the parties that can decrypt the message, which allows you to figure out if you can decrypt a message.

I think it might be easier to reason about stuff if you would separate "encryption" utilties from this shared-keychain. So this keychain stores keys that are encrypted, but the keys for providing the encryption for storing these keys for this keychain perhaps needs to originate from a lower level keychain.

It could in practice also be possible that keys in this keychain are allso encrypted by keys that also exist in this keychain, but then you need to make sure you are loading the database in the right order kind of (which is possible in practice there is a oplog behind the scenes that allows you to link commits)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to revoke keys I think you need a revoked flag on the keypair object, or something like that. Else you could in theory also just use the delete function of the document store and assumes keys are only valid if they exist in the keychain


@field({ type: Identity })
identity: Identity;

@field({ type: EncryptProvide })
encryptProvider: EncryptProvide<KeyExchangeOptions>;

@field({ type: DecryptProvider })
decryptProvider: DecryptProvider;

// TODO: Offer rotate keys functionality. revokes all unrevoked keys and adds a new key which it returns
// TODO: Write Key updater that either adds a given key or generates a new one.
// TODO: the key updater should always set the revoked flag for keys with a recipient

// TODO: Write a key getter that get's my latest key. If no valid key is found, run rotate and return newly created
// TODO: Write a key getter that get's the latest key's of recipients. If no valid key is found for a recipient, add a new, temporary key for the recipient.
// TODO: Write a key getter that get's a specified key, no matter the revoked status
async getKey(
parameters:
| { owner?: Uint8Array; publicKey?: never }
| {
owner?: never;
publicKey?: X25519PublicKey | Ed25519PublicKey | Secp256k1PublicKey;
} = {}
) {
const { owner = this.identity.publicKey.bytes, publicKey } = parameters;
// Create Filter options
const queries =
// Looking for specific public keys. We don't care about revocation status.
publicKey != undefined
? [
new ByteMatchQuery({
key: ["keypair", "publicKey", "publicKey"],
value: publicKey.publicKey
})
]
: // Looking for latest key from a specific owner.
[
new And([
// Only non-revoked
new BoolQuery({ key: "revoked", value: false }),
// Recipient has to be undefined
// TODO: Is this correct?
new MissingField({ key: "recipient" }),
// And from the specified owner
new ByteMatchQuery({
key: "owner",
value: owner
})
])
];
// TODO: How can we limit this to only fetch the last entry - so a single one?
const keys = await this.keys.index.search(
new SearchRequest({
query: queries,
sort: new Sort({
key: "timestamp",
direction: SortDirection.DESC
})
})
);
// Create a new key in case we can't find a match
// If owner is not me, create a new, temporary key with me as owner and recipient as the requested owner
// If owner is me, create a new permanent key with
if (keys.length === 0) {
//return compare(this.identity.publicKey.bytes, owner) ?
}
return keys[0].keypair.decrypt(this.decryptProvider);
}

// TODO: Unsure about encryptProvider here honestly - should this be layer 1 keychain?
async open(parameters: {
identity: Identity;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

identity already exist if you do

this.identity inside open (it is a global thing on 'this')

encryptProvider: EncryptProvide<KeyExchangeOptions>;
decryptProvider: DecryptProvider;
}) {
this.identity = parameters.identity;
this.encryptProvider = parameters.encryptProvider;
this.decryptProvider = parameters.decryptProvider;
// TODO: This should be opened with the layer 1 encryption provider
await this.keys.open({
type: EncryptedExtendedKey,
canPerform: async (operation, { entry }) => {
// Sender can put, identified by masterkey
const owner = (
operation instanceof PutOperation
? operation.value
: await this.keys.index.get(operation.key)
)?.owner;
for (const signer of entry.signatures) {
if (signer.publicKey.equals(owner)) return true;
}
return false;
},
index: {
// TODO: not sure about this
key: ["keyPair", "publicKey"],
// -> I want to be able to search .pub and .owner properties
fields: async (doc, context) => {
{
return {
...doc,
timestamp: context.created
};
}
},
canRead: () => true,
canSearch: () => true
},
// TODO: Potentially unsafe, as untrusted nodes might hide entries
canReplicate: () => true
});
}

async encrypt<T>(plainText: T, recipients: Uint8Array[]) {
if (recipients.length === 0) throw new Error("No recipients specified");
// TODO: Get and Feed my own key into this.
const myEncryptionKeypair = await X25519Keypair.create();
const encryptionProvider = createLocalEncryptProvider(myEncryptionKeypair);
// TODO: Get and Feed recipient keys into .encrypt
const recipientPublicKeys = await Promise.all([
X25519PublicKey.create(),
X25519PublicKey.create()
]);
return new DecryptedThing({ value: plainText }).encrypt(
encryptionProvider,
{ type: "publicKey", receiverPublicKeys: recipientPublicKeys }
);
}

async decrypt<T>(encryptedThing: EncryptedThing<T>, clazz: AbstractType<T>) {
const keyResolver = async <PublicKey extends X25519PublicKey | Uint8Array>(
key: X25519PublicKey | Uint8Array
) => {
// TODO: fetch keys here based on key parameter and return them
if (key instanceof X25519PublicKey)
return X25519Keypair.create() as Promise<
KeyResolverReturnType<PublicKey>
>;
return undefined;
};
const decryptionProvider = createDecrypterFromKeyResolver(keyResolver);
const decryptedThing = await encryptedThing.decrypt(decryptionProvider);
return decryptedThing.getValue(clazz);
}
}
Loading
Loading