Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add DEP docs #567

Merged
merged 86 commits into from
Jan 25, 2023
Merged
Show file tree
Hide file tree
Changes from 84 commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
31aa300
docs: frontpage
youngjoon-lee Dec 22, 2022
23b2399
fix
youngjoon-lee Dec 22, 2022
d0eae6f
fix
youngjoon-lee Dec 22, 2022
2d0aa14
fix
youngjoon-lee Dec 22, 2022
915e8c9
fix
youngjoon-lee Dec 22, 2022
963f489
fix
youngjoon-lee Dec 22, 2022
d0cae23
consume data
youngjoon-lee Dec 23, 2022
ef00c1e
spec
youngjoon-lee Dec 23, 2022
f6faecc
typo
youngjoon-lee Dec 23, 2022
e43756c
link
youngjoon-lee Dec 23, 2022
6b226b8
tmp
youngjoon-lee Dec 23, 2022
6274357
Update .gitbook/0-about-panacea/1-roadmap.md
Jan 4, 2023
37085a4
Update .gitbook/0-about-panacea/1-roadmap.md
Jan 4, 2023
427af07
Update .gitbook/1-users/3-data-exchange/0-about-dep.md
Jan 4, 2023
0fa399f
Update .gitbook/1-users/3-data-exchange/0-about-dep.md
Jan 4, 2023
4423709
Update .gitbook/1-users/3-data-exchange/0-about-dep.md
Jan 4, 2023
701637e
Update .gitbook/1-users/3-data-exchange/0-about-dep.md
Jan 4, 2023
d18fdb6
Update .gitbook/1-users/3-data-exchange/0-about-dep.md
Jan 4, 2023
71708ed
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
7d54caf
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
9f4a242
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
fa7116b
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
fa9d0ce
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
7205c04
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
38d124a
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
92ce731
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
13c9776
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
7d9b4df
Update .gitbook/3-protocol-devs/1-dep-specs/0-overview.md
Jan 4, 2023
9174493
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
fc5e809
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
28f6c56
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
34bb9cf
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
8cb5975
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 4, 2023
1643046
docs: add data-deal specs for protocol devs (#585)
0xHansLee Jan 5, 2023
950574f
Update .gitbook/0-about-panacea/1-roadmap.md
Jan 6, 2023
be07e99
Update .gitbook/1-users/3-data-exchange/1-consume-data.md
Jan 6, 2023
9a04d48
revert unintentional changes
youngjoon-lee Jan 6, 2023
984539f
add jwt guide
youngjoon-lee Jan 6, 2023
9dd634e
deal json
youngjoon-lee Jan 6, 2023
f62e72e
docs: rename buyer and seller
Jan 6, 2023
39daf84
docs: add list of operating oracle nodes
audtlr24 Jan 9, 2023
7187b83
Merge remote-tracking branch 'origin/docs/560' into docs/560
audtlr24 Jan 9, 2023
a8d0a7b
docs: add operate oracle nodes to SUMMARY.md
audtlr24 Jan 9, 2023
05f1204
docs: add data validation spec for protocol devs (#596)
gyuguen Jan 9, 2023
a151c11
docs: add data provider consent spec for protocol devs (#592)
0xHansLee Jan 10, 2023
d4976bd
docs: add user guide for provider (#594)
audtlr24 Jan 10, 2023
e2cdd32
docs: incentives for protocol devs (#601)
0xHansLee Jan 10, 2023
5b43933
docs: add oracle installation (#607)
audtlr24 Jan 12, 2023
4efe303
docs: oracle-registration for oracle operators (#609)
0xHansLee Jan 12, 2023
bab1547
docs: update oracle info (#616)
0xHansLee Jan 12, 2023
612606f
docs: add a genesis oracle registration docs (#613)
gyuguen Jan 12, 2023
d70675d
docs: add oracle initialization docs (#611)
inchori Jan 13, 2023
f09c838
docs: add DEP user flow spec (#597)
Jan 13, 2023
97788f5
docs: running oracle node (#617)
audtlr24 Jan 13, 2023
9362564
docs: oracle upgrade for oracle operators (#619)
0xHansLee Jan 13, 2023
074efbf
docs: add confidential oracle docs (#599)
inchori Jan 13, 2023
140f721
docs: add verify remote report (#620)
gyuguen Jan 13, 2023
0530076
Merge branch 'main' into docs/560
youngjoon-lee Jan 13, 2023
7d247d8
remove unnecessary images
youngjoon-lee Jan 13, 2023
b3ac6d1
add tj
youngjoon-lee Jan 13, 2023
d5b448c
docs: add terms of agreement (#621)
0xHansLee Jan 13, 2023
5798157
Update 0-panacea-ecosystem.md
tjyoon0324 Jan 18, 2023
a3e26aa
Update 1-roadmap.md
tjyoon0324 Jan 18, 2023
4652176
Update 0-about-dep.md
tjyoon0324 Jan 18, 2023
a03df74
Update 1-consume-data.md
tjyoon0324 Jan 18, 2023
069ef75
Update 2-provide-data.md
tjyoon0324 Jan 18, 2023
7d797b8
Update 1-consume-data.md
tjyoon0324 Jan 18, 2023
ef094f8
Update 1-user-flow.md
tjyoon0324 Jan 18, 2023
4d2f1c0
Update 2-data-deal.md
tjyoon0324 Jan 18, 2023
220a535
Update 3-data-provider-consent.md
tjyoon0324 Jan 19, 2023
91d180e
Update 3-data-provider-consent.md
tjyoon0324 Jan 19, 2023
b44506e
Update 4-data-validation.md
tjyoon0324 Jan 19, 2023
13b8c58
Update 2-data-deal.md
tjyoon0324 Jan 19, 2023
6090c08
Update 6-incentives.md
tjyoon0324 Jan 19, 2023
f2a5463
Update 6-incentives.md
tjyoon0324 Jan 19, 2023
8a36429
docs: update 5-confidential-oracle.md (#622)
tjyoon0324 Jan 19, 2023
be7f0b3
Update 1-oracle-installation.md
tjyoon0324 Jan 19, 2023
19db378
Update 2-oracle-intialization.md
tjyoon0324 Jan 20, 2023
66883c8
Update 3-genesis-oracle.md
tjyoon0324 Jan 20, 2023
1e75822
Update 4-oracle-registration.md
tjyoon0324 Jan 20, 2023
342abaa
Update and rename 5-running-node.md to 5-running-oracle-node.md
tjyoon0324 Jan 20, 2023
ed0f996
Update 6-update-oracle-info.md
tjyoon0324 Jan 20, 2023
7afe97e
Update 7-oracle-upgrade.md
tjyoon0324 Jan 20, 2023
769e1b8
Update 8-verify-remote-report.md
tjyoon0324 Jan 20, 2023
1439033
Update .gitbook/5-oracles/1-operate-oracle-nodes/5-running-oracle-nod…
tjyoon0324 Jan 20, 2023
8d8c3d2
Update 5-running-oracle-node.md
tjyoon0324 Jan 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@

## Mission

Allow patients to own their data, provide it, and get incentivized.
Allow patients to own their data, provide the data to data consumers, and get incentivized.


## Goal

Building decentralized healthcare data sharing/exchange protocols
Build decentralized healthcare data sharing/exchange protocols


## Tech Stack
Expand All @@ -18,7 +18,7 @@ Building decentralized healthcare data sharing/exchange protocols

## Panacea blockchain

The Panacea is a public blockchain based on the Cosmos SDK and the Tendermint.
Panacea is a public blockchain based on the Cosmos SDK and the Tendermint.

As the Tendermint implements a partially synchronous BFT (Byzantine fault-tolerant) consensus protocol,
Panacea can provide a high-performance, consistent, and secure decentralized network
Expand Down Expand Up @@ -51,11 +51,11 @@ Then, data verifiers (receivers) can verify that the data has been not tampered
### Data Exchange Coordination

Panacea provides a feature of coordinating data exchange deals.
Data consumers can publish deals publicly, specifying which data they want to purchase and how much budget they are willing to pay.
And, data providers (holders) who agree to provide their data with consumers can provide their data securely and earn rewards in MED.
Data consumers can publish deals publicly, specifying 1) which data they want to purchase and 2) how much budget they are willing to pay.
Data providers (holders) who agree to provide their data with the data consumers can provide their data securely and earn MED as rewards.

This on-chain data exchange coordination must work with some off-chain components to verify data validity/integrity and securely deliver data.
For example, some malicious data providers can try to sell data that data consumers don't want or data that was generated by themselves (not by trusted issuers).
This on-chain data exchange coordination must work with some off-chain components to verify data validity/integrity and to deliver data securely.
For example, some malicious data providers ccould try to sell data that the data consumers don't want or that was forged by themselves (not by trusted issuers).
Therefore, data should be validated by off-chain data validators before being delivered to data consumers, as described in the [Data Exchange Validation](#data-exchange-validation-with-confidential-computing) section.


Expand All @@ -67,30 +67,26 @@ Using [Panacea Verifiable Credential SDK](https://github.com/medibloc/vc-sdk), d
Since [Verifiable Credential](https://www.w3.org/TR/vc-data-model/) is a standard format defined by W3C, any data receivers can verify credentials without any compatibility issues using the data issuer's (holder's) cryptographic public key that is corresponding with their DID.
In other words, the Verifiable Credential SDK works with DID management features of the Panacea blockchain.

In addition, data holders can present their data by masking privacy-sensitive fields that don't need to be revealed.
That can be achieved by Zero-knowledge Proof using BBS+ signatures so that data receivers can verify data integrity even though some fields in the data are masked (technically, ' tampered').
In addition, data holders can present their data while masking privacy-sensitive fields.
This can be achieved by Zero-knowledge Proof using BBS+ signatures so that data receivers can verify data integrity even though some fields in the data are masked (technically, ' tampered').

### Data Exchange Validation with Confidential Computing

As described in the [Data Exchange Coordination](#data-exchange-coordination) section, Panacea blockchain provides data exchange coordination. However, data that is exchanged between providers and consumers cannot be validated/delivered through a public blockchain because all transaction data is exposed to everyone.
Even if transaction data is encrypted, validator nodes should be able to decrypt data to verify data validity/integrity.
That could be a huge vulnerability, allowing data to be leaked to anyone other than intended data consumers.
To avoid this vulnerability, we could implement this data exchange validation as smart contracts run on [Secret Network](https://scrt.network/), but still, we didn't want to expose private data on the public blockchain even if all data is encrypted.
To avoid this vulnerability, we could have implemented this data exchange validation as smart contracts run on [Secret Network](https://scrt.network/), but still, we didn't want to expose private data on the public blockchain even if all data is encrypted.

Instead of solving this problem on chain, we have introduced an off-chain decentralized oracle powered by confidential computing ([Intel SGX](https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/overview.html)).
Like Secret Network, all oracle nodes are run in the secure enclave. All data is encrypted so that only oracle nodes can decrypt it to verify data validity/integrity. If data is verified successfully, it is re-encrypted by oracle nodes for the data consumer.
Because this process is executed in the secure enclave, decrypted data cannot be stolen by anyone, even oracle node operators.

To avoid the Byzantine problem between decentralized oracle nodes, oracle nodes borrow the voting power from the Panacea blockchain.
Only operators who are running validators on Panacea can run oracle nodes. Then, an oracle node has the same voting power as a corresponding validator.
If an oracle node acts maliciously, its validator's stake is slashed.

### Secure Data Storage

Based on DIDs, Verifiable Credentials, and Data Exchange Coordination/Validation, it seems that applications that handle privacy data can be implemented.
However, to actually implement applications, it must be decided specifically how data will be stored and how it will be transmitted in decentralized environments.
Based on DIDs, Verifiable Credentials, and Data Exchange Coordination/Validation, it seems that applications that handle privacy data can be implemented on top of Panacea.
However, to actually implement applications, the specific method on how data will be stored and how it will be transmitted in decentralized environments should be decided.

Thanks to [IPFS](https://ipfs.io/), data can be transmitted by [content addressing](https://docs.ipfs.tech/concepts/content-addressing/) in environments where ecosystem participants don't know each other's IP addresses and ports.
But, IPFS is also a public network basically that is unsuitable for storing privacy data even if it is encrypted. IPFS also provides a way to set up private clusters, but it is not flexible enough to be used for data exchange based on public blockchains.
However, IPFS is also a public network, which is unsuitable for storing privacy data even if it is encrypted. IPFS also provides a way to set up private clusters, but it is not flexible enough to be used for based on public blockchains.

MediBloc team is researching many secure decentralized data storages including edge databases.
To solve this problem, MediBloc team is still researching many secure decentralized data storages including edge databases.
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,15 @@ Now, MediBloc would like to shift its focus to boosting the potential of Panacea

Nowadays, many businesses and technologies are data-driven. Many companies are already familiar with handling large dataset and deriving new values by analyzing sets of data. But, secure data exchange is still the one of the hardest area for data-driven industries. Data requesters want well-refined data or fine-grained raw data for successful data analysis. But, data owners (individuals) don’t want their privacy exposed and abused. Additionally, Web3 users are already aware that proper rewards should be guaranteed for their data and actions transparently on Web3. Traditional systems in Web2 have solved this issue in various ways, but MediBloc believes that we all can build more transparent and reliable systems for secure data exchange in Web3 ecosystem.

Our data exchange protocol has the concept of data Pool, so that anyone can specify the type and the quantity of the data they want. Also, they can specify how much cryptocurrency they are willing to pay for the data. All of these data pools are recorded in Panacea and everyone who wants to sell their data can see all data pools. Data sellers can choose data pools by checking how many parts of their data to be shared to data buyers. Then, they sign the consents for data exchange. Verified off-chain data validators validate whether data provided by data sellers conforms to criteria that data pool creator has specified. If all the requirements are met, the data is provided to data buyers via secure connections and the promised amount of cryptocurrency is transferred to data sellers. In this entire protocol, data is not recorded on any blockchain such as Panacea. All data transmissions are performed off-chain and Panacea guarantees all agreements for data exchanges and transparent payments.
Our data exchange protocol has the concept of data Pool, so that anyone can specify the type and the quantity of the data they want. Also, they can specify how much cryptocurrency they are willing to pay for the data. All of these data pools are recorded in Panacea and everyone who wants to sell their data can see all data pools. Data providers can choose data pools by checking how many parts of their data to be shared to data consumers. Then, they sign the consents for data exchange. Verified off-chain data validators validate whether data provided by data providers conforms to criteria that data pool creator has specified. If all the requirements are met, the data is provided to data consumers via secure connections and the promised amount of cryptocurrency is transferred to data providers. In this entire protocol, data is not recorded on any blockchain such as Panacea. All data transmissions are performed off-chain and Panacea guarantees all agreements for data exchanges and transparent payments.

This data exchange protocol is being developed to be as general as possible, so that not only the healthcare data but also all the other types of data can be handled by the protocol. Since Panacea and data exchange protocol is publicly opened, any service providers can build their own services on the top of the data exchange protocol, so that their users can exchange their data securely and get proper rewards. As the first use case, MediBloc is going to build a healthcare data marketplace service based on this protocol.
Well, it sounds like the protocol should work well, right? However, there are so many issues that we have to resolve. For privacy and security, data sellers should be able to expose only a small part of their data that is really desired by data buyers. Data transmission must be secure, so that anyone cannot steal data. In order to guarantee the right of data buyers, all criteria that data buyers specified has to be validated clearly before the payment is finalized. In addition, the ecosystem should be attractive enough for many data sellers and buyers to join.
Well, it sounds like the protocol should work well, right? However, there are many issues that we have to resolve. For privacy and security, data providers should be able to expose only a small part of their data that is really desired by data consumers. Also, data transmission must be secure, so that no one can steal or intercept the data. In order to guarantee the right of data consumers, all criteria that data consumers specified has to be validated clearly before the payment is finalized. Last but not least, the ecosystem should be attractive enough for many data providers and consumers to join.

In order to resolve these challenges, the team is developing this data exchange protocol with several latest technologies.
The detailed tech stack of the data exchange protocol is described in the [Panacea Ecosystem](./panacea-ecosystem.md) document.
The detailed tech stack of the data exchange protocol is described in the [Panacea Ecosystem](./0-panacea-ecosystem.md) document.

There will be more details that we have to solve, and we know that all of them cannot be achieved in one step. Hence, we will complete this big task step by step. In 2022, MediBloc will release the v0 of data exchange protocol as a proof of concept that includes only essential features. Also, a data marketplace web service will be introduced as a simple example service based on the protocol. Based on this proof of concepts, the data exchange protocol will be improved as v1 from 2023 with enhanced security and interoperability. MediBloc has already opened all source codes and progresses publicly on GitHub. We encourage anyone to join the project and share your insights.
There will be more detailed issues that we would have to solve, and we know that all of them cannot be solved in one step. Hence, we will complete this big task step by step. In 2022, MediBloc have released the v0 of data exchange protocol as a proof of concept that includes only essential features on testnet. Based on this proof of concepts, the data exchange protocol will be improved as v1 in 2023 with enhanced security and interoperability. Also, MediBloc will be introducing dApps for data providers using data exchange protocol and keep designing the services that go on top of data exchange protocol. MediBloc has already opened all source codes and progresses publicly on GitHub. We encourage anyone to join the project and share your insights.
We are so excited and thrilled to share our vision to achieve our goal to become the world’s best patient centric health data platform. Thank you for your continued support!

--
File renamed without changes.
43 changes: 43 additions & 0 deletions .gitbook/1-users/3-data-exchange/0-about-dep.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# About Data Exchange Protocol

Data Exchange Protocol (hereafter 'DEP') is a communication layer for sharing and exchanging various types of data
between two parties in decentralized environments.


## What can you do with DEP?

Data consumers can open data deals by specifying the type, the quantity, and the pricing of the data that they are willing to consume.

Data providers can choose and participate in the deals that match the data that they have when they are willing to provide.

To guarantee data consumers only receive the data that match the criteria specified by them,
decentralized oracles verify and issue certificates for all data being provided by the data provider.

Panacea manages the status of all data deals and data sharing consents,
ensuring data providers and ecosystem operators are rewarded appropriately.


## Motivation and Goals

### Data Ownership and Sovereignty

The ultimate goal of owning our own data is having a control about how our data is used.

### Decentralized off-chain data validation

o guarantee data consumers only receive the data that match the criteria specified by them,
decentralized oracles verify and issue certificates for all data being provided by the data provider.

### Privacy

Throughout the entire process of data verification and transmission,
the data content must not be exposed to anyone other than the consumer intended by the data provider.

### Generalized data exchange

Not only healthcare data, but various types of data should be covered through this protocol.

### Open-sourced protocol

All protocol specifications and implementations must be open-sourced, so any participants can understand
how data is exchanged and how privacy is guaranteed.
Loading