feat: Implementing Pluto Repositories

[curtis-h]

Description

Large reconsideration of Pluto, reintroducing an implementation that abstracts the storage layer, so we can write the logic and error handling without being coupled to a specific db etc.

The Pluto class now acts as an orchestration layer, only handling Domain objects, their logic and relationships.
The repository layer handles mapping Domain <-> Storable Models.
The Store abtracts the storage implementation with a simplified interface, handling Storable Models.

BREAKING CHANGES:

• Domain instances now expect to track their storage ID with uuid property.
• updated Pluto signature to use null instead of undefined
• Pluto.storePrismDID signature changed - unused arguments (keyPathIndex, metaId) removed
• Pluto CredentialMetadata
  • updated to use Domain.CredentialMetadata instance
  • fetchCredentialMetadata renamed to getCredentialMetadata
• Pluto LinkSecret fns updated to use Domain.LinkSecret instance
  • CredentialRequestOptions updated to use Domain.LinkSecret and linkSecretName removed
  • CreatePresentationProof options updated to use Domain.LinkSecret
• Pluto storeMediator now takes Domain.Mediator, instead of three separate DIDs
• Pluto storePrivateKeys renamed to storePrivateKey as it only handles a single key
• PrismDIDs
  • PrismDID added
  • PrismDIDInfo removed
  • PrismDIDMethodId removed - functionality added to PrismDID`
  • Pluto fns
    • getAllPrismDIDs now returns Domain.PrismDIDs
    • getDIDInfoByDID getDIDInfoByAlias getPrismLastKeyPathIndex - removed

Jira link

https://input-output.atlassian.net/browse/ATL-6148
https://input-output.atlassian.net/browse/ATL-5861
https://input-output.atlassian.net/browse/ATL-5863

Checklist

• Self-reviewed the diff
• New code has inline documentation
• New code has proper comments/tests
• Any changes not covered by tests have been tested manually

[github-actions]

Lines	Statements	Branches	Functions
	67.07% (1691/2521)	53.37% (648/1214)	69.75% (512/734)

JUnit

Tests	Skipped	Failures	Errors	Time
383	2 💤	0 ❌	0 🔥	56.782s ⏱️

[milosh86]

Hey @curtis-h, great work really!

TBH, I like the implementation a lot, but I miss a bit of higher level context to be able to reason about it properly, like:

• what are the decision drivers here?
• what specific problem are we addressing exactly? why is it worth the effort?
• what's the ideal solution and under what assumptions?
• what are the tradeoffs of the chosen approach?

I think SDK is mature enough now to start addressing bigger changes in a more structured way (with a bit of good friction). Otherwise, we are going to solve the same problems again and again.

Please consider this only as a proposal to discuss inside the team, I'm ok with proceeding with this one as is, it (de facto) looks better than the existing implementation.

[curtis-h]

Hey @curtis-h, great work really!

TBH, I like the implementation a lot, but I miss a bit of higher level context to be able to reason about it properly, like:

• what are the decision drivers here?
• what specific problem are we addressing exactly? why is it worth the effort?
• what's the ideal solution and under what assumptions?
• what are the tradeoffs of the chosen approach?

I think SDK is mature enough now to start addressing bigger changes in a more structured way (with a bit of good friction). Otherwise, we are going to solve the same problems again and again.

Please consider this only as a proposal to discuss inside the team, I'm ok with proceeding with this one as is, it (de facto) looks better than the existing implementation.

Thanks @milosh86
here's an attempt to answer the questions:

what are the decision drivers here?

• Pluto interface large, unwieldy to implement
• any change to interface affects users downstream
• lack of testing / provability on our side

what specific problem are we addressing exactly? why is it worth the effort?

• Bloat, not all functions in the interface are required or even used in some cases.
• Pluto is currently an empty box for our users, with no guidance or tests to prove what they build.

what's the ideal solution and under what assumptions?

• Ideally, a fully implemented secure storage so there is no work for our users.
• Assumes no liability for security. Also that the users will be happy with our choice of store.

what are the tradeoffs of the chosen approach?

• Implmentation and Security is still the users responsibility.
• Logic and orchestration is managed and proven by us.

[curtis-h]

Schemas

• path to migrations
• encrypt individual fields within a table?
• needs definition

Query Selectors

• gt , lt , eq etc

Migrations

Rollbacks?

• later

Encrypted data

• in or out of control?
• if we offer Model, should we opine on what fields should be encrypted?
• what?
  • PrivateKeys
• password must be entered by user
  • keep in memory or remove after use?

[elribonazo]

Have added the missing demos and changes the alias as we agreed.

Awesome work man, well done!

[github-actions]

# npm audit report

ip  2.0.0
Severity: moderate
NPM IP package incorrectly identifies some private IP addresses as public - https://github.com/advisories/GHSA-78xj-cgh5-2h22
fix available via `npm audit fix`
node_modules/npm/node_modules/ip

1 moderate severity vulnerability

To address all issues, run:
  npm audit fix

[elribonazo]

The whole point of bringing this is to facilitate maintenance and future migrations on the user, developed side.

Migration paths are a key feature or our solution and not including them makes it harder for the user to maintain, etc.

[elribonazo]

I think this now bundles a complete migration solution that will support the existing community storage packages and allow us to maintain migrations on our side.

We can upgrade model attributes from A to B, but we can also completely remove A and migrate all its rows to a new model which is completely different.

[github-actions]

# npm audit report

ip  2.0.0
Severity: moderate
NPM IP package incorrectly identifies some private IP addresses as public - https://github.com/advisories/GHSA-78xj-cgh5-2h22
fix available via `npm audit fix`
node_modules/npm/node_modules/ip

1 moderate severity vulnerability

To address all issues, run:
  npm audit fix

[milosh86]

@curtis-h @elribonazo I reviewed this version in detail and I like it, looks quite clean and haven't noticed anything potentially problematic. We introduced RxDB, but in return we got some nice features out of the box.

@elribonazo one question for you. Are you happy now how this work relates to the pluto-encrypted package. Do you think that migration path for users is obvious?

UPDATE: Great job both of you 🙌💪

[elribonazo]

Are you happy now how this work relates to the pluto-encrypted package. Do you think that migration path for users is obvious?

@milosh86, yes really because in the current scenario this will not require much changes from the user in terms of code, re-implementing the storages, etc. So users still have encrypted (inmemory, or leveldb, or indexdb)

The migration is specified in the models and because models are right now provided by our package, this means that migration of data is something transparent for the users. The data will grow over time and this will facilitate transition to new models. This was in place before but because it was in pluto-encrypted and not in the SDK, for every change in the SDK, i have to release another version of pluto-encrypted.

This will never happen anymore, pluto-encrypted/database, pluto-encrypted/encryption will be deprecated and only the storages will be maintained.

Last, despite we have facilitated lots of things for our users, we are still not taking a decision on where it should be stored and how it should be encrypted, that's something that very advanced user's could implement themselves, the encryption happens at storage level and because storage doesn't speak models, doesn't know what they are maintenance is extremely simple for users.

[github-actions]

# npm audit report

ip  2.0.0
Severity: moderate
NPM IP package incorrectly identifies some private IP addresses as public - https://github.com/advisories/GHSA-78xj-cgh5-2h22
fix available via `npm audit fix`
node_modules/npm/node_modules/ip

1 moderate severity vulnerability

To address all issues, run:
  npm audit fix