Update Stronghold #691

abdulmth · 2022-03-07T08:34:36Z

Description of change

Update the Stronghold dependency to use the commit 969df405661ba4977f2cf30e9909cef7e30cefa2.
This update includes new error handling and procedures API.

Bug fix (a non-breaking change which fixes an issue)
Enhancement (a non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation Fix

PhilippGackstatter

Looks good, but we can remove a few things and also get rid of some error variants afterwards.

PhilippGackstatter · 2022-03-07T13:24:23Z

identity-account/src/stronghold/store.rs

-  pub async fn get(&self, location: Location) -> Result<Vec<u8>> {
-    match self.get_strict(location).await {
+  pub async fn get(&self, location: Location) -> IotaStrongholdResult<Option<Vec<u8>>> {
+    match self.get_strict(location.vault_path().to_vec()).await {
      Ok(data) => Ok(data),
-      Err(Error::StrongholdResult(message)) if message == STRONG_404 => Ok(Vec::new()),
+      Err(StrongholdError::StrongholdResult(message)) if message == STRONG_404 => Ok(Some(Vec::new())),
      Err(error) => Err(error),
    }
  }

  /// Gets a record.
-  pub async fn get_strict(&self, location: Location) -> Result<Vec<u8>> {
+  pub async fn get_strict(&self, location: Vec<u8>) -> IotaStrongholdResult<Option<Vec<u8>>> {
    let scope: _ = Context::scope(self.path, &self.name, &self.flags).await?;
-    let (data, status): (Vec<u8>, _) = scope.read_from_store(location).await;
-
-    status.to_result()?;
-
-    Ok(data)
+    Ok(scope.read_from_store(location).await?)
  }


We can remove get_strict, the STRONG_404 constant and use:

/// Gets a record or returns `None` if it doesn't exist. pub async fn get(&self, location: Location) -> IotaStrongholdResult<Option<Vec<u8>>> { let scope: _ = Context::scope(self.path, &self.name, &self.flags).await?; Ok(scope.read_from_store(location.vault_path().to_vec()).await?) }

The behaviour of read_from_store in stronghold has changed and it no longer returns an error if it doesn't find an entry but None instead, which is what we want.

PhilippGackstatter · 2022-03-07T13:28:45Z

identity-account/src/stronghold/error.rs

+  #[error("Record Error")]
+  RecordError,


It looks like the Records type is actually unused and you could probably remove all that. Also remove Snapshot::records in that case. Then you could also remove this error variant.

Do we not need records at all, even to implement listing all DIDs in a Stronghold #567 ?

I removed Records for now.

Do we not need records at all, even to implement listing all DIDs in a Stronghold #567 ?

I guess we could use it for that, yeah. I actually thought this was wrapping the stronghold record hints API which is going to be removed with the refactor, which is why I didn't look that closely at what Records was doing. I think we can also just reintroduce it in a future PR, if it turns out to be useful then, since it's already removed now.

I reverted the commit that removed the records 😅

PhilippGackstatter · 2022-03-07T13:45:45Z

identity-account/src/stronghold/context.rs

-  fn event_listeners(&self) -> Result<MutexGuard<'_, Vec<Listener>>> {
+  fn event_listeners(&self) -> IotaStrongholdResult<MutexGuard<'_, Vec<Listener>>> {
    self
      .event_listeners
      .lock()
-      .map_err(|_| Error::StrongholdMutexPoisoned("listeners"))
+      .map_err(|_| StrongholdError::StrongholdMutexPoisoned("listeners"))
  }


Looks like we could get rid of the error variant by doing:

match self.event_listeners.lock() { Ok(guard) => Ok(guard), Err(err) => Ok(err.into_inner()), }

And similarly for the other functions. Then we should be able to remove StrongholdError::StrongholdMutexPoisoned. Thanks for the hint on this one, @olivereanderson!

Do we want to give the guard and allow access if the mutex is poisoned? it sounds to me more of an error case, not sure if we want to ignore it completely.

I'm not entirely sure what to do here. A typical solution is to unwrap on the .lock() call, so if some other thread panicked, we panic as well. The current solution returns an error, and that at least gives the caller a chance to react to it. Basically, if some other part of the program already panicked, which we try to avoid in the first place, then it seems okay-ish to call unwrap(), because the error in the other thread was already severe enough to warrant a panic.
Using PoisonError::into_inner grants access to the MutexGuard despite the lock being poisoned, but it's a bit unclear to me whether the lock semantics are still being upheld, e.g. can two threads that haven't panicked access the data concurrently? I strongly assume not, in which case doing this seems okay.

I'm mainly trying to get rid of the error variant, because it doesn't neatly fall into a category of errors, which is what we will have to come up with to implement a more generic/abstract StorageError for the storage interface, but that's probably not that important to consider here.

Does anyone else have an opinion on what's best here, @cycraig or @olivereanderson?

@olivereanderson had a neat suggestion, which is to use the parking_lot::Mutex instead, which does not have poisoning in the first place, so we can avoid the error without a potential panic. parking_lot is not a direct dependency of the account yet, but it is used by our transitive dependencies (like tokio) already, so adding it seems totally fine.

Thanks Philipp and Oliver for the suggestion. I replaced it with parking_lot::Mutex.

identity-account/src/stronghold/error.rs

identity-account/src/stronghold/store.rs

PhilippGackstatter · 2022-03-07T14:36:25Z

identity-account/src/storage/stronghold.rs

+        // No state data found
+        if data.is_empty() {
+          return Ok(None);
+        }


I think since Store::get returns None if the key wasn't found, this doesn't need to be handled anymore, the way it was previously. In other words, we want IdentityState::from_json_slice to error if data.is_empty() because that indicates a storage error.

I removed the .is_empty() check.

This reverts commit fe8c90a.

PhilippGackstatter

Looks good to me now!

The only thing left is making sure there is no dependency on identity_account::error::Error within the stronghold directory. Try temporarily commenting the StrongholdError(#[from] StrongholdError), variant in the account error, and check that there are no errors within the stronghold directory. That will make it easier to split the account and storage into separate crates.

identity-account/src/stronghold/context.rs

PhilippGackstatter · 2022-03-09T10:12:32Z

identity-account/src/stronghold/context.rs

@@ -37,7 +40,7 @@ pub struct Context {

 async fn clear_expired_passwords() -> Result<()> {


This should also return an IotaStrongholdResult.

identity-account/src/stronghold/records.rs

@@ -48,18 +50,23 @@ impl Records<'_> {
  }

  pub async fn index(&self) -> Result<RecordIndex> {


Abdulrahim Al Methiab added 6 commits February 20, 2022 18:06

start handling stronghold errors

3527208

Merge branch 'dev' into chore/upgrade_stronghold

c574cb6

project compiles with new stronghold version

7f4d54e

fix tests

03dd14d

fix io error mapping

8f161be

improvements

95ff04d

abdulmth self-assigned this Mar 7, 2022

abdulmth added Chore Tedious, typically non-functional change Rust Related to the core Rust code. Becomes part of the Rust changelog. Added A new feature that requires a minor release. Part of "Added" section in changelog labels Mar 7, 2022

abdulmth marked this pull request as ready for review March 7, 2022 09:43

abdulmth requested a review from PhilippGackstatter March 7, 2022 09:43

abdulmth changed the title ~~Upgrade Stronghold~~ Update Stronghold Mar 7, 2022

PhilippGackstatter suggested changes Mar 7, 2022

View reviewed changes

Abdulrahim Al Methiab added 6 commits March 7, 2022 16:31

remove unnecessary empty check

9810cbf

remove records.rs

fe8c90a

remove get_strict

1c46543

remove StrongholdMutexPoisoned error by using parking_lot:Mutex

5dc49b1

remove unnecessary empty check

802ef8d

fix format issue

8a78ebe

cycraig mentioned this pull request Mar 8, 2022

Add separate identity-iota-core, identity-account-storage crates #693

Merged

10 tasks

Revert "remove records.rs"

dab4265

This reverts commit fe8c90a.

abdulmth requested a review from PhilippGackstatter March 9, 2022 08:58

PhilippGackstatter approved these changes Mar 9, 2022

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Stronghold #691

Update Stronghold #691

abdulmth commented Mar 7, 2022 •

edited

Loading

PhilippGackstatter left a comment

PhilippGackstatter Mar 7, 2022

PhilippGackstatter Mar 7, 2022

cycraig Mar 7, 2022

abdulmth Mar 8, 2022

PhilippGackstatter Mar 9, 2022

abdulmth Mar 9, 2022

PhilippGackstatter Mar 7, 2022

abdulmth Mar 7, 2022

PhilippGackstatter Mar 7, 2022 •

edited

Loading

PhilippGackstatter Mar 7, 2022

abdulmth Mar 8, 2022

PhilippGackstatter Mar 7, 2022

abdulmth Mar 8, 2022

PhilippGackstatter left a comment

PhilippGackstatter Mar 9, 2022

		@@ -37,7 +40,7 @@ pub struct Context {

		async fn clear_expired_passwords() -> Result<()> {

		@@ -48,18 +50,23 @@ impl Records<'_> {
		}

		pub async fn index(&self) -> Result<RecordIndex> {

Update Stronghold #691

Update Stronghold #691

Conversation

abdulmth commented Mar 7, 2022 • edited Loading

Description of change

PhilippGackstatter left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PhilippGackstatter Mar 7, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

PhilippGackstatter left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

abdulmth commented Mar 7, 2022 •

edited

Loading

PhilippGackstatter Mar 7, 2022 •

edited

Loading