Enable Merklized Distributions in Assets Pallet #5400

shawntabrizi · 2024-08-19T02:27:50Z

Depends on: #3881

This PR introduces a way for an asset issuer to distribute their token to many users using a single merkle root.

Users then present a merkle proof which is verified, and triggers the do_mint function.

A tracking storage ensures that each user only claims their distribution a single time.

The design allows for multiple distributions per asset id.

This reverts commit 1cfb29f.

Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>

…abrizi/polkadot-sdk into shawntabrizi-proving-trie

shawntabrizi · 2024-08-19T02:31:41Z

This PR is blocked because the data structures in binary-merkle-tree crate are not usable in the runtime because usize does not work with Encode, Decode, or TypeInfo.

…/shawntabrizi/polkadot-sdk into pr/5400

kianenigma · 2024-09-19T13:21:57Z

cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_assets_pool.rs

@@ -538,4 +538,20 @@ impl<T: frame_system::Config> pallet_assets::WeightInfo for WeightInfo<T> {
 			.saturating_add(T::DbWeight::get().reads(1_u64))
 			.saturating_add(T::DbWeight::get().writes(1_u64))
 	}
+
+	fn mint_distribution() -> Weight {
+		Weight::default()


TODO prior to merge

kianenigma · 2024-09-19T13:22:19Z

substrate/frame/assets/src/benchmarking.rs

@@ -139,6 +139,29 @@ fn assert_event<T: Config<I>, I: 'static>(generic_event: <T as Config<I>>::Runti
 	frame_system::Pallet::<T>::assert_has_event(generic_event.into());
 }

+// fn generate_merkle_trie<T: Config<I>, I: 'static>(items: u32) -> (DistributionHashOf<T, I>,


not needed?

kianenigma · 2024-09-19T13:24:40Z

substrate/frame/assets/src/lib.rs

@@ -404,6 +407,9 @@ pub mod pallet {
 		/// used to set up auto-incrementing asset IDs for this collection.
 		type CallbackHandle: AssetsCallback<Self::AssetId, Self::AccountId>;

+		/// A type used to verify merkle proofs used for distributions.
+		type VerifyExistenceProof: VerifyExistenceProof<Hash: Parameter + MaxEncodedLen + Default>;


Suggested change

type VerifyExistenceProof: VerifyExistenceProof<Hash: Parameter + MaxEncodedLen + Default>;

type VerifyExistenceProof: VerifyExistenceProof<Hash = <Self as frame_system>::Hash>;

or sth along these lines?

kianenigma · 2024-09-19T13:25:49Z

substrate/frame/assets/src/types.rs

+
+#[derive(Eq, PartialEq, Copy, Clone, RuntimeDebug, Encode, Decode, TypeInfo, MaxEncodedLen)]
+pub struct DistributionInfo<AssetId, Hash> {
+	// The asset id we are distributing.
+	pub asset_id: AssetId,
+	// The merkle root which represents all the balances to distribute.
+	pub merkle_root: Hash,
+	// Whether the distribution is still active.
+	pub active: bool,
+}


Suggested change

#[derive(Eq, PartialEq, Copy, Clone, RuntimeDebug, Encode, Decode, TypeInfo, MaxEncodedLen)]

pub struct DistributionInfo<AssetId, Hash> {

// The asset id we are distributing.

pub asset_id: AssetId,

// The merkle root which represents all the balances to distribute.

pub merkle_root: Hash,

// Whether the distribution is still active.

pub active: bool,

}

/// Foo bar baz

#[derive(Eq, PartialEq, Copy, Clone, RuntimeDebug, Encode, Decode, TypeInfo, MaxEncodedLen)]

pub struct DistributionInfo<AssetId, Hash> {

/// The asset id we are distributing.

pub asset_id: AssetId,

/// The merkle root which represents all the balances to distribute.

pub merkle_root: Hash,

/// Whether the distribution is still active.

pub active: bool,

}

kianenigma · 2024-09-19T13:26:36Z

substrate/frame/assets/src/lib.rs

@@ -1798,6 +1848,92 @@ pub mod pallet {
 			)?;
 			Ok(())
 		}
+
+		/// Mint a distribution of assets of a particular class.


Something about this can go into the main top docs and then in https://docs.rs/pallet-assets/latest/pallet_assets/?

kianenigma

Some small nits remaining, but the overall logic is good

Ank4n

First pass. Haven't looked yet at the proving trie code.

Ank4n · 2024-09-19T20:47:13Z

substrate/frame/assets/src/lib.rs

+		DistributionActive,
+		/// The proof provided could not be verified.
+		BadProof,
+		/// The a leaf node was extracted from the proof, but it did not match the expected format.


Suggested change

/// The a leaf node was extracted from the proof, but it did not match the expected format.

/// The leaf node was extracted from the proof, but it did not match the expected format.

Ank4n · 2024-09-19T20:51:46Z

substrate/frame/assets/src/functions.rs

+	pub(super) fn do_mint_distribution(
+		id: T::AssetId,
+		merkle_root: DistributionHashOf<T, I>,
+		maybe_check_issuer: Option<T::AccountId>,


why Option? Is always called with Some() ?

Ank4n · 2024-09-19T21:06:50Z

substrate/frame/assets/src/functions.rs

+		if all_refunded {
+			Self::deposit_event(Event::<T, I>::DistributionCleaned { distribution_id });
+			// Refund weight only the amount we actually used.
+			Ok(Some(T::WeightInfo::destroy_distribution(refund_count)).into())


What is the incentive to clean up state? Shouldn't we refund all fees?

Ank4n · 2024-09-19T21:07:41Z

substrate/frame/assets/src/functions.rs

+
+		info.active = false;
+
+		MerklizedDistribution::<T, I>::insert(&distribution_id, info);


Is this never cleaned? Would we not just keep bloating state then?

Ank4n · 2024-09-19T21:10:28Z

substrate/frame/assets/src/lib.rs

+		) -> DispatchResult {
+			let origin = ensure_signed(origin)?;
+			let id: T::AssetId = id.into();
+			Self::do_mint_distribution(id, merkle_root, Some(origin))?;


Thinking out loud: Should we not take some kind of deposit here (charging for the storage), that is only refunded when state (MerklizedDistributionTracker) is cleaned up? What stops some one to create large number of distributions?

Ank4n · 2024-09-19T21:12:46Z

substrate/frame/assets/src/lib.rs

+		///
+		/// Weight: `O(N)` where `N` is the maximum number of elements that can be removed at once.
+		#[pallet::call_index(36)]
+		#[pallet::weight(T::WeightInfo::destroy_distribution(100u32))] // TODO


right?

Suggested change

#[pallet::weight(T::WeightInfo::destroy_distribution(100u32))] // TODO

#[pallet::weight(T::WeightInfo::destroy_distribution(T::RemoveItemsLimit::get()))] // TODO

shawntabrizi and others added 28 commits March 28, 2024 14:05

initial ideas

d50d5e4

Update proving_trie.rs

7f4422e

create trait

bdc0c84

use trait

da385ab

clean up trait and basic trie further

6ab4bfb

use trait in session historical

1cfb29f

fix api, add basic end to end test

3080c7b

fix test

39b0fca

Revert "use trait in session historical"

599f576

This reverts commit 1cfb29f.

Merge branch 'master' into shawntabrizi-proving-trie

a1c8886

Merge branch 'master' into shawntabrizi-proving-trie

6bde2a3

Update substrate/primitives/runtime/src/proving_trie.rs

ead7951

Co-authored-by: Ankan <10196091+Ank4n@users.noreply.github.com>

Merge branch 'master' into shawntabrizi-proving-trie

0cac048

fix some feedback

acfe734

update name

45f4287

docs and multi value proof

efbe1c9

improve test

cfa62ce

add multi-value query test

885fd94

Merge branch 'master' into shawntabrizi-proving-trie

18a5c2e

Create pr_3881.prdoc

5e3e518

Merge branch 'shawntabrizi-proving-trie' of https://github.com/shawnt…

f35cacc

…abrizi/polkadot-sdk into shawntabrizi-proving-trie

use v1

e6c3759

Merge branch 'master' into shawntabrizi-proving-trie

89679c3

initial idea

b40c96a

some bs code to make things compile

37052ee

more stuff

08f2b35

complete logic

118a51c

add verification check

44bc5bb

Merge branch 'master' into shawntabrizi-proving-trie

6e853ba

shawntabrizi added 3 commits September 16, 2024 20:52

make work with binary tree

122f9cc

clean dead code

4d26deb

Merge branch 'master' into shawntabrizi-assets-distribution

8e6fcd5

shawntabrizi marked this pull request as ready for review September 17, 2024 02:23

shawntabrizi requested a review from a team as a code owner September 17, 2024 02:23

shawntabrizi added T1-FRAME This PR/Issue is related to core FRAME, the framework. T2-pallets This PR/Issue is related to a particular pallet. and removed T1-FRAME This PR/Issue is related to core FRAME, the framework. labels Sep 17, 2024

shawntabrizi added 15 commits September 17, 2024 11:04

Merge branch 'master' into shawntabrizi-assets-distribution

b17a7c4

two benchmarks

fcfda22

Merge branch 'shawntabrizi-assets-distribution' of https://github.com…

6bf7b91

…/shawntabrizi/polkadot-sdk into pr/5400

more generic, remove keys limit

42cdc26

test fixes

12ed0fb

update API name for consistent

9c8ab8a

destroy distribution benchmark

b4f480c

need to abstract this

40bd086

initial stuff

7e22961

more

1b13d9a

fix up binary tree

1ac0545

refactor

7aaabe3

fixes

9013156

update api for a single opaque blob

c98b882

Merge branch 'master' into shawntabrizi-assets-distribution

42259eb

kianenigma reviewed Sep 19, 2024

View reviewed changes

kianenigma approved these changes Sep 19, 2024

View reviewed changes

Ank4n reviewed Sep 19, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable Merklized Distributions in Assets Pallet #5400

Enable Merklized Distributions in Assets Pallet #5400

shawntabrizi commented Aug 19, 2024 •

edited

Loading

shawntabrizi commented Aug 19, 2024

kianenigma Sep 19, 2024

kianenigma Sep 19, 2024 •

edited

Loading

kianenigma Sep 19, 2024

kianenigma Sep 19, 2024

kianenigma Sep 19, 2024

kianenigma left a comment

Ank4n left a comment

Ank4n Sep 19, 2024

Ank4n Sep 19, 2024

Ank4n Sep 19, 2024

Ank4n Sep 19, 2024

Ank4n Sep 19, 2024

Ank4n Sep 19, 2024

	type VerifyExistenceProof: VerifyExistenceProof<Hash: Parameter + MaxEncodedLen + Default>;
	type VerifyExistenceProof: VerifyExistenceProof<Hash = <Self as frame_system>::Hash>;

	/// The a leaf node was extracted from the proof, but it did not match the expected format.
	/// The leaf node was extracted from the proof, but it did not match the expected format.


		info.active = false;

		MerklizedDistribution::<T, I>::insert(&distribution_id, info);

	#[pallet::weight(T::WeightInfo::destroy_distribution(100u32))] // TODO
	#[pallet::weight(T::WeightInfo::destroy_distribution(T::RemoveItemsLimit::get()))] // TODO

Enable Merklized Distributions in Assets Pallet #5400

Are you sure you want to change the base?

Enable Merklized Distributions in Assets Pallet #5400

Conversation

shawntabrizi commented Aug 19, 2024 • edited Loading

shawntabrizi commented Aug 19, 2024

Choose a reason for hiding this comment

kianenigma Sep 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kianenigma left a comment

Choose a reason for hiding this comment

Ank4n left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shawntabrizi commented Aug 19, 2024 •

edited

Loading

kianenigma Sep 19, 2024 •

edited

Loading