contracts: Don't rely on reserved balances keeping an account alive #13369

athei · 2023-02-12T23:25:01Z

Motivation

Up until now we relied on automatic storage deposits to contracts to keep a contract's account from being reaped. Those storage deposit were reserved in order to prevent the contract from spending it. A nice side effect was that we could use it to guarantee the existence of a contract's account as long as the contract itself existed.

With #12951 reserved balance does no longer contribute to the existential deposit. We need to make modifications to stay compatible.

High Level Changes

The high level changes we made need to address the following needs.

Keep the contract from spending the deposit

The storage deposit will not be send to the contracts account but to a different account. Since this account is not controlled by anyone it is safe to leave the balance as free. Separating the accounts will also make sure that no slashing bugs in other pallets will break our assumptions.
We add a consumer reference to this new deposit account. This is not strictly necessary as no one can spend those funds , though.

Make sure the contract's account does not get reaped

On contract instantiation we always collect the ed from origin (as part of the storage deposit) and use it to pull the contract's account into existence.
We take a consumer reference to the contract's account. This makes sure that any attempt by the contract to send the ed away will fail the transaction in question. Even slashing will not remove the account and leave the ed. Not sure if any security risks can arise from this.

Why the huge diff?

The diff seems to be quite big for something that seems like a small change. The reasons are the following.

Test changes

Always sending the ed to a new contract broke a lost of tests and created a lot of changes. Especially since it affected the emitted events.

Refactoring of `storage.rs`

Since I needed to do some changes to the file which had rather old code I took the opportunity to move some code around. I also made all the fields private when this didn't require even more refactoring. In the future they should all become private.

Moved address derivation into its own module

This began to clutter lib.rs too much.

gavofyork · 2023-02-13T10:00:49Z

The consumer reference on the deposit account means that it cannot have its balance reduced below ED (until the consumer reference is lifted). Is this acceptable/handled properly?

Unless there are no production chains using this pallet (and I believe there are) then there will need to be migration code to transfer the existing storage deposit into the new account. Instead of doing it all at once, it could be that it gets transferred lazily, either when the contract is used or when the contract account is pinged by a "upgrade-this-account" extrinsic with Pays::No if the account was able to be upgraded.

athei · 2023-02-13T13:05:40Z

The consumer reference on the deposit account means that it cannot have its balance reduced below ED (until the consumer reference is lifted). Is this acceptable/handled properly?

Yes. The base deposit (just for the contract itself) is > ed and is only removed when the contract self destructs. The account can't be used for any slashable activity so there is no way it can be reduced below the ed without the contract being removed (which also removes the reference).

Unless there are no production chains using this pallet (and I believe there are) then there will need to be migration code to transfer the existing storage deposit into the new account.

There is shiden (astar kusma parachain) which is relying on my migrations for some time. Also Astar and Azero are scheduled to launch the pallet on their mainnet with the latest polkadot release.

Instead of doing it all at once, it could be that it gets transferred lazily, either when the contract is used or when the contract account is pinged by a "upgrade-this-account" extrinsic with Pays::No if the account was able to be upgraded.

This "upgraded-this-account" logic is only available after #12951 is merged, right?

agryaznov

Some docs need an update, like e.g. here:

substrate/frame/contracts/primitives/src/lib.rs

Lines 50 to 54 in 50ab7fb

    
           /// How much balance was deposited and reserved during execution in order to pay for storage. 
        
           /// 
        
           /// The storage deposit is never actually charged from the caller in case of [`Self::result`] 
        
           /// is `Err`. This is because on error all storage changes are rolled back. 
        
           pub storage_deposit: StorageDeposit<Balance>,

and here:

substrate/frame/contracts/primitives/src/lib.rs

Lines 158 to 169 in 50ab7fb

    
           pub enum StorageDeposit<Balance> { 
        
           	/// The transaction reduced storage consumption. 
        
           	/// 
        
           	/// This means that the specified amount of balance was transferred from the involved 
        
           	/// contracts to the call origin. 
        
           	Refund(Balance), 
        
           	/// The transaction increased overall storage usage. 
        
           	/// 
        
           	/// This means that the specified amount of balance was transferred from the call origin 
        
           	/// to the contracts involved. 
        
           	Charge(Balance), 
        
           }

Also, this PR does not affect the way amount is reserved (and unreserved) on the contract code uploader account:

substrate/frame/contracts/src/wasm/code_cache.rs

Line 91 in 50ab7fb

T::Currency::reserve(&owner_info.owner, owner_info.deposit)

and

substrate/frame/contracts/src/wasm/code_cache.rs

Line 139 in 50ab7fb

T::Currency::unreserve(&owner_info.owner, owner_info.deposit);

Just to double-check if that's ok. Per my understanding, once #12951 is merged, we probably can't leave it as is:

There are no equivalents for ReservableCurrency::reserve or ::unreserve: an identifier must always be used.

frame/contracts/src/storage/meter.rs

agryaznov · 2023-02-13T10:47:06Z

frame/contracts/src/storage/meter.rs

@@ -285,7 +282,7 @@ where
 	pub fn absorb(


needs doc comments update

this lil niggle is not addressed

You sure? I think I did.

frame/contracts/src/exec.rs

gavofyork · 2023-02-13T16:51:04Z

This "upgraded-this-account" logic is only available after #12951 is merged, right?

Surely this logic can be merged without breaking 12951, no? I don't think anything in the current pre-12951 logic would break these changes. So then you'd write this migration extrinsic in the contracts pallet, and ideally provide a script which could be run to migrate all accounts over lazily.

The general idea is implemented for migration in 12951, but obviously this migration logic would be specific to the contracts pallet, so you'd need to introduce the extrinsic yourself in this PR.

athei · 2023-02-13T17:46:01Z

Ahh okay I get it this statement is about the general approach to lazy migrations. In general, yes I want all contract migrations to be lazy as they tend to be heavy.

My confusion is about the order of migrations: What if the migration in #12951 is applied before the contracts migration?

In my mind (correct me if I am wrong) it makes only sense in this order:

Merge this PR including the to be written lazy migration for all accounts.
Merge Deprecate Currency; introduce holds and freezing into fungible traits #12951.
Port all code away from Currency (including contracts).
Only then apply the migration in Deprecate Currency; introduce holds and freezing into fungible traits #12951.

gavofyork · 2023-02-14T12:36:34Z

Merging #12951 and beginning lazy migration of #12951 is really the same action. The logic cannot be merged without requiring any accounts touched to be bought up to date with the new logic.

I'd feel better if we did:

Merge this PR including the to be written lazy migration for all accounts.
(For chains actively using Contracts pallet): Migrate all contracts to the new logic.
Apply Deprecate Currency; introduce holds and freezing into fungible traits #12951, and allow migration to proceed lazily.
Deprecate Currency and migrate all pallets to use fungible, migrating lazily.

athei · 2023-02-14T16:42:26Z

After some discussion in the FRAME channel we agreed that we can merge this without the storage migration in order to get #12951 in quickly. Reason is that no chain that uses pallet-contracts actively depends on master.

Plan is:

Merge this.
Merge Deprecate Currency; introduce holds and freezing into fungible traits #12951
Create lazy/multi block migrations for pallet-contracts

The added benefit is that I can then assume that accounts are already upgraded and exist without reserved balance (when I touch the account in my migration it will be lazily upgraded). This assumptions makes my migration easier: I can just move over all the reserved balance to the deposit account without destroying the contract's account.

All of that will be included in the same release which is then safe to use for existing pallet-contracts deployments.

frame/contracts/src/address.rs

Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>

athei · 2023-02-15T18:44:52Z

Also, this PR does not affect the way amount is reserved (and unreserved) on the contract code uploader account:
Just to double-check if that's ok. Per my understanding, once #12951 is merged, we probably can't leave it as is:
There are no equivalents for ReservableCurrency::reserve or ::unreserve: an identifier must always be used.

#12951 does not remove ReservableCurrency, yet. The quote is from the part where Gav talks about how to eventually port away from (Reserveable)Currency. But this will happen in a later PR (after #12951). Right now we just need to make sure that the existing reserve logic makes sense with the changes in #12951 so it can be merged.

We have the assumption (before this PR) that reserved balance contributes to the existential deposit. This was true so far. This changes with #12951 because it makes other things easier and more consistent. The solution to this involves moving away from reserves but this is not because ReservableCurrency is removed with #12951. It will still be there for some time.

We only had this assumption for the storage deposits that go to contracts. We don't have this assumption when reserving for the code. Hence we don't need to change this part (yet).

frame/contracts/src/address.rs

frame/contracts/src/benchmarking/mod.rs

Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

athei · 2023-02-16T12:01:52Z

bot bench -v PIPELINE_SCRIPTS_REF=bm-fallback $ pallet dev pallet_contracts

command-bot · 2023-02-16T12:01:57Z

@athei https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2402302 was started for your command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev pallet_contracts. Check out https://gitlab.parity.io/parity/mirrors/substrate/-/pipelines?page=1&scope=all&username=group_605_bot to know what else is being executed currently.

Comment bot cancel 34-7325575e-e1b2-4ad3-bf81-c85638c05db0 to cancel this command or bot cancel to cancel all commands in this pull request.

command-bot · 2023-02-16T14:08:32Z

@athei Command "$PIPELINE_SCRIPTS_DIR/commands/bench/bench.sh" pallet dev pallet_contracts has finished. Result: https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2402302 has finished. If any artifacts were generated, you can download them from https://gitlab.parity.io/parity/mirrors/substrate/-/jobs/2402302/artifacts/download.

agryaznov

can complain nothing but some typos in documentation

agryaznov · 2023-02-16T14:10:20Z

frame/contracts/src/storage/meter.rs

@@ -285,7 +282,7 @@ where
 	pub fn absorb(


this lil niggle is not addressed

frame/contracts/primitives/src/lib.rs

Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>

athei · 2023-02-16T15:09:10Z

bot merge

paritytech-processbot · 2023-02-16T15:09:39Z

Waiting for commit status.

…13369) * Move storage deposits to their own account * Take ed for contract's account from origin * Apply suggestions from code review Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * Update stale docs * Use 16 bytes prefix for address derivation * Update frame/contracts/src/address.rs Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> * Fix merge * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_contracts * Update frame/contracts/primitives/src/lib.rs Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> --------- Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> Co-authored-by: command-bot <>

… alive (#13369)" This reverts commit c23c308.

…aritytech#13369) * Move storage deposits to their own account * Take ed for contract's account from origin * Apply suggestions from code review Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * Update stale docs * Use 16 bytes prefix for address derivation * Update frame/contracts/src/address.rs Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> * Fix merge * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_contracts * Update frame/contracts/primitives/src/lib.rs Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> --------- Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> Co-authored-by: command-bot <>

…13369) * Move storage deposits to their own account * Take ed for contract's account from origin * Apply suggestions from code review Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * Update stale docs * Use 16 bytes prefix for address derivation * Update frame/contracts/src/address.rs Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> * Fix merge * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_contracts * Update frame/contracts/primitives/src/lib.rs Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> --------- Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> Co-authored-by: command-bot <>

…aritytech#13369) * Move storage deposits to their own account * Take ed for contract's account from origin * Apply suggestions from code review Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> * Update stale docs * Use 16 bytes prefix for address derivation * Update frame/contracts/src/address.rs Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> * Fix merge * ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_contracts * Update frame/contracts/primitives/src/lib.rs Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> --------- Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com> Co-authored-by: command-bot <>

athei added 2 commits February 11, 2023 12:45

Move storage deposits to their own account

b0f14f2

Take ed for contract's account from origin

50ab7fb

athei requested review from agryaznov and gavofyork February 12, 2023 23:25

github-actions bot added the A0-please_review Pull request needs code review. label Feb 12, 2023

athei added B7-runtimenoteworthy C1-low PR touches the given topic and has a low impact on builders. D9-needsaudit 👮 PR contains changes to fund-managing logic that should be properly reviewed and externally audited and removed A0-please_review Pull request needs code review. labels Feb 12, 2023

the-right-joyce added A0-please_review Pull request needs code review. B1-note_worthy Changes should be noted in the release notes T1-runtime This PR/Issue is related to the topic “runtime”. and removed A0-please_review labels Feb 13, 2023

agryaznov suggested changes Feb 13, 2023

View reviewed changes

athei requested a review from xermicus February 14, 2023 16:43

xermicus reviewed Feb 15, 2023

View reviewed changes

frame/contracts/src/address.rs Outdated Show resolved Hide resolved

frame/contracts/src/address.rs Outdated Show resolved Hide resolved

frame/contracts/src/address.rs Outdated Show resolved Hide resolved

athei and others added 3 commits February 15, 2023 19:05

Apply suggestions from code review

1190527

Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com> Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>

Update stale docs

1ee7dd6

Use 16 bytes prefix for address derivation

4c57c30

athei requested review from xermicus and agryaznov February 15, 2023 18:45

xermicus approved these changes Feb 16, 2023

View reviewed changes

frame/contracts/src/address.rs Outdated Show resolved Hide resolved

frame/contracts/src/benchmarking/mod.rs Show resolved Hide resolved

athei and others added 3 commits February 16, 2023 12:44

Update frame/contracts/src/address.rs

01240db

Co-authored-by: Cyrill Leutwiler <bigcyrill@hotmail.com>

Merge branch 'master' into at/deposit-rework

1cf7fde

Fix merge

c9797c0

".git/.scripts/commands/bench/bench.sh" pallet dev pallet_contracts

39305d7

agryaznov approved these changes Feb 16, 2023

View reviewed changes

Update frame/contracts/primitives/src/lib.rs

a066564

Co-authored-by: Sasha Gryaznov <hi@agryaznov.com>

paritytech-processbot bot merged commit e8178c9 into master Feb 16, 2023

paritytech-processbot bot deleted the at/deposit-rework branch February 16, 2023 15:58

rossbulat pushed a commit that referenced this pull request Feb 19, 2023

Revert "contracts: Don't rely on reserved balances keeping an account…

f65a281

… alive (#13369)" This reverts commit c23c308.

athei mentioned this pull request Mar 19, 2023

contracts: Migrate to fungible traits #13643

Closed

kiltbot mentioned this pull request Apr 3, 2023

[AUTOMATIC] Update Polkadot dependencies from 0.9.38 to 0.9.39 KILTprotocol/kilt-node#495

Closed

librelois mentioned this pull request Apr 6, 2023

Update substrate/polkadot/cumulus from v0.9.38 to v0.9.40 moonbeam-foundation/moonbeam#2200

Closed

athei mentioned this pull request May 4, 2023

contracts: Multi block migrations #14045

Merged

ascjones mentioned this pull request May 12, 2023

🦖 fix e2e test, account for contract initial free balance use-ink/ink#1777

Merged

kacperzuk-neti mentioned this pull request Jun 23, 2023

Polkadot v0.9.43 liberland/liberland_substrate#295

Merged

15 tasks

Marcin-Radecki mentioned this pull request Feb 8, 2024

A0-4022: Enable pallet contract migrations v12 to v15 Cardinal-Cryptography/aleph-node#1610

Merged

ashutoshvarma mentioned this pull request Feb 27, 2024

feat: update to polkadot v1.1.0 release AstarNetwork/Astar#1182

Merged

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

contracts: Don't rely on reserved balances keeping an account alive #13369

contracts: Don't rely on reserved balances keeping an account alive #13369

athei commented Feb 12, 2023 •

edited

Loading

gavofyork commented Feb 13, 2023

athei commented Feb 13, 2023

agryaznov left a comment

agryaznov Feb 13, 2023

agryaznov Feb 16, 2023

athei Feb 16, 2023

gavofyork commented Feb 13, 2023

athei commented Feb 13, 2023 •

edited

Loading

gavofyork commented Feb 14, 2023

athei commented Feb 14, 2023 •

edited

Loading

athei commented Feb 15, 2023 •

edited

Loading

athei commented Feb 16, 2023

command-bot bot commented Feb 16, 2023 •

edited

Loading

command-bot bot commented Feb 16, 2023

agryaznov left a comment

agryaznov Feb 16, 2023

athei commented Feb 16, 2023

paritytech-processbot bot commented Feb 16, 2023

	/// How much balance was deposited and reserved during execution in order to pay for storage.
	///
	/// The storage deposit is never actually charged from the caller in case of [`Self::result`]
	/// is `Err`. This is because on error all storage changes are rolled back.
	pub storage_deposit: StorageDeposit<Balance>,

	pub enum StorageDeposit<Balance> {
	/// The transaction reduced storage consumption.
	///
	/// This means that the specified amount of balance was transferred from the involved
	/// contracts to the call origin.
	Refund(Balance),
	/// The transaction increased overall storage usage.
	///
	/// This means that the specified amount of balance was transferred from the call origin
	/// to the contracts involved.
	Charge(Balance),
	}

contracts: Don't rely on reserved balances keeping an account alive #13369

contracts: Don't rely on reserved balances keeping an account alive #13369

Conversation

athei commented Feb 12, 2023 • edited Loading

Motivation

High Level Changes

Keep the contract from spending the deposit

Make sure the contract's account does not get reaped

Why the huge diff?

Test changes

Refactoring of storage.rs

Moved address derivation into its own module

gavofyork commented Feb 13, 2023

athei commented Feb 13, 2023

agryaznov left a comment

Choose a reason for hiding this comment

agryaznov Feb 13, 2023

Choose a reason for hiding this comment

agryaznov Feb 16, 2023

Choose a reason for hiding this comment

athei Feb 16, 2023

Choose a reason for hiding this comment

gavofyork commented Feb 13, 2023

athei commented Feb 13, 2023 • edited Loading

gavofyork commented Feb 14, 2023

athei commented Feb 14, 2023 • edited Loading

athei commented Feb 15, 2023 • edited Loading

athei commented Feb 16, 2023

command-bot bot commented Feb 16, 2023 • edited Loading

command-bot bot commented Feb 16, 2023

agryaznov left a comment

Choose a reason for hiding this comment

agryaznov Feb 16, 2023

Choose a reason for hiding this comment

athei commented Feb 16, 2023

paritytech-processbot bot commented Feb 16, 2023

athei commented Feb 12, 2023 •

edited

Loading

Refactoring of `storage.rs`

athei commented Feb 13, 2023 •

edited

Loading

athei commented Feb 14, 2023 •

edited

Loading

athei commented Feb 15, 2023 •

edited

Loading

command-bot bot commented Feb 16, 2023 •

edited

Loading