Minting data types and initial LedgerDb support #1537

eranrund · 2022-02-23T21:11:31Z

Motivation

As part of the minting proof of concept work we know that we need the following:

A data structure to represent the currently active minting configurations - this controls who is currently allowed to mint which tokens, and at what limit
This configuration needs to be stored inside the ledger, so that at any point we can have consensus query the configuration for a given token id, as well as update the total amount minted via a given configuration. This will be used by the new minting transaction validation code and the updated block forming logic.
New SCP transactions for minting and updating the minting configuration.

In this PR

This PR contains some of the initial building blocks for this:

New data structures to represent a multisig-signed minting transaction, multisig-signed set-minting-config transaction and other associated data structures
API changes to LedgerDB that allow us to store the list of currently active minting configurations per token id, the list of already-seen nonces, and the total amount minted by a specific configuration.

Future Work

Include the new transaction types inside BlockContents and modify append_block() to propagate new minting configurations into the new MintConfigStore
Transaction validation code for the new transaction types
SCP changes to support the new transaction types
Enclave block forming changes to support minting
Tests
Probably many other things...

consensus/service/src/config/tokens.rs

ledger/db/src/lib.rs

wjuan-mob · 2022-02-28T20:16:14Z

ledger/migration/src/lib.rs

+
+                let mut db_txn = env.begin_rw_txn().expect("Failed starting rw transaction");
+                metadata_store
+                    .set_version_to_latest(&mut db_txn)


Sorry for my ignorance, but is there a benefit of setting to latest vs specifying 20220222 here?

Good catch, that was actually a potential bug!
We need to be explicit with the version we set to (and not just set to latest) so that incremental migration steps would work.

wjuan-mob · 2022-02-28T20:17:34Z

ledger/db/src/tx_out_store.rs

@@ -796,7 +796,7 @@ pub mod tx_out_store_tests {
    use std::path::Path;
    use tempdir::TempDir;

-    fn get_env() -> Environment {
+    pub fn get_env() -> Environment {


If we're making this a public function, maybe we should add a doc comment? Or is it not necessary here?

I'll add a comment, but in general I don't think we made it a priority to document test code thoroughly.

wjuan-mob · 2022-02-28T20:20:36Z

ledger/db/src/lib.rs

 pub use tx_out_store::TxOutStore;

 pub const MAX_LMDB_FILE_SIZE: usize = 2usize.pow(40); // 1 TB
-pub const MAX_LMDB_DATABASES: u32 = 22; // maximum number of databases in the lmdb file
+pub const MAX_LMDB_DATABASES: u32 = 24; // maximum number of databases in the lmdb file


Maybe it would help to note somewhere in the PR body that we are adding 2 DBs? A mapping of token id -> currently active mint configurations and a mapping of nonce -> SetMintConfigTx?

I think that becomes clear when reading the code, not sure where I would note this / who would ever look at it after it gets merged.

wjuan-mob · 2022-02-28T20:27:39Z

ledger/db/src/mint_config_store.rs

+//!      1) It allows transaction validation code to figure out if a mint
+//! transaction is allowed to mint.
+//!      2) It enables keeping track of how much was minted using a given
+//! configuration. This is used to enforce the per-configuration mint limit. 2)


Nit: I think this 2) should start on the next line.

wjuan-mob · 2022-02-28T20:39:46Z

ledger/db/src/mint_config_store.rs

+    }
+
+    /// Set mint configurations for a given token.
+    pub fn set_active_mint_configs(


Sorry I was a little confused by the intention of this. Could we add a test that demonstrates the expected behaviour when we deactivate a mint config via this method or reactivate a mint config via this method?

I believe this is covered by set_get_behaves_correctly

I believe that set_get_behaves_correctly is testing setting the mint config and retrieving that same config for 2 different tokens. What I'm unsure about is the behaviour of deactivating an active config on a token, e.g. we go from having an active mint config to an empty active mint config, perhaps after having already minted some tokens. Then reactivating the previously deactivated config.

Minting configurations can be replaced at any time and do not affect anything previously minted, they just control who can currently mint. There is no deactivation - other than by replacing the currently active configuration with an empty oneset_replaces_configuration checks that an existing configuration can be replaced by a new one. I'll add a test that verifies an empty one is accepted.

wjuan-mob · 2022-02-28T20:49:41Z

consensus/service/src/config/tokens.rs

@@ -447,7 +574,7 @@ mod tests {
        assert_eq!(tokens, tokens2);

        // Validation should fail since allow_any_fee cannot be used on non-MOB tokens.
-        assert!(tokens.validate().is_err());
+        assert_validation_error(&tokens, "MOB token configuration not found");


Nit: Shouldn't this be: allow_any_fee can only be used for MOB

jcape · 2022-02-28T20:51:21Z

consensus/service/src/config/tokens.rs

+    /// DER representation of Ed25519 public keys.
+    #[derive(Serialize, Deserialize)]
+    struct DerSignerSet {
+        signers: Vec<String>,


DER is the binary format, and that gets Base64 encoded for PEM, so other places where I'm talking about DER, I'm using Vec<Vec<u8>>.

I guess my question is: why don't we take PEM-encoded keys as config, and pass DER bytes around (or just use SignerSet as-is)?

c.f. https://docs.rs/pem/latest/pem/fn.parse_many.html

remoun · 2022-02-28T22:33:03Z

consensus/service/src/config/tokens.rs

@@ -521,4 +655,176 @@ mod tests {
            Some(1)
        );
    }
+
+    #[test]
+    fn master_minters_deserialize_serialize_deserialize_works() {


Nit: This test only does one ser-der round trip, so remove the first deserialize_

remoun · 2022-02-28T22:42:14Z

ledger/db/src/ledger_trait.rs

+    fn get_active_mint_configs(&self, token_id: TokenId) -> Result<Vec<ActiveMintConfig>, Error>;
+
+    /// Update the total minted amount for a given MintConfig.
+    fn update_total_minted(&self, mint_config: &MintConfig, amount: u64) -> Result<(), Error>;


This seems racy if there are multiple active minters. I think it would be more robust as a log_minted kind of method that increments the total_minted, which would be more retryable.

IIUC we're only planning on having one minter, so this might not be an issue in practice, but also the log_minted approach could enforce that the total is monotonically increasing.

I don't believe this is racy since this the read-modify-write is done inside a single LMDB transaction.

Just to clarify, the total minted amount is tracked per minting configuration. So there could be multiple MintConfigs currently configured, but the total minted amount is tracked individually for each one.

A followup PR might actually remove this from the public interface, since it can be done inside append_block - as the block has information about who minted what so there will not be a need for an external caller to call this.

remoun · 2022-03-01T00:08:46Z

consensus/service/src/config/tokens.rs

-                .collect(),
-            threshold: signer_set.threshold(),
+                .map(|signer| Pem {
+                    tag: String::from("FOO"),


How about "signer"?

Suggested change

tag: String::from("FOO"),

tag: "signer".to_string(),

oops, I missed that. it should be PUBLIC KEY to match what openssl generates, but in general has no meaningful implication.

consensus/service/src/config/tokens.rs

remoun · 2022-03-01T18:32:40Z

ledger/db/src/mint_config_store.rs

+            .ok_or_else(|| Error::InvalidMintConfig("Mint config not found".to_string()))?;
+
+        // Update the total minted amount.
+        active_mint_config.total_minted = amount;


Consider enforcing that amount > total_minted here, and adding a test that exercises it.

jcape · 2022-03-01T19:29:33Z

ledger/db/src/lib.rs

    #[allow(clippy::unreadable_literal)]
-    const LATEST_VERSION: u64 = 20200707;
+    const LATEST_VERSION: u64 = 20220222;


Suggested change

#[allow(clippy::unreadable_literal)]

const LATEST_VERSION: u64 = 20200707;

const LATEST_VERSION: u64 = 20220222;

#[allow(clippy::inconsistent_digit_grouping)]

const LATEST_VERSION: u64 = 2022_02_22;

remoun · 2022-03-01T21:00:05Z

consensus/service/src/config/tokens.rs

+            signers = """
+            -----BEGIN PUBLIC KEY-----
+            MCowBQYDK2VwAyEAyj6m0NRTlw/R28Q+R7vBakwybuaNFneKrvRVAYNp5WQ=
+            -----END PUBLIC KEY-----
+            -----BEGIN PUBLIC KEY-----
+            MCowBQYDK2VwAyEAl3XVo/DeiTjHn8dYQuEtBjQrEWNQSKpfzw3X9dewSVY=
+            -----END PUBLIC KEY-----
+            """


Optional/Equivalent: These keys seem to repeat across test cases, so how about we use consts for them? e.g.

let input = format!(r#" ... signers=""" {} {} """ "#, PEM_ENCODED_KEY_1, PEM_ENCODED_KEY_2, );

My personal opinion is that I think its fairly minimal and easier to read without the string formatting. Part of what I like in having the full TOML/JSON is that it serves as an easy-to-read example of what the file should look like.

* initial mint data structures * basic mint config in ledger * basic get/set test * another test * update total minted api+tests * add migration code * enforce limit * lock and lint * fmt * update comment and remove pub * update lock files * misc pr review fixes * add a test for setting an active configuration with no configs * small fixes * use PEM encoding for signers * fix tag * ensure total minted amount cannot decrease * use inconsistent_digit_grouping to make version numbers more readable * fmt

* Add master minters configuration to TokensConfig (#1504) * minting configuration wip * fmt * fix indent * fix test * add test, be more aggressive about validation * use serde(with=...) * one line * Minting data types and initial LedgerDb support (#1537) * initial mint data structures * basic mint config in ledger * basic get/set test * another test * update total minted api+tests * add migration code * enforce limit * lock and lint * fmt * update comment and remove pub * update lock files * misc pr review fixes * add a test for setting an active configuration with no configs * small fixes * use PEM encoding for signers * fix tag * ensure total minted amount cannot decrease * use inconsistent_digit_grouping to make version numbers more readable * fmt * MintTx/Config protobuf conversions (#1558) * add external.proto minting messages and api conversion traits * tests * mint tx conversions * lint * fix module name * Store mint-related transations in LedgerDb (#1587) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * Expose LedgerDb API for looking up txs by nonce (#1602) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * missing mock changes * fix fmt * Update ledger/db/src/mint_config_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * fix typo * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * Minting transaction validation (#1593) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * missing mock changes * fix fmt * Update ledger/db/src/mint_config_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * fix typo * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * initial work on validation * SetMintConfigTx validation + tests * reorganize * MintTx validation and tests * lint * add comments * update comments * Update transaction/core/src/mint/validation/common.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update transaction/core/src/mint/validation/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * address pr review comment * make nonce constant length Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * SetMintConfigTx -> MintConfigTx (#1607) * SetMintConfigTx -> MintConfigTx * more renames * Small fixes following the confidential-token-id merging (#1609) * post rebase fixes * fix tests * Eran/merge master 2022 03 14 (#1629) * Update grpcio to 0.10 (#1592) * Bump pretty_assertions from 1.1.0 to 1.2.0 (#1610) Bumps [pretty_assertions](https://github.com/colin-kiegel/rust-pretty-assertions) from 1.1.0 to 1.2.0. - [Release notes](https://github.com/colin-kiegel/rust-pretty-assertions/releases) - [Changelog](https://github.com/colin-kiegel/rust-pretty-assertions/blob/main/CHANGELOG.md) - [Commits](https://github.com/colin-kiegel/rust-pretty-assertions/commits) --- updated-dependencies: - dependency-name: pretty_assertions dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * delete slam, fixes #1574 (#1611) slam is no longer needed because fog-distro, which began life as a fork of it, has superceded it and slam is no longer used in CD * introduce FormBlockInputs and break out some code from form_block into its own method (#1625) * introduce FormBlockInputs and break out some code from form_block into its own method * add some missing code * add missing import * remove the Copy trait from ConsensusValue (#1628) * remove the Copy and Display traits from ConsensusValue since they make it hard to add transaction types that cant implement them * add back display * fmt Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Beck <beck.ct@gmail.com> * Support reaching consensus on MintConfigTxs (#1630) * initial work on stubbing out SetMintConfigTx * MintTxManager replaces stubs * grpc api * fmt * mint client and misc improvements * interate towards tx working * actual validation taking place * iterate * append block from client works * wip * rename and lint * fix nonce length * comine, cleanups, validate that nonce is not already in ledger * cleanups * lint * make test code reusable * add tests * structopt -> clap * rebase fixes * fix test * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * address review comments * Update transaction/core/test-utils/src/mint.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * use dedup_by+truncate * renames * undo incorrect change * typo Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Support for reaching consensus on MintTx, mint client improvements (#1641) * stub out ProposeMintTx * expose get_active_mint_config_for_mint_tx, more plumbing * undo wrong change * MintTxManager impls for MintTx * mint client support subcommands * basic MintTx flow works, getting NoOutputs * mobilecoind api filtering for token_id * minting is working :) * client improvements, local network script defaults to having two mintable tokens * tests * more tests yay * testsssssssss * test fixes and linting * try without block version * back to block v3 * Update consensus/service/src/api/client_api_service.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * new line Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update ledger/db/src/mint_tx_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * pr comments * improve comment * use const * Update ledger/db/src/mint_tx_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * lint Co-authored-by: sugargoat <sugargoat@mobilecoin.com> Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Beck <beck.ct@gmail.com>

* Add master minters configuration to TokensConfig (#1504) * minting configuration wip * fmt * fix indent * fix test * add test, be more aggressive about validation * use serde(with=...) * one line * Minting data types and initial LedgerDb support (#1537) * initial mint data structures * basic mint config in ledger * basic get/set test * another test * update total minted api+tests * add migration code * enforce limit * lock and lint * fmt * update comment and remove pub * update lock files * misc pr review fixes * add a test for setting an active configuration with no configs * small fixes * use PEM encoding for signers * fix tag * ensure total minted amount cannot decrease * use inconsistent_digit_grouping to make version numbers more readable * fmt * MintTx/Config protobuf conversions (#1558) * add external.proto minting messages and api conversion traits * tests * mint tx conversions * lint * fix module name * Store mint-related transations in LedgerDb (#1587) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * Expose LedgerDb API for looking up txs by nonce (#1602) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * missing mock changes * fix fmt * Update ledger/db/src/mint_config_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * fix typo * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * Minting transaction validation (#1593) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * missing mock changes * fix fmt * Update ledger/db/src/mint_config_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * fix typo * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * initial work on validation * SetMintConfigTx validation + tests * reorganize * MintTx validation and tests * lint * add comments * update comments * Update transaction/core/src/mint/validation/common.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update transaction/core/src/mint/validation/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * address pr review comment * make nonce constant length Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * SetMintConfigTx -> MintConfigTx (#1607) * SetMintConfigTx -> MintConfigTx * more renames * Small fixes following the confidential-token-id merging (#1609) * post rebase fixes * fix tests * Eran/merge master 2022 03 14 (#1629) * Update grpcio to 0.10 (#1592) * Bump pretty_assertions from 1.1.0 to 1.2.0 (#1610) Bumps [pretty_assertions](https://github.com/colin-kiegel/rust-pretty-assertions) from 1.1.0 to 1.2.0. - [Release notes](https://github.com/colin-kiegel/rust-pretty-assertions/releases) - [Changelog](https://github.com/colin-kiegel/rust-pretty-assertions/blob/main/CHANGELOG.md) - [Commits](https://github.com/colin-kiegel/rust-pretty-assertions/commits) --- updated-dependencies: - dependency-name: pretty_assertions dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * delete slam, fixes #1574 (#1611) slam is no longer needed because fog-distro, which began life as a fork of it, has superceded it and slam is no longer used in CD * introduce FormBlockInputs and break out some code from form_block into its own method (#1625) * introduce FormBlockInputs and break out some code from form_block into its own method * add some missing code * add missing import * remove the Copy trait from ConsensusValue (#1628) * remove the Copy and Display traits from ConsensusValue since they make it hard to add transaction types that cant implement them * add back display * fmt Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Beck <beck.ct@gmail.com> * Support reaching consensus on MintConfigTxs (#1630) * initial work on stubbing out SetMintConfigTx * MintTxManager replaces stubs * grpc api * fmt * mint client and misc improvements * interate towards tx working * actual validation taking place * iterate * append block from client works * wip * rename and lint * fix nonce length * comine, cleanups, validate that nonce is not already in ledger * cleanups * lint * make test code reusable * add tests * structopt -> clap * rebase fixes * fix test * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * address review comments * Update transaction/core/test-utils/src/mint.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * use dedup_by+truncate * renames * undo incorrect change * typo Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Support for reaching consensus on MintTx, mint client improvements (#1641) * stub out ProposeMintTx * expose get_active_mint_config_for_mint_tx, more plumbing * undo wrong change * MintTxManager impls for MintTx * mint client support subcommands * basic MintTx flow works, getting NoOutputs * mobilecoind api filtering for token_id * minting is working :) * client improvements, local network script defaults to having two mintable tokens * tests * more tests yay * testsssssssss * test fixes and linting * try without block version * back to block v3 * Update consensus/service/src/api/client_api_service.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * new line Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update ledger/db/src/mint_tx_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * pr comments * improve comment * use const * Update ledger/db/src/mint_tx_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * lint * add MasterMintersMap * pass MasterMintersMap around * enclave validates MintConfigTxs * fix typo * lint * more lint * fix block index * try less concurrency * Revert "try less concurrency" This reverts commit 0e4209f. Co-authored-by: sugargoat <sugargoat@mobilecoin.com> Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Beck <beck.ct@gmail.com>

* Add master minters configuration to TokensConfig (#1504) * minting configuration wip * fmt * fix indent * fix test * add test, be more aggressive about validation * use serde(with=...) * one line * Minting data types and initial LedgerDb support (#1537) * initial mint data structures * basic mint config in ledger * basic get/set test * another test * update total minted api+tests * add migration code * enforce limit * lock and lint * fmt * update comment and remove pub * update lock files * misc pr review fixes * add a test for setting an active configuration with no configs * small fixes * use PEM encoding for signers * fix tag * ensure total minted amount cannot decrease * use inconsistent_digit_grouping to make version numbers more readable * fmt * MintTx/Config protobuf conversions (#1558) * add external.proto minting messages and api conversion traits * tests * mint tx conversions * lint * fix module name * Store mint-related transations in LedgerDb (#1587) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * Expose LedgerDb API for looking up txs by nonce (#1602) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * missing mock changes * fix fmt * Update ledger/db/src/mint_config_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * fix typo * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * Minting transaction validation (#1593) * wip * remove BlockContents::new to make adding new fields less tedious * fix typos/lints * add todo * fix typos * more defaults * tests * iterate * more tests * remove unneeded public api * more tests * more tests * stricter validation * reorganize code * lint * add mint migration code * mint_tx_store test no. 1 * dont allow overwriting existing blocks, iterate on test * pr fixes * lint * more tests * add test * remove unneeded type declaration * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * missing mock changes * fix fmt * Update ledger/db/src/mint_config_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * fix typo * Update ledger/db/src/mint_config_store.rs Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * rename test to better explain what its testing * be more strict about having outputs when MintTxs are present * add public ledger api for looking up mint txs by nonce, change to return block index since that is more useful * initial work on validation * SetMintConfigTx validation + tests * reorganize * MintTx validation and tests * lint * add comments * update comments * Update transaction/core/src/mint/validation/common.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update transaction/core/src/mint/validation/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * address pr review comment * make nonce constant length Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: sugargoat <sugargoat@mobilecoin.com> * SetMintConfigTx -> MintConfigTx (#1607) * SetMintConfigTx -> MintConfigTx * more renames * Small fixes following the confidential-token-id merging (#1609) * post rebase fixes * fix tests * Eran/merge master 2022 03 14 (#1629) * Update grpcio to 0.10 (#1592) * Bump pretty_assertions from 1.1.0 to 1.2.0 (#1610) Bumps [pretty_assertions](https://github.com/colin-kiegel/rust-pretty-assertions) from 1.1.0 to 1.2.0. - [Release notes](https://github.com/colin-kiegel/rust-pretty-assertions/releases) - [Changelog](https://github.com/colin-kiegel/rust-pretty-assertions/blob/main/CHANGELOG.md) - [Commits](https://github.com/colin-kiegel/rust-pretty-assertions/commits) --- updated-dependencies: - dependency-name: pretty_assertions dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * delete slam, fixes #1574 (#1611) slam is no longer needed because fog-distro, which began life as a fork of it, has superceded it and slam is no longer used in CD * introduce FormBlockInputs and break out some code from form_block into its own method (#1625) * introduce FormBlockInputs and break out some code from form_block into its own method * add some missing code * add missing import * remove the Copy trait from ConsensusValue (#1628) * remove the Copy and Display traits from ConsensusValue since they make it hard to add transaction types that cant implement them * add back display * fmt Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Beck <beck.ct@gmail.com> * Support reaching consensus on MintConfigTxs (#1630) * initial work on stubbing out SetMintConfigTx * MintTxManager replaces stubs * grpc api * fmt * mint client and misc improvements * interate towards tx working * actual validation taking place * iterate * append block from client works * wip * rename and lint * fix nonce length * comine, cleanups, validate that nonce is not already in ledger * cleanups * lint * make test code reusable * add tests * structopt -> clap * rebase fixes * fix test * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * address review comments * Update transaction/core/test-utils/src/mint.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * use dedup_by+truncate * renames * undo incorrect change * typo Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Support for reaching consensus on MintTx, mint client improvements (#1641) * stub out ProposeMintTx * expose get_active_mint_config_for_mint_tx, more plumbing * undo wrong change * MintTxManager impls for MintTx * mint client support subcommands * basic MintTx flow works, getting NoOutputs * mobilecoind api filtering for token_id * minting is working :) * client improvements, local network script defaults to having two mintable tokens * tests * more tests yay * testsssssssss * test fixes and linting * try without block version * back to block v3 * Update consensus/service/src/api/client_api_service.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * new line Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update ledger/db/src/mint_tx_store.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * pr comments * improve comment * use const * support multiple configs and multiple signers * reusable config objects * support multiple signers * offline flow pt 1 * offline flow pt 2 * remove unneeded dep * Update consensus/mint-client/src/bin/main.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * fix incorrect suggestion, DRY nonce generation * improve usage doc * DRY file loading * Trigger CI * Update consensus/mint-client/src/bin/main.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * Update consensus/mint-client/src/config.rs Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> * pr fixes Co-authored-by: sugargoat <sugargoat@mobilecoin.com> Co-authored-by: Remoun Metyas <remoun@mobilecoin.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Chris Beck <beck.ct@gmail.com>

eranrund marked this pull request as ready for review February 24, 2022 00:26

remoun added this to the (2022-Q1) Multiple Token Types milestone Feb 24, 2022

remoun assigned eranrund Feb 24, 2022

eranrund force-pushed the eran/mint-config-types-and-db branch from 8fa4e42 to 58ede1a Compare February 24, 2022 22:40

eranrund force-pushed the feature/minting branch from 9fa33cf to 04baaa4 Compare February 24, 2022 22:41

remoun approved these changes Feb 25, 2022

View reviewed changes

consensus/service/src/config/tokens.rs Show resolved Hide resolved

wjuan-mob self-requested a review February 28, 2022 19:53

remoun reviewed Feb 28, 2022

View reviewed changes

ledger/db/src/lib.rs Outdated Show resolved Hide resolved

ledger/db/src/lib.rs Outdated Show resolved Hide resolved

wjuan-mob approved these changes Feb 28, 2022

View reviewed changes

jcape reviewed Feb 28, 2022

View reviewed changes

remoun suggested changes Feb 28, 2022

View reviewed changes

eranrund force-pushed the feature/minting branch from 04baaa4 to b0a6b06 Compare February 28, 2022 23:27

eranrund added 14 commits February 28, 2022 15:28

initial mint data structures

e787ae9

basic mint config in ledger

ab98b48

basic get/set test

80e1620

another test

213c1b6

update total minted api+tests

7ff1fb1

add migration code

34a2f9d

enforce limit

3fa61d9

lock and lint

fd39c61

fmt

c1ce83a

update comment and remove pub

7d1b1d7

update lock files

0f5631d

misc pr review fixes

fe211a6

add a test for setting an active configuration with no configs

5579e19

small fixes

0535112

eranrund force-pushed the eran/mint-config-types-and-db branch from f4ab016 to 0535112 Compare February 28, 2022 23:28

use PEM encoding for signers

a3cae16

remoun reviewed Mar 1, 2022

View reviewed changes

fix tag

0cd6d2a

eranrund requested a review from remoun March 1, 2022 01:17

remoun approved these changes Mar 1, 2022

View reviewed changes

jcape approved these changes Mar 1, 2022

View reviewed changes

eranrund added 2 commits March 1, 2022 12:23

ensure total minted amount cannot decrease

25f67e3

use inconsistent_digit_grouping to make version numbers more readable

ac22be6

remoun approved these changes Mar 1, 2022

View reviewed changes

fmt

91c3af8

mfaulk approved these changes Mar 1, 2022

View reviewed changes

eranrund merged commit 3d1b79d into feature/minting Mar 2, 2022

eranrund deleted the eran/mint-config-types-and-db branch March 2, 2022 00:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Minting data types and initial LedgerDb support #1537

Minting data types and initial LedgerDb support #1537

eranrund commented Feb 23, 2022

wjuan-mob Feb 28, 2022

eranrund Feb 28, 2022

wjuan-mob Feb 28, 2022

eranrund Feb 28, 2022

wjuan-mob Feb 28, 2022

eranrund Feb 28, 2022

wjuan-mob Feb 28, 2022

wjuan-mob Feb 28, 2022

eranrund Feb 28, 2022

wjuan-mob Feb 28, 2022

eranrund Feb 28, 2022

wjuan-mob Feb 28, 2022

eranrund Feb 28, 2022

jcape Feb 28, 2022 •

edited

Loading

jcape Feb 28, 2022

eranrund Mar 1, 2022

remoun Feb 28, 2022

remoun Feb 28, 2022

eranrund Feb 28, 2022

eranrund Feb 28, 2022

remoun Mar 1, 2022

eranrund Mar 1, 2022

remoun Mar 1, 2022

jcape Mar 1, 2022

eranrund Mar 1, 2022

remoun Mar 1, 2022

eranrund Mar 1, 2022

Minting data types and initial LedgerDb support #1537

Minting data types and initial LedgerDb support #1537

Conversation

eranrund commented Feb 23, 2022

Motivation

In this PR

Future Work

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcape Feb 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jcape Feb 28, 2022 •

edited

Loading