L2 Block Source & Metadata Storage implementations #1983

MitchTurner · 2024-06-20T20:35:52Z

Closes: #1955

This actually implements the port adapters from the updater service to access our live databases while running.

Will follow up with another PR that replaces the current updater with the new one.

Checklist

Breaking changes are clearly marked as such in the PR description and changelog
New behavior is reflected in tests
The specification matches the implemented behavior (link update PR if changes are needed)

Before requesting review

I have reviewed the code myself
I have created follow-up issues caused by this PR and linked them here

After merging, notify other teams

[Add or remove entries as needed]

Rust SDK
Sway compiler
Platform documentation (for out-of-organization contributors, the person merging the PR will do this)
Someone else?

…ce for previous block

xgreenx · 2024-07-02T13:13:48Z

crates/fuel-core/src/database/database_description/gas_price.rs

+    }
+
+    fn name() -> &'static str {
+        "gas_price_service_storage"


Suggested change

"gas_price_service_storage"

"gas_price"

xgreenx · 2024-07-02T13:16:43Z

crates/fuel-gas-price-algorithm/gas-price-analysis/Cargo.lock

I think we can remove lock file for gas-price-analysis binary. It is not required binary to build

xgreenx · 2024-07-02T13:20:35Z

crates/fuel-gas-price-algorithm/src/v1_no_da.rs

+    pub fn update_l2_block_data(
+        &mut self,
+        height: u32,
+        fullness: (u64, u64),


Suggested change

fullness: (u64, u64),

used: u64, capacity: u64

[nit]s: Looks more readable=)

xgreenx · 2024-07-02T13:21:34Z

crates/fuel-gas-price-algorithm/src/v1_no_da.rs

+
+    fn update_exec_gas_price(&mut self, used: u64, capacity: u64) {
+        let mut exec_gas_price = self.new_exec_price;
+        // TODO: Do we want to capture this error? I feel like we should assume capacity isn't 0


capacity type could be NonZeroU64 to avoid this error=)

xgreenx · 2024-07-02T13:23:40Z

crates/fuel-gas-price-algorithm/src/v1_no_da.rs

+mod tests;
+
+#[derive(Debug, thiserror::Error, PartialEq)]
+pub enum Error {


Where do we use it?

xgreenx · 2024-07-02T13:49:29Z

crates/services/gas_price_service/src/fuel_gas_price_updater.rs

        let inner = metadata_storage
-            .get_metadata()
+            .get_metadata(&target_block_height)
            .await?


The same here, I think we don't need to require usage of async it can be sync.

xgreenx · 2024-07-02T13:50:40Z

crates/services/gas_price_service/src/fuel_gas_price_updater/tests.rs

@@ -13,7 +13,7 @@ struct FakeL2BlockSource {

 #[async_trait::async_trait]
 impl L2BlockSource for FakeL2BlockSource {
-    async fn get_l2_block(&self, _height: BlockHeight) -> Result<BlockInfo> {
+    async fn get_l2_block(&mut self, _height: BlockHeight) -> Result<BlockInfo> {
        let block = self.l2_block.lock().await.recv().await.unwrap();


Do we need to use lock now if it is &mut self?

The lock wasn't becasue this wasn't mut, it's because we need to access the value in another thread in the test.

xgreenx · 2024-07-02T13:51:23Z

crates/services/gas_price_service/src/fuel_gas_price_updater/tests.rs

        Ok(self.inner.lock().await.clone())
    }

-    async fn set_metadata(&self, metadata: UpdaterMetadata) -> Result<()> {
+    async fn set_metadata(&mut self, metadata: UpdaterMetadata) -> Result<()> {
        let _ = self.inner.lock().await.replace(metadata);


The same here, do we need to use lock, if you use &mut self?

Same for here.

xgreenx · 2024-07-02T13:52:26Z

...es/gas_price_service/src/fuel_gas_price_updater/fuel_core_storage_adapter/l2_source_tests.rs

+    let settings = FakeSettings::new(gas_price_factor, block_gas_limit);
+
+    let import_result = block_to_import_result(block.clone());
+    let blocks: Vec<Arc<dyn Deref<Target = ImportResult> + Send + Sync>> =


Suggested change

let blocks: Vec<Arc<dyn Deref<Target = ImportResult> + Send + Sync>> =

let blocks: Vec<SharedImportResult> =

xgreenx · 2024-07-02T13:52:50Z

...es/gas_price_service/src/fuel_gas_price_updater/fuel_core_storage_adapter/l2_source_tests.rs

+    ConsensusParameters::V1(params)
+}
+
+fn block_to_import_result(block: Block) -> Arc<ImportResult> {


Suggested change

fn block_to_import_result(block: Block) -> Arc<ImportResult> {

fn block_to_import_result(block: Block) -> SharedImportResult {

…ve-adapter

Part of #1624 The [previous PR](#1983) added all the code for working with the dynamic gas price, but didn't plug it into our services. This PR does the actual dep injection. It also adds gas price estimation to AlgoirthmV0. This change includes new flags for the CLI: - "starting-gas-price" - the starting gas price for the gas price algorithm - "gas-price-change-percent" - the percent change for each gas price update - "gas-price-threshold-percent" - the threshold percent for determining if the gas price will be increase or decreased And the following CLI flags are serving a new purpose - "min-gas-price" - the minimum gas price that the gas price algorithm will return ## Checklist - [ ] Breaking changes are clearly marked as such in the PR description and changelog - [ ] New behavior is reflected in tests - [ ] [The specification](https://github.com/FuelLabs/fuel-specs/) matches the implemented behavior (link update PR if changes are needed) ### Before requesting review - [x] I have reviewed the code myself - [ ] I have created follow-up issues caused by this PR and linked them here ### After merging, notify other teams [Add or remove entries as needed] - [ ] [Rust SDK](https://github.com/FuelLabs/fuels-rs/) - [ ] [Sway compiler](https://github.com/FuelLabs/sway/) - [ ] [Platform documentation](https://github.com/FuelLabs/devrel-requests/issues/new?assignees=&labels=new+request&projects=&template=NEW-REQUEST.yml&title=%5BRequest%5D%3A+) (for out-of-organization contributors, the person merging the PR will do this) - [ ] Someone else? --------- Co-authored-by: human <jamesturner@Zenobia.hsd1.wa.comcast.net> Co-authored-by: Green Baneling <XgreenX9999@gmail.com> Co-authored-by: Hannes Karppila <2204863+Dentosal@users.noreply.github.com>

@MitchTurner

## What's Changed * L2 Block Source & Metadata Storage implementations by @MitchTurner in #1983 * Weekly `cargo update` by @github-actions in #2029 * Add V0 algorithm to actual services by @MitchTurner in #2025 * Weekly `cargo update` by @github-actions in #2039 * Add syncronization for Gas Price Database and On Chain Database on startup by @MitchTurner in #2041 * Ignore the message receipt if transaction is reverted by @xgreenx in #2045 * feat: add chain config to Docker images by @mchristopher in #2052 * Blob tx support and fuel-vm 0.56.0 by @Dentosal in #1988 * Disable SMT for `ContractsAssets` and `ContractsState` by @xgreenx in #2048 ## New Contributors * @mchristopher made their first contribution in #2052 **Full Changelog**: v0.31.0...v0.32.0

Part of FuelLabs/fuel-core#1624 The [previous PR](FuelLabs/fuel-core#1983) added all the code for working with the dynamic gas price, but didn't plug it into our services. This PR does the actual dep injection. It also adds gas price estimation to AlgoirthmV0. This change includes new flags for the CLI: - "starting-gas-price" - the starting gas price for the gas price algorithm - "gas-price-change-percent" - the percent change for each gas price update - "gas-price-threshold-percent" - the threshold percent for determining if the gas price will be increase or decreased And the following CLI flags are serving a new purpose - "min-gas-price" - the minimum gas price that the gas price algorithm will return ## Checklist - [ ] Breaking changes are clearly marked as such in the PR description and changelog - [ ] New behavior is reflected in tests - [ ] [The specification](https://github.com/FuelLabs/fuel-specs/) matches the implemented behavior (link update PR if changes are needed) ### Before requesting review - [x] I have reviewed the code myself - [ ] I have created follow-up issues caused by this PR and linked them here ### After merging, notify other teams [Add or remove entries as needed] - [ ] [Rust SDK](https://github.com/FuelLabs/fuels-rs/) - [ ] [Sway compiler](https://github.com/FuelLabs/sway/) - [ ] [Platform documentation](https://github.com/FuelLabs/devrel-requests/issues/new?assignees=&labels=new+request&projects=&template=NEW-REQUEST.yml&title=%5BRequest%5D%3A+) (for out-of-organization contributors, the person merging the PR will do this) - [ ] Someone else? --------- Co-authored-by: human <jamesturner@Zenobia.hsd1.wa.comcast.net> Co-authored-by: Green Baneling <XgreenX9999@gmail.com> Co-authored-by: Hannes Karppila <2204863+Dentosal@users.noreply.github.com>

@MitchTurner

## What's Changed * L2 Block Source & Metadata Storage implementations by @MitchTurner in FuelLabs/fuel-core#1983 * Weekly `cargo update` by @github-actions in FuelLabs/fuel-core#2029 * Add V0 algorithm to actual services by @MitchTurner in FuelLabs/fuel-core#2025 * Weekly `cargo update` by @github-actions in FuelLabs/fuel-core#2039 * Add syncronization for Gas Price Database and On Chain Database on startup by @MitchTurner in FuelLabs/fuel-core#2041 * Ignore the message receipt if transaction is reverted by @xgreenx in FuelLabs/fuel-core#2045 * feat: add chain config to Docker images by @mchristopher in FuelLabs/fuel-core#2052 * Blob tx support and fuel-vm 0.56.0 by @Dentosal in FuelLabs/fuel-core#1988 * Disable SMT for `ContractsAssets` and `ContractsState` by @xgreenx in FuelLabs/fuel-core#2048 ## New Contributors * @mchristopher made their first contribution in FuelLabs/fuel-core#2052 **Full Changelog**: FuelLabs/fuel-core@v0.31.0...v0.32.0

MitchTurner and others added 30 commits June 7, 2024 09:22

Add the algorithm code

ac1aa41

WIP

bc9efa0

Add some tests to define the updater behavior

31ea5c4

Add a couple more tests, WIP getting the second to pass

7e6546a

Fix test, refactor out single update into two

6707c3e

Refactor

09a0c95

WIP rewrite the chart generation stuff

da3c887

Add profit stuff to chart, add rewards

cdfc206

Add some naive profit corrections

15a45ed

Remove the old stuff that is not being used

03c6a4a

Add actual algo that the updater generates, fix tests accordingly

459f8fd

Update gas price with l2 block data

b713739

Use algo generation in simulation

91fe558

Straighten out the profit chart to be aligned after every da_recording

5c3c277

Fix the noise

da7ea98

Refactor to calculate exec gas price eagarly and calculate da gas pri…

f05c594

…ce for previous block

Refactor a bit to allow p and d values, rename some stuff

644c2da

Add d value to tests

9c40f42

WIP

0783439

Use real gas limit values stablize algo a bit.

d7d7d55

Set params to somewhat steady state

106569d

Add average profit values

6db37a6

Cleanup, add naive optimisation

781424f

Refactor

88793e9

Replace d value :), start adding max change

b7c10c6

Add change amount tests, remove unused params

e245f98

Remove avg window

95ed269

Refactor and cleanup

95ab959

Merge remote-tracking branch 'origin' into feature/gas-price-algo

53eebfe

fix fields

5d8d7bb

xgreenx reviewed Jul 2, 2024

View reviewed changes

MitchTurner added 3 commits July 2, 2024 11:44

Make PR review changes

1cd7da2

Make PR review changes

4249fb5

Make metadata methods sync

f117695

MitchTurner requested a review from xgreenx July 2, 2024 21:37

MitchTurner and others added 15 commits July 2, 2024 14:52

Fix test to save metadata before trying to get l2 block

b280ad5

Merge remote-tracking branch 'origin' into feature/l2-block-source-li…

737aa78

…ve-adapter

Appease Clippy-sama

d6dde40

Fix v0 algo file name

149f666

Fmt

ccd2246

Remove unused lock

31b7fcf

Remove unused async

e9c49e3

Merge remote-tracking branch 'origin' into feature/l2-block-source-li…

08a12cb

…ve-adapter

Update WASM

d2adec3

Merge remote-tracking branch 'origin' into feature/l2-block-source-li…

267ddde

…ve-adapter

Update code to work with storage trait changes, build wasm bytecode

6ad67d4

Merge remote-tracking branch 'origin' into feature/l2-block-source-li…

d6cc3e7

…ve-adapter

Update WASM

bc84ece

Move changelog entry to unreleased

acfb2f3

Swap order of updating l2 block with saving metadata

c76ad67

xgreenx approved these changes Jul 8, 2024

View reviewed changes

MitchTurner merged commit dded7cc into master Jul 8, 2024
33 checks passed

MitchTurner deleted the feature/l2-block-source-live-adapter branch July 8, 2024 17:10

MitchTurner mentioned this pull request Jul 9, 2024

Add V0 algorithm to actual services #2025

Merged

9 tasks

MitchTurner mentioned this pull request Aug 2, 2024

Release v0.32.0 #2053

Merged

xgreenx mentioned this pull request Aug 29, 2024

Implement algorithm to predict gas price for estimate_gas_price endpoint #1653

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

L2 Block Source & Metadata Storage implementations #1983

L2 Block Source & Metadata Storage implementations #1983

MitchTurner commented Jun 20, 2024 •

edited

Loading

xgreenx Jul 2, 2024

xgreenx Jul 2, 2024

xgreenx Jul 2, 2024

xgreenx Jul 2, 2024

xgreenx Jul 2, 2024

xgreenx Jul 2, 2024

xgreenx Jul 2, 2024

MitchTurner Jul 2, 2024

xgreenx Jul 2, 2024

MitchTurner Jul 2, 2024

xgreenx Jul 2, 2024

xgreenx Jul 2, 2024

	let blocks: Vec<Arc<dyn Deref<Target = ImportResult> + Send + Sync>> =
	let blocks: Vec<SharedImportResult> =

	fn block_to_import_result(block: Block) -> Arc<ImportResult> {
	fn block_to_import_result(block: Block) -> SharedImportResult {

L2 Block Source & Metadata Storage implementations #1983

L2 Block Source & Metadata Storage implementations #1983

Conversation

MitchTurner commented Jun 20, 2024 • edited Loading

Checklist

Before requesting review

After merging, notify other teams

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MitchTurner commented Jun 20, 2024 •

edited

Loading