556 add prices table to indexeddb #764

Valentine1898 · 2024-03-14T14:55:49Z

partially close #556

Valentine1898 · 2024-03-18T12:24:08Z

packages/constants/src/assets.ts

+export const NUMERAIRE_TOKEN = 'test_usd';
+export const NUMERAIRE_TOKEN_ID = localAssets.find(
+  metadata => metadata.display === NUMERAIRE_TOKEN,
+)!.penumbraAssetId!;
+


In the future we can easily change the NUMERAIRE_TOKEN if necessary

See apps/extension/.env. We should try to use env variables for this given mainnet/testnet will have a different numeraire. Looks like we started this already with USDC_ASSET_ID. We should update or replace that.

Valentine1898 · 2024-03-18T12:25:53Z

packages/query/src/block-processor.ts

+      let numerairePerUnit = 0;
+      let pricedAsset: AssetId | undefined = undefined;
+
+      // case for trading pair <pricedAsset,numéraire>


We cannot know in advance the order of assets in a trading pair, so we must handle both cases

Valentine1898 · 2024-03-18T12:27:28Z

packages/query/src/block-processor.ts

+const calculatePrice = (delta: Amount, unfilled: Amount, lambda: Amount): number => {
+  const filledAmount = subtractAmounts(delta, unfilled);
+  return isZero(delta) || isZero(lambda) || isZero(filledAmount)
+    ? 0
+    : divideAmounts(lambda, filledAmount).toNumber();
+};


https://discord.com/channels/824484045370818580/938328380917559326/1217859679057870858 <- price calculation discussions here

Valentine1898 · 2024-03-18T12:29:35Z

packages/types/src/indexed-db.ts

@@ -171,6 +178,10 @@ export interface PenumbraDb extends DBSchema {
    key: string; // bech32-encoded validator identity key
    value: Jsonified<ValidatorInfo>;
  };
+  PRICES: {
+    key: string[]; // composite key [base64 EstimatedPrice['priced_asset'][inner'], base64 EstimatedPrice['numeraire']['inner']]


Using the composite key ensures that we will not have two records for the combination priced_asset:numeraire and will always store only the most recent value

You can also do:

key: [ Jsonified<Required<EstimatedPrice>['pricedAsset']['inner']>, Jsonified<Required<EstimatedPrice>['numeraire']['inner']>, ];

grod220 · 2024-03-18T14:18:19Z

packages/constants/src/assets.ts

+export const NUMERAIRE_TOKEN = 'test_usd';
+export const NUMERAIRE_TOKEN_ID = localAssets.find(
+  metadata => metadata.display === NUMERAIRE_TOKEN,
+)!.penumbraAssetId!;
+


See apps/extension/.env. We should try to use env variables for this given mainnet/testnet will have a different numeraire. Looks like we started this already with USDC_ASSET_ID. We should update or replace that.

grod220 · 2024-03-18T14:18:53Z

packages/query/src/block-processor.ts

+      if (compactBlock.swapOutputs.length)
+        await this.updatePrices(compactBlock.swapOutputs, compactBlock.height);


Let's add brackets here

grod220 · 2024-03-18T14:37:34Z

packages/query/src/block-processor.ts

@@ -388,4 +409,46 @@ export class BlockProcessor implements BlockProcessorInterface {
      await this.indexedDb.upsertValidatorInfo(validatorInfoResponse.validatorInfo);
    }
  }
+
+  private async updatePrices(swapOutputs: BatchSwapOutputData[], height: bigint) {


Can you add documentation (here or calculatePrice()) on some of these terms like delta/lambda/unfilled means?

grod220 · 2024-03-18T14:41:42Z

packages/query/src/block-processor.ts

+
+      await this.indexedDb.updatePrice(pricedAsset, NUMERAIRE_TOKEN_ID, numerairePerUnit, height);
+    }
+  }


I think it's worth adding some tests for this. Some ideas:

We move updatePrices into its own file as a separate function (not on the class). This would require passing in the deps (like storage) into the function. When separated, we can write tests.

If not, just start creating a test suite block-processor.test.ts passing in deps on the class constructor.

grod220 · 2024-03-18T14:44:51Z

packages/storage/src/indexed-db/index.ts

+    });
+
+    await this.u.update({
+      table: 'PRICES',


This is missing the key right? or is it automatic?

This works automatically if the key is specified via keypath

grod220 · 2024-03-18T14:49:56Z

packages/types/src/indexed-db.ts

@@ -171,6 +178,10 @@ export interface PenumbraDb extends DBSchema {
    key: string; // bech32-encoded validator identity key
    value: Jsonified<ValidatorInfo>;
  };
+  PRICES: {
+    key: string[]; // composite key [base64 EstimatedPrice['priced_asset'][inner'], base64 EstimatedPrice['numeraire']['inner']]


You can also do:

key: [ Jsonified<Required<EstimatedPrice>['pricedAsset']['inner']>, Jsonified<Required<EstimatedPrice>['numeraire']['inner']>, ];

grod220

Nice job with those tests. Think this is fine for now, but we should have a follow up PR enabling multiple numeraires.

grod220 · 2024-03-19T08:39:21Z

packages/services/src/index.ts

@@ -110,6 +115,7 @@ export class Services implements ServicesInterface {
      viewServer,
      querier: this.querier,
      indexedDb,
+      numeraireAssetId: numeraireAssetId,


Think you can do:

- numeraireAssetId: numeraireAssetId, + numeraireAssetId,

Valentine1898 force-pushed the 556-add-prices-table-to-indexeddb branch from 612b0e0 to 7e2ddeb Compare March 15, 2024 18:38

Valentine1898 commented Mar 18, 2024

View reviewed changes

Valentine1898 force-pushed the 556-add-prices-table-to-indexeddb branch from 8b85b79 to 7a10bda Compare March 18, 2024 12:30

Valentine1898 marked this pull request as ready for review March 18, 2024 12:30

Valentine1898 changed the title ~~WIP: 556 add prices table to indexeddb~~ 556 add prices table to indexeddb Mar 18, 2024

Valentine1898 requested review from hdevalence and grod220 March 18, 2024 12:34

grod220 reviewed Mar 18, 2024

View reviewed changes

Valentine1898 force-pushed the 556-add-prices-table-to-indexeddb branch from 7a10bda to a7a91cd Compare March 18, 2024 18:52

Valentine1898 added 8 commits March 19, 2024 02:45

PRICES table added

b99a4c5

wip

ac9dd80

save prices

39a702d

impl numerairePerUnit calculation

64fe1d8

add amount tests

6a7f461

review fixes

dd069f9

add test for price-indexer

82e357c

use .env for numeraireAssetId

6d6ebac

Valentine1898 force-pushed the 556-add-prices-table-to-indexeddb branch from c8a4dae to 6d6ebac Compare March 19, 2024 00:45

Valentine1898 requested a review from grod220 March 19, 2024 00:52

grod220 approved these changes Mar 19, 2024

View reviewed changes

review fix

3f9afef

Valentine1898 self-assigned this Mar 19, 2024

Valentine1898 merged commit 958aaea into main Mar 19, 2024
6 checks passed

Valentine1898 deleted the 556-add-prices-table-to-indexeddb branch March 19, 2024 09:11

Valentine1898 mentioned this pull request Mar 19, 2024

Improve pricing method #794

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

556 add prices table to indexeddb #764

556 add prices table to indexeddb #764

Valentine1898 commented Mar 14, 2024 •

edited

Loading

Valentine1898 Mar 18, 2024

grod220 Mar 18, 2024

Valentine1898 Mar 18, 2024

Valentine1898 Mar 18, 2024

Valentine1898 Mar 18, 2024

grod220 Mar 18, 2024

grod220 Mar 18, 2024

grod220 Mar 18, 2024

grod220 Mar 18, 2024

grod220 Mar 18, 2024

grod220 Mar 18, 2024

Valentine1898 Mar 19, 2024

grod220 Mar 18, 2024

grod220 left a comment

grod220 Mar 19, 2024

		if (compactBlock.swapOutputs.length)
		await this.updatePrices(compactBlock.swapOutputs, compactBlock.height);

556 add prices table to indexeddb #764

556 add prices table to indexeddb #764

Conversation

Valentine1898 commented Mar 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

grod220 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Valentine1898 commented Mar 14, 2024 •

edited

Loading