BCF-2317 One relayer per chain id #9788

krehermann · 2023-07-12T19:09:36Z

This change removes chainset from the core Application api. This enables 1:1 relayer:chain, which is a necessary step to encapsulate each chain in a LOOPP.

The Key ideas:

introduce RelayerChainInteroperator which more or less replaces chainset
- This interface provides a clean split between access to loop.Relayer and newly introduced LegacyChains
- The implementation still internal uses chainsets, but each chainset is explicitly scoped to one chain. This is an implementation detail that enables this refactor to use existing code and it can and will be refactored away in the future.
Adds Identity to relayer as (name, id) tuple eg "solana", "solana-chain-abc" which is used consistently in the api

Most of the changes are due to the many, many tentacles of chainset through tests

Note: this change will spawn one relayer per chain id even if the chain id is not used by a job. We may want to implement lazy relayer initialization.

First sketch... Adds plumbing for Solana only to get quick feedback Key ideas: - Identify relayer as (name, id) tuple eg "solana", "solana-chain-abc" - Create one relayer per identifier and plumb a map[relay_id]relayer to delegate - Use the existing job spec to recreate the relayer_id and do the lookup Note: this change will spawn one relayer per chain id even if the chain id is not used by a job. We may want to implement lazy relayer initialization.

github-actions · 2023-07-12T19:10:03Z

I see that you haven't updated any CHANGELOG files. Would it make sense to do so?

…acy compatibility. use relayer factory. begin removing chainset from evm

core/services/chainlink/application.go

…it used to work unless validation was mocked...

… legacy chain

patrickhuie19 · 2023-08-16T17:28:57Z

core/services/ocr2/delegate.go

+	relayID := relay.Identifier{Network: spec.Relay, ChainID: chainID}
+	relayer, err := d.RelayGetter.Get(relayID)
+	if err != nil {
+		return nil, fmt.Errorf("failed to get relay %s is it enabled?: %w", spec.Relay, err)


Unrelated to the scope of this PR, but would this realistically ever happen? Is there job spec validation that should protect against this?

patrickhuie19 · 2023-08-16T17:35:23Z

core/services/chainlink/relayer_chain_interoperators_test.go

+		{name: "all chains",
+			initFuncs: []chainlink.CoreRelayerChainInitFunc{chainlink.InitSolana(testctx, factory, chainlink.SolanaFactoryConfig{
+				Keystore:      keyStore.Solana(),
+				SolanaConfigs: cfg.SolanaConfigs()},
+			),
+				chainlink.InitEVM(testctx, factory, chainlink.EVMFactoryConfig{
+					RelayerConfig: evm.RelayerConfig{
+						GeneralConfig:    cfg,
+						EventBroadcaster: pg.NewNullEventBroadcaster(),
+						MailMon:          &utils.MailboxMonitor{},
+					},
+					CSAETHKeystore: keyStore,
+				}),
+				chainlink.InitStarknet(testctx, factory, chainlink.StarkNetFactoryConfig{
+					Keystore:        keyStore.StarkNet(),
+					StarknetConfigs: cfg.StarknetConfigs()},
+				),
+				chainlink.InitCosmos(testctx, factory, chainlink.CosmosFactoryConfig{
+					Keystore:         keyStore.Cosmos(),
+					CosmosConfigs:    cfg.CosmosConfigs(),
+					EventBroadcaster: pg.NewNullEventBroadcaster(),
+				},
+				),
+			},
+			expectedEVMChainCnt: 2,
+			expectedEVMNodeCnt:  3,
+			expectedEVMRelayerIds: []relay.Identifier{
+				{Network: relay.EVM, ChainID: relay.ChainID(evmChainID1.String())},
+				{Network: relay.EVM, ChainID: relay.ChainID(evmChainID2.String())},
+			},
+
+			expectedSolanaChainCnt: 2,
+			expectedSolanaNodeCnt:  2,
+			expectedSolanaRelayerIds: []relay.Identifier{
+				{Network: relay.Solana, ChainID: relay.ChainID(solanaChainID1)},
+				{Network: relay.Solana, ChainID: relay.ChainID(solanaChainID2)},
+			},
+
+			expectedStarknetChainCnt: 2,
+			expectedStarknetNodeCnt:  4,
+			expectedStarknetRelayerIds: []relay.Identifier{
+				{Network: relay.StarkNet, ChainID: relay.ChainID(starknetChainID1)},
+				{Network: relay.StarkNet, ChainID: relay.ChainID(starknetChainID2)},
+			},
+
+			expectedCosmosChainCnt: 2,
+			expectedCosmosNodeCnt:  2,
+			expectedCosmosRelayerIds: []relay.Identifier{
+				{Network: relay.Cosmos, ChainID: relay.ChainID(cosmosChainID1)},
+				{Network: relay.Cosmos, ChainID: relay.ChainID(cosmosChainID2)},
+			},
+		},
+	}


/nit formatting

Suggested change

{name: "all chains",

initFuncs: []chainlink.CoreRelayerChainInitFunc{chainlink.InitSolana(testctx, factory, chainlink.SolanaFactoryConfig{

Keystore: keyStore.Solana(),

SolanaConfigs: cfg.SolanaConfigs()},

),

chainlink.InitEVM(testctx, factory, chainlink.EVMFactoryConfig{

RelayerConfig: evm.RelayerConfig{

GeneralConfig: cfg,

EventBroadcaster: pg.NewNullEventBroadcaster(),

MailMon: &utils.MailboxMonitor{},

},

CSAETHKeystore: keyStore,

}),

chainlink.InitStarknet(testctx, factory, chainlink.StarkNetFactoryConfig{

Keystore: keyStore.StarkNet(),

StarknetConfigs: cfg.StarknetConfigs()},

),

chainlink.InitCosmos(testctx, factory, chainlink.CosmosFactoryConfig{

Keystore: keyStore.Cosmos(),

CosmosConfigs: cfg.CosmosConfigs(),

EventBroadcaster: pg.NewNullEventBroadcaster(),

},

),

},

expectedEVMChainCnt: 2,

expectedEVMNodeCnt: 3,

expectedEVMRelayerIds: []relay.Identifier{

{Network: relay.EVM, ChainID: relay.ChainID(evmChainID1.String())},

{Network: relay.EVM, ChainID: relay.ChainID(evmChainID2.String())},

},

expectedSolanaChainCnt: 2,

expectedSolanaNodeCnt: 2,

expectedSolanaRelayerIds: []relay.Identifier{

{Network: relay.Solana, ChainID: relay.ChainID(solanaChainID1)},

{Network: relay.Solana, ChainID: relay.ChainID(solanaChainID2)},

},

expectedStarknetChainCnt: 2,

expectedStarknetNodeCnt: 4,

expectedStarknetRelayerIds: []relay.Identifier{

{Network: relay.StarkNet, ChainID: relay.ChainID(starknetChainID1)},

{Network: relay.StarkNet, ChainID: relay.ChainID(starknetChainID2)},

},

expectedCosmosChainCnt: 2,

expectedCosmosNodeCnt: 2,

expectedCosmosRelayerIds: []relay.Identifier{

{Network: relay.Cosmos, ChainID: relay.ChainID(cosmosChainID1)},

{Network: relay.Cosmos, ChainID: relay.ChainID(cosmosChainID2)},

},

},

}

{name: "all chains",

initFuncs: []chainlink.CoreRelayerChainInitFunc{

chainlink.InitSolana(testctx, factory, chainlink.SolanaFactoryConfig{

Keystore: keyStore.Solana(),

SolanaConfigs: cfg.SolanaConfigs(),

}),

chainlink.InitEVM(testctx, factory, chainlink.EVMFactoryConfig{

RelayerConfig: evm.RelayerConfig{

GeneralConfig: cfg,

EventBroadcaster: pg.NewNullEventBroadcaster(),

MailMon: &utils.MailboxMonitor{},

},

CSAETHKeystore: keyStore,

}),

chainlink.InitStarknet(testctx, factory, chainlink.StarkNetFactoryConfig{

Keystore: keyStore.StarkNet(),

StarknetConfigs: cfg.StarknetConfigs(),

}),

chainlink.InitCosmos(testctx, factory, chainlink.CosmosFactoryConfig{

Keystore: keyStore.Cosmos(),

CosmosConfigs: cfg.CosmosConfigs(),

EventBroadcaster: pg.NewNullEventBroadcaster(),

}),

},

expectedEVMChainCount: 2,

expectedEVMNodeCount: 3,

expectedEVMRelayerIds: []relay.Identifier{

{Network: relay.EVM, ChainID: relay.ChainID(evmChainID1.String())},

{Network: relay.EVM, ChainID: relay.ChainID(evmChainID2.String())},

},

expectedSolanaChainCount: 2,

expectedSolanaNodeCount: 2,

expectedSolanaRelayerIds: []relay.Identifier{

{Network: relay.Solana, ChainID: relay.ChainID(solanaChainID1)},

{Network: relay.Solana, ChainID: relay.ChainID(solanaChainID2)},

},

expectedStarknetChainCount: 2,

expectedStarknetNodeCount: 4,

expectedStarknetRelayerIds: []relay.Identifier{

{Network: relay.StarkNet, ChainID: relay.ChainID(starknetChainID1)},

{Network: relay.StarkNet, ChainID: relay.ChainID(starknetChainID2)},

},

expectedCosmosChainCount: 2,

expectedCosmosNodeCount: 2,

expectedCosmosRelayerIds: []relay.Identifier{

{Network: relay.Cosmos, ChainID: relay.ChainID(cosmosChainID1)},

{Network: relay.Cosmos, ChainID: relay.ChainID(cosmosChainID2)},

},

},

patrickhuie19 · 2023-08-16T17:39:33Z

core/chains/chain_set.go

-type Chains interface {
+// ChainStatuser is a generic interface for chain configuration.
+type ChainStatuser interface {
+	// must return [ErrNotFound] if the id is not found
 	ChainStatus(ctx context.Context, id string) (types.ChainStatus, error)
 	ChainStatuses(ctx context.Context, offset, limit int) ([]types.ChainStatus, int, error)


Out of scope for this PR, but documentation here would be helpful to me. How are offset and limit related to Chain Statuses? When concrete types implicitly satisfy this interface, what are some basic rules for how offset and limit can be used?

patrickhuie19 · 2023-08-16T17:45:56Z

core/chains/chain_set.go

Is the eventual goal to migrate all the functionality in chain_set to chain_kv? Worth adding a README.md here (i.e. core/chains/README.md) and quickly documenting the different use cases?

patrickhuie19

Legend 🚀

Had a few comments around documentation that I would find helpful, but certainly not blocking.

cl-sonarqube-production · 2023-08-16T21:11:47Z

SonarQube Quality Gate

77.8% Coverage on New Code (is less than 80%)

See analysis details on SonarQube

krehermann added 4 commits July 14, 2023 15:15

hacking

0e6748a

hacking. track legacy chains and loop relayers. use type cast for leg…

c749813

…acy compatibility. use relayer factory. begin removing chainset from evm

progress on evm chainset conversion

f8d9f0e

get build working.

311f118

jmank88 reviewed Jul 19, 2023

View reviewed changes

core/services/chainlink/application.go Show resolved Hide resolved

krehermann added 12 commits July 20, 2023 09:10

remove evm chain from public interface

4bd3df6

complete evm chainset api removal and get first test to work

628be97

part way thru test refactor

abbd7b3

fix all tests but mocking GetRelayers

98a6263

get last test to compile

a0a3726

Merge branch 'develop' into BCF-2317_relayer_chain_id

e238690

go mod tidy

3ae25f0

return non-nil legacy evm chains for compatibility

e5f92c2

fix shell tests

b7dc4db

on the way to mocks for testing

31b333e

fix legacy chain Get to be backward compatible. fix nil maps

c9bf000

Merge branch 'develop' into BCF-2317_relayer_chain_id

c431d7a

krehermann force-pushed the BCF-2317_relayer_chain_id branch from 373924e to c431d7a Compare July 24, 2023 20:51

krehermann added 10 commits July 24, 2023 15:36

fix signature. fix relayer opts init

2820050

fix bad merge

bef797a

more test fixes

7b01c97

ensure default chain from relayer extenders

44acf5a

Merge branch 'develop' into BCF-2317_relayer_chain_id

95c2b38

fix legacychain.Get() empty string handling to be backward compatible.

e08d33d

fmt.errorf fixes, nil map fix in cosmos, test fixes

d8faa8b

fix test validation errors in job runner. spec was bad; not sure why …

64a4347

…it used to work unless validation was mocked...

fix retryable eth tx failure and err fmting

a31b9a5

err message fix in test and default chain handling in cltest for mock…

7e144d6

… legacy chain

patrickhuie19 reviewed Aug 16, 2023

View reviewed changes

patrickhuie19 previously approved these changes Aug 16, 2023

View reviewed changes

krehermann dismissed stale reviews from patrickhuie19, samsondav, and jmank88 via 5683a8f August 16, 2023 20:26

krehermann force-pushed the BCF-2317_relayer_chain_id branch from 5683a8f to 1a83c4b Compare August 16, 2023 20:32

patrickhuie19 previously approved these changes Aug 16, 2023

View reviewed changes

Merge branch 'develop' into BCF-2317_relayer_chain_id

a464e57

krehermann dismissed patrickhuie19’s stale review via a464e57 August 16, 2023 20:53

krehermann force-pushed the BCF-2317_relayer_chain_id branch from 1a83c4b to a464e57 Compare August 16, 2023 20:53

patrickhuie19 approved these changes Aug 16, 2023

View reviewed changes

krehermann enabled auto-merge August 16, 2023 20:59

krehermann added this pull request to the merge queue Aug 16, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 16, 2023

krehermann added this pull request to the merge queue Aug 16, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 16, 2023

krehermann added this pull request to the merge queue Aug 16, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 17, 2023

krehermann added this pull request to the merge queue Aug 17, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 17, 2023

krehermann added this pull request to the merge queue Aug 17, 2023

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Aug 17, 2023

jmank88 added this pull request to the merge queue Aug 17, 2023

Merged via the queue into develop with commit 77f0074 Aug 17, 2023
92 of 93 checks passed

jmank88 deleted the BCF-2317_relayer_chain_id branch August 17, 2023 12:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BCF-2317 One relayer per chain id #9788

BCF-2317 One relayer per chain id #9788

krehermann commented Jul 12, 2023 •

edited

Loading

github-actions bot commented Jul 12, 2023

patrickhuie19 Aug 16, 2023

patrickhuie19 Aug 16, 2023

patrickhuie19 Aug 16, 2023

patrickhuie19 Aug 16, 2023

patrickhuie19 left a comment

cl-sonarqube-production bot commented Aug 16, 2023

BCF-2317 One relayer per chain id #9788

BCF-2317 One relayer per chain id #9788

Conversation

krehermann commented Jul 12, 2023 • edited Loading

github-actions bot commented Jul 12, 2023

patrickhuie19 Aug 16, 2023

Choose a reason for hiding this comment

patrickhuie19 Aug 16, 2023

Choose a reason for hiding this comment

patrickhuie19 Aug 16, 2023

Choose a reason for hiding this comment

patrickhuie19 Aug 16, 2023

Choose a reason for hiding this comment

patrickhuie19 left a comment

Choose a reason for hiding this comment

cl-sonarqube-production bot commented Aug 16, 2023

krehermann commented Jul 12, 2023 •

edited

Loading