Use in process Relayer for race detection #996
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
agouin
requested review from
jackzampolin,
jtieri,
boojamya and
mark-rushakoff
as code owners
relayer/chains/cosmos/provider.go
Outdated
| @@ -245,5 +247,7 @@ func (cc *CosmosProvider) BlockTime(ctx context.Context, height int64) (time.Tim | |||
| } | |||
|
|
|||
| func (cc *CosmosProvider) SetMetrics(m *processor.PrometheusMetrics) { | |||
There was a problem hiding this comment.
This is one of those kinds of fields that should never change after the struct is initialized.
It would be better if the PrometheusMetrics value were part of the NewCosmosProvider function signature, then we can do away with the SetMetrics method altogether.
There was a problem hiding this comment.
Since NewProvider is an interface method that is called initially in initConfig before we ever construct the prometheus metrics, I kept the SetMetrics method but made sure it is only called once during StartRelayer instead of during construction of the CosmosChainProcessors, which allows the mutex to be isolated to the TotalFees as you suggested.
agouin
force-pushed
the
andrew/local_relayer
branch
from
82e7d34
to
e28e3b2
Compare
agouin
force-pushed
the
andrew/local_relayer
branch
from
e28e3b2
to
9c94595
Compare
Comment on lines
+238
to
+240
| if r.cancel == nil { | ||
| return nil | ||
| } |
There was a problem hiding this comment.
Why guard against a nil cancel here?
The doc comment on the cancel field says it is set in StartRelayer. It doesn't seem valid to call StopRelayer without first calling StartRelayer.
There was a problem hiding this comment.
ibctest makes assumptions that StopRelayer is idempotent, e.g.:
https://github.com/strangelove-ventures/ibctest/blob/7653dfa748149c0bc92277e752b35bd4c2f7b94e/test_setup.go#L101
There was a problem hiding this comment.
Okay. I didn't expect that.
mark-rushakoff
approved these changes
Sep 20, 2022
jtieri
added a commit
that referenced
this pull request
Oct 11, 2022
* Merge PR #919: Clean up and move scripts folder and dev-env * clean up scripts * clean up Makerfile * cleanup configs * fix shebang * update data folder path * move to examples dir * update build-gaia path for make * add-dir description * examples * examples readme * add killall note to readme * fix link * remove extra config dir * update path in demo.md * readme nit * Use self-hosted runner for ibctest integration tests (#943) * bugfix - intermittent channel handshake failure (#937) * bugfix for intermittent channel handshake failure * Add unit tests for connection and channel state caches * Log messages in failed txs * Tidy * Adding MinGasAmount (#940) * Adding MinGasAmount * go.mod updates * Updating ibctest/go.mod * Adding min-gas-amount to CHANGELOG and troubleshooting * Merge PR #953: cosmos - begin and end block event parsing * Add begin and end block events * tidy * Handle multiple ibc messages in begin block and end block * Fix logic with accumulator * tidy * Fix nil pointer for begin/end block events (#956) * Fix bug with tracking processing channel msgs (#970) * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * undo previous changes regarding purging cache on max retries * undo more changes made in shouldSendChannelMessage * more undo * add debug output on msg send and channel cache purge * add debug output * remove pointless log * add logs in process latest msgs * more logs * logs * remove all debug logs and implement fix for tracking processing packets correctly * Merge PR #974: Clarify demo * update demo * remove extra dev-env * Merge PR #963: Additional Metrics * Adding block height and wallet balance metrics * Fixing duplicate registration * Assigning metrics during NewCosmosChainProcessor * Fixing latest height to use ChainId * Metrics are optional... * Merge PR #981: Update IBCdata struct Update IBCdata struct based on recent changes made at the Cosmos Chain Registry * Fix flattening of events (#979) * Fix flattening of events * Parse events without flattened tx.Log * Add missing write ack to switch case * Fix unnecessary retention of irrelevant messages (#980) * Merge PR #987: Use config.lock file to guard access to config.yaml in the case of linking multiple paths concurrently * ignore recv packet with empty ack bytes (#985) Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> * Relay multiple paths from the same process (#988) * wip: add test for relaying on many paths This test instantiates a relayer with the event processor that relays on two paths across three chains. This was ported from previous WIP that attempted to add multi-path support to the legacy processor for the relayer; multi-path support in the legacy processor had some subtle bugs that caused the test to fail. Now with the new event processor, the test is failing in a different way that we need to further debug. * Multiple paths same process with test * go mod tidy * fix old tests * Update to ibctest with race fix * Fix legacy relayer Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> * Default to events processor (#989) * Default to events processor * events processor is now default * sort packet messages by sequence number (#992) only process 1 packet at a time on ordered channels * path processor race (#995) * fix path processor indexing * clone maps before passing to pathprocessor * Merge should Clone * pre-size maps * Use in process Relayer for race detection (#996) * clone maps before passing to pathprocessor * Add local relayer implementation to help catch race conditions in CI * use existing relayerfactory * Isolate mutex to totalFees * handle feedback * fix fatalf format * Merge PR #999: Add ability to run all paths in config easil * Add ability to run all paths in config easily * PR review fixes * Fix tests and patch readme * Merge PR #1005: Fix legacy race condition * fix counterparty path filter (#1000) * fix counterparty path filter * Filter fix test * Add denylist test and add makefile and gh action * Slim test for non-self-hosted runner * Update ibctest to latest main * Make better assertion for denylist acks. Constants for allowlist/denylist. Validate filterRule in CLI * Use isolated prometheus registry per relayer instance instead of prometheus default registry * run path filter tests in parallel * dockerfile: move TARGET* args for fast workflow & ... (#993) - Single COPY step for go mod and sum - Use ENV GOOS & GOARCH from TARGET* args Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> * Respect the `override` flag upon client creation (#997) * respect the override flag * add test case for --override usage * add missing godoc comment for test * disable CountTotal (#1009) * Fix account sequence mismatch errors (#1007) * Use mutex for tx account sequence query through tx broadcast, track account sequence number on chain provider * Make updateNextAccountSequence private * Retry for account sequence mismatch errors * Allow decrementing account sequence if node says it is expected * Add test for handleAccountSequenceMismatchError * Keystore implementation for substrate chain (#922) * add substrate keystore implementation * implement key provider for substrate chain * clean up keystore code * run go mod tidy moving rename test to _test * remove verbs from error messages * correct interface method comments in Keyring and Info types * initialize log in NewProvider method * move type definitions to method implementation files * check if KeyDirectory is empty before overwriting it in the NewProvider method * remove comments in ExportPrivKeyArmor * bump substrate-rpc-client to latest master * change network type to uint16 * update go.mod in ibctest * feat: add chain processor implementation (#924) * feat: add chain processor implementation * fix: go-schnorrkel version fix * fix: go mod update dockertest * feature/ implement substrate event parser (#923) * feat: implement substrate event parser * fix: event parser switch fallthrough * fix: go-schnorrkel version fix * fix: go mod update for dcokertest * create helper functions between substrate <> ibc events * use intoIBCEventType * fix: event parser accumulation support * fix: spearate parsing and handling of ibc messages * feat: parachain header parse * fix: packet accumulator separate switch case * fix: go.mod docker issue * fix: ics11-beefy repo version fix * fix: accumulator events to messages * fix: add connection event type conversion * fix: ibctest go.mod fix Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com> * fix: go.mod Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com> Co-authored-by: Andrew Gouin <andrew@gouin.io> Co-authored-by: Joe Abbey <joe.abbey@gmail.com> Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com> Co-authored-by: JeremyParish69 <95667791+JeremyParish69@users.noreply.github.com> Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> Co-authored-by: pratikbin <68642400+pratikbin@users.noreply.github.com> Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> Co-authored-by: Osho Emmanuel <oshoklinsmann@gmail.com> Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com>
oshorefueled
added a commit
to ComposableFi/relayer
that referenced
this pull request
Oct 20, 2022
* Merge PR cosmos#919: Clean up and move scripts folder and dev-env * clean up scripts * clean up Makerfile * cleanup configs * fix shebang * update data folder path * move to examples dir * update build-gaia path for make * add-dir description * examples * examples readme * add killall note to readme * fix link * remove extra config dir * update path in demo.md * readme nit * Use self-hosted runner for ibctest integration tests (cosmos#943) * bugfix - intermittent channel handshake failure (cosmos#937) * bugfix for intermittent channel handshake failure * Add unit tests for connection and channel state caches * Log messages in failed txs * Tidy * Adding MinGasAmount (cosmos#940) * Adding MinGasAmount * go.mod updates * Updating ibctest/go.mod * Adding min-gas-amount to CHANGELOG and troubleshooting * Merge PR cosmos#953: cosmos - begin and end block event parsing * Add begin and end block events * tidy * Handle multiple ibc messages in begin block and end block * Fix logic with accumulator * tidy * Fix nil pointer for begin/end block events (cosmos#956) * Fix bug with tracking processing channel msgs (cosmos#970) * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * undo previous changes regarding purging cache on max retries * undo more changes made in shouldSendChannelMessage * more undo * add debug output on msg send and channel cache purge * add debug output * remove pointless log * add logs in process latest msgs * more logs * logs * remove all debug logs and implement fix for tracking processing packets correctly * Merge PR cosmos#974: Clarify demo * update demo * remove extra dev-env * Merge PR cosmos#963: Additional Metrics * Adding block height and wallet balance metrics * Fixing duplicate registration * Assigning metrics during NewCosmosChainProcessor * Fixing latest height to use ChainId * Metrics are optional... * Merge PR cosmos#981: Update IBCdata struct Update IBCdata struct based on recent changes made at the Cosmos Chain Registry * Fix flattening of events (cosmos#979) * Fix flattening of events * Parse events without flattened tx.Log * Add missing write ack to switch case * Fix unnecessary retention of irrelevant messages (cosmos#980) * Merge PR cosmos#987: Use config.lock file to guard access to config.yaml in the case of linking multiple paths concurrently * ignore recv packet with empty ack bytes (cosmos#985) Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> * Relay multiple paths from the same process (cosmos#988) * wip: add test for relaying on many paths This test instantiates a relayer with the event processor that relays on two paths across three chains. This was ported from previous WIP that attempted to add multi-path support to the legacy processor for the relayer; multi-path support in the legacy processor had some subtle bugs that caused the test to fail. Now with the new event processor, the test is failing in a different way that we need to further debug. * Multiple paths same process with test * go mod tidy * fix old tests * Update to ibctest with race fix * Fix legacy relayer Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> * Default to events processor (cosmos#989) * Default to events processor * events processor is now default * sort packet messages by sequence number (cosmos#992) only process 1 packet at a time on ordered channels * path processor race (cosmos#995) * fix path processor indexing * clone maps before passing to pathprocessor * Merge should Clone * pre-size maps * Use in process Relayer for race detection (cosmos#996) * clone maps before passing to pathprocessor * Add local relayer implementation to help catch race conditions in CI * use existing relayerfactory * Isolate mutex to totalFees * handle feedback * fix fatalf format * Merge PR cosmos#999: Add ability to run all paths in config easil * Add ability to run all paths in config easily * PR review fixes * Fix tests and patch readme * Merge PR cosmos#1005: Fix legacy race condition * fix counterparty path filter (cosmos#1000) * fix counterparty path filter * Filter fix test * Add denylist test and add makefile and gh action * Slim test for non-self-hosted runner * Update ibctest to latest main * Make better assertion for denylist acks. Constants for allowlist/denylist. Validate filterRule in CLI * Use isolated prometheus registry per relayer instance instead of prometheus default registry * run path filter tests in parallel * dockerfile: move TARGET* args for fast workflow & ... (cosmos#993) - Single COPY step for go mod and sum - Use ENV GOOS & GOARCH from TARGET* args Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> * Respect the `override` flag upon client creation (cosmos#997) * respect the override flag * add test case for --override usage * add missing godoc comment for test * disable CountTotal (cosmos#1009) * Fix account sequence mismatch errors (cosmos#1007) * Use mutex for tx account sequence query through tx broadcast, track account sequence number on chain provider * Make updateNextAccountSequence private * Retry for account sequence mismatch errors * Allow decrementing account sequence if node says it is expected * Add test for handleAccountSequenceMismatchError * Keystore implementation for substrate chain (cosmos#922) * add substrate keystore implementation * implement key provider for substrate chain * clean up keystore code * run go mod tidy moving rename test to _test * remove verbs from error messages * correct interface method comments in Keyring and Info types * initialize log in NewProvider method * move type definitions to method implementation files * check if KeyDirectory is empty before overwriting it in the NewProvider method * remove comments in ExportPrivKeyArmor * bump substrate-rpc-client to latest master * change network type to uint16 * update go.mod in ibctest * feat: add chain processor implementation (cosmos#924) * feat: add chain processor implementation * fix: go-schnorrkel version fix * fix: go mod update dockertest * feature/ implement substrate event parser (cosmos#923) * feat: implement substrate event parser * fix: event parser switch fallthrough * fix: go-schnorrkel version fix * fix: go mod update for dcokertest * create helper functions between substrate <> ibc events * use intoIBCEventType * fix: event parser accumulation support * fix: spearate parsing and handling of ibc messages * feat: parachain header parse * fix: packet accumulator separate switch case * fix: go.mod docker issue * fix: ics11-beefy repo version fix * fix: accumulator events to messages * fix: add connection event type conversion * fix: ibctest go.mod fix Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com> * fix: go.mod Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com> Co-authored-by: Andrew Gouin <andrew@gouin.io> Co-authored-by: Joe Abbey <joe.abbey@gmail.com> Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com> Co-authored-by: JeremyParish69 <95667791+JeremyParish69@users.noreply.github.com> Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> Co-authored-by: pratikbin <68642400+pratikbin@users.noreply.github.com> Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> Co-authored-by: Osho Emmanuel <oshoklinsmann@gmail.com> Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com>
oshorefueled
added a commit
to oshorefueled/relayer
that referenced
this pull request
Oct 20, 2022
* Merge PR cosmos#919: Clean up and move scripts folder and dev-env * clean up scripts * clean up Makerfile * cleanup configs * fix shebang * update data folder path * move to examples dir * update build-gaia path for make * add-dir description * examples * examples readme * add killall note to readme * fix link * remove extra config dir * update path in demo.md * readme nit * Use self-hosted runner for ibctest integration tests (cosmos#943) * bugfix - intermittent channel handshake failure (cosmos#937) * bugfix for intermittent channel handshake failure * Add unit tests for connection and channel state caches * Log messages in failed txs * Tidy * Adding MinGasAmount (cosmos#940) * Adding MinGasAmount * go.mod updates * Updating ibctest/go.mod * Adding min-gas-amount to CHANGELOG and troubleshooting * Merge PR cosmos#953: cosmos - begin and end block event parsing * Add begin and end block events * tidy * Handle multiple ibc messages in begin block and end block * Fix logic with accumulator * tidy * Fix nil pointer for begin/end block events (cosmos#956) * Fix bug with tracking processing channel msgs (cosmos#970) * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * undo previous changes regarding purging cache on max retries * undo more changes made in shouldSendChannelMessage * more undo * add debug output on msg send and channel cache purge * add debug output * remove pointless log * add logs in process latest msgs * more logs * logs * remove all debug logs and implement fix for tracking processing packets correctly * Merge PR cosmos#974: Clarify demo * update demo * remove extra dev-env * Merge PR cosmos#963: Additional Metrics * Adding block height and wallet balance metrics * Fixing duplicate registration * Assigning metrics during NewCosmosChainProcessor * Fixing latest height to use ChainId * Metrics are optional... * Merge PR cosmos#981: Update IBCdata struct Update IBCdata struct based on recent changes made at the Cosmos Chain Registry * Fix flattening of events (cosmos#979) * Fix flattening of events * Parse events without flattened tx.Log * Add missing write ack to switch case * Fix unnecessary retention of irrelevant messages (cosmos#980) * Merge PR cosmos#987: Use config.lock file to guard access to config.yaml in the case of linking multiple paths concurrently * ignore recv packet with empty ack bytes (cosmos#985) Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> * Relay multiple paths from the same process (cosmos#988) * wip: add test for relaying on many paths This test instantiates a relayer with the event processor that relays on two paths across three chains. This was ported from previous WIP that attempted to add multi-path support to the legacy processor for the relayer; multi-path support in the legacy processor had some subtle bugs that caused the test to fail. Now with the new event processor, the test is failing in a different way that we need to further debug. * Multiple paths same process with test * go mod tidy * fix old tests * Update to ibctest with race fix * Fix legacy relayer Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> * Default to events processor (cosmos#989) * Default to events processor * events processor is now default * sort packet messages by sequence number (cosmos#992) only process 1 packet at a time on ordered channels * path processor race (cosmos#995) * fix path processor indexing * clone maps before passing to pathprocessor * Merge should Clone * pre-size maps * Use in process Relayer for race detection (cosmos#996) * clone maps before passing to pathprocessor * Add local relayer implementation to help catch race conditions in CI * use existing relayerfactory * Isolate mutex to totalFees * handle feedback * fix fatalf format * Merge PR cosmos#999: Add ability to run all paths in config easil * Add ability to run all paths in config easily * PR review fixes * Fix tests and patch readme * Merge PR cosmos#1005: Fix legacy race condition * fix counterparty path filter (cosmos#1000) * fix counterparty path filter * Filter fix test * Add denylist test and add makefile and gh action * Slim test for non-self-hosted runner * Update ibctest to latest main * Make better assertion for denylist acks. Constants for allowlist/denylist. Validate filterRule in CLI * Use isolated prometheus registry per relayer instance instead of prometheus default registry * run path filter tests in parallel * dockerfile: move TARGET* args for fast workflow & ... (cosmos#993) - Single COPY step for go mod and sum - Use ENV GOOS & GOARCH from TARGET* args Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> * Respect the `override` flag upon client creation (cosmos#997) * respect the override flag * add test case for --override usage * add missing godoc comment for test * disable CountTotal (cosmos#1009) * Fix account sequence mismatch errors (cosmos#1007) * Use mutex for tx account sequence query through tx broadcast, track account sequence number on chain provider * Make updateNextAccountSequence private * Retry for account sequence mismatch errors * Allow decrementing account sequence if node says it is expected * Add test for handleAccountSequenceMismatchError * Keystore implementation for substrate chain (cosmos#922) * add substrate keystore implementation * implement key provider for substrate chain * clean up keystore code * run go mod tidy moving rename test to _test * remove verbs from error messages * correct interface method comments in Keyring and Info types * initialize log in NewProvider method * move type definitions to method implementation files * check if KeyDirectory is empty before overwriting it in the NewProvider method * remove comments in ExportPrivKeyArmor * bump substrate-rpc-client to latest master * change network type to uint16 * update go.mod in ibctest * feat: add chain processor implementation (cosmos#924) * feat: add chain processor implementation * fix: go-schnorrkel version fix * fix: go mod update dockertest * feature/ implement substrate event parser (cosmos#923) * feat: implement substrate event parser * fix: event parser switch fallthrough * fix: go-schnorrkel version fix * fix: go mod update for dcokertest * create helper functions between substrate <> ibc events * use intoIBCEventType * fix: event parser accumulation support * fix: spearate parsing and handling of ibc messages * feat: parachain header parse * fix: packet accumulator separate switch case * fix: go.mod docker issue * fix: ics11-beefy repo version fix * fix: accumulator events to messages * fix: add connection event type conversion * fix: ibctest go.mod fix Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com> * fix: go.mod Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com> Co-authored-by: Andrew Gouin <andrew@gouin.io> Co-authored-by: Joe Abbey <joe.abbey@gmail.com> Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com> Co-authored-by: JeremyParish69 <95667791+JeremyParish69@users.noreply.github.com> Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> Co-authored-by: pratikbin <68642400+pratikbin@users.noreply.github.com> Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> Co-authored-by: Osho Emmanuel <oshoklinsmann@gmail.com> Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com>
jtieri
added a commit
that referenced
this pull request
Oct 21, 2022
* Merge PR #919: Clean up and move scripts folder and dev-env * clean up scripts * clean up Makerfile * cleanup configs * fix shebang * update data folder path * move to examples dir * update build-gaia path for make * add-dir description * examples * examples readme * add killall note to readme * fix link * remove extra config dir * update path in demo.md * readme nit * Use self-hosted runner for ibctest integration tests (#943) * bugfix - intermittent channel handshake failure (#937) * bugfix for intermittent channel handshake failure * Add unit tests for connection and channel state caches * Log messages in failed txs * Tidy * Adding MinGasAmount (#940) * Adding MinGasAmount * go.mod updates * Updating ibctest/go.mod * Adding min-gas-amount to CHANGELOG and troubleshooting * Merge PR #953: cosmos - begin and end block event parsing * Add begin and end block events * tidy * Handle multiple ibc messages in begin block and end block * Fix logic with accumulator * tidy * Fix nil pointer for begin/end block events (#956) * Fix bug with tracking processing channel msgs (#970) * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * undo previous changes regarding purging cache on max retries * undo more changes made in shouldSendChannelMessage * more undo * add debug output on msg send and channel cache purge * add debug output * remove pointless log * add logs in process latest msgs * more logs * logs * remove all debug logs and implement fix for tracking processing packets correctly * Merge PR #974: Clarify demo * update demo * remove extra dev-env * Merge PR #963: Additional Metrics * Adding block height and wallet balance metrics * Fixing duplicate registration * Assigning metrics during NewCosmosChainProcessor * Fixing latest height to use ChainId * Metrics are optional... * Merge PR #981: Update IBCdata struct Update IBCdata struct based on recent changes made at the Cosmos Chain Registry * Fix flattening of events (#979) * Fix flattening of events * Parse events without flattened tx.Log * Add missing write ack to switch case * Fix unnecessary retention of irrelevant messages (#980) * Merge PR #987: Use config.lock file to guard access to config.yaml in the case of linking multiple paths concurrently * ignore recv packet with empty ack bytes (#985) Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> * Relay multiple paths from the same process (#988) * wip: add test for relaying on many paths This test instantiates a relayer with the event processor that relays on two paths across three chains. This was ported from previous WIP that attempted to add multi-path support to the legacy processor for the relayer; multi-path support in the legacy processor had some subtle bugs that caused the test to fail. Now with the new event processor, the test is failing in a different way that we need to further debug. * Multiple paths same process with test * go mod tidy * fix old tests * Update to ibctest with race fix * Fix legacy relayer Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> * Default to events processor (#989) * Default to events processor * events processor is now default * sort packet messages by sequence number (#992) only process 1 packet at a time on ordered channels * path processor race (#995) * fix path processor indexing * clone maps before passing to pathprocessor * Merge should Clone * pre-size maps * Use in process Relayer for race detection (#996) * clone maps before passing to pathprocessor * Add local relayer implementation to help catch race conditions in CI * use existing relayerfactory * Isolate mutex to totalFees * handle feedback * fix fatalf format * Merge PR #999: Add ability to run all paths in config easil * Add ability to run all paths in config easily * PR review fixes * Fix tests and patch readme * Merge PR #1005: Fix legacy race condition * fix counterparty path filter (#1000) * fix counterparty path filter * Filter fix test * Add denylist test and add makefile and gh action * Slim test for non-self-hosted runner * Update ibctest to latest main * Make better assertion for denylist acks. Constants for allowlist/denylist. Validate filterRule in CLI * Use isolated prometheus registry per relayer instance instead of prometheus default registry * run path filter tests in parallel * dockerfile: move TARGET* args for fast workflow & ... (#993) - Single COPY step for go mod and sum - Use ENV GOOS & GOARCH from TARGET* args Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> * Respect the `override` flag upon client creation (#997) * respect the override flag * add test case for --override usage * add missing godoc comment for test * disable CountTotal (#1009) * Fix account sequence mismatch errors (#1007) * Use mutex for tx account sequence query through tx broadcast, track account sequence number on chain provider * Make updateNextAccountSequence private * Retry for account sequence mismatch errors * Allow decrementing account sequence if node says it is expected * Add test for handleAccountSequenceMismatchError * Bumps versions of some libs (#1010) * bumps * run cmds for go mod tidy Co-authored-by: Andrew Gouin <andrew@gouin.io> Co-authored-by: jtieri <justin@thetieris.com> * Add test case for multiple channels on one connection + remove integration tests in `_test` (#1021) * test: add multiple channels on one connection test case via ibctest * chore: remove old docker integration test make cmds * chore: remove github CI actions for old docker integration tests * chore: remove `_test` directory containing the old docker integration tests * Keystore implementation for substrate chain (#922) * add substrate keystore implementation * implement key provider for substrate chain * clean up keystore code * run go mod tidy moving rename test to _test * remove verbs from error messages * correct interface method comments in Keyring and Info types * initialize log in NewProvider method * move type definitions to method implementation files * check if KeyDirectory is empty before overwriting it in the NewProvider method * remove comments in ExportPrivKeyArmor * bump substrate-rpc-client to latest master * change network type to uint16 * update go.mod in ibctest * feat: add chain processor implementation (#924) * feat: add chain processor implementation * fix: go-schnorrkel version fix * fix: go mod update dockertest * feature/ implement substrate event parser (#923) * feat: implement substrate event parser * fix: event parser switch fallthrough * fix: go-schnorrkel version fix * fix: go mod update for dcokertest * create helper functions between substrate <> ibc events * use intoIBCEventType * fix: event parser accumulation support * fix: spearate parsing and handling of ibc messages * feat: parachain header parse * fix: packet accumulator separate switch case * fix: go.mod docker issue * fix: ics11-beefy repo version fix * fix: accumulator events to messages * fix: add connection event type conversion * fix: ibctest go.mod fix Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com> * Sync/feat substrate main merge (#1016) * Merge PR #919: Clean up and move scripts folder and dev-env * clean up scripts * clean up Makerfile * cleanup configs * fix shebang * update data folder path * move to examples dir * update build-gaia path for make * add-dir description * examples * examples readme * add killall note to readme * fix link * remove extra config dir * update path in demo.md * readme nit * Use self-hosted runner for ibctest integration tests (#943) * bugfix - intermittent channel handshake failure (#937) * bugfix for intermittent channel handshake failure * Add unit tests for connection and channel state caches * Log messages in failed txs * Tidy * Adding MinGasAmount (#940) * Adding MinGasAmount * go.mod updates * Updating ibctest/go.mod * Adding min-gas-amount to CHANGELOG and troubleshooting * Merge PR #953: cosmos - begin and end block event parsing * Add begin and end block events * tidy * Handle multiple ibc messages in begin block and end block * Fix logic with accumulator * tidy * Fix nil pointer for begin/end block events (#956) * Fix bug with tracking processing channel msgs (#970) * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * add debug output * add more debug output * NEEDS MOAR DEBUG OUTPUT * test purging MsgTimeout on src vs. counterparty * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * reverse channel close msg purging * don't delete channel close msgs until another iteration in getUnrelayedPacketsAndAcks * undo previous changes regarding purging cache on max retries * undo more changes made in shouldSendChannelMessage * more undo * add debug output on msg send and channel cache purge * add debug output * remove pointless log * add logs in process latest msgs * more logs * logs * remove all debug logs and implement fix for tracking processing packets correctly * Merge PR #974: Clarify demo * update demo * remove extra dev-env * Merge PR #963: Additional Metrics * Adding block height and wallet balance metrics * Fixing duplicate registration * Assigning metrics during NewCosmosChainProcessor * Fixing latest height to use ChainId * Metrics are optional... * Merge PR #981: Update IBCdata struct Update IBCdata struct based on recent changes made at the Cosmos Chain Registry * Fix flattening of events (#979) * Fix flattening of events * Parse events without flattened tx.Log * Add missing write ack to switch case * Fix unnecessary retention of irrelevant messages (#980) * Merge PR #987: Use config.lock file to guard access to config.yaml in the case of linking multiple paths concurrently * ignore recv packet with empty ack bytes (#985) Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> * Relay multiple paths from the same process (#988) * wip: add test for relaying on many paths This test instantiates a relayer with the event processor that relays on two paths across three chains. This was ported from previous WIP that attempted to add multi-path support to the legacy processor for the relayer; multi-path support in the legacy processor had some subtle bugs that caused the test to fail. Now with the new event processor, the test is failing in a different way that we need to further debug. * Multiple paths same process with test * go mod tidy * fix old tests * Update to ibctest with race fix * Fix legacy relayer Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> * Default to events processor (#989) * Default to events processor * events processor is now default * sort packet messages by sequence number (#992) only process 1 packet at a time on ordered channels * path processor race (#995) * fix path processor indexing * clone maps before passing to pathprocessor * Merge should Clone * pre-size maps * Use in process Relayer for race detection (#996) * clone maps before passing to pathprocessor * Add local relayer implementation to help catch race conditions in CI * use existing relayerfactory * Isolate mutex to totalFees * handle feedback * fix fatalf format * Merge PR #999: Add ability to run all paths in config easil * Add ability to run all paths in config easily * PR review fixes * Fix tests and patch readme * Merge PR #1005: Fix legacy race condition * fix counterparty path filter (#1000) * fix counterparty path filter * Filter fix test * Add denylist test and add makefile and gh action * Slim test for non-self-hosted runner * Update ibctest to latest main * Make better assertion for denylist acks. Constants for allowlist/denylist. Validate filterRule in CLI * Use isolated prometheus registry per relayer instance instead of prometheus default registry * run path filter tests in parallel * dockerfile: move TARGET* args for fast workflow & ... (#993) - Single COPY step for go mod and sum - Use ENV GOOS & GOARCH from TARGET* args Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> * Respect the `override` flag upon client creation (#997) * respect the override flag * add test case for --override usage * add missing godoc comment for test * disable CountTotal (#1009) * Fix account sequence mismatch errors (#1007) * Use mutex for tx account sequence query through tx broadcast, track account sequence number on chain provider * Make updateNextAccountSequence private * Retry for account sequence mismatch errors * Allow decrementing account sequence if node says it is expected * Add test for handleAccountSequenceMismatchError * Keystore implementation for substrate chain (#922) * add substrate keystore implementation * implement key provider for substrate chain * clean up keystore code * run go mod tidy moving rename test to _test * remove verbs from error messages * correct interface method comments in Keyring and Info types * initialize log in NewProvider method * move type definitions to method implementation files * check if KeyDirectory is empty before overwriting it in the NewProvider method * remove comments in ExportPrivKeyArmor * bump substrate-rpc-client to latest master * change network type to uint16 * update go.mod in ibctest * feat: add chain processor implementation (#924) * feat: add chain processor implementation * fix: go-schnorrkel version fix * fix: go mod update dockertest * feature/ implement substrate event parser (#923) * feat: implement substrate event parser * fix: event parser switch fallthrough * fix: go-schnorrkel version fix * fix: go mod update for dcokertest * create helper functions between substrate <> ibc events * use intoIBCEventType * fix: event parser accumulation support * fix: spearate parsing and handling of ibc messages * feat: parachain header parse * fix: packet accumulator separate switch case * fix: go.mod docker issue * fix: ics11-beefy repo version fix * fix: accumulator events to messages * fix: add connection event type conversion * fix: ibctest go.mod fix Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com> * fix: go.mod Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com> Co-authored-by: Andrew Gouin <andrew@gouin.io> Co-authored-by: Joe Abbey <joe.abbey@gmail.com> Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com> Co-authored-by: JeremyParish69 <95667791+JeremyParish69@users.noreply.github.com> Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> Co-authored-by: pratikbin <68642400+pratikbin@users.noreply.github.com> Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> Co-authored-by: Osho Emmanuel <oshoklinsmann@gmail.com> Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com> * Feature/ implement SubstrateIBCHeader (#986) * feat: add SubstrateIBCHeader type and implementation * fix: ibctest go mod tidy * feature/ implement substrate event parser (#923) * feat: implement substrate event parser * fix: event parser switch fallthrough * fix: go-schnorrkel version fix * fix: go mod update for dcokertest * create helper functions between substrate <> ibc events * use intoIBCEventType * fix: event parser accumulation support * fix: spearate parsing and handling of ibc messages * feat: parachain header parse * fix: packet accumulator separate switch case * fix: go.mod docker issue * fix: ics11-beefy repo version fix * fix: accumulator events to messages * fix: add connection event type conversion * fix: ibctest go.mod fix Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com> Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com> * feat: copy cosmos message handler and assign to substrate (#1017) * Feature/ Substrate TX methods (#1011) * fix: error fix and dependencies in tx * fix wrong value instead of reference * cleanup: remove unused types and functions * fix: fix go.mod * fix: add fslock to go.mod * fix: review code smells removal * fix: modify sendmessages * fix: review comments * feat: add events to messages * fix: unexported fields * fix: go mod tidy _test * Merge branch 'feat-substrate' into upstream_update * go mod ibctest module Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com> Co-authored-by: Andrew Gouin <andrew@gouin.io> Co-authored-by: Joe Abbey <joe.abbey@gmail.com> Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com> Co-authored-by: JeremyParish69 <95667791+JeremyParish69@users.noreply.github.com> Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com> Co-authored-by: Mark Rushakoff <mark.rushakoff@gmail.com> Co-authored-by: pratikbin <68642400+pratikbin@users.noreply.github.com> Co-authored-by: pratikbin <pratikbin+010101@no-reply-git.luolix.top> Co-authored-by: Jacob Gadikian <jacobgadikian@gmail.com> Co-authored-by: jtieri <justin@thetieris.com> Co-authored-by: Mehdi Valinejad <6095314+vjdmhd@users.noreply.github.com> Co-authored-by: Blas Rodriguez Irizar <rodrigblas@gmail.com>
agouin
added a commit
that referenced
this pull request
Nov 15, 2022
* clone maps before passing to pathprocessor * Add local relayer implementation to help catch race conditions in CI * use existing relayerfactory * Isolate mutex to totalFees * handle feedback * fix fatalf format
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reason: Catch race conditions in CI
Implements
ibctestibc.Relayerinterface using the local relayer code. This allows thego test -racetesting to have the most insight into potential races.Fixes a potential race with the prometheus metrics in the cosmos chain provider.