Skip to content

Commit

Permalink
feat: merge relayer (#94)
Browse files Browse the repository at this point in the history
* Flush query should include begin and end block events (cosmos#1125)

* Include begin and end block events

* disable flushing when termination condition is set

* Still flush for FlushLifecycle

* Add sort for flush logging to avoid confusion

* pre_init messages (cosmos#1131)

* Wire initial messages into path processor caches so that retry logic will occur

* Fix counterparty keys

* Remove debug log

* fix default coin type (cosmos#1134)

* fix slip44 default

* Add test case

* build: bump to Go 1.20 + bump deps (cosmos#1132)

* build: bump to Go 1.20 + bump deps

This bumps the Go version to 1.20 and also bumps the SDK version to 0.47.0 and ibc-go to v7.0.0

* chore: update GH workflows to use Go 1.20 + update interchaintest deps

* chore: update missing deps in go.sum + use 1.20 in dockerfiles

* chore: bump to `setup-go/v4` and remove caching step

* chore: bump to `checkout/v3` and remove caching step

* chore: bump 1.20.2

* Fix flushing acks (cosmos#1139)

* Fix ordered channel closure (cosmos#1142)

* Fix ordered channel closure

* Increase timeout for scenarios test

* Fix tracking of init messages when IDs aren't the same

* bump interchaintest to include explicit port bindings

* Fix flush termination condition (cosmos#1141)

* bump to main sha (cosmos#1143)

* Pre-filter flush channels (cosmos#1146)

* Add channel close correlation (cosmos#1145)

* Add channel close correlation

* Switch to pre-close key

* make tx channel-close cli command work, add test coverage

* more sweet code removals

* update comment

* Fix flush on ordered channels (cosmos#1150)

* Fix flush on ordered channels

* Queue all packets at nextseqrecv or above

* Now that we have periodic flushing, skip blocks if they can't be queried (cosmos#1154)

* Lock config file for all write operations (cosmos#1156)

* Lock config file for all write operations

* Fix linter errs

* tidy

* more tidy

* Penumbra support v2 (cosmos#1144)

* Penumbra buf go gen

* Use go prefix override

* wip: penumbra relayer provider

remove copied-over cosmos provider tests, rename processor cosmos -> penumbra

rename ccp -> pcp

reformat into new relayer dir structure

update penumbra types to point to buf.build

building again

* fix penumbra

* fix: implement MsgSubmitMisbehaviour

* fix: remove unnecessary proto file + fix msg type cast

* chore: add removal of penumbra client protos in protocgen

* working penumbra relayer functionality

wip: unbase64?

wip: multiple messages per penumbra tx

wip: stub SendMessages impl

wip: attempt to split out common method

wip: use random anchor and work around path renaming

wip: improve logging

wip: changes during pairing

https://www.youtube.com/watch?v=RYonSOkZ5ZE

clean up logs

skip height bug workaround

drop debug panic

* update penumbra chain processor connection and channel message processing

* cleanup logging statements for review

Responding to review comments, honoring the style guide for logging, and
removing some unused reference code that was commented out while
debugging.

---------

Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Ava Howell <ava@penumbra.zone>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* --time-threshold example use cases (cosmos#1155)

* Make ICA waits more explicit (cosmos#1157)

* Make ICA waits more explicit

* poll for timeout

* poll for channel close confirm

* Comment out sqlite db file for scenarios tests

* Bump github.com/docker/docker in /interchaintest (cosmos#1160)

Bumps [github.com/docker/docker](https://github.com/docker/docker) from 20.10.19+incompatible to 20.10.24+incompatible.
- [Release notes](https://github.com/docker/docker/releases)
- [Commits](moby/moby@v20.10.19...v20.10.24)

---
updated-dependencies:
- dependency-name: github.com/docker/docker
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* bump version in docs (cosmos#1158)

* Bump github.com/opencontainers/runc in /interchaintest (cosmos#1153)

Bumps [github.com/opencontainers/runc](https://github.com/opencontainers/runc) from 1.1.3 to 1.1.5.
- [Release notes](https://github.com/opencontainers/runc/releases)
- [Changelog](https://github.com/opencontainers/runc/blob/v1.1.5/CHANGELOG.md)
- [Commits](opencontainers/runc@v1.1.3...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/opencontainers/runc
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Fix QueryConnectionsUsingClient for cosmos (cosmos#1162)

* Make min query loop duration configurable (cosmos#1164)

* penumbra provider: update generated protos (cosmos#1168)

* Add sr25519 support (cosmos#1120)

* Migrate to cometbft

* Additional replaces

* Register comet proto types

* Add sr25519 support

* bump ictest

* Add keys test

* Update supported algos comment

* penumbra: update protos (cosmos#1170)

Updating the generated protos for Penumbra support. After lightly
editing the `scripts/protocgen.sh` script, I ran `make proto-gen`
and then committed the changes `relayer/chains/penumbra/`. Other
automatically updated protos I intentionally excluded from this PR, to
avoid side-effects.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>

* chore: add path name to logs in message processor (cosmos#1171)

* Fix multiple conn open init (cosmos#1173)

* allow register with extra_codecs (cosmos#1175)

* Harry/fee middleware (cosmos#1174)

* Register Counterparty relayer cmd and fee middleware test

* debugging the command

* debugging and finalizing the fee_middleware_test.

* debugging and finalizing the fee_middleware_test.

* merged with updated repo

* clear out some commanded code

* nits and suggestions post review

* more nits

* added one val no fullnode as chainspec

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>

* fix: nil receiver initiate for path (cosmos#1177)

* fix nil receiver initiate for path

ensure path get written to config

* add change doc

* feat: add max-gas-amount parameter in chain configs (cosmos#1178)

* add max fee

* add test

* add change doc

* Update cregistry/chain_info.go

* dep: bump sdk from v0.47.0 to v0.47.2 (cosmos#1180)

* bump sdk from v0.47.0 to v0.47.2

fix btcutil dep

* fix dep of hdkeychain

* Harry/rly tx channel (cosmos#1183)

* made a new method "logChannelOpenMessage" to log the newly opened channel into info level

* added fields

* some changes

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>

* Harry/rly tx transfer (cosmos#1184)

* made a new method "logChannelOpenMessage" to log the newly opened channel into info level

* added fields

* some changes

* recreated issue 1151, added logs

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* Better Error Messaging when failing to query the Block Height (cosmos#1189)

* better block data errors

* remove redundant field

* penumbra: update protos (cosmos#1181)

Matches the latest protos shipped with the Penumbra Testnet 52.

Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>

* Neutron launch fixes and optimizations (cosmos#1185)

* pipe max msgs through path processor

* only apply max msgs to packet msgs

* multiple msgs simultaneously on ordered chans

* flush should be more frequent if it fails or does not complete

* fix legacy

* handle feedback

* Problem: fixes in ibc-go v7.0.1 are not included (cosmos#1205)

* Problem: fixes in ibc-go v7.0.1 are not included

* add change doc

* Harry/rly address (cosmos#1204)

* added addresCmd to root and keys.go

* nicks

* nick

* made a common method "showAddressByChainAndKey" to be used by both addressCmd and keysShowCmd

---------

Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: Andrew Gouin <andrew@gouin.io>

* deps: update to ibc-go v7.1.0-rc0 (cosmos#1207)

* Export wallet address for Prometheus metrics (cosmos#1206)

* export relayer address for pro

* address in updateFeesSpent

* make error messages consistent

* log error rather than return

* handle 0 balance

* chore: fix issue

* fix: connection key include

* chore: comment out flush

* fix: remove SET method from providerConfig

* chore: replace static naming with constant

* chore: packet timeout after relayer update

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Andrew Gouin <andrew@gouin.io>
Co-authored-by: Justin Tieri <37750742+jtieri@users.noreply.github.com>
Co-authored-by: Henry de Valence <hdevalence@hdevalence.ca>
Co-authored-by: Ava Howell <ava@penumbra.zone>
Co-authored-by: jtieri <justin@thetieris.com>
Co-authored-by: Ava Howell <ava@avahowell.me>
Co-authored-by: Conor Schaefer <conor@penumbralabs.xyz>
Co-authored-by: Dan Kanefsky <56059752+boojamya@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Conor Schaefer <conorsch@users.noreply.github.com>
Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Cosmos-Harry <110472914+Cosmos-Harry@users.noreply.github.com>
Co-authored-by: Harry <harrycosmos@Harrys-MacBook-Pro.local>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Keefer Taylor | Tessellated <keefer@tessellated.io>
Co-authored-by: izyak <test@test.com>
Co-authored-by: viveksharmapoudel <viveksharmapoudel@gmail.com>
  • Loading branch information
18 people authored and izyak committed Sep 7, 2023
1 parent d0c5d34 commit bbbde50
Show file tree
Hide file tree
Showing 26 changed files with 284 additions and 249 deletions.
22 changes: 13 additions & 9 deletions .github/workflows/interchaintest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,10 @@ jobs:
runs-on: self-hosted
steps:
- name: Set up Go 1.20
uses: actions/setup-go@v4
uses: actions/setup-go@v1
with:
go-version: '1.20'
go-version: 1.20
id: go

- name: checkout relayer
uses: actions/checkout@v2
Expand All @@ -31,9 +32,10 @@ jobs:
runs-on: self-hosted
steps:
- name: Set up Go 1.20
uses: actions/setup-go@v4
uses: actions/setup-go@v1
with:
go-version: '1.20'
go-version: 1.20
id: go

- name: checkout relayer
uses: actions/checkout@v2
Expand All @@ -52,9 +54,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.20
uses: actions/setup-go@v4
uses: actions/setup-go@v1
with:
go-version: '1.20'
go-version: 1.20
id: go

- name: checkout relayer
uses: actions/checkout@v2
Expand Down Expand Up @@ -136,9 +139,10 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Set up Go 1.20
uses: actions/setup-go@v4
uses: actions/setup-go@v1
with:
go-version: '1.20'
go-version: 1.20
id: go

- name: checkout relayer
uses: actions/checkout@v2
Expand All @@ -151,4 +155,4 @@ jobs:
${{ runner.os }}-go-
- name: interchaintest
run: make interchaintest-scenario
run: make interchaintest-scenario
2 changes: 1 addition & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v4
with:
go-version: '1.20'
go-version: 1.20

- run: echo https://github.com/cosmos/relayer/blob/${GITHUB_REF#refs/tags/}/CHANGELOG.md#${GITHUB_REF#refs/tags/} > ../release_notes.md

Expand Down
272 changes: 148 additions & 124 deletions cmd/appstate.go
Original file line number Diff line number Diff line change
Expand Up @@ -194,19 +194,9 @@ func (a *appState) performConfigLockingOperation(ctx context.Context, operation
return fmt.Errorf("failed to initialize config from file: %w", err)
}

path, ok := a.Config.Paths[pathName]

if !ok {
return fmt.Errorf("config does not exist for that path: %s", pathName)
}
if clientSrc != "" {
path.Src.ClientID = clientSrc
}
if clientDst != "" {
path.Dst.ClientID = clientDst
}
if connectionSrc != "" {
path.Src.ConnectionID = connectionSrc
// perform the operation that requires config flock.
if err := operation(); err != nil {
return err
}

// validate config after changes have been made.
Expand All @@ -226,12 +216,13 @@ func (a *appState) performConfigLockingOperation(ctx context.Context, operation
if err := os.WriteFile(cfgPath, out, 0600); err != nil {
return fmt.Errorf("failed to write config file at %s: %w", cfgPath, err)
}

return nil
}

func (a *appState) GetConfigProviderNameFromChainId(chainId string) (string, error) {

chains := a.Config.Chains
chains := a.config.Chains
for k, v := range chains {
if v.ChainID() == chainId {
return k, nil
Expand All @@ -242,7 +233,7 @@ func (a *appState) GetConfigProviderNameFromChainId(chainId string) (string, err
}

func (a *appState) CheckIfProviderType(providerName string, providerType string) bool {
providers := a.Config.Wrapped()
providers := a.config.Wrapped()
for p, v := range providers.ProviderConfigs {
if p == providerName && v.Type == providerType {
return true
Expand All @@ -251,121 +242,154 @@ func (a *appState) CheckIfProviderType(providerName string, providerType string)
return false
}

func (a *appState) UpdateConfigsIfContainIcon(cmd *cobra.Command, src *relayer.Chain, dst *relayer.Chain) error {
// func (a *appState) UpdateConfigsIfContainIcon(cmd *cobra.Command, src *relayer.Chain, dst *relayer.Chain) error {

// ctx := context.Background()
// eg, egCtx := errgroup.WithContext(ctx)

// eg.Go(func() error {
// var err error
// err = a.UpdateProviderIfIcon(cmd, egCtx, src)
// if err != nil {
// return err
// }

// return nil

// })
// eg.Go(func() error {
// var err error
// err = a.UpdateProviderIfIcon(cmd, egCtx, dst)
// if err != nil {
// return err
// }
// return nil

// })

// if err := eg.Wait(); err != nil {
// return err
// }

// return nil

// }

// func (a *appState) UpdateProviderIfIcon(cmd *cobra.Command, ctx context.Context, chain *relayer.Chain) error {

// providerName, err := a.GetConfigProviderNameFromChainId(chain.ChainID())
// if err != nil {
// return err
// }

// if !a.CheckIfProviderType(providerName, "icon") {
// return nil
// }
// // height, err := chain.ChainProvider.QueryLatestHeight(ctx)
// // if err != nil {
// // return errors.New(fmt.Sprintf("Error fetching chain latest height %s ", chain.ChainID()))
// // }

// // err = a.OverwriteChainConfig(cmd, providerName, "btpHeight", height)
// // if err != nil {
// // return errors.New(fmt.Sprintf("Error updating BTPHeight of config of chain %s ", chain.ChainID()))
// // }
// return nil
// }

// func (a *appState) OverwriteChainConfig(
// cmd *cobra.Command,
// providerName string,
// fieldName string,
// fieldValue interface{},
// ) error {

// // use lock file to guard concurrent access to config.yaml
// lockFilePath := path.Join(a.homePath, "config", "config.lock")
// fileLock := flock.New(lockFilePath)
// err := fileLock.Lock()
// if err != nil {
// return fmt.Errorf("failed to acquire config lock: %w", err)
// }
// defer func() {
// if err := fileLock.Unlock(); err != nil {
// a.Log.Error("error unlocking config file lock, please manually delete",
// zap.String("filepath", lockFilePath),
// )
// }
// }()

// if err := initConfig(cmd, a); err != nil {
// return fmt.Errorf("failed to initialize config from file: %w", err)
// }

// wrappedConfig := a.config.Wrapped()
// err = setProviderConfigField(wrappedConfig, providerName, fieldName, fieldValue)
// if err != nil {
// return err
// }

// out, err := yaml.Marshal(wrappedConfig)
// if err != nil {
// return err
// }

// cfgPath := a.viper.ConfigFileUsed()

// // Overwrite the config file.
// if err := os.WriteFile(cfgPath, out, 0600); err != nil {
// return fmt.Errorf("failed to write config file at %s: %w", cfgPath, err)
// }

// return nil
// }

// func setProviderConfigField(cfg *ConfigOutputWrapper, providerName string, fieldToChange string, newValue interface{}) error {
// providerConfigs := cfg.ProviderConfigs
// providerConfigWrapper, ok := providerConfigs[providerName]
// if !ok {
// return fmt.Errorf("ProviderConfigWrapper %s not found", providerName)
// }
// providerConfigValue := providerConfigWrapper.Value
// if err := providerConfigValue.Set(fieldToChange, newValue); err != nil {
// return err
// }
// providerConfigWrapper.Value = providerConfigValue

// return nil
// }

ctx := context.Background()
eg, egCtx := errgroup.WithContext(ctx)
// updatePathConfig overwrites the config file concurrently,
// locking to read, modify, then write the config.
func (a *appState) updatePathConfig(
ctx context.Context,
pathName string,
clientSrc, clientDst string,
connectionSrc, connectionDst string,
) error {
if pathName == "" {
return errors.New("empty path name not allowed")
}

eg.Go(func() error {
var err error
err = a.UpdateProviderIfIcon(cmd, egCtx, src)
if err != nil {
return err
return a.performConfigLockingOperation(ctx, func() error {
path, ok := a.config.Paths[pathName]
if !ok {
return fmt.Errorf("config does not exist for that path: %s", pathName)
}

return nil

})
eg.Go(func() error {
var err error
err = a.UpdateProviderIfIcon(cmd, egCtx, dst)
if err != nil {
return err
if clientSrc != "" {
path.Src.ClientID = clientSrc
}
if clientDst != "" {
path.Dst.ClientID = clientDst
}
if connectionSrc != "" {
path.Src.ConnectionID = connectionSrc
}
if connectionDst != "" {
path.Dst.ConnectionID = connectionDst
}
return nil

})

if err := eg.Wait(); err != nil {
return err
}

return nil

}

func (a *appState) UpdateProviderIfIcon(cmd *cobra.Command, ctx context.Context, chain *relayer.Chain) error {

providerName, err := a.GetConfigProviderNameFromChainId(chain.ChainID())
if err != nil {
return err
}

if !a.CheckIfProviderType(providerName, "icon") {
return nil
}
height, err := chain.ChainProvider.QueryLatestHeight(ctx)
if err != nil {
return errors.New(fmt.Sprintf("Error fetching chain latest height %s ", chain.ChainID()))
}

err = a.OverwriteChainConfig(cmd, providerName, "btpHeight", height)
if err != nil {
return errors.New(fmt.Sprintf("Error updating BTPHeight of config of chain %s ", chain.ChainID()))
}
return nil
}

func (a *appState) OverwriteChainConfig(
cmd *cobra.Command,
providerName string,
fieldName string,
fieldValue interface{},
) error {

// use lock file to guard concurrent access to config.yaml
lockFilePath := path.Join(a.HomePath, "config", "config.lock")
fileLock := flock.New(lockFilePath)
err := fileLock.Lock()
if err != nil {
return fmt.Errorf("failed to acquire config lock: %w", err)
}
defer func() {
if err := fileLock.Unlock(); err != nil {
a.Log.Error("error unlocking config file lock, please manually delete",
zap.String("filepath", lockFilePath),
)
}
}()

if err := initConfig(cmd, a); err != nil {
return fmt.Errorf("failed to initialize config from file: %w", err)
}

wrappedConfig := a.Config.Wrapped()
err = setProviderConfigField(wrappedConfig, providerName, fieldName, fieldValue)
if err != nil {
return err
}

out, err := yaml.Marshal(wrappedConfig)
if err != nil {
return err
}

cfgPath := a.Viper.ConfigFileUsed()

// Overwrite the config file.
if err := os.WriteFile(cfgPath, out, 0600); err != nil {
return fmt.Errorf("failed to write config file at %s: %w", cfgPath, err)
}

return nil
}

func setProviderConfigField(cfg *ConfigOutputWrapper, providerName string, fieldToChange string, newValue interface{}) error {
providerConfigs := cfg.ProviderConfigs
providerConfigWrapper, ok := providerConfigs[providerName]
if !ok {
return fmt.Errorf("ProviderConfigWrapper %s not found", providerName)
}
providerConfigValue := providerConfigWrapper.Value
if err := providerConfigValue.Set(fieldToChange, newValue); err != nil {
return err
}
providerConfigWrapper.Value = providerConfigValue

return nil
}

// updatePathConfig overwrites the config file concurrently,
Expand Down
2 changes: 2 additions & 0 deletions cmd/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,8 @@ func (iw *ProviderConfigYAMLWrapper) UnmarshalYAML(n *yaml.Node) error {
iw.Value = new(penumbra.PenumbraProviderConfig)
case "archway":
iw.Value = new(archway.ArchwayProviderConfig)
case "penumbra":
iw.Value = new(penumbra.PenumbraProviderConfig)
default:
return fmt.Errorf("%s is an invalid chain type, check your config file", iw.Type)
}
Expand Down
Loading

0 comments on commit bbbde50

Please sign in to comment.