Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tools/cosmovisor): cosmovisor batch upgrades #21790

Merged
merged 30 commits into from
Oct 10, 2024

Conversation

psiphi5
Copy link
Contributor

@psiphi5 psiphi5 commented Sep 18, 2024

Description

Closes: #20630

This PR adds a add-batch-upgrade to cosmovisor that allows to add multiple named upgrades at specific heights.


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title, you can find examples of the prefixes below:
  • confirmed ! in the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • reviewed "Files changed" and left comments if necessary
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • updated the relevant documentation or specification, including comments for documenting Go code
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic, API design and naming, documentation is accurate, tests and test coverage

Summary by CodeRabbit

  • New Features

    • Introduced the add-batch-upgrade command for efficient batch upgrades.
    • Added the prepare-upgrade command for streamlined upgrade preparation.
    • Added the cosmovisor show-upgrade-info command to display upgrade details.
    • Implemented functionality to pass stdin for enhanced input handling.
    • Introduced a new command for batch processing of upgrades.
  • Improvements

    • Enhanced upgrade management capabilities and usability with new commands and configurations.
    • Improved configuration flexibility by allowing specification of the Cosmos gRPC endpoint via an environment variable.
    • Updated the init command to write configuration to a specified default path and accept a configuration flag.
    • Updated the config command to display configuration from a provided config file or environment variables.
  • Bug Fixes

    • Resolved permission issues with the add-upgrade command.
    • Ensured proper parsing of stdout and stderr outputs.
  • Dependency Updates

    • Updated module dependencies for improved functionality and performance.

Copy link
Contributor

coderabbitai bot commented Sep 18, 2024

📝 Walkthrough

Walkthrough

The pull request introduces enhancements to the tools/cosmovisor module, including the addition of new commands for batch upgrades and improvements to existing functionalities. Key changes include the introduction of the add-batch-upgrade and prepare-upgrade commands, modifications to the init and config commands, and the implementation of features for managing multiple upgrades efficiently. The changelog has been updated to reflect these changes, and new methods have been added to the Config struct to support the new functionalities.

Changes

Files Change Summary
tools/cosmovisor/CHANGELOG.md Updated to include new features (add-batch-upgrade, prepare-upgrade, cosmovisor show-upgrade-info) and improvements to existing commands.
tools/cosmovisor/args.go Added UpgradeInfoBatchFilePath method, DaemonGrpcEndpoint field, and modified DetailString method in the Config struct.
tools/cosmovisor/cmd/cosmovisor/root.go Introduced NewBatchAddUpgradeCmd method to the command-line interface for batch processing of upgrades.
tools/cosmovisor/go.mod Updated dependencies: added direct requirements for github.com/cosmos/cosmos-sdk and github.com/fsnotify/fsnotify, removed indirect requirements, and updated google.golang.org/grpc version.

Assessment against linked issues

Objective Addressed Explanation
Support multiple upgrades in upgrade-info.json (Issue #20630) The implementation details for handling multiple upgrades are not fully clear.
Change upgrade-info.json structure to an array of objects (Issue #20630) The changes do not indicate that the structure of upgrade-info.json has been modified to an array format.
Allow staging multiple upgrades without manual intervention (Issue #20630)

Possibly related PRs

Suggested labels

C:x/upgrade

Suggested reviewers

  • facundomedica
  • tac0turtle

📜 Recent review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 28d03fe and 4c12e9b.

📒 Files selected for processing (5)
  • tools/cosmovisor/CHANGELOG.md (1 hunks)
  • tools/cosmovisor/args.go (1 hunks)
  • tools/cosmovisor/args_test.go (1 hunks)
  • tools/cosmovisor/cmd/cosmovisor/root.go (1 hunks)
  • tools/cosmovisor/go.mod (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (5)
  • tools/cosmovisor/CHANGELOG.md
  • tools/cosmovisor/args.go
  • tools/cosmovisor/args_test.go
  • tools/cosmovisor/cmd/cosmovisor/root.go
  • tools/cosmovisor/go.mod

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions github-actions bot added the C:Cosmovisor Issues and PR related to Cosmovisor label Sep 18, 2024
@julienrbrt julienrbrt self-assigned this Sep 18, 2024
@psiphi5 psiphi5 marked this pull request as ready for review September 19, 2024 06:23
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 12

Outside diff range and nitpick comments (7)
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (3)

16-16: Correct grammatical error in command's short description

The Short description should be "Add APP upgrade binaries to cosmovisor" instead of "Add APP upgrades binary to cosmovisor".

Apply this diff to fix the grammatical error:

- Short:        "Add APP upgrades binary to cosmovisor",
+ Short:        "Add APP upgrade binaries to cosmovisor",

23-24: Use GoDoc style comments for exported functions

The exported function AddBatchUpgrade should have a comment that starts with the function name to comply with GoDoc conventions.

Apply this diff to update the comment:

- // AddBatchUpgrade takes in multiple specified upgrades and creates a single
- // batch upgrade file out of them
+ // AddBatchUpgrade takes in multiple specified upgrades and creates a single
+ // batch upgrade file out of them.

Ensure the comment starts with the function name and ends with a period.


13-13: Add GoDoc comment for exported function NewBatchAddUpgradeCmd

The exported function NewBatchAddUpgradeCmd lacks a GoDoc comment. Adding a comment improves documentation and code readability.

Apply this diff to add the comment:

+ // NewBatchAddUpgradeCmd creates a new Cobra command for batch adding upgrades.
  func NewBatchAddUpgradeCmd() *cobra.Command {
tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (2)

Line range hint 68-70: Avoid using panic; return an error instead

Using panic for expected errors is discouraged as it can cause the application to crash unexpectedly. Instead, return the error to allow it to be handled gracefully by the caller. Also, consider providing a more descriptive error message.

Apply this diff to fix the issue:

 if err := plan.ValidateBasic(); err != nil {
-    panic(fmt.Errorf("something is wrong with cosmovisor: %w", err))
+    return fmt.Errorf("invalid upgrade plan: %w", err)
 }

88-123: Unexport functions that are not intended to be public

The functions GetConfig and AddUpgradeCmd are currently exported due to their capitalized names. If these functions are only used within this package, consider renaming them to getConfig and addUpgradeCmd to reflect that they are unexported, following Go conventions and the Uber Go Style Guide.

Apply this diff to make the changes:

 // GetConfig returns a Config using passed-in flag
-func GetConfig(cmd *cobra.Command) (*cosmovisor.Config, error) {
+func getConfig(cmd *cobra.Command) (*cosmovisor.Config, error) {
     // ...

 // AddUpgradeCmd parses input flags and adds upgrade info to manifest
-func AddUpgradeCmd(cmd *cobra.Command, args []string) error {
+func addUpgradeCmd(cmd *cobra.Command, args []string) error {
     // ...

 func NewAddUpgradeCmd() *cobra.Command {
     addUpgrade := &cobra.Command{
         // ...
-        RunE:         AddUpgradeCmd,
+        RunE:         addUpgradeCmd,
     }
     // ...
tools/cosmovisor/process.go (1)

47-62: Use descriptive variable names for better readability

The variable uInfos is not descriptive. Consider renaming it to upgradePlans or plans to improve readability and adhere to the style guide recommendations on naming.

Apply this diff to rename uInfos to upgradePlans:

-var uInfos []upgradetypes.Plan
+var upgradePlans []upgradetypes.Plan

And update all occurrences of uInfos within the function accordingly.

tools/cosmovisor/args.go (1)

112-112: Adjust the function comment for GoDoc style compliance

Per GoDoc conventions, the comment should start with the function name and be a complete sentence that describes its behavior.

Update the comment as follows:

-// UpgradeInfoBatchFilePath is the same as UpgradeInfoFilePath but with a batch suffix.
+// UpgradeInfoBatchFilePath returns the same path as UpgradeInfoFilePath but with a ".batch" suffix.
Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 81ec7ea and 3fe1cff58bad115be9558e366f2f9f29d408c89f.

Files selected for processing (7)
  • tools/cosmovisor/CHANGELOG.md (1 hunks)
  • tools/cosmovisor/args.go (1 hunks)
  • tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (4 hunks)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
  • tools/cosmovisor/cmd/cosmovisor/root.go (1 hunks)
  • tools/cosmovisor/go.mod (1 hunks)
  • tools/cosmovisor/process.go (4 hunks)
Additional context used
Path-based instructions (6)
tools/cosmovisor/cmd/cosmovisor/root.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/CHANGELOG.md (1)

Pattern **/*.md: "Assess the documentation for misspellings, grammatical errors, missing documentation and correctness"

tools/cosmovisor/process.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/args.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

Markdownlint
tools/cosmovisor/CHANGELOG.md

39-39: Expected: 1; Actual: 0; Below
Headings should be surrounded by blank lines

(MD022, blanks-around-headings)


40-40: null
Lists should be surrounded by blank lines

(MD032, blanks-around-lists)

GitHub Check: CodeQL
tools/cosmovisor/process.go

[notice] 201-201: Spawning a Go routine
Spawning a Go routine may be a possible source of non-determinism

Additional comments not posted (3)
tools/cosmovisor/cmd/cosmovisor/root.go (1)

22-22: Addition of NewBatchAddUpgradeCmd() follows conventions and is well-integrated.

The new command NewBatchAddUpgradeCmd() is consistent with the existing command structure. It adheres to the naming conventions outlined in the Uber Golang style guide, using the New prefix for constructor functions that return a command.

tools/cosmovisor/go.mod (1)

8-9: Verify the necessity of adding dependencies as direct requirements

Please confirm that github.com/cometbft/cometbft v0.38.9 and github.com/fsnotify/fsnotify v1.7.0 are directly imported in the cosmovisor codebase. If these packages are only used indirectly through other dependencies, they may not need to be listed as direct requirements in the go.mod file. Keeping only necessary direct dependencies helps maintain a cleaner dependency graph and reduces potential conflicts.

Run the following script to check for direct imports of these packages:

tools/cosmovisor/args.go (1)

113-115: LGTM!

The new method correctly adds the batch upgrade info file path functionality and follows existing code patterns.

tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (1)
tools/cosmovisor/process.go (1)

Line range hint 1-600: Minor deviations from Uber Go Style Guide.

The code follows the Uber Go Style Guide for the most part. There are a few minor deviations:

  • Some lines are longer than 120 characters. Consider breaking them into multiple lines for better readability.
  • Some error messages don't start with lowercase. Error messages should start with lowercase for consistency.
Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 3fe1cff58bad115be9558e366f2f9f29d408c89f and 7e1bf65048d5f7fb2dac8ac7a8bc849299d03b43.

Files selected for processing (4)
  • tools/cosmovisor/CHANGELOG.md (1 hunks)
  • tools/cosmovisor/args.go (5 hunks)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
  • tools/cosmovisor/process.go (3 hunks)
Files skipped from review as they are similar to previous changes (2)
  • tools/cosmovisor/CHANGELOG.md
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go
Additional context used
Path-based instructions (2)
tools/cosmovisor/args.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/process.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

Additional comments not posted (8)
tools/cosmovisor/process.go (4)

47-63: LGTM!

The loadBatchUpgradeFile function correctly loads and sorts the batch upgrade plans. The error handling looks good.


68-177: LGTM!

The BatchWatcher function correctly implements the batch upgrade process. It handles new blocks, updates the upgrade-info.json file at the appropriate heights, and reloads the upgrade plans if the batch file is modified. The error handling and logging look good.


201-202: LGTM!

The changes to the Run function to start the BatchWatcher in a separate goroutine look good. Using a context to manage the lifecycle of the BatchWatcher is a nice touch.

Also applies to: 214-215


207-207: LGTM!

Calling cancel() when a signal is received is the correct way to gracefully exit the BatchWatcher.

tools/cosmovisor/args.go (4)

41-41: LGTM!

The new constant EnvCometBftRpcEndpoint is defined correctly and follows the existing naming convention.


72-72: Looks good!

The new CometBftRpcEndpoint field is added to the Config struct correctly, with appropriate tags and a reasonable default value.


114-117: LGTM!

The new UpgradeInfoBatchFilePath function is implemented correctly and provides a clear way to derive the batch upgrade info file path.


217-221: Looks good!

The GetConfigFromEnv function and DetailString method are updated appropriately to handle the new CometBftRpcEndpoint field.

Also applies to: 559-559

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (1)
tools/cosmovisor/process.go (1)

68-177: The BatchUpgradeWatcher function logic is correct, but consider refactoring for better readability and maintainability.

The BatchUpgradeWatcher function is implemented correctly and handles the core logic of watching for upgrades and replacing the upgrade-info files. However, the function is quite long and could be refactored into smaller functions for better readability and maintainability. For example:

  • Extract the logic for replacing the upgrade-info and batch files into a separate function.
  • Consider propagating errors up to the caller instead of logging and returning.
  • Add a way to gracefully stop the watcher.
Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 2fb3b68fc5beea40a0195c4bc818c37a00a03e3e and 66c7be2760d1255f846fa3efe6137c7d580aad71.

Files selected for processing (4)
  • tools/cosmovisor/args_test.go (1 hunks)
  • tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (4 hunks)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
  • tools/cosmovisor/process.go (3 hunks)
Files skipped from review as they are similar to previous changes (1)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go
Additional context used
Path-based instructions (3)
tools/cosmovisor/args_test.go (2)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/process.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

Additional comments not posted (11)
tools/cosmovisor/cmd/cosmovisor/add_upgrade.go (6)

22-22: LGTM!

Setting the RunE field to AddUpgradeCmd is the correct way to specify the handler function for the add-upgrade command.


31-32: Refactoring the function signature is a good improvement!

Accepting parameters directly instead of relying on command-line arguments enhances the modularity and reusability of the addUpgrade function. This change aligns with best practices for writing clean and maintainable code.


36-36: LGTM!

Converting the upgrade name to lowercase, when not disabled by configuration, is a good practice to ensure consistency in naming conventions across the codebase.


Line range hint 66-82: The code segment handles upgrades at a specific height correctly.

The logic for creating an upgrade plan, validating it, and saving it to the upgrade-info.json file is implemented correctly. The code segment aligns with the best practices mentioned in the past review comment about setting appropriate file permissions for non-executable files.


88-100: Extracting the configuration retrieval logic is a good practice.

Moving the configuration retrieval logic into the getConfigFromCmd function improves code organization and reusability. The function handles errors appropriately and returns them to the caller, which aligns with best practices for error handling in Go.


102-122: The AddUpgradeCmd function is implemented correctly.

The function follows the expected logic for a command handler. It parses the input flags, retrieves the configuration, and calls the refactored addUpgrade function with the appropriate parameters. The error handling is done appropriately, and the code segment aligns with the changes made to the addUpgrade function signature.

tools/cosmovisor/process.go (3)

47-63: LGTM!

The loadBatchUpgradeFile function is implemented correctly:

  • It handles the file not found error gracefully.
  • It uses the Config for the file path to allow configurability.
  • Error messages include the file path for easier debugging.
  • It sorts the slice of upgrade plans by height to ensure correct order of processing.

95-101: Good use of polling to wait for the chain process to be ready.

Polling the RPC endpoint to wait for the chain process to be ready is a good way to ensure the watcher starts at the correct time and doesn't miss any blocks.


214-214: Starting the BatchUpgradeWatcher in a separate goroutine is a good way to ensure it runs concurrently with the main process.

The changes to the Run function to start the BatchUpgradeWatcher in a separate goroutine are implemented correctly:

  • It allows the watcher to run concurrently with the main process.
  • It passes a context to the watcher, which allows it to be cancelled when the Run function exits.
  • It starts the watcher before waiting for an upgrade, which ensures it doesn't miss any upgrades.
tools/cosmovisor/args_test.go (2)

480-480: Approve the addition of the CometBftRpcEndpoint parameter.

The addition of the CometBftRpcEndpoint parameter to the Config struct enhances the configurability of the application and aligns with the goal of improving integration with CometBFT.


480-480: Verify the handling of the CometBftRpcEndpoint parameter.

Please ensure that the CometBftRpcEndpoint parameter is properly validated and handled throughout the codebase to maintain the integrity of the application.

Verification successful

CometBftRpcEndpoint parameter is handled consistently, but could benefit from additional validation

The CometBftRpcEndpoint parameter is properly used throughout the codebase with a default value and environment variable support. However, consider implementing these improvements:

  • Add explicit URL format validation when setting the endpoint.
  • Enhance error handling when using the endpoint in HTTP requests and client creation.
  • Include documentation about security considerations for this parameter.

These suggestions aim to improve robustness and security, but the current implementation is functional.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify if the `CometBftRpcEndpoint` parameter is properly validated and handled.

# Test: Search for occurrences of `CometBftRpcEndpoint`. 
# Expect: Proper validation and handling of the parameter.
rg --type go -A 5 $'CometBftRpcEndpoint'

Length of output: 2643

@psiphi5
Copy link
Contributor Author

psiphi5 commented Sep 20, 2024

Hi @julienrbrt I've finished the implementation of this feature, would love a code review.

@psiphi5
Copy link
Contributor Author

psiphi5 commented Sep 23, 2024

ping @facundomedica, this PR is ready for review

@julienrbrt
Copy link
Member

I'll review today! Thank you

@psiphi5
Copy link
Contributor Author

psiphi5 commented Sep 24, 2024

ping @julienrbrt lmk if you have any questions about my code

Copy link
Member

@julienrbrt julienrbrt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice work! I do think we need to make it more hacky and not use cometbft websocket.
We are trying to abstract away from the consensus.

tools/cosmovisor/args.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🧹 Outside diff range and nitpick comments (3)
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1)

14-22: Consider simplifying the command usage message.

While the command structure is correct, the Use field is still quite lengthy and might be confusing for users. Consider simplifying it or moving detailed usage information to the Long description field for better clarity.

Good job on addressing the minimum arguments requirement with cobra.MinimumNArgs(1).

tools/cosmovisor/args.go (2)

72-72: LGTM: New Config field added correctly.

The CosmosGrpcEndpoint field is added with appropriate tags and a sensible default value. Consider adding a brief comment explaining the purpose of this field for better documentation.

-	CosmosGrpcEndpoint       string        `toml:"cosmos_grpc_endpoint" mapstructure:"cosmos_grpc_endpoint" default:"localhost:9090"`
+	CosmosGrpcEndpoint       string        `toml:"cosmos_grpc_endpoint" mapstructure:"cosmos_grpc_endpoint" default:"localhost:9090"` // gRPC endpoint for Cosmos SDK

114-117: LGTM: New method for batch upgrade file path added.

The UpgradeInfoBatchFilePath method is concise and follows the single responsibility principle. Consider adding a brief comment explaining the purpose of this method and its relation to the batch upgrade feature.

-// UpgradeInfoBatchFilePath is the same as UpgradeInfoFilePath but with a batch suffix.
+// UpgradeInfoBatchFilePath returns the path for the batch upgrade info file.
+// It's the same as UpgradeInfoFilePath but with a ".batch" suffix, used for managing multiple upgrades.
 func (cfg *Config) UpgradeInfoBatchFilePath() string {
 	return cfg.UpgradeInfoFilePath() + ".batch"
 }
📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 66c7be2760d1255f846fa3efe6137c7d580aad71 and ba5462f61cfbedc74b19aa25c0f55629565af45f.

📒 Files selected for processing (5)
  • tools/cosmovisor/args.go (5 hunks)
  • tools/cosmovisor/args_test.go (1 hunks)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
  • tools/cosmovisor/go.mod (1 hunks)
  • tools/cosmovisor/process.go (4 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
tools/cosmovisor/args.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/args_test.go (2)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/process.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

🔇 Additional comments (11)
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (2)

1-12: LGTM: Package and imports are appropriate.

The package declaration and import statements are correct and relevant for the functionality implemented in this file.


24-42: LGTM: Argument processing and validation look good.

The function correctly processes and validates the input arguments. Good job on sanitizing upgradeName using filepath.Base() to prevent potential path traversal vulnerabilities.

tools/cosmovisor/go.mod (4)

8-8: Approval: Direct dependency on cosmos-sdk added

The addition of github.com/cosmos/cosmos-sdk v0.50.7 as a direct dependency is appropriate. This change indicates that the Cosmovisor tool is now explicitly using features from the Cosmos SDK, which aligns with the PR objectives of enhancing upgrade management capabilities.


8-15: Summary: Dependency updates align with PR objectives

The changes to the go.mod file, particularly the addition of direct dependencies on cosmos-sdk, fsnotify, and grpc, align well with the PR objectives of enhancing the Cosmovisor tool for managing upgrades. These dependencies suggest improvements in functionality related to:

  1. Direct interaction with the Cosmos SDK
  2. File system monitoring (possibly for upgrade-related files)
  3. Enhanced gRPC communication

These changes are likely to support the new batch upgrade feature mentioned in the PR objectives, potentially improving the efficiency and reliability of the upgrade process for node operators.


15-15: Approval: Direct dependency on gRPC added

The addition of google.golang.org/grpc v1.66.2 as a direct dependency is appropriate. This change suggests that the Cosmovisor tool may be implementing or enhancing gRPC-based communication, possibly for interacting with the Cosmos SDK or other components during the upgrade process.

Please verify the usage of gRPC in the codebase:

#!/bin/bash
# Search for gRPC usage in the codebase
rg --type go 'grpc\.' tools/cosmovisor

9-9: Approval: Direct dependency on fsnotify added

The addition of github.com/fsnotify/fsnotify v1.7.0 as a direct dependency is appropriate. This change suggests that the Cosmovisor tool may be implementing file watching functionality, possibly for monitoring upgrade-related files.

Please verify the usage of this package in the codebase:

✅ Verification successful

Action Required: Re-verify fsnotify Usage with Corrected Script

Please run the following shell script to accurately search for fsnotify. usage in the tools/cosmovisor directory:


Verification Successful: fsnotify Usage Confirmed

The github.com/fsnotify/fsnotify v1.7.0 dependency is actively used in tools/cosmovisor/process.go:

  • watcher, err := fsnotify.NewWatcher()
  • if event.Op&(fsnotify.Write|fsnotify.Create) != 0 {

This confirms that adding fsnotify as a direct dependency is appropriate.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for fsnotify usage in the codebase
rg --type go 'fsnotify\.' tools/cosmovisor

Length of output: 324


Script:

# 
#!/bin/bash
# Search for fsnotify usage in the codebase
rg --type go 'fsnotify\.' tools/cosmovisor

Length of output: 194

tools/cosmovisor/args.go (3)

41-41: LGTM: New environment variable constant added correctly.

The new EnvCosmosGrpcEndpoint constant is added consistently with the existing naming convention and placed appropriately at the end of the constant block.


563-563: LGTM: CosmosGrpcEndpoint added to DetailString method.

The addition of CosmosGrpcEndpoint to the configEntries slice ensures that this new configuration is included in the detailed string representation of the Config. This change is consistent with how other fields are handled in this method.


Line range hint 1-638: Overall assessment: New CosmosGrpcEndpoint feature well-implemented.

The changes introduce a new CosmosGrpcEndpoint configuration option consistently throughout the file. The implementation follows existing patterns and conventions in the codebase. Minor suggestions have been made for improved documentation and removal of redundant code. These changes appear to support the batch upgrade feature mentioned in the PR objectives without introducing any significant issues.

tools/cosmovisor/args_test.go (2)

Line range hint 1-1000: Verify impact on other files

The addition of the CosmosGrpcEndpoint field in the Config struct might have implications for other parts of the codebase. It's important to ensure that any code that uses the Config struct is updated to handle this new field appropriately.

To check for potential impacts, you can run the following script:

#!/bin/bash
# Description: Search for uses of the Config struct in other files

# Test: Look for files that import and use the Config struct
rg --type go -A 5 'import \(.*"cosmossdk.io/tools/cosmovisor".*\)' | rg 'cosmovisor\.Config'

# If any results are found, these files might need to be updated to handle the new CosmosGrpcEndpoint field

Line range hint 483-1000: Update test cases to include CosmosGrpcEndpoint

With the addition of the CosmosGrpcEndpoint field in the Config struct, the test cases in TestGetConfigFromEnv should be updated to include this new field. This ensures that the new configuration option is properly tested.

To verify if the test cases need updating, you can run the following script:

tools/cosmovisor/args.go Outdated Show resolved Hide resolved
tools/cosmovisor/args_test.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Outdated Show resolved Hide resolved
tools/cosmovisor/process.go Show resolved Hide resolved
@psiphi5 psiphi5 requested a review from julienrbrt September 28, 2024 15:01
Copy link
Member

@julienrbrt julienrbrt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

amazing work! thank you, just missing some handling when passing a file I think

if err := json.Unmarshal(b, &batch); err != nil {
return fmt.Errorf("error unmarshalling upgrade file %s: %w", upgradeFile, err)
}
return copyFile(upgradeFile, cfg.UpgradeInfoBatchFilePath())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks incomplete. Where is batch used? And why aren't we calling add upgrade there as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no different than processUpgradeList where we end with writing out the batch file to the home directory (

err = os.WriteFile(cfg.UpgradeInfoBatchFilePath(), batchData, 0o600)
). Here we're already getting a batch file from the user so we just validate it and copy it over. We then watch this file during cosmovisor run.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

processUpgradeList calls addUpgrade, which place the binaries at the right place.
This doesn't.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right, my bad. upgradetypes.Plan doesn't really have field we can use for the binary path, what do you think of changing the file format for --upgrade-file to be a CSV with rows name,path/to/binary,height? I'd imagine it'd be easier for users to make than a JSON file anyway.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Outside diff range and nitpick comments (2)
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (2)

71-121: Use a constant for file permissions

The function correctly uses 0o600 permissions when writing the batch upgrade file, which is good for security. However, it's a good practice to define such values as constants for better maintainability and readability.

Consider adding a constant at the package level:

const batchUpgradeFilePerms = 0o600

Then use this constant when writing the file:

err = os.WriteFile(cfg.UpgradeInfoBatchFilePath(), batchData, batchUpgradeFilePerms)

This makes the code more self-documenting and easier to maintain if the permissions need to be changed in the future.


123-142: Add a comment explaining the CSV format

The function correctly processes a CSV batch upgrade file, but it would be helpful to add a comment explaining the expected format of the CSV file for better documentation.

Consider adding a comment at the beginning of the function:

// processUpgradeFile takes in a CSV batch upgrade file, parses it and calls processUpgradeList
// The CSV file should be headerless with each row containing three fields:
// 1. upgrade-name
// 2. path-to-exec
// 3. upgrade-height
func processUpgradeFile(cfg *cosmovisor.Config, upgradeFile string) error {
    // ... existing code ...
}

This comment will help users understand the expected format of the CSV file without having to look at the command description.

📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 1d21f55 and 250eae9.

📒 Files selected for processing (1)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (4)
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (4)

17-48: LGTM! Consider adding validation for required flags.

The NewBatchAddUpgradeCmd function is well-implemented, following Cobra best practices. The command description, usage, and example are clear and informative. The mutual exclusivity of flags is correctly enforced.

Consider adding a check to ensure that at least one of the flags (--upgrade-file or --upgrade-list) is provided. You can use cmd.MarkFlagRequired("upgrade-file") and handle the case in the RunE function.


72-122: LGTM! Consider optimizing file operations and improving error messages.

The processUpgradeList function effectively handles the core logic of batch upgrades. The implementation is solid, with good error handling and secure practices like using filepath.Base for upgradeName.

Consider these minor improvements:

  1. Use os.Create with os.O_WRONLY|os.O_CREATE|os.O_TRUNC flags instead of os.WriteFile for better control over file permissions.
  2. Enhance error messages to include more context, especially for file operations.

Example for point 1:

file, err := os.OpenFile(cfg.UpgradeInfoBatchFilePath(), os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0600)
if err != nil {
    return fmt.Errorf("error creating batch file: %w", err)
}
defer file.Close()
_, err = file.Write(batchData)
if err != nil {
    return fmt.Errorf("error writing to batch file: %w", err)
}

These changes will provide more control over file operations and improve error reporting.


124-143: LGTM! Consider enhancing error handling for CSV parsing.

The processUpgradeFile function effectively handles CSV file parsing and delegates the processing to processUpgradeList. The implementation is solid, with good use of the csv package and proper error handling.

Consider enhancing the error handling for CSV parsing to provide more context about invalid records:

 records, err := r.ReadAll()
 if err != nil {
-    return fmt.Errorf("error parsing upgrade CSV file %s: %w", upgradeFile, err)
+    if perr, ok := err.(*csv.ParseError); ok {
+        return fmt.Errorf("error parsing upgrade CSV file %s at line %d, column %d: %w", upgradeFile, perr.Line, perr.Column, perr.Err)
+    }
+    return fmt.Errorf("error parsing upgrade CSV file %s: %w", upgradeFile, err)
 }

This change will provide more detailed information about parsing errors, making it easier for users to identify and correct issues in their CSV files.


1-143: Add unit tests for the new batch upgrade functionality

The implementation of batch upgrade functionality is well-structured and meets the objectives outlined in the PR summary. However, to ensure reliability and prevent regressions, it's crucial to add comprehensive unit tests.

Consider adding unit tests for the following scenarios:

  1. Parsing and validating upgrade lists
  2. Processing CSV upgrade files
  3. Creating and combining individual upgrade files
  4. Error handling for various input scenarios

Would you like assistance in generating a basic structure for these unit tests?

📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 250eae9 and 7e8112d.

📒 Files selected for processing (1)
  • tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go (1)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.

Copy link
Member

@julienrbrt julienrbrt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm! one nit

tools/cosmovisor/cmd/cosmovisor/batch_upgrade.go Outdated Show resolved Hide resolved
@julienrbrt
Copy link
Member

@psiphi5
Copy link
Contributor Author

psiphi5 commented Oct 2, 2024

Could you fix the failing unit tests: https://github.com/cosmos/cosmos-sdk/actions/runs/11145765964/job/30976132402?pr=21790 ?

ah it was due to the new grpc config; should be fixed now

@psiphi5
Copy link
Contributor Author

psiphi5 commented Oct 3, 2024

hey @julienrbrt do we need an additional review before this can be merged?

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: .coderabbit.yml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between ebc78ba and 28d03fe.

📒 Files selected for processing (2)
  • tools/cosmovisor/args_test.go (1 hunks)
  • tools/cosmovisor/go.mod (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • tools/cosmovisor/go.mod
🧰 Additional context used
📓 Path-based instructions (1)
tools/cosmovisor/args_test.go (2)

Pattern **/*.go: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.


Pattern **/*_test.go: "Assess the unit test code assessing sufficient code coverage for the changes associated in the pull request"

🔇 Additional comments (1)
tools/cosmovisor/args_test.go (1)

480-480: Update test cases and usage for new GRPCAddress field

The addition of the GRPCAddress field to the Config struct is a significant change that likely requires updates to existing test cases and possibly other parts of the code that use this struct.

Please ensure that:

  1. Existing test cases are updated to include the new GRPCAddress field.
  2. New test cases are added to specifically test the behavior related to this field.
  3. Any other parts of the codebase that create or use Config instances are updated accordingly.

To help verify these changes, you can run the following script:

This will help identify areas of the code that might need updates due to the new field.

tools/cosmovisor/args_test.go Outdated Show resolved Hide resolved
@julienrbrt
Copy link
Member

hey @julienrbrt do we need an additional review before this can be merged?

Yes, someone else from the team will review and then we can merge.

@psiphi5
Copy link
Contributor Author

psiphi5 commented Oct 8, 2024

hey @julienrbrt do we need an additional review before this can be merged?

Yes, someone else from the team will review and then we can merge.

hi @julienrbrt do you think we can get a second review from the team soon?

@julienrbrt
Copy link
Member

Yes. No worries it will get reviewed.

@julienrbrt julienrbrt enabled auto-merge October 9, 2024 14:07
@julienrbrt julienrbrt disabled auto-merge October 9, 2024 14:22
@julienrbrt
Copy link
Member

Could you fix the linting and the cosmovisor tests?

@psiphi5
Copy link
Contributor Author

psiphi5 commented Oct 10, 2024

Could you fix the linting and the cosmovisor tests?

should be fixed now

@julienrbrt julienrbrt added this pull request to the merge queue Oct 10, 2024
Merged via the queue into cosmos:main with commit 5b53cca Oct 10, 2024
70 of 72 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C:Cosmovisor Issues and PR related to Cosmovisor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: Have multiple upgrades in upgrade-info.json and support it in Cosmovisor
3 participants