Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor ClusterMetadata defaults and validation #4385

Conversation

vytautas-karpavicius
Copy link
Contributor

What changed?
Refactored ClusterMetadata configuration validation and filling in defaults.

Why?
To follow structure already established with other config sections. They have validate function that checks for invalid values and fillDefaults which sets default values if they are not provided.

ClusterMetadata had this logic scattered in a few other places. Collected them and put into separate cluster.go file.

I'm planning to add more validation/defaults to this section later, that is why I decided to refactor it first.

How did you test it?
Added new unit tests for all validation logic. Instead of panics, normal errors are returned that are easier to tests against.

I also suggest returning them as multierr as this would allow listing all the problems at once, instead of fix -> start server -> discover another error -> fix cycle.

Potential risks

Release notes

Documentation Changes

@coveralls
Copy link

coveralls commented Aug 18, 2021

Pull Request Test Coverage Report for Build 8535af03-8e99-4f98-88aa-cca91d8bc4c7

  • 90 of 108 (83.33%) changed or added relevant lines in 6 files are covered.
  • 19 unchanged lines in 6 files lost coverage.
  • Overall coverage increased (+0.08%) to 56.489%

Changes Missing Coverage Covered Lines Changed/Added Lines %
common/cluster/metadata.go 4 6 66.67%
common/config/config.go 10 13 76.92%
tools/cli/domainUtils.go 0 6 0.0%
cmd/server/cadence/server.go 0 7 0.0%
Files with Coverage Reduction New Missed Lines %
common/config/config.go 2 83.72%
common/task/weightedRoundRobinTaskScheduler.go 2 88.6%
common/cache/lru.go 3 90.73%
common/task/fifoTaskScheduler.go 4 83.51%
service/history/execution/mutable_state_builder.go 4 69.8%
service/history/task/transfer_standby_task_executor.go 4 90.4%
Totals Coverage Status
Change from base Build e88b4784-5874-47ab-9197-306b055c39ac: 0.08%
Covered Lines: 78803
Relevant Lines: 139501

💛 - Coveralls

@vytautas-karpavicius vytautas-karpavicius marked this pull request as ready for review August 18, 2021 16:57
@vytautas-karpavicius vytautas-karpavicius requested review from longquanzheng and a team August 18, 2021 16:58
CurrentClusterName string `yaml:"currentClusterName"`
// ClusterInformation contains information for each cluster within the replication group
// Key is the clusterName
ClusterInformation map[string]ClusterInformation `yaml:"clusterInformation"`
Copy link
Contributor

@longquanzheng longquanzheng Aug 18, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to "ReplicationGroup"?

e.g. ReplicationGroup map[string]ClusterInformation

Or maybe "ClusterGroup" as it may not do any replication in one cluster setup.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, having the same name for both map name and item within that map does not seem right.

ClusterGroup seems reasonable. Or maybe just simply Clusters?
I think it would also make sense to rename ClusterMetadata to (maybe) ClusterGroupMetadata as it contains config for multiple clusters not just single one.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having "Group" explicitly is more friendly as the "s' is quite hard to read/recognize.

Yes, I love the idea of ClusterGroupMetadata

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed:

  • ClusterInformation to ClusterGroup
  • ClusterMetadata to ClusterGroupMetadata
    This also means introducing new fields in yaml deprecating old ones. Otherwise having them different could lead to some confusion.

ClusterMetadata *ClusterMetadata `yaml:"clusterMetadata"`
// ClusterGroupMetadata is the config containing all valid clusters and active cluster
ClusterGroupMetadata *ClusterGroupMetadata `yaml:"clusterGroupMetadata"`
// ClusterMetadata is deprecated. Please use ClusterGroupMetadata
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good tip! Updated.

Copy link
Contributor

@longquanzheng longquanzheng left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a comment about the deprecation notation. Otherwise LGTM.

@vytautas-karpavicius vytautas-karpavicius merged commit c6ef3c9 into cadence-workflow:master Aug 23, 2021
@vytautas-karpavicius vytautas-karpavicius deleted the cluster-config-validate branch August 23, 2021 09:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants