Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug to query header search attributes correctly in visibility #6163

Merged
merged 5 commits into from
Jul 10, 2024

Conversation

shijiesheng
Copy link
Contributor

What changed?

Why?

  • search attributes cannot contain . and this assumption exists in multiple places in the code
  • sqlparser library does not allow - in the column name in where clause.

How did you test it?

unit test

Potential risks

This feature only exists in pre-releases and internally it's not rolled out. Additionally, this feature is also behind a feature flag so it's very safe.

Release notes

Documentation Changes

Copy link

codecov bot commented Jul 9, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 72.76%. Comparing base (1e2ef3d) to head (78e5d53).
Report is 1 commits behind head on master.

Additional details and impacted files
Files Coverage Δ
common/definition/indexedKeys.go 75.00% <ø> (ø)
...ervice/history/task/transfer_task_executor_base.go 81.33% <100.00%> (+0.25%) ⬆️
tools/cli/admin_cluster_commands.go 27.65% <ø> (ø)

... and 9 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e2ef3d...78e5d53. Read the comment docs.

@@ -50,6 +51,8 @@ const (
defaultDomainName = "defaultDomainName"
)

var nonAlphanumericRegex = regexp.MustCompile(`[^a-zA-Z0-9]+`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's better to not spread these visibility store constraints across the codebase. can you move sanitizedHeaderKey to common folder or somewhere in visibility folders?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sense. Now it's in the common folder

@@ -36,7 +36,7 @@ import (

// An indirection for the prompt function so that it can be mocked in the unit tests
var promptFn = prompt
var validSearchAttributeKey = regexp.MustCompile(`^[a-zA-Z][a-zA-Z_.-0-9]*$`)
var validSearchAttributeKey = regexp.MustCompile(`^[a-zA-Z][a-zA-Z_0-9]*$`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. this regex should live close to other search attribute handling code in a central place.

@@ -30,6 +30,7 @@ import (
"github.com/uber/cadence/common"
"github.com/uber/cadence/common/backoff"
"github.com/uber/cadence/common/definition"
"github.com/uber/cadence/common/elasticsearch/validator"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

elasticseach is a plugin. rest of the codebase shouldn't directly depend on this package. same search attribute constrationts apply to Pinot and other future visibility stores as well. so let's move this validator to common/visibility or somewhere generic like that

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@shijiesheng shijiesheng enabled auto-merge (squash) July 9, 2024 23:12
@shijiesheng shijiesheng merged commit e758b78 into master Jul 10, 2024
18 of 19 checks passed
@shijiesheng shijiesheng deleted the fix-index branch July 10, 2024 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants