Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Go modules #1270

Closed
wants to merge 2 commits into from
Closed

Update Go modules #1270

wants to merge 2 commits into from

Conversation

SuperQ
Copy link
Member

@SuperQ SuperQ commented May 5, 2023

  • Update Go modules for new protobuf version.
  • Update Go modules in sub projects.
  • Enable dependabot for sub projects.

@SuperQ SuperQ requested review from beorn7 and bwplotka May 5, 2023 06:31
* Update Go modules for new protobuf version.
* Update Go modules in sub projects.
* Enable dependabot for sub projects.
* Update golangci-lint action.
* Update CircleCI orb.

Signed-off-by: SuperQ <superq@gmail.com>
@SuperQ
Copy link
Member Author

SuperQ commented May 8, 2023

This is going to require a bunch of test refactoring.

@beorn7
Copy link
Member

beorn7 commented May 9, 2023

Please ping me again when the tests are passing. :)

Signed-off-by: SuperQ <superq@gmail.com>
@beorn7
Copy link
Member

beorn7 commented May 28, 2023

General idea looks OK. But there are even more code locations where we have utilized the text formatted protobuf, which still fail.

I guess you don't even need to change from angle bracket to curly braces with the new comparison helper.

Also, relevant lint warnings and now a merge conflict (which, I'm afraid, has to do with even more tests).

@SuperQ
Copy link
Member Author

SuperQ commented May 29, 2023

Yea, I'm not sure about what to do to fix a few of these, which is why I did only a partial update so far.

  • Tests for scrape output that test text proto.
  • Inline text proto errors in the text scrape format.

Ideas:

  • Write a stableprototext package.
  • Take the scrape output and marshal it back and compare.

Copy link
Member

@bwplotka bwplotka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this! Generally looking good.

Indeed we could write small function to convert new output to old, but we might as well refactor as you do now in one go (thanks for amazing work!). Please make sure to keep Examples in, they are important to have example type (for documentation reasons).

Tests for scrape output that test text proto.
Inline text proto errors in the text scrape format.

Can you point us to problematic parts? Would love to take closer look on what's best approach there 🤗

@@ -106,7 +106,7 @@ func (v *InfoVec) MustCurryWith(labels prometheus.Labels) *InfoVec {
return vec
}

func ExampleMetricVec() {
func TestExampleMetricVec(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it meant to be Example for a reason (it's nicely highlighted on Go docs) 🤔

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, didn't know that.

@SuperQ
Copy link
Member Author

SuperQ commented May 30, 2023

@bwplotka When we print some errors in the registry output, we embed part of the protobuf as output.

For example this section of registry.go.

We check for errors like:

An error has occurred while serving metrics:

collected metric "name" { label:<name:"constname" value:"\377" > label:<name:"labelname" value:"different_val" > counter:<value:42 > } has a label named "constname" whose value is not utf8: "\xff"

@bwplotka
Copy link
Member

Ok, but that is again - expecting in unit test a different format, so we just need to switch to new text format in expected output? 🤔

@SuperQ
Copy link
Member Author

SuperQ commented May 30, 2023

@bwplotka No, it's not just that. The new protobuf prototext package explicitly randomizes the whitespace in the output. Making every test slightly different.

https://github.com/protocolbuffers/protobuf-go/blob/v1.30.0/internal/encoding/text/encode.go#L226-L228

@SuperQ
Copy link
Member Author

SuperQ commented May 30, 2023

Let me try calling detrand.Disable() explicitly to see if that helps.

@beorn7
Copy link
Member

beorn7 commented Jun 1, 2023

For the Go example use case, the non-deterministic text output is really annoying. We cannot even implement our own comparison function (like one that ignores whitespace).

@bwplotka
Copy link
Member

bwplotka commented Jun 1, 2023

So there is some truth that tests comparing text encoding is a little brittle and blocks serialization innovations (?). If we follow suggestions I think we have to get rid of checking exact errors and parse text format or proto format to structure and compare semantically those (e.g. here)

@bwplotka
Copy link
Member

bwplotka commented Jun 1, 2023

But examples, yea.. but it's also not text encoding, it's our string that just happen to use this to show what metrics we talk about also on errors. Wonder how hard would be to write our our strangifier or unrandomizing it by strings.Replace(str, ' ', ' ') 🤔

@bwplotka
Copy link
Member

bwplotka commented Jun 1, 2023

Will try to help here tomorrow one day @SuperQ to try to solve some cases here in this PR

@SuperQ
Copy link
Member Author

SuperQ commented Aug 12, 2023

Closing in favor of #1323

@SuperQ SuperQ closed this Aug 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants