Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xdsmatcher: add evaluation framework for xDS matching API #511

Merged
merged 5 commits into from
Jan 20, 2022

Conversation

snowp
Copy link
Contributor

@snowp snowp commented Nov 10, 2021

No description provided.

Signed-off-by: Snow Pettersen <snowp@lyft.com>
@snowp snowp marked this pull request as ready for review November 10, 2021 19:06
@alecholmez alecholmez self-requested a review November 10, 2021 21:19
@alecholmez alecholmez self-assigned this Nov 10, 2021
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Signed-off-by: Snow Pettersen <snowp@lyft.com>
Signed-off-by: Snow Pettersen <snowp@lyft.com>
@snowp
Copy link
Contributor Author

snowp commented Dec 13, 2021

@alecholmez wanna give this one a look? LMK how I can help moving this forward

@snowp
Copy link
Contributor Author

snowp commented Dec 13, 2021

Maybe @howardjohn if you're interested in this bit as well, I'd appreciate the eyes :)

@alecholmez
Copy link
Contributor

@alecholmez wanna give this one a look? LMK how I can help moving this forward

Yup looking this over, sorry got a bit wrapped up

Copy link
Contributor

@alecholmez alecholmez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think overall this looks nice. The API is pretty friendly so long as you understand the original matching API. Overall nothing major from my end. Maybe just some more comments here and there and beefing up the docs for godoc but other than that it looks good!

xdsmatcher/README.md Outdated Show resolved Hide resolved
// report this back up.
if result.NeedMoreData {
m.logger.log("matcher tree requires more data")
return result, nil
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is a failure does it warrant an error returned?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as i understand it a result with "needs more data" is valid, might just need a small comment adjustment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is not an error, it's the complex data type understanding that this data may change in the future, so it's unable to conclusively say that the data doesn't match.

An example here is how this is used for matching HTTP streams in Envoy: if you run into a matcher that tries to match against trailers before trailers are seen, we respond "NeedMoreData" so that the caller knows to call this again when trailers are made available.

xdsmatcher/pkg/matcher/matcher.go Outdated Show resolved Hide resolved
}

// Internal logger that helps debugging by providing indention during matching to provide some visual guidance to the current nesting level.
// TODO(snowp): Make this optional, not ideal if this ends up being used outside of testing scenarios.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we pull this out before it gets merged? Or at least make sure its optional?

@@ -0,0 +1,33 @@
package types
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe some comments in this file above the important types/interfaces might be useful? Like the Matcher interface for example. Just so it shows up nicely in godoc for people consuming the framework

return fooInput{}, nil
}

// Implement types.DataInput for the input type.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just so I understand correctly, is the InputType the registered type URL in the init() function? I'd assume so since we are attempting a cast below that never fails

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make things super generic, the entire tree acts against a specific InputType (simple case here is just a string, but can be a complex data structure). Once the tree is built, you pass a value of type InputType to it, which is then passed to all DataInputs (which tell us how to extract a specific value out of the complex data type).

So the InputType is never registered, but you must be defining DataInputs that all support the specific type that you provide to the matcher on evaluation time

r, _ := m.Match(data)
if r.MatchResult != nil && r.MatchResult.Action != nil {
// resulting action
action, ok := r.MatchResult.Action.(action)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw that Actions are interfaces, what kind of things could we do on this resulting variable? Or would that be for the consumer of this API to decide?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah they are opaque objects that are defined using extension points in the config. You'd expect to check the type of these by casting to the type you might care about, then you can do whatever you want using the resulting casted type (which may have more interesting fields/functions defined)

Signed-off-by: Snow Pettersen <snowp@lyft.com>
@snowp
Copy link
Contributor Author

snowp commented Jan 12, 2022

Updated, ptal @alecholmez

@snowp
Copy link
Contributor Author

snowp commented Jan 19, 2022

ping @alecholmez

Copy link
Contributor

@alecholmez alecholmez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think from me reading this it looks good to go. I might have more questions once I start using it but looks ready to merge on my end. The tests definitely help

@snowp snowp merged commit b7b3397 into envoyproxy:main Jan 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants