Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add strict checks to reference client to validate a server's connect error and end stream JSON #790

Merged
merged 3 commits into from
Feb 14, 2024

Conversation

jhump
Copy link
Member

@jhump jhump commented Feb 13, 2024

This makes the checks significantly more strict than the base connect-go implementation, in order to report a number of possible issues in server JSON responses:

  1. This will detect duplicate keys in JSON objects/maps. Without an explicit check, "encoding/json" ignores duplicate keys, letting later values for the same key overwrite earlier ones. Sadly, there's no configuration to set this uniformly, so it had to be done by overriding the UnmarshalJSON behavior of each type to explicitly check. 😦
  2. This ensures that all keys use the correct case. Sadly, even with explicit field names in struct tags, "encoding/json" is still lenient and uses a case-insensitive match for field names. 😢
  3. This double-checks the JSON type of each value. There are a few ways in which "encoding/json" sometimes allows undesired types -- mainly with the use of a JSON null for map, slice, and pointer fields. So this performs an extra check of the actual types in the raw JSON.
  4. This makes sure that the keys and values in the end-stream "metadata" map are valid HTTP header names and values.
  5. This also makes sure that if a "debug" key is present in an error detail, that the message it describes actually matches the message described in the binary data of the "value" key.

@jhump jhump force-pushed the jh/connect-checks branch from 4700764 to 2fc3ad9 Compare February 13, 2024 17:29
@jhump jhump force-pushed the jh/connect-checks branch from 2fc3ad9 to b872a4d Compare February 13, 2024 18:35
@jhump jhump requested a review from smaye81 February 13, 2024 18:37
@jhump
Copy link
Member Author

jhump commented Feb 13, 2024

@smaye81, this one's pretty big. Sorry about that. Though the test cases are a pretty big chunk of the lines changed. But there's also a non-trivial amount of logic to closely scrutinize the JSON data and complain about it.

Comment on lines 429 to 444
var endStream connectEndStream
if err := json.Unmarshal(endStreamJSON, &endStream); err != nil {
printer.Printf("connect end stream JSON is malformed: %v", err)
return
}
// Extra checks, since encoding/json above is lenient.
if _, err := checkNoDuplicateKeys("", json.NewDecoder(bytes.NewReader(endStreamJSON))); err != nil {
printer.Printf("connect end stream JSON: %v", err)
return
}
var asAny map[string]any
if err := json.Unmarshal(endStreamJSON, &asAny); err != nil {
// note: since above unmarshal step succeeded, this should never fail
printer.Printf("connect end stream JSON is malformed: %v", err)
return
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth moving out this logic into its own function? I noticed that examineConnectError and examineConnectErrorDetail perform very similar logic (albeit with some minor variations). All 3 seem to do a series of validation steps against a json.RawMessage and then use an internal.Printer to report an error. It may not even be worth it, but just an observation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting point. I looked and was able to consolidate this into an examineJSON helper function. Just pushed a change with that.

@jhump jhump merged commit 208a0b5 into main Feb 14, 2024
4 checks passed
@jhump jhump deleted the jh/connect-checks branch February 14, 2024 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants