Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Make Go SDK parse response by result type #1021

Merged
merged 3 commits into from
Mar 23, 2022
Merged

fix: Make Go SDK parse response by result type #1021

merged 3 commits into from
Mar 23, 2022

Conversation

jeremytchang
Copy link
Collaborator

@jeremytchang jeremytchang commented Mar 16, 2022

Changes:

  • Go SDK Parse response based of type defined in go sdk models, the result type. This fixes methods returning string type. This handles all method result types, string, struct, interface{}.
  • Added additional integration tests around queries, looks, and
    dashboards. This fills out the basic integration test requirements.
  • Added unit tests
  • Return error body in addition to error status code for more descriptive response errors.

Ideally we want the content type header to be the source of truth when parsing the response. However, currently, go sdk model generation doesn't support union type for results, so we cannot parse according to Content Type Header. For now, parse based off result type.

Take for example RunQuery. It returns 4 content type headers text, application/json, image/png, and image/jpeg. Result types should be string, map[string]interface{}, byte[]. Right now result type is only string here. Passing in a string type to the decoder means we cannot parse to map[string]interface{} or byte[].
In this case, it is an antipattern to force the Go SDK caller to cast the string result to what they need, but it at least works.

For the future:
We need to update sdk model generation to use interface{} (any) for union types, and then use Content Type Header to parse response accordingly. This will break downstream dependencies. Issue tracking this.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@jeremytchang
Copy link
Collaborator Author

jeremytchang commented Mar 16, 2022

@jkaster The TS SDK exhibits the antipattern and returns string for text/json/image content types. https://github.com/looker-open-source/sdk-codegen/blob/main/packages/sdk/src/4.0/methods.ts#L8253

@jkaster
Copy link
Contributor

jkaster commented Mar 17, 2022

@jkaster The TS SDK exhibits the antipattern and returns string for text/json/image content types. https://github.com/looker-open-source/sdk-codegen/blob/main/packages/sdk/src/4.0/methods.ts#L825

Not quite. See the existing tests that verify the image payloads for various SDK methods. The string in that case is a byte array rather than a UTF-8 decoded string.

@jkaster
Copy link
Contributor

jkaster commented Mar 17, 2022

For more strictly typed languages, you may need to indicate either byte array or string as the return type. For kotlin and swift, I hacked together a solution we've never cleaned up that implements binary payloads via the "streaming" parallel functions for the SDKs.

@github-actions

This comment has been minimized.

panoskoug
panoskoug previously approved these changes Mar 22, 2022
return fmt.Errorf("response error. status=%s. error parsing error body", res.Status)
}

return fmt.Errorf("response error. status=%s. error=%s", res.Status, string(b))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am wondering if this could break any code that relies on the string.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this could, though consumers reallllllly shouldn't be matching on strings. I'll be implementing better error context in the future so consumers can match on the status code and/or specific errors. #795 (comment)

The possibility of breakage is not great, but while we try to fully support the Go SDK, consider it in beta and we will provide best effort to not break downstream dependencies.

@jkaster
Copy link
Contributor

jkaster commented Mar 22, 2022

This is a major improvement. Can you add a test for content_thumbnail that verifies the PNG and SVG payload? See the TS tests for PNG bytestream verification code.

@github-actions

This comment has been minimized.

@jeremytchang
Copy link
Collaborator Author

jeremytchang commented Mar 23, 2022

@jkaster Added png/svg integration test. Made use of golang standard library for checking mime type https://pkg.go.dev/net/http#DetectContentType

@github-actions

This comment has been minimized.

Currently go sdk model generation doesn't support union type for responses like
RunQuery. So we cannot parse according to Content Type Header. For now, parse
based off model type.

We need to update sdk model generation to use interface{}
for union types, and then use Content Type Header to parse response accordingly.
This will break downstream dependencies.

- Added additional integration tests around queries looks and
  dashboards.
- Parse response based of type defined in go sdk models, the result type
- Added unit tests
- Return error body in addition to error status code.
@github-actions
Copy link
Contributor

Go Tests

    7 files      7 suites   5m 17s ⏱️
  42 tests   42 ✔️ 0 💤 0 ❌
111 runs  111 ✔️ 0 💤 0 ❌

Results for commit 7e07cfa.

Copy link
Contributor

@jkaster jkaster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great. Thanks for adding the PNG/SVG tests. Nice that Go has a MIME type detection function

@jeremytchang
Copy link
Collaborator Author

@panoskoug Thanks for going out of your way for the approval!

@jeremytchang jeremytchang merged commit c1675ab into main Mar 23, 2022
@jeremytchang jeremytchang deleted the jc/go_sdk branch March 23, 2022 17:59
@github-actions

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants