Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON Type #1455

Merged
merged 8 commits into from
Jan 17, 2025
Merged

Add JSON Type #1455

merged 8 commits into from
Jan 17, 2025

Conversation

SpencerTorres
Copy link
Member

@SpencerTorres SpencerTorres commented Dec 20, 2024

Note: There are some server-side bugs that need to be fixed/merged, including a protocol update. However, this PR can still be reviewed as is.

Summary

Implement JSON column type. Fully resolves #1430.

Usage

The user can use JSON in 3 ways:

  1. By using the new clickhouse.JSON type to assemble JSON data by path strings.
  2. By using a struct or map. The struct can be any nested structure, but the map must be map[string]any.
  3. By using strings. In this mode the user handles their own json.(Un)Marshal and the strings get sent to the server as-is.

The JSON column also supports the clickhouse.Dynamic type for handling a path with multiple data types. Structs can also include a tag to specify type, such as json:"my_field" chType:"Array(String)".

See examples folder for sample code.

Checklist

Delete items not relevant to your PR:

  • Unit and integration tests covering the common scenarios were added

@SpencerTorres
Copy link
Member Author

Updated with fixes and suggestions from Variant PR. There are still multiple server side bugs that will prevent certain features from working, but common scenarios work such as:

  • Read/Write JSON using key/value pairs
  • Write JSON using strings
  • Read/Write JSON using structs (except stdlib sql)

Once #1454 is merged then we can see if any tests are broken in the test matrix. Tests are passing on latest Go/ClickHouse versions.

Base automatically changed from dynamic_type to main January 13, 2025 15:25
@SpencerTorres
Copy link
Member Author

Looks like the tests are passing against main.

Note that JSON string tests are skipped while we wait for: ClickHouse/ClickHouse#73924

@jkaflik ready for review

Copy link
Contributor

@jkaflik jkaflik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Lets have it merged and iterate in next steps.

@SpencerTorres SpencerTorres merged commit 577ea49 into main Jan 17, 2025
12 checks passed
@SpencerTorres SpencerTorres deleted the json_type branch January 17, 2025 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for new JSON type
2 participants