Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(bigquery)!: Inline type-annotated ARRAY literals #4671

Merged
merged 1 commit into from
Jan 29, 2025

Conversation

VaggelisD
Copy link
Collaborator

@VaggelisD VaggelisD commented Jan 29, 2025

Fixes #4670

#3751 canonicalized BigQuery's inlined constructors into casts which may not be correct for certain roundtrips, as shown in the issue:

  • Safe
bq> SELECT ARRAY<INT64>[1, 2, 3], CAST([1, 2, 3] AS ARRAY<INT64>);
f0_	f1_
"[1,2,3]"	"[1,2,3]"

  • Not safe (type mismatch)
bq> SELECT CAST([1, 2, 3] AS ARRAY<FLOAT64>);
Casting between arrays with incompatible element types is not supported: Invalid cast from ARRAY<INT64> to ARRAY<FLOAT64> at [1:13]


This PR fixes this particular scenario by preserving the inlined ARRAY type annotation & literal value for BQ's roundtrip.

Docs

BQ ARRAY Literals

@VaggelisD VaggelisD changed the title fix(bigquery): Inline type-annotated ARRAY literals fix(bigquery)!: Inline type-annotated ARRAY literals Jan 29, 2025
@georgesittas
Copy link
Collaborator

Does this only concern arrays? What about structs?

@VaggelisD
Copy link
Collaborator Author

Yep, I tried STRUCT related queries in a similar fashion but I don't think the same typing issue arises, e.g this is fine:

bq> SELECT STRUCT<FLOAT64>(1), CAST(STRUCT(1) AS STRUCT<FLOAT64>);
f0_	f1_
"{
  ""f0_"": {
    ""_field_1"": ""1.0""
  }
}"	"{
  ""f1_"": {
    ""_field_1"": ""1.0""
  }
}"

Would you have other (possibly offending) scenarios in mind?

@georgesittas
Copy link
Collaborator

Cool, no this is what I had in mind.

@georgesittas georgesittas merged commit c45f174 into main Jan 29, 2025
7 checks passed
@georgesittas georgesittas deleted the vaggelisd/bq_arrays branch January 29, 2025 15:32
@sean-rose
Copy link

Yep, I tried STRUCT related queries in a similar fashion but I don't think the same typing issue arises, e.g this is fine:

While casting STRUCTs like that doesn't result in an outright BigQuery error, IMO it'd be preferable to also preserve BigQuery's typed struct literal syntax since it's more idiomatic, succinct, and avoids BigQuery potentially doing more work to first construct theSTRUCT and then convert it (I'm speculating on this last point).

@georgesittas
Copy link
Collaborator

georgesittas commented Jan 29, 2025

Makes sense. @VaggelisD can we try to preserve these constructors when roundtripping BigQuery SQL?

@VaggelisD
Copy link
Collaborator Author

Yep, will give it a shot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BigQuery type-annotated array literals don't survive roundtrip through SQLGlot
3 participants