Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ocient): convert pyocient GIS data types to geoJSON #23814

Merged

Conversation

jwilliams-ocient
Copy link
Contributor

SUMMARY

This PR addresses a bug that sometimes causes the Ocient engine to emit invalid polygon and linestring objects. See commit messages for more info.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@jwilliams-ocient jwilliams-ocient marked this pull request as draft April 25, 2023 15:24
@codecov
Copy link

codecov bot commented Apr 25, 2023

Codecov Report

Merging #23814 (f0d7a5d) into master (357745f) will decrease coverage by 0.02%.
The diff coverage is 25.00%.

❗ Current head f0d7a5d differs from pull request most recent head bb12cad. Consider uploading reports for the commit bb12cad to get more accurate results

@@            Coverage Diff             @@
##           master   #23814      +/-   ##
==========================================
- Coverage   68.31%   68.29%   -0.02%     
==========================================
  Files        1951     1951              
  Lines       75385    75401      +16     
  Branches     8185     8185              
==========================================
+ Hits        51496    51498       +2     
- Misses      21786    21800      +14     
  Partials     2103     2103              
Flag Coverage Δ
hive 53.17% <25.00%> (-0.02%) ⬇️
mysql 78.93% <25.00%> (-0.03%) ⬇️
postgres 79.00% <25.00%> (-0.03%) ⬇️
presto 53.10% <25.00%> (-0.02%) ⬇️
python 82.80% <25.00%> (-0.04%) ⬇️
sqlite 77.53% <25.00%> (-0.03%) ⬇️
unit 53.09% <25.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/db_engine_specs/ocient.py 51.26% <25.00%> (-6.03%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

The previous implementation did not emit a Deck.gl-compatible GIS
types correctly in all cases. Now, the Ocient engine will always
return geoJSON for geospatial types, including types that are
upcasted and placed into type-safe GIS containers (i.e. a Polygon[]
geo collection that contains point and linestring objects).
@jwilliams-ocient jwilliams-ocient force-pushed the user/jwilliams/ocient-geospatial-cleanup branch from 736b627 to 287bba1 Compare April 26, 2023 17:25
@pull-request-size pull-request-size bot added size/L and removed size/M labels Apr 26, 2023
@jwilliams-ocient jwilliams-ocient changed the title fix[ocient]: convert pyocient GIS data types to geoJSON or Polyline fix[ocient]: convert pyocient GIS data types to geoJSON Apr 26, 2023
@jwilliams-ocient jwilliams-ocient changed the title fix[ocient]: convert pyocient GIS data types to geoJSON fix(ocient): convert pyocient GIS data types to geoJSON Apr 26, 2023
@jwilliams-ocient jwilliams-ocient marked this pull request as ready for review April 26, 2023 19:29
@jwilliams-ocient
Copy link
Contributor Author

Closing and reopening to kick off new CI run

@jwilliams-ocient
Copy link
Contributor Author

There's a CSV test broken on the master branch. Once the fix has been merged I will close/reopen

Copy link
Member

@rusackas rusackas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but would love to get some additional eyes on this, so pinging @villebro @john-bodley @michael-s-molina for extra 👀

@rusackas rusackas requested a review from villebro May 11, 2023 01:42
Copy link
Member

@villebro villebro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks for the added tests!

@jwilliams-ocient
Copy link
Contributor Author

Alright, I think this one is ready to merge. Thanks @villebro , @rusackas !

@rusackas rusackas merged commit f0a26fd into apache:master May 17, 2023
@rusackas
Copy link
Member

Thank YOU!

@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 3.0.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L 🚢 3.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants