Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: escape bind-like strings in virtual table query #17111

Merged
merged 1 commit into from
Oct 14, 2021

Conversation

villebro
Copy link
Member

@villebro villebro commented Oct 14, 2021

SUMMARY

When executing a chart query with a virtual table that contains a literal value with a colon suffixed by text characters, SQLAlchemy interprets these as being bind parameters when wrapped in a text element and compiled. This PR ensures that these are escaped using the same pattern that SQLAlchemy uses to detect bind parameters (the regex is available as a private variable in TextClause). Relevant test cases are added (notice that the Postgres cast to ::TIMESTAMP is not escaped, as SQLAlchemy doesn't consider it to be a bind parameter). To my knowledge there is no cleaner way to do this, and I also found several threads where the author of SQLAlchemy instructed people to solve this issue by mechanically escaping the colon in the statement, which is what this proposal in practice does.

Closes #17098

When executing the query SELECT ':00 :abc' as abc, 123 as num in SQL Lab, the query executes succesfully:
image

BEFORE

When used as a virtual table in a chart query, SQLAlchemy interprets there to be two bind parameters present, namely 00 and abc, resulting in the following error:
image

AFTER

After escaping the bind-like strings, the query renders as expected.
image

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@villebro villebro force-pushed the villebro/escape-sqla branch 2 times, most recently from bcc066b to 7a8f182 Compare October 14, 2021 07:50
@villebro villebro force-pushed the villebro/escape-sqla branch from 7a8f182 to 8f9c411 Compare October 14, 2021 08:57
@codecov
Copy link

codecov bot commented Oct 14, 2021

Codecov Report

Merging #17111 (8f9c411) into master (b9ff85d) will decrease coverage by 0.21%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #17111      +/-   ##
==========================================
- Coverage   76.88%   76.67%   -0.22%     
==========================================
  Files        1031     1031              
  Lines       55197    55209      +12     
  Branches     7506     7506              
==========================================
- Hits        42436    42329     -107     
- Misses      12509    12628     +119     
  Partials      252      252              
Flag Coverage Δ
hive ?
mysql 81.93% <100.00%> (+<0.01%) ⬆️
postgres 81.94% <100.00%> (+<0.01%) ⬆️
presto ?
python 82.03% <100.00%> (-0.41%) ⬇️
sqlite 81.61% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/connectors/sqla/models.py 85.61% <100.00%> (-1.64%) ⬇️
superset/utils/core.py 89.97% <100.00%> (+0.01%) ⬆️
superset/db_engines/hive.py 0.00% <0.00%> (-85.19%) ⬇️
superset/db_engine_specs/hive.py 69.76% <0.00%> (-17.06%) ⬇️
superset/db_engine_specs/presto.py 83.47% <0.00%> (-6.49%) ⬇️
superset/views/database/mixins.py 81.03% <0.00%> (-1.73%) ⬇️
superset/models/core.py 89.26% <0.00%> (-0.74%) ⬇️
superset/db_engine_specs/base.py 88.17% <0.00%> (-0.39%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b9ff85d...8f9c411. Read the comment docs.

@villebro villebro requested a review from dpgaspar October 14, 2021 09:33
Copy link
Member

@dpgaspar dpgaspar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunate edge case

@villebro villebro merged commit 434b576 into apache:master Oct 14, 2021
@villebro villebro deleted the villebro/escape-sqla branch October 14, 2021 16:03
@villebro villebro added the v1.4 label Oct 15, 2021
eschutho pushed a commit to preset-io/superset that referenced this pull request Oct 27, 2021
opus-42 pushed a commit to opus-42/incubator-superset that referenced this pull request Nov 14, 2021
@mistercrunch mistercrunch added 🍒 1.4.0 🍒 1.4.1 🍒 1.4.2 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset-io size/M v1.4 🍒 1.4.0 🍒 1.4.1 🍒 1.4.2 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SQLAlchemy error on exploring virtual dataset
4 participants