Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sqla): avoid unnecessary groupby in samples request #18579

Merged
merged 1 commit into from
Feb 4, 2022

Conversation

villebro
Copy link
Member

@villebro villebro commented Feb 4, 2022

SUMMARY

When requesting samples from a dataset, a GROUP BY was incorrectly added to the query due to the metrics parameter in the QueryObject being set to an empty list. This adds unnecessary computational expense to the query and doesn't reflect the raw data correctly by hiding potential duplicates in the underlying data. This PR fixes the bug, adds assertions to existing tests to ensure the sample query is executed without a GROUP BY clause and adds instructions to the API spec on what the difference is between a null vs empty array for the metrics parameter.

TESTING INSTRUCTIONS

  1. Go to explore and open the Network tab in the Chrome debugger
  2. Click on "Samples"
  3. Notice that the query in the chart data request previously included a GROUP BY clause.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@villebro villebro changed the title fix(sqla): avoid unnecessary groupby for when no metrics fix(sqla): avoid unnecessary groupby in samples request Feb 4, 2022
@codecov
Copy link

codecov bot commented Feb 4, 2022

Codecov Report

Merging #18579 (36eceb0) into master (8b0634c) will decrease coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18579      +/-   ##
==========================================
- Coverage   66.29%   66.29%   -0.01%     
==========================================
  Files        1594     1594              
  Lines       62623    62623              
  Branches     6312     6312              
==========================================
- Hits        41518    41516       -2     
- Misses      19456    19458       +2     
  Partials     1649     1649              
Flag Coverage Δ
hive 52.07% <100.00%> (-0.07%) ⬇️
mysql 81.24% <100.00%> (-0.01%) ⬇️
postgres 81.29% <100.00%> (-0.01%) ⬇️
presto 51.91% <100.00%> (-0.07%) ⬇️
python 81.72% <100.00%> (-0.01%) ⬇️
sqlite 80.99% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/charts/schemas.py 99.34% <ø> (ø)
superset/common/query_actions.py 93.05% <100.00%> (ø)
superset/connectors/sqla/models.py 88.29% <0.00%> (-0.23%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8b0634c...36eceb0. Read the comment docs.

Copy link
Member

@amitmiran137 amitmiran137 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ! nice to feel the value of chart data separated test suite!

@villebro
Copy link
Member Author

villebro commented Feb 4, 2022

LGTM ! nice to feel the value of chart data separated test suite!

Agreed, the chart data test suite is a great tool to help harden the poorly tested sqla connector 👍

@villebro villebro merged commit 55cd7fb into apache:master Feb 4, 2022
shcoderAlex pushed a commit to casual-precision/superset that referenced this pull request Feb 7, 2022
ofekisr pushed a commit to nielsen-oss/superset that referenced this pull request Feb 8, 2022
bwang221 pushed a commit to casual-precision/superset that referenced this pull request Feb 10, 2022
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.5.0 labels Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels preset-io size/S 🚢 1.5.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants