Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[select-star] Adding optional schema to view #6051

Merged

Conversation

john-bodley
Copy link
Member

@john-bodley john-bodley commented Oct 6, 2018

This PR updates a couple of aspects of the select-star logic including:

  • Adding an optional schema to the /select_star route.
  • Making the /select_star route return a JSON response as opposed to HTML.
  • Ensured that the /select_star route includes a partition (if appropriate).
  • Ensured that the /table route includes a partition (if appropriate)†.
  • Added a where_latest_partition for Presto.
  • Changed the default Hive where_latest_partition behavior to return the first partition if the table is multi-partitioned. Previous it would return False is the table was partitioned but contained more than one partition key.

† Note this improves the performance of the SQL Lab preview by including a partition predicate even though there is additional overhead of finding the latest partition. At Airbnb for a large daily partitioned table with several years of data this reduced the query time from ~ 75 seconds to ~ 3 seconds. This also helps to ensure that the data preview is associated with the partition (guaranteed for the single partitioned case) shown in the UI.

screen shot 2018-10-05 at 10 25 13 pm

to: @graceguo-supercat @kristw @michellethomas @mistercrunch @timifasubaa @williaster

@john-bodley john-bodley force-pushed the john-bodley-fix-select-star-view branch from d4a7915 to 99c7819 Compare October 6, 2018 19:26
@codecov-io
Copy link

codecov-io commented Oct 6, 2018

Codecov Report

Merging #6051 into master will increase coverage by 0.1%.
The diff coverage is 46.66%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master    #6051     +/-   ##
=========================================
+ Coverage   77.74%   77.84%   +0.1%     
=========================================
  Files          46       46             
  Lines        9400     9412     +12     
=========================================
+ Hits         7308     7327     +19     
+ Misses       2092     2085      -7
Impacted Files Coverage Δ
superset/views/core.py 73.92% <100%> (+0.18%) ⬆️
superset/db_engine_specs.py 55.61% <27.27%> (+0.79%) ⬆️
superset/connectors/sqla/models.py 81.2% <0%> (+0.75%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9dcf8e1...fc924e3. Read the comment docs.

@@ -1 +0,0 @@
{{ content |safe }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh god that was awful, thanks for getting rid of this.

# table is not partitioned
return False
for c in columns:
if c['name'] == col_name:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: c.get() is generally safer as it won't ever raise KeyError, though in this case the schema should be safe

@mistercrunch
Copy link
Member

LGTM

@john-bodley john-bodley force-pushed the john-bodley-fix-select-star-view branch from 99c7819 to fc924e3 Compare October 8, 2018 16:51
@john-bodley john-bodley merged commit 1ee08fc into apache:master Oct 8, 2018
@john-bodley john-bodley deleted the john-bodley-fix-select-star-view branch October 8, 2018 17:32
john-bodley added a commit to john-bodley/superset that referenced this pull request Oct 8, 2018
betodealmeida pushed a commit to lyft/incubator-superset that referenced this pull request Oct 12, 2018
bipinsoniguavus pushed a commit to ThalesGroup/incubator-superset that referenced this pull request Dec 26, 2018
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.28.0 labels Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 0.28.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants