Revert "Allow bigint to be used as time column in Presto" #7670

john-bodley · 2019-06-07T05:16:12Z

Per #7218 (comment) including a BIGINT column type as a time column in Presto is problematic as it could represent either a Unix time in seconds or milliseconds.

The current correct way to handle Unix time columns is to set the python_date_format as either epoch or epoch_ms (depending on the encoding) which results in queries comprising the correct fidelity:

SELECT COUNT(*) AS "count"
FROM "superset"."logs"
WHERE "dttm" >= 1553644800
  AND "dttm" <= 1554249600
ORDER BY "count" DESC
LIMIT 10000;

Note #7656 plans to auto-detect the encoding of temporal columns.

to: @betodealmeida @mistercrunch

* Fix race condition when fetching results in SQL Lab * Fix lint (cherry picked from commit ca6a73b)

in some cases, people want a time filter only on filter box, without specifying dimensions (filters), this allows that (cherry picked from commit e39b169)

* Fix logo / branding files SVG was busted in the branding folder. Files were partly corrupt. This takes back the original logo, and offers a with-Apache and without-Apache option. SVG renders nicely. The pngs are also bigger. * add ASF license

* Remove ASF license from github templates (PR and issues) It shows as people open issues and PRs and it's confusing... * add to rat-excludes

…6964) * direct display for pre-selected tab * update parents * add AnchorLink component * add unit tests

* [SQL Lab] fix table metadata loading spinner The loading spinner used to be close to the new table name that was just added, and recently flickers and disappears early. This puts the spinner where it's expected to be. * Use class * Addressing comments * adapt CSS * add missing license

* chore: add bot config files and update issue templates * fix: exclude SIP

* fix: make SIPs exempt from stale bots * update pr template

* left align, close to button it's related to * text-muted, so it's a bit more subtle * fix required func props, where no func it actually passed

* [WiP] debugging and improving 'Time Table' closes #6948 * Lint * Remove passing down props from CollectionControl * Declarative passthrough of props * remove console.error

* Add all database access permission to 'Alpha' role * Grant all database access to Alpha

This reverts commit 9b4f5ad.

THEN is also a keyword in SQL

* fix: time series table * fix: add default value for label * fix: use prop values if defined * fix: revert CollectionControl changes (cherry picked from commit 5dab983)

* [load_examples] download data at runtime When running `superset load_examples` to load example data sets, Superset used to load from the local package. This created a few issues notably around licensing (what are these datasets licensed as?) and around package size. For now, I moved the data sets here: https://github.com/apache-superset/examples-data Altered the logic to download the data from where it is stored. * flakes

* feat: bump superset-ui versions to 0.11 * fix: update version number again * fix: package-lock.json * test: disable large number test * fix: revert cors

* Adding permission for can_only_access_owned_queries * Fixing lint adding typing to variable * Adding test for queryview and enabling /queryview/api/read * Fixing issues with python typing

* fix: SqlaColumn.type overflow on mysql Hitting a new error when loading examples around a datatype that exceeds the 32 chars limit on SqlaColumn.type Type includes the 'COLLATE utf8mb4_general_ci' suffix which is too verbose and not needed in that context. * fix tests

…ith single value setting (#7494)

* Adding codecov targets and thresholds * Changing diff coverage rules to 1% threshold * Changing codecov threshold to 0 and removing ignore

* Fix colors in ellipsis * Change method name to align with convention * Fix js lint

* Add Filter on DatabaseView that filters DBs Based on Role Access * Update with mistercrunch's feedback

Closes #6224

This reverts commit 1ae000a.

mistercrunch

I agree here, seen it both ways so much I couldn't tell which one is more common...

* Add new escape characters to OnPasteSelect * Add Unit Tests to onPasteSelect_spec * Update new line to tab escape character * Fix indentation in 2 cases

* Apply dashboardFilter to dashboards * added missing comma

)

* Refactor db_engine_specs into package * Rename bigquery class and add epoch funcs * Fix flake8 errors * Dynamically load all engine specs * Fix linting errors and unit tests * Implement Snowflake epoch time funcs * Implement Teradata epoch time func * Fix presto datasource query and remove unused import * Fix broken datasource query * Add mypy ignore for false positive * Add missing license files * Make create_time_grains_tuple public * Fix flake8 quote * Fix incorrect licence header

michellethomas and others added 30 commits April 8, 2019 15:08

Fix race condition when fetching results in SQL Lab (#7198) (#7242)

52473c5

* Fix race condition when fetching results in SQL Lab * Fix lint (cherry picked from commit ca6a73b)

[filter_box] allow empty filters list (#7220) (#7244)

5ef2712

in some cases, people want a time filter only on filter box, without specifying dimensions (filters), this allows that (cherry picked from commit e39b169)

Tweaks to word cloud controls (#7212)

442cfc6

Fix logo / branding files (#7105)

5192270

* Fix logo / branding files SVG was busted in the branding folder. Files were partly corrupt. This takes back the original logo, and offers a with-Apache and without-Apache option. SVG renders nicely. The pngs are also bigger. * add ASF license

fix: Remove ASF license from github templates (PR and issues) (#7217)

149b31d

* Remove ASF license from github templates (PR and issues) It shows as people open issues and PRs and it's confusing... * add to rat-excludes

[cache] Render label when cached (#7164)

9216639

add parent ids into Dashboard layout metadata (#6945)

139f299

feat(dashboard): direct link to single chart/tab/header in dashboard (#…

c50e6bc

…6964) * direct display for pre-selected tab * update parents * add AnchorLink component * add unit tests

update migration Revision for #6945 (#7258)

1104aca

Run 'npm audit fix' to address vulnerabilities (#7263)

b225507

chore: add bot config files and update issue templates (#7249)

6c38cb1

* chore: add bot config files and update issue templates * fix: exclude SIP

Deprecate npm run backend-sync and related logic (#7211)

0c3e46f

fix: make SIPs exempt from stale bots (#7278)

f9db605

* fix: make SIPs exempt from stale bots * update pr template

fix: make the bot more descriptive (#7279)

80070d0

Improve code smell, use py3k super() call (#7100)

763db8f

Minor improvements to <Hotkeys /> component (#7261)

14647fc

* left align, close to button it's related to * text-muted, so it's a bit more subtle * fix required func props, where no func it actually passed

bugfix: improve 'Time Table' (#6959)

9b4f5ad

* [WiP] debugging and improving 'Time Table' closes #6948 * Lint * Remove passing down props from CollectionControl * Declarative passthrough of props * remove console.error

Bump nvd3 dep and 'npm audit fix' (#7292)

b1ebfbc

Add all database access permission to 'Alpha' role (#7271)

9dfa80d

* Add all database access permission to 'Alpha' role * Grant all database access to Alpha

Add details to release instructions (#7296)

d253c5c

Revert "bugfix: improve 'Time Table' (#6959)" (#7304)

154f6ab

This reverts commit 9b4f5ad.

[slice] Fix, remove duplicate label_columns property (#7113)

1ffd3b1

Update AceEditorWrapper.jsx (#7315)

7aa6c4e

THEN is also a keyword in SQL

fix: time series table (#7302) (#7312)

9341995

* fix: time series table * fix: add default value for label * fix: use prop values if defined * fix: revert CollectionControl changes (cherry picked from commit 5dab983)

feat: bump superset-ui versions to 0.11 (#7294)

ab8f4af

* feat: bump superset-ui versions to 0.11 * fix: update version number again * fix: package-lock.json * test: disable large number test * fix: revert cors

Remove LICENSE entry around dataset (#7318)

38dd33e

Adding permission for can_only_access_owned_queries (#7234)

51068f0

* Adding permission for can_only_access_owned_queries * Fixing lint adding typing to variable * Adding test for queryview and enabling /queryview/api/read * Fixing issues with python typing

fix: typos in translations placeholders (#7161)

07d9dbd

mistercrunch and others added 11 commits June 3, 2019 09:52

make filter_values() macro to properly accept value from filter box w…

78c1674

…ith single value setting (#7494)

[fix] SQL parsing of table names (#7490)

45b41aa

Adding codecov targets and thresholds (#7433)

6d1f6e9

* Adding codecov targets and thresholds * Changing diff coverage rules to 1% threshold * Changing codecov threshold to 0 and removing ignore

Fix colors in ellipsis (#7632)

ddd7f8f

* Fix colors in ellipsis * Change method name to align with convention * Fix js lint

[dashboard] click tab anchor link (#7640)

f99ae1a

Add Filter on DatabaseView that filters DBs Based on Role Access (#7618)

5470d10

* Add Filter on DatabaseView that filters DBs Based on Role Access * Update with mistercrunch's feedback

[Profile] Fix react gravitar size prop type warning (#7647)

25da4c7

[SQL Lab] CTAS now uses the right async configuration (#7175)

1cd0e7f

Closes #6224

Revert "Boxplot should not require a datetime column (#5096)" (#7655)

d62c37b

This reverts commit 1ae000a.

[docs] Fix, deprecate fabmanager cli (#7660)

de9f61e

pull-request-size bot added the size/XS label Jun 7, 2019

mistercrunch approved these changes Jun 7, 2019

View reviewed changes

1AB9502 and others added 10 commits June 6, 2019 22:24

Add new escape characters to OnPasteSelect (#7638)

271612c

* Add new escape characters to OnPasteSelect * Add Unit Tests to onPasteSelect_spec * Update new line to tab escape character * Fix indentation in 2 cases

Apply DashboardFilter to the dashboards in SliceModeView (#7475)

2793bdd

* Apply dashboardFilter to dashboards * added missing comma

docs: Add Tenable to the List of Users (#7610)

883a02a

[SQL Lab] Show warning when user used up localStorage (#7572)

39d67cb

[dashboard] pass dashboard filters to share chart url in dropdown (#7642

f3091c5

)

[SQL Lab] Clarify SQL Lab query and display limits (#7641)

f7812a3

Add epoch funcs for BigQuery (#7567)

38abf4e

Remove duplicate paragraph in celery tasks documentation (#7392)

f3e5805

Revert "Allow bigint to be used as time column in Presto" #7670

b1cb742

john-bodley force-pushed the revert-7218-VIZ-414 branch from c1bbe29 to b1cb742 Compare June 8, 2019 22:30

john-bodley closed this Jun 8, 2019

john-bodley pushed a commit to john-bodley/superset that referenced this pull request Jun 8, 2019

Revert "Allow bigint to be used as time column in Presto" apache#7670

375bc76

john-bodley mentioned this pull request Jun 8, 2019

Revert "Allow bigint to be used as time column in Presto" #7670 #7678

Merged

john-bodley added a commit that referenced this pull request Jun 10, 2019

Revert "Allow bigint to be used as time column in Presto" #7670 (#7678)

49390b3

cccs-rc pushed a commit to CybercentreCanada/superset that referenced this pull request Mar 6, 2024

Revert "Allow bigint to be used as time column in Presto" apache#7670

493ee5d

mistercrunch deleted the revert-7218-VIZ-414 branch March 25, 2024 21:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revert "Allow bigint to be used as time column in Presto" #7670

Revert "Allow bigint to be used as time column in Presto" #7670

john-bodley commented Jun 7, 2019 •

edited

Loading

mistercrunch left a comment

Revert "Allow bigint to be used as time column in Presto" #7670

Revert "Allow bigint to be used as time column in Presto" #7670

Conversation

john-bodley commented Jun 7, 2019 • edited Loading

mistercrunch left a comment

Choose a reason for hiding this comment

john-bodley commented Jun 7, 2019 •

edited

Loading