Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(dashboard): native filters do not preserve bigint #24413

Conversation

justinpark
Copy link
Member

SUMMARY

There is an overflow issue when searching for bigint records in the native filters, i.e., the value passed to the backend is a rounded form of the specified UI value.
This hotfix handles the bigint value in a string form to preserve the value in JSON

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

After:

Screenshot 2023-06-15 at 4 26 17 PM

Before:

Screenshot 2023-06-15 at 4 34 39 PM

TESTING INSTRUCTIONS

  • Specs
  • Go to dashboard and add a number field in a native filter
  • type any big number and check the request payload

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

extraFilters.push({
col: column,
op: '>=',
val: Number(search),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@justinpark do you know what additional value using Number(...) provides? This is how the backend handles the filtering logic which casts strings to numbers and thus maybe it's ok for the backend to handle both cases, i.e., we could remove lines 54–60.

We could include a comment of the form,

// for numeric columns the backend casts the search string to the appropriate type—preventing JavaScript overflow. Additionally we use the fairly crude(and insufficient)  >= operator as a proxy for detecting whether the search string is contained within the column given there is no trivial way of dynamically casting the the column type.

@codecov
Copy link

codecov bot commented Jun 16, 2023

Codecov Report

Merging #24413 (4e0b3eb) into master (892de8a) will decrease coverage by 0.05%.
The diff coverage is 85.71%.

❗ Current head 4e0b3eb differs from pull request most recent head 260d421. Consider uploading reports for the commit 260d421 to get more accurate results

@@            Coverage Diff             @@
##           master   #24413      +/-   ##
==========================================
- Coverage   68.96%   68.91%   -0.05%     
==========================================
  Files        1904     1904              
  Lines       74092    73922     -170     
  Branches     8120     8120              
==========================================
- Hits        51100    50946     -154     
+ Misses      20880    20865      -15     
+ Partials     2112     2111       -1     
Flag Coverage Δ
javascript 55.65% <73.68%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ackages/superset-ui-core/src/utils/featureFlags.ts 100.00% <ø> (ø)
superset-frontend/src/SqlLab/fixtures.ts 100.00% <ø> (ø)
superset-frontend/src/pages/DatasetList/index.tsx 57.76% <ø> (ø)
...rset-frontend/src/profile/components/Favorites.tsx 80.00% <ø> (ø)
superset/charts/api.py 87.07% <ø> (ø)
superset/config.py 91.97% <ø> (ø)
superset/connectors/sqla/models.py 90.37% <ø> (-0.07%) ⬇️
superset/dashboards/api.py 92.58% <ø> (ø)
superset/dashboards/schemas.py 99.02% <ø> (-0.01%) ⬇️
superset/datasets/api.py 87.95% <ø> (ø)
... and 12 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@john-bodley
Copy link
Member

#24418 is an alternative formulation which also addresses searching/filtering of numerical values.

@justinpark justinpark closed this Jun 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants