fix: Native filter dynamic numeric search #24418
Conversation
| @@ -1173,7 +1173,13 @@ def handle_single_value(value: Optional[FilterValue]) -> Optional[FilterValue]: | |||
| if isinstance(value, str): | |||
| value = value.strip("\t\n") | |||
|
|
|||
| if target_generic_type == utils.GenericDataType.NUMERIC: | |||
| if ( | |||
There was a problem hiding this comment.
I'm going to turn a blind eye to the fact that this function is copy-and-pasted in multiple places. Furthermore it's rather scary there's zero unit tests for the filter_values_handler method.
john-bodley
force-pushed
the
john-bodley--filter-select-like
branch
from
66b970d
to
c4602ae
Compare
john-bodley
changed the title
Codecov Report
@@ Coverage Diff @@
## master #24418 +/- ##
==========================================
- Coverage 68.91% 68.91% -0.01%
==========================================
Files 1904 1904
Lines 73920 73920
Branches 8119 8118 -1
==========================================
- Hits 50944 50941 -3
- Misses 20865 20868 +3
Partials 2111 2111
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
john-bodley
force-pushed
the
john-bodley--filter-select-like
branch
from
c4602ae
to
06c1168
Compare
john-bodley
force-pushed
the
john-bodley--filter-select-like
branch
from
06c1168
to
729cc0e
Compare
| @@ -120,6 +120,8 @@ describe('Select buildQuery', () => { | |||
| }); | |||
| expect(queryContext.queries.length).toEqual(1); | |||
| const [query] = queryContext.queries; | |||
| expect(query.filters).toEqual([{ col: 'my_col', op: '>=', val: 123 }]); | |||
| expect(query.filters).toEqual([ | |||
| { col: 'my_col', op: 'ILIKE', val: '%123%' }, | |||
There was a problem hiding this comment.
btw i think LIKE can be enough for since it matches only numbers, right?
There was a problem hiding this comment.
nvm. I don't find any performance benefit of using LIKE over ILIKE.
I'm good with it
(cherry picked from commit 652bf64)
mistercrunch
added
🏷️ bot
SUMMARY
This PR provides a fix for #14710 in addition to being an alternative formulation for #24413. Additionally it provides
LIKEandILIKEoperator support for SIMPLE ad-hoc filters irrespective of the column type, i.e., now all operators are valid.This change came about give how #14710 implemented dynamic (backend) filtering of numerical values for native filters.
Hypothetically lets assume there was an
idcolumn which comprised of the set of integers from 0 to 100,000 and a user searched for100, then:{col: 'id', op: '>=', val: 100}.100substring resulting in only 10 matches: 100, 1,001, 1,002, 1,003, 1,004, 1,005, 1,006, 1,007, 1,008, 1,009.The right way to see if a number is contained within another number is to treat the numbers as a string and provide a filter of the form
{col: 'id', op: 'ILIKE', val: '%100%'}. This also remedies the BIGINT issue where in #24413 the numerical values were treated as a string.The ad-hoc filters include the
ILIKEandLIKEoperator which work exclusively withSTRINGtypes. This PR simply provides "support" for using both these operators by casting the column (if necessary) to aSTRING, i.e., it means that theILIKEandLIKEoperator work for all SIMPLE filters irrespective of the column type, which can then be leveraged by the native filter search functionality.BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
BEFORE
AFTER
Dynamic casting
TESTING INSTRUCTIONS
CI.
ADDITIONAL INFORMATION