Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

expression/json: add builtin function JSON_SEARCH #8704

Merged
merged 12 commits into from
Apr 18, 2019

Conversation

pingyu
Copy link
Contributor

@pingyu pingyu commented Dec 17, 2018

What problem does this PR solve?

Add json builtin function JSON_SEARCH described in #7546

What is changed and how it works?

Add struct builtinJSONSearchSig, 2 functions for walking and utility for string quoting to BinaryJSON, and 3 utility function for json PathExpression

Check List

Tests

  • Unit test: YES
  • Integration test: YES
  • Manual test (add detailed scripts or steps below)

mysql>
mysql> SET @j = '["abc", [{"k": "10"}, "def"], {"x":"abc"}, {"y":"bcd"}]';
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'one', 'abc');
+-------------------------------+
| JSON_SEARCH(@j, 'one', 'abc') |
+-------------------------------+
| "$[0]" |
+-------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', 'abc');
+-------------------------------+
| JSON_SEARCH(@j, 'all', 'abc') |
+-------------------------------+
| ["$[0]", "$[2].x"] |
+-------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', 'ghi');
+-------------------------------+
| JSON_SEARCH(@j, 'all', 'ghi') |
+-------------------------------+
| NULL |
+-------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '10');
+------------------------------+
| JSON_SEARCH(@j, 'all', '10') |
+------------------------------+
| "$[1][0].k" |
+------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '10', NULL, '$');
+-----------------------------------------+
| JSON_SEARCH(@j, 'all', '10', NULL, '$') |
+-----------------------------------------+
| "$[1][0].k" |
+-----------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '10', NULL, '$[]');
+--------------------------------------------+
| JSON_SEARCH(@j, 'all', '10', NULL, '$[
]') |
+--------------------------------------------+
| "$[1][0].k" |
+--------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '10', NULL, '$.k');
+---------------------------------------------+
| JSON_SEARCH(@j, 'all', '10', NULL, '$
.k') |
+---------------------------------------------+
| "$[1][0].k" |
+---------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '10', NULL, '$[][0].k');
+-------------------------------------------------+
| JSON_SEARCH(@j, 'all', '10', NULL, '$[
][0].k') |
+-------------------------------------------------+
| "$[1][0].k" |
+-------------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '10', NULL, '$[1]');
+--------------------------------------------+
| JSON_SEARCH(@j, 'all', '10', NULL, '$[1]') |
+--------------------------------------------+
| "$[1][0].k" |
+--------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '10', NULL, '$[1][0]');
+-----------------------------------------------+
| JSON_SEARCH(@j, 'all', '10', NULL, '$[1][0]') |
+-----------------------------------------------+
| "$[1][0].k" |
+-----------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', 'abc', NULL, '$[2]');
+---------------------------------------------+
| JSON_SEARCH(@j, 'all', 'abc', NULL, '$[2]') |
+---------------------------------------------+
| "$[2].x" |
+---------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '%a%');
+-------------------------------+
| JSON_SEARCH(@j, 'all', '%a%') |
+-------------------------------+
| ["$[0]", "$[2].x"] |
+-------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '%b%');
+-------------------------------+
| JSON_SEARCH(@j, 'all', '%b%') |
+-------------------------------+
| ["$[0]", "$[2].x", "$[3].y"] |
+-------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '%b%', NULL, '$[0]');
+---------------------------------------------+
| JSON_SEARCH(@j, 'all', '%b%', NULL, '$[0]') |
+---------------------------------------------+
| "$[0]" |
+---------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '%b%', NULL, '$[2]');
+---------------------------------------------+
| JSON_SEARCH(@j, 'all', '%b%', NULL, '$[2]') |
+---------------------------------------------+
| "$[2].x" |
+---------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '%b%', NULL, '$[1]');
+---------------------------------------------+
| JSON_SEARCH(@j, 'all', '%b%', NULL, '$[1]') |
+---------------------------------------------+
| NULL |
+---------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '%b%', '', '$[1]');
+-------------------------------------------+
| JSON_SEARCH(@j, 'all', '%b%', '', '$[1]') |
+-------------------------------------------+
| NULL |
+-------------------------------------------+
1 row in set (0.00 sec)

mysql> SELECT JSON_SEARCH(@j, 'all', '%b%', '', '$[3]');
+-------------------------------------------+
| JSON_SEARCH(@j, 'all', '%b%', '', '$[3]') |
+-------------------------------------------+
| "$[3].y" |
+-------------------------------------------+
1 row in set (0.00 sec)

mysql>
mysql>
mysql> set @j3 = '{"\"hello世界\"": "world", "a": [1, "2", {"aa": "bb"}, 4.0, {"aa": "cc"}], "b": true, "c": ["d"]}';
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT JSON_SEARCH(@j3, 'all', 'world');
+----------------------------------+
| JSON_SEARCH(@j3, 'all', 'world') |
+----------------------------------+
| "$."\"hello世界\""" |
+----------------------------------+
1 row in set (0.00 sec)

mysql>

Code changes

  • Has exported function/method change
    BinaryJSON.Walk, PathExpression.String

Side effects

  • NO

Related changes

  • Need to update the documentation
    JSON_SEARCH supported

This change is Reviewable

@shenli shenli added contribution This PR is from a community contributor. component/expression labels Dec 17, 2018
@pingyu pingyu changed the title Expression json search expression: add builtin function JSON_SEARCH Dec 17, 2018
@pingyu pingyu changed the title expression: add builtin function JSON_SEARCH expression/json: add builtin function JSON_SEARCH Dec 17, 2018
@zz-jason
Copy link
Member

@XuHuaiyu PTAL

@pingyu
Copy link
Contributor Author

pingyu commented Dec 24, 2018

@XuHuaiyu PTAL, thanks!

@pingyu pingyu force-pushed the expression_json_search branch from f125510 to 187a132 Compare January 5, 2019 12:56
@zz-jason
Copy link
Member

zz-jason commented Jan 5, 2019

@pingyu CI is failed, could you fix it in your spare time?

@zz-jason zz-jason requested review from XuHuaiyu and qw4990 January 5, 2019 13:57
@pingyu pingyu force-pushed the expression_json_search branch from 187a132 to f05d889 Compare January 5, 2019 15:22
expression/builtin_json.go Outdated Show resolved Hide resolved
@codecov-io
Copy link

codecov-io commented Jan 5, 2019

Codecov Report

Merging #8704 into master will decrease coverage by 0.4301%.
The diff coverage is 82.5%.

@@               Coverage Diff                @@
##             master      #8704        +/-   ##
================================================
- Coverage   77.9673%   77.5371%   -0.4302%     
================================================
  Files           407        404         -3     
  Lines         82718      81971       -747     
================================================
- Hits          64493      63558       -935     
- Misses        13452      13686       +234     
+ Partials       4773       4727        -46

@pingyu
Copy link
Contributor Author

pingyu commented Jan 6, 2019

@zz-jason CI was fixed. PTAL. Thanks.

@qw4990
Copy link
Contributor

qw4990 commented Jan 24, 2019

Nice work! @pingyu

But I think variable resultSet in builtinJSONSearchSig. evalJSON used to store paths have been searched can be removed.
It's more graceful to let BinaryJSON.Walk guarantee all paths should be searched only once.

expression/builtin_json.go Outdated Show resolved Hide resolved
@pingyu pingyu force-pushed the expression_json_search branch from 87bb029 to d1673ca Compare January 26, 2019 17:05
@qw4990
Copy link
Contributor

qw4990 commented Jan 31, 2019

LGTM

@qw4990 qw4990 added the status/LGT1 Indicates that a PR has LGTM 1. label Jan 31, 2019
expression/builtin_json.go Outdated Show resolved Hide resolved
expression/builtin_json.go Outdated Show resolved Hide resolved
expression/builtin_json.go Outdated Show resolved Hide resolved
@pingyu pingyu force-pushed the expression_json_search branch from d1673ca to 1a17445 Compare February 15, 2019 15:27
expression/builtin_json.go Outdated Show resolved Hide resolved
expression/builtin_json.go Outdated Show resolved Hide resolved
expression/builtin_json.go Outdated Show resolved Hide resolved
@morgo
Copy link
Contributor

morgo commented Mar 8, 2019

@zz-jason PTAL

@qw4990 qw4990 requested review from alivxxx and winoros March 13, 2019 11:35
@pingyu pingyu force-pushed the expression_json_search branch from 7c2b59e to ce821b3 Compare March 17, 2019 02:58
@pingyu pingyu force-pushed the expression_json_search branch from ce821b3 to c8bc4a8 Compare March 30, 2019 06:29
@pingyu pingyu force-pushed the expression_json_search branch from c8bc4a8 to 8f17c6b Compare April 18, 2019 11:31
Copy link
Contributor

@alivxxx alivxxx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alivxxx
Copy link
Contributor

alivxxx commented Apr 18, 2019

/run-all-tests

@alivxxx alivxxx added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Apr 18, 2019
@alivxxx alivxxx merged commit a594287 into pingcap:master Apr 18, 2019
@morgo morgo mentioned this pull request Jun 24, 2019
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/expression contribution This PR is from a community contributor. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants