Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(query): Support access Redis data from dictionaries via the dict_get function. #16389

Merged
merged 48 commits into from
Sep 9, 2024

Conversation

Winnie-Hong0927
Copy link
Contributor

@Winnie-Hong0927 Winnie-Hong0927 commented Sep 4, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

This pull request introduces support for accessing Redis data from dictionaries using the dict_get function. Other source like MySQL, PostgreSQL will be introduced in the future PRs.

  1. Dictionary Creation with Redis Source:
CREATE OR REPLACE DICTIONARY d(key string not null, value string not null) PRIMARY KEY key SOURCE(redis(host='127.0.0.1' port='6379'))

options are used to provide the Redis connection URL, host and port are required options, and username, password, db_index are optional options.

connect to Redis and set some test keys and values

redis-cli -h 127.0.0.1 -p 6379
127.0.0.1:6379> set a abc
OK
127.0.0.1:6379> set b def
OK
127.0.0.1:6379>
  1. Data Access with dict_get Function:
MySQL [(none)]> create or replace table t(a string);
Query OK, 0 rows affected (0.201 sec)

MySQL [(none)]> insert into t values('a'),('b'),('c');
Query OK, 3 rows affected (0.095 sec)

MySQL [(none)]> select a, dict_get(d, 'value', a) from t;
+------+---------------------------------+
| a | dict_get(default.d, 'value', a) |
+------+---------------------------------+
| a | abc |
| b | def |
| c | |
+------+---------------------------------+
3 rows in set (0.173 sec)
  • The result corresponds to the value stored in Redis associated with the keys in table column a.
  • If the key is not exists in the Redis, a default value will be inserted instead.

This PR also introduces a mock_source, which simulates a Redis server to test the dict_get function. Future work will expand this mock data source functionality to include support for other sources such as MySQL, PostgreSQL, and more.

part of #15901

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@b41sh b41sh changed the title feat(query): Support access data from dictionaries via the dict_get function. feat(query): Support access Redis from dictionaries via the dict_get function. Sep 6, 2024
@b41sh b41sh changed the title feat(query): Support access Redis from dictionaries via the dict_get function. feat(query): Support access Redis data from dictionaries via the dict_get function. Sep 6, 2024
@b41sh b41sh marked this pull request as ready for review September 7, 2024 04:42
@b41sh b41sh requested review from sundy-li and BohuTANG September 7, 2024 04:45
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Sep 9, 2024
@b41sh b41sh enabled auto-merge September 9, 2024 06:31
@b41sh b41sh added this pull request to the merge queue Sep 9, 2024
github-merge-queue bot pushed a commit that referenced this pull request Sep 9, 2024
…t_get` function. (#16389)

* feat: dict_get to redis.

* fix

* fix: resolve_dict_get

* fix

* fix

* fix

* fix and fmt

* feat: add transform_dict_get

* feat: add lazy_static

* feat: add key's type Number and prepare to update transform more.

* update: bind_create_dictionary

* update: transform default expr

* fix: transform and binder and type_check

* update: operator & cache.

* fix

* fix: operator

* fix: transform--operators

* update: transform&resolve&argument

* fix

* update: binder & test.

* feat: redis-server & test.

* fmt

* update: test.

* fix: binder & test.

* fix: binder & test.

* update: errorcode & test

* fix

* fix

* fix

* fix

* update: binder.

* fix

* fix: schema & transform

* merge

* update:test

* fix

* fix

* fix.

* update: binder & transform.

* fix: dict_get test

* update.

* update
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 9, 2024
@b41sh b41sh added this pull request to the merge queue Sep 9, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Sep 9, 2024
@b41sh b41sh added this pull request to the merge queue Sep 9, 2024
@BohuTANG BohuTANG removed this pull request from the merge queue due to a manual request Sep 9, 2024
@BohuTANG BohuTANG merged commit 9ac1368 into databendlabs:main Sep 9, 2024
71 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-query Area: databend query C-feature Category: feature lgtm This PR has been approved by a maintainer pr-feature this PR introduces a new feature to the codebase size:XL This PR changes 500-999 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants