Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce an intermediate version that is compatible with vacuum2 #16354

Merged
merged 8 commits into from
Sep 5, 2024

Conversation

SkyFan2002
Copy link
Member

@SkyFan2002 SkyFan2002 commented Aug 30, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

#16049 can be rolled back to this PR, and this PR can be rolled back to older versions. #16049 cannot be directly rolled back to versions before this PR.

v1_main: 1.2.630-nightly-fb9fc9a680 (Latest version of the main branch at the time of testing)

v2_this_pr: This pull request

v3_vacuum2_pr: Pull request #16049 (Implementing vacuum2 functionality)

Test snapshot_location_from_uuid()

Execute the following statements in v3_vacuum2_pr:

create or replace table t1(a int);
insert into t1 values (1);
insert into t1 values (2);
insert into t1 values (3);

Change databend query to v1_main, and execute:

root@localhost:8000/default> create or replace stream s1 on table t1 at (snapshot => '0191c068a10b7ba8954393ccd7f819ce');
error: APIError: ResponseError with 2013: No historical data found at given point

Since v1_main is not compatible with v3_vacuum2_pr, the error is expected, and this indicates that downgrade from v3_vacuum2_pr to v1_main is not applicable, if new data has been written by using v3_vacuum2_pr.

Change databend query to v2_this_pr, and execute:

root@localhost:8000/default> create or replace stream s1 on table t1 at (snapshot => '0191c068a10b7ba8954393ccd7f819ce');

CREATE
OR REPLACE stream s1 ON TABLE t1 at (snapshot = > '0191c068a10b7ba8954393ccd7f819ce')

0 row written in 0.167 sec. Processed 0 row, 0B (0 row/s, 0B/s)

This shows that this PR v2_this_pr is compatible with v3_vacuum2_pr: After upgraded to v3_vacuum2_pr, and wrote down some new data, it still possible to downgrade to v2_this_pr.

Test block_id_from_location()

Execute the following statements in v3_vacuum2_pr:

create or replace table t1(a int, b int);
create or replace table t2(a int, b int);
create or replace stream s2 on table t1 append_only = true;
insert into t2 values(2,2),(3,3),(4,4);
insert into t1 values(1,1),(2,3),(3,3);

Change databend query to v1_main, and execute:

root@localhost:8000/default> merge into t1 using t2 on t1.a=t2.a when matched then update set t1.b=t2.b when not matched then insert *;
error: APIError: ResponseError with 1001: invalid character: expected an optional prefix of `urn:uuid:` followed by [0-9a-fA-F-], found `g` at 1

Change databend query to v2_this_pr, and execute:

root@localhost:8000/default> merge into t1 using t2 on t1.a=t2.a when matched then update set t1.b=t2.b when not matched then insert *;

MERGE INTO t1 USING t2 ON t1.a = t2.a
WHEN matched THEN
UPDATE
SET
  t1.b = t2.b
  WHEN NOT matched THEN
INSERT
  *

-[ RECORD 1 ]-----------------------------------
number of rows inserted: 1
 number of rows updated: 2

1 row read in 1.320 sec. Processed 6 row, 47B (4.55 rows/s, 35B/s)

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • Compatible test

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Aug 30, 2024
# Conflicts:
#	src/query/storages/common/table_meta/src/meta/mod.rs
#	src/query/storages/common/table_meta/src/meta/utils.rs
@SkyFan2002 SkyFan2002 marked this pull request as ready for review September 4, 2024 12:37
@dosubot dosubot bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Sep 4, 2024
@dosubot dosubot bot added A-query Area: databend query A-storage Area: databend storage labels Sep 4, 2024
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:M This PR changes 30-99 lines, ignoring generated files. labels Sep 5, 2024
dantengsky
dantengsky previously approved these changes Sep 5, 2024
@dosubot dosubot bot added the lgtm This PR has been approved by a maintainer label Sep 5, 2024
@dantengsky dantengsky dismissed their stale review September 5, 2024 07:31

ci not pass

@SkyFan2002
Copy link
Member Author

#16397, @dantengsky

@BohuTANG BohuTANG merged commit 015778c into databendlabs:main Sep 5, 2024
71 checks passed
@dantengsky dantengsky changed the title feat: introduce an intermediate version that is compatible with both … feat: introduce an intermediate version that is compatible with vacuum2 Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-query Area: databend query A-storage Area: databend storage lgtm This PR has been approved by a maintainer pr-feature this PR introduces a new feature to the codebase size:L This PR changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants