Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add unsorted decompressed chunk path even if we have sorted ones #6879

Open
wants to merge 197 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
197 commits
Select commit Hold shift + click to select a range
76c1f97
Add unsorted decompressed chunk path even if we have sorted ones
akuzm May 3, 2024
868d253
label the path w/o sorting with the proper pathkeys
akuzm May 3, 2024
eef6db3
ref
akuzm May 3, 2024
ca9d1c3
set all parameters
akuzm May 3, 2024
f7b1ec5
accept the transparent_decompression ref
akuzm May 6, 2024
848c6ee
simplify?
akuzm May 6, 2024
397717d
tmp debug
akuzm May 7, 2024
b975589
Fix the flaky merge_append_partially_compressed test
akuzm May 7, 2024
7643343
reference merge_append_partially_compressed-*
akuzm May 7, 2024
4f5b34f
reference REL_14_9-96-g162b38a068 merge_append_partially_compressed-*
akuzm May 7, 2024
68062c7
reference REL_13_9 merge_append_partially_compressed-*
akuzm May 7, 2024
2767664
Fix some flakiness in transparent_decompression test
akuzm May 7, 2024
f0f4ebc
reference REL_13_9 transparent_decompression-*
akuzm May 7, 2024
c8b1bac
reference REL_16_0-116-g67738dbf9c transparent_decompression-*
akuzm May 7, 2024
5f2764f
reference REL_14_9-96-g162b38a068 transparent_decompression-*
akuzm May 7, 2024
06c087b
add vacuum as well
akuzm May 7, 2024
5d44042
reference REL_16_1 transparent_decompression-*
akuzm May 7, 2024
3476da5
reference REL_14_11 transparent_decompression-*
akuzm May 7, 2024
5ffd24a
reference REL_13_9 transparent_decompression-*
akuzm May 7, 2024
4fba0f5
Merge remote-tracking branch 'akuzm/flaky-merge' into HEAD
akuzm May 7, 2024
0951fb0
Remove unneeded Sort over Sort
akuzm May 7, 2024
5bbf5df
capitalization
akuzm May 7, 2024
2670505
add vacuum
akuzm May 7, 2024
3b62a11
vacuum analyze
akuzm May 7, 2024
d7e656a
Merge remote-tracking branch 'akuzm/transparent-flaky' into HEAD
akuzm May 7, 2024
f92a7ca
try to make less invasive changes to prevent unexplainable flakiness in
akuzm May 8, 2024
b04d9fb
Merge remote-tracking branch 'akuzm/transparent-flaky' into HEAD
akuzm May 8, 2024
6be134e
Merge remote-tracking branch 'akuzm/flaky-merge' into HEAD
akuzm May 8, 2024
5e496c0
Merge remote-tracking branch 'akuzm/double-sort' into HEAD
akuzm May 8, 2024
0859a3a
bigger table?
akuzm May 8, 2024
e509896
even less changes
akuzm May 8, 2024
f50947e
even bigger table?
akuzm May 8, 2024
f45fa6e
Revert "even bigger table?"
akuzm May 13, 2024
c5e89c6
debug
akuzm May 14, 2024
51c734c
add some asymmetry
akuzm May 16, 2024
5d2acc1
Merge remote-tracking branch 'origin/main' into HEAD
akuzm May 16, 2024
f80c21a
debug solo test
akuzm May 21, 2024
6343044
add vacuum
akuzm May 21, 2024
5e60f30
Merge remote-tracking branch 'origin/main' into HEAD
akuzm May 21, 2024
6a27712
Merge remote-tracking branch 'origin/main' into HEAD
akuzm May 23, 2024
acc14cf
add some limits
akuzm May 23, 2024
bdc0ceb
ref
akuzm May 23, 2024
f601114
smaller limit
akuzm May 23, 2024
451b60f
limit 11
akuzm May 23, 2024
c465b58
work_mem 10MB
akuzm May 23, 2024
e8dfb2a
1 MB
akuzm May 23, 2024
e5a660b
no limit
akuzm May 23, 2024
51fc7a2
Merge remote-tracking branch 'origin/main' into HEAD
akuzm May 29, 2024
6a75d46
no limit fixed
akuzm May 29, 2024
e88d7a6
Revert "no limit"
akuzm May 29, 2024
5ac83d8
limit 11 again
akuzm May 29, 2024
6ff0bd1
work_mem 64kB
akuzm May 29, 2024
780ec7d
Revert "limit 11 again"
akuzm May 29, 2024
f53983a
no limit again
akuzm May 29, 2024
7d05527
limit 11 again
akuzm May 29, 2024
0e789c4
Revert "no limit fixed"
akuzm May 29, 2024
65583c8
limit 11
akuzm May 23, 2024
3269794
Merge remote-tracking branch 'origin/main' into HEAD
akuzm May 29, 2024
0fa86fd
ref
akuzm May 29, 2024
211e3a5
try something else
akuzm May 29, 2024
ad7c269
disable sort in more places
akuzm May 31, 2024
611aa73
Merge remote-tracking branch 'origin/main' into HEAD
akuzm May 31, 2024
c48faf5
disable sort in more test
akuzm May 31, 2024
b45a8d2
more cases
akuzm Jun 3, 2024
8e16fcc
one more
akuzm Jun 3, 2024
561abdc
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 3, 2024
be54bb6
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 4, 2024
267cc31
cleanup
akuzm Jun 4, 2024
21ad1dc
no idea what all this code is doing
akuzm Jun 4, 2024
033c6c9
cleanup
akuzm Jun 4, 2024
c22b2d7
reference REL_16_3 transparent_decompression-*
akuzm Jun 4, 2024
d40119f
reference REL_14_11 transparent_decompression-*
akuzm Jun 4, 2024
8a3c167
benchmark double sort (2024-06-04 no. 1)
akuzm Jun 4, 2024
842e044
cleanup
akuzm Jun 4, 2024
0f2bb7f
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 4, 2024
30f5201
benchmark double sort (2024-06-04 no. 2)
akuzm Jun 4, 2024
c1a0be4
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 5, 2024
b86bdd3
Merge remote-tracking branch 'akuzm/double-sort' into HEAD
akuzm Jun 5, 2024
dba8293
tmp wrong[:
akuzm Jun 5, 2024
85a1926
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 5, 2024
5dfd578
Merge remote-tracking branch 'akuzm/double-sort' into HEAD
akuzm Jun 5, 2024
0039a6f
temp debug
akuzm Jun 12, 2024
8b81b42
small fix
akuzm Jun 12, 2024
20ff42b
Choose cheapest aggregated path as base for chunk-wise aggregation
akuzm Jun 12, 2024
f9508a3
test refs
akuzm Jun 12, 2024
cca447a
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 12, 2024
561cd28
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 12, 2024
371c9ae
Merge remote-tracking branch 'akuzm/cheap-agg' into HEAD
akuzm Jun 12, 2024
2516cec
cleanup after merge
akuzm Jun 12, 2024
769a346
reference REL_16_3 transparent_decompression-* partitionwise-*
akuzm Jun 17, 2024
5fe7c52
reference REL_14_11 transparent_decompression-* partitionwise-*
akuzm Jun 17, 2024
20b6940
parallel plans
akuzm Jun 18, 2024
e017290
remove debug
akuzm Jun 18, 2024
b0d9c44
easier diff
akuzm Jun 18, 2024
c66df25
cleanup
akuzm Jun 18, 2024
3b9e42b
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 18, 2024
a3f53d0
Merge remote-tracking branch 'akuzm/cheap-agg' into HEAD
akuzm Jun 18, 2024
06a2606
fixes for projection
akuzm Jun 18, 2024
9a0a988
stabilize the cagg tests
akuzm Jun 19, 2024
21fb3ae
Support chunkwise aggregation with projection
akuzm Jun 19, 2024
00288ac
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 19, 2024
4b0a1be
tmp debug -- cost based partial aggregation paths
akuzm Jun 20, 2024
59ec486
fix for custom agg paths
akuzm Jun 20, 2024
bab2db4
remove debug
akuzm Jun 20, 2024
7b0a442
fix the tests
akuzm Jun 21, 2024
e785032
stabilize the telemetry_stats
akuzm Jun 22, 2024
55fedd2
partitionwise test
akuzm Jun 22, 2024
b23a737
delete old alternative ref
akuzm Jun 22, 2024
63c7ab4
add vacuum analyze to watermark update test
akuzm Jun 22, 2024
7b795b5
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jun 26, 2024
9115937
try order by in caggs
akuzm Jun 27, 2024
392e342
reference REL_16_3 append-* parallel-* partitionwise-* query-* cagg_u…
akuzm Jun 27, 2024
8f23e8b
reference REL_14_11 append-* parallel-* partitionwise-* query-* cagg_…
akuzm Jun 27, 2024
a9168f6
try another way
akuzm Jun 27, 2024
fb9e1ac
unify test
akuzm Jun 27, 2024
bda8e9b
benchmark partitionwise projection (2024-06-27 no. 1)
akuzm Jun 27, 2024
b856e45
ref?
akuzm Jun 27, 2024
9239b8c
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jul 7, 2024
52825e9
refs
akuzm Jul 7, 2024
e7d187d
disable on v14
akuzm Jul 16, 2024
be2821f
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jul 16, 2024
e87fb2a
test
akuzm Jul 23, 2024
af7f8e1
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Jul 23, 2024
19a156f
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Aug 21, 2024
bb37597
oops
akuzm Aug 21, 2024
144b89e
refs
akuzm Aug 21, 2024
bc72b3d
lost update test changes
akuzm Aug 21, 2024
5020c49
remove projection
akuzm Aug 21, 2024
de95e08
typo
akuzm Aug 21, 2024
8b1b711
not sure
akuzm Aug 21, 2024
be5ec30
benchmark partitionwise projection (2024-08-21 no. 3)
akuzm Aug 21, 2024
8a75199
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Aug 26, 2024
b5b607c
Merge remote-tracking branch 'akuzm/chunkwise-projection' into HEAD
akuzm Aug 26, 2024
1eae5b5
reference REL_16_3 transparent_decompression-*
akuzm Aug 26, 2024
20c81f9
Merge origin/main into tmp (using imerge)
akuzm Sep 10, 2024
8009a76
fixes after merge
akuzm Sep 10, 2024
cfff28a
Merge origin/main into heads/tmp (using imerge)
akuzm Dec 5, 2024
f1c5782
fix the reference after merge
akuzm Dec 5, 2024
4820a50
hypercore test
akuzm Dec 5, 2024
10518fd
reference REL_17_0-80-gb7467ab71c transparent_decompression-*
akuzm Dec 5, 2024
2fa3344
reference REL_14_11 transparent_decompression-*
akuzm Dec 5, 2024
2385f1d
reference REL_16_4-111-g925b3aa857 transparent_decompression-*
akuzm Dec 5, 2024
d75dc9e
try to replicate the move of chunkwise to tsl
akuzm Dec 5, 2024
6194250
Merge remote-tracking branch 'akuzm/cheap-agg' into tmp
akuzm Dec 5, 2024
d18bef0
cleanup
akuzm Dec 5, 2024
eb0cfe4
partialize the entire pathlist
akuzm Dec 5, 2024
518132b
fix another hypercore test
akuzm Dec 5, 2024
7af9df6
reference REL_17_0-80-gb7467ab71c cagg_union_view-*
akuzm Dec 5, 2024
d786a65
fix the debug guc
akuzm Dec 5, 2024
12e5dd4
reference REL_17_0-80-gb7467ab71c cagg_union_view-* continuous_aggs-*…
akuzm Dec 5, 2024
11b1db5
reference REL_14_11 cagg_union_view-* continuous_aggs-* transparent_d…
akuzm Dec 5, 2024
f9186ea
reference REL_16_4-111-g925b3aa857 cagg_union_view-* continuous_aggs-…
akuzm Dec 5, 2024
2ae3095
Merge remote-tracking branch 'akuzm/cheap-agg' into HEAD
akuzm Dec 5, 2024
fc4d4f4
fixes
akuzm Dec 5, 2024
62f0610
fix debug
akuzm Dec 5, 2024
0f0c14e
vector agg memory test is fixed now
akuzm Dec 5, 2024
455e92b
benchmark chunkwise aggregation path changes (2024-12-09 no. 1)
akuzm Dec 9, 2024
5f5879a
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Dec 9, 2024
c7c8f60
reference REL_15_7 transparent_decompression-*
akuzm Dec 9, 2024
457826d
Merge remote-tracking branch 'akuzm/cheap-agg' into HEAD
akuzm Dec 10, 2024
75c8515
cleanup
akuzm Dec 10, 2024
506172c
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Dec 10, 2024
37bb919
cleanup
akuzm Dec 10, 2024
cc52ef2
reference REL_17_0-80-gb7467ab71c transparent_decompression-* merge_a…
akuzm Dec 10, 2024
7cc64ee
add a test for another fixed issue
akuzm Dec 10, 2024
2bce2ac
cleanup
akuzm Dec 12, 2024
705c43f
reference REL_14_11 transparent_decompression-* merge_append_partiall…
akuzm Dec 12, 2024
bbbd8eb
reference REL_16_4-111-g925b3aa857 transparent_decompression-* merge_…
akuzm Dec 12, 2024
6865182
fix MergeAppend with sorting by time_bucket
akuzm Dec 12, 2024
a0cb824
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Dec 12, 2024
ad892ea
benchmark unsorted decompression (2024-12-12 no. 4)
akuzm Dec 12, 2024
3939ba2
Merge remote-tracking branch 'origin/main' into HEAD
akuzm Dec 17, 2024
c2d293f
reference
akuzm Dec 17, 2024
c5ace45
reference REL_17_0-80-gb7467ab71c transparent_decompression-* merge_a…
akuzm Dec 17, 2024
e592158
reference REL_14_11 transparent_decompression-* merge_append_partiall…
akuzm Dec 17, 2024
65fdd96
reference REL_16_4-111-g925b3aa857 transparent_decompression-* merge_…
akuzm Dec 17, 2024
1812105
more const
akuzm Dec 17, 2024
757fb52
remove the macro
akuzm Dec 17, 2024
3e7ec2d
make a separate function
akuzm Dec 17, 2024
8b0a093
brrace
akuzm Dec 17, 2024
8010a6b
unify parallel path handling
akuzm Dec 17, 2024
8528478
benchmark unsorted paths (2024-12-17 no. 2)
akuzm Dec 17, 2024
ae8149a
fix the minmax initplan cost
akuzm Dec 18, 2024
81d4d85
benchmark unsorted paths (2024-12-18 no. 3)
akuzm Dec 18, 2024
6ca7143
some fixes and gather over sort
akuzm Dec 19, 2024
21bfc85
Revert "Chunk-wise agg: add Gather above Sort"
akuzm Dec 19, 2024
3916fe0
accept the refs
akuzm Dec 19, 2024
e834b23
benchmark unsorted paths (2024-12-23 no. 5)
akuzm Dec 23, 2024
a084f90
create parallel index paths for compressed table
akuzm Dec 23, 2024
f790af4
benchmark unsorted paths (2024-12-23 no. 6)
akuzm Dec 23, 2024
96a129b
benchmark unsorted paths (2024-12-23 no. 8)
akuzm Dec 23, 2024
ad804a3
fix
akuzm Dec 23, 2024
b3beeda
optimizations 1
akuzm Dec 24, 2024
c3287de
optimization 2
akuzm Dec 24, 2024
8db0166
optimizations.....
akuzm Dec 24, 2024
d4825e4
batch sorted merge
akuzm Dec 24, 2024
1c81dc1
benchmark unsorted paths (2024-12-24 no. 9)
akuzm Dec 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cleanup
  • Loading branch information
akuzm committed Dec 12, 2024
commit 2bce2accb943e3984d23c9567c450cdb5df72a0f
21 changes: 10 additions & 11 deletions tsl/src/nodes/decompress_chunk/decompress_chunk.c
Original file line number Diff line number Diff line change
@@ -760,12 +760,7 @@ make_chunk_sorted_path(PlannerInfo *root, RelOptInfo *chunk_rel, const Hypertabl
return NULL;
}

/* Copy the decompress chunk path because the original can be recycled in add_path, and our
* sorted path must be independent. */
if (!ts_is_decompress_chunk_path(path))
{
return NULL;
}
Assert(ts_is_decompress_chunk_path(path));

/* Iterate over the sort_pathkeys and generate all possible useful sorting */
List *useful_pathkeys = NIL;
@@ -802,12 +797,16 @@ make_chunk_sorted_path(PlannerInfo *root, RelOptInfo *chunk_rel, const Hypertabl
return NULL;
}

/* Create the sorted path for these useful_pathkeys */
if (pathkeys_contained_in(useful_pathkeys, path->pathkeys))
{
return NULL;
}
/*
* We should be given an unsorted DecompressChunk path.
*/
Assert(path->pathkeys == NIL);

/*
* Create the sorted path for these useful_pathkeys. Copy the decompress
* chunk path because the original can be recycled in add_path, and our
* sorted path must be independent.
*/
DecompressChunkPath *path_copy = copy_decompress_chunk_path((DecompressChunkPath *) path);

Path *sorted_path = (Path *)
Loading