Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-17513: [C++] Compare RLE Arrays #14063

Closed
wants to merge 131 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
131 commits
Select commit Hold shift + click to select a range
53323d4
add type-only parts from rle branch
zagto Jul 29, 2022
4eb0b46
handle rle type in ToString for type ids
zagto Jul 20, 2022
b4f94c0
handle RLE in ARROW_GENERATE_FOR_ALL_TYPES
zagto Jul 20, 2022
12df4bc
imlement NotImplemented status for rle in MakeFormatterImpl
zagto Jul 20, 2022
bf68567
add RunLengthEncodedArray class
zagto Jul 27, 2022
b90a5fe
type_fwd: add RunLengthEncodedType
zagto Jul 27, 2022
21259d8
include new array_encoded header in array.h
zagto Jul 27, 2022
2689543
introduce type traits for rle/encoding types
zagto Jul 28, 2022
83754f5
add methods for rle in type visitor abstract classes
zagto Jul 28, 2022
735fa04
split ARROW_GENERATE_FOR_ALL_TYPES
zagto Jul 29, 2022
672b2a5
actually add scalar type visitor
zagto Jul 28, 2022
7091760
add comments
zagto Jul 29, 2022
687b344
Merge branch 'scalar-visitor' into rle-type
zagto Jul 29, 2022
e4d480a
stub rle type in various visitors
zagto Jul 28, 2022
05f7404
more stubs
zagto Jul 28, 2022
f62046b
more stubs
zagto Jul 28, 2022
cf548bc
formatting
zagto Jul 29, 2022
8e21a09
update/stub more visitors
zagto Jul 29, 2022
0a33530
one more visitor
zagto Jul 29, 2022
a59335c
gtest_util: add rle to type list
zagto Jul 29, 2022
7741fe9
remove some unused methods for now
zagto Jul 29, 2022
77b5650
add rle_util
zagto Jul 29, 2022
3fcdddd
update the rle_util functions
zagto Jul 30, 2022
3bfef48
type: RLE only has one buffer
zagto Aug 1, 2022
0f391a4
naming
zagto Aug 1, 2022
8380e79
wip: testing for VisitMergedRuns
zagto Aug 1, 2022
6878b5b
array_encoded: add methods for working with RunLengthEncodedArray
zagto Aug 2, 2022
fe8c4a4
add rle array tests
zagto Aug 2, 2022
adfcac5
make HasValidityBitmap return false for rle
zagto Aug 2, 2022
caf19be
make null count of rle arrays always zero
zagto Aug 2, 2022
0ef0128
handle rle in GetNumBuffers
zagto Aug 2, 2022
d503b98
fix setting offset in RunLengthEncodedArray::Make
zagto Aug 2, 2022
47e186b
fix testing status strings
zagto Aug 2, 2022
a0a053c
Merge branch 'rle-type' into rle-util
zagto Aug 2, 2022
1624e1e
mark constructors as explicit
zagto Aug 2, 2022
b65f2a6
doxygen: add group for encoded arrays
zagto Aug 2, 2022
adc4e31
fix comment
zagto Aug 2, 2022
c94e1dd
wip: test for VisitMergedRuns
zagto Aug 3, 2022
25e9e2d
make shared_ptr reference parameters const
zagto Aug 3, 2022
991541d
Merge branch 'rle-type' into rle-util
zagto Aug 3, 2022
8260b92
actually fix const reference parameters
zagto Aug 3, 2022
4324f72
type_traits: remove misleading bytes_required method for rle
zagto Aug 3, 2022
b318d43
type_internal: stub visitor for rle
zagto Aug 3, 2022
557a058
type: move run ends into child and set children array in constructor
zagto Aug 3, 2022
e0bf74b
update RunLengthEncodedArray class
zagto Aug 3, 2022
2565e2f
pandas: correctly detect rle type as not supported
zagto Aug 3, 2022
0133f2e
Merge branch 'rle-type' into rle-util
zagto Aug 3, 2022
d8623d7
fix copy-paste error in VisitMergedRuns test
zagto Aug 3, 2022
23ca398
remove invalid testcase in FindPhysicalOffset test
zagto Aug 3, 2022
13b49ac
order RunLengthEncodedArray arguments like the child arrays in format
zagto Aug 3, 2022
d81cc30
hopefully fix compiling parquet
zagto Aug 3, 2022
587860f
fix test
zagto Aug 3, 2022
634eafe
fix RunLengthEncodedArray constructor calls
zagto Aug 3, 2022
2ef0f2f
Merge branch 'rle-type' into rle-util
zagto Aug 3, 2022
60527e0
rle_util_test: formatting
zagto Aug 3, 2022
aaab59c
update rle utilitties for new format
zagto Aug 3, 2022
117da6f
rle_util: formatting
zagto Aug 3, 2022
30d7d67
give rle type one buffer since I found examples of code assuming one
zagto Aug 4, 2022
81520bc
formatting
zagto Aug 4, 2022
05e98c6
fix comment
zagto Aug 4, 2022
9121faf
Merge branch 'rle-type' into rle-util
zagto Aug 4, 2022
c78fbf3
rle_util: fixes and make FindPhysicalOffset take element count instre…
zagto Aug 4, 2022
f032fbc
rle_util tests fixed
zagto Aug 4, 2022
9775fb8
stub rle in another visitor in parquet
zagto Aug 4, 2022
438b268
Merge branch 'rle-type' into rle-util
zagto Aug 4, 2022
ffdfb1c
rle_util: add comments
zagto Aug 4, 2022
8bb5828
fix comments
zagto Aug 4, 2022
d2731a0
builder_base: use VisitScalarTypeInline
zagto Aug 4, 2022
5de04c2
Merge branch 'scalar-visitor' into rle-type
zagto Aug 4, 2022
ff3ab7c
remove no longer used visitor stub
zagto Aug 4, 2022
1b646b6
formatting
zagto Aug 4, 2022
0381dd5
Merge branch 'rle-type' into rle-util
zagto Aug 4, 2022
af3e625
Revert "remove no longer used visitor stub"
zagto Aug 4, 2022
be3ef70
Merge branch 'rle-type' into rle-util
zagto Aug 4, 2022
71e7bde
add more user friendly methods to get physical offset and length
zagto Aug 4, 2022
ec1e004
add test
zagto Aug 4, 2022
63b5ce2
fix GetPhysicalLength method
zagto Aug 4, 2022
1d31bdd
better test for physical offset/length
zagto Aug 4, 2022
6f58c4b
VisitMergedRuns test: test inverted case
zagto Aug 5, 2022
2c211c4
VisitMergedRuns: fix handling both arrays ending inside a run
zagto Aug 5, 2022
1b6fc84
implement rle_util visitor variant for a single array
zagto Aug 10, 2022
66f2d81
add merged rle iterator to replace visitor
zagto Aug 10, 2022
dd3eeb3
MergedRunsIterator: use pointer instead of reference_wrapper
zagto Aug 15, 2022
7f37695
fix single input MergedRunsIterator constructor
zagto Aug 15, 2022
fea250e
fix . -> mixup
zagto Aug 15, 2022
b45bd6a
fix rle iterator
zagto Aug 15, 2022
f4b3f8a
fix AddArtificialOffsetInChildArray
zagto Aug 15, 2022
c2dcab4
also test rle iterator on single input array
zagto Aug 15, 2022
fa74062
naming: DataArray -> ValuesArray
zagto Aug 15, 2022
b894269
remove old VisitMergedRuns/VisitRuns functions that are replaced by i…
zagto Aug 15, 2022
d3e5cac
mark rle_util accessors as inline
zagto Aug 15, 2022
f8ee87a
rename rle iterator test
zagto Aug 15, 2022
fd26ec0
Merge branch 'master' into scalar-visitor
zagto Aug 15, 2022
98cf6e4
Merge branch 'scalar-visitor' into rle-type
zagto Aug 15, 2022
d350081
fix rle stub in parquet path_internal
zagto Aug 15, 2022
281a470
Merge branch 'rle-type' into rle-util
zagto Aug 15, 2022
99a264a
fix GetPhysicalLength function
zagto Aug 18, 2022
2d03dbf
rle compare
zagto Aug 24, 2022
bbd81f6
byte-swapping RLE arrays should now just work
zagto Aug 24, 2022
5c60fbd
Merge branch 'rle-type' into rle-util
zagto Aug 24, 2022
59f5fa2
fix handling of 0 length arrays at the end of run ends array
zagto Sep 1, 2022
7c46a05
Merge branch 'master' into scalar-visitor
zagto Sep 1, 2022
5933a21
Merge branch 'scalar-visitor' into rle-type
zagto Sep 1, 2022
07c6446
Merge branch 'rle-type' into rle-util
zagto Sep 1, 2022
a35b651
fix diff error message
zagto Sep 6, 2022
f9ddf79
fix offset handling
zagto Sep 7, 2022
f55717d
fix zero-lenght arrays in rle iterator
zagto Sep 7, 2022
5bdc984
rle_util_test: avoid using array span objects beyond thier lifetime
zagto Sep 7, 2022
286fe6c
rle iterator: add more accessor variants
zagto Sep 7, 2022
f6d78d8
formatting
zagto Sep 7, 2022
e1b8b7d
Merge branch 'rle-util' into rle-compare
zagto Sep 7, 2022
335d550
rle_compare: use index_into_array
zagto Sep 7, 2022
25bc98b
mark rle diff as not supported correctly
zagto Sep 8, 2022
de64932
formatting
zagto Sep 8, 2022
a9e6aba
more formatting
zagto Sep 8, 2022
2eeb0fd
fix rle type construction add test
zagto Sep 13, 2022
3b5ace1
test rle type string
zagto Sep 13, 2022
6733c8f
rle_util_test: fix too big Slice() on NullArray
zagto Sep 13, 2022
0d80e3a
add diagram for MergedRunsInterator test
zagto Sep 13, 2022
8731559
fix typo
zagto Sep 13, 2022
2272661
rle type/array: support non-int32 run ends arrays
zagto Sep 26, 2022
acf5404
Merge branch 'rle-type' into rle-util
zagto Sep 26, 2022
d3716fa
Merge branch 'master' into scalar-visitor
zagto Sep 26, 2022
4a4a6ac
Merge branch 'scalar-visitor' into rle-type
zagto Sep 26, 2022
54db9a8
Merge branch 'rle-type' into rle-util
zagto Sep 26, 2022
bda49e7
type_fwd: C++17 compatibility
zagto Sep 26, 2022
580e548
Merge branch 'rle-type' into rle-util
zagto Sep 26, 2022
90a0814
rle_util: support different types for run ends array
zagto Oct 7, 2022
8c23788
fix run-ends type detection in GetPhysicalOffset and GetPhysicalLength
zagto Oct 7, 2022
8534f88
test mutltple run ends types in rle offset/length test
zagto Oct 7, 2022
080add2
Merge branch 'rle-util' into rle-compare
zagto Oct 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'master' into scalar-visitor
  • Loading branch information
zagto committed Aug 15, 2022
commit fd26ec03a93242a70d44563579caf31374c2c4fc

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.