Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR Tracking the development of v0.12 #218

Draft
wants to merge 201 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
201 commits
Select commit Hold shift + click to select a range
3a22612
Fix the deps of example and integration test
chmp Jun 19, 2024
553b629
Merge pull request #194 from chmp/feature/fix-deps
chmp Jun 19, 2024
3c9ee49
Remove deprecated API
chmp Jun 19, 2024
5f08a06
Use &[FieldRef] instead of &[Field] in arrow API
chmp Jun 19, 2024
d797a69
Remove from_arrow_fields / to_arrow_fields in SerdeArrowSchema
chmp Jun 19, 2024
8525e5f
Update docs
chmp Jun 19, 2024
01b496f
Remove SerdeArrowSchema::new(), Overwrites::new()
chmp Jun 19, 2024
1f658a6
Merge pull request #195 from chmp/feature/remove-deprecated-functions
chmp Jun 19, 2024
6e268f7
Start to implement a common array abstraction
chmp Jul 18, 2024
3dd7828
No longer check all feature combinations per default
chmp Jul 18, 2024
f0fce3a
Address warnings / clippy
chmp Jul 18, 2024
c5efc81
Remove unused View <-> Array conversions, add offset to bitbuffer views
chmp Jul 18, 2024
b3bdb91
Export BitsWithOffset
chmp Jul 18, 2024
00eac96
Impl Date32, Date64, Timestmap serialization
chmp Jul 18, 2024
d43e44e
Add support for Time32, Time64 serialization
chmp Jul 18, 2024
d978af7
Add Duration serialization
chmp Jul 18, 2024
421cd1a
Add Decimal128 serialization
chmp Jul 18, 2024
40942ac
Implement Utf8, LargeUtf8, Binary, LargeBinary serialization
chmp Jul 18, 2024
dc2ed0d
Refactor arrow list construction
chmp Jul 18, 2024
1f4fdef
Implement FixedSizeList, List, LargeList, Struct
chmp Jul 21, 2024
f3a5a9d
Implement FixedSizeBinary
chmp Jul 21, 2024
063c5db
Implement Dictionary
chmp Jul 21, 2024
7461c8b
Implement Map
chmp Jul 21, 2024
c70c39c
Implement Union
chmp Jul 21, 2024
d738eec
Address clippy
chmp Jul 21, 2024
73ecc2d
Move unused markers into view / types submodule to fix PR
chmp Jul 21, 2024
fd513cc
Merge pull request #197 from chmp/feature/arrow-abstraction
chmp Jul 21, 2024
7d48bef
Implement ArrayView based Deserializers for Null, Bool, {UI,I}nt{8,16…
chmp Jul 21, 2024
0938c22
Implement Date32, Date64
chmp Jul 21, 2024
6e62986
Implement Time32, Time64
chmp Jul 21, 2024
d8a388c
Reformat code
chmp Jul 21, 2024
a82a839
Implement Duration, Timestamp
chmp Jul 23, 2024
f1b724f
Implement Utf8, LargeUtf8, Binary, LargeBinaray
chmp Jul 23, 2024
32eebfc
Fix arrow=37 support, start to implement ListArray support
chmp Jul 23, 2024
fb66cc7
Fix arrow==37 support
chmp Jul 23, 2024
a6b6d7a
Implement List, Struct
chmp Jul 23, 2024
a76a66a
Implement FixedSizeList
chmp Jul 23, 2024
8d7d76e
Implement Map
chmp Jul 24, 2024
5da2ea9
Implement Dictionary
chmp Jul 24, 2024
2475512
Implement Union
chmp Jul 24, 2024
7424407
Implement FixedSizeBinary
chmp Jul 24, 2024
658de29
Merge pull request #200 from chmp/feature/arrow-abstraction-deseriali…
chmp Jul 24, 2024
76b29a8
Add tests to confirm that from_type does not work DateTime and Uuid
chmp Aug 3, 2024
35c6646
Document issues with from_type for Uuid and DateTime
chmp Aug 3, 2024
f8cff9c
Merge pull request #206 from chmp/feature/203-uuid-deserialization
chmp Aug 3, 2024
1e839aa
Remove stray comment
chmp Aug 3, 2024
0c77a6f
Refactor code, remove unnecessary abstractions
chmp Aug 3, 2024
e0a5bd5
More refactoring
chmp Aug 3, 2024
7f572a9
More refactoring
chmp Aug 3, 2024
2877b79
Implement Null, Int, UInt, Float
chmp Aug 3, 2024
a66a33c
Implement Date32, Date64, Time32, Time64, Timestamp, Duration
chmp Aug 3, 2024
ef74957
Implement Decimal128
chmp Aug 3, 2024
a43ca4a
Implement Boolean
chmp Aug 3, 2024
94ba5ac
Add Utft, LargeUtf8, Binary, LargeBinary
chmp Aug 3, 2024
87ee1af
Add List, LargeList, Struct
chmp Aug 3, 2024
ba796f5
Implement Map
chmp Aug 3, 2024
d9c2e24
Add Union support
chmp Aug 3, 2024
0d3230c
Implement Dictionary
chmp Aug 3, 2024
1b63937
Implement Null view
chmp Aug 3, 2024
c69a1b0
Implement Int, UInt, Float view
chmp Aug 3, 2024
486921a
Implement Date32, Date64, Decimal128 view
chmp Aug 3, 2024
50fe251
Implement Time32, Time64, Timestamp, Duration view
chmp Aug 3, 2024
3c07cfa
Implement Boolean view
chmp Aug 3, 2024
4716655
Implement Utf8, LargeUtf8, Binary, LargeBinary view
chmp Aug 3, 2024
74d161c
Implement List, LargeList, Struct view
chmp Aug 3, 2024
7de38b4
Implement Map view
chmp Aug 3, 2024
f0e9855
Implement Union view
chmp Aug 3, 2024
a29a876
Implement Dictionary view
chmp Aug 3, 2024
c30281d
Remove unnecessary abstraction
chmp Aug 3, 2024
6215a5c
Merge pull request #207 from chmp/feature/move-arrow2-to-array-abstra…
chmp Aug 3, 2024
f98a5ca
Add known cfgs to Cargo.toml
chmp Aug 4, 2024
b2324cb
Merge pull request #208 from chmp/feature/add-known-cfgs
chmp Aug 4, 2024
058ab4e
Fix clippy
chmp Aug 4, 2024
fa1b7b2
Remove unused cfg
chmp Aug 4, 2024
ce7835b
Use arrays directly in builders (where sensible)
chmp Aug 4, 2024
2ac97de
Reformat
chmp Aug 4, 2024
8947432
Use fieldmeta directly in UnionBuilder
chmp Aug 4, 2024
a686388
Use FieldMeta directly in builders
chmp Aug 4, 2024
0f3a270
Move field lookup logic for StructBuilder into separate struct
chmp Aug 4, 2024
a6dfeff
Rename utils.rs -> simple_serializer.rs
chmp Aug 4, 2024
167f16b
Simplify meta hanlding date64_builder
chmp Aug 4, 2024
a299dbe
Fix clippy
chmp Aug 4, 2024
d858956
Use BitsWithOffset directly in primitive deserializers
chmp Aug 4, 2024
8b6ff63
Use view directly in BinaryDeserializer
chmp Aug 4, 2024
6488901
Use views directly for {Float,Integer,Sting}Deserializer
chmp Aug 4, 2024
5be7f0f
Use views in {Decimal,Dictionary,Time}Deserializers
chmp Aug 4, 2024
d3d44d3
Remove BitBuffer
chmp Aug 4, 2024
b5f43c1
Change unused markers
chmp Aug 4, 2024
5975da4
Merge pull request #209 from chmp/feature/refactor-serializer-inernals
chmp Aug 4, 2024
3d343b9
Remove unused BaseDataTypeDisplay
chmp Aug 4, 2024
c674a61
Add serde tests for fields
chmp Aug 4, 2024
a1aaa61
Add custom Serialize impl for Field
chmp Aug 4, 2024
35f8ee8
Remove metadata / schema split
chmp Aug 4, 2024
f04554b
Broken. Switch API to use arrow::Field internally
chmp Aug 5, 2024
e07641b
Remove ArrowOrCustomField
chmp Aug 5, 2024
f6779f6
Implement Arrow -> Arrow field conversion
chmp Aug 5, 2024
0646dc2
Initial impl of deserialization logic
chmp Aug 5, 2024
26964a7
Implement more type conversions
chmp Aug 7, 2024
904164d
Fully implement Dictionary
chmp Aug 7, 2024
ad38848
Fix schema serialization tests
chmp Aug 7, 2024
d6f6df8
Fix various copy and paste errors
chmp Aug 7, 2024
dde9ed1
Fix copy and paste error
chmp Aug 7, 2024
2ac156f
Fix type id handling for UnionArrays
chmp Aug 7, 2024
f9e90ab
Ensure nullable fields are nullable, even without explicit nullable
chmp Aug 7, 2024
894ab0c
Fix more copy and paste errors
chmp Aug 7, 2024
6602a14
Fix tensor field extension types
chmp Aug 7, 2024
8c6991b
Fix schema tests (+ Bug in serialize)
chmp Aug 7, 2024
8617372
Delete unused code
chmp Aug 7, 2024
5659f74
Move all tests into a single file
chmp Aug 7, 2024
bb74c18
Include type ids in union arrays, similar to arrow
chmp Aug 7, 2024
3102fcf
Move ext types into own modules, add serialization test for FixedShap…
chmp Aug 7, 2024
deaf588
Add serialization test for VariableShapeTensorField
chmp Aug 7, 2024
31d544c
Update changelog
chmp Aug 7, 2024
c95d5f0
Validate map entries
chmp Aug 7, 2024
cca0efe
Fix todos, remove unused code
chmp Aug 7, 2024
0f25d6e
fix format
chmp Aug 7, 2024
f038a3f
Merge pull request #210 from chmp/feature/refactor-schema-impl
chmp Aug 7, 2024
e2231b4
Ensure I8 can be used to serialize and deserialize bool
chmp Aug 7, 2024
03a7949
Move array-ext into utils
chmp Aug 7, 2024
635b176
Allow to construct a Deserializer directly from internal arrays
chmp Aug 7, 2024
ed138a2
Test that bools serialized from ints are true for any non-zero value
chmp Aug 7, 2024
079184f
Add Bool8 extension type
chmp Aug 7, 2024
949e5bc
Fix cross references
chmp Aug 7, 2024
580c539
Add doc example, update extension type docs
chmp Aug 8, 2024
e89b0b4
Fix clippy
chmp Aug 8, 2024
2924824
Rework crate-internal imports
chmp Aug 8, 2024
8e4ccf4
Merge pull request #212 from chmp/feature/211-bool8
chmp Aug 8, 2024
aa147bb
Add missed change
chmp Aug 8, 2024
1eda9f8
Update changelog
chmp Aug 8, 2024
9162036
Merge pull request #214 from chmp/feature/211-bool8-changelog
chmp Aug 8, 2024
e84f901
Add test that checks for the inclusion of the field path
chmp Aug 8, 2024
564c266
Start to document the Error format
chmp Aug 8, 2024
0df5d56
Add machinery to annotate errors with extra information
chmp Aug 8, 2024
1216f45
Merge branch 'main' into develop-0.12
chmp Aug 31, 2024
b04066d
Merge branch 'develop-0.12' into feature/37-improve-error-messages
chmp Aug 31, 2024
fbc28f4
Reformat code
chmp Aug 31, 2024
c920987
Rename assert_error -> assert_error_contains
chmp Aug 31, 2024
04414f9
Reformat code
chmp Aug 31, 2024
78b9689
Format code
chmp Aug 31, 2024
525a89c
Merge branch 'develop-0.12' into feature/37-improve-error-messages
chmp Aug 31, 2024
50731d5
Implement annotated error message for int / struct builders, include …
chmp Aug 31, 2024
e84a813
Add tests for top-level and struct fields
chmp Aug 31, 2024
d5fba86
Simplify annotated error impl
chmp Aug 31, 2024
4a5ca14
Add annotations to binary, bool, date32, date64, decimal, dictionary,…
chmp Aug 31, 2024
0731612
Rewrite FieldLookup helper to ensure ArrayBuilder is Send + Sync
chmp Sep 1, 2024
ac8bcaf
Assert that all public types implement Send + Sync
chmp Sep 1, 2024
d99cbee
Execute test workflow on PRs for develop branches
chmp Sep 1, 2024
4d7c4a8
Merge pull request #227 from chmp/feature/225-array-builder-send
chmp Sep 1, 2024
8dc0ab0
Merge remote-tracking branch 'origin/develop-0.12' into feature/37-im…
chmp Sep 1, 2024
1a22e6e
Simplify error impl
chmp Sep 1, 2024
bd93301
Make the original arrow error available as source()
chmp Sep 1, 2024
f08c276
Implement annotations for all builders
chmp Sep 1, 2024
3e97139
Introduce error context to add additional info
chmp Sep 1, 2024
dd70221
Use explicit context calls instead of relying on wrapper classes
chmp Sep 1, 2024
1aef722
Use ArrayBuilder as return value of take
chmp Sep 1, 2024
f5434f6
Add some style guides
chmp Sep 1, 2024
45b1b05
Improve filed path formatting, remove unncessary Result
chmp Sep 1, 2024
218586b
Add error contexts to OuterSequence builder contruction
chmp Sep 1, 2024
f7e2225
Review error messages in serialization
chmp Sep 1, 2024
8e62280
Include data types in context for serialization
chmp Sep 1, 2024
d771918
Add "data_type" to documentation style guide
chmp Sep 1, 2024
b94935c
Tune error messages
chmp Sep 1, 2024
f525905
Further improve error messages
chmp Sep 1, 2024
d93a05d
Fix clippy
chmp Sep 1, 2024
16866ac
Merge pull request #217 from chmp/feature/37-improve-error-messages
chmp Sep 1, 2024
2180e63
Make paths available throughout the deserializers
chmp Sep 1, 2024
8dc7424
Add context info for default impls in SimpleDeserializer
chmp Sep 1, 2024
93580db
Implement Float, Integer, List, Map, Null, String, Time Deserializers
chmp Sep 1, 2024
acb71c2
Rename path => field, impl binary, bool deserializer
chmp Sep 9, 2024
c60958c
Add try_ function to simplify error handling code
chmp Sep 9, 2024
19c3fdf
Mutate map annotations directly
chmp Sep 9, 2024
2c6af57
Use try_().ctx() in most deserializers
chmp Sep 10, 2024
c9231f9
Add context only for non-annotated errors
chmp Sep 10, 2024
2e72489
use try_().ctx() in serializer
chmp Sep 10, 2024
8f8e717
Unify error message format
chmp Sep 10, 2024
095ed90
Merge pull request #228 from chmp/feature/37-error-messages-deserializer
chmp Sep 10, 2024
9e513a7
Add arrow=53
chmp Sep 11, 2024
b9ec6bf
Fix x.py add-arrow-version
chmp Sep 11, 2024
5688bdb
Fix clippy
chmp Sep 11, 2024
e5ee1e1
Revert Copy change to support previous Arrow versions
chmp Sep 11, 2024
2f57a6f
Update changelog
chmp Sep 11, 2024
69c0609
Merge pull request #229 from chmp/feature/add-arrow-53
chmp Sep 11, 2024
095b5cd
Address clippy
chmp Sep 11, 2024
9976af6
Reformat code
chmp Sep 11, 2024
5b21a9c
Merge pull request #230 from chmp/feature/fix-clippy
chmp Sep 11, 2024
d01fb59
Implement error context for SchemaLike::from_type
chmp Sep 16, 2024
a6f2751
Start to implement error context in from_samples
chmp Sep 16, 2024
c49f239
Implement error context for from_samples
chmp Sep 16, 2024
78d1a6d
Update changelog
chmp Sep 16, 2024
abc7909
Fix capitalization of the error messages
chmp Sep 16, 2024
411fc93
Add hint for coerce_numbers when tracing different number formats
chmp Sep 16, 2024
bbe8a3e
Merge pull request #231 from chmp/feature/add-context-to-tracer-errors
chmp Sep 16, 2024
d199d64
Implement the display reprenstations of anyhow
chmp Sep 16, 2024
607d461
Update changelog
chmp Sep 16, 2024
23b8cb6
Merge pull request #232 from chmp/feature/anyhow-display-rep
chmp Sep 16, 2024
16fe8f6
Remove Ref + ?Sized from from_value and from_samples
chmp Sep 16, 2024
5d94db0
Remove Ref + ?Sized from ArrayBuilder::push / ArrayBuilder::extend
chmp Sep 16, 2024
b79f86d
Remove Ref + ?Sized for to_arrow, to_record_batch, to_arrow2
chmp Sep 16, 2024
eae1992
Merge pull request #233 from chmp/feature/remove-refs-and-sized
chmp Sep 16, 2024
16bbcc9
Add support for FixedSizeBinary / FixedSizeList for arrow2
chmp Sep 16, 2024
a9d75c8
Merge pull request #234 from chmp/feature/add-fixed-size-support-arrow2
chmp Sep 16, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"workflow_dispatch": {},
"pull_request": {
"branches": [
"main"
"main",
"develop-*"
],
"types": [
"opened",
Expand Down
Loading