Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: add Allocator type param to MutableBuffer #6336

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
+460 −82
Open

feat: add Allocator type param to MutableBuffer #6336

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
8be6078
add feature gate
waynexia Aug 29, 2024
c12e5f8
move reallocate
waynexia Aug 29, 2024
609d93c
adapt from vec
waynexia Aug 29, 2024
227e667
midify workflow and miri shell
waynexia Aug 30, 2024
05fd1cb
also modify clippy command
waynexia Aug 30, 2024
cbcbc51
fix style and clippy
waynexia Aug 30, 2024
b8d2924
add document and example
waynexia Sep 2, 2024
4934043
customize allocator in example
waynexia Sep 18, 2024
d2fc15e
Merge branch 'master' into allocator-api
waynexia Sep 18, 2024
36eccf6
modify ci for nightly feature
waynexia Sep 18, 2024
fa587a0
remove one job
waynexia Sep 18, 2024
6b28b10
limit freeze methods to only available for the default allocator
waynexia Sep 19, 2024
ceef772
fix doc links
waynexia Sep 19, 2024
02d73ad
add unit test for custom allocator
waynexia Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 22 additions & 5 deletions .github/workflows/arrow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,8 @@ jobs:
submodules: true
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
- name: Test arrow-buffer with all features
run: cargo test -p arrow-buffer --all-features
- name: Test arrow-buffer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found it confusing at first why this line doesn't follow the model of the rest of the tests in this workflow which run with --all-features

I am also concerned this may remove coverage for some of the other features such as prettyprint, ffi and pyarrow

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

arrow-buffer only has one feature gate allocator_api from this PR. This specific change won't affect other tests like prettyprint as the command -p arrow-buffer only runs on this sub-crate. But I agree it's easy to forget when the new feature comes in the future... Sadly, cargo seems not to support opt-out one feature in CLI 😢

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, the same problem comes to the main arrow lib because I added a delegate feature allocator_api = ["arrow-buffer/allocator_api"]. Looks like we have to enumerate all available features except allocator_api in CI?

run: cargo test -p arrow-buffer
- name: Test arrow-data with all features
run: cargo test -p arrow-data --all-features
- name: Test arrow-schema with all features
Expand Down Expand Up @@ -163,8 +163,8 @@ jobs:
uses: ./.github/actions/setup-builder
- name: Setup Clippy
run: rustup component add clippy
- name: Clippy arrow-buffer with all features
run: cargo clippy -p arrow-buffer --all-targets --all-features -- -D warnings
- name: Clippy arrow-buffer
run: cargo clippy -p arrow-buffer --all-targets -- -D warnings
- name: Clippy arrow-data with all features
run: cargo clippy -p arrow-data --all-targets --all-features -- -D warnings
- name: Clippy arrow-schema with all features
Expand Down Expand Up @@ -192,8 +192,25 @@ jobs:
- name: Clippy arrow-row with all features
run: cargo clippy -p arrow-row --all-targets --all-features -- -D warnings
- name: Clippy arrow with all features
run: cargo clippy -p arrow --all-features --all-targets -- -D warnings
# `allocator_api` is ignored as it requires nightly toolchain
run: cargo clippy -p arrow -F csv -F json -F ipc -F ipc_compression -F prettyprint -F chrono-tz -F ffi -F pyarrow --all-targets -- -D warnings
- name: Clippy arrow-integration-test with all features
run: cargo clippy -p arrow-integration-test --all-targets --all-features -- -D warnings
- name: Clippy arrow-integration-testing with all features
run: cargo clippy -p arrow-integration-testing --all-targets --all-features -- -D warnings

clippy-nightly:
name: Clippy
runs-on: ubuntu-latest
container:
image: amd64/rust
steps:
- uses: actions/checkout@v4
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
with:
rust-version: nightly
- name: Setup Clippy
run: rustup component add clippy
- name: Clippy arrow-buffer with all features
run: cargo clippy -p arrow-buffer --all-features --all-targets -- -D warnings
2 changes: 1 addition & 1 deletion .github/workflows/miri.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ cargo miri setup
cargo clean

echo "Starting Arrow MIRI run..."
cargo miri test -p arrow-buffer
cargo miri test -p arrow-buffer --features allocator_api
cargo miri test -p arrow-data --features ffi
cargo miri test -p arrow-schema --features ffi
cargo miri test -p arrow-ord
Expand Down
3 changes: 3 additions & 0 deletions arrow-buffer/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ name = "arrow_buffer"
path = "src/lib.rs"
bench = false

[features]
allocator_api = []

[dependencies]
bytes = { version = "1.4" }
num = { version = "0.4", default-features = false, features = ["std"] }
Expand Down
Loading
Loading