Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(rust): fix build for wasm #9502

Merged
merged 4 commits into from
Aug 16, 2023
Merged

fix(rust): fix build for wasm #9502

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bc27df4
fix(rust) fix build for wasm
lorepozo Jul 31, 2023
52d8a49
move wasm check to Makefile
lorepozo Aug 14, 2023
afd30de
use excluded features when checking wasm
lorepozo Aug 14, 2023
5c5bbbb
update github workflow steps for check-wasm
lorepozo Aug 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 113 additions & 0 deletions .github/workflows/test-rust.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,3 +112,116 @@ jobs:

- name: Run cargo hack
run: cargo hack check -p polars --each-feature --no-dev-deps

build-other-targets:
runs-on: ubuntu-latest
strategy:
fail-fast: true
matrix:
include:
- name: Wasm
target: wasm32-unknown-unknown
features: >
-F abs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that familiar with this. What does -F mean in this context?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is used further down on line 226 — it specifies a feature:

cargo build --target ${{ matrix.target }} -p polars --no-default-features ${{ matrix.features }}

-F algo
-F approx_unique
-F arg_where
-F asof_join
-F avro
-F avx512
-F bigidx
-F binary_encoding
-F checked_arithmetic
-F chunked_ids
-F coalesce
-F concat_str
-F cse
-F cum_agg
-F cumulative_eval
-F dataframe_arithmetic
-F date_offset
-F decompress
-F describe
-F diagonal_concat
-F diff
-F docs
-F dot_diagram
-F dot_product
-F dtype-date
-F dtype-datetime
-F dtype-duration
-F dtype-slim
-F dtype-struct
-F dtype-time
-F dynamic_groupby
-F ewma
-F fmt_no_tty
-F fused
-F groupby_list
-F horizontal_concat
-F interpolate
-F is_first
-F is_in
-F is_last
-F is_unique
-F lazy
-F lazy_regex
-F list_count
-F list_eval
-F list_take
-F list_to_struct
-F log
-F merge_sorted
-F meta
-F mode
-F moment
-F ndarray
-F object
-F partition_by
-F pct_change
-F pivot
-F product
-F propagate_nans
-F random
-F range
-F rank
-F reinterpret
-F repeat_by
-F rolling_window
-F round_series
-F row_hash
-F rows
-F search_sorted
-F semi_anti_join
-F serde
-F serde-lazy
-F sign
-F simd
-F sort_multiple
-F string_encoding
-F string_from_radix
-F string_justify
-F strings
-F take_opt_iter
-F temporal
-F timezones
-F to_dummies
-F top_k
-F trigonometry
-F true_div
-F unique_counts
-F zip_with

steps:
- uses: actions/checkout@v3
- name: Set up Rust
run: |
rustup target add ${{ matrix.target }}
rustup show
- name: Cache Rust
uses: Swatinem/rust-cache@v2
with:
save-if: ${{ github.ref_name == 'main' }}
- name: Compile everything
run: cargo build --target ${{ matrix.target }} -p polars --no-default-features ${{ matrix.features }}

4 changes: 3 additions & 1 deletion crates/polars-io/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@ chrono-tz = { workspace = true, optional = true }
fast-float = { version = "0.2", optional = true }
flate2 = { version = "1", optional = true, default-features = false }
futures = { workspace = true, optional = true }
home = { version = "0.5.4" }
lexical = { version = "6", optional = true, default-features = false, features = ["std", "parse-integers"] }
lexical-core = { version = "0.8", optional = true }
memchr = { workspace = true }
Expand All @@ -42,6 +41,9 @@ simdutf8 = { version = "0.1", optional = true }
tokio = { version = "1.26", features = ["net"], optional = true }
url = { workspace = true, optional = true }

[target.'cfg(not(target_family = "wasm"))'.dependencies]
home = "0.5.4"

[dev-dependencies]
tempdir = "0.3.7"

Expand Down
5 changes: 3 additions & 2 deletions crates/polars-io/src/utils.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
use std::path::{Path, PathBuf};

use home::home_dir;
use polars_core::frame::DataFrame;
use polars_core::prelude::*;

Expand All @@ -16,7 +15,9 @@ use crate::ArrowSchema;
pub fn resolve_homedir(path: &Path) -> PathBuf {
// replace "~" with home directory
if path.starts_with("~") {
if let Some(homedir) = home_dir() {
// home crate does not compile on wasm https://github.com/rust-lang/cargo/issues/12297
#[cfg(not(target_family = "wasm"))]
if let Some(homedir) = home::home_dir() {
return homedir.join(path.strip_prefix("~").unwrap());
}
}
Expand Down
8 changes: 8 additions & 0 deletions crates/polars-utils/src/wasm.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,14 @@ impl Pool {
rayon::current_num_threads()
}

pub fn current_thread_index(&self) -> Option<usize> {
rayon::current_thread_index()
}

pub fn current_thread_has_pending_tasks(&self) -> Option<bool> {
None
}

pub fn install<OP, R>(&self, op: OP) -> R
where
OP: FnOnce() -> R + Send,
Expand Down