Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(rust): Purge arrow-rs support #19312

Merged
merged 5 commits into from
Oct 19, 2024
Merged

Conversation

orlp
Copy link
Collaborator

@orlp orlp commented Oct 18, 2024

This was a legacy holdover from before polars-arrow was vendored, we never used this.

@github-actions github-actions bot added internal An internal refactor or improvement rust Related to Rust Polars labels Oct 18, 2024
Copy link

codecov bot commented Oct 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.22%. Comparing base (00650b9) to head (94a822e).
Report is 15 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #19312      +/-   ##
==========================================
+ Coverage   80.02%   80.22%   +0.20%     
==========================================
  Files        1528     1523       -5     
  Lines      209871   209495     -376     
  Branches     2419     2434      +15     
==========================================
+ Hits       167954   168073     +119     
+ Misses      41366    40867     -499     
- Partials      551      555       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ritchie46
Copy link
Member

A rebase away.

@ritchie46 ritchie46 merged commit f88bd6a into pola-rs:main Oct 19, 2024
19 checks passed
@deanm0000
Copy link
Collaborator

:(

@eitsupi
Copy link
Contributor

eitsupi commented Oct 20, 2024

Isn't this a breaking change?

@ritchie46
Copy link
Member

For rust it is. And it will be in a major version bump. @deanm0000 did you use this? As it is arrow, you can still use ffi for this.

@ritchie46 ritchie46 added the breaking rust Change that breaks backwards compatibility for the Rust crate label Oct 20, 2024
@deanm0000
Copy link
Collaborator

I was using to interop with geoarrow, or try to anyway

@c-peters c-peters added the accepted Ready for implementation label Oct 21, 2024
jhorstmann added a commit to jhorstmann/polars that referenced this pull request Oct 23, 2024
@sportfloh
Copy link
Contributor

Hi, @ritchie46
I used this to convert arrow-rs RecordBatches to a polars DataFrame like this:

pub fn record_batch_to_dataframe(batch: &RecordBatch) -> Result<DataFrame, PolarsError> {
    let schema = batch.schema();
    let mut columns = Vec::with_capacity(batch.num_columns());

    for (i, column) in batch.columns().iter().enumerate() {
        let arrow = Box::<dyn polars_arrow::array::Array>::from(&**column);
        columns.push(Series::from_arrow(
            schema.fields().get(i).unwrap().name().into(),
            arrow,
        )?);
    }

    Ok(DataFrame::from_iter(columns))
}

Is there an ffi example? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accepted Ready for implementation breaking rust Change that breaks backwards compatibility for the Rust crate internal An internal refactor or improvement rust Related to Rust Polars
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

6 participants