Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(python): multiple read_excel updates #14039

Merged
merged 1 commit into from
Jan 30, 2024

Conversation

alexander-beedie
Copy link
Collaborator

@alexander-beedie alexander-beedie commented Jan 27, 2024

  • Fixes casting from Datetime to Time for specific pre-epoch values where the time was 00:00:00
    (See: fix(rust,python): some temporal conversion errors for datetimes earlier than 1970-01-01 #14050)
  • Type inference fix when using "calamine" engine (added missing all when determining if a cast should be applied).
  • Deprecate read_csv_options in favour of more generic read_options, which can now be passed through to other engines.
  • Minor optimisation when using "calamine" engine (use eq, not eq_missing).
  • Additional unit test coverage.

@github-actions github-actions bot added fix Bug fix python Related to Python Polars labels Jan 27, 2024
@alexander-beedie alexander-beedie added the A-io-excel Area: reading/writing Excel files label Jan 27, 2024
@alexander-beedie alexander-beedie changed the title fix(python): fix dtype refinement check in read_excel when using the new "calamine" engine fix(python): multiple read_excel updates, fix for datetime → time casts for 00:00:00 before the epoch Jan 27, 2024
@alexander-beedie alexander-beedie force-pushed the read-excel-fix branch 4 times, most recently from 171c5e0 to 709f1e9 Compare January 28, 2024 06:54
@alexander-beedie alexander-beedie marked this pull request as draft January 28, 2024 08:22
@alexander-beedie
Copy link
Collaborator Author

alexander-beedie commented Jan 28, 2024

FYI: temporarily parking this in Draft as I have now found several issues with our datetime conversions relating to the i64 sign-change around the epoch, and will split out the fixes as a separate PR for clearer analysis/review (all the Excel-related updates are fine though).


Update: rebased on top of the temporal fixes (see #14050) - good to go.

@alexander-beedie alexander-beedie changed the title fix(python): multiple read_excel updates, fix for datetime → time casts for 00:00:00 before the epoch fix(python): multiple read_excel updates Jan 28, 2024
@alexander-beedie alexander-beedie marked this pull request as ready for review January 30, 2024 08:26
@pola-rs pola-rs deleted a comment from MarcoGorelli Jan 30, 2024
@ritchie46 ritchie46 merged commit 24a21c3 into pola-rs:main Jan 30, 2024
24 checks passed
@alexander-beedie alexander-beedie deleted the read-excel-fix branch January 30, 2024 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-io-excel Area: reading/writing Excel files fix Bug fix python Related to Python Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants