Python Polars 0.20.12

Warning

This release was deleted from PyPI. Please use the 0.20.13 release instead.

🚀 Performance improvements

• auto-tune concurrency budget (#14753)
• Don't materialize for broadcasting fill_null value and default value of replace (#14736)
• Improve performance of boolean filters 1-100x. (#14746)

🐞 Bug fixes

• fix hashing specialization (#14754)
• Sum after filter in aggregation context sometimes returned NULL (#14752)
• Allow list.contains() for list of categoricals (#14744)
• Fix bug where alias was ignored in COUNT(*) optimization (#14738)
• Fix DataFrame.sum for decimals (#14732)

📖 Documentation

• Link to plugins tutorial more prominently (#14727)

📦 Build system

• update ahash (#14731)

🛠️ Other improvements

• update ahash (#14731)
• Use datetime_to_int util for AnyValue conversion (#14743)
• Refactor utils/convert.py module (#14739)

Thank you to all our contributors for making this release possible!
@MarcoGorelli, @c-peters, @nameexhaustion, @orlp, @petrosbar, @ritchie46 and @stinodego

Python Polars 0.20.11

Warning

This release was deleted from PyPI. Please use the 0.20.13 release instead.

🏆 Highlights

• fast path for COUNT(*) queries (#14574)

⚠️ Deprecations

• Deprecate passing time_unit=None to Datetime constructor (#14708)
• Rename Expr.meta.write_json/Expr.from_json to Expr.meta.serialize/Expr.deserialize (#14490)
• Deprecate default value for ignore_nulls for ewm methods (#14663)
• Deprecate DataFrame/LazyFrame.approx_n_unique (#14594)

🚀 Performance improvements

• 2-3x speedup in creating literals/Series of type Date (#14716)
• fix accidental quadratic utf8 validation in parquet (#14705)
• Add __slots__ to most Polars classes (#13236)
• fast path for COUNT(*) queries (#14574)
• Elide the total order wrapper for non-(float/option) types (#14648)
• add utf8-validation fast paths for utf8view (#14644)
• don't reassign chunks back to df owner (#14633)
• If there are many small chunks in write_parquet(), convert to a single chunk (#14484) (#14487)
• Polars thread pool was not used properly in various functions (#14583)

✨ Enhancements

• Change default for maximum number of Series items printed to 10 to match DataFrame (#14703)
• Change default number of rows printed in Notebooks for DataFrame/Series to 10 (#14536)
• Infer values columns in DataFrame.pivot when values is None (#14477)
• fast path for COUNT(*) queries (#14574)
• let rolling accept index_column of type UInt32 or UInt64 (#14669)
• Treat float -0.0 == 0.0 and -NaN == NaN in group-by, joins and unique (#14617)
• Improve consistency of dtype inference from Python types (#14600)
• Properly cache object-stores (#14598)

🐞 Bug fixes

• Fix parallel strategy for LazyFrame not being applied (#14696)
• Block slice pushdown past non-literal projections or when the projection doesn't contain any columns from the input (#14684)
• Fix number of rows printed in DataFrame/Series repr (edge cases) (#14548)
• Fix contention panics in file gc threads (#14690)
• Fix feature combination (#14688)
• Only push predicates depending on the subset columns past unique() (#14668)
• Properly handle a single empty RecordBatch in from_arrow (#14683)
• More accurate type hints for binary file-like inputs (#14674)
• Reading RLE_DICTIONARY-encoded parquet incorrectly coalesced NULL to empty string in some cases (#14670)
• use correct flooring division/modulo operator in literal optimizer and const_lhs <> series ops (#14671)
• Enable is_in for string in categorical/enum (#14576)
• Fixes a read_database issue loading specific datetime types from SQL Server backends (#14627)
• Polars thread pool was not used properly in various functions (#14583)

📖 Documentation

• Improve some DataType docstrings (#14719)
• Fix bad link due to boldness in pl.count (#14691)
• Improve docstrings for ewm_* and rolling_* methods (#14667)
• Improve examples for Series.binary.encode and Series.binary.decode. (#14579)
• Add examples for Series.kurtosis (#14681)
• Fix docstring for LazyGroupBy.len (#14661)
• Separate "writing a plugin" from "registering an expression" in user guide, add some extra links, don't use deprecated _register_plugin (#14621)
• Fix code block path for group by example in getting started guide (#14612)
• Add missing 'string' column in reading-writing Rust example to match Python example (#14597)

📦 Build system

• Limit CMake threads to fix crash compiling libz-ng-sys on macOS (#14715)

🛠️ Other improvements

• Limit CMake threads to fix crash compiling libz-ng-sys on macOS (#14715)
• Fix make requirements when conda environment is active (#14693)
• update rustc (#14678)
• redundant imports all crates (#14662)
• Avoid unnecessary cast in Series constructor (#14650)
• Add test on selecting Enum columns (#14628)
• Use uv for make requirements (#14618)
• Rename coverage file (#14607)
• Add a lint-only Makefile option (#14602)
• No longer use SeriesView in Series.to_numpy (#14588)

Thank you to all our contributors for making this release possible!
@Kylea650, @MarcoGorelli, @Object905, @alexander-beedie, @bsubei, @c-peters, @eLVas, @itamarst, @mbuhidar, @mcrumiller, @nameexhaustion, @orlp, @rijkvp, @ritchie46 and @stinodego

Python Polars 0.20.10

⚠️ Deprecations

• Add allow_copy parameter to DataFrame.to_numpy (#14569)

🚀 Performance improvements

• Avoid loading pandas in from_arrow when array has 0 chunks (#14562)

✨ Enhancements

• Warn on inefficient use of map_elements for additional string functions (#14565)
• Add allow_copy parameter to DataFrame.to_numpy (#14569)
• Improve read_database interop with sqlalchemy Session and various Result objects (#14557)
• Warn on inefficient use of map_elements for temporal attributes/methods (#14529)

🐞 Bug fixes

• Semi-join and multiple keys outer-join did not respect POLARS_MAX_THREADS (#14571)
• Correct sorted flag of chunked gather (#14570)

📖 Documentation

• Fix typo of "Cartesian" product (#14585)
• Mention in contributing guide that PR titles should start with an uppercase letter (#14584)
• Fix markdown newline for rendering function description in VSCode (#14567)

🛠️ Other improvements

• Refactor code coverage workflow (#14563)
• Disable status from code coverage (#14545)

Thank you to all our contributors for making this release possible!
@CBell045, @alexander-beedie, @c-peters, @nameexhaustion, @ritchie46 and @stinodego

Python Polars 0.20.9

🚀 Performance improvements

• use owned arithmetic in horizontal_sum (#14525)

✨ Enhancements

• Add writable flag to DataFrame.to_numpy (#14520)
• flatten aliases (#14512)
• Make formatting more consistent in DOT graphs (#14486)
• add flush operator to streaming operators (#14500)

🐞 Bug fixes

• ensure the streaming dispatcher can replace placeholders in unions (#14537)
• Ensure series are contiguous prior to transpose (#14527)
• write csv header if necessary when finishing sinks (#14518)
• fix logical dtypes in take_chunked (#14517)
• fix binary-offset row-encode (#14514)
• race conditions in OOC writing (#14510)
• Remove is_numeric check on Series.std/var (#14493)
• Error on invalid schema input in DataFrame constructor (#14483)

📖 Documentation

• Fix docstring example for Config.save_to_file (#14533)
• Fix infer_schema_length param description (#14233)
• Clean up grammar and capitalization in README.md (#14488)
• Add examples for Series.bin.ends_with, Series.bin.starts_with, Series.bin.decode, Series.bin.encode. (#14478)

🛠️ Other improvements

• add code coverage CI (#14532)
• Re-enable streaming OOC tests (#14522)
• Use constant for checking Sphinx building (#14502)

Thank you to all our contributors for making this release possible!
@FBruzzesi, @NedJWestern, @c-peters, @dannyfriar, @i-aki-y, @jdanford, @mbuhidar, @mcrumiller, @ritchie46, @stinodego and @taki-mekhalfa

Python Polars 0.20.8

🏆 Highlights

• Implemented tree formatting for LogicalPlan (#14221)

⚠️ Deprecations

• Deprecate positional args in pivot to prepare new functionality (#14428)

🚀 Performance improvements

• Combine small chunks in sinks for streaming pipelines (#14346)
• reduce heap allocs in expression/logical-plan iteration (#14440)
• simplify and speed up cum_sum and cum_prod (#14409)
• simplify negated predicates to improve row groups skipping (#14370)

✨ Enhancements

• Increase verbosity of duplicate column error message (#11899)
• change print to warn in reading csv from python file like object (#14469)
• Raise if pivot would introduce duplicate column names (#14431)
• apply negate in simplify expression pass (#14436)
• restrict more cloud interop to semaphore budget (#14435)
• Implement min/max for categorical dtype (#14112)
• Hide polars.testing.* in pytest stack traces (#14399)
• expose numpy view to integer types (#14405)
• Allow column name input in clip (#14410)
• add boolean rle decoding for parquet (#14403)
• Allow brackets in SQL join conditions (#14263)
• Implemented tree formatting for LogicalPlan (#14221)
• Implement mean_horizontal expression (#14369)
• support decimal comparison (#14338)
• Implements arr.shift (#14298)
• Implements list.n_unique (#14306)
• Do not panic when casting from an empty Series to pl.Decimal (#14330)
• unset WRITEABLE flag in zero-copy output (#14283)
• Support Categorical/Enum in Series.to_numpy (#14275)
• add parametric testing support for the Array dtype (#14265)

🐞 Bug fixes

• don't gc after variadic buffers are written (#14473)
• Increase verbosity of duplicate column error message (#11899)
• Return appropriate data type for duration mean and median (#14376)
• change print to warn in reading csv from python file like object (#14469)
• regression in out-of-core group-by by new string-type (#14464)
• DataFrame.pivot was returning incorrect results when multiple columns were passed to index and one of them was Struct (#14438)
• remove literal Series from projection state (#14437)
• pivot was producing incorrect results when (single) index was Struct (#14308)
• Error on some invalid clip inputs (#14416)
• Series.hist panicking on empty/all-null (#14407)
• rechunk series when apply_lambda (#14406)
• Raise if invalid strategy is passed to map_elements (#14397)
• Require exact checking for Decimals in assertion utils (#14357)
• fix ufunc for unlimited column args (#14328)
• Handle chunked Series in Series.to_numpy (#14341)
• Remove duplicated content in error messages (#8107)
• Fix set_operation if the input is sliced and be broadcast (#14303)
• Wrap par_iter in list.to_struct by POOL.install (#14304)
• Do not panic when casting from an empty Series to pl.Decimal (#14330)
• Preserve name when casting to Enum (#14320)
• list.get does not work on list of decimals (#14276)
• relax precision when up scaling (#14270)
• Allow format object series with registry (#14272)

📖 Documentation

• Update read_database docstring note about getting the connection URI string for sqlalchemy (#14461)
• Fix typo in plugins section (#14402)
• Add debugging section to contributing docs (#10576)
• Define what a 'character' means in slice / len_chars (#14395)
• Clarify behavior of DataFrame.rows_by_key (#14149)
• Fix some typos (#14394)
• Realign file structure of user guide (#14360)
• Rust examples for data structures in user guide (#14339)
• Add deprecation period policy example for post-1.0.0 (#14184)
• Add example for Series.bin.contains (#14297)
• Small clarifications in the contributing guide (#14310)
• Fix capitalization of user guide references (#14291)
• Fix explode docstring mentioning String types (#14285)
• Update deltalake docstrings to new link (#14282)

🛠️ Other improvements

• Ignore unclosed file warnings for now (#14467)
• Raise better error in import timings test (#14441)
• Refactor arg_min/max test case (#14439)
• Skip some OOC tests that fail randomly in the CI (#14434)
• Bump release drafter to v6 (#14429)
• Set specific temp dir for OOC tests (#14420)
• Bump setup-graphviz action to v2 (#14418)
• Minor test refactor (#14404)
• Update make clean command (#14408)
• Internal rename of _or to or_ in PyO3 (same for _xor/_and) (#14393)
• Minor refactor of DataFrame.to_numpy structured code (#14348)
• Update Series.to_numpy to handle Decimal/Time types in Rust (#14296)
• Add test for Series.to_numpy with timezones (#14337)
• Bump ruff version to 0.2.0 (#14294)
• Temporarily fix failing deltalake test (#14288)
• remove dataframe consortium standard api entrypoint (#14279)

Thank you to all our contributors for making this release possible!
@BGR360, @CaselIT, @MarcoGorelli, @migi, @NedJWestern, @Vincenthays, @alexander-beedie, @deanm0000, @dependabot, @dependabot[bot], @engdoreis, @flisky, @grinya007, @itamarst, @janosh, @kalekundert, @lukemanley, @mbuhidar, @mcrumiller, @petrosbar, @r-brink, @rben01, @reswqa, @ritchie46, @stinodego, @taki-mekhalfa and @thomasfrederikhoeck

Python Polars 0.20.7

⚠️ Deprecations

• Rename threadpool_size to thread_pool_size (#14236)

🚀 Performance improvements

• prune parquet row groups when is_not_null is used (#14260)
• Avoid unnecessary copies in Series.to_numpy for boolean/temporal types (#14261)
• use is_between to skip parquet row groups (#14244)
• Use a compression API that is designed for this use case (#11699) (#14194)
• Use UnitVec in polars-plan traversal (#14199)
• use UnitVec in streaming joins (#14197)
• improve ChunkId (#14175)
• improve iteration performance (#14126)
• elide unneeded work in window? (#14108)
• run window functions more in parallel (#14095)
• improve skip row group using statistics condition (#14056)

✨ Enhancements

• add u8/i8/u16/i16 parsers to CSV reader (#14241)
• move F-order data in and out of numpy to polars zero copy (#14259)
• read arrow-c-interface without requiring pyarrow (#14254)
• Implements list.gather_every (#14253)
• Implements prefix/suffix_fields (#14251)
• Change Series.to_numpy to return f64 for Int32/UInt32 Series with nulls instead of f32 (#14240)
• Polish decimal arithmetic (#14172)
• improved read_excel format detection, and support for excel 97-2004 workbooks (#14234)
• Introduce arr.to_struct (#14202)
• Supports map fields name of struct (#14203)
• make IdxVec generic as UnitVec (#14196)
• add new arithmetic kernels (#14026)
• Supports unique and hash_rows for null column (#14111)
• Implement arithmetic operations for Null columns (#14107)
• support pd.Index in from_pandas and elsewhere (#14087)
• Allow renaming expressions with keyword syntax in group_by (#14071)
• raise more informative error message if someone lands on Expr.__bool__ (#14067)
• Adapt extend_constant to function expr architecture and expressify it (#14058)
• add integer negation (#14049)
• list & array measures of dispersion (#13245)
• gc binview when writing ipc (#14035)
• When calling convert_time_zone on time-zone-naive datetime, convert as if converting from UTC (#13960)

🐞 Bug fixes

• deduplicate recursive growables (#14264)
• Fix glimpse overload signature (#14258)
• allow set operations on list of categoricals (#14110)
• any/all_horizontal with single input has incorrect type (#14256)
• load numpy array with np array values #14237 (#14238)
• Make Series.to_numpy on booleans without nulls return bool type (#14239)
• fix ufunc in agg (change __ufunc_array__ so it uses is_elementwise=True parameter) (#14135)
• Fix join validation for String types (#14229)
• enable windows test coverage for read_excel "calamine" (fastexcel) engine (#14171)
• make csv parser more robust to edge cases (#14210)
• Fix for set_operations of binary dtype (#14152)
• fix read_csv date/datetime inference and parsing (#14113)
• don't see files as hive partitions (#14128)
• allow eval on list of categoricals (#14132)
• Forbid casting from Date to Time and vice versa (#14127)
• preserve old naming convention for multi-value pivot (this will change in 1.0 to no longer redundantly have the column name in the middle) (#14120)
• Implements gt/lt cmp for null dtype (#14119)
• ignore comments at beginning of csv if schema provided (#14115)
• fix pivot when multiple columns are passed. Output is now aligned with what tidyverse / pandas.pivot_table would do (#14048)
• multiple read_excel updates (#14039)
• some temporal conversion errors for datetimes earlier than 1970-01-01 (#14050)
• Preserve name when casting from categorical (#14085)
• respect Object dtype designation (#14072)
• fix cse bug when window function is nested (#14070)
• Fix melt panic when there are no value vars (#14057)
• json_encode should respect the logical type (#14063)
• improve skip row group using statistics condition (#14056)
• Raise for .dt.epoch and .dt.timestamp for Duration dtype (#13962)
• handle SliceSink with empty data (#14025)
• Allow Series.to_pandas for categorical types (#14028)
• correct field type schema inference (using read_csv) (#14042)
• Use int formatter for unsigned ints (#14043)

📖 Documentation

• fix code block in user-guide/lazy/schemas (#14228)
• Add visualization page to user guide (#13052)
• Fix typo in contributing guide (#14181)
• Small improvements Ecosystem page (#14176)
• fix code blocks in user-guide/concepts/data-structures (#14146)
• Document that Kleene logic is followed in any_horizontal and all_horizontal (#14148)
• Fix description of return_dtype parameter for map_elements and map_batches (#14114)
• Fix bullet point formatting in CI contributing guide (#14117)
• Add documentation on replacement strings to str.replace and str.replace_all (#13382)
• Replace alternatives page with more objective comparison (#13784)
• Note that only one name operation is allowed per expression (#14075)
• Improve deprecation message of dtype_if_empty param (#14068)
• fix more docstring bullet points (#14065)

🛠️ Other improvements

• Reorganize NumPy interop tests (#14257)
• additional dataframe test coverage (#14243)
• Remove *args in Series.to_numpy (#14248)
• Move metadata utils to meta module (#14230)
• remove unused method DataFrame._from_dicts (#14212)
• make gather_chunked completely generic (#14195)
• Add .cargo directory to .gitignore (#14191)
• take_chunked to polars-ops (#14185)
• Issue a warning when running doctests on Python 3.11 or lower (#14187)
• Run cargo update (#14160)
• merge take kernels (#14137)
• improve From<Ca> -> Vec (#14123)
• hoist boolean -> string cast (#14122)
• remove unused argument (#14014)

Thank you to all our contributors for making this release possible!
@JulianCologne, @MarcoGorelli, @Vincenthays, @Wainberg, @alexander-beedie, @apcamargo, @braaannigan, @c-peters, @deanm0000, @dependabot, @dependabot[bot], @dpinol, @edavisau, @eitsupi, @flisky, @grinya007, @ion-elgreco, @itamarst, @lukemanley, @mcrumiller, @orlp, @r-brink, @reswqa, @ritchie46, @stinodego and @taki-mekhalfa

Rust Polars 0.37.0

🏆 Highlights

• new implementation for String/Binary type. (#13748)

💥 Breaking changes

• Remove DatetimeChunked::convert_time_zone (#14046)
• Rename LiteralValue::to_anyvalue to LiteralValue::to_any_value (#14033)
• Rename drop_columns to drop (#13754)
• Rename pl.count() to pl.len() (#13719)
• Rename row_count_name/row_count_offset parameters in IO functions to row_index_* (#13563)
• Rename with_row_count to with_row_index (#13494)

🚀 Performance improvements

• prune parquet row groups when is_not_null is used (#14260)
• use is_between to skip parquet row groups (#14244)
• Use a compression API that is designed for this use case (#11699) (#14194)
• Use UnitVec in polars-plan traversal (#14199)
• use UnitVec in streaming joins (#14197)
• improve ChunkId (#14175)
• improve iteration performance (#14126)
• elide unneeded work in window? (#14108)
• run window functions more in parallel (#14095)
• improve skip row group using statistics condition (#14056)
• improve string/binary reverse performance (#14016)
• optimize DataFrame.describe by presorting columns (#13822)
• elide redundant bound checks. (#13909)
• speedup boolean filter (#13905)
• speedup binview filter (#13902)
• improve binview filter (#13878)
• apply string view GC more conservatively (#13850)
• add optimized BinaryViewArray comparison kernels (#13839)
• lazy cache binview bytes len (#13830)
• fast-path for eager int_range (#13811)
• Optimize arr.sum for inner non-null bool (#13800)
• directly embed data ptr in Buffer (#13744)
• elide parallelism restriction on generic rolling expressions (#13662)
• ensure time groups are parallelized (#13660)
• do not eagerly compute bitcount (#13562)
• optimise SQL engine string concat (#13499)
• remove lifetime requirement from CategoricalChunkedBuilder (#13319)

✨ Enhancements

• add u8/i8/u16/i16 parsers to CSV reader (#14241)
• Implements list.gather_every (#14253)
• Implements prefix/suffix_fields (#14251)
• Polish decimal arithmetic (#14172)
• Introduce arr.to_struct (#14202)
• Supports map fields name of struct (#14203)
• make IdxVec generic as UnitVec (#14196)
• add new arithmetic kernels (#14026)
• Supports unique and hash_rows for null column (#14111)
• Implement arithmetic operations for Null columns (#14107)
• Add strict/non-strict construction of Boolean/Binary series (#14073)
• Improve Series::from_any_values logic (#14052)
• Adapt extend_constant to function expr architecture and expressify it (#14058)
• add integer negation (#14049)
• list & array measures of dispersion (#13245)
• gc binview when writing ipc (#14035)
• When calling convert_time_zone on time-zone-naive datetime, convert as if converting from UTC (#13960)
• DataFrame supports explode by array column (#13958)
• improve binary formatting (#13981)
• preserve Enum information when going to IPC (#13943)
• support kwargs in plugin 'field' functions and raise error on unsupported binview layout (#13944)
• support cast decimal to utf8 (#13829)
• add SQL support for timestamp precision modifier (#13936)
• support negative indexing and expressions for LEFT, RIGHT and SUBSTR SQL string funcs (#13888)
• Introduce explode for ArrayNameSpace (#13923)
• raise better error message for .dt.time on Date column (#13932)
• List set_operations supports float (#13920)
• Add ignore_nulls for arr.join (#13919)
• register 'set_sorted' as batch/elementwise (#13896)
• move Enum/Categorical categories to binview (#13882)
• Add ignore_nulls for list.join (#13701)
• Add ignore_nulls for pl.concat_str (#13877)
• fix parquet for binview (#13873)
• support mmap for binview in OOC (#13872)
• implement ffi for binview (#13871)
• Support zero fill null strategy for binary and string columns (#13869)
• Implement/fix unary minus operator -pl.col(...) (#13776)
• extend SQL EXTRACT with "century", "millennium", and "timezone" parts (#13634)
• fix binview ipc format (#13842)
• add SQL support for numeric and/or decimal types (#13739)
• improve panic message (#13836)
• Expressify str.zfill (#13790)
• new implementation for String/Binary type. (#13748)
• Add nulls_last for Series.sort (#13794)
• Impl count_matches for array namespace (#13675)
• Add nulls_last for list/array.sort (#13795)
• Rename drop_columns to drop (#13754)
• convert fixed-offset timezones to respective Etc timezone from time zone database (#13738)
• Expressify str.slice (#13747)
• implement binview for polars-row (#13736)
• implement binview for polars-json (#13737)
• add architecture for polars-flavored IPC (#13734)
• implement binview comparison kernels (#13715)
• raise default frame/series repr height from 8 to 10 (#13699)
• write parquet ColumnOrder (#13672)
• Impl contains for ArrayNameSpace (#13638)
• improve rolling() expression formatting (#13657)
• Implement is_between in Rust (#11945)
• Expressify pattern of str.extract (#13607)
• Impl join for ArrayNameSpace (#13586)
• add SQL engine support for string cast to json (#13624)
• add SQL engine support for EXTRACT and DATE_PART (#13603)
• add BinaryView to parquet writer/reader. (#13489)
• add SQL engine support for POSITION and STRPOS (#13585)
• is_in support for array dtype (#13559)
• add new str.find expression, returning the index of a regex pattern or literal substring (#13561)
• add SQL engine support for LIKE and ILIKE pattern matching (#13522)
• improve hive partition pruning (#13358) (#13426)
• don't rechunk by default in lazy scans (#13518)
• Add cum_count expression function (#13478)
• add SQL engine support for IF control flow function (#13491)
• add SQL engine support for MOD function (#13502)
• return datetime for datetime mean & median (#13417)
• add SQL engine support for CONCAT_WS string function (#13483)
• BinaryView/Utf8View IPC support (#13464)
• Implement wasm Pool::scope (#13476)
• add SQL engine support for RIGHT and REVERSE string functions (#13461)
• implement BinaryView and Utf8View in polars-arrow (#13243)
• add SQL engine support for variadic string CONCAT function (#13428)
• add support for AND in SQL join-clause context (#13242)
• Impl ordering ops for array namespace (#13414)
• add SQL engine support for REPLACE string function (#13431)
• add SQL engine support for SIGN function (#13429)
• add SQL engine support for IFNULL function (#13432)
• additional SQL support for bytes, bit, and hex literals (#13389)

🐞 Bug fixes

• deduplicate recursive growables (#14264)
• Fix glimpse overload signature (#14258)
• allow set operations on list of categoricals (#14110)
• any/all_horizontal with single input has incorrect type (#14256)
• load numpy array with np array values #14237 (#14238)
• Fix join validation for String types (#14229)
• make csv parser more robust to edge cases (#14210)
• Fix for set_operations of binary dtype (#14152)
• fix read_csv date/datetime inference and parsing (#14113)
• don't see files as hive partitions (#14128)
• allow eval on list of categoricals (#14132)
• add missing conditional compile flag for StringFunction::Find (#14129)
• Forbid casting from Date to Time and vice versa (#14127)
• preserve old naming convention for multi-value pivot (this will change in 1.0 to no longer redundantly have the column name in the middle) (#14120)
• Implements gt/lt cmp for null dtype (#14119)
• ignore comments at beginning of csv if schema provided (#14115)
• fix pivot when multiple columns are passed. Output is now aligned with what tidyverse / pandas.pivot_table would do (#14048)
• some temporal conversion errors for datetimes earlier than 1970-01-01 (#14050)
• Preserve name when casting from categorical (#14085)
• fix cse bug when window function is nested (#14070)
• Fix melt panic when there are no value vars (#14057)
• json_encode should respect the logical type (#14063)
• improve skip row group using statistics condition (#14056)
• Raise for .dt.epoch and .dt.timestamp for Duration dtype (#13962)
• handle SliceSink with empty data (#14025)
• correct field type schema inference (using read_csv) (#14042)
• Map AnyValue::Null to datatype Null (#14045)
• Use int formatter for unsigned ints (#14043)
• quick fix for multiple chunks binary reverse (#14024)
• count matches on list categorical (#14021)
• list.min/max with empty and/or None elements (#14018)
• allow get access to list of categoricals (#14015)
• Fix casting from categorical to numeric (#13957)
• read_csv preserve whitespace and newlines (#13934)
• append decimal with different scale (#13977)
• Allow casting integer types to Enum (#13955)
• arg_min/max on categoricals should respect ordering (#13998)
• serialize decimal type (#13997)
• check input type for arr/list.contains (#13959)
• Allow dtype merge when inner dtype is enum (#13938)
• recurse less in streaming shared sinks (#13930)
• ensure order is preserved if streaming from different sources (#13922)
• Fix is_not_null for Struct columns (#13921)
• make 100 * pl.col(pl.Boolean).mean() work (#13725)
• allow extract of numeric from str AnyValue (#13865)
• single-element .dt.time() and .dt.date() should always preserve sortedness (#13808)
• prune emtpy chunks before set operations (#13898)
• treat null columns as zero in sum_horizontal (#13880)
• include null count in rolling window validity with min_periods (#13863)
• don't return NaN as free memory fraction (#13860)
• parquet hybrid RLE encoding did not always align to bit width (#13883)
• Add ignore_nulls for list.join (#13701)
• .dt.time() was panicking for datetimes prior to unix epoch (#13812)
• Correct err message of check_map_output_len (#13854)
• allow list creation of decimals (#13851)
  ...

Python Polars 0.20.6

🏆 Highlights

• new implementation for String/Binary type. (#13748)

⚠️ Deprecations

• Deprecate dtype_if_empty parameter for Series constructor (#13976)

🚀 Performance improvements

• improve string/binary reverse performance (#14016)
• add "calamine" support to read_excel, using fastexcel (~8-10x speedup) (#14000)
• optimize DataFrame.describe by presorting columns (#13822)
• elide redundant bound checks. (#13909)
• speedup boolean filter (#13905)
• speedup binview filter (#13902)
• allow python threads in read_ functions (#13886)
• improve binview filter (#13878)
• apply string view GC more conservatively (#13850)
• add optimized BinaryViewArray comparison kernels (#13839)
• lazy cache binview bytes len (#13830)
• fast-path for eager int_range (#13811)
• Optimize arr.sum for inner non-null bool (#13800)

✨ Enhancements

• Add UnstableWarning for unstable functionality (#13948)
• DataFrame supports explode by array column (#13958)
• add "calamine" support to read_excel, using fastexcel (~8-10x speedup) (#14000)
• improve binary formatting (#13981)
• preserve Enum information when going to IPC (#13943)
• support calling describe on a LazyFrame (#13982)
• support kwargs in plugin 'field' functions and raise error on unsupported binview layout (#13944)
• support cast decimal to utf8 (#13829)
• add SQL support for timestamp precision modifier (#13936)
• support negative indexing and expressions for LEFT, RIGHT and SUBSTR SQL string funcs (#13888)
• Introduce explode for ArrayNameSpace (#13923)
• unify Series/DataFrame describe code (#13720)
• raise better error message for .dt.time on Date column (#13932)
• List set_operations supports float (#13920)
• Add ignore_nulls for arr.join (#13919)
• register 'set_sorted' as batch/elementwise (#13896)
• move Enum/Categorical categories to binview (#13882)
• Add ignore_nulls for list.join (#13701)
• Add ignore_nulls for pl.concat_str (#13877)
• Align int_range and int_ranges signatures (#13867)
• fix parquet for binview (#13873)
• support mmap for binview in OOC (#13872)
• implement ffi for binview (#13871)
• Support zero fill null strategy for binary and string columns (#13869)
• allow df.rename and lf.rename to take a renaming function (#13708)
• Implement/fix unary minus operator -pl.col(...) (#13776)
• extend SQL EXTRACT with "century", "millennium", and "timezone" parts (#13634)
• fix binview ipc format (#13842)
• add SQL support for numeric and/or decimal types (#13739)
• improve panic message (#13836)
• Expressify str.zfill (#13790)
• new implementation for String/Binary type. (#13748)
• Add typing to hvplot plot namespace (#13813)
• Add nulls_last for Series.sort (#13794)
• allow ftp URLs, improve URL check (#13781)

🐞 Bug fixes

• count matches on list categorical (#14021)
• list.min/max with empty and/or None elements (#14018)
• Make to_pandas() work for Dataframe and Series with dtype Object (#13910)
• raise for pl.concat(how="align") when no columns are shared between frames (#13941)
• Fix casting from categorical to numeric (#13957)
• read_csv preserve whitespace and newlines (#13934)
• omit implicit 'site' from import-timing test (#14009)
• append decimal with different scale (#13977)
• Use date_as_object=False as default for Series.to_pandas (just like DataFrame.to_pandas) (#13984)
• serialize decimal type (#13997)
• check input type for arr/list.contains (#13959)
• Fix max_colname_length formatting in glimpse() (#13969)
• Allow dtype merge when inner dtype is enum (#13938)
• recurse less in streaming shared sinks (#13930)
• ensure order is preserved if streaming from different sources (#13922)
• Fix is_not_null for Struct columns (#13921)
• convert object-dtyped NumPy str/bytes arrays to pl.String/pl.Binary instead of pl.Object (#13712)
• allow extract of numeric from str AnyValue (#13865)
• single-element .dt.time() and .dt.date() should always preserve sortedness (#13808)
• prune emtpy chunks before set operations (#13898)
• treat null columns as zero in sum_horizontal (#13880)
• include null count in rolling window validity with min_periods (#13863)
• Fix interchange protocol for new String type (#13881)
• parquet hybrid RLE encoding did not always align to bit width (#13883)
• Add ignore_nulls for list.join (#13701)
• .dt.time() was panicking for datetimes prior to unix epoch (#13812)
• allow list creation of decimals (#13851)
• ensure kwargs filter behaviour matches docstring (expect equivalence with eq) (#13864)
• Implement abs for Decimal, error on Date/Time/Datetime (#13821)
• rolling nested groups deadlock (#13835)
• gather_every should work on agg context (#13810)
• Fix segfault of is_in (#13814)
• don't panic on full null qcut (#13815)
• validate operator arithmetic with None, fix Series edge-case (#13780)

📖 Documentation

• Add missing doc entries (#14006)
• add missing len to rst file (#13999)
• Improve structure of user guide (#13951)
• Improve structure of user guide (#13639)
• Introduce ecosystem page in user guide (#13903)
• Mention deltalake write support in README (#13890)
• use proper argument names in the code blocks of api.rst (#13866)

🛠️ Other improvements

• make Enums an actual datatype (#14011)
• omit implicit 'site' from import-timing test (#14009)
• Constructor improvements - part 1 (#14001)
• Add glimpse test (#13979)
• Move PyO3 ChunkedArray conversion logic into its own module (#13973)
• Fix xdist streaming group (#13974)
• Fix spurious test failures (#13961)
• minor describe tidy-up, and slight rewording of some Exception docstrings (#13942)
• Fix pip warning filter return code (#13935)
• Minor refactor of PyO3 conversions module (#13929)
• move filter to polars-compute (#13897)
• Revert pandas warning filter (#13893)
• Make functions in expr/general non-anonymous (#13832)
• Fix doctests (#13831)
• Refactor Python release workflow (#13807)

Thank you to all our contributors for making this release possible!
@ByteNybbler, @JulianCologne, @MarcoGorelli, @Wainberg, @alexander-beedie, @c-peters, @dependabot, @dependabot[bot], @edavisau, @flisky, @ion-elgreco, @itamarst, @jacksonthall22, @kstoneriv3, @mcrumiller, @mkucijan, @nameexhaustion, @orlp, @petrosbar, @r-brink, @reswqa, @ritchie46, @stinodego, @taki-mekhalfa, @thomasaarholt and @valorien

Python Polars 0.20.6-rc.1

🏆 Highlights

• new implementation for String/Binary type. (#13748)

🚀 Performance improvements

• speedup boolean filter (#13905)
• speedup binview filter (#13902)
• allow python threads in read_ functions (#13886)
• improve binview filter (#13878)
• apply string view GC more conservatively (#13850)
• add optimized BinaryViewArray comparison kernels (#13839)
• lazy cache binview bytes len (#13830)
• fast-path for eager int_range (#13811)
• Optimize arr.sum for inner non-null bool (#13800)

✨ Enhancements

• register 'set_sorted' as batch/elementwise (#13896)
• move Enum/Categorical categories to binview (#13882)
• Add ignore_nulls for list.join (#13701)
• Add ignore_nulls for pl.concat_str (#13877)
• Align int_range and int_ranges signatures (#13867)
• fix parquet for binview (#13873)
• support mmap for binview in OOC (#13872)
• implement ffi for binview (#13871)
• Support zero fill null strategy for binary and string columns (#13869)
• allow df.rename and lf.rename to take a renaming function (#13708)
• Implement/fix unary minus operator -pl.col(...) (#13776)
• extend SQL EXTRACT with "century", "millennium", and "timezone" parts (#13634)
• fix binview ipc format (#13842)
• add SQL support for numeric and/or decimal types (#13739)
• improve panic message (#13836)
• Expressify str.zfill (#13790)
• new implementation for String/Binary type. (#13748)
• Add typing to hvplot plot namespace (#13813)
• Add nulls_last for Series.sort (#13794)
• allow ftp URLs, improve URL check (#13781)

🐞 Bug fixes

• prune emtpy chunks before set operations (#13898)
• treat null columns as zero in sum_horizontal (#13880)
• include null count in rolling window validity with min_periods (#13863)
• Fix interchange protocol for new String type (#13881)
• parquet hybrid RLE encoding did not always align to bit width (#13883)
• Add ignore_nulls for list.join (#13701)
• .dt.time() was panicking for datetimes prior to unix epoch (#13812)
• allow list creation of decimals (#13851)
• ensure kwargs filter behaviour matches docstring (expect equivalence with eq) (#13864)
• Implement abs for Decimal, error on Date/Time/Datetime (#13821)
• rolling nested groups deadlock (#13835)
• gather_every should work on agg context (#13810)
• Fix segfault of is_in (#13814)
• don't panic on full null qcut (#13815)
• validate operator arithmetic with None, fix Series edge-case (#13780)

📖 Documentation

• Mention deltalake write support in README (#13890)
• use proper argument names in the code blocks of api.rst (#13866)

🛠️ Other improvements

• move filter to polars-compute (#13897)
• Revert pandas warning filter (#13893)
• Make functions in expr/general non-anonymous (#13832)
• Fix doctests (#13831)
• Refactor Python release workflow (#13807)

Thank you to all our contributors for making this release possible!
@ByteNybbler, @MarcoGorelli, @Wainberg, @alexander-beedie, @dependabot, @dependabot[bot], @edavisau, @flisky, @ion-elgreco, @itamarst, @kstoneriv3, @mcrumiller, @mkucijan, @nameexhaustion, @orlp, @reswqa, @ritchie46, @stinodego, @taki-mekhalfa and @thomasaarholt

Python Polars 0.20.5

⚠️ Deprecations

• Deprecate default delimiter value for str.concat (#13690)
• Rename pl.count() to pl.len() (#13719)
• Deprecate dt.with_time_unit in favor of cast(pl.Int64).cast(pl.Datetime(time_unit, time_zone)) (#13667)

🚀 Performance improvements

• directly embed data ptr in Buffer (#13744)

✨ Enhancements

• Impl count_matches for array namespace (#13675)
• Add nulls_last for list/array.sort (#13795)
• convert fixed-offset timezones to respective Etc timezone from time zone database (#13738)
• allow read_excel to load from remote http locations (#13753)
• Expressify str.slice (#13747)
• implement binview for polars-row (#13736)
• implement binview for polars-json (#13737)
• add architecture for polars-flavored IPC (#13734)
• implement binview comparison kernels (#13715)
• raise default frame/series repr height from 8 to 10 (#13699)

🐞 Bug fixes

• do not read data for zero-length compressed buffer (#13791)
• Fix the non-null test of transpose (#13783)
• Raise error instead of panic when joining on wildcard/nth (#13742)
• str.concat correctly ignore single null value (#13751)
• Selectors by_name and by_dtype should allow empty list as input (#11024)
• Keep Series attributes docstrings when read by Sphinx (#13731)
• fix error message when creating DataFrame from 0-dimensional NumPy array (#13729)
• support corr() for single-column DataFrames (#13728)
• Use NonZeroUsize for batch_size parameter in write_csv/sink_csv/scan_ndjson (#13726)
• error instead of panicking in sql if empty function (#13691)

📖 Documentation

• Fix typo in deprecation message of with_row_count (#13793)
• Fix incorrect "coming from pandas" syntax (#13767)
• Improve streaming section of the user guide (#13750)
• improve n_unique and approx_n_unique docs (#13752)
• add missing Series.str.find reference (#13717)
• Be more explicit about behaviour in str.strip_chars / strip_chars_start / strip_chars_end docstrings (#13697)
• Add doc example for datetime_ranges (#13695)
• document %A and %B to get day name and month name (#13678)

🛠️ Other improvements

• Make pl.duration non-anonymous (#13762)
• Add test for describe on Object types (#13689)
• Only run bytecode parser CI workflow for Python 3.9/3.10 (#13664)

Thank you to all our contributors for making this release possible!
@29antonioac, @MarcoGorelli, @NedJWestern, @Wainberg, @alexander-beedie, @cgevans, @henryharbeck, @langestefan, @orlp, @petrosbar, @r-brink, @reswqa, @ritchie46, @stinodego and @universalmind303