Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

REGR: write compressed pickle files with protocol=5 #39376

Merged
merged 2 commits into from
Jan 26, 2021
Merged

REGR: write compressed pickle files with protocol=5 #39376

merged 2 commits into from
Jan 26, 2021

Conversation

twoertwein
Copy link
Member

@twoertwein twoertwein commented Jan 24, 2021 •
edited
Loading

I don't know why the change from #37056 fails in combination with protocol=5 and compression. @ig248

An alternative fix would be to have get_handle insert a write buffer for all compressions (similar to #38728) to avoid if/else in format specific to_*

@twoertwein twoertwein added the IO Pickle read_pickle, to_pickle label Jan 24, 2021
@jreback jreback added this to the 1.2.2 milestone Jan 24, 2021
jreback
jreback requested changes Jan 24, 2021
View reviewed changes
pandas/io/pickle.py Outdated Show resolved Hide resolved
@jreback jreback changed the title REGR: wirte compressed pickle files with protocol=5 REGR: write compressed pickle files with protocol=5 Jan 24, 2021
jreback
jreback approved these changes Jan 26, 2021
View reviewed changes
pandas/tests/io/test_pickle.py Show resolved Hide resolved
@jreback @simonjayhawkins
Update doc/source/whatsnew/v1.2.2.rst
4411ea8 
Co-authored-by: Simon Hawkins <simonjayhawkins@gmail.com>
@jreback jreback merged commit 5c2fb96 into pandas-dev:master Jan 26, 2021
@jreback
Copy link
Contributor

jreback commented Jan 26, 2021

thanks @twoertwein

@jreback
Copy link
Contributor

jreback commented Jan 26, 2021

@meeseeksdev backport 1.2.x

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Jan 26, 2021
@twoertwein @meeseeksmachine
Backport PR pandas-dev#39376: REGR: write compressed pickle files wit…
e4c5978 
…h protocol=5
@meeseeksmachine meeseeksmachine mentioned this pull request Jan 26, 2021
Merged
simonjayhawkins pushed a commit that referenced this pull request Jan 27, 2021
@meeseeksmachine @twoertwein
Backport PR #39376: REGR: write compressed pickle files with protocol…
06b4887 
…=5 (#39428)

Co-authored-by: Torsten Wörtwein <twoertwein@users.noreply.github.com>
@twoertwein twoertwein deleted the compressed_pickle branch February 8, 2021 00:43
@PaulFlanaganGenscape PaulFlanaganGenscape mentioned this pull request Feb 10, 2021
Closed
@jakirkham jakirkham mentioned this pull request Oct 21, 2022
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback approved these changes

@simonjayhawkins simonjayhawkins simonjayhawkins left review comments

Assignees
No one assigned
Labels
IO Pickle read_pickle, to_pickle
Projects
None yet
Milestone
1.2.2
Development

Successfully merging this pull request may close these issues.

BUG:to_pickle() raises TypeError when compressing large dataframe
3 participants
@twoertwein @jreback @simonjayhawkins