Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor code base #26

Merged
merged 24 commits into from
Mar 9, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Changelog

## Version 2.0.0 - 2021-03-09
### Added
- `BaseCompresser` class and subclasses to handle compression streams.
- `BasePicklerIO` class and subclasses to handle dump and load of python objects.
- Registry of `BaseCompresser` classes with the mappings that relate them to compression names, file extensions and default read and write modes.
- Registry of `BasePicklerIO` classes with the mappings that relate them to pickler names.
- Singledispatch functions `compress_pickle.io.base.compress_and_serialize` and `compress_pickle.io.base.uncompress_and_unserialize` that implement the core input/output functionality.
- `pickler_method` and `pickler_kwargs` arguments for all `dump`, `dumps`, `load` and `loads`. This now enables users to choose the pickler backend.
- Added support for several pickler backends such as `marshal`, `dill` and `cloudpickle`.
- Added the ability for compression methods to support more than one file extension. Now `bz2` and `lzma` also are applied for the `.bz2` and `.xz` extensions respectively.

### Removed
- The `optimize` argument of the `dump` and `dumps` function was removed. To use the same functionality, you should pass the argument `pickler_method="optimized_pickle"` to `dump` and `dumps`.
- The argument `unhandled_extensions` has been removed from `dump`, `dumps`, `load` and `loads`. The current behavior is to always raise an exception when the compression protocol is unknown.
- All functions from the `compress_pickle.utils` module with the exception of `_stringify_path`. All of the logic implemented by the old functions has been refactored into the registries, compressers and picklerIO classes.
11 changes: 7 additions & 4 deletions azure-pipelines.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ stages:
test.unittest: True
test.style: True
test.docs: True
Python39:
python.version: '3.9'
imageName: 'ubuntu-latest'
test.unittest: True
test.style: True
test.docs: True
Python37-Windows:
python.version: '3.7'
imageName: 'windows-latest'
Expand All @@ -45,10 +51,7 @@ stages:
- bash: |
python -m pip install --upgrade pip
pip install -r requirements.txt
if [[ $(test.style) = "True" ]] || [[ $(test.docs) = "True" ]]
then
python -m pip install -r requirements-dev.txt
fi
python -m pip install -r requirements-dev.txt
python -m pip install .[full]
displayName: 'Install dependencies'

Expand Down
17 changes: 5 additions & 12 deletions compress_pickle/__init__.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,7 @@
from .compress_pickle import dump, load, dumps, loads
from .utils import (
get_known_compressions,
validate_compression,
get_default_compression_mapping,
get_compression_write_mode,
get_compression_read_mode,
set_default_extensions,
infer_compression_from_filename,
preprocess_path,
open_compression_stream,
)
from . import utils
from . import compressers
from . import picklers
from . import io

__version__ = "1.2.0"
__version__ = "2.0.0"
Loading