Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add pandas ExtensionArray for storing homogeneous ragged arrays #687

Merged
merged 48 commits into from
Mar 1, 2019
Merged

Add pandas ExtensionArray for storing homogeneous ragged arrays #687

merged 48 commits into from
Mar 1, 2019

Commits on Jan 12, 2019

  1. Fix for pandas 0.24.0rc1

    @jonmmease
    jonmmease committed Jan 12, 2019
    Configuration menu
    Copy the full SHA
    fc148de View commit details
    Browse the repository at this point in the history

Commits on Jan 13, 2019

  1. Initial RaggedArray implementation

    @jonmmease
    jonmmease committed Jan 13, 2019
    Configuration menu
    Copy the full SHA
    864a235 View commit details
    Browse the repository at this point in the history

  2. Add the extension test suite provided by pandas and fix tests. 

    Something in the fixes for these tests removed the need to for the ._values hack!
    @jonmmease
    jonmmease committed Jan 13, 2019
    Configuration menu
    Copy the full SHA
    440e207 View commit details
    Browse the repository at this point in the history

  3. Import register_extension_dtype from pandas public location

    @jonmmease
    jonmmease committed Jan 13, 2019
    Configuration menu
    Copy the full SHA
    2f18587 View commit details
    Browse the repository at this point in the history

Commits on Jan 14, 2019

  1. Fix copy/paste error

    @jonmmease
    jonmmease committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    5f46b8e View commit details
    Browse the repository at this point in the history

  2. KeyError -> IndexError

    @jonmmease
    jonmmease committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    a6b3c27 View commit details
    Browse the repository at this point in the history

  3. Document, validate, and test fast-path RaggedArray construction 

    from start_indices, flat_array, and mask arrays
    @jonmmease
    jonmmease committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    fbc5065 View commit details
    Browse the repository at this point in the history

  4. Support indexing RaggedArray with a list

    @jonmmease
    jonmmease committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    527e9d6 View commit details
    Browse the repository at this point in the history

  5. Create single RaggedDtype() instance per RaggedArray

    @jonmmease
    jonmmease committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    8d1c34b View commit details
    Browse the repository at this point in the history

  6. Allow astype() to cast RaggedArray to other extension array types

    @jonmmease
    jonmmease committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    dad6cc2 View commit details
    Browse the repository at this point in the history

  7. Allow RaggedArray constructor to accept a RaggedArray to copy

    @jonmmease
    jonmmease committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    fff0c3e View commit details
    Browse the repository at this point in the history

  8. Remove mask property and consider missing to be equivalent to empty

    @jonmmease
    jonmmease committed Jan 14, 2019
    Configuration menu
    Copy the full SHA
    478b655 View commit details
    Browse the repository at this point in the history

Commits on Jan 15, 2019

  1. More test fixes for [] being null

    @jonmmease
    jonmmease committed Jan 15, 2019
    Configuration menu
    Copy the full SHA
    9d84b3c View commit details
    Browse the repository at this point in the history

  2. Update datashader/datatypes.py 

    Co-Authored-By: jonmmease <jon.mease@gmail.com>
    @jbednar @jonmmease
    jbednar and jonmmease authored Jan 15, 2019
    Configuration menu
    Copy the full SHA
    d71f866 View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2019

  1. Add RaggedElement wrapper class for internal pandas operations 

    Add additional ExtensionArray test suites
    @jonmmease
    jonmmease committed Jan 16, 2019
    Configuration menu
    Copy the full SHA
    4cd7b4c View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2019

  1. Override fillna is RaggedArray and enable test

    @jonmmease
    jonmmease committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    16aff67 View commit details
    Browse the repository at this point in the history

  2. Add vectorized equality operators

    @jonmmease
    jonmmease committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    5772ade View commit details
    Browse the repository at this point in the history

  3. pass start_indices and flat_array arrays as args to _validate_ragged_… 

    …properties

rather than dict key/value pairs.
    @jonmmease
    jonmmease committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    939405b View commit details
    Browse the repository at this point in the history

  4. Add copy arg to RaggedArray constructor

    @jonmmease
    jonmmease committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    7f355d2 View commit details
    Browse the repository at this point in the history

  5. +=

    @jonmmease
    jonmmease committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    9e44946 View commit details
    Browse the repository at this point in the history

  6. Fix missing return

    @jonmmease
    jonmmease committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    a52728a View commit details
    Browse the repository at this point in the history

  7. Parameterize RaggedDtype by element type 

    This way the element dtype can be specified in the ragged datatype string

e.g.

>>> pd.Series([[1, 2], [2, 3, 4], None], dtype='Ragged[uint16]')
Out[13]:
0      [1 2]
1    [2 3 4]
2        NaN
dtype: Ragged[uint16]
    @jonmmease
    jonmmease committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    75f914d View commit details
    Browse the repository at this point in the history

  8. Remove tuple conversions in RaggedElement

    @jonmmease
    jonmmease committed Jan 17, 2019
    Configuration menu
    Copy the full SHA
    32f4a3c View commit details
    Browse the repository at this point in the history

Commits on Jan 18, 2019

  1. Designate _RaggedElement as an internal class

    @jonmmease
    jonmmease committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    27403a7 View commit details
    Browse the repository at this point in the history

  2. numba jit utility functions

    @jonmmease
    jonmmease committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    e93c24d View commit details
    Browse the repository at this point in the history

  3. Don't auto-import RaggedArray unless pandas is at least version 0.24.0

    @jonmmease
    jonmmease committed Jan 18, 2019
    Configuration menu
    Copy the full SHA
    3fda786 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2019

  1. wrap _compute_*_bounds static methods with compute_*_bounds methods 

    This allows subclasses to override how the DataFrame is used to compute the bounds
    @jonmmease
    jonmmease committed Jan 20, 2019
    Configuration menu
    Copy the full SHA
    04453ce View commit details
    Browse the repository at this point in the history

  2. Small refactor to remove the need for a specialized _PolygonLike glyp… 

    …h_dispatch
    @jonmmease
    jonmmease committed Jan 20, 2019
    Configuration menu
    Copy the full SHA
    642a858 View commit details
    Browse the repository at this point in the history

  3. Refactor to extract required_columns glyph method

    @jonmmease
    jonmmease committed Jan 20, 2019
    Configuration menu
    Copy the full SHA
    97bccf5 View commit details
    Browse the repository at this point in the history

  4. Initial cvs.lines and LinesXY glyph

    @jonmmease
    jonmmease committed Jan 20, 2019
    Configuration menu
    Copy the full SHA
    2860511 View commit details
    Browse the repository at this point in the history

  5. WIP of LinesRagged type

    @jonmmease
    jonmmease committed Jan 20, 2019
    Configuration menu
    Copy the full SHA
    d7cf092 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2019

  1. Merge branch 'master' into enh_ragged 

    # Conflicts:
#	datashader/glyphs.py
#	datashader/tests/test_pandas.py
    @jonmmease
    jonmmease committed Feb 7, 2019
    Configuration menu
    Copy the full SHA
    e781a0f View commit details
    Browse the repository at this point in the history

  2. Remove unused canvas.lines method

    @jonmmease
    jonmmease committed Feb 7, 2019
    Configuration menu
    Copy the full SHA
    ea08fd1 View commit details
    Browse the repository at this point in the history

Commits on Feb 8, 2019

  1. Add RaggedArray line aggregation support for pandas

    @jonmmease
    jonmmease committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    1b02b0d View commit details
    Browse the repository at this point in the history

  2. Dask ragged array support

    @jonmmease
    jonmmease committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    2314311 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' into enh_ragged

    @jonmmease
    jonmmease committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    2078aad View commit details
    Browse the repository at this point in the history

  4. flake8

    @jonmmease
    jonmmease committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    f4a40eb View commit details
    Browse the repository at this point in the history

  5. Add validation for LinesAxis1Ragged

    @jonmmease
    jonmmease committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    59b0b3a View commit details
    Browse the repository at this point in the history

  6. Exception handling on import for pandas < 0.24

    @jonmmease
    jonmmease committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    c48429e View commit details
    Browse the repository at this point in the history

  7. Add pandas >=0.24.1 as testing dependency so that we can test RaggedA… 

    …rray support
    @jonmmease
    jonmmease committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    cdecd85 View commit details
    Browse the repository at this point in the history

  8. absolute import

    @jonmmease
    jonmmease committed Feb 8, 2019
    Configuration menu
    Copy the full SHA
    7c8b953 View commit details
    Browse the repository at this point in the history

Commits on Feb 9, 2019

  1. specify that int lists should cast to int64 numpy arrays 

    To address AppVeyor failures
    @jonmmease
    jonmmease committed Feb 9, 2019
    Configuration menu
    Copy the full SHA
    c846f0c View commit details
    Browse the repository at this point in the history

Commits on Feb 23, 2019

  1. Merge branch 'master' into enh_ragged

    @jonmmease
    jonmmease committed Feb 23, 2019
    Configuration menu
    Copy the full SHA
    4145fb9 View commit details
    Browse the repository at this point in the history

Commits on Feb 24, 2019

  1. Remove parameterized args from skipped tests 

    No reason to skip every combination, and this was causing pytest-xdist
to throw an internal error when running tests in parallel
    @jonmmease
    jonmmease committed Feb 24, 2019
    Configuration menu
    Copy the full SHA
    cad7d0a View commit details
    Browse the repository at this point in the history

  2. Add Dask optimized bounds calculations for ragged list glyph

    @jonmmease
    jonmmease committed Feb 24, 2019
    Configuration menu
    Copy the full SHA
    89d1d51 View commit details
    Browse the repository at this point in the history

Commits on Feb 28, 2019

  1. Apply suggestions from code review 

    Co-Authored-By: jonmmease <jon.mease@gmail.com>
    @jbednar @jonmmease
    jbednar and jonmmease authored Feb 28, 2019
    Configuration menu
    Copy the full SHA
    92eaab2 View commit details
    Browse the repository at this point in the history

  2. Refer to parent docstrings rather than duplicate

    @jonmmease
    jonmmease committed Feb 28, 2019
    Configuration menu
    Copy the full SHA
    1538909 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2019

  1. Remove docstring references

    @jonmmease
    jonmmease committed Mar 1, 2019
    Configuration menu
    Copy the full SHA
    c42f0df View commit details
    Browse the repository at this point in the history