Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pandas module for Pandas integration, fixes 1826 #1919

Merged
merged 4 commits into from
Feb 24, 2022

Conversation

jmao-denver
Copy link
Contributor

@jmao-denver jmao-denver commented Feb 1, 2022

  1. to_pandas() and to_table() to convert between DH tables and DF
  2. test cases showed some lingering issues with DH NULL handling
  3. performance is potentially an issue that has yet been addressed.

fixes #1826

pyintegration/deephaven2/column.py Outdated Show resolved Hide resolved
pyintegration/deephaven2/dherror.py Outdated Show resolved Hide resolved
pyintegration/tests/test_dtypes.py Show resolved Hide resolved
pyintegration/deephaven2/dtypes.py Outdated Show resolved Hide resolved
pyintegration/deephaven2/dtypes.py Outdated Show resolved Hide resolved
pyintegration/tests/test_pandas.py Outdated Show resolved Hide resolved
pyintegration/tests/test_pandas.py Outdated Show resolved Hide resolved
pyintegration/tests/test_pandas.py Show resolved Hide resolved
pyintegration/tests/test_pandas.py Outdated Show resolved Hide resolved
pyintegration/tests/test_pandas.py Outdated Show resolved Hide resolved
1. to_pandas() and to_table() to convert between DH tables and DF
2. test cases showed some lingering issues with DH NULL handling
3. performance is potentially an issue that has yet been addressed.

Add an incomplete to_numpy() w/o 2d support

also fixed some unit test failures

Add multicolumns to numpy support

also fix a DHError empty root cause issue

Remove print statement in unit tests

Minor improvement to DHError

Respond to code review comments

1. Removed the remap parameter because users can always perform the value
remapping using table operations which are way faster
2. Added or edited docstrings
3. Split the pandas module into pandas and numpy to avoid confusion
4. Increased test coverage
5. to_table in numpy remains to be done

Address code review comments

1. optimizaiton around 2D numpy array generation
2. more code cleanup and docstring edit

Resolve a merge conflict with base repo/main

Add a test for suuport Pandas cat col in to_table
pyintegration/deephaven2/dtypes.py Outdated Show resolved Hide resolved
pyintegration/deephaven2/dtypes.py Outdated Show resolved Hide resolved
pyintegration/deephaven2/numpy.py Outdated Show resolved Hide resolved
pyintegration/deephaven2/numpy.py Outdated Show resolved Hide resolved
pyintegration/deephaven2/numpy.py Outdated Show resolved Hide resolved
pyintegration/tests/test_numpy.py Show resolved Hide resolved
pyintegration/tests/test_numpy.py Show resolved Hide resolved
pyintegration/tests/test_numpy.py Show resolved Hide resolved
pyintegration/tests/test_pandas.py Show resolved Hide resolved
pyintegration/tests/test_pandas.py Show resolved Hide resolved
Copy link
Contributor

@jjbrosnan jjbrosnan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me for the most part other than a few minor things on comments/documentation and naming conventions

@jmao-denver jmao-denver dismissed chipkent’s stale review February 24, 2022 17:20

addressed all the comments

@jmao-denver jmao-denver merged commit 17d060a into deephaven:main Feb 24, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Feb 24, 2022
@jmao-denver jmao-denver deleted the feature-1826 branch February 8, 2023 18:11
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reorg and improve java_to_python/python_to_java/npy modules
3 participants