Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Investigating SQL Support. #11

Merged
merged 8 commits into from
Feb 15, 2024
Merged

Investigating SQL Support. #11

merged 8 commits into from
Feb 15, 2024

Commits on Feb 13, 2024

  1. Profiling SQL via Dask 

    Beginning to investigate SQL support. Right now, for anything beyond a small Xarray dataset, this is terribly slow.
    @alxmrs
    alxmrs committed Feb 13, 2024
    Configuration menu
    Copy the full SHA
    9b1b423 View commit details
    Browse the repository at this point in the history

Commits on Feb 15, 2024

  1. Adding pyspy profiling tool. 

    Also adding in baseline profile.
    @alxmrs
    alxmrs committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    a28498b View commit details
    Browse the repository at this point in the history

  2. Removed unboxing, seems to have improved performance (from looking at… 

    … the flame graph).
    @alxmrs
    alxmrs committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    893cd8c View commit details
    Browse the repository at this point in the history

  3. Added a groupby profile test.

    @alxmrs
    alxmrs committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    fa97615 View commit details
    Browse the repository at this point in the history

  4. RM bad profile.

    @alxmrs
    alxmrs committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    9a49670 View commit details
    Browse the repository at this point in the history

  5. Added type info to Dataframes. 

    Added dtypes from the xr.Dataset to the pd.DataFrame. This allows SQL aggegrations to work, which need type info.

This changed the core `unravel` operation, so I've added sanity profile traces.
    @alxmrs
    alxmrs committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    2240611 View commit details
    Browse the repository at this point in the history

  6. Removed TODO

    @alxmrs
    alxmrs committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    62400a0 View commit details
    Browse the repository at this point in the history

  7. Adding groupby_air.py profile.

    @alxmrs
    alxmrs committed Feb 15, 2024
    Configuration menu
    Copy the full SHA
    0d689bc View commit details
    Browse the repository at this point in the history