groupby_bins based on quantiles

[jonas-spaeth]

Currently possible

groupby_bins allows to specify bins either as integer (specifying the number of bins) or list (specifying the explicit interval edges). If bins is an integer, the bins are automatically created such that each intervals span the same width (e.g., from 0-2, from 2-4 and 4-6), because it relies on pd.cut.

Currently not directly possible

I often need to bin data based on quantiles, i.e., in groups of the same size. This is possible using pd.qcut, but it is not possible using xarray, as far that I am aware?

Wouldn't it be nice to be able to specify whether groupby_bins(..., bins=3) should create intervals of equal length or intervals such that the groups are of equal size?

Here is an example:

import xarray as xr
import numpy as np

# create DataArray with 9 floats
np.random.seed(1)
n = 9
data = xr.DataArray(np.random.normal(size=n), dims='x', coords=dict(x=range(n)))
print(data, end='\n\n')

<xarray.DataArray (x: 9)>
array([ 1.62434536, -0.61175641, -0.52817175, -1.07296862, 0.86540763,
-2.3015387 , 1.74481176, -0.7612069 , 0.3190391 ])
Coordinates:

• x (x) int64 0 1 2 3 4 5 6 7 8

# group DataArray into 3 bins
data_grouped = data.groupby_bins(data, bins=3)
print(data_grouped, end='\n\n')

DataArrayGroupBy, grouped over 'group_bins'
3 groups with labels (0.396,, 1.745], ..., (-2.306,....

# groupby_bins creates intervals of equal lengths (based on pd.cut)
intervals = data_grouped.groups.keys()
length_of_intervals = [l.length for l in intervals]
print(length_of_intervals, end='\n\n')

[1.3490000000000002, 1.349, 1.3530000000000002]

# therefore, number of items in each bin is not necessarily equal
indices_in_interval = data_grouped.groups.values()
number_of_items_in_interval = [len(i) for i in indices_in_interval]
print(number_of_items_in_interval, end='\n\n')

[3, 4, 2]

# binning based on quantiles requires explicit specification of bins
terciles = data.quantile([1/3, 2/3])
data_grouped_q = data.groupby_bins(data, bins=[-np.inf, *terciles, np.inf])
indices_in_interval_q = data_grouped_q.groups.values()
number_of_items_in_interval_q = [len(i) for i in indices_in_interval_q]
print(number_of_items_in_interval_q, end='\n\n')

[3, 3, 3]

# would be nice to have that option integrated in groupby_bins
try:
    data.groupby_bins(data, bins=3, method='q')
except:
    print("not implemented")

not implemented

Implementation

could be easy I guess, just using an if statement to either execute

binned, bins = pd.cut(group.values, bins, **cut_kwargs, retbins=True)  # so far

or

binned, bins = pd.qcut(group.values, bins, **cut_kwargs, retbins=True)  # new