Struggling to go from single indexing to multi-indexing

[Ashwolaa]

Hi,

I am new to xarray and I am already very impressed by some of the indexing that is accessible. Nevertheless, I am currently facing an issue to fully benefit from the potentiality of the library.

I am loading a log file (a big csv file with many columns) that records many parameters in my experiment with pandas which I then convert to a Dataset file.
Each data point is then associated to some parameters.

The dataset looks like this;
ds = xr.Dataset(
data_vars={
"delay1": ("index", [1, 1, 1, 1, 2, 2, 2, 2]),
"delay2": ("index", [10, 10, 20, 20, 10, 10, 20, 20]),
"isOn": ("index", [True, False, True, False, True, False, True, False]),
"data": ("index", [0, 10, 20, 30, 40, 50, 60 ,70]),
},
coords={"index": [0, 1, 2, 3, 4, 5, 6, 7,]}
)

Most of the information gets repeated here, as such I would like to use a different representation. In this representation, I would:

• only store the unique variables in "delay1", "delay2", "isOn" as coordinates
• reshape the data accordingly.

The final output would then look something like this:

ds = xr.Dataset(
data_vars={
"data": (("delay1","delay2","isOn"), a 3D object),
},
coords={"delay1": [1, 2],"delay2": [10, 20],"isOn": [True, False]}
)

I struggled to do it on my own with the API and I was wondering if there exists already some functions to implement this behavior easily or if I should aim for some other ways to treat the data before storing them into the dataset.

Thanks!

[dcherian]

It is not easy to figure this out sadly.

from xarray.indexes import PandasMultiIndex

coord_names = ["isOn", "delay2", "delay1"]

(
    ds
    # drop the existing index variable (required for set_xindex)
    .drop_vars("index")
    # set the desired new dimensions as coordinate variables
    .set_coords(coord_names)
    # Assign a PandasMultiIndex
    .set_xindex(coord_names, PandasMultiIndex)
    # "unstack" to a nD array
    .unstack("index")
)

[Ashwolaa]

Fantastic, this works flawlessly!

Thank you very much!