zip + combinations clobbers data on x86?

[lgray]

Version of Awkward Array

2.4.1

Description and code to reproduce

Starting from investigation in: dask-contrib/dask-histogram#100

import hist
import numpy as np
import awkward as ak

if __name__ == "__main__":

    ar = ak.Array(200*np.random.random((100_000, 3)).astype(np.float32) + 32)

    electrons = ak.zip({"pt": ar})

    combos = ak.combinations(electrons, 2, fields=["first", "second"])

    for _ in range(10):
        x = hist.Hist.new.Variable([0, 10, 20, 30, 40, 80, 120, 200], name="x").Double()
        print(x.fill(x=ak.flatten(combos.second.pt)).values(flow=True))

results in:

[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]
[2.00000e+00 6.00000e+00 0.00000e+00 0.00000e+00 1.20520e+04 5.98710e+04
 6.01770e+04 1.20042e+05 4.78500e+04]

Note there are no underflows or values less than 32 in the dataset.

If I print out ak.flatten(combos.second.pt) before the histogram filling step the error magically goes away.
Still might be a histogramming issue.

[lgray]

@iasonkrom @NJManganelli

[lgray]

Interestingly:

import numpy as np
import awkward as ak

if __name__ == "__main__":

    ar = ak.Array(200*np.random.random((100_000, 3)).astype(np.float32) + 32)

    electrons = ak.zip({"pt": ar})

    combos = ak.combinations(electrons, 2, fields=["first", "second"])

    for _ in range(10):
        print(np.histogram(ak.flatten(combos.second.pt), bins=[0, 10, 20, 30, 40, 80, 120, 200]))

Functions correctly and produces the correct output.
So the problem is somewhere on the histogramming side maybe? This is really odd.

[lgray]

If I go to just using the boost_histogram python bindings, rather than hist, the problem is still there.

@henryiii @HDembinski FYI this appears to be some deeply odd interaction between awkward array and boost histogram.

We may want to move the issue there?

[lgray]

Interestingly - if I switch dtype from np.float32 to np.float64 (i.e. just remove the astype) this reproducer segfaults when trying to fill the histogram. This change also functions correctly on arm64.

[ikrommyd]

casting to .astype(np.float64) fixes the problem though

[lgray]

At a request from @agoose77 the following functions correctly:

import boost_histogram as bh
import numpy as np
import awkward as ak

if __name__ == "__main__":

    ar = ak.Array(200*np.random.random((100_000, 3)).astype(np.float32) + 32)

    electrons = ak.zip({"pt": ar})

    combos = ak.combinations(electrons, 2, fields=["first", "second"])

    for _ in range(10):
        x = bh.Histogram(bh.axis.Variable([0, 10, 20, 30, 40, 80, 120, 200]))
        print(x.fill(ak.flatten(combos.second.pt).to_numpy()).values(flow=True))

with either np.float32 or default dtypes.

[ikrommyd]

filling with x=ak.flatten(ak.values_astype(combos.second.pt, np.float64)) while using .astype(np.float32) in generation also fixes it for me. Not doing .astype in the generation though always seg faults