You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Python coverage workflow regularly hangs - leading to the job taking 6 hours and then terminating due to the job time limit. We should figure out what causes this.
It happens when ASYNC is enabled. Here's some example logs:
ss.s.s.s................................................................ [ 8%]
........................................................................ [ 16%]
........................................................................ [ 25%]
...............................................................s........ [ 33%]
..s..................................................................... [ 42%]
...................................s..s................................. [ 50%]
........................................................................ [ 58%]
thread 'polars-3' panicked at /Users/runner/work/polars/polars/crates/polars-io/src/parquet/read/read_impl.rs:763:58:
called `Result::unwrap()` on an `Err` value: (Ok([shape: (3, 2)
┌─────┬─────┐
│ a ┆ b │
│ --- ┆ --- │
│ i64 ┆ i64 │
╞═════╪═════╡
│ 1 ┆ 4 │
│ 2 ┆ 5 │
│ 3 ┆ 6 │
└─────┴─────┘]), 3, 18446744073709551612)
stack backtrace:
......................................................................... [ 67%]
......................................................sss.......xs...... [ 75%]
Fatal Python error: Aborted
Thread 0x0000700002968000 (most recent call first):
..[gw3] node down: Not properly terminated
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 534 in read
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 567 in from_io
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 1160 in _thread_receiver
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 341 in run
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn
Thread 0x00007ff859e9c7c0 (most recent call first):
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pyarrow/dataset.py", line 486 in _filesystem_dataset
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pyarrow/dataset.py", line 794 in dataset
File "/Users/runner/work/polars/polars/py-polars/tests/unit/io/test_pyarrow_dataset.py", line 22 in helper_dataset_test
File "/Users/runner/work/polars/polars/py-polars/tests/unit/io/test_pyarrow_dataset.py", line 150 in test_dataset_foo
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/python.py", line 162 in pytest_pyfunc_call
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/python.py", line 16[32](https://github.com/pola-rs/polars/actions/runs/9763018518/job/26947877229?pr=17359#step:12:33) in runtest
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 173 in pytest_runtest_call
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 241 in <lambda>
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 341 in from_call
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 240 in call_and_report
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 135 in runtestprotocol
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/runner.py", line 116 in pytest_runtest_protocol
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/xdist/remote.py", line 195 in run_one_test
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/xdist/remote.py", line 174 in pytest_runtestloop
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/main.py", line [33](https://github.com/pola-rs/polars/actions/runs/9763018518/job/26947877229?pr=17359#step:12:34)9 in _main
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/main.py", line 285 in wrap_session
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/_pytest/main.py", line 332 in pytest_cmdline_main
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_callers.py", line 103 in _multicall
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_manager.py", line 120 in _hookexec
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/pluggy/_hooks.py", line 513 in __call__
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/xdist/remote.py", line 393 in <module>
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 1291 in executetask
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line [34](https://github.com/pola-rs/polars/actions/runs/9763018518/job/26947877229?pr=17359#step:12:35)1 in run
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 411 in _perform_spawn
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 389 in integrate_as_primary_thread
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 1273 in serve
File "/Users/runner/work/polars/polars/.venv/lib/python3.12/site-packages/execnet/gateway_base.py", line 1806 in serve
File "<string>", line 8 in <module>
File "<string>", line 1 in <module>
Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt199[37](https://github.com/pola-rs/polars/actions/runs/9763018518/job/26947877229?pr=17359#step:12:38), numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, yaml._yaml, markupsafe._speedups, sqlalchemy.cyextension.collections, sqlalchemy.cyextension.immutabledict, sqlalchemy.cyextension.processors, sqlalchemy.cyextension.resultproxy, sqlalchemy.cyextension.util, greenlet._greenlet, pyarrow.lib, zstandard.backend_c, pyarrow._fs, pyarrow._azurefs, pyarrow._hdfs, pyarrow._gcsfs, pyarrow._s3fs, pyarrow._compute, pyarrow._acero, pyarrow._csv, pyarrow._json, pandas._libs.tslibs.ccalendar, pandas._libs.tslibs.np_datetime, pandas._libs.tslibs.dtypes, pandas._libs.tslibs.base, pandas._libs.tslibs.nattype, pandas._libs.tslibs.timezones, pandas._libs.tslibs.fields, pandas._libs.tslibs.timedeltas, pandas._libs.tslibs.tzconversion, pandas._libs.tslibs.timestamps, pandas._libs.properties, pandas._libs.tslibs.offsets, pandas._libs.tslibs.strptime, pandas._libs.tslibs.parsing, pandas._libs.tslibs.conversion, pandas._libs.tslibs.period, pandas._libs.tslibs.vectorized, pandas._libs.ops_dispatch, pandas._libs.missing, pandas._libs.hashtable, pandas._libs.algos, pandas._libs.interval, pandas._libs.lib, pandas._libs.ops, pandas._libs.hashing, pandas._libs.arrays, pandas._libs.tslib, pandas._libs.sparse, pandas._libs.internals, pandas._libs.indexing, pandas._libs.index, pandas._libs.writers, pandas._libs.join, pandas._libs.window.aggregations, pandas._libs.window.indexers, pandas._libs.reshape, pandas._libs.groupby, pandas._libs.json, pandas._libs.parsers, pandas._libs.testing, pyarrow._dataset, pyarrow._dataset_orc, pyarrow._parquet, pyarrow._parquet_encryption, pyarrow._dataset_parquet_encryption, pyarrow._dataset_parquet, adbc_driver_manager._lib, adbc_driver_manager._reader (total: 80)
F
replacing crashed worker gw3
..............s......................................................s.. [ 84%]
........................................................................ [ 92%]
The Python coverage workflow regularly hangs - leading to the job taking 6 hours and then terminating due to the job time limit. We should figure out what causes this.
It happens when ASYNC is enabled. Here's some example logs:
Links to some failed workflows:
The text was updated successfully, but these errors were encountered: