You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Projects including Fugue itself needs to test with different backends. The current way to test is to manually create Spark, Dask, Ray and Duckdb sessions, test and stop them. In order to get good unit testing performance, it's actually tricky to setup those backends. Another issues is, if there is session level pytest fixture, then some sessions will be started and stopped in each single test, adding significant overhead.
So, we will create fixtures for Fugue and related projects. As the first release, we will have these hard coded fixtures (with the best possible performance for unit tests)
Spark support, fugue_spark_session
Dask support: fugue_dask_client
Ray support: fugue_ray_session
Duckdb support: fugue_duckdb_connection
Make them configurable in the common pytest config
Make them available automatically as long as fugue is installed
Create enumerated engine (parameterized), so it will automatically test all available backend
To use them, we just need to pip install fugue with the specific extra, then here is an example:
Projects including Fugue itself needs to test with different backends. The current way to test is to manually create Spark, Dask, Ray and Duckdb sessions, test and stop them. In order to get good unit testing performance, it's actually tricky to setup those backends. Another issues is, if there is session level pytest fixture, then some sessions will be started and stopped in each single test, adding significant overhead.
So, we will create fixtures for Fugue and related projects. As the first release, we will have these hard coded fixtures (with the best possible performance for unit tests)
fugue_spark_session
fugue_dask_client
fugue_ray_session
fugue_duckdb_connection
To use them, we just need to pip install fugue with the specific extra, then here is an example:
Currently, for spark backend, please keep using pytest-spark and
spark_session
provided by the package.The text was updated successfully, but these errors were encountered: