Integrate ephemeral database(s) with CI for read/write testing #12815
Labels
A-io-database
Area: reading/writing to databases
A-sql
Area: Polars SQL functionality
accepted
Ready for implementation
ci
Related to the continuous integration setup
enhancement
New feature or an improvement of an existing feature
internal
An internal refactor or improvement
Description
@stinodego: As promised ;)
Currently
read_database
andwrite_database
unit tests rely on SQLite when running on CI, which is insufficient to provide quality testing due to the numerous differences between that and something like PostgreSQL, MSSQL, Redshift, Snowflake, etc. While extending these functions I've had to rely on validating new functionality against a containerised PostgreSQL (and a few other DBs) running off a NAS at home, or using Polars at work where we have data in (amongst others) Redshift, Databricks, and BigQuery stores.We should enhance CI (and the associated unit tests) to, at a minimum, support an ephemeral PostgreSQL test target as well. If there are other databases that support simple containerisation (or some equivalent facility) then we could think about adding more than PostgreSQL later 🤔
The text was updated successfully, but these errors were encountered: