This library provides APIs to write a spark dataframe to various target source systems.
This library supports below target systems:
- Local File System
- AWS
- GCP
User can use this library to write/persist spark dataframe to the local file system in various file formats. Supported file formats are:
- CSV
- JSON
- XML
- Parquet
Please see the detailed documentation here.
User can use this library to write/persist spark dataframe to various storage options provided by AWS.
Currently, we are supporting below options:
- S3 Buckets
- Redshift Tables
Please see the detailed documentation here.
User can use this library to write/persist spark dataframe to various storage options provided by google cloud system.
Currently, we are supporting below options:
- GCS Buckets
- BigQuery Tables
Please see the detailed documentation here.