| title | sidebar_position | description |
|---|---|---|
Storage options |
1 |
An overview of the available options for storing Snowplow data in data warehouses and lakes |
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import AzureExperimental from "@site/docs/reusable/azure-experimental/_index.md";
Data warehouses and data lakes are primary destinations for Snowplow data. For other options, see the destinations overview page.
:::note Cloud
The cloud selection is for where your Snowplow pipeline runs. The warehouse itself can be deployed in any cloud.
:::
| Destination | Type | Loader application | Status |
|---|---|---|---|
| Redshift (including Redshift serverless) |
Batching (recommended) or micro-batching |
Snowplow RDB Loader | Production-ready |
| Snowflake | Batching (recommended) or micro-batching |
Snowplow RDB Loader | Production-ready |
| Databricks | Batching (recommended) or micro-batching |
Snowplow RDB Loader | Production-ready |
| Postgres | Streaming | Postgres Loader | Not recommended for production use |
| Destination | Type | Loader application | Status |
|---|---|---|---|
| BigQuery | Streaming | BigQuery Loader | Production-ready |
| Snowflake | Micro-batching | Snowplow RDB Loader | Production-ready |
| Databricks | Micro-batching | Snowplow RDB Loader | Early release |
| Postgres | Streaming | Postgres Loader | Not recommended for production use |
| Destination | Type | Loader application | Status |
|---|---|---|---|
| Snowflake | Micro-batching | Snowplow RDB Loader | Early release |
| Databricks | Micro-batching (via a data lake) |
Lake Loader | Early release |
| Synapse Analytics | Micro-batching (via a data lake) |
Lake Loader | Early release |
All lake loaders are micro-batching.
| Lake | Format | Compatibility | Loader application | Status |
|---|---|---|---|---|
| S3 | TSV/JSON | Amazon Athena | S3 Loader | Only recommended for use with RDB Batch Transformer or for failed events |
| Lake | Format | Compatibility | Loader application | Status |
|---|---|---|---|---|
| GCS | Delta | Databricks | Lake Loader | Early release |
| GCS | JSON | BigQuery | GCS Loader | Only recommended for failed events |
| Lake | Format | Compatibility | Loader application | Status |
|---|---|---|---|---|
| ADLS Gen2 | Delta | Synapse Analytics, Fabric, Databricks | Lake Loader | Early release |