diff --git a/_docs/_user_guide/data_and_analytics/cloud_ingestion.md b/_docs/_user_guide/data_and_analytics/cloud_ingestion.md index fa9b4089955..afbbf84e5ea 100644 --- a/_docs/_user_guide/data_and_analytics/cloud_ingestion.md +++ b/_docs/_user_guide/data_and_analytics/cloud_ingestion.md @@ -8,7 +8,7 @@ page_order: 0.1 page_type: landing guide_top_header: "Braze Cloud Data Ingestion" -guide_top_text: "

What is Braze Cloud Data Ingestion?

Braze Cloud Data Ingestion (CDI) allows you to set up a direct connection from your data storage solution to Braze to sync relevant user or catalog data, and delete users. When synced to Braze, this data can be leveraged for use cases such as personalization or segmentation. Cloud Data Ingestion's flexible integration supports complex data structures including nested JSON and arrays of objects.

**Braze Cloud Data Ingestion capabilities:**
- Create a simple integration directly from your data warehouse or file storage solution to Braze in just a few minutes.
- Securely sync user data, including attributes, events, and purchases from your data warehouse to Braze.
- Close the data loop with Braze by combining Cloud Data Ingestion with Currents or Snowflake Data Sharing.

**Cloud Data Ingestion can sync data from**:
- Amazon Redshift
- Databricks
- Google BigQuery
- Microsoft Fabric
- S3
- Snowflake" +guide_top_text: "

What is it?

Braze Cloud Data Ingestion (CDI) allows you to set up a direct connection from your data storage solution to Braze to sync relevant user or catalog data, and delete users. When synced to Braze, this data can be leveraged for use cases such as personalization or segmentation. Cloud Data Ingestion's flexible integration supports complex data structures including nested JSON and arrays of objects.

**Braze Cloud Data Ingestion capabilities:**
- Create a simple integration directly from your data warehouse or file storage solution to Braze in just a few minutes.
- Securely sync user data, including attributes, events, and purchases from your data warehouse to Braze.
- Close the data loop with Braze by combining Cloud Data Ingestion with Currents or Snowflake Data Sharing.

**Cloud Data Ingestion can sync data from**:
- Amazon Redshift
- Databricks
- Google BigQuery
- Microsoft Fabric
- S3
- Snowflake" guide_featured_title: "Section articles" guide_featured_list: diff --git a/_docs/_user_guide/data_and_analytics/cloud_ingestion/connected_sources.md b/_docs/_user_guide/data_and_analytics/cloud_ingestion/connected_sources.md index 77261d7560d..036631c0a48 100644 --- a/_docs/_user_guide/data_and_analytics/cloud_ingestion/connected_sources.md +++ b/_docs/_user_guide/data_and_analytics/cloud_ingestion/connected_sources.md @@ -1,7 +1,7 @@ --- nav_title: Connected Sources article_title: Connected Sources -description: "This reference article covers how to use Braze Cloud Data Ingestion to sync relevant data with your Snowflake, Redshift, BigQuery, and Databricks integration." +description: "This page covers how to use Braze Cloud Data Ingestion to sync relevant data with your Snowflake, Redshift, BigQuery, and Databricks integration." page_order: 2 page_type: reference @@ -13,7 +13,7 @@ page_type: reference After adding a connected source to your Braze workspace, you can create a CDI segment within Segment Extensions. CDI segments let you write SQL that directly queries your data warehouse (using data there that’s made available through your CDI Connected Source), and creates and maintains a group of users that can be targeted within Braze. -For more information on creating a segment with this source, view [CDI segments]({{site.baseurl}}/user_guide/engagement_tools/segments/segment_extension/cdi_segments/). +For more information on creating a segment with this source, refer to [CDI segments]({{site.baseurl}}/user_guide/engagement_tools/segments/segment_extension/cdi_segments/). {% alert warning %} Because connected sources run on your data warehouse directly, you will incur all costs associated with running these queries in your data warehouse. Connected sources don't consume data points, and CDI segments don't consume SQL segment credits. @@ -141,7 +141,7 @@ When connecting different workspaces to the same Snowflake account, you must cre #### Step 2.4: Allow Braze IPs in your Snowflake network policy (optional) -Depending on the configuration of your Snowflake account, you may need to allow the following IP addresses in your Snowflake network policy. For more information on doing this, view the relevant Snowflake documentation on [modifying a network policy](https://docs.snowflake.com/en/user-guide/network-policies.html#modifying-network-policies). +Depending on the configuration of your Snowflake account, you may need to allow the following IP addresses in your Snowflake network policy. For more information on doing this, refer to the relevant Snowflake documentation on [modifying a network policy](https://docs.snowflake.com/en/user-guide/network-policies.html#modifying-network-policies). {% subtabs %} {% subtab United States (US) %} @@ -194,7 +194,7 @@ If you have a firewall or other network policies, you must give Braze network ac You may also need to change your security groups to allow Braze access to your data in Redshift. Make sure to explicitly allow inbound traffic on the IPs below and on the port used to query your Redshift cluster (default is 5439). You should explicitly allow Redshift TCP connectivity on this port even if the inbound rules are set to "allow all". In addition, it is important that the endpoint for the Redshift cluster be publicly accessible in order for Braze to connect to your cluster. -If you don't want your Redshift cluster to be publicly accessible, you can set up a VPC and EC2 instance to use an ssh tunnel to access the Redshift data. For more information, see [AWS: How do I access a private Amazon Redshift cluster from my local machine?](https://repost.aws/knowledge-center/private-redshift-cluster-local-machine) +If you don't want your Redshift cluster to be publicly accessible, you can set up a VPC and EC2 instance to use an ssh tunnel to access the Redshift data. For more information, refer to [AWS: How do I access a private Amazon Redshift cluster from my local machine?](https://repost.aws/knowledge-center/private-redshift-cluster-local-machine) {% subtabs %} {% subtab United States (US) %} @@ -241,7 +241,7 @@ You may choose to grant access to all tables in a dataset, or grant privileges o The `create table` permission is required so Braze can create a table with your CDI Segment query results before updating the segment in Braze. Braze will create a temporary table per segment, and the table will only persist while Braze is updating the segment. -After creating the service account and granting permissions, generate a JSON key. For more information, view [Google Cloud: Create and delete service account keys](https://cloud.google.com/iam/docs/keys-create-delete). You'll upload this to the Braze dashboard later. +After creating the service account and granting permissions, generate a JSON key. For more information, refer to [Google Cloud: Create and delete service account keys](https://cloud.google.com/iam/docs/keys-create-delete). You'll upload this to the Braze dashboard later. #### Step 2.2: Allow access to Braze IPs @@ -335,29 +335,29 @@ Braze will connect to your Fabric warehouse using a service principal with Entra * Principal ID (also called application ID) for the service principal * Client secret for Braze to authenticate -1. In the Azure portal, navigate to Microsoft Entra admin center, and then App Registrations +1. In the Azure portal, navigate to the Microsoft Entra admin center, and then **App Registrations**. 2. Select **+ New registration** under **Identity > Applications > App registrations** 3. Enter a name, and select `Accounts in this organizational directory only` as the supported account type. Then, select **Register**. 4. Select the application (service principal) you just created, then navigate to **Certificates & secrets > + New client secret** -5. Enter a description for the secret, and set an expiry period for the secret. Then, click add. +5. Enter a description for the secret, and set an expiry period for the secret. Then, select **Add**. 6. Note the client secret created to use in the Braze setup. {% alert note %} -Azure does not allow unlimited expiry on service principal secrets. Remember to refresh the credentials before they expire in order to maintain the flow of data to Braze. +Azure doesn't allow unlimited expiry on service principal secrets. Remember to refresh the credentials before they expire in order to maintain the flow of data to Braze. {% endalert %} #### Step 2.2: Grant access to Fabric resources -You will provide access for Braze to connect to your Fabric instance. In your Fabric admin portal, navigate to **Settings > Governance and insights > Admin portal > Tenant settings**. +You will provide access for Braze to connect to your Fabric instance. In your Fabric admin portal, navigate to **Settings** > **Governance and insights** > **Admin portal** > **Tenant settings**. * In **Developer settings** enable "Service principals can use Fabric APIs" so Braze can connect using Microsoft Entra ID. * In **OneLake settings** enable "Users can access data stored in OneLake with apps external to Fabric" so that the service principal can access data from an external app. #### Step 2.3: Get warehouse connection string + You will need the SQL endpoint for your warehouse in order for Braze to connect. To retrieve the SQL endpoint, go to the **workspace** in Fabric, and in the list of items, hover over the warehouse name and select **Copy SQL connection string**. ![The "Fabric Console" page in Microsoft Azure, where users should retrieve the SQL Connection String.]({% image_buster /assets/img/cloud_ingestion/fabric_1.png %}) - #### Step 2.4: Allow Braze IPs in Firewall (Optional) Depending on the configuration of your Microsoft Fabric account, you may need to allow the following IP addresses in your firewall to allow traffic from Braze. For more information on enabling this, see the relevant documentation on [Entra Conditional Access](https://learn.microsoft.com/en-us/fabric/security/protect-inbound-traffic#entra-conditional-access). @@ -602,7 +602,7 @@ You may set up multiple sources with Braze, but each source should be configured ## Using the connected source -After the source is created, it can be used to create one or more CDI segments. For more information on creating a segment with this source, see the [CDI Segments documentation]({{site.baseurl}}/user_guide/engagement_tools/segments/segment_extension/cdi_segments/). +After the source is created, you can use it to create one or more CDI segments. For more information on creating a segment with this source, refer to the [CDI Segments documentation]({{site.baseurl}}/user_guide/engagement_tools/segments/segment_extension/cdi_segments/). {% alert note %} If queries are consistently timing out and you have set a maximum runtime of 60 minutes, consider trying to optimize your query execution time or dedicating more compute resources (such as a larger warehouse) to the Braze user. diff --git a/_docs/_user_guide/data_and_analytics/cloud_ingestion/delete_users.md b/_docs/_user_guide/data_and_analytics/cloud_ingestion/delete_users.md index 60f6309f144..e602defca85 100644 --- a/_docs/_user_guide/data_and_analytics/cloud_ingestion/delete_users.md +++ b/_docs/_user_guide/data_and_analytics/cloud_ingestion/delete_users.md @@ -3,23 +3,25 @@ nav_title: Delete Users with CDI article_title: Delete Users with Cloud Data Ingestion page_order: 30 page_type: reference -description: "This reference article provides an overview of the process for deleting users with Cloud Data Ingestion." +description: "This pgae provides an overview of the process for deleting users with Cloud Data Ingestion." --- # Delete users with Cloud Data Ingestion +> This page discusses the process for deleting users with Cloud Data Ingestion. + User delete syncs are supported for all available Cloud Data Ingestion data sources. -## Integration configuration +## Configuring the integration -Follow the standard process to [create a new integration in the Braze dashboard]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views) for the data warehouse you want to connect to. Ensure that you include a role that can access the delete table. On the **Create import sync** page, set the **Data Type** to **Delete Users**. This will ensure the proper actions are taken during the integration run to delete users. +Follow the standard process to [create a new integration in the Braze dashboard]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views) for the data warehouse you want to connect to. Ensure that you include a role that can access the delete table. On the **Create import sync** page, set the **Data Type** to **Delete Users** so that the proper actions are taken during the integration run to delete users. ![]({% image_buster /assets/img/cloud_ingestion/deletion_1.png %}) -## Source data configuration +## Configuring source data -Source tables for user deletes should include one or more user identifier types and an `UPDATED_AT` timestamp. Payload columns are not supported for user delete data. +Source tables for user deletes should include one or more user identifier types and an `UPDATED_AT` timestamp. Payload columns aren't supported for user delete data. ### `UPDATED_AT` diff --git a/_docs/_user_guide/data_and_analytics/cloud_ingestion/faqs.md b/_docs/_user_guide/data_and_analytics/cloud_ingestion/faqs.md index 9bb5359fbc3..d42958093d1 100644 --- a/_docs/_user_guide/data_and_analytics/cloud_ingestion/faqs.md +++ b/_docs/_user_guide/data_and_analytics/cloud_ingestion/faqs.md @@ -3,13 +3,13 @@ nav_title: FAQs article_title: Cloud Data Ingestion FAQs page_order: 100 page_type: FAQ -description: "This article answers frequently asked questions about Cloud Data Ingestion." +description: "This page answers frequently asked questions about Cloud Data Ingestion." toc_headers: h2 --- # Frequently Asked Questions (FAQ) -> These are the answers to some frequently asked questions for Cloud Data Ingestion. +> This page contains answers to some frequently asked questions for Cloud Data Ingestion. ## Why was I emailed: "Error in CDI Sync"? @@ -17,7 +17,7 @@ This type of email usually means there's an issue with your CDI setup. Here are ### CDI can't access the data warehouse or table using your credentials -This could mean the credentials in CDI are incorrect or are misconfigured on the data warehouse. For more information, see [Data Warehouse Integrations]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/). +This could mean the credentials in CDI are incorrect or are misconfigured on the data warehouse. For more information, refer to [Data Warehouse Integrations]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/). ### The table cannot be found @@ -25,7 +25,7 @@ Try updating your integration with the correct database configuration or create ### The catalog cannot be found -The catalog set up in the integration does not exist in the Braze catalog. A catalog can be removed after the integration was set up. To resolve the issue, either update the integration to use a different catalog or create a new catalog that matches the catalog name in the integration. +The catalog set up in the integration doesn't exist in the Braze catalog. A catalog can be removed after the integration was set up. To resolve the issue, either update the integration to use a different catalog or create a new catalog that matches the catalog name in the integration. ## Why was I emailed: "Row errors in your CDI sync"? @@ -41,11 +41,11 @@ Test Connection is running on your data warehouse, so increasing warehouse capac ### Error connecting to Snowflake instance: Incoming request with IP is not allowed to access Snowflake -Try adding the official Braze IPs to your IP allowlist. For more information, see [Data Warehouse Integrations]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/). +Try adding the official Braze IPs to your IP allowlist. For more information, refer to [Data Warehouse Integrations]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/). ### Error executing SQL due to customer config: 002003 (42S02): SQL compilation error: does not exist or not authorized -If the table does not exist, create the table. If the table does exist, verify that the user and role have permissions to read from the table. +If the table doesn't exist, create the table. If the table does exist, verify that the user and role have permissions to read from the table. ### Could not use schema @@ -61,7 +61,7 @@ If you receive this error, allow that user access to your Snowflake account. ### Error connecting to Snowflake instance with current and old key -If you receive this error, make sure the user is using the current public key as seen in your braze dashboard. +If you receive this error, make sure the user is using the current public key as displayed in your Braze dashboard. {% endtab %} {% tab Redshift %} @@ -96,7 +96,7 @@ Test Connection is running on your data warehouse, so increasing warehouse capac ### User does not have permission to query table -If you receive this error, add User permissions to query the table. +If you receive this error, add user permissions to query the table. ### Your usage exceeded the custom quota diff --git a/_docs/_user_guide/data_and_analytics/cloud_ingestion/file_storage_integrations.md b/_docs/_user_guide/data_and_analytics/cloud_ingestion/file_storage_integrations.md index 64b5cc72afe..ad04117b52c 100644 --- a/_docs/_user_guide/data_and_analytics/cloud_ingestion/file_storage_integrations.md +++ b/_docs/_user_guide/data_and_analytics/cloud_ingestion/file_storage_integrations.md @@ -1,7 +1,7 @@ --- nav_title: File Storage Integrations article_title: File Storage Integrations -description: "This reference article covers Braze Cloud Data Ingestion and how to sync relevant data from S3 to Braze" +description: "This page covers Braze Cloud Data Ingestion and how to sync relevant data from S3 to Braze" page_order: 3 page_type: reference @@ -9,7 +9,7 @@ page_type: reference # File storage integrations -> This article covers how to set up Cloud Data Ingestion support and sync relevant data from S3 to Braze. +> This page covers how to set up Cloud Data Ingestion support and sync relevant data from S3 to Braze. You can use Cloud Data Ingestion (CDI) for S3 to directly integrate one or more S3 buckets in your AWS account with Braze. When new files are published to S3, a message is posted to SQS, and Braze Cloud Data Ingestion takes in those new files. @@ -23,7 +23,7 @@ The integration requires the following resources: ## AWS definitions -First, let's just define some of the terms used during this task. +First, let's define some of the terms used during this task. | Word | Definition | | --- | --- | @@ -55,7 +55,7 @@ Create an SQS queue to track when objects are added to the bucket you’ve creat Be sure to create this SQS in the same region you created the bucket in. {% endalert %} -Be sure to take note of the ARN and the URL of the SQS as you’ll be using it frequently during this configuration. +Make sure you take note of the ARN and the URL of the SQS as you’ll be using it frequently during this configuration.

![]({% image_buster /assets/img/cloud_ingestion/s3_ARN.png %})

@@ -194,7 +194,7 @@ Unlike with data warehouse sources, the `UPDATED_AT` column is not required nor {% endalert %} {% alert note %} -Files added to the S3 source bucket should not exceed 512MB. Files larger than 512MB will result in an error and will not be synced to Braze. +Files added to the S3 source bucket should not exceed 512MB. Files larger than 512MB will result in an error and won't be synced to Braze. {% endalert %} {% tabs %} diff --git a/_docs/_user_guide/data_and_analytics/cloud_ingestion/integrations.md b/_docs/_user_guide/data_and_analytics/cloud_ingestion/integrations.md index 1e81327dca8..a504cf7d1a0 100644 --- a/_docs/_user_guide/data_and_analytics/cloud_ingestion/integrations.md +++ b/_docs/_user_guide/data_and_analytics/cloud_ingestion/integrations.md @@ -1,7 +1,7 @@ --- nav_title: Data Warehouse Integrations article_title: Data Warehouse Integrations -description: "This reference article covers how to use Braze Cloud Data Ingestion to sync relevant data with your Snowflake, Redshift, BigQuery, and Databricks integration." +description: "This page covers how to use Braze Cloud Data Ingestion to sync relevant data with your Snowflake, Redshift, BigQuery, and Databricks integration." page_order: 2 page_type: reference @@ -9,9 +9,9 @@ page_type: reference # Data warehouse storage integrations -> This article covers how to use Braze Cloud Data Ingestion (CDI) to sync relevant data with your Snowflake, Redshift, BigQuery, and Databricks integration. +> This page covers how to use Braze Cloud Data Ingestion (CDI) to sync relevant data with your Snowflake, Redshift, BigQuery, and Databricks integration. -## Product setup +## Setting up data warehouse integrations Cloud Data Ingestion integrations require some setup on the Braze side and in your data warehouse instance. Follow these steps to set up the integration: @@ -111,7 +111,7 @@ GRANT USAGE ON WAREHOUSE BRAZE_INGESTION_WAREHOUSE TO ROLE BRAZE_INGESTION_ROLE; ``` {% alert note %} -The warehouse will need to have the **auto-resume** flag on. If not, you will need to grant Braze additional `OPERATE` privileges on the warehouse for us to turn it on when it's time to run the query. +The warehouse needs to have the **auto-resume** flag on. If not, you will need to grant Braze additional `OPERATE` privileges on the warehouse for us to turn it on when it's time to run the query. {% endalert %} #### Step 1.4: Set up the user @@ -354,7 +354,7 @@ You can name the schema and table as you'd like, but the column names should mat #### Step 1.2: Create an Access Token -In order for Braze to access Databricks, a personal access token needs to be created. +For Braze to access Databricks, a personal access token needs to be created. 1. In your Databricks workspace, select your Databricks username in the top bar, and then select **User Settings** from the dropdown. 2. On the Access tokens tab, select **Generate new token**. @@ -391,18 +391,18 @@ Braze will connect to your Fabric warehouse using a service principal with Entra * Client secret for Braze to authenticate 1. In the Azure portal, navigate to Microsoft Entra admin center, and then App Registrations -2. Select **+ New registration** under **Identity > Applications > App registrations** -3. Enter a name, and select `Accounts in this organizational directory only` as the supported account type. Then, select **Register**. -4. Select the application (service principal) you just created, then navigate to **Certificates & secrets > + New client secret** -5. Enter a description for the secret, and set an expiry period for the secret. Then, click add. +2. Select **+ New registration** under **Identity** > **Applications** > **App registrations**. +3. Enter a name, and then select `Accounts in this organizational directory only` as the supported account type. Then, select **Register**. +4. Select the application (service principal) you just created, then navigate to **Certificates & secrets** > **+ New client secret**. +5. Enter a description for the secret, and set an expiry period for the secret. Then, select **Add**. 6. Note the client secret created to use in the Braze setup. {% alert note %} -Azure does not allow unlimited expiry on service principal secrets. Remember to refresh the credentials before they expire in order to maintain the flow of data to Braze. +Azure doesn't allow unlimited expiry on service principal secrets. Remember to refresh the credentials before they expire to maintain the flow of data to Braze. {% endalert %} #### Step 1.2: Grant access to Fabric resources -You will provide access for Braze to connect to your Fabric instance. In your Fabric admin portal, navigate to **Settings > Governance and insights > Admin portal > Tenant settings**. +You will provide access for Braze to connect to your Fabric instance. In your Fabric admin portal, navigate to **Settings** > **Governance and insights** > **Admin portal** > **Tenant settings**. * In **Developer settings** enable "Service principals can use Fabric APIs" so Braze can connect using Microsoft Entra ID. * In **OneLake settings** enable "Users can access data stored in OneLake with apps external to Fabric" so that the service principal can access data from an external app. @@ -430,7 +430,7 @@ CREATE OR ALTER TABLE [warehouse].[schema].[CDI_table_name] GO ``` -You can name the warehouse, schema, and table/view as you'd like, but the column names should match the preceding definition. +You can name the warehouse, schema, and table or view as you'd like, but the column names should match the preceding definition. - `UPDATED_AT` - The time this row was updated in or added to the table. We will only sync rows that have been added or updated since the last sync. - **User identifier columns** - Your table may contain one or more user identifier columns. Each row should only contain one identifier (either `external_id`, the combination of `alias_name` and `alias_label`, `braze_id`, `phone`, or `email`). A source table may have columns for one or more identifier types. @@ -472,7 +472,7 @@ Depending on the configuration of your Microsoft Fabric account, you may need to {% tabs %} {% tab Snowflake %} -In the Braze Dashbord, go to **Data Settings > Cloud Data Ingestion**, click **Create New Data Sync**, and select **Snowflake Import**. +In the Braze Dashbord, go to **Data Settings** > **Cloud Data Ingestion**, select **Create New Data Sync**, and then select **Snowflake Import**. #### Step 2.1: Add Snowflake connection information and source table @@ -507,11 +507,11 @@ ALTER USER BRAZE_INGESTION_USER SET rsa_public_key='Braze12345...'; {% endtab %} {% tab Redshift %} -In the Braze Dashbord, go to **Data Settings > Cloud Data Ingestion**, click **Create New Data Sync**, and select **Amazon Redshift Import**. +In the Braze Dashbord, go to **Data Settings** > **Cloud Data Ingestion**, select **Create New Data Sync**, and then select **Amazon Redshift Import**. #### Step 2.1: Add Redshift connection information and source table -Input the information for your Redshift data warehouse and source table. If you are using a private network tunnel, toggle the slider and input the tunnel information. Then proceed to the next step. +Input the information for your Redshift data warehouse and source table. If you're using a private network tunnel, toggle the slider and input the tunnel information. Then proceed to the next step. ![The "Create new import sync" page for Redshift in the Braze dashboard, set to Step 1: "Set up connection".]({% image_buster /assets/img/cloud_ingestion/ingestion_6.png %}) @@ -532,7 +532,7 @@ You will also choose the data type and sync frequency. Frequency can be anywhere {% endtab %} {% tab BigQuery %} -In the Braze Dashbord, go to **Data Settings > Cloud Data Ingestion**, click **Create New Data Sync**, and select **Google BigQuery Import**. +In the Braze Dashbord, go to **Data Settings** > **Cloud Data Ingestion**, select **Create New Data Sync**, and then select **Google BigQuery Import**. #### Step 2.1: Add BigQuery connection information and source table @@ -544,7 +544,7 @@ Upload the JSON key and provide a name for the service account, then input the d Next, choose a name for your sync and input contact emails. We'll use this contact information to notify you of any integration errors, such as unexpected removal of access to the table. -Contact emails will only receive notifications of global or sync-level errors such as missing tables, permissions, and others. They will not receive row-level issues. Global errors indicate critical problems with the connection that prevent syncs from running. Such problems can include the following: +Contact emails will only receive notifications of global or sync-level errors such as missing tables, permissions, and others. They won't receive row-level issues. Global errors indicate critical problems with the connection that prevent syncs from running. Such problems can include the following: - Connectivity issues - Lack of resources @@ -558,7 +558,7 @@ You will also choose the data type and sync frequency. Frequency can be anywhere {% endtab %} {% tab Databricks %} -In the Braze Dashbord, go to **Data Settings > Cloud Data Ingestion**, click **Create New Data Sync**, and select **Databricks Import**. +In the Braze Dashbord, go to **Data Settings** > **Cloud Data Ingestion**, select **Create New Data Sync**, and then select **Databricks Import**. #### Step 2.1: Add Databricks connection information and source table @@ -586,7 +586,7 @@ You will also choose the data type and sync frequency. Frequency can be anywhere #### Step 2.1: Set up a Cloud Data Ingestion sync -You will create a new data sync for Microsoft Fabric. In the Braze dashbord, go to **Data Settings > Cloud Data Ingestion**, click **Create New Data Sync**, and select **Microsoft Fabric Import**. +You will create a new data sync for Microsoft Fabric. In the Braze dashbord, go to **Data Settings** > **Cloud Data Ingestion**, select **Create New Data Sync**, and then select **Microsoft Fabric Import**. #### Step 2.2: Add Microsoft Fabric connection information and source table diff --git a/_docs/_user_guide/data_and_analytics/cloud_ingestion/overview.md b/_docs/_user_guide/data_and_analytics/cloud_ingestion/overview.md index 8fdd679c81b..f2b648bb563 100644 --- a/_docs/_user_guide/data_and_analytics/cloud_ingestion/overview.md +++ b/_docs/_user_guide/data_and_analytics/cloud_ingestion/overview.md @@ -3,7 +3,7 @@ nav_title: Overview article_title: Cloud Data Ingestion Overview page_order: 0 page_type: reference -description: "This reference article provides an overview of Cloud Data Ingestion, best practices, and product limitations." +description: "This page provides an overview of Cloud Data Ingestion, best practices, and product limitations." --- @@ -13,9 +13,9 @@ description: "This reference article provides an overview of Cloud Data Ingestio ## How it works -With Braze Cloud Data Ingestion (CDI), you set up an integration between your data warehouse instance and Braze workspace to sync data on a recurring basis. This sync runs on a schedule you set, and each integration can have a different schedule. Syncs can run as frequently as every 15 minutes or as infrequently as once per month. For customers who need syncs to occur more frequently than 15 minutes, please speak with your customer success manager, or consider using REST API calls for real-time data ingestion. +With Braze Cloud Data Ingestion (CDI), you set up an integration between your data warehouse instance and Braze workspace to sync data on a recurring basis. This sync runs on a schedule you set, and each integration can have a different schedule. Syncs can run as frequently as every 15 minutes or as infrequently as once per month. If you need syncs to occur more frequently than 15 minutes, contact your customer success manager or consider using REST API calls for real-time data ingestion. -When a sync runs, Braze will directly connect to your data warehouse instance, retrieve all new data from the specified table, and update the corresponding data on your Braze dashboard. Each time the sync runs, any updated data will be reflected in Braze. +When a sync runs, Braze directly connects to your data warehouse instance, retrieves all new data from the specified table, and updates the corresponding data on your Braze dashboard. Each time the sync runs, any updated data will be reflected in Braze. ## Supported data sources @@ -391,7 +391,7 @@ However, when Request 2 occurs, Braze starts with the original attribute values When the requests are finished, Request 2 will overwrite the update from Request 1. Due to this, it's best to stagger your updates so you can prevent requests from being overwritten. -### Create JSON string from another table +### Create a JSON string from another table If you prefer to store each attribute in its own column internally, you need to convert those columns to a JSON string to populate the sync with Braze. To do that, you can use a query like: @@ -508,7 +508,7 @@ FROM [braze].[users] ; ### Use the `UPDATED_AT` timestamp We use the `UPDATED_AT` timestamp to track what data has been synced successfully to Braze. If many rows are written with the same timestamp while a sync is running, this may lead to duplicate data being synced to Braze. Some suggestions to avoid duplicate data: -- If you are setting up a sync against a `VIEW`, do not use `CURRENT_TIMESTAMP` as the default value. This will cause all data to sync every time the sync runs because the `UPDATED_AT` field will evaluate to the time our queries are run. +- If you're setting up a sync against a `VIEW`, don't use `CURRENT_TIMESTAMP` as the default value. This will cause all data to sync every time the sync runs because the `UPDATED_AT` field will evaluate to the time our queries are run. - If you have very long-running pipelines or queries writing data to your source table, avoid running these concurrently with a sync, or avoid using the same timestamp for every row inserted. - Use a transaction to write all rows that have the same timestamp. diff --git a/_docs/_user_guide/data_and_analytics/cloud_ingestion/sync_catalogs_data.md b/_docs/_user_guide/data_and_analytics/cloud_ingestion/sync_catalogs_data.md index 2480d04b9a3..f18e899450a 100644 --- a/_docs/_user_guide/data_and_analytics/cloud_ingestion/sync_catalogs_data.md +++ b/_docs/_user_guide/data_and_analytics/cloud_ingestion/sync_catalogs_data.md @@ -3,16 +3,17 @@ nav_title: Sync and Delete Catalog Data article_title: Sync and Delete Catalog Data page_order: 4 page_type: reference -description: "This reference article provides an overview of how to sync catalog data." +description: "This page provides an overview of how to sync catalog data." --- # Sync and delete catalog data +> This page discusses how to sync catalog data. ## Step 1: Create a new catalog -Before creating a new Cloud Data Ingestion (CDI) integration for [catalogs]({{site.baseurl}}/user_guide/personalization_and_dynamic_content/catalogs/), you will need to create a new catalog or identify an existing catalog you want to use for the integration. There are a few ways to create a new catalog and any of these will work for the CDI integration: +Before creating a new Cloud Data Ingestion (CDI) integration for [catalogs]({{site.baseurl}}/user_guide/personalization_and_dynamic_content/catalogs/), you need to create a new catalog or identify an existing catalog you want to use for the integration. There are a few ways to create a new catalog and any of these will work for the CDI integration: - Upload a [CSV]({{site.baseurl}}/user_guide/personalization_and_dynamic_content/catalogs/catalog/#method-1-upload-csv) - Create a catalog in the [Braze dashboard]({{site.baseurl}}/user_guide/personalization_and_dynamic_content/catalogs/catalog/#method-2-create-in-browser) or during CDI setup. - Create a catalog using the [Create catalog endpoint]({{site.baseurl}}/api/endpoints/catalogs/catalog_management/synchronous/post_create_catalog/) @@ -39,7 +40,7 @@ The setup for a catalog sync closely follows the process for [user-data CDI inte DELETED BOOLEAN ); ``` -2. Set up a role, warehouse, and user and grant proper permissions. If you already have credentials from an existing sync, you can reuse them, just make sure to extend access to the catalog source table. +2. Set up a role, warehouse, and user and grant proper permissions. If you already have credentials from an existing sync, you can reuse them, but make sure to extend access to the catalog source table. ```json CREATE ROLE BRAZE_INGESTION_ROLE; @@ -53,13 +54,13 @@ The setup for a catalog sync closely follows the process for [user-data CDI inte CREATE USER BRAZE_INGESTION_USER; GRANT ROLE BRAZE_INGESTION_ROLE TO USER BRAZE_INGESTION_USER; ``` -3. If your Snowflake account has network policies, allowlist the Braze IPs so the CDI service can connect. For a list of IPs, see the [Cloud Data Ingestion]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views). -4. In the Braze dashboard, navigate to **Technology Partners > Snowflake**, and create a new sync. +3. If your Snowflake account has network policies, allowlist the Braze IPs so the CDI service can connect. For a list of IPs, refer to the [Cloud Data Ingestion]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views). +4. In the Braze dashboard, navigate to **Technology Partners** > **Snowflake**, and create a new sync. 5. Enter connection details (or reuse existing credentials) and the source table. 6. Proceed to step 2 of the setup flow, select the “Catalogs” sync type, and input the integration name and schedule. Note that the name of the integration should **exactly match** the name of the catalog you previously created. 7. Choose a sync frequency and proceed to the next step. 8. Add the public key displayed on the dashboard to the user you created for Braze to connect to Snowflake. To complete this step, you will need someone with `SECURITYADMIN` access or higher in Snowflake. -9. Click **Test Connection** to ensure everything works as expected. +9. Select **Test Connection** so that everything works as expected. 10. Save the sync, and use the synced catalog data for all your personalization use cases. {% endtab %} {% tab Redshift %} @@ -78,7 +79,7 @@ The setup for a catalog sync closely follows the process for [user-data CDI inte deleted boolean ) ``` -2. Set up a user and grant proper permissions. If you already have credentials from an existing sync, you can reuse them, just make sure to extend access to the catalog source table. +2. Set up a user and grant proper permissions. If you already have credentials from an existing sync, you can reuse them, but make sure to extend access to the catalog source table. {% raw %} ```json CREATE USER braze_user PASSWORD '{password}'; @@ -86,7 +87,7 @@ The setup for a catalog sync closely follows the process for [user-data CDI inte GRANT SELECT ON TABLE CATALOGS_SYNC TO braze_user; ``` {% endraw %} -3. If you have a firewall or other network policies, you must give Braze network access to your Redshift instance. Allow access from the below IPs corresponding to your Braze dashboard’s region. For a list of IPs, see the [Cloud Data Ingestion]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views). +3. If you have a firewall or other network policies, you must give Braze network access to your Redshift instance. Allow access from the below IPs corresponding to your Braze dashboard’s region. For a list of IPs, refer to the [Cloud Data Ingestion]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views). {% endtab %} {% tab BigQuery %} @@ -118,7 +119,7 @@ CREATE TABLE `BRAZE-CLOUD-PRODUCTION.INGESTION.CATALOGS_SYNC` {:start="2"} -2. Set up a user and grant proper permissions. If you already have credentials from an existing sync, you can reuse those—just make sure to extend access to the catalog source table. +2. Set up a user and grant proper permissions. If you already have credentials from an existing sync, you can reuse those—but make sure to extend access to the catalog source table. The service account should have the below permissions: - BigQuery Connection User: This will allow Braze to make connections. - BigQuery User: This will provide Braze access to run queries, read dataset metadata, and list tables. @@ -126,7 +127,7 @@ The service account should have the below permissions: - BigQuery Job User: This will provide Braze access to run jobs

After creating the service account and granting permissions, generate a JSON key. Refer to [Keys create and delete](https://cloud.google.com/iam/docs/keys-create-delete) for more information. You'll update this to the Braze dashboard later. {:start="3"} -3. If you have network policies in place, you must give Braze network access to your BigQuery instance. For a list of IPs, see the [Cloud Data Ingestion]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views). +3. If you have network policies in place, you must give Braze network access to your BigQuery instance. For a list of IPs, refer to the [Cloud Data Ingestion]({{site.baseurl}}/user_guide/data_and_analytics/cloud_ingestion/integrations/#step-1-set-up-tables-or-views). {% endtab %} {% tab Databricks %}