-
Notifications
You must be signed in to change notification settings - Fork 4.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Source Smartsheets: fix docs to certify for Beta (#12203)
* #5520 fix scrambled columns bug * #5520 source smartsheets: add changelog item * #5520 move pytest to optional setup requirements * #12003 source smartsheets: implement incremental read + tests * #12003 source smartsheet: add changelog * #12003 source smartsheets: fix merge conflict on unit tests * #12003 source smartsheets: fix startdate in spec * #11759 source smartsheets: fix doc to certify for Beta
- Loading branch information
Showing
3 changed files
with
65 additions
and
61 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,93 +1,94 @@ | ||
# Smartsheets | ||
|
||
### Table of Contents | ||
This page guides you through the process of setting up the Smartsheets source connector. | ||
|
||
* [Sync Details](smartsheets.md#sync-details) | ||
* [Column datatype mapping](smartsheets.md#column-datatype-mapping) | ||
* [Features](smartsheets.md#Features) | ||
* [Performance Considerations](smartsheets.md#performance-considerations) | ||
* [Getting Started](smartsheets.md#getting-started) | ||
* [Requirements](smartsheets.md#requirements) | ||
* [Setup Guide](smartsheets.md#setup-guide) | ||
* [Configuring the source in the Airbyte UI](smartsheets.md#configuring-the-source-in-the-airbyte-ui) | ||
## Prerequisites | ||
|
||
## Sync Details | ||
|
||
The Smartsheet Source is written to pull data from a single Smartsheet spreadsheet. Unlike Google Sheets, Smartsheets only allows one sheet per Smartsheet - so a given Airbyte connector instance can sync only one sheet at a time. | ||
|
||
To replicate multiple spreadsheets, you can create multiple instances of the Smartsheet Source in Airbyte, reusing the API token for all your sheets that you need to sync. | ||
|
||
**Note: Column headers must contain only alphanumeric characters or `_` , as specified in the** [**Airbyte Protocol**](../../understanding-airbyte/airbyte-specification.md). | ||
To configure the Smartsheet Source for syncs, you'll need the following: | ||
|
||
### Column datatype mapping | ||
* A Smartsheets API access token - generated by a Smartsheets user with at least **read** access | ||
* The ID of the spreadsheet you'd like to sync | ||
|
||
The data type mapping adopted by this connector is based on the Smartsheet [documentation](https://smartsheet.redoc.ly/tag/columnsRelated#section/Column-Types). | ||
## Step 1: Set up Smartsheets | ||
|
||
**NOTE**: For any column datatypes interpreted by Smartsheets beside `DATE` and `DATETIME`, this connector's source schema generation assumes a `string` type, in which case the `format` field is not required by Airbyte. | ||
### Obtain a Smartsheets API access token | ||
|
||
| Integration Type | Airbyte Type | Airbyte Format | | ||
|:-----------------|:-------------|:---------------------| | ||
| `TEXT_NUMBER` | `string` | | | ||
| `DATE` | `string` | `format: date` | | ||
| `DATETIME` | `string` | `format: date-time` | | ||
| `anything else` | `string` | | | ||
You can generate an API key for your account from a session of your Smartsheet webapp by clicking: | ||
|
||
The remaining column datatypes supported by Smartsheets are more complex types \(e.g. Predecessor, Dropdown List\) and are not supported by this connector beyond its `string` representation. | ||
* Account (top-right icon) | ||
* Apps & Integrations | ||
* API Access | ||
* Generate new access token | ||
|
||
### Features | ||
For questions on advanced authorization flows, refer to [this](https://www.smartsheet.com/content-center/best-practices/tips-tricks/api-getting-started). | ||
|
||
This source connector only supports Full Refresh Sync. Since Smartsheets only allows 5000 rows per sheet, it's likely that the Full Refresh Sync Mode will suit the majority of use-cases. | ||
### Prepare the spreadsheet ID of your Smartsheet | ||
|
||
| Feature | Supported? | | ||
|:------------------|:-----------| | ||
| Full Refresh Sync | Yes | | ||
| Incremental Sync | No | | ||
| Namespaces | No | | ||
You'll also need the ID of the Spreadsheet you'd like to sync. Unlike Google Sheets, this ID is not found in the URL. You can find the required spreadsheet ID from your Smartsheet app session by going to: | ||
|
||
### Performance considerations | ||
* File | ||
* Properties | ||
|
||
At the time of writing, the [Smartsheets API rate limit](https://developers.smartsheet.com/blog/smartsheet-api-best-practices#:~:text=The%20Smartsheet%20API%20currently%20imposes,per%20minute%20per%20Access%20Token.) is 300 requests per minute per API access token. This connector makes 6 API calls per sync operation. | ||
## Step 2: Set up the Smartsheets connector in Airbyte | ||
|
||
## Getting started | ||
**For Airbyte Cloud:** | ||
|
||
### Requirements | ||
1. [Log into your Airbyte Cloud](https://cloud.airbyte.io/workspaces) account. | ||
2. In the left navigation bar, click Sources. In the top-right corner, click **+new source**. | ||
3. On the Set up the source page, enter the name for the Smartsheets connector and select **Smartsheets** from the Source type dropdown. | ||
4. Authenticate via OAuth2.0 using the API access token from Prerequisites | ||
5. Enter the start date and the ID of the spreadsheet you want to sync | ||
6. Submit the form | ||
|
||
To configure the Smartsheet Source for syncs, you'll need the following: | ||
**For Airbyte OSS:** | ||
1. Navigate to the Airbute Open Source dashboard | ||
2. Set the name for your source | ||
3. Enter the API access token from Prerequisites | ||
4. Enter the ID of the spreadsheet you want to sync | ||
5. Enter a start sync date | ||
6. Click **Set up source** | ||
|
||
* A Smartsheets API access token - generated by a Smartsheets user with at least **read** access | ||
* The ID of the spreadsheet you'd like to sync | ||
## Supported sync modes | ||
|
||
### Setup guide | ||
The Smartsheets source connector supports the following[ sync modes](https://docs.airbyte.com/cloud/core-concepts#connection-sync-modes): | ||
- Full Refresh | Overwrite | ||
- Full Refresh | Append | ||
- Incremental | Append | ||
- Incremental | Deduped | ||
|
||
#### Obtain a Smartsheets API access token | ||
## Performance considerations | ||
|
||
You can generate an API key for your account from a session of your Smartsheet webapp by clicking: | ||
At the time of writing, the [Smartsheets API rate limit](https://developers.smartsheet.com/blog/smartsheet-api-best-practices#be-practical-adhere-to-rate-limiting-guidelines) is 300 requests per minute per API access token. | ||
|
||
* Account \(top-right icon\) | ||
* Apps & Integrations | ||
* API Access | ||
* Generate new access token | ||
## Supported streams | ||
|
||
For questions on advanced authorization flows, refer to [this](https://www.smartsheet.com/content-center/best-practices/tips-tricks/api-getting-started). | ||
This source provides a single stream per spreadsheet with a dynamic schema, depending on your spreadsheet structure. | ||
For example, having a spreadsheet `Customers`, the connector would introduce a stream with the same name and properties typed according to Data type map (see [below](https://docs.airbyte.com/integrations/sources/smartsheets/#data-type-map)). | ||
|
||
#### The spreadsheet ID of your Smartsheet | ||
## Important highlights | ||
The Smartsheet Source is written to pull data from a single Smartsheet spreadsheet. Unlike Google Sheets, Smartsheets only allows one sheet per Smartsheet - so a given Airbyte connector instance can sync only one sheet at a time. To replicate multiple spreadsheets, you can create multiple instances of the Smartsheet Source in Airbyte, reusing the API token for all your sheets that you need to sync. | ||
|
||
You'll also need the ID of the Spreadsheet you'd like to sync. Unlike Google Sheets, this ID is not found in the URL. You can find the required spreadsheet ID from your Smartsheet app session by going to: | ||
**Note: Column headers must contain only alphanumeric characters or `_` , as specified in the** [**Airbyte Protocol**](../../understanding-airbyte/airbyte-specification.md). | ||
|
||
* File | ||
* Properties | ||
## Data type map | ||
The data type mapping adopted by this connector is based on the Smartsheet [documentation](https://smartsheet-platform.github.io/api-docs/index.html?python#column-types). | ||
|
||
### Configuring the source in the Airbyte UI | ||
**NOTE**: For any column datatypes interpreted by Smartsheets beside `DATE` and `DATETIME`, this connector's source schema generation assumes a `string` type, in which case the `format` field is not required by Airbyte. | ||
|
||
To setup your new Smartsheets source, Airbyte will need: | ||
| Integration Type | Airbyte Type | Airbyte Format | | ||
|:-----------------|:-------------|:---------------------| | ||
| `TEXT_NUMBER` | `string` | | | ||
| `DATE` | `string` | `format: date` | | ||
| `DATETIME` | `string` | `format: date-time` | | ||
| `anything else` | `string` | | | ||
|
||
1. Your API access token | ||
2. The spreadsheet ID | ||
The remaining column datatypes supported by Smartsheets are more complex types (e.g. Predecessor, Dropdown List) and are not supported by this connector beyond its `string` representation. | ||
|
||
## Changelog | ||
|
||
| Version | Date | Pull Request | Subject | | ||
|:--------|:-----------|:---------------------------------------------------------|:----------------------------------------------------------| | ||
| 0.1.10 | 2022-04-15 | [12077](https://github.com/airbytehq/airbyte/pull/12077) | Implement incremental read and improve code test coverage | | ||
| 0.1.11 | 2022-04-27 | [12203](https://github.com/airbytehq/airbyte/pull/12203) | Doc improvements | | ||
| 0.1.10 | 2022-04-15 | [12077](https://github.com/airbytehq/airbyte/pull/12077) | Implement incremental read and improve code test coverage | | ||
| 0.1.9 | 2022-04-12 | [11911](https://github.com/airbytehq/airbyte/pull/11911) | Bugfix: scrambled columns | | ||
| 0.1.8 | 2022-02-04 | [9792](https://github.com/airbytehq/airbyte/pull/9792) | Added oauth support | |