Skip to content

Commit

Permalink
Source Smartsheets: fix docs to certify for Beta (#12203)
Browse files Browse the repository at this point in the history 
* #5520 fix scrambled columns bug

* #5520 source smartsheets: add changelog item

* #5520 move pytest to optional setup requirements

* #12003 source smartsheets: implement incremental read + tests

* #12003 source smartsheet: add changelog

* #12003 source smartsheets: fix merge conflict on unit tests

* #12003 source smartsheets: fix startdate in spec

* #11759 source smartsheets: fix doc to certify for Beta
  • Loading branch information
@davydov-d
davydov-d authored Apr 28, 2022
1 parent fd4b71e commit 87beaf5
Show file tree
Hide file tree
Showing 3 changed files with 65 additions and 61 deletions.
2 changes: 1 addition & 1 deletion airbyte-integrations/connectors/source-smartsheets/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,5 @@ COPY $CODE_PATH ./$CODE_PATH
ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]

LABEL io.airbyte.version=0.1.10
LABEL io.airbyte.version=0.1.11
LABEL io.airbyte.name=airbyte/source-smartsheets
9 changes: 6 additions & 3 deletions airbyte-integrations/connectors/source-smartsheets/source_smartsheets/spec.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,20 +11,23 @@
"title": "Access Token",
"description": "The Access Token for making authenticated requests. Find in the main menu: Account > Apps & Integrations > API Access",
"type": "string",
"order": 0,
"airbyte_secret": true
},
"spreadsheet_id": {
"title": "Sheet ID",
"description": "The spreadsheet ID. Find in the spreadsheet menu: File > Properties",
"type": "string"
"type": "string",
"order": 1
},
"start_datetime": {
"title": "Start Datetime",
"title": "Start Datetime (Optional)",
"type": "string",
"examples": ["2000-01-01T13:00:00", "2000-01-01T13:00:00-07:00"],
"description": "ISO 8601, for instance: `YYYY-MM-DDTHH:MM:SS`, `YYYY-MM-DDTHH:MM:SS+HH:MM`",
"format": "date-time",
"default": "2020-01-01T00:00:00+00:00"
"default": "2020-01-01T00:00:00+00:00",
"order": 2
}
}
},
Expand Down
115 changes: 58 additions & 57 deletions docs/integrations/sources/smartsheets.md
View file
Original file line number Diff line number Diff line change
@@ -1,93 +1,94 @@
# Smartsheets

### Table of Contents
This page guides you through the process of setting up the Smartsheets source connector.

* [Sync Details](smartsheets.md#sync-details)
* [Column datatype mapping](smartsheets.md#column-datatype-mapping)
* [Features](smartsheets.md#Features)
* [Performance Considerations](smartsheets.md#performance-considerations)
* [Getting Started](smartsheets.md#getting-started)
* [Requirements](smartsheets.md#requirements)
* [Setup Guide](smartsheets.md#setup-guide)
* [Configuring the source in the Airbyte UI](smartsheets.md#configuring-the-source-in-the-airbyte-ui)
## Prerequisites

## Sync Details

The Smartsheet Source is written to pull data from a single Smartsheet spreadsheet. Unlike Google Sheets, Smartsheets only allows one sheet per Smartsheet - so a given Airbyte connector instance can sync only one sheet at a time.

To replicate multiple spreadsheets, you can create multiple instances of the Smartsheet Source in Airbyte, reusing the API token for all your sheets that you need to sync.

**Note: Column headers must contain only alphanumeric characters or `_` , as specified in the** [**Airbyte Protocol**](../../understanding-airbyte/airbyte-specification.md).
To configure the Smartsheet Source for syncs, you'll need the following:

### Column datatype mapping
* A Smartsheets API access token - generated by a Smartsheets user with at least **read** access
* The ID of the spreadsheet you'd like to sync

The data type mapping adopted by this connector is based on the Smartsheet [documentation](https://smartsheet.redoc.ly/tag/columnsRelated#section/Column-Types).
## Step 1: Set up Smartsheets

**NOTE**: For any column datatypes interpreted by Smartsheets beside `DATE` and `DATETIME`, this connector's source schema generation assumes a `string` type, in which case the `format` field is not required by Airbyte.
### Obtain a Smartsheets API access token

| Integration Type | Airbyte Type | Airbyte Format |
|:-----------------|:-------------|:---------------------|
| `TEXT_NUMBER` | `string` | |
| `DATE` | `string` | `format: date` |
| `DATETIME` | `string` | `format: date-time` |
| `anything else` | `string` | |
You can generate an API key for your account from a session of your Smartsheet webapp by clicking:

The remaining column datatypes supported by Smartsheets are more complex types \(e.g. Predecessor, Dropdown List\) and are not supported by this connector beyond its `string` representation.
* Account (top-right icon)
* Apps & Integrations
* API Access
* Generate new access token

### Features
For questions on advanced authorization flows, refer to [this](https://www.smartsheet.com/content-center/best-practices/tips-tricks/api-getting-started).

This source connector only supports Full Refresh Sync. Since Smartsheets only allows 5000 rows per sheet, it's likely that the Full Refresh Sync Mode will suit the majority of use-cases.
### Prepare the spreadsheet ID of your Smartsheet

| Feature | Supported? |
|:------------------|:-----------|
| Full Refresh Sync | Yes |
| Incremental Sync | No |
| Namespaces | No |
You'll also need the ID of the Spreadsheet you'd like to sync. Unlike Google Sheets, this ID is not found in the URL. You can find the required spreadsheet ID from your Smartsheet app session by going to:

### Performance considerations
* File
* Properties

At the time of writing, the [Smartsheets API rate limit](https://developers.smartsheet.com/blog/smartsheet-api-best-practices#:~:text=The%20Smartsheet%20API%20currently%20imposes,per%20minute%20per%20Access%20Token.) is 300 requests per minute per API access token. This connector makes 6 API calls per sync operation.
## Step 2: Set up the Smartsheets connector in Airbyte

## Getting started
**For Airbyte Cloud:**

### Requirements
1. [Log into your Airbyte Cloud](https://cloud.airbyte.io/workspaces) account.
2. In the left navigation bar, click Sources. In the top-right corner, click **+new source**.
3. On the Set up the source page, enter the name for the Smartsheets connector and select **Smartsheets** from the Source type dropdown.
4. Authenticate via OAuth2.0 using the API access token from Prerequisites
5. Enter the start date and the ID of the spreadsheet you want to sync
6. Submit the form

To configure the Smartsheet Source for syncs, you'll need the following:
**For Airbyte OSS:**
1. Navigate to the Airbute Open Source dashboard
2. Set the name for your source
3. Enter the API access token from Prerequisites
4. Enter the ID of the spreadsheet you want to sync
5. Enter a start sync date
6. Click **Set up source**

* A Smartsheets API access token - generated by a Smartsheets user with at least **read** access
* The ID of the spreadsheet you'd like to sync
## Supported sync modes

### Setup guide
The Smartsheets source connector supports the following[ sync modes](https://docs.airbyte.com/cloud/core-concepts#connection-sync-modes):
- Full Refresh | Overwrite
- Full Refresh | Append
- Incremental | Append
- Incremental | Deduped

#### Obtain a Smartsheets API access token
## Performance considerations

You can generate an API key for your account from a session of your Smartsheet webapp by clicking:
At the time of writing, the [Smartsheets API rate limit](https://developers.smartsheet.com/blog/smartsheet-api-best-practices#be-practical-adhere-to-rate-limiting-guidelines) is 300 requests per minute per API access token.

* Account \(top-right icon\)
* Apps & Integrations
* API Access
* Generate new access token
## Supported streams

For questions on advanced authorization flows, refer to [this](https://www.smartsheet.com/content-center/best-practices/tips-tricks/api-getting-started).
This source provides a single stream per spreadsheet with a dynamic schema, depending on your spreadsheet structure.
For example, having a spreadsheet `Customers`, the connector would introduce a stream with the same name and properties typed according to Data type map (see [below](https://docs.airbyte.com/integrations/sources/smartsheets/#data-type-map)).

#### The spreadsheet ID of your Smartsheet
## Important highlights
The Smartsheet Source is written to pull data from a single Smartsheet spreadsheet. Unlike Google Sheets, Smartsheets only allows one sheet per Smartsheet - so a given Airbyte connector instance can sync only one sheet at a time. To replicate multiple spreadsheets, you can create multiple instances of the Smartsheet Source in Airbyte, reusing the API token for all your sheets that you need to sync.

You'll also need the ID of the Spreadsheet you'd like to sync. Unlike Google Sheets, this ID is not found in the URL. You can find the required spreadsheet ID from your Smartsheet app session by going to:
**Note: Column headers must contain only alphanumeric characters or `_` , as specified in the** [**Airbyte Protocol**](../../understanding-airbyte/airbyte-specification.md).

* File
* Properties
## Data type map
The data type mapping adopted by this connector is based on the Smartsheet [documentation](https://smartsheet-platform.github.io/api-docs/index.html?python#column-types).

### Configuring the source in the Airbyte UI
**NOTE**: For any column datatypes interpreted by Smartsheets beside `DATE` and `DATETIME`, this connector's source schema generation assumes a `string` type, in which case the `format` field is not required by Airbyte.

To setup your new Smartsheets source, Airbyte will need:
| Integration Type | Airbyte Type | Airbyte Format |
|:-----------------|:-------------|:---------------------|
| `TEXT_NUMBER` | `string` | |
| `DATE` | `string` | `format: date` |
| `DATETIME` | `string` | `format: date-time` |
| `anything else` | `string` | |

1. Your API access token
2. The spreadsheet ID
The remaining column datatypes supported by Smartsheets are more complex types (e.g. Predecessor, Dropdown List) and are not supported by this connector beyond its `string` representation.

## Changelog

| Version | Date | Pull Request | Subject |
|:--------|:-----------|:---------------------------------------------------------|:----------------------------------------------------------|
| 0.1.10 | 2022-04-15 | [12077](https://github.com/airbytehq/airbyte/pull/12077) | Implement incremental read and improve code test coverage |
| 0.1.11 | 2022-04-27 | [12203](https://github.com/airbytehq/airbyte/pull/12203) | Doc improvements |
| 0.1.10 | 2022-04-15 | [12077](https://github.com/airbytehq/airbyte/pull/12077) | Implement incremental read and improve code test coverage |
| 0.1.9 | 2022-04-12 | [11911](https://github.com/airbytehq/airbyte/pull/11911) | Bugfix: scrambled columns |
| 0.1.8 | 2022-02-04 | [9792](https://github.com/airbytehq/airbyte/pull/9792) | Added oauth support |

0 comments on commit 87beaf5

Please sign in to comment.