This tap pulls raw data from a AirTable database.
Feature Name | Supported | Comment |
---|---|---|
Full Import | ✅ | |
Partial Import | ❌ | No incremental import supported |
Start Date Selection | ❌ | |
Import Empty Tables | ❌ | Empty streams will not generate a table |
Custom Data | ✅ | Dynamic schemas |
Retroactive Updating | ❌ | No historical data |
Dynamic Column Selection | ✅ | Select optional, non mandatory columns |
This is a Singer tap that produces JSON-formatted data following the Singer spec.
- Schema generation is dynamic, the streams could not be known before execution
- Synch is done as FULL TABLE only
There is no predefined schema for this integration. The integration will dynamically load the tables & columns which are defined within your specific system. For this source you can use full imports. Every time the source syncs, it will fully get all your data.
Quotas
The API is limited to 5 requests per second, per base. If you exceed this rate, you will receive a 429 status code and will need to wait 30 seconds before subsequent requests will succeed. This limit is the same across all pricing tiers and increased rate limits are not currently available.
python3 -m venv ~/.virtualenvs/tap-airtable
source ~/.virtualenvs/tap-airtable/bin/activate
pip install -e .
Configuration Key | required | Description |
---|---|---|
metadata_url | - | Airtable metadata URL, at the time of the update: "https://api.airtable.com/v2/meta/" |
records_url | - | Airtable content URL, at the time of the update: "https://api.airtable.com/v0/" |
token | yes | Airtable Token |
base_id | - | Airtable base ID to export |
selected_by_default | - | Default for every table in the base. If set to true, all of the tables in the schema will be syncronized |
remove_emojis | - | Filter out emojis from the scyncronization |
{
"metadata_url":"https://api.airtable.com/v2/meta/",
"records_url":"https://api.airtable.com/v0/",
"token":"airtable_token",
"base_id": "airtable_base_id",
"selected_by_default": true,
"remove_emojis": false
}
The tap can be invoked in discovery mode to find the available tables and columns in the database:
$ tap-airtable --config config.json --discover
A discovered catalog is output, with a JSON-schema description of each table. A source table directly corresponds to a Singer stream.
The selected-by-default
fields is used to enable the sync of the tables. If set to 'true', all of the tables will be
selected in the properties.json
git clone https://github.com/datamill-co/target-postgres
cd target-postgres
python3 -m venv ~/.virtualenvs/target-postgres
source ~/.virtualenvs/target-postgres/bin/activate
pip install target-postgres
Complete the config.json
{
"metadata_url":"https://api.airtable.com/v2/meta/",
"records_url":"https://api.airtable.com/v0/",
"token":"airtable-api-key",
"base_id": "base-id",
"selected_by_default": true
}
From the home directory of the project
tap-airtable -c config.json --properties properties.json | ~/.virtualenvs/target-postgres/bin/target-postgres
Authentication: App Credentials
Settings: None
Schema type: Dynamic
Update Type: Full import
To authorize, an API Key/ Token should be generated first. For reference check here.