Skip to content

Commit

Permalink
Improving docusaurus sidebar generation (#1927) (#14369)
Browse files Browse the repository at this point in the history
* Improving docusaurus sidebar generation (#1927)

* Added "Career & open positions" folder to sidebar, adjusted "Project overview" folder

* Deleted "career-and-open-positions" folder from sidebar
  • Loading branch information
Mykyta-Serbynevskyi authored Jul 8, 2022
1 parent e1c2ca2 commit a2c194a
Show file tree
Hide file tree
Showing 379 changed files with 600 additions and 16,653 deletions.
29 changes: 29 additions & 0 deletions docs/01-homepage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
---
slug: /
---

# Welcome to Airbyte Docs

Whether you are an Airbyte user or contributor, we have docs for you!

### For Airbyte Cloud users

Browse the [connector catalog](02-integrations/README.md) to find the connector you want. In case the connector is not yet supported on Airbyte Cloud, consider using [Airbyte Open Source](#for-airbyte-open-source-users).

Next, check out the [step-by-step tutorial](03-cloud/README.md) to sign up for Airbyte Cloud, understand Airbyte [concepts](03-cloud/01-core-concepts.md), and run your first sync. Then learn how to [manage your Airbyte Cloud account](03-cloud/02-managing-airbyte-cloud.md).

### For Airbyte Open Source users

Browse the [connector catalog](02-integrations/README.md) to find the connector you want. If the connector is not yet supported on Airbyte Open Source, [build your own connector](08-connector-development/README.md).

Next, check out the [Airbyte Open Source QuickStart](04-quickstart/01-deploy-airbyte.md). Then learn how to [deploy](05-deploying-airbyte/01-local-deployment.md) and [manage](06-operator-guides/01-upgrading-airbyte.md) Airbyte Open Source in your cloud infrastructure.


To get help with Airbyte deployments, check out the [Troubleshooting & FAQ](07-troubleshooting/README.md), chat with Support on [Discourse](https://discuss.airbyte.io/), or join us on [Community Slack](https://slack.airbyte.io/).

### For Airbyte contributors

To contribute to Airbyte code, connectors, and documentation, refer to our [Contributing Guide](09-contributing-to-airbyte/README.md).

[![GitHub stars](https://img.shields.io/github/stars/airbytehq/airbyte?style=social&label=Star&maxAge=2592000)](https://github.com/airbytehq/airbyte/stargazers/) [![GitHub Workflow Status](https://img.shields.io/github/workflow/status/airbytehq/airbyte/Airbyte%20CI)](https://github.com/airbytehq/airbyte/actions/workflows/gradle.yml) [![License](https://img.shields.io/static/v1?label=license&message=MIT&color=brightgreen)](https://github.com/airbytehq/airbyte/tree/a9b1c6c0420550ad5069aca66c295223e0d05e27/LICENSE/README.md) [![License](https://img.shields.io/static/v1?label=license&message=ELv2&color=brightgreen)](https://github.com/airbytehq/airbyte/tree/a9b1c6c0420550ad5069aca66c295223e0d05e27/LICENSE/README.md)

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ Setup through Airbyte Cloud will be exactly the same as the open-source setup, e

#### Provider Specific Information

* In case of GCS, it is necessary to provide the content of the service account keyfile to access private buckets. See settings of [BigQuery Destination](../destinations/bigquery.md)
* In case of GCS, it is necessary to provide the content of the service account keyfile to access private buckets. See settings of [BigQuery Destination](../02-destinations/bigquery.md)
* In case of AWS S3, the pair of `aws_access_key_id` and `aws_secret_access_key` is necessary to access private S3 buckets.
* In case of AzBlob, it is necessary to provide the `storage_account` in which the blob you want to access resides. Either `sas_token` [(info)](https://docs.microsoft.com/en-us/azure/storage/blobs/sas-service-create?tabs=dotnet) or `shared_key` [(info)](https://docs.microsoft.com/en-us/azure/storage/common/storage-account-keys-manage?tabs=azure-portal) is necessary to access private blobs.

Expand Down Expand Up @@ -119,7 +119,7 @@ Please see (or add) more at `airbyte-integrations/connectors/source-file/integra

In order to read large files from a remote location, this connector uses the [smart\_open](https://pypi.org/project/smart-open/) library. However, it is possible to switch to either [GCSFS](https://gcsfs.readthedocs.io/en/latest/) or [S3FS](https://s3fs.readthedocs.io/en/latest/) implementations as it is natively supported by the `pandas` library. This choice is made possible through the optional `reader_impl` parameter.

* Note that for local filesystem, the file probably have to be stored somewhere in the `/tmp/airbyte_local` folder with the same limitations as the [CSV Destination](../destinations/local-csv.md) so the `URL` should also starts with `/local/`.
* Note that for local filesystem, the file probably have to be stored somewhere in the `/tmp/airbyte_local` folder with the same limitations as the [CSV Destination](../02-destinations/local-csv.md) so the `URL` should also starts with `/local/`.
* The JSON implementation needs to be tweaked in order to produce more complex catalog and is still in an experimental state: Simple JSON schemas should work at this point but may not be well handled when there are multiple layers of nesting.

## Changelog
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ To set up Google Sheets as a source in Airbyte OSS:

Each sheet in the selected spreadsheet is synced as a separate stream. Each selected column in the sheet is synced as a string field.

**Note: Sheet names and column headers must contain only alphanumeric characters or `_`, as specified in the** [**Airbyte Protocol**](../../understanding-airbyte/airbyte-protocol.md). For example, if your sheet or column header is named `the data`, rename it to `the_data`. This restriction does not apply to non-header cell values.
**Note: Sheet names and column headers must contain only alphanumeric characters or `_`, as specified in the** [**Airbyte Protocol**](../../10-understanding-airbyte/02-airbyte-protocol.md). For example, if your sheet or column header is named `the data`, rename it to `the_data`. This restriction does not apply to non-header cell values.

Airbyte only supports replicating [Grid](https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets/sheets#SheetType) sheets.

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

:::caution

This connector is graveyarded and will not be receiving any updates from the Airbyte team. Its functionalities have been replaced by the [Airbyte CDK](../../connector-development/cdk-python/README.md), which allows you to create source connectors for any HTTP API.
This connector is graveyarded and will not be receiving any updates from the Airbyte team. Its functionalities have been replaced by the [Airbyte CDK](../../08-connector-development/07-cdk-python/README.md), which allows you to create source connectors for any HTTP API.

:::

## Overview

This connector allows you to generally connect to any HTTP API. In order to use this connector, you must manually bring it in as a custom connector. The steps to do this can be found [here](../../connector-development/tutorials/cdk-tutorial-python-http/7-use-connector-in-airbyte.md).
This connector allows you to generally connect to any HTTP API. In order to use this connector, you must manually bring it in as a custom connector. The steps to do this can be found [here](../../08-connector-development/02-cdk-tutorial-python-http/7-use-connector-in-airbyte.md).

## Where do I find the Docker image?

Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ We use [SQL Server's change data capture feature](https://docs.microsoft.com/en-

Some extra setup requiring at least _db\_owner_ permissions on the database\(s\) you intend to sync from will be required \(detailed [below](mssql.md#setting-up-cdc-for-mssql)\).

Please read the [CDC docs](../../understanding-airbyte/cdc.md) for an overview of how Airbyte approaches CDC.
Please read the [CDC docs](../../10-understanding-airbyte/10-cdc.md) for an overview of how Airbyte approaches CDC.

### Should I use CDC for MSSQL?

Expand All @@ -66,7 +66,7 @@ Please read the [CDC docs](../../understanding-airbyte/cdc.md) for an overview o

#### CDC Limitations

* Make sure to read our [CDC docs](../../understanding-airbyte/cdc.md) to see limitations that impact all databases using CDC replication.
* Make sure to read our [CDC docs](../../10-understanding-airbyte/10-cdc.md) to see limitations that impact all databases using CDC replication.
* There are some critical issues regarding certain datatypes. Please find detailed info in [this Github issue](https://github.com/airbytehq/airbyte/issues/4542).
* CDC is only available for SQL Server 2016 Service Pack 1 \(SP1\) and later.
* _db\_owner_ \(or higher\) permissions are required to perform the [neccessary setup](mssql.md#setting-up-cdc-for-mssql) for CDC.
Expand Down
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ Your database user should now be ready for use with Airbyte.

#### CDC Limitations

* Make sure to read our [CDC docs](../../understanding-airbyte/cdc.md) to see limitations that impact all databases using CDC replication.
* Make sure to read our [CDC docs](../../10-understanding-airbyte/10-cdc.md) to see limitations that impact all databases using CDC replication.
* Our CDC implementation uses at least once delivery for all change records.

**1. Enable binary logging**
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@

The PokéAPI is primarly used as a tutorial and educational resource, as it requires zero dependencies. Learn how Airbyte and this connector works with these tutorials:

- [Airbyte Quickstart: An Introduction to Deploying and Syncing](../../quickstart/deploy-airbyte.md)
- [Airbyte CDK Speedrun: A Quick Primer on Building Source Connectors](../../connector-development/tutorials/cdk-speedrun.md)
- [Airbyte Quickstart: An Introduction to Deploying and Syncing](../../04-quickstart/01-deploy-airbyte.md)
- [Airbyte CDK Speedrun: A Quick Primer on Building Source Connectors](../../08-connector-development/01-cdk-speedrun.md)
- [How to Build ETL Sources in Under 30 Minutes: A Video Tutorial](https://www.youtube.com/watch?v=kJ3hLoNfz_E&t=13s&ab_channel=Airbyte)

## Features
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ We use [logical replication](https://www.postgresql.org/docs/10/logical-replicat

We use `pgoutput` as a default plugin, which is included in Postgres 10+. Also `wal2json` plugin is supported, please read [the section on replication plugins below](postgres.md#select-replication-plugin) for more information.

Please read the [CDC docs](../../understanding-airbyte/cdc.md) for an overview of how Airbyte approaches CDC.
Please read the [CDC docs](../../10-understanding-airbyte/10-cdc.md) for an overview of how Airbyte approaches CDC.

### Should I use CDC for Postgres?

Expand All @@ -118,7 +118,7 @@ Please read the [CDC docs](../../understanding-airbyte/cdc.md) for an overview o

#### CDC Limitations

* Make sure to read our [CDC docs](../../understanding-airbyte/cdc.md) to see limitations that impact all databases using CDC replication.
* Make sure to read our [CDC docs](../../10-understanding-airbyte/10-cdc.md) to see limitations that impact all databases using CDC replication.
* CDC is only available for Postgres 10+.
* Airbyte requires a replication slot configured only for its use. Only one source should be configured that uses this replication slot. Instructions on how to set up a replication slot can be found below.
* Log-based replication only works for master instances of Postgres.
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ For example, having a spreadsheet `Customers`, the connector would introduce a s
## Important highlights
The Smartsheet Source is written to pull data from a single Smartsheet spreadsheet. Unlike Google Sheets, Smartsheets only allows one sheet per Smartsheet - so a given Airbyte connector instance can sync only one sheet at a time. To replicate multiple spreadsheets, you can create multiple instances of the Smartsheet Source in Airbyte, reusing the API token for all your sheets that you need to sync.

**Note: Column headers must contain only alphanumeric characters or `_` , as specified in the** [**Airbyte Protocol**](../../understanding-airbyte/airbyte-protocol.md).
**Note: Column headers must contain only alphanumeric characters or `_` , as specified in the** [**Airbyte Protocol**](../../10-understanding-airbyte/02-airbyte-protocol.md).

## Data type map
The data type mapping adopted by this connector is based on the Smartsheet [documentation](https://smartsheet-platform.github.io/api-docs/index.html?python#column-types).
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
1 change: 1 addition & 0 deletions docs/02-integrations/02-destinations/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Destinations
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -63,5 +63,5 @@ You can also copy the output file to your host machine, the following command wi
docker cp airbyte-scheduler:/tmp/airbyte_local/{destination_path}/{filename}.csv .
```

Note: If you are running Airbyte on Windows with Docker backed by WSL2, you have to use similar step as above or refer to this [link](../../operator-guides/locating-files-local-destination.md) for an alternative approach.
Note: If you are running Airbyte on Windows with Docker backed by WSL2, you have to use similar step as above or refer to this [link](../../06-operator-guides/08-locating-files-local-destination.md) for an alternative approach.

Original file line number Diff line number Diff line change
Expand Up @@ -63,5 +63,5 @@ You can also copy the output file to your host machine, the following command wi
docker cp airbyte-scheduler:/tmp/airbyte_local/{destination_path}/{filename}.jsonl .
```

Note: If you are running Airbyte on Windows with Docker backed by WSL2, you have to use similar step as above or refer to this [link](../../operator-guides/locating-files-local-destination.md) for an alternative approach.
Note: If you are running Airbyte on Windows with Docker backed by WSL2, you have to use similar step as above or refer to this [link](../../06-operator-guides/08-locating-files-local-destination.md) for an alternative approach.

File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Loading

0 comments on commit a2c194a

Please sign in to comment.