Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

📖 Removes $ from terminal commands to allow direct copying. #17467

Merged
merged 2 commits into from
Sep 30, 2022
Merged

📖 Removes $ from terminal commands to allow direct copying. #17467

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2f4e878
Removes $ from terminal commands to allow direct copying.
Sep 30, 2022
241b113
Fixes build by adding missing link.
Sep 30, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/connector-development/config-based/tutorial/1-create-source.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@
Let's start by cloning the Airbyte repository:

```bash
$ git clone git@github.com:airbytehq/airbyte.git
$ cd airbyte
git clone git@github.com:airbytehq/airbyte.git
cd airbyte
```

Airbyte provides a code generator which bootstraps the scaffolding for our connector.

```bash
$ cd airbyte-integrations/connector-templates/generator
$ ./generate.sh
cd airbyte-integrations/connector-templates/generator
./generate.sh
```

This will bring up an interactive helper application. Use the arrow keys to pick a template from the list. Select the `Configuration Based Source` template and then input the name of your connector. The application will create a new directory in `airbyte/airbyte-integrations/connectors/` with the name of your new connector.
Expand Down
10 changes: 5 additions & 5 deletions docs/connector-development/config-based/tutorial/2-install-dependencies.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,18 @@ If this is the case on your machine, substitute the `python` commands with `pyth
The subsequent `python` invocations will use the virtual environment created for the connector.

```bash
$ cd ../../connectors/source-exchange-rates-tutorial
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install -r requirements.txt
cd ../../connectors/source-exchange-rates-tutorial
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```

These steps create an initial python environment, and install the dependencies required to run an API Source connector.

Let's verify everything works as expected by running the Airbyte `spec` operation:

```bash
$ python main.py spec
python main.py spec
```

You should see an output similar to the one below:
Expand Down
4 changes: 2 additions & 2 deletions docs/connector-development/config-based/tutorial/3-connecting-to-the-API-source.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ connectionSpecification:
Because of the sensitive nature of the access key, we recommend storing this config in the `secrets` directory because it is ignored by git.

```bash
$ echo '{"access_key": "<your_access_key>", "base": "USD"}' > secrets/config.json
echo '{"access_key": "<your_access_key>", "base": "USD"}' > secrets/config.json
```

## Updating the connector definition
Expand Down Expand Up @@ -197,7 +197,7 @@ check:
We can now run the `check` operation, which verifies the connector can connect to the API source.

```bash
$ python main.py check --config secrets/config.json
python main.py check --config secrets/config.json
```

which should now succeed with logs similar to:
Expand Down
10 changes: 5 additions & 5 deletions docs/connector-development/config-based/tutorial/4-reading-data.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,20 @@ Let's define the stream schema in `source-exchange-rates-tutorial/source_exchang
You can download the JSON file describing the output schema with all currencies [here](https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-cdk/python/docs/tutorials/http_api_source_assets/exchange_rates.json) for convenience and place it in `schemas/`.

```bash
$ curl https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-cdk/python/docs/tutorials/http_api_source_assets/exchange_rates.json > source_exchange_rates_tutorial/schemas/rates.json
curl https://raw.githubusercontent.com/airbytehq/airbyte/master/airbyte-cdk/python/docs/tutorials/http_api_source_assets/exchange_rates.json > source_exchange_rates_tutorial/schemas/rates.json
```

We can also delete the boilerplate schema files

```bash
$ rm source_exchange_rates_tutorial/schemas/customers.json
$ rm source_exchange_rates_tutorial/schemas/employees.json
rm source_exchange_rates_tutorial/schemas/customers.json
rm source_exchange_rates_tutorial/schemas/employees.json
```

Reading from the source can be done by running the `read` operation

```bash
$ python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
```

The logs should show that 1 record was read from the stream.
Expand All @@ -55,7 +55,7 @@ The logs should show that 1 record was read from the stream.
The `--debug` flag can be set to print out debug information, including the outgoing request and its associated response

```bash
$ python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json --debug
python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json --debug
```

## Next steps
Expand Down
6 changes: 3 additions & 3 deletions docs/connector-development/config-based/tutorial/5-incremental-reads.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ streams:
You can test these changes by executing the `read` operation:

```bash
$ python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
```

By reading the output record, you should see that we read historical data instead of the latest exchange rate.
Expand Down Expand Up @@ -226,7 +226,7 @@ check:
Running the `read` operation will now read all data for all days between start_date and now:

```bash
$ python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json
```

The operation should now output more than one record:
Expand Down Expand Up @@ -281,7 +281,7 @@ We can simulate incremental syncs by creating a state file containing the last s
Running the `read` operation will now only read data for dates later than the given state:

```bash
$ python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json --state integration_tests/sample_state.json
python main.py read --config secrets/config.json --catalog integration_tests/configured_catalog.json --state integration_tests/sample_state.json
```

There shouldn't be any data read if the state is today's date:
Expand Down
4 changes: 2 additions & 2 deletions docs/connector-development/config-based/tutorial/6-testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ and `integration_tests/abnormal_state.json` with
You can run the acceptance tests with the following commands:

```bash
$ docker build . -t airbyte/source-exchange-rates-tutorial:dev
$ python -m pytest integration_tests -p integration_tests.acceptance
docker build . -t airbyte/source-exchange-rates-tutorial:dev
python -m pytest integration_tests -p integration_tests.acceptance
```

## Next steps:
Expand Down
2 changes: 1 addition & 1 deletion docs/integrations/sources/dv360.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,4 +67,4 @@ This source is constrained by the limits set by the DBM API. You can read more a

| Version | Date | Pull Request | Subject |
| :--- | :--- | :--- | :--- |
| 0.1.1 | 2022--| []() | Release Native Display & Video 360 Connector |
| 0.1.1 | 2022--| [11828](https://github.com/airbytehq/airbyte/pull/11828) | Release Native Display & Video 360 Connector |