Skip to content

Commit

Permalink
Adding example 16 to the 01_data_catalog.md to show how to load csv f… (
Browse files Browse the repository at this point in the history
  • Loading branch information
Isy89 authored Jan 31, 2022
1 parent eb82e89 commit 02598bf
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions docs/source/05_data/01_data_catalog.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,27 @@ dev_abs:
account_name: accountname
account_key: key
```
Example 16: Loading a CSV file stored in a remote location through SSH

```eval_rst
.. note:: This example requires [Paramiko](https://www.paramiko.org) to be installed (`pip install paramiko`).
```
```yaml
cool_dataset:
type: pandas.CSVDataSet
filepath: "sftp:///path/to/remote_cluster/cool_data.csv"
credentials: cluster_credentials
```
All parameters required to establish the SFTP connection can be defined through `fs_args` or in `credentials.yml` as follows:

```yaml
cluster_credentials:
username: my_username
host: host_address
port: 22
password: password
```
The list of all available parameters is given in the [Paramiko documentation](https://docs.paramiko.org/en/2.4/api/client.html#paramiko.client.SSHClient.connect).

## Creating a Data Catalog YAML configuration file via CLI

Expand Down

0 comments on commit 02598bf

Please sign in to comment.