Add Content Tab to document python and Console SDK #613
Conversation
docs/guides/first_pipeline.md
Outdated
|
|
||
| ``` | ||
| fondant run local pipeline.py | ||
| ``` | ||
|
|
||
| Note: For local testing purposes, the pipeline will only download the first 10,000 images. If you want to download the full dataset, you will need to modify the component arguments in the pipeline.py file, specifically the following part: | ||
| **Note:** For local testing purposes, the pipeline will only download the first 10,000 images. If you want to download the full dataset, you will need to modify the component arguments in the pipeline.py file, specifically the following part: |
There was a problem hiding this comment.
FYI, we can add actual notes: https://squidfunk.github.io/mkdocs-material/reference/admonitions
There was a problem hiding this comment.
I see you did this below :)
docs/pipeline.md
Outdated
| @@ -70,33 +70,131 @@ dependencies between components to build the DAG. | |||
| Currently Fondant supports linear DAGs with single dependencies. Support for non-linear DAGs will be | |||
| available in future releases. | |||
|
|
|||
| ## Compiling and Running a pipeline | |||
| ## Compiling a pipeline | |||
|
|
|||
| Once all your components are added to your pipeline you can use different compilers to run your | |||
| pipeline: | |||
|
|
|||
| !!! note "IMPORTANT" | |||
| When using other runners you will need to make sure that your new environment has access to: | |||
docs/runners/local.md
Outdated
| gcp_mounr_dir = CloudCredentialsMount.GCP.value | ||
| compiler = DockerCompiler(extra_volumnes=gcp_mounr_dir) |
There was a problem hiding this comment.
| gcp_mounr_dir = CloudCredentialsMount.GCP.value | |
| compiler = DockerCompiler(extra_volumnes=gcp_mounr_dir) | |
| gcp_mount_dir = CloudCredentialsMount.GCP.value | |
| compiler = DockerCompiler(extra_volumes=gcp_mount_dir) |
I also think extra_volumes needs to be a list.
Same changes in the other tabs.
There was a problem hiding this comment.
I changed the script to also accept a string (single volume to mount)
There was a problem hiding this comment.
Missed that 😅
There's also some typo's in these lines though.
There was a problem hiding this comment.
woops fixed it
There was a problem hiding this comment.
One more typo remaining 🙂
"extra_volumnes" -> "extra_volumes"
Co-authored-by: Robbe Sneyders <robbe.sneyders@gmail.com>
docs/runners/local.md
Outdated
| fondant run local <pipeline_ref> | ||
| ``` | ||
|
|
||
| If you want to use remote paths (GCS, S3, etc.) you can use the `--auth-gcp`, `--auth-aws` or `--auth-azure`. |
There was a problem hiding this comment.
Maybe it makes sense to split it up as well (similar to the example below).
Selecting/activate the right session/profile for different cloud providers works differently.
If we would add the commands for each cloud provider everything would be here on the page.
Added Content Tabs to document the CLI and python SDK