-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create spaceflights-pandas-viz
starter
#152
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
5cb0aa7
Add files to spaceflights-viz
merelcht 8f5d0fb
Rename to spaceflights-pandas-viz + lint
merelcht ee4a4d7
Minor edits
merelcht a0b843b
Update readme, should be revised in a later task
merelcht 8734ad6
Merge branch 'main' into create-spaceflights-viz
merelcht File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,43 @@ | ||
# The `spaceflights-pandas-viz` Kedro starter | ||
|
||
## Overview | ||
|
||
This is a completed version of the [spaceflights tutorial project](https://docs.kedro.org/en/stable/tutorial/spaceflights_tutorial.html) described in the [online Kedro documentation](https://docs.kedro.org) and the extra tutorial sections on [visualisation with Kedro-Viz](https://docs.kedro.org/en/stable/visualisation/index.html) and [experiment tracking with Kedro-Viz](https://docs.kedro.org/en/stable/experiment_tracking/index.html). This project includes the data required to run it. | ||
|
||
The tutorial works through the steps necessary to create this project. To learn the most about Kedro, we recommend that you start with a blank template as the tutorial describes, and follow the workflow. However, if you prefer to read swiftly through the documentation and get to work on the code, you may want to generate a new Kedro project using this [starter](https://docs.kedro.org/en/stable/kedro_project_setup/starters.html) because the steps have been done for you. | ||
|
||
To use this starter, create a new Kedro project using the commands below. To make sure you have the required dependencies, run it in your virtual environment (see [our documentation about virtual environments](https://docs.kedro.org/en/stable/get_started/install.html#virtual-environments) for guidance on how to get set up): | ||
|
||
```bash | ||
pip install kedro | ||
kedro new --starter=spaceflights-pandas-viz | ||
cd <my-project-name> # change directory into newly created project directory | ||
``` | ||
|
||
This will give you the complete project and project template. If you would prefer to have a reduced project template you can use `add-ons` instead and select `Kedro-Viz` as add-on with an example: | ||
```bash | ||
pip install kedro | ||
kedro new --add-ons=XXX | ||
cd <my-project-name> # change directory into newly created project directory | ||
``` | ||
|
||
Install the required dependencies: | ||
|
||
```bash | ||
pip install -r src/requirements.txt | ||
``` | ||
|
||
Now you can run the project: | ||
|
||
```bash | ||
kedro run | ||
``` | ||
|
||
To visualise the default pipeline, run: | ||
```bash | ||
kedro viz | ||
``` | ||
|
||
This will open the default browser and display the following pipeline visualisation: | ||
|
||
![](./images/pipeline_visualisation_with_layers.png) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
{ | ||
"project_name": "Spaceflights Pandas Viz", | ||
"repo_name": "{{ cookiecutter.project_name.strip().replace(' ', '-').replace('_', '-').lower() }}", | ||
"python_package": "{{ cookiecutter.project_name.strip().replace(' ', '_').replace('-', '_').lower() }}", | ||
"kedro_version": "{{ cookiecutter.kedro_version }}" | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
project_name: | ||
title: "Project Name" | ||
text: | | ||
Please enter a human readable name for your new project. | ||
Spaces, hyphens, and underscores are allowed. | ||
regex_validator: "^[\\w -]{2,}$" | ||
error_message: | | ||
It must contain only alphanumeric symbols, spaces, underscores and hyphens and | ||
be at least 2 characters long. |
151 changes: 151 additions & 0 deletions
151
spaceflights-pandas-viz/{{ cookiecutter.repo_name }}/.gitignore
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,151 @@ | ||
########################## | ||
# KEDRO PROJECT | ||
|
||
# ignore all local configuration | ||
conf/local/** | ||
!conf/local/.gitkeep | ||
|
||
# ignore potentially sensitive credentials files | ||
conf/**/*credentials* | ||
|
||
# ignore everything in the following folders | ||
data/** | ||
|
||
# except their sub-folders | ||
!data/**/ | ||
|
||
# also keep all .gitkeep files | ||
!.gitkeep | ||
|
||
# keep also the example dataset | ||
!data/01_raw/* | ||
|
||
|
||
########################## | ||
# Common files | ||
|
||
# IntelliJ | ||
.idea/ | ||
*.iml | ||
out/ | ||
.idea_modules/ | ||
|
||
### macOS | ||
*.DS_Store | ||
.AppleDouble | ||
.LSOverride | ||
.Trashes | ||
|
||
# Vim | ||
*~ | ||
.*.swo | ||
.*.swp | ||
|
||
# emacs | ||
*~ | ||
\#*\# | ||
/.emacs.desktop | ||
/.emacs.desktop.lock | ||
*.elc | ||
|
||
# JIRA plugin | ||
atlassian-ide-plugin.xml | ||
|
||
# C extensions | ||
*.so | ||
|
||
### Python template | ||
# Byte-compiled / optimized / DLL files | ||
__pycache__/ | ||
*.py[cod] | ||
*$py.class | ||
|
||
# Distribution / packaging | ||
.Python | ||
build/ | ||
develop-eggs/ | ||
dist/ | ||
downloads/ | ||
eggs/ | ||
.eggs/ | ||
lib/ | ||
lib64/ | ||
parts/ | ||
sdist/ | ||
var/ | ||
wheels/ | ||
*.egg-info/ | ||
.installed.cfg | ||
*.egg | ||
MANIFEST | ||
|
||
# PyInstaller | ||
# Usually these files are written by a python script from a template | ||
# before PyInstaller builds the exe, so as to inject date/other infos into it. | ||
*.manifest | ||
*.spec | ||
|
||
# Installer logs | ||
pip-log.txt | ||
pip-delete-this-directory.txt | ||
|
||
# Unit test / coverage reports | ||
htmlcov/ | ||
.tox/ | ||
.coverage | ||
.coverage.* | ||
.cache | ||
nosetests.xml | ||
coverage.xml | ||
*.cover | ||
.hypothesis/ | ||
|
||
# Translations | ||
*.mo | ||
*.pot | ||
|
||
# Django stuff: | ||
*.log | ||
.static_storage/ | ||
.media/ | ||
local_settings.py | ||
|
||
# Flask stuff: | ||
instance/ | ||
.webassets-cache | ||
|
||
# Scrapy stuff: | ||
.scrapy | ||
|
||
# Sphinx documentation | ||
docs/_build/ | ||
|
||
# PyBuilder | ||
target/ | ||
|
||
# Jupyter Notebook | ||
.ipynb_checkpoints | ||
|
||
# pyenv | ||
.python-version | ||
|
||
# celery beat schedule file | ||
celerybeat-schedule | ||
|
||
# SageMath parsed files | ||
*.sage.py | ||
|
||
# Environments | ||
.env | ||
.venv | ||
env/ | ||
venv/ | ||
ENV/ | ||
env.bak/ | ||
venv.bak/ | ||
|
||
# mkdocs documentation | ||
/site | ||
|
||
# mypy | ||
.mypy_cache/ |
34 changes: 34 additions & 0 deletions
34
spaceflights-pandas-viz/{{ cookiecutter.repo_name }}/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
# {{ cookiecutter.project_name }} | ||
|
||
## Overview | ||
|
||
This is your new Kedro project, which was generated using `Kedro {{ cookiecutter.kedro_version }}`. | ||
|
||
Take a look at the [Kedro documentation](https://docs.kedro.org) to get started. | ||
|
||
## Rules and guidelines | ||
|
||
In order to get the best out of the template: | ||
|
||
* Don't remove any lines from the `.gitignore` file we provide | ||
* Make sure your results can be reproduced by following a [data engineering convention](https://docs.kedro.org/en/stable/faq/faq.html#what-is-data-engineering-convention) | ||
* Don't commit data to your repository | ||
* Don't commit any credentials or your local configuration to your repository. Keep all your credentials and local configuration in `conf/local/` | ||
|
||
## How to install dependencies | ||
|
||
Declare any dependencies in `src/requirements.txt` for `pip` installation. | ||
|
||
To install them, run: | ||
|
||
``` | ||
pip install -r src/requirements.txt | ||
``` | ||
|
||
## How to run your Kedro pipeline | ||
|
||
You can run your Kedro project with: | ||
|
||
``` | ||
kedro run | ||
``` |
22 changes: 22 additions & 0 deletions
22
spaceflights-pandas-viz/{{ cookiecutter.repo_name }}/conf/README.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
# What is this for? | ||
|
||
This folder should be used to store configuration files used by Kedro or by separate tools. | ||
|
||
This file can be used to provide users with instructions for how to reproduce local configuration with their own credentials. You can edit the file however you like, but you may wish to retain the information below and add your own section in the section titled **Instructions**. | ||
|
||
## Local configuration | ||
|
||
The `local` folder should be used for configuration that is either user-specific (e.g. IDE configuration) or protected (e.g. security keys). | ||
|
||
> *Note:* Please do not check in any local configuration to version control. | ||
|
||
## Base configuration | ||
|
||
The `base` folder is for shared configuration, such as non-sensitive and project-related configuration that may be shared across team members. | ||
|
||
WARNING: Please do not put access credentials in the base configuration folder. | ||
|
||
## Instructions | ||
|
||
## Find out more | ||
You can find out more about configuration from the [user guide documentation](https://docs.kedro.org/en/stable/configuration/configuration_basics.html). |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.