Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for customization of pipeline templates #2701

Merged
merged 71 commits into from
Sep 12, 2023
Merged
Show file tree
Hide file tree
Changes from 69 commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
d56a396
Remove non-user-facing release notes about pip pin (#2854)
deepyaman Jul 28, 2023
d87bd72
Allow for customization of pipeline templates
jasonmhite Jul 28, 2023
20f7812
Update lookup based on feedback
jasonmhite Jul 28, 2023
c27eb11
Add CLI flag for pipeline creation template -t/--template
jasonmhite Jul 28, 2023
b3bdb46
First basic unit test
jasonmhite Jul 28, 2023
1ddb98d
Add test to check overriding template path on the CLI
jasonmhite Jul 28, 2023
9ec4a53
Linter formatting fixes
jasonmhite Jul 28, 2023
b43dbc2
Release 0.18.12 (#2871)
SajidAlamQB Aug 1, 2023
51a022e
Fix broken link to Prefect website in deploy guide (#2885)
deepyaman Aug 2, 2023
141d502
Clarify the <micropkg_name> argument to kedro micropkg package (#2835)
JonathanDCohen Aug 2, 2023
c373b67
Allow registering of custom resolvers to `OmegaConfigLoader` (#2869)
ankatiyar Aug 2, 2023
d20af76
Document the use of custom resolvers with `OmegaConfigLoader` (#2896)
ankatiyar Aug 7, 2023
307c186
Update kedro pipeline create to use new /conf file structure (#2856)
DimedS Aug 7, 2023
81dd36f
Update kedro catalog create to use new /conf structure (#2884)
DimedS Aug 8, 2023
5d11f7f
Add migration steps for `ConfigLoader` to `OmegaConfigLoader` (#2887)
merelcht Aug 8, 2023
65c7f33
Fix #2498 Adding logging issue 2498 (#2842)
laizaparizotto Aug 8, 2023
480b320
Try only trigger docs build when release notes updated (#2907)
merelcht Aug 8, 2023
51a0745
Add Python 3.11 support to Kedro (#2851)
SajidAlamQB Aug 8, 2023
80516dd
Revise FAQs and README (#2909)
stichbury Aug 8, 2023
b4468de
Update Generator example (#2858)
noklam Aug 8, 2023
c4b0256
update docs to reflect change to /conf file structure (#2913)
DimedS Aug 9, 2023
f753762
Change CONTRIBUTING.md file based on PR #2894 (#2914)
lrcouto Aug 9, 2023
35335cb
Move contribution guidelines from CONTRIBUTING.md to the Wiki (#2894)
lrcouto Aug 10, 2023
d6e8454
Remove redundant pages and direct users to wiki (#2917)
lrcouto Aug 14, 2023
75a78a2
Deprecate abstract "DataSet" in favor of "Dataset" (#2746)
deepyaman Aug 14, 2023
9bc7a46
Reorganise and improve the data catalog documentation (#2888)
stichbury Aug 18, 2023
9e2095b
Add line about Viz to PR template (#2929)
tynandebold Aug 18, 2023
1e55b01
Add architecture graphic back to docs with revisions (#2916)
stichbury Aug 18, 2023
9c7121b
Add kedro catalog resolve command (#2891)
AhdraMeraliQB Aug 18, 2023
ffb46b5
Replace "DataSet" with "Dataset" in Markdown files (#2735)
deepyaman Aug 18, 2023
0111685
Update h1,h2,h3 font sizes in the docs pages (#2938)
tynandebold Aug 18, 2023
53cc179
Automatically trigger `kedro-starters` release on the release of `ked…
ankatiyar Aug 18, 2023
4559ddf
Create issues_metrics.yml (#2814)
noklam Aug 18, 2023
6b12c69
Clean up setuptools and wheel requirements to align with PEP-518 (#2927)
lrcouto Aug 18, 2023
fa97e35
Clean up `kedro pipeline create` outdated docs (#2945)
noklam Aug 21, 2023
786c7d6
Add globals feature for `OmegaConfigLoader` using a globals resolver …
ankatiyar Aug 21, 2023
87b568f
Consolidate two `ruff-pre-commit` entries into one (#2881)
deepyaman Aug 22, 2023
2676c8c
Fix typos across the documentation (#2956)
ankatiyar Aug 22, 2023
53cc5d7
Setup Vale linter as a GHA workflow (#2953)
ankatiyar Aug 22, 2023
476a183
Fix README to show graphics on PyPI (#2961)
stichbury Aug 23, 2023
9426a1b
Add some Vale styles (#2963)
stichbury Aug 23, 2023
230751b
Minor changes to test + release notes + docs
ankatiyar Aug 24, 2023
4c79063
Update anyconfig requirement from ~=0.10.0 to >=0.10,<0.14 (#2876)
dependabot[bot] Aug 24, 2023
55bfbe7
Move default template to static `pyproject.toml`, take 2 (#2853)
astrojuanlu Aug 25, 2023
500a10c
Add deprecation warnings to top-level use of layer in catalog definit…
SajidAlamQB Aug 25, 2023
5a310ae
Update on credentials.md (#2787)
jmnunezd Aug 26, 2023
6e2f457
Cap pluggy 1.3 release (#2981)
DimedS Aug 29, 2023
ed5ed36
Make vale linter only run when PR opened or reopened (#2982)
ankatiyar Aug 29, 2023
53b152a
Update merge-gatekeeper.yml (#2960)
noklam Aug 29, 2023
c9841e2
Configure starters to use OmegaConfigLoader (#2974)
lrcouto Aug 29, 2023
ca5108b
Stop OmegaConfigLoader from reading config from hidden directory like…
noklam Aug 30, 2023
71c1118
Add migration steps for `TemplatedConfigLoader` to `OmegaConfigLoader…
merelcht Aug 30, 2023
9896691
Introduce a sentinel value _NO_VALUE to improve Global resolvers to s…
ankatiyar Aug 30, 2023
8c6c371
Release 0.18.13 (#2988)
ankatiyar Aug 31, 2023
cb9a0cd
Fix docstrings on kedro/extras/datasets (#2995)
lrcouto Sep 1, 2023
ee2f5a2
Minor docs changes on data section to create a PR and test Vale style…
stichbury Sep 1, 2023
be0c34a
PiP/pyproject.toml and Conda/Meta.yaml Sync (#2922)
rxm7706 Sep 1, 2023
7ebbc12
fix typo for build (#3001)
rxm7706 Sep 4, 2023
f9485f0
Add hook example to access `metadata` (#2998)
noklam Sep 5, 2023
2578560
Bump release notes version
jasonmhite Sep 6, 2023
0d77938
Expand docs for customized pipeline templates.
jasonmhite Sep 6, 2023
c12a5de
Update release notes and docs
ankatiyar Sep 7, 2023
c710d38
Update language linter to also run when PR converted ready for review
ankatiyar Sep 7, 2023
8b7cb9a
Fix ci - Use `--resolver=backtracking` with `kedro build-reqs` in e2e…
ankatiyar Sep 6, 2023
9620f80
Update docs
ankatiyar Sep 7, 2023
88971f7
Update style to catch some more US spellings
stichbury Sep 8, 2023
94c769f
Make dataset factory resolve nested dict properly (#2993)
ankatiyar Sep 7, 2023
c6786be
Apply suggestions from code review
jasonmhite Sep 8, 2023
3742bed
Resolve suggestions from code review
jasonmhite Sep 8, 2023
9bd9212
Merge branch 'main' into feature/pipeline-templates
ankatiyar Sep 12, 2023
30e838b
Revert changes ukspelling.yml
ankatiyar Sep 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .circleci/continue_config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -48,9 +48,6 @@ commands:

setup_requirements:
steps:
- run:
name: Install pip setuptools
command: make install-pip-setuptools
- run:
name: Install venv for some pre-commit hooks
command: conda install -y virtualenv
Expand Down
1 change: 1 addition & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,4 @@
- [ ] Updated the documentation to reflect the code changes
- [ ] Added a description of this change in the [`RELEASE.md`](https://github.com/kedro-org/kedro/blob/main/RELEASE.md) file
- [ ] Added tests to cover my changes
- [ ] Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team
6 changes: 6 additions & 0 deletions .github/styles/Kedro/Spellings.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
extends: spelling
message: "Did you really mean '%s'?"
level: warning
ignore:
- Kedro/ignore.txt
- Kedro/ignore-names.txt
11 changes: 11 additions & 0 deletions .github/styles/Kedro/abbreviations.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
extends: substitution
message: "Use '%s' instead of abbreviations like '%s'."
ignorecase: true
level: warning
nonword: true
action:
name: replace
swap:
'\b(?:eg|e\.g\.|eg\.)[\s,]': for example
'\b(?:ie|i\.e\.|ie\.)[\s,]': that is
'\b(?:etc)[\s\n,.]': and more
13 changes: 13 additions & 0 deletions .github/styles/Kedro/gender.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
extends: existence
message: "Use a gender-neutral pronoun instead of '%s'."
level: error
ignorecase: true
tokens:
- he/she
- s/he
- \(s\)he
- \bhe\b
- \bhim\b
- \bhis\b
- \bshe\b
- \bher\b
56 changes: 56 additions & 0 deletions .github/styles/Kedro/headings.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
extends: capitalization
message: "'%s' should use sentence-style capitalization."
level: warning
scope: heading
match: $sentence
exceptions:
- Kedro
- Kedro-Viz
- Airflow
- Amazon
- Amazon Web Services
- AWS Step Functions
- AWS Systems Manager
- Azure
- Azure App Service
- Azure App Service Plan
- Azure Blob Storage
- Azure Event Hub
- CI/CD
- DataCatalog
- Data Catalog
- Docker
- Docker Compose
- Docker Swarm
- Dockerfile
- GitHub
- GitHub Actions
- Google
- Google Analytics
- Google Cloud
- Google Cloud Functions
- GraphQL
- Hook
- Hooks
- IDs
- Jenkins
- JFrog
- JFrog Artifactory
- Jira
- Kafka
- Kubernetes
- Kubernetes Engine
- Kubernetes Pod
- Kubernetes Service
- Lambda
- Linux
- MySQL
- Python
- QuantumBlack
- QuantumBlack Labs
- Red Hat
- Redis
- Slack
- Ubuntu
- Unix
- URLs
111 changes: 111 additions & 0 deletions .github/styles/Kedro/ignore-names.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
Ahdra
Aris
Alves
Alam
Andrew
Andrii
Anton
Antony
Ankita
Batori
Braccialli
Brummitt
Cano
Carvalho
Cvetanka
Czakon
Chan
Comym
Couto
Daiha
Dada
Danov
Datta
DeBold
Deepyaman
Deriabin
Diaz
Dmitrii
Dmitry
Emamjome
Erins
Gabriel
Gundaniya
Guilherme
Hamza
Hoang
Holzer
Huong
Helmhold
Ignacio
Ivaniuk
Ivan
Jakub
Jannic
Jankowski
Jitendra
Jiri
Jo
Joel
Juan
Kadiri
Kaiser
Kaltsas
Kanchwala
Katiyar
Khan
Kirilenko
Kiyohito
Klein
Kunii
Kumar
Lais
Laura
Liam
Lim
Lorena
Machado
Mackay
Malte
Malloy
Marcin
Meisam
Merali
Merel
Milne
Nasef
Nechevska
Nero
Nguyen
Nikolaos
Nikolic
Nok
Okwa
Oza
Paricio
Patel
Peteris
Pilla
Rashida
Ravi
Richard
Schwarzmann
Sorokin
Stichbury
Susanna
Sajid
Stephanie
Takieddine
Theisen
Tynan
Tsaousis
Valtazanos
Vladimir
Waylon
Westenra
Wong
Wrigley
Yolan
Yetunde
Zablocki
Zain
31 changes: 31 additions & 0 deletions .github/styles/Kedro/ignore.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
Kedro
Kedro's
Kubeflow
Databricks
Conda
conda
Cookiecutter
config
fsspec
Kaggle
namespace
namespaces
namespaced
regressors
repo
Repo
dbx
MLflow
csv
yaml
matplotlib
Matplotlib
IPython
APIs
networkx
Plotly
Pylint
SQLAlchemy
Astro
Xebia
pytest
12 changes: 12 additions & 0 deletions .github/styles/Kedro/inclusive.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
extends: substitution
message: "Use '%s' instead of '%s'."
ignorecase: true
level: error
action:
name: replace
swap:
# bad: good
'black ?list': 'disallow list|exclude list'
'master': primary
'slave': secondary
'white ?list': 'allow list|include list'
24 changes: 24 additions & 0 deletions .github/styles/Kedro/links.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
extends: substitution
message: "Avoid vague text in links like '%s' unless you can pair it with more descriptive text."
link: 'https://github.com/kedro-org/kedro/wiki/Kedro-documentation-style-guide'
ignorecase: true
scope: raw
nonword: true
level: error

swap:
# For the word 'here' in Markdown and HTML links
'\[here\]\(.*?\)': 'here'
'<a\s*href\s*=\s*".*?".*?>\s*here\s*</a>': 'here'

# For the word 'this' in Markdown and HTML links
'\[this\]\(.*?\)': 'this'
'<a\s*href\s*=\s*".*?".*?>\s*this\s*</a>': 'this'

# For the word 'page' in Markdown and HTML links
'\[page\]\(.*?\)': 'page'
'<a\s*href\s*=\s*".*?".*?>\s*page\s*</a>': 'page'

# For the phrase 'this page' in Markdown and HTML links
'\[this page\]\(.*?\)': 'this page'
'<a\s*href\s*=\s*".*?".*?>\s*this page\s*</a>': 'this page'
6 changes: 6 additions & 0 deletions .github/styles/Kedro/oxfordcomma.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
extends: existence
message: "Use the Oxford comma in '%s'."
scope: sentence
level: suggestion
tokens:
- '(?:[^,]+,){1,}\s\w+\s(?:and|or)'
11 changes: 11 additions & 0 deletions .github/styles/Kedro/pronouns.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
extends: existence
message: "Avoid first-person singular pronouns such as '%s'."
level: warning
nonword: true
tokens:
- (?<=^|\s)I(?=\s)
- (?<=^|\s)I,(?=\s)
- \bI'm\b
- (?<=\s)[Mm]e\b
- (?<=\s)[Mm]y\b
- (?<=\s)[Mm]ine\b
11 changes: 11 additions & 0 deletions .github/styles/Kedro/sentencelength.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
extends: occurrence
message: "Try to keep your sentence length to 30 words or fewer."
level: suggestion
# Here, we're counting the number of words
# in a sentence.
#
# If there are more than 30, we'll flag it.
scope: sentence
ignorecase: false
max: 30
token: (\w+)
Loading