Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow for customization of pipeline templates #2701

Merged
merged 71 commits into from
Sep 12, 2023
Merged

Allow for customization of pipeline templates #2701

merged 71 commits into from
Sep 12, 2023

Commits on Sep 11, 2023

  1. Remove non-user-facing release notes about pip pin (#2854) 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @deepyaman @jasonmhite
    deepyaman authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d56a396 View commit details
    Browse the repository at this point in the history

  2. Allow for customization of pipeline templates 

    Controlled via `settings.PIPELINE_TEMPLATE_PATH`.

Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d87bd72 View commit details
    Browse the repository at this point in the history

  3. Update lookup based on feedback 

    Instead of lookups on `settings.py` just statically look for a `templates`
directory in the project root. If this folder is present it will override
the default global template, otherwise fall back to the previous global
default.

Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    20f7812 View commit details
    Browse the repository at this point in the history

  4. Add CLI flag for pipeline creation template -t/--template 

    Precedence for template_path is:
   command line > project templates/pipeline dir > global default

pipeline create now will also print the template folder used.

Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c27eb11 View commit details
    Browse the repository at this point in the history

  5. First basic unit test 

    TODO is testing the CLI flag too, and maybe testing that the cleanup works
with custom templates.

Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    b3bdb46 View commit details
    Browse the repository at this point in the history

  6. Add test to check overriding template path on the CLI 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    1ddb98d View commit details
    Browse the repository at this point in the history

  7. Linter formatting fixes 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9ec4a53 View commit details
    Browse the repository at this point in the history

  8. Release 0.18.12 (#2871) 

    * bump version and prepare for new release

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update RELEASE.md

Added community contributions

* add DeltaTableDataSet to rst

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* temporarily use the fixed kedro-plugins branch

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* test docsbuild on merels branch

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* use kedro-datasets new release

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

---------

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @SajidAlamQB @stichbury
    @astrojuanlu @jasonmhite
    3 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    b43dbc2 View commit details
    Browse the repository at this point in the history

  9. Fix broken link to Prefect website in deploy guide (#2885) 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @deepyaman @jasonmhite
    deepyaman authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    51a022e View commit details
    Browse the repository at this point in the history

  10. Clarify the <micropkg_name> argument to kedro micropkg package (#2835) 

    Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @JonathanDCohen @astrojuanlu
    @merelcht @jasonmhite
    3 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    141d502 View commit details
    Browse the repository at this point in the history

  11. Allow registering of custom resolvers to OmegaConfigLoader (#2869) 

    * Allow registering of custom resolvers to OCL

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Complete doc string

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add test for overwritten resolvers

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update test for overwritten resolvers

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Remove replace=True by default

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update release notes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update release notes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add debug level log for registering new resolver

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c373b67 View commit details
    Browse the repository at this point in the history

  12. Document the use of custom resolvers with OmegaConfigLoader (#2896) 

    * Update docs for custom resolvers for OCL

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update FAQ with correct link

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Fix typo

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>

* Add note for oc.env

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Suggestions from code review

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @merelcht @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d20af76 View commit details
    Browse the repository at this point in the history

  13. Update kedro pipeline create to use new /conf file structure (#2856) 

    * add parameters_pipelineNane.yml template in the conf folder

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* del yml template from parameters folder

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* modify delete pipeline command to use correct path to yml files

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* modify test_pipeline move parameters foldername to the yml filename

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* collect configs to package not only from parameters folder, but from core conf folder also

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* move parameters foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* move parameters foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* Add changes to RELEASE.md

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* Update RELEASE.md

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>

* Update RELEASE.md, change to release 0.18.13

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* save old structure for 'pipeline delete' command for backward compatibility

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* modify test_pipeline for 'pipeline delete' backward compability

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

---------

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @DimedS @merelcht @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    307c186 View commit details
    Browse the repository at this point in the history

  14. Update kedro catalog create to use new /conf structure (#2884) 

    * modify test_catalog, remove nesting in the catalog folder

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* move catalog foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* Update RELEASE.md

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* Modify docs, fix broken link

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

---------

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @DimedS @jasonmhite
    DimedS authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    81dd36f View commit details
    Browse the repository at this point in the history

  15. Add migration steps for ConfigLoader to OmegaConfigLoader (#2887) 

    * Add migration steps for CL to OCL

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @merelcht @jasonmhite
    merelcht authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    5d11f7f View commit details
    Browse the repository at this point in the history

  16. Fix #2498 Adding logging issue 2498 (#2842) 

    * Fixing typo n on line 114 file /docs/source/nodes_and_pipelines/run_a_pipeline.md from issue #2607

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>

* Removed redundant packages at lines 66, 67 and 68 on file kedro/docs/source/nodes_and_pipelines/run_a_pipeline.md, packages:  from kedro.io import AbstractDataSet, DataCatalog, MemoryDataSet,  from kedro.pipeline import Pipeline and  from kedro.runner.runner import AbstractRunner

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>

* Add logger object for logging when computing micropackage version

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>

* Added new format to the logging

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>

---------

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @laizaparizotto @merelcht @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    65c7f33 View commit details
    Browse the repository at this point in the history

  17. Try only trigger docs build when release notes updated (#2907) 

    Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @merelcht @jasonmhite
    merelcht authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    480b320 View commit details
    Browse the repository at this point in the history

  18. Add Python 3.11 support to Kedro (#2851) 

    * add 3.11

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* limit number of python version tests

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* fix setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* delta-spark version

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* try original setup

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* remove 3.11 from circleci

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin matplotlib to 3.7.2 for python 3.11

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update test_import.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add noks test adjustments

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* replace eq with is for multi_catalog

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* minor fixes

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* remove matplotlib 3.11 pin

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* matplotlib pin for 3.11

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* run original matplotlib pin with ubuntu only

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* run circleci

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin spark < 3.4 as its breaking

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* spark datasets revert earlier changes back to original

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* delta

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* remove all windows tests from GHA

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* skip coverage as not needed

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Checking only WIN tests (will revert)

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update unit-tests.yml

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* undo previous commit

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add msbuild

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add ubuntu tests back in and remove 3.11 tests from circleci add in msbuild for the other tests

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* set hdf5 disable check and lint

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin tables for 3.8

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin 3.8 tables

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* lint

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* remove 3.11 completely from circleci and tidy up

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add 3.11 to pyarrow pin for docs

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* undo previous commit

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* update rtd to python 3.11

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* try 3.10 on rtd

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* fix outdated pylint to ruff

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* changes based on review

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add mock for pyarrow and deltalake for docs

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* undo previous commit

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update .readthedocs.yml

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update conf.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* changes based on review

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* fix comments

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* test running 3.10 parallel

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Add to release notes and update badges on docs

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Add migration tips

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

---------

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @SajidAlamQB @jasonmhite
    SajidAlamQB authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    51a0745 View commit details
    Browse the repository at this point in the history

  19. Revise FAQs and README (#2909) 

    * Revise FAQs and README

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add back the data layers FAQ as I've no idea where else it fits

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* minor changes from review

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @stichbury @jasonmhite
    stichbury authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    80516dd View commit details
    Browse the repository at this point in the history

  20. Update Generator example (#2858) 

    * update release note

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* update to binary mode  when opening file

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Simplify the implementation of Dataset

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* simplify the example - wip

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update the nodes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* update example

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update example

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Apply suggestions from code review

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* add more instruction to guide user to create the custom dataset

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* update the reference from `pipeline.py` -> `nodes.py`

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Collapse the nodes.py function into a expandable section

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Update nodes.md

* Update nodes.md

---------

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @noklam @stichbury
    @merelcht @jasonmhite
    3 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    b4468de View commit details
    Browse the repository at this point in the history

  21. update docs to reflect change to /conf file structure (#2913) 

    * update docs to reflect change to /conf file structure

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* update docs, folder structure diagrams

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

---------

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @DimedS @jasonmhite
    DimedS authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c4b0256 View commit details
    Browse the repository at this point in the history

  22. Change CONTRIBUTING.md file based on PR #2894 (#2914) 

    Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @lrcouto @jasonmhite
    lrcouto authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    f753762 View commit details
    Browse the repository at this point in the history

  23. Move contribution guidelines from CONTRIBUTING.md to the Wiki (#2894) 

    * Add drafts for new contributing.md and wiki page

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Add contribution details to wiki page draft

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Remove contributing-to-Kedro.md draft from the branch

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Fix broken link to Prefect website in deploy guide (#2885)

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Clarify the <micropkg_name> argument to kedro micropkg package (#2835)

Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Allow registering of custom resolvers to `OmegaConfigLoader` (#2869)

* Allow registering of custom resolvers to OCL

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Complete doc string

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add test for overwritten resolvers

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update test for overwritten resolvers

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Remove replace=True by default

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update release notes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update release notes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add debug level log for registering new resolver

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Some adjustments to ordering of text

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Document the use of custom resolvers with `OmegaConfigLoader` (#2896)

* Update docs for custom resolvers for OCL

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update FAQ with correct link

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Fix typo

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>

* Add note for oc.env

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Suggestions from code review

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Update kedro pipeline create to use new /conf file structure (#2856)

* add parameters_pipelineNane.yml template in the conf folder

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* del yml template from parameters folder

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* modify delete pipeline command to use correct path to yml files

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* modify test_pipeline move parameters foldername to the yml filename

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* collect configs to package not only from parameters folder, but from core conf folder also

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* move parameters foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* move parameters foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* Add changes to RELEASE.md

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* Update RELEASE.md

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>

* Update RELEASE.md, change to release 0.18.13

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* save old structure for 'pipeline delete' command for backward compatibility

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* modify test_pipeline for 'pipeline delete' backward compability

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

---------

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Update CONTRIBUTING.md

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Fix broken link

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Update kedro catalog create to use new /conf structure (#2884)

* modify test_catalog, remove nesting in the catalog folder

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* move catalog foldername to the yml filename, remove nesting

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* Update RELEASE.md

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

* Modify docs, fix broken link

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>

---------

Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Add migration steps for `ConfigLoader` to `OmegaConfigLoader` (#2887)

* Add migration steps for CL to OCL

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Fix #2498 Adding logging issue 2498 (#2842)

* Fixing typo n on line 114 file /docs/source/nodes_and_pipelines/run_a_pipeline.md from issue #2607

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>

* Removed redundant packages at lines 66, 67 and 68 on file kedro/docs/source/nodes_and_pipelines/run_a_pipeline.md, packages:  from kedro.io import AbstractDataSet, DataCatalog, MemoryDataSet,  from kedro.pipeline import Pipeline and  from kedro.runner.runner import AbstractRunner

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>

* Add logger object for logging when computing micropackage version

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>

* Added new format to the logging

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>

---------

Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Try only trigger docs build when release notes updated (#2907)

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Add Python 3.11 support to Kedro (#2851)

* add 3.11

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* limit number of python version tests

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* fix setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* delta-spark version

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* try original setup

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* remove 3.11 from circleci

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin matplotlib to 3.7.2 for python 3.11

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update test_import.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add noks test adjustments

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* replace eq with is for multi_catalog

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* minor fixes

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* remove matplotlib 3.11 pin

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* matplotlib pin for 3.11

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* run original matplotlib pin with ubuntu only

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* run circleci

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin spark < 3.4 as its breaking

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* spark datasets revert earlier changes back to original

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* delta

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* remove all windows tests from GHA

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* skip coverage as not needed

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Checking only WIN tests (will revert)

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update unit-tests.yml

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* undo previous commit

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add msbuild

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add ubuntu tests back in and remove 3.11 tests from circleci add in msbuild for the other tests

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* set hdf5 disable check and lint

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin tables for 3.8

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* pin 3.8 tables

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* lint

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* remove 3.11 completely from circleci and tidy up

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add 3.11 to pyarrow pin for docs

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* undo previous commit

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* update rtd to python 3.11

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* try 3.10 on rtd

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* fix outdated pylint to ruff

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* changes based on review

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add mock for pyarrow and deltalake for docs

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* undo previous commit

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update setup.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update .readthedocs.yml

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update conf.py

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* changes based on review

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* fix comments

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* test running 3.10 parallel

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Add to release notes and update badges on docs

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Add migration tips

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

---------

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Revise FAQs and README (#2909)

* Revise FAQs and README

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add back the data layers FAQ as I've no idea where else it fits

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* minor changes from review

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update Generator example (#2858)

* update release note

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* update to binary mode  when opening file

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Simplify the implementation of Dataset

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* simplify the example - wip

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update the nodes

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* update example

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Update example

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Apply suggestions from code review

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* add more instruction to guide user to create the custom dataset

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* update the reference from `pipeline.py` -> `nodes.py`

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Collapse the nodes.py function into a expandable section

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Update nodes.md

* Update nodes.md

---------

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Update CONTRIBUTING.md

---------

Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com>
Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Co-authored-by: Jonathan Cohen <jon@joncohen.dev>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Dmitry Sorokin <40151847+DimedS@users.noreply.github.com>
Co-authored-by: Laíza Milena Scheid Parizotto <41296740+laizaparizotto@users.noreply.github.com>
Co-authored-by: Sajid Alam <90610031+SajidAlamQB@users.noreply.github.com>
Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @lrcouto @deepyaman
    @JonathanDCohen @astrojuanlu @merelcht @ankatiyar @stichbury @DimedS @laizaparizotto @SajidAlamQB @noklam @jasonmhite
    11 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    35335cb View commit details
    Browse the repository at this point in the history

  24. Remove redundant pages and direct users to wiki (#2917) 

    * Remove redundant pages and direct users to wiki

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Lint docs changes

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Few tweaks to index page

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @lrcouto @stichbury @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    d6e8454 View commit details
    Browse the repository at this point in the history

  25. Deprecate abstract "DataSet" in favor of "Dataset" (#2746) 

    * Deprecate abstract "DataSet" in favor of "Dataset"

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Update docs references to abstract "DataSet" class

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Fix `AbstractDataSet` reference to `AbstractDataset`

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Update docs/source/conf.py

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Change remaining `_DEPRECATED_ERROR_CLASSES` to `_DEPRECATED_CLASSES`, update type hints

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Update RELEASE.md

* Reformat kedro/io/__init__.py with Black and isort

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Remove duplicate imports

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Move imports

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

---------

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @deepyaman @jasonmhite
    deepyaman authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    75a78a2 View commit details
    Browse the repository at this point in the history

  26. Reorganise and improve the data catalog documentation (#2888) 

    * First drop of newly organised data catalog docs

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* linter

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Added to-do notes

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Afternoon's work in rewriting/reorganising content

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* More changes

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Further changes

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Another chunk of changes

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Final changes

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Revise ordering of pages

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add new CLI commands to dataset factory docs (#2935)

* Add changes from #2930

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Lint

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Apply suggestions from code review

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Make code snippets collapsable

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

---------

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Bunch of changes from feedback

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* A few more tweaks

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update h1,h2,h3 font sizes

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Add code snippet for using DataCatalog with Kedro config

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Few more tweaks

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/data/data_catalog.md

* Upgrade kedro-datasets for docs

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Improve prose

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com>
Co-authored-by: Tynan DeBold <thdebold@gmail.com>
Co-authored-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @stichbury @AhdraMeraliQB
    @tynandebold @ankatiyar @astrojuanlu @jasonmhite
    5 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9bc7a46 View commit details
    Browse the repository at this point in the history

  27. Add line about Viz to PR template (#2929) 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @tynandebold @jasonmhite
    tynandebold authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9e2095b View commit details
    Browse the repository at this point in the history

  28. Add architecture graphic back to docs with revisions (#2916) 

    * Revise FAQs and README

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Add back the data layers FAQ as I've no idea where else it fits

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* minor changes from review

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Re-add kedro arch diagram, with revised graphic

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* fix broken anchor

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* fix broken anchor

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update docs/source/extend_kedro/architecture_overview.md

Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* Update docs/source/extend_kedro/architecture_overview.md

Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* Update docs/source/extend_kedro/architecture_overview.md

Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* Update docs/source/extend_kedro/architecture_overview.md

Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>

* Changes to architecture page following review

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Change diagram following reivew

* Add links to API docs

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Added in type of users

* Fix linting error

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @stichbury @noklam
    @yetudada @jasonmhite
    3 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    1e55b01 View commit details
    Browse the repository at this point in the history

  29. Add kedro catalog resolve command (#2891) 

    * Implement working draft

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove comments

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove raw_config

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Use config loader to access catalog config

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Use dict comprehension

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove pipeline filtering

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Prevent overwrite or param inclusion

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Trim filepath

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Appease linter

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add test for resolve

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add test to exclude params

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add test for overwrite (not working)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Fix test

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Remove print

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add changes to RELEASE.md

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Trim unreachable code

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Make helper function private

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Refactor

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add changes to the docs

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Refactor 2

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Fix docs build

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add suggestions from code review (docs)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Add suggestions from code review (code)

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Rename catalog_config variable to explicit_datasets

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Apply suggestions from code review

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>

* Fix mocking in test

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Change test fixture name

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

* Lint

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>

---------

Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @AhdraMeraliQB @merelcht @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9c7121b View commit details
    Browse the repository at this point in the history

  30. Replace "DataSet" with "Dataset" in Markdown files (#2735) 

    * LambdaDataSet->LambdaDataset in .md files

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* MemoryDataSet->MemoryDataset in .md files

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* PartitionedDataSet->PartitionedDataset in .md files

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* IncrementalDataSet->IncrementalDataset in .md files

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* CachedDataSet->CachedDataset in .md files

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* DataSetError->DatasetError in .md files

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* DataSetNotFoundError->DatasetNotFoundError in .md files

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Replace "DataSet" with "Dataset" in Markdown files

* Update RELEASE.md

* Fix remaining instance of "*DataSet*"->"*Dataset*"

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* `find . -name '*.md' -print0 | xargs -0 sed -i "" "s/\([^A-Za-z]\)DataSet/\1Dataset/g"`

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Change non-class instances of Dataset to dataset

* Replace any remaining instances of DataSet in docs

* Fix a broken link to docs for `PartitionedDataset`

---------

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @deepyaman @stichbury
    @astrojuanlu @jasonmhite
    3 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    ffb46b5 View commit details
    Browse the repository at this point in the history

  31. Update h1,h2,h3 font sizes in the docs pages (#2938) 

    * Update h1,h2,h3 font sizes

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

* Update h2 font size

Signed-off-by: Tynan DeBold <thdebold@gmail.com>

---------

Signed-off-by: Tynan DeBold <thdebold@gmail.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @tynandebold @stichbury
    @astrojuanlu @jasonmhite
    3 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    0111685 View commit details
    Browse the repository at this point in the history

  32. Automatically trigger kedro-starters release on the release of `ked… 

    …ro` (#2889)

* Add GA for triggering starters release

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update token name

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    53cc179 View commit details
    Browse the repository at this point in the history

  33. Create issues_metrics.yml (#2814) 

    * Create issues_metrics.yml

* Update issues_metrics.yml

Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @noklam @jasonmhite
    noklam authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    4559ddf View commit details
    Browse the repository at this point in the history

  34. Clean up setuptools and wheel requirements to align with PEP-518 (#2927) 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @lrcouto @jasonmhite
    lrcouto authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    6b12c69 View commit details
    Browse the repository at this point in the history

  35. Clean up kedro pipeline create outdated docs (#2945) 

    * remove the README template generate by kedro pipeline create

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* fix docstring for kedro catalog create

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Fix broken link

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* fix broken link

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @noklam @astrojuanlu @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    fa97e35 View commit details
    Browse the repository at this point in the history

  36. Add globals feature for OmegaConfigLoader using a globals resolver (#… 

    …2921)

* Refactor load_and_merge_dir()

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Try adding globals resolver

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Minor change

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add globals resolver

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Revert refactoring

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add test + remove self.globals

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Allow for nested variables in globals

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add documentation

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Typo

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add error message + test

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Apply suggestions from code review

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>

* Split test into multiple tests

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Restrict the globals config_patterns

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Release notes

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update docs/source/configuration/advanced_configuration.md

Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>

* Add helpful error message for keys starting with _

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Enable setting default value for globals resolver

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Typo

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Move test for keys starting with _ to the top

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Fix cross ref link in docs

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @merelcht
    @astrojuanlu @noklam @jasonmhite
    4 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    786c7d6 View commit details
    Browse the repository at this point in the history

  37. Consolidate two ruff-pre-commit entries into one (#2881) 

    * Clean up Ruff configuration in base pyproject.toml

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Consolidate two `ruff-pre-commit` entries into one

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Move Ruff hook to the top, because it's aggressive

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Add `kedro` as `known-first-party` for `isort` run

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Add noqa in docs/source/conf.py

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

* Fix ruff linter precommit config

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Co-authored-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @deepyaman @noklam @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    87b568f View commit details
    Browse the repository at this point in the history

  38. Fix typos across the documentation (#2956) 

    * Fix typos across docs

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Capitalisation stuff

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    2676c8c View commit details
    Browse the repository at this point in the history

  39. Setup Vale linter as a GHA workflow (#2953) 

    * Add vale GHA

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update condition

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update .github/styles/Kedro/ignore.txt

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* add some styles

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Remove styles for now

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @stichbury @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    53cc5d7 View commit details
    Browse the repository at this point in the history

  40. Fix README to show graphics on PyPI (#2961) 

    * Fix for PyPI

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Few more tweaks

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Few more tweaks

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Few more tweaks

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Few more tweaks

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update README.md

* Update README.md

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @stichbury @jasonmhite
    stichbury authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    476a183 View commit details
    Browse the repository at this point in the history

  41. Add some Vale styles (#2963) 

    * add some styles

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Tweaks to styles

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* more tweaks

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Revise some styles

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Revise severity of some styles

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* Update .github/styles/Kedro/words.yml

Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @stichbury @deepyaman @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9426a1b View commit details
    Browse the repository at this point in the history

  42. Minor changes to test + release notes + docs 

    Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    230751b View commit details
    Browse the repository at this point in the history

  43. Update anyconfig requirement from ~=0.10.0 to >=0.10,<0.14 (#2876) 

    Updates the requirements on [anyconfig](https://github.com/ssato/python-anyconfig) to permit the latest version.
- [Changelog](https://github.com/ssato/python-anyconfig/blob/next/NEWS)
- [Commits](ssato/python-anyconfig@RELEASE_0.10.0...RELEASE_0.13.0)

Signed-off-by: Jason Hite <jasonmhite@gmail.com>
---
updated-dependencies:
- dependency-name: anyconfig
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @dependabot @astrojuanlu @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    4c79063 View commit details
    Browse the repository at this point in the history

  44. Move default template to static pyproject.toml, take 2 (#2853) 

    * Remove leftover reference to micropkg setup.py

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Replace outdated extras requirement with kedro-datasets

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Remove outdated mention to environment.yml

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Move flake8 configuration to dedicated file

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Replace setup.py with pyproject.toml in default project template

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Adapt link to moving file in docs

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Revert "Replace outdated extras requirement with kedro-datasets"

This reverts commit dfa8a9e.

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add release notes for pyproject.toml change

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @astrojuanlu @jasonmhite
    astrojuanlu authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    55bfbe7 View commit details
    Browse the repository at this point in the history

  45. Add deprecation warnings to top-level use of layer in catalog definit… 

    …ion (#2964)

* add deprecation

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* lint

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Use FutureWarning

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* changes based on review

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* add link to docs

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>

* Update RELEASE.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @SajidAlamQB @stichbury @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    500a10c View commit details
    Browse the repository at this point in the history

  46. Update on credentials.md (#2787) 

    * Update on credentials.md

Updating example code in credentials.md to make it usable when copy/pasting. Since as of now it will generate a TypeError

Signed-off-by: Jose <jmnunezd123@gmail.com>

* adding a more explicit code example, now we directly show the user that project_path is a pathlib.Path object

Signed-off-by: Jose <jmnunezd123@gmail.com>

* Update docs/source/configuration/credentials.md

Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: Jose <jmnunezd123@gmail.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jmnunezd @stichbury
    @noklam @jasonmhite
    3 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    5a310ae View commit details
    Browse the repository at this point in the history

  47. Cap pluggy 1.3 release (#2981) 

    Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @DimedS @jasonmhite
    DimedS authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    6e2f457 View commit details
    Browse the repository at this point in the history

  48. Make vale linter only run when PR opened or reopened (#2982) 

    Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    ed5ed36 View commit details
    Browse the repository at this point in the history

  49. Update merge-gatekeeper.yml (#2960) 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @noklam @jasonmhite
    noklam authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    53b152a View commit details
    Browse the repository at this point in the history

  50. Configure starters to use OmegaConfigLoader (#2974) 

    * Change starters to use OmegaConfigLoader

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Fix linter error

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Suppress import outside toplevel linting for starters template

Signed-off-by: L. R. Couto <laurarccouto@gmail.com>

* Fix linter error

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Add changes to release notes

Signed-off-by: lrcouto <laurarccouto@gmail.com>

---------

Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: L. R. Couto <laurarccouto@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @lrcouto @jasonmhite
    lrcouto authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c9841e2 View commit details
    Browse the repository at this point in the history

  51. Stop OmegaConfigLoader from reading config from hidden directory like… 

    … `ipynb_checkpoints` (#2977)

* Check plugins implement valid hooks

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Add release note

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Staging work - add custom functions to check hidden folder and files. Tests still failing

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix test - checkpoints should use the same environment

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Revert "Check plugins implement valid hooks"

This reverts commit f10bede.

* Update RELEASE.md

Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>

* fix lint

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @noklam @ankatiyar @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    ca5108b View commit details
    Browse the repository at this point in the history

  52. Add migration steps for TemplatedConfigLoader to OmegaConfigLoader ( 

    #2904)

Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @merelcht @jasonmhite
    merelcht authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    71c1118 View commit details
    Browse the repository at this point in the history

  53. Introduce a sentinel value _NO_VALUE to improve Global resolvers to s… 

    …upport defaults `0` or `None` (#2976)

* Add None support to globals

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add warning when default value is used

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Check keys

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Nok's suggestions

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Create the test to check the non-existing keys

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* add more tests to catch case when global key is not a dict

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Fix the null test

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Introduce sentinel value _NO_VALUE

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* rename test

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Improve error mesasge and raise InterpolationResolutionError when key does not exist and no default

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Fix non exist default test

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Fix test

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Use omegaconf to replace the custom resolving logic

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* uncommented some tests

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* Remove dead code

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update error message

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Co-authored-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @noklam @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9896691 View commit details
    Browse the repository at this point in the history

  54. Release 0.18.13 (#2988) 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    8c6c371 View commit details
    Browse the repository at this point in the history

  55. Fix docstrings on kedro/extras/datasets (#2995) 

    * Fix docstrings on kedro/extras/datasets

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Fix formatting error

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Explicitly define code block

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Fix empty line under code block

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Fix formatting error

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Fix broken link

Signed-off-by: lrcouto <laurarccouto@gmail.com>

* Bump kedro-datasets version

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update links in partitioned and incremental datasets

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update links in partitioned and incremental datasets

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update links in partitioned dataset

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update links in partitioned dataset

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add polars.GenericDataSet to .rst

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: lrcouto <laurarccouto@gmail.com>
Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Co-authored-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @lrcouto @ankatiyar @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    cb9a0cd View commit details
    Browse the repository at this point in the history

  56. Minor docs changes on data section to create a PR and test Vale styles ( 

    #2966)

* Minor changes to create a PR and test Vale styles

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

* fix some vale warnings

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>

---------

Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @stichbury @astrojuanlu @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    ee2f5a2 View commit details
    Browse the repository at this point in the history

  57. PiP/pyproject.toml and Conda/Meta.yaml Sync (#2922) 

    * Resynced

Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com>

* restore lower bound for pluggy

Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com>

---------

Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @rxm7706 @astrojuanlu @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    be0c34a View commit details
    Browse the repository at this point in the history

  58. fix typo for build (#3001) 

    Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @rxm7706 @jasonmhite
    rxm7706 authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    7ebbc12 View commit details
    Browse the repository at this point in the history

  59. Add hook example to access metadata (#2998) 

    * Check plugins implement valid hooks

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* add Metadtahook

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>

* Fix docs according to comments

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

---------

Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @noklam @stichbury @jasonmhite
    2 people authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    f9485f0 View commit details
    Browse the repository at this point in the history

  60. Bump release notes version 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    2578560 View commit details
    Browse the repository at this point in the history

  61. Expand docs for customized pipeline templates. 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    0d77938 View commit details
    Browse the repository at this point in the history

  62. Update release notes and docs 

    Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c12a5de View commit details
    Browse the repository at this point in the history

  63. Update language linter to also run when PR converted ready for review 

    Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c710d38 View commit details
    Browse the repository at this point in the history

  64. Fix ci - Use --resolver=backtracking with kedro build-reqs in e2e… 

    … tests (#3010)

* Bump build version

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Remove telemetry from test default starter

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add package_name back

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Pin build only for 3.7

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Try upgrade pip

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Add constraint

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Update for windows

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Run build-reqs with backtracking resolver

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

* Run build-reqs with backtracking resolver

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>

---------

Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    8b7cb9a View commit details
    Browse the repository at this point in the history

  65. Update docs 

    Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    9620f80 View commit details
    Browse the repository at this point in the history

  66. Update style to catch some more US spellings 

    Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @stichbury @jasonmhite
    stichbury authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    88971f7 View commit details
    Browse the repository at this point in the history

  67. Make dataset factory resolve nested dict properly (#2993) 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @ankatiyar @jasonmhite
    ankatiyar authored and jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    94c769f View commit details
    Browse the repository at this point in the history

  68. Apply suggestions from code review 

    Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite @stichbury
    jasonmhite and stichbury committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    c6786be View commit details
    Browse the repository at this point in the history

  69. Resolve suggestions from code review 

    Signed-off-by: Jason Hite <jasonmhite@gmail.com>
    @jasonmhite
    jasonmhite committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    3742bed View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2023

  1. Merge branch 'main' into feature/pipeline-templates

    @ankatiyar
    ankatiyar authored Sep 12, 2023
    Configuration menu
    Copy the full SHA
    9bd9212 View commit details
    Browse the repository at this point in the history

  2. Revert changes ukspelling.yml 

    Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
    @ankatiyar
    ankatiyar authored Sep 12, 2023
    Configuration menu
    Copy the full SHA
    30e838b View commit details
    Browse the repository at this point in the history