From c083fb09942a46faa172109dd62691370a6b711c Mon Sep 17 00:00:00 2001 From: Nok Date: Wed, 13 Mar 2024 12:00:20 +0000 Subject: [PATCH 1/3] add docs Signed-off-by: Nok --- RELEASE.md | 1 + docs/source/kedro_project_setup/session.md | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/RELEASE.md b/RELEASE.md index e22d9f9a47..c34051551d 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -11,6 +11,7 @@ * Methods `_is_project` and `_find_kedro_project` have been moved to `kedro.utils`. We recommend not using private methods in your code, but if you do, please update your code to use the new location. ## Documentation changes +* Add mentions about correct usage of `configure_project` with `multiprocessing`. ## Community contributions diff --git a/docs/source/kedro_project_setup/session.md b/docs/source/kedro_project_setup/session.md index 9bdfa44123..371b61df04 100644 --- a/docs/source/kedro_project_setup/session.md +++ b/docs/source/kedro_project_setup/session.md @@ -63,3 +63,8 @@ This function uses `configure_project`, and additionally reads metadata from `py ### `configure_project` This function reads `settings.py` and `pipeline_registry.py` and registers the configuration before Kedro's run starts. If you have a packaged Kedro project, you only need to run `configure_project` before executing your pipeline. + +#### ValueError: Pacakge name not found +> ValueError: Pacakge name not found. Make sure you have configured the project using 'bootstrap_project'. This should happen automatically if you are using Kedro comman line interface. + +If you are using `multiprocessing`, you need to be careful about this. Depending on your Operating System, you may have [different default](https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods). If the processes are `spawn`, Python will re-import all the modules in each process and thus you need to run `configure_project` again at the start of the new process. For example, this is how Kedro handle this in `ParallelRunner`(https://github.com/kedro-org/kedro/blob/9e883e6a0ba40e3db4497b234dcb3801258e8396/kedro/runner/parallel_runner.py#L84-L85) From de39edcc4fb8d2fa5c78783bac4ab5f91f2d8bca Mon Sep 17 00:00:00 2001 From: Nok Date: Tue, 9 Jul 2024 11:52:24 +0000 Subject: [PATCH 2/3] fix typo Signed-off-by: Nok --- docs/source/kedro_project_setup/session.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/source/kedro_project_setup/session.md b/docs/source/kedro_project_setup/session.md index 371b61df04..e1a95bf668 100644 --- a/docs/source/kedro_project_setup/session.md +++ b/docs/source/kedro_project_setup/session.md @@ -64,7 +64,7 @@ This function uses `configure_project`, and additionally reads metadata from `py This function reads `settings.py` and `pipeline_registry.py` and registers the configuration before Kedro's run starts. If you have a packaged Kedro project, you only need to run `configure_project` before executing your pipeline. -#### ValueError: Pacakge name not found -> ValueError: Pacakge name not found. Make sure you have configured the project using 'bootstrap_project'. This should happen automatically if you are using Kedro comman line interface. +#### ValueError: Package name not found +> ValueError: Package name not found. Make sure you have configured the project using 'bootstrap_project'. This should happen automatically if you are using Kedro command line interface. If you are using `multiprocessing`, you need to be careful about this. Depending on your Operating System, you may have [different default](https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods). If the processes are `spawn`, Python will re-import all the modules in each process and thus you need to run `configure_project` again at the start of the new process. For example, this is how Kedro handle this in `ParallelRunner`(https://github.com/kedro-org/kedro/blob/9e883e6a0ba40e3db4497b234dcb3801258e8396/kedro/runner/parallel_runner.py#L84-L85) From 815593c1ec561f0f9f917145d51bf00d51299cd2 Mon Sep 17 00:00:00 2001 From: Nok Date: Tue, 9 Jul 2024 11:56:09 +0000 Subject: [PATCH 3/3] whitespace Signed-off-by: Nok --- RELEASE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/RELEASE.md b/RELEASE.md index d4a54931bf..04de728c30 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -54,7 +54,7 @@ Many thanks to the following Kedroids for contributing PRs to this release: * Updated the documentation for deploying a Kedro project with Astronomer Airflow. * Used `kedro-sphinx-theme` for documentation. * Add mentions about correct usage of `configure_project` with `multiprocessing`. -* +* # Release 0.19.4 ## Major features and improvements