diff --git a/docs/user-guides/experimental-features.md b/docs/user-guides/experimental-features.md new file mode 100644 index 00000000..d757cb70 --- /dev/null +++ b/docs/user-guides/experimental-features.md @@ -0,0 +1,76 @@ +# Experimental Features + +From time to time, Hera will release a new feature under the "experimental feature" flag while we develop the feature +and ensure stability. Once the feature is stable and we have decided to support it long-term, it will "graduate" into +a fully-supported feature. + +To enable experimental features you must set the feature by name to `True` in the `global_config.experimental_features` +dictionary before using the feature: + +```py +global_config.experimental_features["NAME_OF_FEATURE"] = True +``` + +Note that experimental features are subject to breaking changes in future releases of the same major version. We will +usually announce changes in [the Hera slack channel](https://cloud-native.slack.com/archives/C03NRMD9KPY). + +## Currently supported experimental features: + +### Script IO Models + +Hera provides Pydantic models for you to create subclasses from, which allow you to more easily declare script template +inputs. Any fields that you declare in your subclass of `Input` will become input parameters or artifacts, while +`Output` fields will become output parameters artifacts. The fields that you declare can be `Annotated` as a `Parameter` +or `Artifact`, as any fields with a basic type will become `Parameters`. Turning on the `script_pydantic_io` flag will +automatically enable the `script_annotations` experimental feature. + +To enable Hera input/output models, you must set the `experimental_feature` flag `script_pydantic_io` + +```py +global_config.experimental_features["script_pydantic_io"] = True +``` + +Read the full guide on script pydantic IO in [the script user guide](../user-guides/script-runner-io.md). + +### Decorators for main template types + +Decorators for dags, steps and containers are provided alongside a new script decorator, letting your declare Workflows via Python functions alone. + +To enable the decorators, you must install the optional extras under `experimental` + +```bash +pip install hera[experimental] +``` + +You can then set the `experimental_feature` flag `decorator_syntax` in your code + +```py +global_config.experimental_features["decorator_syntax"] = True +``` + +Read the full guide on decorators in [the decorator user guide](../user-guides/decorators.md). + +## Graduated features + +Once an experimental feature is robust and reliable, we "graduate" them to allow their use without setting the +`experimental_features` flag of the `global_config`. This comes with better support and guarantees for their feature +set. We list graduated features here so you can keep up to date. + +### `RunnerScriptConstructor` (since 5.10) + +The `RunnerScriptConstructor` found in `hera.workflows.script` and seen in the +[callable script example](../examples/workflows/scripts/callable_script.md) is a robust way to run Python functions on +Argo. The image used by the script should be built from the source code package itself and its dependencies, so that the +source code's functions, dependencies, and Hera itself are available to run. The `RunnerScriptConstructor` is also +compatible with Pydantic so supports deserializing inputs to Python objects and serializing outputs to json strings. + +Read [the Script Guide](../user-guides/script-basics.md#runnerscriptconstructor) to learn more! + +### Script Annotations (since 5.19) + +Annotation syntax using `typing.Annotated` is supported for `Parameter`s and `Artifact`s as inputs and outputs for +functions decorated as `scripts`. They use `Annotated` as the type in the function parameters and allow us to simplify +writing scripts with parameters and artifacts that require additional fields such as a `description` or alternative +`name`. + +Read the full guide on script annotations in [the script user guide](../user-guides/script-annotations.md). diff --git a/docs/user-guides/script-runner-io.md b/docs/user-guides/script-runner-io.md index 71477f79..c2549dd8 100644 --- a/docs/user-guides/script-runner-io.md +++ b/docs/user-guides/script-runner-io.md @@ -48,7 +48,6 @@ def my_function(my_input: MyInput) -> MyOutput: another_param_int_output=-1, a_str_param_output="Hello, world!", ) - ``` Using the IO classes requires use of the Hera Runner and the `"script_pydantic_io"` experimental feature flag to be @@ -149,7 +148,6 @@ class MyOutput(Output): @script(constructor="runner") def pydantic_io() -> MyOutput: return MyOutput(exit_code=1, result="Test!", param_int=42, artifact_int=my_input.param_int) - ``` See the full Pydantic IO example [here](../examples/workflows/experimental/script_runner_io.md)! diff --git a/docs/walk-through/experimental-hera-features.md b/docs/walk-through/experimental-hera-features.md index 03a9098f..7984130d 100644 --- a/docs/walk-through/experimental-hera-features.md +++ b/docs/walk-through/experimental-hera-features.md @@ -15,63 +15,4 @@ global_config.experimental_features["NAME_OF_FEATURE"] = True Note that experimental features are subject to breaking changes in future releases of the same major version. We will usually announce changes in [the Hera slack channel](https://cloud-native.slack.com/archives/C03NRMD9KPY). -## Currently supported experimental features: - -### Script IO Models - -Hera provides Pydantic models for you to create subclasses from, which allow you to more easily declare script template -inputs. Any fields that you declare in your subclass of `Input` will become input parameters or artifacts, while -`Output` fields will become output parameters artifacts. The fields that you declare can be `Annotated` as a `Parameter` -or `Artifact`, as any fields with a basic type will become `Parameters`. Turning on the `script_pydantic_io` flag will -automatically enable the `script_annotations` experimental feature. - -To enable Hera input/output models, you must set the `experimental_feature` flag `script_pydantic_io` - -```py -global_config.experimental_features["script_pydantic_io"] = True -``` - -Read the full guide on script pydantic IO in [the script user guide](../user-guides/script-runner-io.md). - -### Decorators for main template types - -Decorators for dags, steps and containers are provided alongside a new script decorator, letting your declare Workflows via Python functions alone. - -To enable the decorators, you must install the optional extras under `experimental` - -```bash -pip install hera[experimental] -``` - -You can then set the `experimental_feature` flag `decorator_syntax` in your code - -```py -global_config.experimental_features["decorator_syntax"] = True -``` - -Read the full guide on decorators in [the decorator user guide](../user-guides/decorators.md). - -## Graduated features - -Once an experimental feature is robust and reliable, we "graduate" them to allow their use without setting the -`experimental_features` flag of the `global_config`. This comes with better support and guarantees for their feature -set. We list graduated features here so you can keep up to date. - -### `RunnerScriptConstructor` (since 5.10) - -The `RunnerScriptConstructor` found in `hera.workflows.script` and seen in the -[callable script example](../examples/workflows/scripts/callable_script.md) is a robust way to run Python functions on -Argo. The image used by the script should be built from the source code package itself and its dependencies, so that the -source code's functions, dependencies, and Hera itself are available to run. The `RunnerScriptConstructor` is also -compatible with Pydantic so supports deserializing inputs to Python objects and serializing outputs to json strings. - -Read [the Script Guide](../user-guides/script-basics.md#runnerscriptconstructor) to learn more! - -### Script Annotations (since 5.19) - -Annotation syntax using `typing.Annotated` is supported for `Parameter`s and `Artifact`s as inputs and outputs for -functions decorated as `scripts`. They use `Annotated` as the type in the function parameters and allow us to simplify -writing scripts with parameters and artifacts that require additional fields such as a `description` or alternative -`name`. - -Read the full guide on script annotations in [the script user guide](../user-guides/script-annotations.md). +Read about current experimental features in [the user guide](../user-guides/experimental-features.md). diff --git a/docs/walk-through/template-features.md b/docs/walk-through/template-features.md index 6fa8a41a..87510c03 100644 --- a/docs/walk-through/template-features.md +++ b/docs/walk-through/template-features.md @@ -1,6 +1,6 @@ # Template Features -This section signposts some `template` features found in Argo and their equivalent in Hera, but are beyond the scope of the Walk Through. +This section signposts some `template` features found in Argo and their equivalent in Hera, which are beyond the scope of the Walk Through. ## Template-Level Lifecycle Hooks diff --git a/docs/walk-through/workflow-features.md b/docs/walk-through/workflow-features.md index 3ddf31cd..70d9ec9e 100644 --- a/docs/walk-through/workflow-features.md +++ b/docs/walk-through/workflow-features.md @@ -1,6 +1,6 @@ # Workflow Features -This section signposts some more Workflow features found in Argo, but are beyond the scope of the Walk Through. +This section signposts some more Workflow features found in Argo, which are beyond the scope of the Walk Through. ## Workflow-Level Lifecycle Hooks diff --git a/mkdocs.yml b/mkdocs.yml index 36c6359d..b8a7d3b1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -32,6 +32,7 @@ nav: - Script Runner IO: user-guides/script-runner-io.md - Decorators: user-guides/decorators.md - Expr Transpiler: user-guides/expr.md + - Experimental Features: user-guides/experimental-features.md - Examples: - About: examples/workflows-examples.md - Idiomatic Hera Examples: