Support default values for component arguments

[PhilippeMoussalli]

Problem Statement

Some components have many arguments associated with them. It is not always necessary for the end-user to define all of them and we might want to stick with default values. Currently this is not available and all the components require an argument.

This feature should enable users to set default values for certain components while defining the components. Those values can then be optionally passed during the pipeline definition.

Proposed Approach

We first define the default arguments at the component_spec level:

compnent_spec.yaml

name:component
description: Component that downloads images based on URLs
image: ghcr.io/ml6team/download_images:latest

consumes:
   ...

produces:
  ...

args:
  min_image_size: #mandatory argument
    description: Minimum size of the images.
    type: int
  timeout: #optional argument
    description: Maximum time (in seconds) to wait when trying to download an image 
    default: 3
    type: int

Then, at the step when we add and parse the arguments in component.py
we check if a default argument is provided, if so we set the optional field when adding the argument:

    def _add_and_parse_args(self):
        parser = argparse.ArgumentParser()
        component_arguments = self._get_component_arguments()

        for arg in component_arguments.values():
            # Input manifest is not required for loading component
            if arg.name == "input_manifest_path":
                input_required = False
            else:
                input_required = True
           
            if arg.default is not None:
               default = arg.default
            else: 
               default = None
            parser.add_argument(
                f"--{arg.name}",
                type=kubeflow2python_type[arg.type],
                required=input_required,
                default=default,
                help=arg.description,
            )

        return parser.parse_args()

Additionally, we need to make the necessary changes to the component_spec.py by adding an optional default field to the Argument dataclass as well as modifying the KubeflowComponentSpec as specified here to include default arguments.

Implementation Steps/Tasks

• Change the component_spec.json to accept an optional default field
• Make necessary change to the argument parser in component.py
• Make necessary changes to the component_spec.py

Potential Impact

Testing

Create a component with optional argument and test whether omitting the argument will return the default values. Also test if we can override the default values successfully.

Additional testing will have to be done when running kubeflow but difficult to test with tests (has to be done manually with a mock submitted pipeline).

Documentation

Extend the Args documentation with examples of default pipelines.

Feedback and Suggestions

Dependent features

None

Additional Notes