[bug] Pipeline component input parameters has been altered from "y" to "true"

[erhwenkuo]

• How did you deploy Kubeflow Pipelines (KFP)?

  Use https://github.com/kubeflow/manifests to deploy kubeflow.

  git clone https://github.com/kubeflow/manifests.git
  cd manifests
  git checkout v1.7-branch

  Modify manifests/apps/pipeline/upstream/base/pipeline/kustomization.yaml:

  ...
  images:
    - name: gcr.io/ml-pipeline/api-server
      newTag: 2.0.0-rc.1
    - name: gcr.io/ml-pipeline/persistenceagent
      newTag: 2.0.0-rc.1
    - name: gcr.io/ml-pipeline/scheduledworkflow
      newTag: 2.0.0-rc.1
    - name: gcr.io/ml-pipeline/frontend
      newTag: 2.0.0-rc.1
    - name: gcr.io/ml-pipeline/viewer-crd-controller
      newTag: 2.0.0-rc.1
    - name: gcr.io/ml-pipeline/visualization-server
      newTag: 2.0.0-rc.1
  

  Then, use one command to apply these manifest:

  while ! kustomize build example | awk '!/well-defined/' | kubectl apply -f -; do echo "Retrying to apply resources"; sleep 10; done

• KFP version:

  version: 2.0.0-rc.1

• KFP SDK version:

  version: 2.0.0-rc.1

Steps to reproduce

1. Create two sample pipeline (v2)

   • pipeline - pipeline-add(x,y)

   from kfp import dsl
   from kfp import compiler
   
   @dsl.component
   def square(x: float) -> float:
       return x ** 2
   
   @dsl.component
   def add(x: float, y: float) -> float:
       return x + y
   
   @dsl.component
   def square_root(x: float) -> float:
       return x ** .5
   
   @dsl.pipeline
   def pythagorean(a: float, b: float) -> float:
       a_sq_task = square(x=a)
       b_sq_task = square(x=b)
       sum_task = add(x=a_sq_task.output, y=b_sq_task.output)
       return square_root(x=sum_task.output).output
   
   compiler.Compiler().compile(pythagorean, 'pipeline-add(x,y).yaml')

• pipeline - pipeline-add(x,z)

  from kfp import dsl
  from kfp import compiler
  
  @dsl.component
  def square(x: float) -> float:
      return x ** 2
  
  @dsl.component
  def add(x: float, z: float) -> float:
      return x + z
  
  @dsl.component
  def square_root(x: float) -> float:
      return x ** .5
  
  @dsl.pipeline
  def pythagorean(a: float, b: float) -> float:
      a_sq_task = square(x=a)
      b_sq_task = square(x=b)
      sum_task = add(x=a_sq_task.output, z=b_sq_task.output)
      return square_root(x=sum_task.output).output
  
  compiler.Compiler().compile(pythagorean, 'pipeline-add(x,z).yaml')

  Note: The key difference of these two pipelines is the input parameter's naming for add() component. add(x,y) v.s add(x,z)!!

2. Click "Pipelines" menu, then click "Upload pipeline" for these two testing pipeline IR yaml

   

3. Verify "Pipeline Spec" for add(x,y)

   

   You can find at line#6, the input parameter of "y" has changed to "true" which cause the pipeline fail.

   

   Line#227 also indicating the same situation.

   Then I try to review the IR yaml that generate by compiler "pipeline-add(x,y).yaml", and the format is correct. We can see two input parameters "x" and "y".

   

4. Verify "Pipeline Spec" for add(x,z)
   I am suspect what cause the issue, so the other version pipeline just change the input parameters from "y" to "z". Checking the "Pipeline Spec" and the result is the same as "pipeline-add(x,z).yaml".

   

   Base on this observation, I think the pipeline api backend might be the one which alter the "y" to "true".

Expected result

"Pipeline Spec" representation & structure shoud be identical as IR yaml.

Materials and reference

I put two version of IR yaml as attachment.

pipeline-add.zip

Labels

/area backend

---

Impacted by this bug? Give it a 👍.

[chensun]

The underlying issue is because gopkg.in/yaml.v2 library implements the problematic YAML 1.1 spec, and threats "y" as boolean by design: go-yaml/yaml#214
This has been fixed in gopkg.in/yaml.v3