Conversion error for parameters when pipelineref.bundle is used

[lbernick]

Webhook error messages show:

{"severity":"error","timestamp":"2022-09-15T14:38:48.420Z","logger":"tekton-pipelines-webhook","caller":"conversion/conversion.go:59","message":"Conversion failed","commit":"698f8a7-dirty","uid":"1c56f711-20c2-454d-90d1-0210280ee8f5","desiredAPIVersion":"tekton.dev/v1","error":"unable to marshal output: json: error calling MarshalJSON for type v1.ParamValue: impossible ParamValues.Type: \"\"","stacktrace":"knative.dev/pkg/webhook/resourcesemantics/conversion.(*reconciler).Convert\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/resourcesemantics/conversion/conversion.go:59\nknative.dev/pkg/webhook.conversionHandler.func1\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/conversion.go:64\nnet/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:2109\nnet/http.(*ServeMux).ServeHTTP\n\tnet/http/server.go:2487\nknative.dev/pkg/webhook.(*Webhook).ServeHTTP\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/webhook.go:262\nknative.dev/pkg/network/handlers.(*Drainer).ServeHTTP\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/network/handlers/drain.go:113\nnet/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2947\nnet/http.(*conn).serve\n\tnet/http/server.go:1991"}

I believe the issue is that param value conversion doesn't handle param type "" (i.e. param type not set-- corresponds to string type)

[JeromeJu]

/assign

[vdemeester]

@lbernick what version does it affect ?

[JeromeJu]

@lbernick what version does it affect ?

Looks like it's the stored version as of v1beta1 according to #5506 (comment).

[lbernick]

I think @vdemeester might be asking about the software version not the api version (to know if we need to backport)-- is that right? I think this change was introduced in 0.39.0 so at least we only have to do one backport 😬

[vdemeester]

@lbernick ah yeah I should have been more clear. And yes, which release, to know if I need to make the PR fixing this in 0.40.0 milestone or not 👼🏼

[lbernick]

Yes let's include it in the 0.40.0 milestone and backport to 0.39.0

[vdemeester]

Well, 0.40.1.. 😅 for some reason I thought it was merged when I chose the commit to be released on.. 🤦🏼

[mike-serchenia]

@vdemeester
Release 0.40.2
still see the same issue for some reason

{"severity":"error","timestamp":"2022-10-07T05:09:46.799Z","logger":"tekton-pipelines-webhook","caller":"conversion/conversion.go:59","message":"Conversion failed","commit":"40a438a","uid":"722ad1f6-0b60-4105-aa49-04966ee6ecaf","desiredAPIVersion":"tekton.dev/v1","error":"unable to marshal output: json: error calling MarshalJSON for type v1.ParamValue: impossible ParamValues.Type: \"\"","stacktrace":"knative.dev/pkg/webhook/resourcesemantics/conversion.(*reconciler).Convert\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/resourcesemantics/conversion/conversion.go:59\nknative.dev/pkg/webhook.conversionHandler.func1\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/conversion.go:64\nnet/http.HandlerFunc.ServeHTTP\n\tnet/http/server.go:2084\nnet/http.(*ServeMux).ServeHTTP\n\tnet/http/server.go:2462\nknative.dev/pkg/webhook.(*Webhook).ServeHTTP\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/webhook/webhook.go:262\nknative.dev/pkg/network/handlers.(*Drainer).ServeHTTP\n\tknative.dev/pkg@v0.0.0-20220818004048-4a03844c0b15/network/handlers/drain.go:113\nnet/http.serverHandler.ServeHTTP\n\tnet/http/server.go:2916\nnet/http.(*conn).serve\n\tnet/http/server.go:1966"}

Installed

https://github.com/tektoncd/pipeline/releases/download/v0.40.2/release.notags.yaml
https://github.com/tektoncd/triggers/releases/download/v0.21.0/release.notags.yaml
https://github.com/tektoncd/triggers/releases/download/v0.21.0/interceptors.notags.yaml

[vdemeester]

hum 🤔 #5506 defenitely got in 0.40.1 though.
@mike-serchenia can you share the task/pipeline used to reproduce this ?

[mike-serchenia]

@vdemeester Please let me know how if I can help to debug it

Client version: 0.26.0
Pipeline version: v0.40.2
Triggers version: v0.21.0
Dashboard version: v0.29.2

PR

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  annotations:
    github.tekton.dev/log-server: https://tekton-log.mydomain.com/logs
    github.tekton.dev/name: '{{ with index .Annotations "tekton.ec.ai/memberOf" }}{{.}}
      / {{ end }}{{ with index .Annotations "tekton.ec.ai/pipeline" }}{{.}}{{ else
      }}{{ index .Labels "tekton.dev/pipeline" }}{{ end }} / {{ index .Labels "tekton.dev/pipelineTask"
      }}'
    github.tekton.dev/owner: mikes
    github.tekton.dev/ref: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
    github.tekton.dev/repo: git-yamls
    github.tekton.dev/url: https://tekton.mydomain.com/#/namespaces/{{ .Namespace }}/pipelineruns/{{
      index .Labels "tekton.dev/pipelineRun" }}?pipelineTask={{ index .Labels "tekton.dev/pipelineTask"
      }}
  creationTimestamp: "2022-10-07T16:46:23Z"
  generateName: kubernetes-sanity-run-
  generation: 1
  labels:
    ec.ai/component: tekton-pipelines
    ec.ai/component_group: tekton
    ec.ai/contract: ec
    ec.ai/data_classification: internal
    ec.ai/environment: src
    ec.ai/managed_by: gitops
    ec.ai/sub_component: pipelinerun
    ec.ai/team: platform
    ec.ai/technical_owner: rluckie
    tekton.dev/pipeline: kubernetes-sanity
  name: kubernetes-sanity-run-bg6h2
  namespace: tekton
  resourceVersion: "59144069"
  uid: a7e67d00-744e-41dc-abf5-1630e662be47
spec:
  params:
  - name: clone-url
    value: https://github.com/mikes/git-yamls.git
  - name: clone-ref
    value: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
  - name: repo-name
    value: git-yamls
  - name: repo-owner-name
    value: mikes
  - name: whitesource-api-key
    value: apiKeyDev
  - name: whitesource-project-version
    value: 1b1b1b1b
  - name: sonar-opts
    value:
    - -Dsonar.pullrequest.key=2701
    - -Dsonar.pullrequest.branch=mike-serchenia-patch-1
    - -Dsonar.pullrequest.base=main
  pipelineRef:
    bundle: us-east4-docker.pkg.dev/mikes-app-source/tekton-catalog/pipeline/kubernetes:0.1
    name: kubernetes-sanity
  podTemplate:
    nodeSelector:
      node_pool: tekton-pipelines
    tolerations:
    - effect: NoSchedule
      key: tekton-pipelines
      operator: Equal
      value: "true"
  serviceAccountName: tekton-build-bot
  timeouts:
    finally: 10m0s
    pipeline: 1h0m0s
    tasks: 30m0s
  workspaces:
  - name: source
    volumeClaimTemplate:
      metadata:
        creationTimestamp: null
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
        storageClassName: rook-ceph-block
      status: {}
  - name: cache
    persistentVolumeClaim:
      claimName: cephfs-pvc
status:
  conditions:
  - lastTransitionTime: "2022-10-07T16:46:53Z"
    message: 'Tasks Completed: 1 (Failed: 0, Cancelled 0), Incomplete: 1, Skipped:
      0'
    reason: Running
    status: Unknown
    type: Succeeded
  pipelineSpec:
    params:
    - name: clone-url
      type: string
    - name: clone-ref
      type: string
    tasks:
    - name: checkout
      params:
      - name: url
        value: https://github.com/mikes/git-yamls.git
      - name: revision
        value: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
      taskRef:
        bundle: gcr.io/tekton-releases/catalog/upstream/git-clone:0.4
        kind: Task
        name: git-clone
      workspaces:
      - name: output
        workspace: source
    - name: lint
      runAfter:
      - checkout
      taskRef:
        bundle: us-east4-docker.pkg.dev/mikes-app-source/tekton-catalog/task/kustomize-lint:0.1
        kind: Task
        name: kustomize-lint
      workspaces:
      - name: source
        workspace: source
    workspaces:
    - name: source
  startTime: "2022-10-07T16:46:24Z"
  taskRuns:
    kubernetes-sanity-run-bg6h2-checkout:
      pipelineTaskName: checkout
      status:
        completionTime: "2022-10-07T16:46:52Z"
        conditions:
        - lastTransitionTime: "2022-10-07T16:46:52Z"
          message: All Steps have completed executing
          reason: Succeeded
          status: "True"
          type: Succeeded
        podName: kubernetes-sanity-run-bg6h2-checkout-pod
        startTime: "2022-10-07T16:46:26Z"
        steps:
        - container: step-clone
          imageID: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init@sha256:db18a9c1607c8cbbcd72f61d0c4d795b9ff528669deacd5f8a1672e4ef198ffd
          name: clone
          terminated:
            containerID: containerd://1ae406984c89765f4c33af71526faee500f0428e2b795a4ac1ee56fe5930e22e
            exitCode: 0
            finishedAt: "2022-10-07T16:46:52Z"
            message: '[{"key":"commit","value":"7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122","type":1},{"key":"url","value":"https://github.com/mikes/git-yamls.git","type":1}]'
            reason: Completed
            startedAt: "2022-10-07T16:46:50Z"
        taskResults:
        - name: commit
          type: string
          value: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
        - name: url
          type: string
          value: https://github.com/mikes/git-yamls.git
        taskSpec:
          description: |-
            These Tasks are Git tasks to work with repositories used by other tasks in your Pipeline.
            The git-clone Task will clone a repo from the provided url into the output Workspace. By default the repo will be cloned into the root of your Workspace. You can clone into a subdirectory by setting this Task's subdirectory param. This Task also supports sparse checkouts. To perform a sparse checkout, pass a list of comma separated directory patterns to this Task's sparseCheckoutDirectories param.
          params:
          - description: Repository URL to clone from.
            name: url
            type: string
          - default: ""
            description: Revision to checkout. (branch, tag, sha, ref, etc...)
            name: revision
            type: string
          - default: ""
            description: Refspec to fetch before checking out revision.
            name: refspec
            type: string
          - default: "true"
            description: Initialize and fetch git submodules.
            name: submodules
            type: string
          - default: "1"
            description: Perform a shallow clone, fetching only the most recent N
              commits.
            name: depth
            type: string
          - default: "true"
            description: Set the `http.sslVerify` global git config. Setting this
              to `false` is not advised unless you are sure that you trust your git
              remote.
            name: sslVerify
            type: string
          - default: ""
            description: Subdirectory inside the `output` Workspace to clone the repo
              into.
            name: subdirectory
            type: string
          - default: ""
            description: Define the directory patterns to match or exclude when performing
              a sparse checkout.
            name: sparseCheckoutDirectories
            type: string
          - default: "true"
            description: Clean out the contents of the destination directory if it
              already exists before cloning.
            name: deleteExisting
            type: string
          - default: ""
            description: HTTP proxy server for non-SSL requests.
            name: httpProxy
            type: string
          - default: ""
            description: HTTPS proxy server for SSL requests.
            name: httpsProxy
            type: string
          - default: ""
            description: Opt out of proxying HTTP/HTTPS requests.
            name: noProxy
            type: string
          - default: "true"
            description: Log the commands that are executed during `git-clone`'s operation.
            name: verbose
            type: string
          - default: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.21.0
            description: The image providing the git-init binary that this Task runs.
            name: gitInitImage
            type: string
          - default: /tekton/home
            description: |
              Absolute path to the user's home directory. Set this explicitly if you are running the image as a non-root user or have overridden
              the gitInitImage param with an image containing custom user configuration.
            name: userHome
            type: string
          results:
          - description: The precise commit SHA that was fetched by this Task.
            name: commit
            type: string
          - description: The precise URL that was fetched by this Task.
            name: url
            type: string
          steps:
          - env:
            - name: HOME
              value: /tekton/home
            - name: PARAM_URL
              value: https://github.com/mikes/git-yamls.git
            - name: PARAM_REVISION
              value: 7b4347c3e6a2caa2c90d1a50b09c3f19c9a72122
            - name: PARAM_REFSPEC
            - name: PARAM_SUBMODULES
              value: "true"
            - name: PARAM_DEPTH
              value: "1"
            - name: PARAM_SSL_VERIFY
              value: "true"
            - name: PARAM_SUBDIRECTORY
            - name: PARAM_DELETE_EXISTING
              value: "true"
            - name: PARAM_HTTP_PROXY
            - name: PARAM_HTTPS_PROXY
            - name: PARAM_NO_PROXY
            - name: PARAM_VERBOSE
              value: "true"
            - name: PARAM_SPARSE_CHECKOUT_DIRECTORIES
            - name: PARAM_USER_HOME
              value: /tekton/home
            - name: WORKSPACE_OUTPUT_PATH
              value: /workspace/output
            - name: WORKSPACE_SSH_DIRECTORY_BOUND
              value: "false"
            - name: WORKSPACE_SSH_DIRECTORY_PATH
            - name: WORKSPACE_BASIC_AUTH_DIRECTORY_BOUND
              value: "false"
            - name: WORKSPACE_BASIC_AUTH_DIRECTORY_PATH
            image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.21.0
            name: clone
            resources: {}
            script: |
              #!/usr/bin/env sh
              set -eu
                  # Script was cut 
          workspaces:
          - description: The git repo will be cloned onto the volume backing this
              Workspace.
            name: output
          - description: |
              A .ssh directory with private key, known_hosts, config, etc. Copied to
              the user's home before git commands are executed. Used to authenticate
              with the git remote when performing the clone. Binding a Secret to this
              Workspace is strongly recommended over other volume types.
            name: ssh-directory
            optional: true
          - description: |
              A Workspace containing a .gitconfig and .git-credentials file. These
              will be copied to the user's home before any git commands are run. Any
              other files in this Workspace are ignored. It is strongly recommended
              to use ssh-directory over basic-auth whenever possible and to bind a
              Secret to this Workspace over other volume types.
            name: basic-auth
            optional: true
    kubernetes-sanity-run-bg6h2-lint:
      pipelineTaskName: lint
      status:
        conditions:
        - lastTransitionTime: "2022-10-07T16:47:11Z"
          message: Not all Steps in the Task have finished executing
          reason: Running
          status: Unknown
          type: Succeeded
        podName: kubernetes-sanity-run-bg6h2-lint-pod
        startTime: "2022-10-07T16:46:53Z"
        steps:
        - container: step-lint
          imageID: k8s.gcr.io/kustomize/kustomize@sha256:01c5882c5cb98dee17e1f90ff756e5e4696cf058e671ead9077815333533911d
          name: lint
          running:
            startedAt: "2022-10-07T16:47:10Z"
        taskSpec:
          params:
          - default: |
              check_ignore () { # this function checks if argument contains substring from "ignore" array
              ignore="scripts static-sites"
              for folder in $ignore
                  # Script was cut 
            name: SCRIPT
            type: string
          - default: 4.1.3
            name: KUSTOMIZE_VERSION
            type: string
          steps:
          - env:
            - name: SCRIPT_CONTENTS
              value: |
                check_ignore () { # this function checks if argument contains substring from "ignore" array
                ignore="scripts static-sites"
                for folder in $ignore
                  # Script was cut 
            image: k8s.gcr.io/kustomize/kustomize:v4.1.3
            name: lint
            resources: {}
            script: |
              #!/usr/bin/env sh
              apk add --no-cache bash
              printf '%s' "${SCRIPT_CONTENTS}" > /tmp/kustomize-lint.sh
              /bin/bash /tmp/kustomize-lint.sh
            workingDir: /workspace/source
          workspaces:
          - name: source

[mike-serchenia]

pipeline

---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: kubernetes-sanity
spec:
  params:
    - name: clone-url
      type: string
    - name: clone-ref
      type: string
  workspaces:
    - name: source
  tasks:
    - name: checkout
      taskRef:
        name: git-clone
        bundle: gcr.io/tekton-releases/catalog/upstream/git-clone:0.4
      params:
        - name: url
          value: $(params.clone-url)
        - name: revision
          value: $(params.clone-ref)
      workspaces:
        - name: output
          workspace: source
    - name: lint
      runAfter:
        - checkout
      taskRef:
        name: kustomize-lint
        bundle: us-east4-docker.pkg.dev/my-project-080808/tekton-catalog/task/kustomize-lint:0.1
      workspaces:
        - name: source
          workspace: source          

[vdemeester]

Ah ok, it's using the bundle, which are not going through the conversion webhook or anything. We should already be running .SetDefaults on what the bundle fetched, so there might be a regression or something overlooked.
cc @JeromeJu

[vdemeester]

@mike-serchenia but indeed, your bug is still there, I'll re-open and update the title.

[vdemeester]

(We also need to validate if this is also happening for spec coming out of resolvers)

[mike-serchenia]

i was checking requirements to use resolvers
https://tekton.dev/docs/pipelines/bundle-resolver/

A cluster with the alpha feature gate enabled could be an issue since GKE Alpha has the following limitations
https://cloud.google.com/kubernetes-engine/docs/concepts/alpha-clusters#limitations

to be more specific:

• Automatically deleted after 30 days

[lbernick]

Hey @mike-serchenia, just two quick notes--

1. remote resolution/resolvers will be in beta in the next pipelines release-- no need for a feature flag in v1beta1
2. alpha feature here refers to a tekton alpha feature, not gke alpha feature (we should probably clarify this!). see these docs for more info

hope this is helpful!

[tirelibirefe]

the same problem still exists in v.0.50.2; Tekton can not be used because of this problem.

[JeromeJu]

the same problem still exists in v.0.50.2; Tekton can not be used because of this problem.

Hi @tirelibirefe , thanks for your report! Would you mind provide more details on the Pipeline/Task that you are running that we might be able take a look into it?

[tirelibirefe]

Hello @JeromeJu
Thank you for your attention.
Unfortunately the customer took Tekton out of scope because of the error; so I won't be able to provide any additional information regarding to the problem.
Thanks & Regards