Task bundle cannot be resolved due to its size

[arewm]

Expected Behavior

The bundle resolver can accurately resolve any bundle that has previously been pushed using tkn bundle push.

Actual Behavior

I have a long task definition that we are using. A recent change for it resulted in the bundle resolver failing to resolve the bundle with an error .

couldn't get version/kind; json parse error: invalid character '\x00' after object key:value pair

In the reproduction below, I focus on the tkn bundle CLI as it was easier for me to reproduce and report. This issue is not limited to the CLI as we also saw it with the bundle resolver in a deployment.

Steps to Reproduce the Problem

1. Create a long task definition
2. tkn bundle list the bundle

I have created two bundles that are one character different. You can see that the smaller one succeeds while the larger one fails (both were generated with tkn bundle push).

$ tkn bundle list quay.io/arewm/buildah-remote-oci-ta-bundle:error task buildah-remote-oci-ta
*Warning*: This is an experimental command, it's usage and behavior can change in the next release(s)
Error: failed to decode layer sha256:8f3658ecc315500cf9237492e3fa5ec1baa4e8546bea3ab3d5b69e9c3b0f03e3 to a Tekton object: couldn't get version/kind; json parse error: invalid character '\x00' after object key:value pair
$ tkn bundle list quay.io/arewm/buildah-remote-oci-ta-bundle:success task buildah-remote-oci-ta
*Warning*: This is an experimental command, it's usage and behavior can change in the next release(s)
task.tekton.dev/buildah-remote-oci-ta
$ oras manifest fetch quay.io/arewm/buildah-remote-oci-ta-bundle:error --pretty
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 238,
    "digest": "sha256:57308bf4a2fa648d08c31c8f1dd56a7e384429d4e9825f736b1252a358986a57"
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 10793,
      "digest": "sha256:8f3658ecc315500cf9237492e3fa5ec1baa4e8546bea3ab3d5b69e9c3b0f03e3",
      "annotations": {
        "dev.tekton.image.apiVersion": "v1",
        "dev.tekton.image.kind": "task",
        "dev.tekton.image.name": "buildah-remote-oci-ta"
      }
    }
  ]
}
$ oras manifest fetch quay.io/arewm/buildah-remote-oci-ta-bundle:success --pretty
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 238,
    "digest": "sha256:5d5478cd1f98788d596111054c9aa522dc2f9cdfcd220449047e58261ede5fc6"
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 10787,
      "digest": "sha256:cc8ecf86bbe7b0a84921a23ab28e432b4dfd4e16c7278fb92a66bb41d12498db",
      "annotations": {
        "dev.tekton.image.apiVersion": "v1",
        "dev.tekton.image.kind": "task",
        "dev.tekton.image.name": "buildah-remote-oci-ta"
      }
    }
  ]
}

Extracting the content from the OCI artifact, we can see the difference:

$ oras blob fetch quay.io/arewm/buildah-remote-oci-ta-bundle@sha256:8f3658ecc315500cf9237492e3fa5ec1baa4e8546bea3ab3d5b69e9c3b0f03e3 --output - | tar -C error -zxvpf -
x buildah-remote-oci-ta
$ oras blob fetch quay.io/arewm/buildah-remote-oci-ta-bundle@sha256:cc8ecf86bbe7b0a84921a23ab28e432b4dfd4e16c7278fb92a66bb41d12498db --output - | tar -C success -zxvpf -
x buildah-remote-oci-ta
$ cat error/buildah-remote-oci-ta | jq > error/error.jq
$ cat success/buildah-remote-oci-ta | jq > success/success.jq
$ diff error/error.jq success/success.jq
21c21
<         "description": "Name of secret which contains subscription activation key type- string",
---
>         "description": "Name of secret which contains subscription activation key type- strin",

Additional Info

I reproduced the error with the tkn cli

$ tkn version
Client version: 0.38.1
Chains version: v0.23.0
Pipeline version: v0.65.1
Triggers version: v0.30.0
Operator version: devel

[ipanova]

This bug blocks some further work related to image mode konflux-ci/build-definitions#1530. The commit was reverted because otherwise it triggers this tekton bug.
Is there any chance it can be prioritized or temporarily workarounded in some way?