RHCOS 4.11 - builds that detect no changes are blowing up during ostree container encapsulate

[miabbott]

Might be another instance of coreos/coreos-assembler#972 but going to capture it here anyways

Started to see RHCOS build pipelines fail when there were no changes across all arches; makes me suspect something recently landed in cosa or ostree to make it more prevalent

Input state hash: cec9cd090b2585587a661bb006360d644073479fa57ec98923e7e236a65e61b9
No apparent changes since previous commit; use --force-nocache to override
commit: df85cf45b34069d62fdcd083e142f6cc0c970ab57ab7b32ce2c7bdc75c269e2e image: 60a74d92661930e4f9e9d248eee7e59f9d09bb41df259388dffc7d3c6058c2b0
Commit df85cf45b34069d62fdcd083e142f6cc0c970ab57ab7b32ce2c7bdc75c269e2e unchanged; reusing previous build's rpm-ostree metadata
ostree diff commit from: df85cf45b34069d62fdcd083e142f6cc0c970ab57ab7b32ce2c7bdc75c269e2e
ostree diff commit to:   df85cf45b34069d62fdcd083e142f6cc0c970ab57ab7b32ce2c7bdc75c269e2e

New image input checksum: 60a74d92661930e4f9e9d248eee7e59f9d09bb41df259388dffc7d3c6058c2b0
None
New build ID: 411.86.202204130154-0-1
Running:  ostree container encapsulate --copymeta=rpmostree.inputhash --repo=/srv/rhcos/rhcos-rhcos-4.11/tmp/repo --label=coreos-assembler.image-config-checksum=a63a225142788970b32bb6571c3b120811785ac63152b900e588df3db9e0f997 --label=coreos-assembler.image-input-checksum=60a74d92661930e4f9e9d248eee7e59f9d09bb41df259388dffc7d3c6058c2b0 --label=org.opencontainers.image.source=https://github.com/openshift/os --label=org.opencontainers.image.revision=v0.0.2-567-g129a6739d384d783c9c2cdba9f95d1fc744383a0 411.86.202204130154-0-1 oci-archive:rhcos-411.86.202204130154-0-1-ostree.ppc64le.ociarchive.tmp:latest
�[0m�[31merror: �[0mBuilding oci: Writing ostree root to blob: Exporting commit: No such metadata object 50773817e4519629fb061cb3cfe4ddae0a996c12336d087042481fbeab1a380c.dirmeta

[miabbott]

The coreos-assembler short-log FWIW:

$ git shortlog 6b50db1558d0e03f9be94cbb6cce3fac5d3a0c56..3102e153f1686c468032a7d53f779d2e31ffaf95
Colin Walters (6):
      build: Ensure JSON is sorted for reproducibility
      buildextend-metal: Parse image JSON, not YAML
      build: Move `extra-kargs` default into `image-default.yaml`
      build: Checksum image JSON, not YAML
      build: Clean up image JSON generation, use it consistently
      live: Move `squashfs-compression` def into `image-default.yaml`

[cgwalters]

Ohh right, it could actually be my changes, let me see...

[cgwalters]

Ah yes, I think it actually is coreos/coreos-assembler#972 because basically we switched from checksumming image.yaml to checksumming image.json.

So...let's merge some change to openshift/os (or we could merge coreos/coreos-assembler#2811 which will actually fix this forever!)

[cgwalters]

RHCOS pipelines are green again, and the root cause was still coreos/coreos-assembler#972