Kaniko executor image is not pinned which leaks breaking changes

[hammady]

Expected behavior

The default kaniko executor image to be pinned, upgraded and documented with each new skaffold version.

Actual behavior

The executor image is set to a floating tag (latest) leading to breaking changes to leak from the kaniko repo to skaffold.

Information

• Skaffold version: v1.38.0
• Operating system: macOS 12.6
• Installed via: Homebrew
• Contents of skaffold.yaml:

apiVersion: skaffold/v2beta25
kind: Config
build:
  artifacts:
  - image: example
    kaniko:
      cache:
        cacheCopyLayers: true

Steps to reproduce the behavior

1. Create a multi-stage dockerfile (e.g. FROM busybox:latest \n FROM busybox:latest)
2. Run skaffold build

If you run this code when the executor latest version was v1.9.0 (v1.9.1 was released 4 days ago!), or when the latest version is cached and does not reflect the actual latest, the build passes. Try running this code today on a new node (where the latest is v1.9.1 and the node has no cache), and the build fails with the below error:

error building image: kaniko does not support caching copy layers in multistage builds

This is because the latest executor release introduced this breaking change:

feat: disable cache-copy-layers in multistage builds; closes 2065 GoogleContainerTools/kaniko#2227

From the above-mentioned PR:

And this is only users who have chosen not to pin their dependencies, such as their version of executor. I always pinned the executor image I used in my argo workflows since I wanted to make sure that upstream updates don't change anything unless I want them to. But if somebody doesn't pin the executor, they're saying they're okay with upstream breaking their build on upgrades. And this is a case where upstream breaking your build is a good thing, since you were doing something very bad, you just didn't understand it.

This is only one example of the side-effects of defaulting to a floating tag. As with all open source tools, all dependent library versions should be pinned, and upgraded with new versions of the tool, making sure the new versions are tested and documented in the release notes of the tool.
The current workaround to the above specific example is to either disable copy layer caching or pinning the executor image in the yaml, but this is challenging with a big project with an overwhelming number of microservices and skaffold.yaml files.

I believe this is a regression for #591.