Add retry for docker pull to deal with occasional errors from ghcr

[peterbroadhurst]

Split docker-compose pull into a separate step with retries, as a simple way to address hyperledger/firefly#244

[nguyer]

Closing and re-opening to trigger the DCO check again

[peterbroadhurst]

Tried draft/un-draft too. Think we might need a comment based confirmation of DCO in this case.

[nguyer]

We already do a pull (if --no-pull is not set) here:

firefly-cli/internal/stacks/stack_manager.go

Lines 487 to 492 in e645f53

	if !options.NoPull {
	s.Log.Info("pulling latest versions")
	if err := docker.RunDockerComposeCommand(workingDir, verbose, verbose, "pull"); err != nil {
	return err
	}
	}

Should we just change that existing call to RunDockerComposeCommandWithRetry?

[peterbroadhurst]

Aha, this is a great point… but what it points at (which I totally missed) is the problem is the e2e test is doing a no-pull 🤦‍♂️

So 👍 to moving the retry on the existing pull, but it means this won’t actually fix the issue.

We might need to punt this to after your manifest work, and have a new behavior for the e2e test where it no long specifies no-pull, but instead uses a different (local) image name.

[peterbroadhurst]

I've made the change on this PR, and am up for working on the follow-on change to generate a special manifest in the e2e test code once the manifest work is ready

[peterbroadhurst]

My earlier assertion that the repo part of the docker image would be enough to stop docker-compose pulling it, seems to be proven false:
docker/compose#3660

This leaves us in a bit of a quandary (as many others on that thread are).
We can:

• Use --ignore-pull-failures and still do a pull to hopefully work around occasional ghcr failures
• Move the retry to the whole docker-compose start

The latter worries me, because it could cause us to miss actual startup error conditions like hyperledger/firefly#254

[peterbroadhurst]

Ok - think this latest approach does what we need

• Added a ff pull <STACK> --retires [X] command for e2e to use
• Added a local: true option to the manifest, to allow a particular image to be skipped pulling
• Automatically add local: true for the case the CLI auto-adds FireFly core today

Then a small tweak to the E2E is to do a pull -retires 3 before a start

[nguyer]

Looks good! Thanks for this!

[nguyer]

I tested this version of the CLI with the corresponding core PR hyperledger/firefly#251 and all E2E tests passed locally! 😃