preload-user-images, "ros preload-images" issues

[johnko]

RancherOS Version: git commit 0353296

Where are you running RancherOS? vagrant, virtualbox

I am noticing an issue with PR #1326.

The new "ros preload-images" for docker (not system-docker) will exit 0 (no error code) even if all the images were not loaded, for example if "no space left on device".

Another issue is that the preload in Go code appears to load images in the background, but creates the doneStamp (eg. user.tar.gz.done) right away.

This is different from the old preload.sh script which only creates the doneStamp when "docker load" is complete with no error (because preload.sh was using set -e).

[SvenDowideit]

ah, joy - I noticed that we were throwing away error values too - 1.0.1-rc3 contains #1788 - but I'm guessing there are other places we're doing this :(

[johnko]

For anyone else with this issue, my current workaround is to revert the preload-user-images service in os-config.tpl.yml back to using the os-preload image (and the old preload.sh script).

[johnko]

Unfortunately, I currently don't know enough Go to implement a fix so that the client.ImageLoad(context.Background(), imageReader, false) from here https://github.com/rancher/os/blob/master/cmd/control/preload.go#L88 runs in a non-background context.

[johnko]

I was wrong in my assumption that

the old preload.sh script only creates the doneStamp when "docker load" is complete with no error (because preload.sh was using set -e).

The old preload.sh script created the doneStamp (eg. user.tar.gz.done) even if there was an error (eg. no space left on device) because of eval ${CAT} || :, and it would still exit 0 (with no error code).

But at least I could see the error message with sudo system-docker logs preload-user-images.

[niusmallnan]

1. Try to reproduce with v1.1.2. I can see images are loaded, but cannot see the done stamp file. So there may be a bug.
2. I will add more logs output for preloading images.
   Checking system-docker preload: /var/log/boot/ros-sysinit.log
   Checking user-docker preload: system-docker logs preload-user-images

[kingsd041]

Tested with RancherOS v1.2.0-rc2
Pre-loading process only reads each new archive once. When starting rancheros again, you see the following log:

# system-docker logs preload-user-images
···
time="2018-02-02T02:29:43Z" level=info msg="Skipping to preload the file: /var/lib/rancher/preload/docker/my-images.tar.xz"
time="2018-02-02T02:29:43Z" level=info msg="Skipping to preload the file: /var/lib/rancher/preload/docker/my-images.tar.xz.done"

# cat /var/log/boot/ros-sysinit.log
...
time="2018-02-02T02:29:38Z" level=info msg="Skipping to preload the file: /var/lib/rancher/preload/system-docker/my-images.tar.xz"
time="2018-02-02T02:29:38Z" level=info msg="Skipping to preload the file: /var/lib/rancher/preload/system-docker/my-images.tar.xz.done"