DO droplet without functioning networking sometimes in some cases

[niusmallnan]

RancherOS Version: (ros os version)
v1.5.0

Where are you running RancherOS? (docker-machine, AWS, GCE, baremetal, etc.)
DigitalOcean

From an user:

We have received and confirmed a bug report related to the new 1.5.0 image. Currently if a user creates a snapshot of a 1.5.0 droplet and creates a new droplet from that image they end up without functioning networking. I am digging into this issue to track down the cause but I wanted to reach out to see if there were any specific changes to network configuration or cloud-init in the 1.5.0 image since we do not see this issue with 1.4.2.

It should be a bug caused by #2524, we want to get the user's network settings before cloud-init is executed. But DO is really special, it is not using DHCP, but the static network settings obtained through metadata.
ROS will use the network configuration left last time if booting from a snapshot, so we have two network configurations on eth0, and we have not been able to remove the old configuration very well.

Two workarounds:

1. Remove /var/lib/rancher/conf/cloud-config.d/network.yml and /var/lib/rancher/conf/metadata before creating snapshot.
2. Reboot the droplet when you cannot get the functioning networking. This is equivalent to resetting the network.

To fix this issue, the best way is to use the network configuration from metadata every time on DO, don't use any cache info.

[kingsd041]

Testing with RancherOS v1.5.1-rc1
Confirmed fix.

Test steps:

1. Upload rancheros v1.5.1-rc1 to DigitalOcean
2. Create a droplet and a snapshot
3. Use this snapshot to create a new droplet
4. Log in to the droplet and ping google.com

Test Results:

[rancher@hailong-ros-151-rc1-1-snapshot-1-s-4vcpu-8gb-nyc3-01 ~]$ ping google.com
PING google.com (172.217.10.238): 56 data bytes
64 bytes from 172.217.10.238: seq=0 ttl=57 time=2.716 ms
64 bytes from 172.217.10.238: seq=1 ttl=57 time=2.238 ms
64 bytes from 172.217.10.238: seq=2 ttl=57 time=2.213 ms