[QUESTION/HELP] Serverlb keeps restarting after moving from 4.4.8 to 5.0.0

[Generic68]

Question / Where do you need Help?

I am trying to migrate from k3d version 4.4.8 to 5.0.0 (eventual intent to move to something more recent). I've migrated from k3d.io/v1alpha2 to k3d.io/v1alpha3 at the same time.

When I try creating a cluster the serverlb gets stuck in a restart loop that appears to be related to an invalid nginx.conf (no issues with 4.4.8). I noticed there's breaking changes moving to this version but I can't see anywhere that it applies to my config.

Attached snippet of docker logs

[2024-01-17T08:29:18+0000] creating initial nginx config (try 3/3) 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: INFO Backend set to file 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: INFO Starting confd 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: INFO Backend source(s) set to /etc/confd/values.yaml 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Loading template resources from confdir /etc/confd 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Found template: /etc/confd/conf.d/nginx.toml 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Loading template resource from /etc/confd/conf.d/nginx.toml 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Retrieving keys from store 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Key prefix set to / 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Key Map: map[string]string{"/ports/30001.tcp/0":"k3d-ws-server-0", "/ports/30002.tcp/0":"k3d-ws-server-0", "/ports/30003.tcp/0":"k3d-ws-server-0", "/ports/30004.tcp/0":"k3d-ws-server-0", "/ports/30005.tcp/0":"k3d-ws-server-0", "/ports/30006.tcp/0":"k3d-ws-server-0", "/ports/30007.tcp/0":"k3d-ws-server-0", "/ports/30008.tcp/0":"k3d-ws-server-0", "/ports/30009.tcp/0":"k3d-ws-server-0", "/ports/31001.tcp/0":"k3d-ws-server-0", "/ports/31002.tcp/0":"k3d-ws-server-0", "/ports/31003.tcp/0":"k3d-ws-server-0", "/ports/31004.tcp/0":"k3d-ws-server-0", "/ports/31005.tcp/0":"k3d-ws-server-0", "/ports/443.tcp/0":"k3d-ws-server-0", "/ports/5672.tcp/0":"k3d-ws-server-0", "/ports/6000.udp/0":"k3d-ws-server-0", "/ports/6001.udp/0":"k3d-ws-server-0", "/ports/6002.udp/0":"k3d-ws-server-0", "/ports/6003.udp/0":"k3d-ws-server-0", "/ports/6004.udp/0":"k3d-ws-server-0", "/ports/6005.udp/0":"k3d-ws-server-0", "/ports/61613.tcp/0":"k3d-ws-server-0", "/ports/61614.tcp/0":"k3d-ws-server-0", "/ports/61615.tcp/0":"k3d-ws-server-0", "/ports/61616.tcp/0":"k3d-ws-server-0", "/ports/6443.tcp/0":"k3d-ws-server-0", "/ports/80.tcp/0":"k3d-ws-server-0", "/settings/workerConnections":"1024"} 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Got the following map from store: map[/ports/30001.tcp/0:k3d-ws-server-0 /ports/30002.tcp/0:k3d-ws-server-0 /ports/30003.tcp/0:k3d-ws-server-0 /ports/30004.tcp/0:k3d-ws-server-0 /ports/30005.tcp/0:k3d-ws-server-0 /ports/30006.tcp/0:k3d-ws-server-0 /ports/30007.tcp/0:k3d-ws-server-0 /ports/30008.tcp/0:k3d-ws-server-0 /ports/30009.tcp/0:k3d-ws-server-0 /ports/31001.tcp/0:k3d-ws-server-0 /ports/31002.tcp/0:k3d-ws-server-0 /ports/31003.tcp/0:k3d-ws-server-0 /ports/31004.tcp/0:k3d-ws-server-0 /ports/31005.tcp/0:k3d-ws-server-0 /ports/443.tcp/0:k3d-ws-server-0 /ports/5672.tcp/0:k3d-ws-server-0 /ports/6000.udp/0:k3d-ws-server-0 /ports/6001.udp/0:k3d-ws-server-0 /ports/6002.udp/0:k3d-ws-server-0 /ports/6003.udp/0:k3d-ws-server-0 /ports/6004.udp/0:k3d-ws-server-0 /ports/6005.udp/0:k3d-ws-server-0 /ports/61613.tcp/0:k3d-ws-server-0 /ports/61614.tcp/0:k3d-ws-server-0 /ports/61615.tcp/0:k3d-ws-server-0 /ports/61616.tcp/0:k3d-ws-server-0 /ports/6443.tcp/0:k3d-ws-server-0 /ports/80.tcp/0:k3d-ws-server-0 /settings/workerConnections:1024] 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Using source template /etc/confd/templates/nginx.tmpl 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Compiling source template /etc/confd/templates/nginx.tmpl 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Comparing candidate config to /etc/nginx/nginx.conf 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: INFO /etc/nginx/nginx.conf has md5sum f20360c79df7383270f7814968376d0a should be 897055d4fcf0c2fadeb3e429ab0c93b5 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: INFO Target config /etc/nginx/nginx.conf out of sync 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: DEBUG Overwriting target config /etc/nginx/nginx.conf 2024-01-17T08:29:18Z k3d-ws-serverlb confd[25]: INFO Target config /etc/nginx/nginx.conf has been updated ===== Initial nginx configuration ===== 2024/01/17 08:29:20 [emerg] 32#32: host not found in upstream "k3d-ws-server-0:30001" in /etc/nginx/nginx.conf:20 nginx: [emerg] host not found in upstream "k3d-ws-server-0:30001" in /etc/nginx/nginx.conf:20 nginx: configuration file /etc/nginx/nginx.conf test failed

My config file:

apiVersion: k3d.io/v1alpha3
kind: Simple
name: ws
volumes:
  - volume: "<pem file for certs>:/etc/pki/ca-trust/source/anchors/<pem file for certs>"
  - volume: "/etc/resolv.conf:/etc/resolv.conf:ro"
    nodeFilters:
      - all
  - volume: "/home/<me>/.k3d/manifests/coredns.yaml:/var/lib/rancher/k3s/server/manifests/coredns.yaml"
    nodeFilters:
      - all
ports:
  - port: 80:80
    nodeFilters:
      - loadbalancer
  - port: 443:443
    nodeFilters:
      - loadbalancer
  - port: 5672:5672
    nodeFilters:
      - loadbalancer
  - port: 61613-61616:61613-61616
    nodeFilters:
      - loadbalancer
  - port: 30001-30009:30001-30009
    nodeFilters:
      - loadbalancer
  - port: 31001-31005:31001-31005
    nodeFilters:
      - loadbalancer
  - port: 6000-6005:6000-6005/udp
    nodeFilters:
      - loadbalancer
options:
  k3s:
    extraArgs:
      - arg: --no-deploy=traefik,coredns
        nodeFilters:
          - server:*
registries:
<below here is custom proxied registries which all function on 4.4.8>

I feel like something is staring me in the face but I'm not sure what.

Scope of your Question

k3d version v5.0.0
k3s version v1.21.5-k3s1 (default)