Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: systemd-resolved issues #580

Closed
ratcashdev opened this issue Sep 30, 2024 · 14 comments
Closed

Bug: systemd-resolved issues #580

ratcashdev opened this issue Sep 30, 2024 · 14 comments
Labels
discussion A general purpose conversation stale Issue answered by the authors and waiting feedback for a long time waiting-feedback Answered by repo owners and waiting reporter feedback

Comments

@ratcashdev
Copy link

ratcashdev commented Sep 30, 2024

What is Happening

starting dns-proxy from within a compose file, like:

services:
  dns-proxy:
    image: defreitas/dns-proxy-server
    hostname: dns
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /etc
        target: /host/etc
    ports:
      - "5380:5380"

ping dns executed from the host says Name or service not known
The logs, in addition contain the entry: Not possible to restart resolved service: service systemd-resolved restart

Actually, the command service is not present on my machine. Only systemctl restart systemd-resolved works.
However, even if I restart resolved manually, DNS lookup does not work through dns-proxy.

What is Expected

systemd-resolved is restarted and DNS lookup works.

Specs

  • Docker Version: docker version 27.1.2
  • DPS Version:
    • When using docker: the docker image version or run the following if you are using the latest docker run --rm defreitas/dns-proxy-server --version 3.25.10-snapshot
  • OS: Linux mymachine 6.10.6-10-MANJARO 1 SMP PREEMPT_DYNAMIC Sat Aug 24 16:17:40 UTC 2024 x86_64 GNU/Linux
  • Attach DPS Log file
12:03:31.664 [main           ] DEB c.m.d.config.dataprovider.ConfigPathBuilder       l=37   m=build                           status=configPathBuilt, path=/app/conf/config.json
12:03:31.675 [main           ] INF c.m.dnsproxyserver.config.dataprovider.JsonConfigsl=69   m=createDefaultConfigOnPath       status=createdDefaultConfigFile, path=/app/conf/config.json
12:03:31.676 [main           ] DEB c.m.d.config.dataprovider.ConfigDAOJson           l=32   m=find                            configPath=/app/conf/config.json
12:03:31.698 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=32   m=setup                           status=binding-docker-events, connectedToDocker=true
12:03:31.698 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=44   m=setupNetwork                    status=dpsNetwork, active=false
12:03:31.698 [main           ] INF c.m.d.s.docker.application.DpsContainerService    l=116  m=tRunningContainersToDpsNetwork  status=autoConnectDpsNetworkDisabled, dpsNetwork=false, dpsNetworkAutoConnect=false
12:03:31.698 [main           ] INF c.m.d.solver.docker.entrypoint.EventListener      l=32   m=onStart                         status=containerAutoConnectToDpsNetworkDisabled
12:03:31.698 [main           ] INF com.mageddo.dnsserver.UDPServerPool               l=31   m=start                           Starting UDP server, addresses=/0.0.0.0:53
12:03:31.698 [main           ] INF c.mageddo.dnsproxyserver.server.dns.ServerStarter l=30   m=start                           status=startingDnsServer, protocol=UDP_TCP, port=53
12:03:31.701 [virtual-44     ] INF com.mageddo.dnsserver.TCPServer                   l=56   m=start0                          status=tcpServerStarting, port=53
12:03:31.706 [main           ] INF com.mageddo.http.WebServer                        l=134  m=start                           status=startingWebServer, port=5380
12:03:36.733 [Thread-3       ] INF c.m.d.dnsconfigurator.DnsConfigurators            l=113  m=getInstance0                    usingDnsConfigurator=DnsConfiguratorLinux
12:03:36.733 [Thread-3       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=122  m=init                            status=using, configFile=ResolvFile(path=/host/etc/systemd/resolved.conf, type=SYSTEMD_RESOLVED)
12:03:36.738 [Thread-3       ] WAR c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=145  m=tryRestartResolved              status=can't restart resolved service, please run: 'service systemd-resolved restart' to apply DPS as default DNS.
Not possible to restart resolved service: 1 : systemd-resolved: unrecognized service
@mageddo mageddo added discussion A general purpose conversation triage Issue, commonly a report, being reviewed by repo contributor and removed discussion A general purpose conversation labels Sep 30, 2024
@mageddo
Copy link
Owner

mageddo commented Sep 30, 2024

Hey @ratcashdev , DPS won´t be able to configure itself as the default DNS when running inside a docker container when the host is using system-resolved.

Yep, you will do need to restart systemd-resolved using service or systemctl command, see more details about the limitation.

If you are using DPS as your primary DNS solver I would recommend follow theses steps to configure it as a service and don't have to reconfigure systemd-resolved at every restart.

In alternative you can bypass systemd-resolved and use resolv.conf without the need to restart any service by following these instructions:

services:
  dns-proxy:
    image: defreitas/dns-proxy-server
    hostname: dns
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /etc
        target: /host/etc
   environment: 
     - MG_RESOLVCONF=/host/etc/resolv.conf
    ports:
      - "5380:5380"

@ratcashdev
Copy link
Author

HI @mageddo thanks for the quick replay and solution.

DPS won´t be able to configure itself as the default DNS when running inside a docker container when the host is using system-resolved.

can you plz elaborate what is the fundamental issue here?

@mageddo
Copy link
Owner

mageddo commented Sep 30, 2024

Basically systemd-resolved need to be restarted and it's a host program, a host program can't be executed from the docker container as expected by the isolation principle, DPS is running inside a docker container so it can´t restart the host system-resolved at the host.

@ratcashdev
Copy link
Author

ratcashdev commented Oct 1, 2024

@mageddo I have tried your suggested setup above. Unfortunately, it seems it still does not work.
here are the logs:

dns-proxy-1  | 06:41:24.864 [main           ] DEB c.m.d.config.dataprovider.ConfigPathBuilder       l=37   m=build                           status=configPathBuilt, path=/app/conf/config.json
dns-proxy-1  | 06:41:24.874 [main           ] INF c.m.dnsproxyserver.config.dataprovider.JsonConfigsl=69   m=createDefaultConfigOnPath       status=createdDefaultConfigFile, path=/app/conf/config.json
dns-proxy-1  | 06:41:24.874 [main           ] DEB c.m.d.config.dataprovider.ConfigDAOJson           l=32   m=find                            configPath=/app/conf/config.json
dns-proxy-1  | 06:41:24.890 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=32   m=setup                           status=binding-docker-events, connectedToDocker=true
dns-proxy-1  | 06:41:24.890 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=44   m=setupNetwork                    status=dpsNetwork, active=false
dns-proxy-1  | 06:41:24.890 [main           ] INF c.m.d.s.docker.application.DpsContainerService    l=116  m=tRunningContainersToDpsNetwork  status=autoConnectDpsNetworkDisabled, dpsNetwork=false, dpsNetworkAutoConnect=false
dns-proxy-1  | 06:41:24.890 [main           ] INF c.m.d.solver.docker.entrypoint.EventListener      l=32   m=onStart                         status=containerAutoConnectToDpsNetworkDisabled
dns-proxy-1  | 06:41:24.890 [main           ] INF com.mageddo.dnsserver.UDPServerPool               l=31   m=start                           Starting UDP server, addresses=/0.0.0.0:53
dns-proxy-1  | 06:41:24.890 [main           ] INF c.mageddo.dnsproxyserver.server.dns.ServerStarter l=30   m=start                           status=startingDnsServer, protocol=UDP_TCP, port=53
dns-proxy-1  | 06:41:24.892 [virtual-44     ] INF com.mageddo.dnsserver.TCPServer                   l=56   m=start0                          status=tcpServerStarting, port=53
dns-proxy-1  | 06:41:24.897 [main           ] INF com.mageddo.http.WebServer                        l=134  m=start                           status=startingWebServer, port=5380
dns-proxy-1  | 06:41:29.911 [Thread-2       ] INF c.m.d.dnsconfigurator.DnsConfigurators            l=113  m=getInstance0                    usingDnsConfigurator=DnsConfiguratorLinux
dns-proxy-1  | 06:41:29.911 [Thread-2       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=91   m=lambda$findBestConfFile$0       status=noValidConfFile, file=/host/etc/resolv.conf
dns-proxy-1  | 06:41:29.911 [Thread-2       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=122  m=init                            status=using, configFile=null

/etc/resolv.conf still stays the same, containing nameserver 127.0.0.53

@mageddo
Copy link
Owner

mageddo commented Oct 1, 2024

I will do a check, looks like DPS hasn't permission to edit resolv.conf

@ratcashdev
Copy link
Author

yeah, pretty much:

-rw-r--r-- 1 systemd-resolve systemd-resolve 920 sep 30 16:23 /run/systemd/resolve/stub-resolv.conf

@mageddo
Copy link
Owner

mageddo commented Oct 7, 2024

@ratcashdev

Got it.

/host/etc/resolv.conf is a symlink

# ls -lha /host/etc/resolv*
lrwxrwxrwx 1 root root   39 Sep 15  2022 /host/etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

So you need to run like below:

services:
  dns-proxy:
    image: defreitas/dns-proxy-server
    hostname: dns
    volumes:
      - /run/systemd/resolve/:/host/run/
    environment:
      MG_RESOLVCONF: /host/run/stub-resolv.conf
    ports:
      - "5380:5380"
dns-proxy-1  | 13:00:31.189 [Thread-3       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=122  m=init                            status=using, configFile=ResolvFile(path=/host/run/stub-resolv.conf, type=RESOLVCONF)

Improving the logs to make the failing condition explicit at #585 .

@mageddo mageddo added discussion A general purpose conversation waiting-feedback Answered by repo owners and waiting reporter feedback and removed triage Issue, commonly a report, being reviewed by repo contributor labels Oct 7, 2024
Copy link

This issue is stale because it has been waiting-feedback for 15 days with no activity.

@github-actions github-actions bot added the stale Issue answered by the authors and waiting feedback for a long time label Oct 23, 2024
Copy link

This issue was closed because it has been inactive for 7 days since being marked as stale, you can reopen it at any time.

@ratcashdev
Copy link
Author

Tested this today. Unfortunately, it still does not work.

Logs look good, but a simple ping to a hostname registered in the compose file fails.

logs:

dns-proxy-1  | 09:20:45.529 [main           ] DEB c.m.d.config.dataprovider.ConfigPathBuilder       l=37   m=build                           status=configPathBuilt, path=/app/conf/config.json
dns-proxy-1  | 09:20:45.536 [main           ] INF c.m.dnsproxyserver.config.dataprovider.JsonConfigsl=69   m=createDefaultConfigOnPath       status=createdDefaultConfigFile, path=/app/conf/config.json
dns-proxy-1  | 09:20:45.537 [main           ] DEB c.m.d.config.dataprovider.ConfigDAOJson           l=32   m=find                            configPath=/app/conf/config.json
dns-proxy-1  | 09:20:45.553 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=32   m=setup                           status=binding-docker-events, connectedToDocker=true
dns-proxy-1  | 09:20:45.553 [main           ] INF c.m.d.s.d.a.DpsDockerEnvironmentSetupService      l=44   m=setupNetwork                    status=dpsNetwork, active=false
dns-proxy-1  | 09:20:45.553 [main           ] INF c.m.d.s.docker.application.DpsContainerService    l=116  m=tRunningContainersToDpsNetwork  status=autoConnectDpsNetworkDisabled, dpsNetwork=false, dpsNetworkAutoConnect=false
dns-proxy-1  | 09:20:45.553 [main           ] INF c.m.d.solver.docker.entrypoint.EventListener      l=32   m=onStart                         status=containerAutoConnectToDpsNetworkDisabled
dns-proxy-1  | 09:20:45.554 [main           ] INF com.mageddo.dnsserver.UDPServerPool               l=31   m=start                           Starting UDP server, addresses=/0.0.0.0:53
dns-proxy-1  | 09:20:45.554 [main           ] INF c.mageddo.dnsproxyserver.server.dns.ServerStarter l=30   m=start                           status=startingDnsServer, protocol=UDP_TCP, port=53
dns-proxy-1  | 09:20:45.556 [virtual-44     ] INF com.mageddo.dnsserver.TCPServer                   l=56   m=start0                          status=tcpServerStarting, port=53
dns-proxy-1  | 09:20:45.562 [main           ] INF com.mageddo.http.WebServer                        l=134  m=start                           status=startingWebServer, port=5380
dns-proxy-1  | 09:20:50.603 [Thread-3       ] INF c.m.d.dnsconfigurator.DnsConfigurators            l=113  m=getInstance0                    usingDnsConfigurator=DnsConfiguratorLinux
dns-proxy-1  | 09:20:50.604 [Thread-3       ] INF c.m.d.dnsconfigurator.linux.DnsConfiguratorLinux  l=122  m=init                            status=using, configFile=ResolvFile(path=/host/run/stub-resolv.conf, type=RESOLVCONF)

/etc/resolv.conf (linked to /run/systemd/resolve/stub-resolv.conf) now looks like this:

# nameserver 127.0.0.53 # dps-comment
options edns0 trust-ad
search .
nameserver 172.18.0.2 # dps-entry

@mageddo
Copy link
Owner

mageddo commented Oct 31, 2024

@ratcashdev can you confirm that DPS is being used as the default DNS? can you see query entries at DPS logs?

If your /etc/resolv.conf keeps with nameserver 172.18.0.2 # dps-entry then it's supposed to work.

You can ping dps-sample.dev or host.docker to check if DPS is working.

@ratcashdev
Copy link
Author

ratcashdev commented Oct 31, 2024

@mageddo unfortunately none of the pings work and I don't see any name queries in the DPS logs.

mageddo added a commit that referenced this issue Oct 31, 2024
…y ip #580 (#592)

* treating npe at network without gateway ip mapping

* release notes

* [Gradle Release Plugin] - new version commit:  '3.30.5-snapshot'.
@mageddo
Copy link
Owner

mageddo commented Oct 31, 2024

Looks like systemd-resolved is acting and ignoring resolv.conf. If you really want to run DPS inside docker container you can set an static IP for DPS container and configure it on systemd-resolved as following (you will need to use DPS 3.30.5-snapshot):

services:
  dns-proxy:
    image: defreitas/dns-proxy-server:3.30.5-snapshot
    hostname: dns
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
    networks:
      my_network:
        ipv4_address: 171.18.0.10  
    environment:
      - MG_LOG_LEVEL=DEBUG

networks:
  my_network:
    driver: bridge
    ipam:
      config:
        - subnet: 171.18.0.0/16
sudo nano /etc/systemd/resolved.conf

Locate the DNS= line in the [Resolve] section and add the container's static IP:

[Resolve]
DNS=171.18.0.10 
FallbackDNS=

Restart the systemd-resolved service to apply the changes: DPS will be used when available

sudo systemctl restart systemd-resolved

@ratcashdev
Copy link
Author

Thanks for the suggestion. My primary use case was to start dns-proxy from inside docker-compose to allow seamless communication between the host and other containers using host names during integration testing. The requirement to restart resolved makes this a non-practical approach (to the extent of being practically unusable) for my use case.
Thanks for your efforts, either way!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion A general purpose conversation stale Issue answered by the authors and waiting feedback for a long time waiting-feedback Answered by repo owners and waiting reporter feedback
Projects
None yet
Development

No branches or pull requests

2 participants