FTL crashing

[kunago]

In raising this issue, I confirm the following (please check boxes, eg [X]) Failure to fill the template will close your issue:

• I have read and understood the contributors guide.
• The issue I am reporting can be replicated
• The issue I am reporting isn't a duplicate

How familiar are you with the codebase?:

1 - not familiar at all

---

[BUG | ISSUE] Expected Behaviour:
Pihole works flawlessly for weeks/months/years without any interaction.

[BUG | ISSUE] Actual Behaviour:
Well, pihole does not as FTL crashes once in a while and by this I mean once in 2-3 days.

[BUG | ISSUE] Steps to reproduce:

• Install pihole via an official script.
• Run it and let it run for some time.
• FTP craches and the server cannot be restarted unless the machine is rebooted.

Log file output [if available]

[2019-07-01 20:59:02.673 860] Notice: Database size is 1208.14 MB, deleted 9467 rows
[2019-07-01 21:05:45.920 860] Resizing "/FTL-strings" from 39583744 to 39587840
[2019-07-01 21:09:32.028 860] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2019-07-01 21:09:32.028 860] ---------------------------->  FTL crashed!  <----------------------------
[2019-07-01 21:09:32.028 860] !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
[2019-07-01 21:09:32.028 860] Please report a bug at https://github.com/pi-hole/FTL/issues
[2019-07-01 21:09:32.028 860] and include in your report already the following details:
[2019-07-01 21:09:32.028 860] FTL has been running for 388725 seconds
[2019-07-01 21:09:32.028 860] FTL branch: master
[2019-07-01 21:09:32.028 860] FTL version: v4.3.1
[2019-07-01 21:09:32.028 860] FTL commit: b60d63f
[2019-07-01 21:09:32.028 860] FTL date: 2019-05-25 21:37:26 +0200
[2019-07-01 21:09:32.028 860] FTL user: started as pihole, ended as pihole
[2019-07-01 21:09:32.028 860] Received signal: Segmentation fault
[2019-07-01 21:09:32.028 860]      at address: 140631702433792
[2019-07-01 21:09:32.028 860]      with code: SEGV_ACCERR (Invalid permissions for mapped object)
[2019-07-01 21:09:32.029 860] Backtrace:
[2019-07-01 21:09:32.029 860] B[0000]: /usr/bin/pihole-FTL(+0x255e5) [0x5565d0da65e5]
[2019-07-01 21:09:32.029 860] B[0001]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x110e0) [0x7fe7632f90e0]
[2019-07-01 21:09:32.029 860] B[0002]: /usr/bin/pihole-FTL(getTopDomains+0x149) [0x5565d0dad679]
[2019-07-01 21:09:32.029 860] B[0003]: /usr/bin/pihole-FTL(process_request+0x1bd) [0x5565d0da79cd]
[2019-07-01 21:09:32.029 860] B[0004]: /usr/bin/pihole-FTL(+0x25896) [0x5565d0da6896]
[2019-07-01 21:09:32.029 860] B[0005]: /lib/x86_64-linux-gnu/libpthread.so.0(+0x74a4) [0x7fe7632ef4a4]
[2019-07-01 21:09:32.029 860] B[0006]: /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7fe763030d0f]
[2019-07-01 21:09:32.029 860] Thank you for helping us to improve our FTL engine!
[2019-07-01 21:09:32.029 860] FTL terminated!
[2019-07-01 21:16:48.858 215] Using log file /var/log/pihole-FTL.log
[2019-07-01 21:16:48.858 215] ########## FTL started! ##########
[2019-07-01 21:16:48.858 215] FTL branch: master
[2019-07-01 21:16:48.858 215] FTL version: v4.3.1
[2019-07-01 21:16:48.858 215] FTL commit: b60d63f
[2019-07-01 21:16:48.858 215] FTL date: 2019-05-25 21:37:26 +0200
[2019-07-01 21:16:48.858 215] FTL user: pihole
[2019-07-01 21:16:48.858 215] Starting config file parsing (/etc/pihole/pihole-FTL.conf)
[2019-07-01 21:16:48.858 215]    SOCKET_LISTENING: only local
[2019-07-01 21:16:48.858 215]    AAAA_QUERY_ANALYSIS: Show AAAA queries
[2019-07-01 21:16:48.858 215]    MAXDBDAYS: max age for stored queries is 14 days
[2019-07-01 21:16:48.858 215]    RESOLVE_IPV6: Resolve IPv6 addresses
[2019-07-01 21:16:48.858 215]    RESOLVE_IPV4: Resolve IPv4 addresses
[2019-07-01 21:16:48.858 215]    DBINTERVAL: saving to DB file every minute
[2019-07-01 21:16:48.858 215]    DBFILE: Using /etc/pihole/pihole-FTL.db
[2019-07-01 21:16:48.858 215]    MAXLOGAGE: Importing up to 2.0 hours of log data
[2019-07-01 21:16:48.858 215]    PRIVACYLEVEL: Set to 0
[2019-07-01 21:16:48.859 215]    IGNORE_LOCALHOST: Show queries from localhost
[2019-07-01 21:16:48.859 215]    BLOCKINGMODE: Null IPs for blocked domains
[2019-07-01 21:16:48.859 215]    ANALYZE_ONLY_A_AND_AAAA: Disabled. Analyzing all queries
[2019-07-01 21:16:48.859 215]    DBIMPORT: Importing history from database
[2019-07-01 21:16:48.859 215]    PIDFILE: Using /var/run/pihole-FTL.pid
[2019-07-01 21:16:48.859 215]    PORTFILE: Using /var/run/pihole-FTL.port
[2019-07-01 21:16:48.859 215]    SOCKETFILE: Using /var/run/pihole/FTL.sock
[2019-07-01 21:16:48.859 215]    WHITELISTFILE: Using /etc/pihole/whitelist.txt
[2019-07-01 21:16:48.859 215]    BLACKLISTFILE: Using /etc/pihole/black.list
[2019-07-01 21:16:48.859 215]    GRAVITYFILE: Using /etc/pihole/gravity.list
[2019-07-01 21:16:48.859 215]    REGEXLISTFILE: Using /etc/pihole/regex.list
[2019-07-01 21:16:48.859 215]    SETUPVARSFILE: Using /etc/pihole/setupVars.conf
[2019-07-01 21:16:48.859 215]    AUDITLISTFILE: Using /etc/pihole/auditlog.list
[2019-07-01 21:16:48.859 215]    MACVENDORDB: Using /etc/pihole/macvendor.db
[2019-07-01 21:16:48.859 215]    PARSE_ARP_CACHE: Active
[2019-07-01 21:16:48.859 215] Finished config file parsing
[2019-07-01 21:16:48.860 215] Database version is 3
[2019-07-01 21:16:48.860 215] Database successfully initialized
[2019-07-01 21:16:48.861 215] New forward server: 1.1.1.1 (0/512)
[2019-07-01 21:16:48.861 215] New forward server: 1.0.0.1 (1/512)

Device specifics

Hardware Type: VPS, LXC container
OS: Linux 4.15.18-9-pve #1 SMP PVE 4.15.18-30 (Thu, 15 Nov 2018 13:32:46 +0100) x86_64 GNU/Linux

pihole

Pi-hole version is v4.3 (Latest: v4.3.1)
AdminLTE version is v4.3 (Latest: v4.3)
FTL version is v4.3.1 (Latest: v4.3.1)

This template was created based on the work of udemy-dl.

[DL6ER]

The error you're encountering is

Invalid permissions for mapped object

which is likely related to an issue with shared memory.

Can you please run

addr2line -e $(which pihole-FTL) 0x25896 0x269cd 0x2c679 0x255e5

and share the output so we can confirm this?

Do you have any special configuration in this LXC container such as non-persistent storage, maybe a self-cleaning /tmp or shared memory?

[AzureMarker]

Using the binary from https://ftl.pi-hole.net/v4.3.1/pihole-FTL-linux-x86_64, I get these lines:

/root/project/socket.c:325
/root/project/request.c:45
/root/project/api.c:211
/root/project/signals.c:46

The code on those lines isn't especially interesting (link to api.c:211), so we will need some more info from @kunago

[kunago]

My output is exactly the same actually, no additional information.

[AzureMarker]

@kunago so you don't have any information related to this question?

Do you have any special configuration in this LXC container such as non-persistent storage, maybe a self-cleaning /tmp or shared memory?

[kunago]

I am attaching the following screenshots from PROXMOX 5:

As soon as the container was created, I ran the pihole script, so no additional changes were made.

[AzureMarker]

We don't really support Proxmox, but I would suggest looking into how it handles shared memory (/dev/shm).

[DL6ER]

@gituriu1 I see no indication that it was justified that you "conquered" @kunago's issue ticket with your (very likely unrelated) issue. I will separate your posts to keep this conversation clean.

[kunago]

I would be happy to debug but I don't have the knowledge here. If you let me know, I can post any log or output available.

[DL6ER]

@kunago Do you have more information for us so we assist you in fixing your issue?

What is especially interesting is whether your /dev/shm folder is

• (a) present, and
• (b) how it looks like (ls -lh /dev/shm) before and after a crash.

I have no experience with Proxmox so the two screenshots you shared do not tell me much. I see there "Debian". Which version of Debian is this? And which flavor (minimal, normal operating system, other modifications).

[kunago]

At this point I can post the a)

total 88M
-rw------- 1 pihole pihole 324K Jul  8 01:00 FTL-clients
-rw------- 1 pihole pihole  108 Jul  4 18:51 FTL-counters
-rw------- 1 pihole pihole  20M Jul  8 10:17 FTL-domains
-rw------- 1 pihole pihole  20K Jul  4 18:51 FTL-forwarded
-rw------- 1 pihole pihole   48 Jul  8 07:42 FTL-lock
-rw------- 1 pihole pihole  12K Jul  4 18:51 FTL-overTime
-rw------- 1 pihole pihole  43M Jul  8 07:42 FTL-queries
-rw------- 1 pihole pihole   12 Jul  4 18:51 FTL-settings
-rw------- 1 pihole pihole  25M Jul  8 10:12 FTL-strings

I noticed the conversation about 600 permissions. Could that be a problem here as well?

As soon as the crash happens, I will send the b) part.

Debian is a vanilla Debian with no additional modifications, I installed it and ran the pihole script. Nothing more. It is a Debian 7.1 standard version. The system (being LXC container) however identifies itself as

Linux server 4.15.18-9-pve #1 SMP PVE 4.15.18-30 (Thu, 15 Nov 2018 13:32:46 +0100) x86_64 GNU/Linux

[AzureMarker]

The 600 permissions are correct and not a problem.

[DL6ER]

Closing as it seems the crash does not seem to have happened again.