Maltrail won't boot

[youradds]

Question

"sensor" Seem to have stopped running on all my servers since the last update

Support

python3 /opt/maltrail-scripts/maltrail/sensor.py -c /opt/maltrail-scripts/maltrail.conf
Maltrail (sensor) #v0.65 {https://maltrail.github.io}

[*] starting @ 14:10:48 /2024-02-07/

[i] using configuration file '/opt/maltrail-scripts/maltrail.conf'
[i] using '/var/log/maltrail' for log storage
[i] using '/root/.maltrail/trails.csv' for trail storage (last modification: 'Wed, 07 Feb 2024 13:35:12 GMT')
[i] loading trails...
[i] 801,268 trails loaded
[?] in case of any problems with packet capture on virtual interface 'any', please put all monitoring interfaces to promiscuous mode manually (e.g. 'sudo ifconfig eth0 promisc')
[i] opening interface 'any'
[i] setting capture filter 'udp or icmp or (tcp and (tcp[tcpflags] == tcp-syn or port 80 or port 1080 or port 3128 or port 8000 or port 8080 or port 8118))'
[i] preparing capture buffer...
[i] created 3 more processes (out of total 4)
[^] running...
Exception in thread Thread-2 (_):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/maltrail-scripts/maltrail/sensor.py", line 1169, in _
    (header, packet) = _cap.next()
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats
[x] stopping (Ctrl-C pressed)
[i] cleaning up...

[*] ending @ 14:11:05 /2024-02-07/

python3 /opt/maltrail-scripts/maltrail/server.py -c /opt/maltrail-scripts/maltrail.conf
Maltrail (server) #v0.65 {https://maltrail.github.io}

[*] starting @ 14:12:46 /2024-02-07/

[i] using configuration file '/opt/maltrail-scripts/maltrail.conf'
[i] starting HTTP server at http://172.233.60.35:8338/
[^] running...

It used to work fine - so I'm not sure if an update has broken something?

FWIW, I have tried using the standard maltrail.conf file as well, but this also has the issue (I was wondering if maybe something in my custom .conf file was the issue, but it seems not)

[MikhailKasimov]

@stamparm Please, look here

[MikhailKasimov]

Hello!

Have recongnized all updates Imade from yesterday -- everything is as usual, no anomalies.

Could you, please, reproduce the problem with DEBUG settings true:

and attach here /error.log file from /var/log/maltrail folder. Thnx!

[youradds]

Thanks. I don't get anything in /var/log/maltrail/error.log . I deleted the file and run:

python3 /opt/maltrail-scripts/maltrail/sensor.py -c /opt/maltrail-scripts/maltrail/maltrail.conf
Maltrail (sensor) #v0.66 {https://maltrail.github.io}

[*] starting @ 09:27:57 /2024-02-08/

[i] using configuration file '/opt/maltrail-scripts/maltrail/maltrail.conf'
[i] using '/var/log/maltrail' for log storage
[i] using '/root/.maltrail/trails.csv' for trail storage (last modification: 'Wed, 07 Feb 2024 14:19:29 GMT')
[i] loading trails...
[i] 800,447 trails loaded
[?] in case of any problems with packet capture on virtual interface 'any', please put all monitoring interfaces to promiscuous mode manually (e.g. 'sudo ifconfig eth0 promisc')
[i] opening interface 'any'
[i] setting capture filter 'udp or icmp or (tcp and (tcp[tcpflags] == tcp-syn or port 80 or port 1080 or port 3128 or port 8000 or port 8080 or port 8118))'
[^] running...
Exception in thread Thread-2 (_):
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/maltrail-scripts/maltrail/sensor.py", line 1169, in _
    (header, packet) = _cap.next()
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

The error.log is created, but nothing in it. I set:

# Show debug messages (in console output)
SHOW_DEBUG true

maybe I missed something?

If its any use, I'm on Ubuntu 20.04 and also 22.04 (different servers, all with the same issue)

Thanks

[stamparm]

@youradds are you running pcapy-ng or pcapy? Maltrail requires pcapy-ng (as stated here) - especially if running on Python3

furthermore, here author states that for preventing that error, directive #define PY_SSIZE_T_CLEAN has to be declared in python-C binded code, which is already done in pcapy-ng, as you can see here: https://github.com/stamparm/pcapy-ng/blob/master/pcapy.cc#L10

[youradds]

Ahhh that was it! I was indeed using pcapy (I've been using that since day 1, so not sure when that changed? it has been running fine)

Anyway, doing:

pip3 uninstall pcapy
pip3 install pcapy-ng

has done the trick .

Thanks

[MikhailKasimov]

I was indeed using pcapy <-- just interesting where pcapy came from, because /requirements.txt was changed for pcapy-ng 2 years ago: 2aa2da5

[youradds]

Oh wow... well it seems to have been working at least 3 months ago :) I only just noticed it wasn't as I as setting up a new server and couldn't get it to boot (so checked my other servers, and the same issue). Oh well, all working now. Thanks for the quick reply :) I'm going to go over all my other server and uninstall pcapy and put on pcapy-ng :)

[MikhailKasimov]

@stamparm Thanks for 422e5a8 to handle potential similar cases!