ProxySQL crash and restart in loop

[quentinsimon]

OS version : Debian 6.1.112-1 (2024-09-30)
ProxySQL version : ProxySQL version 2.6.6-8-gc03c5ee, codename Truls
MariaDB version : 10.11.6-MariaDB-0+deb12u1

I'm using a NodeJS app that connect to ProxySQL through mysql2 3.11 package
Anytime I'm executing a statement I've got ProxySQL crashing and restarting
No problem in my staging environment, could it be that it can't handle the production traffic ?

Error logs :

Error: signal 11:
/usr/bin/proxysql(_Z13crash_handleri+0x2a)[0x5563ad3b322a]
/lib/x86_64-linux-gnu/libc.so.6(+0x3c050)[0x7f94d465b050]
/lib/x86_64-linux-gnu/libc.so.6(+0x164032)[0x7f94d4783032]
/usr/bin/proxysql(store_param+0x125)[0x5563ada83325]
/usr/bin/proxysql(ma_stmt_execute_generate_simple_request+0x362)[0x5563ada83af2]
/usr/bin/proxysql(ma_stmt_execute_generate_request+0xc5)[0x5563ada84255]
/usr/bin/proxysql(mysql_stmt_execute+0x78)[0x5563ada85e88]
/usr/bin/proxysql(+0xa52ea8)[0x5563ada93ea8]
/usr/bin/proxysql(my_context_spawn+0x41)[0x5563ada96e31]
---- /usr/bin/proxysql(_Z13crash_handleri+0x2a) [0x5563ad3b322a] : crash_handler(int)
To report a crashing bug visit: https://github.com/sysown/proxysql/issues
For support visit: https://proxysql.com/services/support/
2024-12-18 22:46:34 main.cpp:1464:ProxySQL_daemonize_phase3(): [ERROR] ProxySQL crashed. Restarting!

Thanks for your help cause I can't decrypt the error

[Habeeb556]

@quentinsimon I am looking to gather more details about this issue to help troubleshoot:

• Does this issue occur only during peak hours, or is it consistent regardless of the system load? We need to understand if the problem is related to production load or if it's happening during non-peak hours as well.

• Is the production environment still live and operational, or is the application affected due to ProxySQL's constant restarts? It would be helpful to know whether this is causing any service downtime or if it’s limited to query execution failures.

• Have you monitored memory and CPU usage on the server? High resource utilization could potentially be a factor in ProxySQL's crashes and restarts, so insights on this would be valuable.

[quentinsimon]

Hi @Habeeb556,

Thanks for your reply. Here my answers :

• Does this issue occur only during peak hours, or is it consistent regardless of the system load? We need to understand if the problem is related to production load or if it's happening during non-peak hours as well.

It is consistent regardless of the system load

• Is the production environment still live and operational, or is the application affected due to ProxySQL's constant restarts? It would be helpful to know whether this is causing any service downtime or if it’s limited to query execution failures.

The production environment stays up but no queries can't reach database so I've got dozens of ECONNREFUSED error on my NodeJS app logs and API requests fall in HTTP 400 or 500

• Have you monitored memory and CPU usage on the server? High resource utilization could potentially be a factor in ProxySQL's crashes and restarts, so insights on this would be valuable.

Here a screenshot of a top command when it crashes
Before crash :

During crash :

ERROR connect ECONNREFUSED X.X.X.X:6033

Here what can I found in my proxysql logfile

Hope it could help

[Habeeb556]

Hmm, strange. Since it doesn’t seem to be a resource issue, I think it is more of a configuration issue rather than a bug, but I’m not able to fully understand it yet. Could you provide more details to help with further investigation?

Just to clarify, does ProxySQL always restart, regardless of load, even when the Node.js production environment isn’t connected to it? Or does it only restart when there is a connection?

In another way, when you do a fresh installation of ProxySQL does it always restart? If not, when does this happen? I’d like to determine whether the issue is related to the application or a misconfiguration. I need you try to reproduce it with more attempts.