Nginx workers segfault with nginx 1.9.5, naxsi 0.54 if http2 is enabled

[selivan]

UPD: Problem occurs only if http2 is enabled in listen directive.

Here are core dump files, apport crash report and manulay built packages I used on Ubuntu 14.04: https://yadi.sk/d/6m32n8IFjRqrc

nasxi 0.54

nginx -V
nginx version: nginx/1.9.5
built with OpenSSL 1.0.1f 6 Jan 2014
TLS SNI support enabled
configure arguments: --add-module=/home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src/ --add-module=/home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/nginx-upstream-fair/ --with-cc-opt='-g -O2 -fPIE -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_geoip_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_sub_module --with-http_xslt_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_v2_module

Only module I used except naxsi is nginx-upstream-fair.

Problem appeared after couple of minutes of nginx working with production load, synthetic tests did not reproduce it.
nginx.conf:

http {
        # Naxsi
        include /etc/nginx/naxsi_core.rules;
...
server {
        server_name *********;
        include           /etc/nginx/naxsi.rules;
        set $naxsi_extensive_log 1;
        location /RequestDenied {
            return 418;
            access_log /var/log/nginx/denied.log;
        }
...

naxsi_core.rules taken unchanged from source.

naxsi.rules:

#Enables learning mode
LearningMode;
SecRulesEnabled;
#SecRulesDisabled;
DeniedUrl "/RequestDenied";
## check rules
CheckRule "$SQL >= 8" BLOCK;
CheckRule "$RFI >= 8" BLOCK;
CheckRule "$TRAVERSAL >= 4" BLOCK;
CheckRule "$EVADE >= 4" BLOCK;
CheckRule "$XSS >= 8" BLOCK;

Problem also appeared with naxsi 0.54rc3, which I accidentally built first.

[selivan]

gdb /usr/sbin/nginx -c /var/lib/nginx/cores/core
...
Reading symbols from /usr/sbin/nginx...Reading symbols from /usr/lib/debug/.build-id/98/189920b31b8739442099c12774ed5f5a5e04f7.debug...done.
done.
[New LWP 20339]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `nginx: worker process                           '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007feb1dabaf25 in nx_find_wl_in_hash (mstr=mstr@entry=0x7feb1f4449e8, cf=cf@entry=0x7feb1f049018, zone=zone@entry=HEADERS)
    at /home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src//naxsi_runtime.c:352
352         mstr->data[i] = tolower(mstr->data[i]);

Backtrace:

(gdb) bt
#0  0x00007feb1dabaf25 in nx_find_wl_in_hash (mstr=mstr@entry=0x7feb1f4449e8, cf=cf@entry=0x7feb1f049018, zone=zone@entry=HEADERS)
    at /home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src//naxsi_runtime.c:352
#1  0x00007feb1dabb4ca in ngx_http_dummy_is_rule_whitelisted_n (req=req@entry=0x7feb1f443c90, cf=cf@entry=0x7feb1f049018, 
    r=r@entry=0x7feb1f030178, name=name@entry=0x7feb1f4449e8, zone=zone@entry=HEADERS, target_name=target_name@entry=0)
    at /home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src//naxsi_runtime.c:641
#2  0x00007feb1dabcad5 in ngx_http_apply_rulematch_v_n (r=r@entry=0x7feb1f030178, ctx=ctx@entry=0x7feb1f45dff8, req=req@entry=0x7feb1f443c90, 
    name=name@entry=0x7feb1f4449e8, value=value@entry=0x7feb1f4449f8, zone=zone@entry=HEADERS, nb_match=1, target_name=target_name@entry=0)
    at /home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src//naxsi_runtime.c:1049
#3  0x00007feb1dabd40f in ngx_http_basestr_ruleset_n (pool=<optimized out>, name=name@entry=0x7feb1f4449e8, value=value@entry=0x7feb1f4449f8, 
    rules=0x7feb1f01ce20, req=req@entry=0x7feb1f443c90, ctx=ctx@entry=0x7feb1f45dff8, zone=zone@entry=HEADERS)
    at /home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src//naxsi_runtime.c:1429
#4  0x00007feb1dabd7f3 in ngx_http_dummy_headers_parse (main_cf=main_cf@entry=0x7feb1f008e08, cf=cf@entry=0x7feb1f049018, 
    ctx=ctx@entry=0x7feb1f45dff8, r=r@entry=0x7feb1f443c90)
    at /home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src//naxsi_runtime.c:2130
#5  0x00007feb1dabef8e in ngx_http_dummy_data_parse (ctx=ctx@entry=0x7feb1f45dff8, r=r@entry=0x7feb1f443c90)
    at /home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src//naxsi_runtime.c:2154
#6  0x00007feb1dac1020 in ngx_http_dummy_access_handler (r=0x7feb1f443c90)
    at /home/selivan/work/naxsi/nginx-1.9.5-custom/debian/modules/naxsi/naxsi_src//naxsi_skeleton.c:1247
#7  0x00007feb1da5c743 in ngx_http_core_rewrite_phase (r=0x7feb1f443c90, ph=0x7feb1f0ca1f8) at src/http/ngx_http_core_module.c:894
#8  0x00007feb1da57f05 in ngx_http_core_run_phases (r=r@entry=0x7feb1f443c90) at src/http/ngx_http_core_module.c:840
#9  0x00007feb1da57ff7 in ngx_http_handler (r=r@entry=0x7feb1f443c90) at src/http/ngx_http_core_module.c:823
#10 0x00007feb1da6401e in ngx_http_process_request (r=0x7feb1f443c90) at src/http/ngx_http_request.c:1901
#11 0x00007feb1da95958 in ngx_http_v2_run_request (r=0x7feb1f443c90) at src/http/v2/ngx_http_v2.c:3445
#12 ngx_http_v2_state_header_complete (h2c=0x7feb1f3de700, pos=0x7feb1f12ab33 "", end=0x7feb1f12ab33 "") at src/http/v2/ngx_http_v2.c:1704
#13 0x00007feb1da96ad1 in ngx_http_v2_state_header_block (h2c=0x7feb1f3de700, pos=0x7feb1f12ab33 "", end=0x7feb1f12ab33 "")
    at src/http/v2/ngx_http_v2.c:1266
#14 0x00007feb1da92e6d in ngx_http_v2_read_handler (rev=0x7feb1f0fe790) at src/http/v2/ngx_http_v2.c:357
#15 0x00007feb1da4d2d1 in ngx_epoll_process_events (cycle=0x7feb1f005020, timer=<optimized out>, flags=<optimized out>)
    at src/event/modules/ngx_epoll_module.c:822
#16 0x00007feb1da43a7a in ngx_process_events_and_timers (cycle=cycle@entry=0x7feb1f005020) at src/event/ngx_event.c:242
#17 0x00007feb1da4aa85 in ngx_worker_process_cycle (cycle=cycle@entry=0x7feb1f005020, data=data@entry=0x0) at src/os/unix/ngx_process_cycle.c:753
#18 0x00007feb1da4949a in ngx_spawn_process (cycle=cycle@entry=0x7feb1f005020, proc=proc@entry=0x7feb1da4aa30 <ngx_worker_process_cycle>, 
    data=data@entry=0x0, name=name@entry=0x7feb1dace476 "worker process", respawn=respawn@entry=-3) at src/os/unix/ngx_process.c:198
#19 0x00007feb1da4ad30 in ngx_start_worker_processes (cycle=cycle@entry=0x7feb1f005020, n=1, type=type@entry=-3)
    at src/os/unix/ngx_process_cycle.c:358
#20 0x00007feb1da4bb4f in ngx_master_process_cycle (cycle=0x7feb1f005020) at src/os/unix/ngx_process_cycle.c:130
#21 0x00007feb1da27ac4 in main (argc=<optimized out>, argv=<optimized out>) at src/core/nginx.c:415

[blotus]

Hi,

From the full stacktrace, it seems that you are using HTTP/2, and naxsi has not been tested yet with it (we will start working on it soon).
Can you try to remove the http2 parameter from the listen directive in your configuration and see if the crash still occurs ?

[selivan]

Seems that removing http2 fixed the problem, thank you :)

[bbigras]

Maybe related to the "Bugfix: a segmentation fault might occur in a worker process when using HTTP/2." in the latest nginx version.
http://nginx.org/en/CHANGES

I'll test tomorrow.

[rfnx]

I just tested it and unfortunately it is still not working.

[rfnx]

Same issue with nginx 1.9.7

EDIT : same with 1.9.8 & 1.9.9

[rfnx]

@blotus : Hello, is there any news on this issue ? I really want to use naxsi with http2 :)

[buixor]

Hi @rfnx :)

We didn't start working on the http2 compat.
I hope we will find some time in Jan to work on it, but it is not going to come that soon. http/2 brings a lot of new things that will definitely have serious impacts :)

[rfnx]

@buixor Thanks for the answer !

[selivan]

@buixor: may be you could insert some warning or even error if naxsi is used with HTTP/2? It would be a small and easy change. Now it may be a bit confusing for new users.

[123BLiN]

This is so sad :(

[buixor]

Hi,

Sorry for delay :)
Yes, we are going to add a warning at least. Sorry, I didn't really have time to look at the subject in depth, I just already know it's going to be tricky as http/2 seems to have a lot of tricky things (from the RFC) that might have security implication.

I'll keep you posted ;)

[Promaethius]

Hate to make this thread sound like a broken record, but do you have any news concerning http2 compatibility?

[buixor]

Hi,

Don't worry, you request is totally legit.
No news yet, except I started looking at it (no code written yet), I will try to squeeze some time to work on it, but it's definitely in my plans :)

[TheDeadCode]

+1

[aledbf]

+1

[AnoopAlias]

Can you let me the status of this now that nginx 1.10.0 stable is released . I believe the bug affects this version too?

[selivan]

Yep, unlike 1.9.x, which was mainline version, 1.10 is stable version, like 1.8 was. Is there anything you guys are going to do with it? I would even pay money for it (if I could convince my bosses that it's good investment).

[buixor]

Hello,

This is indeed something we need to work on, but as you might understand, we'd better not take the topic lightly. I plan to work on it as soon as we are done with release 0.55 (we have a few open bugs to close first), but tbh we didn't start working on http2 yet.

It will come, I know some of you would have hoped sooner, please a bit more patience.
I will come back to you as soon as we have something testable ;)

[buixor]

Hi,

just started to do some basic tests with http2, and it seems to be working so far.
If any of you can possibly submit a test that triggers any bug, please do so !

[AnoopAlias]

I am using vers=0.55rc1 with nginx-1.10.0 and having http2 enabled in ssl vhost.

So far I have not seen any crashes and a test was blocked by naxsi just fine

2016/04/28 22:02:00 [error] 1463#1463: *891 NAXSI_FMT: ip=122.174.198.241&server=domain.net&uri=/index.php&learning=0&vers=0.55rc1&total_processed=1&total_blocked=1&block=1&cscore0=$XSS&score0=8&zone0=ARGS&id0=1302&var_name0=a, client: 122.174.198.241, server: domain.net, request: "GET /index.php?a=%3C%3E HTTP/2.0", host: "domain.net"

[Promaethius]

The initial comment clarified: "Problem appeared after couple of minutes of nginx working with production load, synthetic tests did not reproduce it." This would make debugging the issue difficult.

[buixor]

@Promaethius : yes exactly, that's why I'm begging for some test cases.
@AnoopAlias : thanks, my initial tests seem to pass as well

[buixor]

@selivan : any chance you have more indications on what caused the crash ?

[selivan]

@buixor: was long time ago, all that I saved is in this ticket. And I can't test new build on production load now, but I'll see if I can use it for 5-10 minutes on one of backends some days later.

[rfnx]

@buixor Hello, thanks again for your work. I tried and naxsi still crashed. This is very easy to reproduce : for me, it happens everytime I click the "connection" button on my wordpress site, to go to the admin connection page (default to /wp-admin).

Configuration :

• nginx version : 1.10.0;
• naxsi version : 0.55rc1;
• http/2 enabled;
• Linux kernel with grsecurity.

My system log after the crash :

kernel: grsec: From 127.0.0.6: Segmentation fault occurred at 0000000000ad37e4 in /usr/bin/nginx[nginx:11381] uid/euid:33/33 gid/egid:33/33, parent /usr/bin/nginx[nginx:11379] uid/euid:0/0 gid/egid:0/0
kernel: grsec: From 127.0.0.6: bruteforce prevention initiated for the next 30 minutes or until service restarted, stalling each fork 30 seconds.  Please investigate the crash report for /usr/bin/nginx[nginx:11381] uid/euid:33/33 gid/egid:33/33, parent /usr/bin/nginx[nginx:11379] uid/euid:0/0 gid/egid:0/0
systemd-coredump[11419]: Process 11381 (nginx) of user 33 dumped core.

                                                        Stack trace of thread 11381:
                                                        #0  0x00000000004c5964 nx_find_wl_in_hash (nginx)
                                                        #1  0x00000000004c5e85 ngx_http_dummy_is_rule_whitelisted_n (nginx)
                                                        #2  0x00000000004c7599 ngx_http_apply_rulematch_v_n (nginx)
                                                        #3  0x00000000004c7ee2 ngx_http_basestr_ruleset_n (nginx)
                                                        #4  0x00000000004c82c5 ngx_http_dummy_headers_parse (nginx)
                                                        #5  0x00000000004c9980 ngx_http_dummy_data_parse (nginx)
                                                        #6  0x00000000004cbc9e n/a (nginx)
                                                        #7  0x00000000004563fc ngx_http_core_rewrite_phase (nginx)
                                                        #8  0x0000000000451895 ngx_http_core_run_phases (nginx)
                                                        #9  0x000000000045c973 ngx_http_process_request (nginx)
                                                        #10 0x00000000004890a6 n/a (nginx)
                                                        #11 0x0000000000489d76 n/a (nginx)
                                                        #12 0x0000000000489fbe n/a (nginx)
                                                        #13 0x0000000000488855 n/a (nginx)
                                                        #14 0x000000000043e750 ngx_event_process_posted (nginx)
                                                        #15 0x0000000000444921 n/a (nginx)
                                                        #16 0x0000000000443350 ngx_spawn_process (nginx)
                                                        #17 0x0000000000444c84 n/a (nginx)
                                                        #18 0x00000000004455a4 ngx_master_process_cycle (nginx)
                                                        #19 0x0000000000421838 main (nginx)
                                                        #20 0x0000031d74329710 __libc_start_main (libc.so.6)
                                                        #21 0x0000000000421d09 _start (nginx)

                                                        Stack trace of thread 11392:
                                                        #0  0x0000031d761333e8 pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)
                                                        #1  0x00000000006f3095 n/a (nginx)
                                                        #2  0x00000000006efb3a n/a (nginx)
                                                        #3  0x000000000050b86e n/a (nginx)
                                                        #4  0x00000000006f4588 n/a (nginx)
                                                        #5  0x0000031d7612d424 start_thread (libpthread.so.0)
                                                        #6  0x0000031d743f0cbd __clone (libc.so.6)

[buixor]

Thanks :)

Any chance you could provide a dump of the http request?
On 29 Apr 2016 22:53, "rfnx" notifications@github.com wrote:

@buixor https://github.com/buixor Hello, thanks again for your work. I
tried again and I still have a crash with naxsi. This is very easy to
reproduce : for me, it happens everytime I click the "connection" button on
my wordpress site, to go to the admin connection page (default to
/wp-admin).

Configuration :

• nginx version : 1.10.0;
• naxsi version : 0.55rc1;
• http/2 enabled;
• Linux kernel with grsecurity.

My system log after the crash :

kernel: grsec: From 127.0.0.6: Segmentation fault occurred at 0000000000ad37e4 in /usr/bin/nginx[nginx:11381] uid/euid:33/33 gid/egid:33/33, parent /usr/bin/nginx[nginx:11379] uid/euid:0/0 gid/egid:0/0
kernel: grsec: From 127.0.0.6: bruteforce prevention initiated for the next 30 minutes or until service restarted, stalling each fork 30 seconds. Please investigate the crash report for /usr/bin/nginx[nginx:11381] uid/euid:33/33 gid/egid:33/33, parent /usr/bin/nginx[nginx:11379] uid/euid:0/0 gid/egid:0/0
systemd-coredump[11419]: Process 11381 (nginx) of user 33 dumped core.

                                                    Stack trace of thread 11381:
                                                    #0  0x00000000004c5964 nx_find_wl_in_hash (nginx)
                                                    #1  0x00000000004c5e85 ngx_http_dummy_is_rule_whitelisted_n (nginx)
                                                    #2  0x00000000004c7599 ngx_http_apply_rulematch_v_n (nginx)
                                                    #3  0x00000000004c7ee2 ngx_http_basestr_ruleset_n (nginx)
                                                    #4  0x00000000004c82c5 ngx_http_dummy_headers_parse (nginx)
                                                    #5  0x00000000004c9980 ngx_http_dummy_data_parse (nginx)
                                                    #6  0x00000000004cbc9e n/a (nginx)
                                                    #7  0x00000000004563fc ngx_http_core_rewrite_phase (nginx)
                                                    #8  0x0000000000451895 ngx_http_core_run_phases (nginx)
                                                    #9  0x000000000045c973 ngx_http_process_request (nginx)
                                                    #10 0x00000000004890a6 n/a (nginx)
                                                    #11 0x0000000000489d76 n/a (nginx)
                                                    #12 0x0000000000489fbe n/a (nginx)
                                                    #13 0x0000000000488855 n/a (nginx)
                                                    #14 0x000000000043e750 ngx_event_process_posted (nginx)
                                                    #15 0x0000000000444921 n/a (nginx)
                                                    #16 0x0000000000443350 ngx_spawn_process (nginx)
                                                    #17 0x0000000000444c84 n/a (nginx)
                                                    #18 0x00000000004455a4 ngx_master_process_cycle (nginx)
                                                    #19 0x0000000000421838 main (nginx)
                                                    #20 0x0000031d74329710 __libc_start_main (libc.so.6)
                                                    #21 0x0000000000421d09 _start (nginx)

                                                    Stack trace of thread 11392:
                                                    #0  0x0000031d761333e8 pthread_cond_timedwait@@GLIBC_2.3.2 (libpthread.so.0)
                                                    #1  0x00000000006f3095 n/a (nginx)
                                                    #2  0x00000000006efb3a n/a (nginx)
                                                    #3  0x000000000050b86e n/a (nginx)
                                                    #4  0x00000000006f4588 n/a (nginx)
                                                    #5  0x0000031d7612d424 start_thread (libpthread.so.0)
                                                    #6  0x0000031d743f0cbd __clone (libc.so.6)

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#227 (comment)

[rfnx]

@buixor Do you want the headers ?

[bbigras]

I also have had a crash. I don't know how to reproduce it yet.

Reading symbols from /usr/sbin/nginx...Reading symbols from /usr/lib/debug/.build-id/7c/6f589235091c270b13731c005969fc22f4c29d.debug...done.
done.
[New LWP 30858]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/i386-linux-gnu/libthread_db.so.1".
Core was generated by `nginx: worker process                   '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x080f880e in nx_find_wl_in_hash (mstr=mstr@entry=0x8823150, cf=cf@entry=0x86ea644, zone=zone@entry=HEADERS) at /home/bbigras/nginx/nginx-1.9.15/debian/modules/naxsi/naxsi_src/naxsi_runtime.c:342
342         mstr->data[i] = tolower(mstr->data[i]);
(gdb) bt
#0  0x080f880e in nx_find_wl_in_hash (mstr=mstr@entry=0x8823150, cf=cf@entry=0x86ea644, zone=zone@entry=HEADERS) at /home/bbigras/nginx/nginx-1.9.15/debian/modules/naxsi/naxsi_src/naxsi_runtime.c:342
#1  0x080f8d5c in ngx_http_dummy_is_rule_whitelisted_n (req=req@entry=0x88225c8, cf=cf@entry=0x86ea644, r=r@entry=0x86c32a4, name=name@entry=0x8823150, zone=zone@entry=HEADERS,
    target_name=target_name@entry=0) at /home/bbigras/nginx/nginx-1.9.15/debian/modules/naxsi/naxsi_src/naxsi_runtime.c:629
#2  0x080fa5b7 in ngx_http_apply_rulematch_v_n (r=r@entry=0x86c32a4, ctx=ctx@entry=0x882330c, req=req@entry=0x88225c8, name=name@entry=0x8823150, value=value@entry=0x8823158, zone=zone@entry=HEADERS,
    nb_match=0, target_name=target_name@entry=0) at /home/bbigras/nginx/nginx-1.9.15/debian/modules/naxsi/naxsi_src/naxsi_runtime.c:1030
#3  0x080faf0c in ngx_http_basestr_ruleset_n (pool=0x88225a0, name=name@entry=0x8823150, value=value@entry=0x8823158, rules=0x86bd8c0, req=req@entry=0x88225c8, ctx=ctx@entry=0x882330c,
    zone=zone@entry=HEADERS) at /home/bbigras/nginx/nginx-1.9.15/debian/modules/naxsi/naxsi_src/naxsi_runtime.c:1397
#4  0x080fb2e3 in ngx_http_dummy_headers_parse (main_cf=main_cf@entry=0x85b8598, cf=cf@entry=0x86ea644, ctx=ctx@entry=0x882330c, r=r@entry=0x88225c8)
    at /home/bbigras/nginx/nginx-1.9.15/debian/modules/naxsi/naxsi_src/naxsi_runtime.c:2078
#5  0x080fcd98 in ngx_http_dummy_data_parse (ctx=ctx@entry=0x882330c, r=r@entry=0x88225c8) at /home/bbigras/nginx/nginx-1.9.15/debian/modules/naxsi/naxsi_src/naxsi_runtime.c:2102
#6  0x080fed3f in ngx_http_dummy_access_handler (r=0x88225c8) at /home/bbigras/nginx/nginx-1.9.15/debian/modules/naxsi/naxsi_src/naxsi_skeleton.c:1228
#7  0x080973c3 in ngx_http_core_rewrite_phase (r=0x88225c8, ph=0x879df94) at src/http/ngx_http_core_module.c:901
#8  0x08092e29 in ngx_http_core_run_phases (r=0x88225c8) at src/http/ngx_http_core_module.c:847
#9  0x080c62f8 in ngx_http_v2_process_request_body (r=0x88225c8, pos=pos@entry=0x87bf711 "{\"Nom\":\"M05021\",\"Type\":\"NonPeinte\"}\320b\r&=LtA\352\373$\343\261\005L\034\067\341Y\357Ր؂ڇ",
    size=size@entry=35, last=1) at src/http/v2/ngx_http_v2.c:3587
#10 0x080c7513 in ngx_http_v2_state_read_data (h2c=0x86a5720, pos=0x87bf711 "{\"Nom\":\"M05021\",\"Type\":\"NonPeinte\"}\320b\r&=LtA\352\373$\343\261\005L\034\067\341Y\357Ր؂ڇ",
    end=0x87bf734 "\320b\r&=LtA\352\373$\343\261\005L\034\067\341Y\357Ր؂ڇ") at src/http/v2/ngx_http_v2.c:922
#11 0x080c8a0a in ngx_http_v2_read_handler (rev=<optimized out>) at src/http/v2/ngx_http_v2.c:362
#12 0x0807f78c in ngx_event_process_posted (cycle=<optimized out>, cycle@entry=0x85b64b8, posted=<optimized out>) at src/event/ngx_event_posted.c:33
#13 0x0807f35a in ngx_process_events_and_timers (cycle=cycle@entry=0x85b64b8) at src/event/ngx_event.c:259
#14 0x08085976 in ngx_worker_process_cycle (cycle=0x85b64b8, data=0x0) at src/os/unix/ngx_process_cycle.c:753
#15 0x080843a6 in ngx_spawn_process (cycle=cycle@entry=0x85b64b8, proc=0x80858f0 <ngx_worker_process_cycle>, data=0x0, name=0x8171fbb "worker process", respawn=respawn@entry=2)
    at src/os/unix/ngx_process.c:198
#16 0x080869f2 in ngx_reap_children (cycle=0x85b64b8) at src/os/unix/ngx_process_cycle.c:621
#17 ngx_master_process_cycle (cycle=<optimized out>, cycle@entry=0x85b2498) at src/os/unix/ngx_process_cycle.c:174
#18 0x080633de in main (argc=3, argv=0xbf8d5dd4) at src/core/nginx.c:367

[Louvremaster]

I have the same problem with nginx 1.10.3 + naxsi 0.55.3 when http2 is enabled.
Will this be fixed?

[danlsgiga]

I actually had to remove all the $HEADERS_VAR:Cookie from all MainRules in the core.rules file. For some reason, having it with HTTP2 causes the segfaults. Removed them and no issues for 2 days.

I know that removing it is not ideal but having HttpOnly; Secure in the headers kinda overcome most of the cookies attacks, so I'll leave it out for now.

Does anyone have more insights on the risks and implications of leaving that out from the core rules?

[buixor]

Hello,

@danlsgiga / @Louvremaster : did this happen with the http2 branch ?

Thanks !

ps: sorry for delay, I was kinda busy, http2 branch is going for merge in next major

[danlsgiga]

Hey @buixor, yeah, http2 branch and the latest nginx release.

[buixor]

That is bad news :( I'm going to look at it when I come back from hollidays, end of April

[ManuelRighi]

Hello,
I have same problem, nginx 1.10.3, naxsi 0.55.2 and http2 enabled :|

[marcelomd]

@danlsgiga @ManuelRighi Did you guys try PR #309 on top of http2 branch?

We've been running HTTP/2 + Naxsi for months with no errors.

The segfaults were caused by Naxsi trying to modify the header part of the request, which lives in read-only memory. See previous posts on this thread.

Unless these are new segfaults, #309 should solve them =)

[ManuelRighi]

Thanks @marcelomd ... I try http2 branch ;)

[buixor]

Eh I guess I should really merge this one into http2 branch, @marcelomd : no update to do on this one ?

[marcelomd]

@buixor Nothing new. Works beautifully for us as it is. Go ahead =)

[gutweiler]

@buixor No problem at all with http2 branch plus MR #309, would be nice to have it in master. 👍

[danlsgiga]

Hi @marcelomd, yeah, I was just using the http2 branch without the suggested PR #309. @buixor It would be nice to have that merged in http2 since I have a job that builds nginx automatically from that branch.
Thanks for the patches!!

[buixor]

merged into http2, which itself should make its way to master for 0.56 :)

[danlsgiga]

Beautiful! Thanks @buixor

[danlsgiga]

After recompiling nginx with the http2 branch with the merge of PR #309 I confirm that all issues and segfaults are gone. I've restored the original core.rules with all the $HEADERS_VAR:Cookie and everything is smooth again.
Thanks everyone!!!!

[ManuelRighi]

Hello, What exactly is the commands for download branch HTTP2 and merger of PR # 309 ?
Thanks

[gutweiler]

@ManuelRighi

It would be:
git clone https://github.com/nbs-system/naxsi.git && \ cd naxsi && \ git checkout http2 && \ git fetch origin pull/309/head:http2-309 && \ git checkout http2-309 && \

But meanwhile #309 is merged into http2 branch. -> you can just checkout the http2 branch.

[ManuelRighi]

@gutweiler thanks ;)

[ManuelRighi]

Hello,
with nginx 1.12.1, pagespeed 1.12.34.2-stable and naxsi with branch http2, I receive this error during compile:

objs/addon/src/ngx_url_async_fetcher.o:/opt/ngx_pagespeed-1.12.34.2-stable/src/ngx_url_async_fetcher.cc:89: first defined here
collect2: error: ld returned 1 exit status
objs/Makefile:382: recipe for target 'objs/nginx' failed
make[2]: *** [objs/nginx] Error 1
make[2]: Leaving directory '/opt/nginx_source/nginx-1.12.1/debian/build-nginx'
Makefile:8: recipe for target 'build' failed
make[1]: *** [build] Error 2
make[1]: Leaving directory '/opt/nginx_source/nginx-1.12.1/debian/build-nginx'
debian/rules:50: recipe for target 'build-arch.nginx' failed
make: *** [build-arch.nginx] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

If I use branch master, no problem, but I have issue with http2 ..... I need to merge http2 on master ?

Thanks

[combro2k]

@ManuelRighi
Looks more like an issue of pagespeed... did you try to compile without naxsi?
If the problem presists, you need to contact pagespeed's developers.
if the problem is gone, compile naxsi without pagespeed and see if that helps.

[ManuelRighi]

@combro2k nginx + pagespeed without naxsi compile is ok

[combro2k]

@ManuelRighi Maybe this is related? apache/incubator-pagespeed-ngx#1194

Did you try to compile it against nginx mainline version?

[ManuelRighi]

Hello,
If I reverse the module order, compile works :)

[buixor]

@ManuelRighi : can you provide extra info ? :D

[ManuelRighi]

If I compile with this ./configure order

--add-module=/opt/ngx_pagespeed-1.12.34.2-stable --add-module=/opt/naxsi-0.55.3/naxsi_src

compile fails. If I reverse order,

--add-module=/opt/naxsi-0.55.3/naxsi_src --add-module=/opt/ngx_pagespeed-1.12.34.2-stable

compile works

[ManuelRighi]

Hello,
nginx 1.10.3 with naxsi 0.55.3 is affected by this bug ?
If yes, If I checkout http2 branch for naxsi, the naxsi version is still the 0.55.3 ?

Thanks

[msrv]

EDIT: Solved it myself, error was caused by compiling as dynamic module. When using naxsi as static module, the http2 version works with the latest nginx version again.

The issue was fixed before by compiling nginx from the http2 branch. But with the nginx release
1.13.6 nginx will not start even if using the http2 branch of nginx.

Any ideas?