Fix flex overflow #1650

mmisono · 2020-11-24T21:04:28Z

Add workaround for flex scanner push-back overflow.

Checklist

Language changes are updated in docs/reference_guide.md
User-visible and non-trivial changes updated in CHANGELOG.md
The new behaviour is covered by tests

mmisono · 2020-11-24T21:05:52Z

@fbs I encountered the following memleaks. Do you think your PRs fix them? Or is this different issue? Maybe my error handling is wrong.

% sudo  ./src/bpftrace -e 'BEGIN {IS_MODULE}'
stdin:1:7-17: ERROR: Too deep recursive or long macro is detected: IS_MODULE, (option)__is_defined(option##_MODULE)
BEGIN {IS_MODULE}
      ~~~~~~~~~~
stdin:1:7-17: ERROR: syntax error, unexpected end of file, expecting }
BEGIN {IS_MODULE}
      ~~~~~~~~~~

=================================================================
==18766==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x5a653d in operator new(unsigned long) (/disk/work/bpftrace2/build/src/bpftrace+0x5a653d)
    #1 0xd16dbc in bpftrace::Parser::parse() /disk/work/bpftrace2/build/src/parser.yy:174:97
    #2 0x922add in bpftrace::Driver::parse() /disk/work/bpftrace2/src/driver.cpp:49:10
    #3 0xa4704f in main /disk/work/bpftrace2/src/main.cpp:733:16
    #4 0x7fffed57e0b2 in __libc_start_main /build/glibc-ZN95T4/glibc-2.31/csu/../csu/libc-start.c:308:16

Direct leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x5a653d in operator new(unsigned long) (/disk/work/bpftrace2/build/src/bpftrace+0x5a653d)
    #1 0xd1e62c in bpftrace::Parser::parse() /disk/work/bpftrace2/build/src/parser.yy:239:83
    #2 0x922add in bpftrace::Driver::parse() /disk/work/bpftrace2/src/driver.cpp:49:10
    #3 0xa4704f in main /disk/work/bpftrace2/src/main.cpp:733:16
    #4 0x7fffed57e0b2 in __libc_start_main /build/glibc-ZN95T4/glibc-2.31/csu/../csu/libc-start.c:308:16

Indirect leak of 432 byte(s) in 1 object(s) allocated from:
    #0 0x5a653d in operator new(unsigned long) (/disk/work/bpftrace2/build/src/bpftrace+0x5a653d)
    #1 0xd17203 in bpftrace::Parser::parse() /disk/work/bpftrace2/build/src/parser.yy:177:93
    #2 0x922add in bpftrace::Driver::parse() /disk/work/bpftrace2/src/driver.cpp:49:10
    #3 0xa4704f in main /disk/work/bpftrace2/src/main.cpp:733:16
    #4 0x7fffed57e0b2 in __libc_start_main /build/glibc-ZN95T4/glibc-2.31/csu/../csu/libc-start.c:308:16

Indirect leak of 8 byte(s) in 1 object(s) allocated from:
    #0 0x5a653d in operator new(unsigned long) (/disk/work/bpftrace2/build/src/bpftrace+0x5a653d)
    #1 0xd6820a in __gnu_cxx::new_allocator<bpftrace::ast::AttachPoint*>::allocate(unsigned long, void const*) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/ext/new_allocator.h:114:27
    #2 0xd68170 in std::allocator_traits<std::allocator<bpftrace::ast::AttachPoint*> >::allocate(std::allocator<bpftrace::ast::AttachPoint*>&, unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/alloc_traits.h:444:20
    #3 0xd67dac in std::_Vector_base<bpftrace::ast::AttachPoint*, std::allocator<bpftrace::ast::AttachPoint*> >::_M_allocate(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_vector.h:343:20
    #4 0xd66f34 in void std::vector<bpftrace::ast::AttachPoint*, std::allocator<bpftrace::ast::AttachPoint*> >::_M_realloc_insert<bpftrace::ast::AttachPoint* const&>(__gnu_cxx::__normal_iterator<bpftrace::ast::AttachPoint**, std::vector<bpftrace::ast::AttachPoint*, std::allocator<bpftrace::ast::AttachPoint*> > >, bpftrace::ast::AttachPoint* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/vector.tcc:440:33
    #5 0xd561ec in std::vector<bpftrace::ast::AttachPoint*, std::allocator<bpftrace::ast::AttachPoint*> >::push_back(bpftrace::ast::AttachPoint* const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_vector.h:1195:4
    #6 0xd171ef in bpftrace::Parser::parse() /disk/work/bpftrace2/build/src/parser.yy:174:169
    #7 0x922add in bpftrace::Driver::parse() /disk/work/bpftrace2/src/driver.cpp:49:10
    #8 0xa4704f in main /disk/work/bpftrace2/src/main.cpp:733:16
    #9 0x7fffed57e0b2 in __libc_start_main /build/glibc-ZN95T4/glibc-2.31/csu/../csu/libc-start.c:308:16

SUMMARY: AddressSanitizer: 488 byte(s) leaked in 4 allocation(s).

fbs · 2020-11-24T21:16:55Z

I don't have any prs related to this open so it must be something else. Maybe the driver.root isn't deleted properly after the error?

mmisono · 2020-11-24T21:44:57Z

hmm.. deleting root_ after error does not solve this.

mmisono · 2020-11-26T21:21:31Z

It seems #1651 accidentally close this PR.

danobi

Small nit but LGTM

src/lexer.l

danobi · 2021-01-12T21:12:34Z

@fbs I encountered the following memleaks. Do you think your PRs fix them? Or is this different issue? Maybe my error handling is wrong.

Maybe yyterminate() prevents some kind of memory cleanup?

Previously, ``` % cat a.h # define B A # define A B ``` then ``` % sudo ./src/bpftrace --include a.h -e 'BEGIN {A}' // infinite loops ``` to solve this problem, limit the numbuer of unputs in the lexer. The magic number 1000 is no particular meanings. I think it's just enough. Now the above example becomes ``` % sudo ./src/bpftrace --include a.h -e 'BEGIN {A}' stdin:1:7-1008: ERROR: Too deep recursive macros detected' BEGIN {A} ~~~ stdin:1:7-1008: ERROR: syntax error, unexpected end of file, expecting } BEGIN {A} ~~~ ```

For example, ``` %cat b.h #define B BBBBBBBBBBBBBBB ``` then, ``` % sudo ./src/bpftrace --include b.h -e 'BEGIN {B}' flex scanner push-back overflow ``` To cope with this, check flex's buffers length (yyleng) and if we try to unput more than the length, stop lexer. Now the above example becomes ``` % sudo ASAN_OPTIONS=detect_leaks=0 ./src/bpftrace --include b.h -e 'BEGIN {B}' stdin:1:7-9: ERROR: Too deep recursive or long macro is detected: B, BBBBBBBBBBBBBBB BEGIN {B} ~~ stdin:1:7-9: ERROR: syntax error, unexpected end of file, expecting } BEGIN {B} ~~ ```

mmisono · 2021-01-13T23:58:13Z

@fbs I encountered the following memleaks. Do you think your PRs fix them? Or is this different issue? Maybe my error handling is wrong.

Maybe yyterminate() prevents some kind of memory cleanup?

OK, I'll merge this for now and create an issue with the memleaks.

mmisono mentioned this pull request Nov 24, 2020

Delete driver.root_ when failing parse #1651

Merged

3 tasks

fbs closed this in #1651 Nov 26, 2020

mmisono reopened this Nov 26, 2020

danobi reviewed Jan 12, 2021

View reviewed changes

src/lexer.l Outdated Show resolved Hide resolved

danobi added the waiting-on-author label Jan 12, 2021

mmisono added 3 commits January 14, 2021 08:28

Update changelog and fuzzing.md

07afec4

mmisono force-pushed the fix_flex_overflow branch from 6195388 to 07afec4 Compare January 13, 2021 23:30

mmisono merged commit 691c5e2 into bpftrace:master Jan 14, 2021

mmisono mentioned this pull request Jan 14, 2021

memleaks when the lexer fails #1674

Open

lenticularis39 mentioned this pull request Aug 1, 2022

Macro recursion limit reached for very simple macro in <linux/sched.h> #2304

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix flex overflow #1650

Fix flex overflow #1650

mmisono commented Nov 24, 2020

mmisono commented Nov 24, 2020

fbs commented Nov 24, 2020

mmisono commented Nov 24, 2020

mmisono commented Nov 26, 2020

danobi left a comment

danobi commented Jan 12, 2021

mmisono commented Jan 13, 2021

Fix flex overflow #1650

Fix flex overflow #1650

Conversation

mmisono commented Nov 24, 2020

Checklist

mmisono commented Nov 24, 2020

fbs commented Nov 24, 2020

mmisono commented Nov 24, 2020

mmisono commented Nov 26, 2020

danobi left a comment

Choose a reason for hiding this comment

danobi commented Jan 12, 2021

mmisono commented Jan 13, 2021