runtime: buildmode=c-shared segfaults with CGo + Go1.5 (and Go1.5.1)

[nirbhayc]

• What version of Go are you using (go version)?
  go version go1.5.1 linux/amd64
• What operating system and processor architecture are you using?
  Linux nirbhay-VirtualBox-1 3.13.0-63-generic Undefined reference "elm" in godocs.js:16 #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
• What did you do?
  (a) Install MariaDB server
  (b) Build User defined function (UDF) shared object file and copy it under /usr/lib/mysql/plugin/
  Code: http://play.golang.org/p/RN7R3M_wzt
  Build: go build -buildmode=c-shared -o hello.so src/hello/hello.go
  (c) Using mysql client connect to the mysqld server and run the following command

DROP FUNCTION IF EXISTS hello;
CREATE FUNCTION hello RETURNS STRING SONAME 'hello.so';
SELECT hello();
DROP FUNCTION hello;

• What did you expect to see?
  The hello() function should print "world".
• What did you see instead?
  Frequent crashes.
  Under valgrind

Version: '10.0.20-MariaDB-debug-log'  socket: '/tmp/my_node1.sock'  port: 16001  Source distribution
==4472== Warning: ignored attempt to set SIGRT32 handler in sigaction();
==4472==          the SIGRT32 signal is used internally by Valgrind
fatal error: rt_sigaction failure

runtime stack:
runtime.throw(0x288effc0, 0x14)
    /home/nirbhay/project/git-repo/nirbhayc/go/go/src/runtime/panic.go:527 +0x92
runtime.setsig(==4472== Thread 26:

Full output : http://pastie.org/10411289

Discussion : https://groups.google.com/forum/#!topic/golang-nuts/_HNmB3oHnl8

[ianlancetaylor]

valgrind doesn't work on Go programs. Your valgrind trace shows that the Go code is crashing because of valgrind. So that really doesn't tell us much.

Can you describe the crashes that you get when not running valgrind? Do you get any kind of stack trace? Can you run the crashing program under a debugger and get a stack trace that way?

[nirbhayc]

Running it under gdb gives me the following stack:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffcc7fd700 (LWP 3002)]
0x00007fffd4d5c081 in ?? ()
(gdb) bt
#0  0x00007fffd4d5c081 in ?? ()
#1  0x00007fffd4d2b6b3 in ?? ()
#2  0x000000c82000ce90 in ?? ()
#3  0x0000000000000000 in ?? ()
(gdb) 

[ianlancetaylor]

Thanks, but unfortunately that's pretty useless. It tells us that the segmentation violation is occurring somewhere in C code.

Can you find out at what stage in your process the crash occurs?

Does "DROP FUNCTION hello;" call dlclose? Because you can't call dlclose on a Go shared library.

[nirbhayc]

Yes, dlcose() is called by mysqld on DROP FUNCTION.

[nirbhayc]

The crash is pretty random. The following happened on CREATE FUNCTION this time:

[New Thread 0x7fffcd7ff700 (LWP 3542)]

Program received signal SIGINT, Interrupt.
[Switching to Thread 0x7fffcd7ff700 (LWP 3542)]
runtime.rtsigprocmask () at /home/nirbhay/project/git-repo/nirbhayc/go/go/src/runtime/sys_linux_amd64.s:206
206     CMPQ    AX, $0xfffffffffffff001
(gdb) bt
#0  runtime.rtsigprocmask () at /home/nirbhay/project/git-repo/nirbhayc/go/go/src/runtime/sys_linux_amd64.s:206
#1  0x00007fffd4d2bfef in runtime.minit ()
    at /home/nirbhay/project/git-repo/nirbhayc/go/go/src/runtime/os1_linux.go:228
#2  0x00007fffd4d33615 in runtime.mstart1 () at /home/nirbhay/project/git-repo/nirbhayc/go/go/src/runtime/proc1.go:707
#3  0x00007fffd4d33574 in runtime.mstart () at /home/nirbhay/project/git-repo/nirbhayc/go/go/src/runtime/proc1.go:691
#4  0x00007fffd4d587bc in runtime.rt0_go ()
    at /home/nirbhay/project/git-repo/nirbhayc/go/go/src/runtime/asm_amd64.s:120
#5  0x0000000000000002 in ?? ()
#6  0x00007fffffffde18 in ?? ()
#7  0x0000000000000002 in ?? ()
#8  0x00007fffffffde18 in ?? ()
#9  0x00007fffcd7ff700 in ?? ()
#10 0x00007ffff7339182 in start_thread (arg=0x7fffcd7ff700) at pthread_create.c:312
#11 0x00007ffff621e47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

[nirbhayc]

Any suggestion on how to decipher the '??'. ? On DROP FUNCTION with a breakpoint on dlopen:

Breakpoint 1, __dlclose (handle=0x7fffda2c5a00) at dlclose.c:42
42  dlclose.c: No such file or directory.
(gdb) n
46  in dlclose.c
(gdb) 
40  in dlclose.c
(gdb) 
46  in dlclose.c
(gdb) 

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffcc7fd700 (LWP 3594)]
0x00007fffd4d5c081 in ?? ()
(gdb) bt
#0  0x00007fffd4d5c081 in ?? ()
#1  0x00007fffd4d2b6b3 in ?? ()
#2  0x000000c82000ce90 in ?? ()
#3  0x0000000000000000 in ?? ()
(gdb) 

The segfault happens in a separate thread.

[ianlancetaylor]

A SIGINT signal is not caused by your program, unless your program explicitly calls raise(SIGINT) or kill(getpid(), SIGINT). A SIGINT normally comes from hitting ^C.

[ianlancetaylor]

Calling dlclose on a Go shared library is just flat out not going to work. Digging into the details is of only academic interest.

The first step to figuring out the ?? is to use "info sharedlibrary" to find out which shared library those addresses are in.

[nirbhayc]

^C could be from my weird touchpad. Sorry about that.
The above stack addresses do not fall in any of the shared libs listed by 'info shared'.

[ianlancetaylor]

Those aren't stack addresses, they are PC addresses. For them to not be in any of the shared libraries is approximately impossible. However, if they are not, then the next step is to look for the addresses in /proc/PID/maps.