SEGFAULT in statically-linked extension

[riyaz-ali]

Based on my previous comment in #2, some static-linking scenarios are supported right now, notably where you use sqlite3_auto_extension() to auto-load the extension for every new database connection. This approach works but is not suitable for all the use cases.

One such use-case is when you want to register the extension manually in Golang, which is what I believe @anacrolix has tried in #16 and also outlined in this comment.

---

Based on inputs from @anacrolix (#2 (comment) and #16 (comment)) and also in my own analysis, I've seen the extension building fine (following the steps outlined below) but crashing at runtime with a SEGFAULT (illegal memory access).

We can reproduce the error based on a simple extension (using _examples/upper/upper.go) and using a modified version of crawshaw.io/sqlite that allow exposing the underlying connection object (somewhat similar to crawshaw:sqlite:master...anacrolix:crawshaw-sqlite:master)

Using a helper function similar to following, we can register the extension:

type UnderlyingConnection *C.struct_sqlite3

func RegisterWith(conn UnderlyingConnection, fn ExtensionFunc) error {
	_, err := fn(&ExtensionApi{db: (*C.struct_sqlite3)(conn)})
	return err
}

On the application side, after opening a new connection (using crawshaw.io/sqlite), we can manually call RegisterWith() to register the extension:

conn, _ := sqlite.OpenConn("file:test.db?mode=memory")
defer conn.Close()

uc := ext.UnderlyingConnection(conn.UnderlyingConnection())  // ext.UnderlyingConnection() is a type-cast
err := ext.RegisterWith(uc, func(api *ext.ExtensionApi) (ext.ErrorCode, error) {
	if err := api.CreateFunction("go_upper", &Upper{}); err != nil {
		return ext.SQLITE_ERROR, err
	}
	return ext.SQLITE_OK, nil
})

To build this, run the following command:

> go build -tags=static .

It builds just fine, but when executed it fails with a SEGFAULT exception:

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x510 pc=0x40d9bee]

runtime stack:
runtime.throw({0x4255bed, 0x7ff85f1a5000})
	/Users/riyaz/golang/go1.17.6/src/runtime/panic.go:1198 +0x71
runtime.sigpanic()
	/Users/riyaz/golang/go1.17.6/src/runtime/signal_unix.go:719 +0x20e

goroutine 1 [syscall]:
runtime.cgocall(0x40da290, 0xc0000a39c8)
	/Users/riyaz/golang/go1.17.6/src/runtime/cgocall.go:156 +0x5c fp=0xc0000a3968 sp=0xc0000a3930 pc=0x4006a9c
go.riyazali.net/sqlite._Cfunc__sqlite3_create_function_v2(0xbb04080, 0x600000008140, 0x1, 0x801, 0x600000008150, 0x40d8870, 0x0, 0x0, 0x40d8af0)
	_cgo_gotypes.go:927 +0x4f fp=0xc0000a39c8 sp=0xc0000a3968 pc=0x40b910f
go.riyazali.net/sqlite.(*ExtensionApi).CreateFunction.func2(0xc0000b8088, 0x600000008140, {0x425e7c0, 0x4340548}, 0x801, 0x600000008150, 0x40d8870, 0x40d8af0)
	/Users/riyaz/Projects/sqlite3/func.go:99 +0x1bf fp=0xc0000a3a90 sp=0xc0000a39c8 pc=0x40bd0df
go.riyazali.net/sqlite.(*ExtensionApi).CreateFunction(0xc0000b8088, {0x424ecb2, 0x8}, {0x425e7c0, 0x4340548})
	/Users/riyaz/Projects/sqlite3/func.go:99 +0x1fa fp=0xc0000a3ce8 sp=0xc0000a3a90 pc=0x40bc75a
main.main.func1(0xc0000b8088)
	/Users/riyaz/Projects/segfault/main.go:17 +0x5a fp=0xc0000a3d58 sp=0xc0000a3ce8 pc=0x40d869a
go.riyazali.net/sqlite.RegisterWith(0xbb04080, 0x42574c0)
	/Users/riyaz/Projects/sqlite3/extension.go:54 +0x52 fp=0xc0000a3db0 sp=0xc0000a3d58 pc=0x40bc3d2
main.main()
	/Users/riyaz/Projects/segfault/main.go:16 +0x105 fp=0xc0000a3f80 sp=0xc0000a3db0 pc=0x40d81a5
runtime.main()
	/Users/riyaz/golang/go1.17.6/src/runtime/proc.go:255 +0x1f3 fp=0xc0000a3fe0 sp=0xc0000a3f80 pc=0x403abf3
runtime.goexit()
	/Users/riyaz/golang/go1.17.6/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc0000a3fe8 sp=0xc0000a3fe0 pc=0x4065f01

[riyaz-ali]

With inputs from @anacrolix and some additional help from gdb, the cause for segmentation fault is narrowed to sqlite3_api object being NULL.

If we look at the sqlite3ext.h header file, we can see that most of the sqlite3_* function calls are re-defined (using macros) to calls on the sqlite3_api object. This works for dynamic extension as the sqlite3 core passes this object to the extension entry-point where it is expected to be passed to SQLITE_EXTENSION_INIT2 macro, which sets the value for the global, static variable.

In case of static-linked extension, sqlite3 documentation suggests setting the SQLITE_CORE macro and calling the extension entry-point manually with the sqlite3 object.

Again, looking at the sqlite3ext.h file, if the SQLITE_CORE macro is present, then the function calls are not re-defined which is what we'd want in a static-linked extension.

[anacrolix]

Sorry if it wasn't clear, but I'd already worked out the cause for the segfault (SQLITE_CORE needs to be defined). Similarly, the fix required was 6b22d30, which I linked to from #16 (comment) and #2 (comment).

[riyaz-ali]

Ah! my bad 😓 I just wanted to add a detailed report about the SEGFAULT based on the comments that popped up in different threads. I've edited the previous comments to reflect that 🙌