Skip to content
This repository has been archived by the owner on Aug 23, 2023. It is now read-only.

panic error : unexpected fault address #782

Closed
motyla opened this issue Dec 11, 2017 · 3 comments
Closed

panic error : unexpected fault address #782

motyla opened this issue Dec 11, 2017 · 3 comments

Comments

@motyla
Copy link

motyla commented Dec 11, 2017

metrictank --version
metrictank (built with go1.9, git hash 0.7.4-315-gb90bd2ae)


Dec 11 07:49:01 metrictank001 metrictank[4412]: unexpected fault address 0xdfbc49e078
Dec 11 07:49:01 metrictank001 metrictank[4412]: fatal error: fault
Dec 11 07:49:01 metrictank001 metrictank[4412]: [signal SIGBUS: bus error code=0x4 addr=0xdfbc49e078 pc=0x9f8cdf]
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 48704832 [running]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.throw(0xc69705, 0x5)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/panic.go:605 +0x95 fp=0xdf4db5d240 sp=0xdf4db5d220 pc=0x42ba65
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.sigpanic()
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/signal_unix.go:364 +0x29d fp=0xdf4db5d290 sp=0xdf4db5d240 pc=0x4425bd
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/klauspost/compress/flate.(*compressor).initDeflate(...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/klauspost/compress/flate/deflate.go:408
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/klauspost/compress/flate.(*compressor).init(0xdfbc3fe000, 0x7f9d146a90a0, 0xda26b24f40, 0xffffffffffffffff, 0xa, 0x0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/klauspost/compress/flate/deflate.go:1179 +0x49f fp=0xdf4db5d3c0 sp=0xdf4db5d290 pc=0x9f8cdf
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/klauspost/compress/flate.NewWriter(0x7f9d146a90a0, 0xda26b24f40, 0xffffffffffffffff, 0xa, 0xa, 0x0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/klauspost/compress/flate/deflate.go:1267 +0x62 fp=0xdf4db5d408 sp=0xdf4db5d3c0 pc=0x9f9882
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/klauspost/compress/gzip.(*Writer).Write(0xd3ac610790, 0xdd460cc000, 0x87a9, 0xca4a3, 0xc7088c, 0xa01a9d, 0xda26b24f40)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/klauspost/compress/gzip/gzip.go:193 +0x358 fp=0xdf4db5d488 sp=0xdf4db5d408 pc=0xa007a8
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/raintank/gziper.(*gzipResponseWriter).Write(0xda26b28f60, 0xdd460cc000, 0x87a9, 0xca4a3, 0xd3ac610790, 0x7f9d146a90a0, 0xdf4db5d528)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/raintank/gziper/gzip.go:122 +0x75 fp=0xdf4db5d4d8 sp=0xdf4db5d488 pc=0xa01315
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/api/middleware.(*TracingResponseWriter).Write(0xda26b28fc0, 0xdd460cc000, 0x87a9, 0xca4a3, 0xdf4db5d568, 0x9b6756, 0xda26b28f60)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/api/middleware/tracer.go:23 +0x78 fp=0xdf4db5d528 sp=0xdf4db5d4d8 pc=0x9b53b8
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*TplRender).Write(0xd9ca996050, 0xdd460cc000, 0x87a9, 0xca4a3, 0x9b7fdd, 0xd9ca996050, 0xc8)
Dec 11 07:49:01 metrictank001 metrictank[4412]: :1 +0x5a fp=0xdf4db5d570 sp=0xdf4db5d528 pc=0x9aadaa
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/api/middleware.(*Context).Write(0xd0a75a5540, 0xdd460cc000, 0x87a9, 0xca4a3, 0x13, 0x0, 0x7f9d74a124b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: :1 +0x61 fp=0xdf4db5d5b8 sp=0xdf4db5d570 pc=0x9b7f41
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/api/response.Write(0x108b120, 0xd0a75a5540, 0x108d1a0, 0xca534dfe60)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/api/response/response.go:26 +0x21e fp=0xdf4db5d6a8 sp=0xdf4db5d5b8 pc=0x9d8b7e
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/api.(*Server).getData(0xc420172780, 0xd0a75a5540, 0xd8a4a7bc00, 0x8, 0x9)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/api/cluster.go:169 +0x148 fp=0xdf4db5d760 sp=0xdf4db5d6a8 pc=0xa05388
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/api.(*Server).(github.com/grafana/metrictank/api.getData)-fm(0xd0a75a5540, 0xd8a4a7bc00, 0x8, 0x9)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/api/routes.go:36 +0x52 fp=0xdf4db5d798 sp=0xdf4db5d760 pc=0xa1dc42
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.call32(0xd3e79e6000, 0xc420151240, 0xdc762666e0, 0x2000000020)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/asm_amd64.s:509 +0x3b fp=0xdf4db5d7c8 sp=0xdf4db5d798 pc=0x45a0ab
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.call(0xb527e0, 0xc420151240, 0x13, 0xc68868, 0x4, 0xda26b29080, 0x2, 0x2, 0xc65780, 0xb527e0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:434 +0x906 fp=0xdf4db5daa0 sp=0xdf4db5d7c8 pc=0x4b39b6
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.Call(0xb527e0, 0xc420151240, 0x13, 0xda26b29080, 0x2, 0x2, 0x0, 0x0, 0x0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:302 +0xa4 fp=0xdf4db5db08 sp=0xdf4db5daa0 pc=0x4b2f94
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).callInvoke(0xd0a75a5460, 0xb527e0, 0xc420151240, 0x1096660, 0xb527e0, 0x2, 0xa01b0d, 0xda26b24f40, 0x0, 0xdf4db5dc20, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 +0x2c9 fp=0xdf4db5dbb8 sp=0xdf4db5db08 pc=0x97ade9
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).Invoke(0xd0a75a5460, 0xb527e0, 0xc420151240, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 +0xca fp=0xdf4db5dc50 sp=0xdf4db5dbb8 pc=0x97a6ea
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).run(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 +0x5e fp=0xdf4db5dcc8 sp=0xdf4db5dc50 pc=0x99760e
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).Next(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 +0x36 fp=0xdf4db5dce0 sp=0xdf4db5dcc8 pc=0x997546
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/api/middleware.Tracer.func1(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/api/middleware/tracer.go:54 +0x54c fp=0xdf4db5ddf8 sp=0xdf4db5dce0 pc=0x9b602c
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.call32(0xc4204507b0, 0xc42058e580, 0xd7093fb448, 0x800000008)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/asm_amd64.s:509 +0x3b fp=0xdf4db5de28 sp=0xdf4db5ddf8 pc=0x45a0ab
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.call(0xb3f1e0, 0xc42058e580, 0x13, 0xc68868, 0x4, 0xd0a75a54e0, 0x1, 0x1, 0xc65780, 0xb3f1e0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:434 +0x906 fp=0xdf4db5e100 sp=0xdf4db5de28 pc=0x4b39b6
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.Call(0xb3f1e0, 0xc42058e580, 0x13, 0xd0a75a54e0, 0x1, 0x1, 0xbbc540, 0xdf4db5e2f8, 0xdf4db5e3f0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:302 +0xa4 fp=0xdf4db5e168 sp=0xdf4db5e100 pc=0x4b2f94
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).callInvoke(0xd0a75a5460, 0xb3f1e0, 0xc42058e580, 0x1096660, 0xb3f1e0, 0x1, 0x0, 0x0, 0x0, 0x0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 +0x2c9 fp=0xdf4db5e218 sp=0xdf4db5e168 pc=0x97ade9
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).Invoke(0xd0a75a5460, 0xb3f1e0, 0xc42058e580, 0x5ae8c, 0x294c50d4, 0xdf4db5e308, 0x44be38, 0x152a9e93ec8d4)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 +0xca fp=0xdf4db5e2b0 sp=0xdf4db5e218 pc=0x97a6ea
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).run(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 +0x5e fp=0xdf4db5e328 sp=0xdf4db5e2b0 pc=0x99760e
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).Next(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 +0x36 fp=0xdf4db5e340 sp=0xdf4db5e328 pc=0x997546
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/api/middleware.RequestStats.func1(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/api/middleware/stats.go:72 +0xa9 fp=0xdf4db5e3d0 sp=0xdf4db5e340 pc=0x9b58d9
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.call32(0xc4204507b0, 0xc4201502c0, 0xdcd3821f08, 0x800000008)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/asm_amd64.s:509 +0x3b fp=0xdf4db5e400 sp=0xdf4db5e3d0 pc=0x45a0ab
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.call(0xb3f1e0, 0xc4201502c0, 0x13, 0xc68868, 0x4, 0xd0a75a54c0, 0x1, 0x1, 0xc65780, 0xb3f1e0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:434 +0x906 fp=0xdf4db5e6d8 sp=0xdf4db5e400 pc=0x4b39b6
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.Call(0xb3f1e0, 0xc4201502c0, 0x13, 0xd0a75a54c0, 0x1, 0x1, 0xbff380, 0xc4204507e0, 0xc9f701)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:302 +0xa4 fp=0xdf4db5e740 sp=0xdf4db5e6d8 pc=0x4b2f94
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).callInvoke(0xd0a75a5460, 0xb3f1e0, 0xc4201502c0, 0x1096660, 0xb3f1e0, 0x1, 0xd92b750598, 0xdf4db5e8f8, 0xdf4db5e8d8, 0x6000000004109d7, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 +0x2c9 fp=0xdf4db5e7f0 sp=0xdf4db5e740 pc=0x97ade9
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).Invoke(0xd0a75a5460, 0xb3f1e0, 0xc4201502c0, 0xdf4db5e908, 0x97b460, 0xb7c0e0, 0xda26b28ea0, 0xdf4db5e8f8)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 +0xca fp=0xdf4db5e888 sp=0xdf4db5e7f0 pc=0x97a6ea
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).run(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 +0x5e fp=0xdf4db5e900 sp=0xdf4db5e888 pc=0x99760e
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).Next(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 +0x36 fp=0xdf4db5e918 sp=0xdf4db5e900 pc=0x997546
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/raintank/gziper.Gziper.func1(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/raintank/gziper/gzip.go:73 +0x178 fp=0xdf4db5e970 sp=0xdf4db5e918 pc=0xa01748
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.call32(0xc4204507b0, 0xc9e958, 0xd9d6496eb8, 0x800000008)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/asm_amd64.s:509 +0x3b fp=0xdf4db5e9a0 sp=0xdf4db5e970 pc=0x45a0ab
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.call(0xb3f1e0, 0xc9e958, 0x13, 0xc68868, 0x4, 0xd0a75a54a0, 0x1, 0x1, 0xc65780, 0xb3f1e0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:434 +0x906 fp=0xdf4db5ec78 sp=0xdf4db5e9a0 pc=0x4b39b6
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.Call(0xb3f1e0, 0xc9e958, 0x13, 0xd0a75a54a0, 0x1, 0x1, 0x0, 0x0, 0x0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:302 +0xa4 fp=0xdf4db5ece0 sp=0xdf4db5ec78 pc=0x4b2f94
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).callInvoke(0xd0a75a5460, 0xb3f1e0, 0xc9e958, 0x1096660, 0xb3f1e0, 0x1, 0x0, 0x0, 0x0, 0x0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 +0x2c9 fp=0xdf4db5ed90 sp=0xdf4db5ece0 pc=0x97ade9
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).Invoke(0xd0a75a5460, 0xb3f1e0, 0xc9e958, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 +0xca fp=0xdf4db5ee28 sp=0xdf4db5ed90 pc=0x97a6ea
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).run(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 +0x5e fp=0xdf4db5eea0 sp=0xdf4db5ee28 pc=0x99760e
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).Next(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 +0x36 fp=0xdf4db5eeb8 sp=0xdf4db5eea0 pc=0x997546
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.Recovery.func1(0xd90cec61b0, 0xc4204f9400)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/recovery.go:161 +0x5b fp=0xdf4db5eee8 sp=0xdf4db5eeb8 pc=0x9a891b
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.call32(0xc4204506f0, 0xc9ec58, 0xd0143fb260, 0x1000000010)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/asm_amd64.s:509 +0x3b fp=0xdf4db5ef18 sp=0xdf4db5eee8 pc=0x45a0ab
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.call(0xb53aa0, 0xc9ec58, 0x13, 0xc68868, 0x4, 0xda26b28f30, 0x2, 0x2, 0xc65780, 0xb53aa0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:434 +0x906 fp=0xdf4db5f1f0 sp=0xdf4db5ef18 pc=0x4b39b6
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.Call(0xb53aa0, 0xc9ec58, 0x13, 0xda26b28f30, 0x2, 0x2, 0x47, 0x0, 0x0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:302 +0xa4 fp=0xdf4db5f258 sp=0xdf4db5f1f0 pc=0x4b2f94
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).callInvoke(0xd0a75a5460, 0xb53aa0, 0xc9ec58, 0x1096660, 0xb53aa0, 0x2, 0xc4204f9400, 0xc425152000, 0x47, 0xa0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 +0x2c9 fp=0xdf4db5f308 sp=0xdf4db5f258 pc=0x97ade9
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).Invoke(0xd0a75a5460, 0xb53aa0, 0xc9ec58, 0x3c, 0x3c, 0xc425152000, 0xc4204f9430, 0xdf4db5f420)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 +0xca fp=0xdf4db5f3a0 sp=0xdf4db5f308 pc=0x97a6ea
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).run(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 +0x5e fp=0xdf4db5f418 sp=0xdf4db5f3a0 pc=0x99760e
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).Next(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:104 +0x36 fp=0xdf4db5f430 sp=0xdf4db5f418 pc=0x997546
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.Logger.func1(0xd90cec61b0, 0xc4204f9400)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/logger.go:43 +0x25c fp=0xdf4db5f670 sp=0xdf4db5f430 pc=0x9a7bec
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.call32(0xc4204506f0, 0xc9ec38, 0xdd15eb64a0, 0x1000000010)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/asm_amd64.s:509 +0x3b fp=0xdf4db5f6a0 sp=0xdf4db5f670 pc=0x45a0ab
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.call(0xb53aa0, 0xc9ec38, 0x13, 0xc68868, 0x4, 0xda26b28f00, 0x2, 0x2, 0xc65780, 0xb53aa0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:434 +0x906 fp=0xdf4db5f978 sp=0xdf4db5f6a0 pc=0x4b39b6
Dec 11 07:49:01 metrictank001 metrictank[4412]: reflect.Value.Call(0xb53aa0, 0xc9ec38, 0x13, 0xda26b28f00, 0x2, 0x2, 0xd92b7505a8, 0xdf4db5faf8, 0xd92b750598)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/reflect/value.go:302 +0xa4 fp=0xdf4db5f9e0 sp=0xdf4db5f978 pc=0x4b2f94
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).callInvoke(0xd0a75a5460, 0xb53aa0, 0xc9ec38, 0x1096660, 0xb53aa0, 0x2, 0x4109d7, 0xc9918fdb00, 0xc0, 0xc0, ...)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:177 +0x2c9 fp=0xdf4db5fa90 sp=0xdf4db5f9e0 pc=0x97ade9
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/go-macaron/inject.(*injector).Invoke(0xd0a75a5460, 0xb53aa0, 0xc9ec38, 0x108d420, 0xd0a75a5460, 0x3, 0xc4202f8280, 0xc9918fdb90)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/go-macaron/inject/inject.go:137 +0xca fp=0xdf4db5fb28 sp=0xdf4db5fa90 pc=0x97a6ea
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Context).run(0xd90cec61b0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/context.go:113 +0x5e fp=0xdf4db5fba0 sp=0xdf4db5fb28 pc=0x99760e
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Router).Handle.func1(0x108c1e0, 0xdfb88addc0, 0xd7ad016e00, 0xda26b28e70)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/router.go:184 +0x2a3 fp=0xdf4db5fc50 sp=0xdf4db5fba0 pc=0x9a9c03
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Router).ServeHTTP(0xc4204f93b0, 0x108c1e0, 0xdfb88addc0, 0xd7ad016e00)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/router.go:286 +0x14c fp=0xdf4db5fcb0 sp=0xdf4db5fc50 pc=0x9a324c
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/gopkg.in/macaron%2ev1.(*Macaron).ServeHTTP(0xc420441d80, 0x108c1e0, 0xdfb88addc0, 0xd7ad016e00)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/gopkg.in/macaron.v1/macaron.go:177 +0x138 fp=0xdf4db5fd18 sp=0xdf4db5fcb0 pc=0x99b788
Dec 11 07:49:01 metrictank001 metrictank[4412]: net/http.serverHandler.ServeHTTP(0xc42076a8f0, 0x108c1e0, 0xdfb88addc0, 0xd7ad016e00)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/net/http/server.go:2619 +0xb4 fp=0xdf4db5fd48 sp=0xdf4db5fd18 pc=0x6dc7d4
Dec 11 07:49:01 metrictank001 metrictank[4412]: net/http.(*conn).serve(0xd3858b25a0, 0x108ce60, 0xd7c193f340)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/net/http/server.go:1801 +0x71d fp=0xdf4db5ffc8 sp=0xdf4db5fd48 pc=0x6d8aed
Dec 11 07:49:01 metrictank001 metrictank[4412]: runtime.goexit()
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xdf4db5ffd0 sp=0xdf4db5ffc8 pc=0x45c8a1
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by net/http.(*Server).Serve
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/net/http/server.go:2720 +0x288
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 1 [chan receive, 960 minutes]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: main.main()
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/metrictank.go:403 +0x15af
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 5 [syscall, 993 minutes]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: os/signal.signal_recv(0x0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/runtime/sigqueue.go:131 +0xa6
Dec 11 07:49:01 metrictank001 metrictank[4412]: os/signal.loop()
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/os/signal/signal_unix.go:22 +0x22
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by os/signal.init.0
Dec 11 07:49:01 metrictank001 metrictank[4412]: /usr/local/go/src/os/signal/signal_unix.go:28 +0x41
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 18 [select]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/raintank/worldping-api/pkg/log.(*Logger).StartLogger(0xc420246100)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/raintank/worldping-api/pkg/log/log.go:234 +0x1e3
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by github.com/grafana/metrictank/vendor/github.com/raintank/worldping-api/pkg/log.newLogger
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/raintank/worldping-api/pkg/log/log.go:167 +0x115
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 19 [select]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0xc4202500e0)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:206 +0x172
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:237 +0x195
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 59 [select, 20 minutes]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0xc42018e900)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:90 +0x119
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.openDB
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db.go:141 +0x528
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 60 [select]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0xc42018e900)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db_state.go:96 +0x13b
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.openDB
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db.go:142 +0x54a
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 61 [select, 20 minutes]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0xc42018e900)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:804 +0x2c8
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.openDB
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db.go:148 +0x6c9
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 62 [select, 20 minutes]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0xc42018e900)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db_compaction.go:751 +0x192
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.openDB
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db.go:149 +0x6eb
Dec 11 07:49:01 metrictank001 metrictank[4412]: goroutine 63 [select, 993 minutes]:
Dec 11 07:49:01 metrictank001 metrictank[4412]: github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.(*DB).jWriter(0xc42018e900)
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db_write.go:37 +0x12b
Dec 11 07:49:01 metrictank001 metrictank[4412]: created by github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb.openDB
Dec 11 07:49:01 metrictank001 metrictank[4412]: /home/ubuntu/.go_workspace/src/github.com/grafana/metrictank/vendor/github.com/syndtr/goleveldb/leveldb/db.go:150 +0x70d
.......

Dieterbe added a commit that referenced this issue Dec 13, 2017
seems it has some useful fixes. see #782
@Dieterbe
Copy link
Contributor

this looks to be either caused by a memory corruption (do you use ECC? virtualization software?) or possibly a bug in some package using unsafe, like go-spew, or judging from the stacktrace , probably the goleveldb library. in #785 you'll find a new build using the latest version of the leveldb package which has received some commits which may or may not fix it. can you tell me about how often this issue occurs, does it seem related to system load, or any particular time of the day? can you try the build in #785 ?

@Dieterbe
Copy link
Contributor

Dieterbe commented Jan 5, 2018

have you seen this issue since upgrading to 0.7.4-542-g929ac1cf or higher?
if not, i will close.
if yes, please answer the questions in the above comment

@motyla
Copy link
Author

motyla commented Jan 5, 2018 via email

@Dieterbe Dieterbe closed this as completed Jan 8, 2018
Aergonus pushed a commit to bloomberg/metrictank that referenced this issue Jan 19, 2018
seems it has some useful fixes. see grafana#782
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants