Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fabio 1.5.12 - panic: runtime error: invalid memory address or nil pointer dereference #719

Closed
evandam opened this issue Nov 14, 2019 · 5 comments
Milestone

Comments

@evandam
Copy link

evandam commented Nov 14, 2019

After upgrading from Fabio 1.5.11, Fabio is failing to start with the following logs:

Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Version 1.5.12 starting
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Go runtime is go1.13.1
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Sending metrics to StatsD on 127.0.0.1:18125 as "fabio"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Sending metrics to StatsD on 127.0.0.1:18125 as "fabio"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Setting GOGC=800
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Setting GOMAXPROCS=1
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] consul: Connecting to "localhost:8500" in datacenter "kitchen"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Admin server access mode "ro"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Admin server listening on ":9998"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] Waiting for first routing table
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] consul: Using dynamic routes
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] consul: Using tag prefix "fabio-"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] consul: Watching KV path "/fabio/kitchen"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: 2019/11/14 22:55:19 [INFO] consul: Watching KV path "/fabio/noroute.html"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: panic: runtime error: invalid memory address or nil pointer dereference
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x4783c6]
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: goroutine 33 [running]:
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: bytes.(*Buffer).Write(0x0, 0xc0001cc000, 0xa, 0x1500, 0x1, 0x1, 0xc0002ded20)
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: #011/usr/local/Cellar/go/1.13.1/libexec/src/bytes/buffer.go:169 +0x26
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: fmt.Fprintf(0xe6a820, 0x0, 0xc5c5a4, 0xc, 0xc000499b10, 0x1, 0x1, 0x1, 0x133fa10, 0x134c040)
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: #011/usr/local/Cellar/go/1.13.1/libexec/src/fmt/print.go:205 +0xa5
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: github.com/fabiolb/fabio/route.Table.Dump(0xc00024dbc0, 0x34, 0xc000250960)
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: #011/Users/frank/src/github.com/fabiolb/fabio/route/table.go:517 +0x849
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: main.logRoutes(0xc00024dbc0, 0x0, 0x0, 0x0, 0x0, 0xc000116340, 0x6)
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: #011/Users/frank/src/github.com/fabiolb/fabio/main.go:531 +0x193
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: main.watchBackend(0xc000106600, 0xc00007cfc0)
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: #011/Users/frank/src/github.com/fabiolb/fabio/main.go:485 +0x5b6
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: created by main.main
Nov 14 22:55:19 default-ubuntu-1804 fabio[5668]: #011/Users/frank/src/github.com/fabiolb/fabio/main.go:136 +0x698
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]: 2019/11/14 22:55:19 [INFO] Setting log level to INFO
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]: 2019/11/14 22:55:19 [INFO] Runtime config
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]: {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "Proxy": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Strategy": "rnd",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Matcher": "prefix",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "NoRouteStatus": 404,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "MaxConn": 10000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "ShutdownWait": 3000000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "DialTimeout": 3000000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "ResponseHeaderTimeout": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "KeepAliveTimeout": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "FlushInterval": 1000000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "GlobalFlushInterval": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "LocalIP": "10.0.2.15",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "ClientIPHeader": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "TLSHeader": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "TLSHeaderValue": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "GZIPContentTypes": null,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "RequestID": "X-Fabio-Id",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "STSHeader": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "MaxAge": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Subdomains": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Preload": false
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "AuthSchemes": {}
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "Registry": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Backend": "consul",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Static": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "NoRouteHTML": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Routes": ""
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "File": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "NoRouteHTMLPath": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "RoutesPath": ""
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Consul": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Addr": "localhost:8500",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Scheme": "http",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Token": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "KVPath": "/fabio/kitchen",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "NoRouteHTMLPath": "/fabio/noroute.html",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "TagPrefix": "fabio-",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Register": true,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ServiceAddr": ":9998",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ServiceName": "fabio",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ServiceTags": [
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "kitchen"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             ],
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ServiceStatus": [
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "passing"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             ],
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CheckInterval": 1000000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CheckTimeout": 3000000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CheckScheme": "http",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CheckTLSSkipVerify": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CheckDeregisterCriticalServiceAfter": "90m",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ChecksRequired": "all",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ServiceMonitors": 1,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "TLS": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "KeyFile": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "CertFile": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "CAFile": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "CAPath": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "InsecureSkipVerify": false
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             }
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Custom": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Host": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Path": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "QueryParams": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Scheme": "https",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CheckTLSSkipVerify": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "PollingInterval": 5,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "NoRouteHTML": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Timeout": 10
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Timeout": 3000000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Retry": 500000000
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "Listen": [
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Addr": ":9999",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Proto": "http",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ReadTimeout": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "WriteTimeout": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CertSource": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "Name": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "Type": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "CertPath": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "KeyPath": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "ClientCAPath": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "CAUpgradeCN": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "Refresh": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "Header": null,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "VaultFetchToken": ""
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "StrictMatch": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "TLSMinVersion": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "TLSMaxVersion": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "TLSCiphers": null,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ProxyProto": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ProxyHeaderTimeout": 0
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         }
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     ],
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "Log": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "AccessFormat": "$remote_host - $header.X-Forwarded-For - [$time_common] \"$request\" $response_status $response_body_size \"$header.Referer\" \"$header.User-Agent\" upstream \"$upstream_addr\"",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "AccessTarget": "stdout",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "RoutesFormat": "detail",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Level": "INFO"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "Metrics": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Target": "statsd",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Prefix": "{{clean .Exec}}",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Names": "{{clean .Host}}.{{clean .Path}}",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Interval": 10000000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Timeout": 10000000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Retry": 500000000,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "GraphiteAddr": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "StatsDAddr": "127.0.0.1:18125",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Circonus": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "APIKey": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "APIApp": "fabio",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "APIURL": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CheckID": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "BrokerID": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "SubmissionURL": ""
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         }
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "UI": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Listen": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Addr": ":9998",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "Proto": "http",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ReadTimeout": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "WriteTimeout": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "CertSource": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "Name": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "Type": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "CertPath": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "KeyPath": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "ClientCAPath": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "CAUpgradeCN": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "Refresh": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "Header": null,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:                 "VaultFetchToken": ""
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "StrictMatch": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "TLSMinVersion": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "TLSMaxVersion": 0,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "TLSCiphers": null,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ProxyProto": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:             "ProxyHeaderTimeout": 0
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Color": "light-green",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Title": "kitchen ",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Access": "ro"
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "Runtime": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "GOGC": 800,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "GOMAXPROCS": 1
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "Tracing": {
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "TracingEnabled": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "CollectorType": "http",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "ConnectString": "http://localhost:9411/api/v1/spans",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "ServiceName": "Fabiolb",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "Topic": "Fabiolb-Kafka-Topic",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "SamplerRate": -1,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "SpanHost": "localhost:9998",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:         "TraceID128Bit": true
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     },
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "ProfileMode": "",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "ProfilePath": "/tmp",
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "Insecure": false,
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]:     "GlobMatchingDisabled": false
Nov 14 22:55:19 default-ubuntu-1804 fabio[5694]: }
@pschultz
Copy link
Member

pschultz commented Nov 18, 2019

You can work around the issue by setting log.routes.format to something other than "detail", i.e. "delta" or "all".

@scalp42
Copy link

scalp42 commented Nov 18, 2019

seeing the same issue here with 1.5.12

@evandam
Copy link
Author

evandam commented Nov 18, 2019

Thanks @pschultz! When is the next release scheduled that this will be included in?

@magiconair
Copy link
Contributor

I’ll try to roll one this week

@magiconair magiconair added this to the 1.5.13 milestone Nov 18, 2019
@dcarbone
Copy link
Contributor

Thanks for getting this released so quick!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants