Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic tailing logs #3115

Closed
pims opened this issue Aug 29, 2017 · 2 comments · Fixed by #3116
Closed

Panic tailing logs #3115

pims opened this issue Aug 29, 2017 · 2 comments · Fixed by #3116

Comments

@pims
Copy link

pims commented Aug 29, 2017

Nomad version

Nomad v0.6.1

Operating system and Environment details

$ uname -a
Linux nomad-test 4.4.0-87-generic #110-Ubuntu SMP Tue Jul 18 12:55:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Issue

panic: assignment to entry in nil map

goroutine 1 [running]:
github.com/hashicorp/nomad/api.(*AllocFS).Logs(0xc4201f9a28, 0xc4203fc5a0, 0xc42042a100, 0x7ffdd220289b, 0x2, 0x1464aee, 0x6, 0x1463237, 0x5, 0x0, ...)
	/opt/gopath/src/github.com/hashicorp/nomad/api/fs.go:201 +0x12f
github.com/hashicorp/nomad/command.(*LogsCommand).followFile(0xc4203fa100, 0xc420408300, 0xc4203fc5a0, 0x0, 0x7ffdd220289b, 0x2, 0x1464aee, 0x6, 0x1463237, 0x5, ...)
	/opt/gopath/src/github.com/hashicorp/nomad/command/logs.go:264 +0x104
github.com/hashicorp/nomad/command.(*LogsCommand).Run(0xc4203fa100, 0xc4200101e0, 0x2, 0x2, 0x0)
	/opt/gopath/src/github.com/hashicorp/nomad/command/logs.go:217 +0x1075
github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42014de60, 0xc42014de60, 0x2a, 0x14c8140)
	/opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:242 +0x229
main.RunCustom(0xc4200101d0, 0x3, 0x3, 0xc42038bb90, 0x0)
	/opt/gopath/src/github.com/hashicorp/nomad/main.go:54 +0xf2d
main.Run(0xc4200101d0, 0x3, 0x3, 0xc4200001a0)
	/opt/gopath/src/github.com/hashicorp/nomad/main.go:24 +0x56
main.main()
	/opt/gopath/src/github.com/hashicorp/nomad/main.go:20 +0x64

Reproduction steps

$ nomad run metabase.nomad
$ nomad status metabase
ID            = metabase
Name          = metabase
Submit Date   = 08/29/17 04:58:42 UTC
Type          = service
Priority      = 50
Datacenters   = dc1
Status        = running
Periodic      = false
Parameterized = false

Summary
Task Group  Queued  Starting  Running  Failed  Complete  Lost
metabase    0       1         0        0       0         0

Allocations
ID        Node ID   Task Group  Version  Desired  Status   Created At
12a23d76  cf71d091  metabase    0        run      pending  08/29/17 04:58:42 UTC

$ nomad logs 12a23d76 -f

panic: assignment to entry in nil map

goroutine 1 [running]:
github.com/hashicorp/nomad/api.(*AllocFS).Logs(0xc4201f9a28, 0xc4203fc5a0, 0xc42042a100, 0x7ffdd220289b, 0x2, 0x1464aee, 0x6, 0x1463237, 0x5, 0x0, ...)
	/opt/gopath/src/github.com/hashicorp/nomad/api/fs.go:201 +0x12f
github.com/hashicorp/nomad/command.(*LogsCommand).followFile(0xc4203fa100, 0xc420408300, 0xc4203fc5a0, 0x0, 0x7ffdd220289b, 0x2, 0x1464aee, 0x6, 0x1463237, 0x5, ...)
	/opt/gopath/src/github.com/hashicorp/nomad/command/logs.go:264 +0x104
github.com/hashicorp/nomad/command.(*LogsCommand).Run(0xc4203fa100, 0xc4200101e0, 0x2, 0x2, 0x0)
	/opt/gopath/src/github.com/hashicorp/nomad/command/logs.go:217 +0x1075
github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli.(*CLI).Run(0xc42014de60, 0xc42014de60, 0x2a, 0x14c8140)
	/opt/gopath/src/github.com/hashicorp/nomad/vendor/github.com/mitchellh/cli/cli.go:242 +0x229
main.RunCustom(0xc4200101d0, 0x3, 0x3, 0xc42038bb90, 0x0)
	/opt/gopath/src/github.com/hashicorp/nomad/main.go:54 +0xf2d
main.Run(0xc4200101d0, 0x3, 0x3, 0xc4200001a0)
	/opt/gopath/src/github.com/hashicorp/nomad/main.go:24 +0x56
main.main()
	/opt/gopath/src/github.com/hashicorp/nomad/main.go:20 +0x64

Nomad Server logs (if appropriate)

2017/08/29 05:08:32.079072 [DEBUG] driver.docker: docker pull metabase/metabase:latest succeeded
    2017/08/29 05:08:32.099007 [DEBUG] driver.docker: Setting default logging options to syslog and unix:///tmp/plugin373405656
    2017/08/29 05:08:32.099328 [DEBUG] driver.docker: Using config for logging: {Type:syslog ConfigRaw:[] Config:map[syslog-address:unix:///tmp/plugin373405656]}
    2017/08/29 05:08:32.099455 [DEBUG] driver.docker: using 10485760 bytes memory for server
    2017/08/29 05:08:32.099550 [DEBUG] driver.docker: using 100 cpu shares for server
    2017/08/29 05:08:32.099680 [DEBUG] driver.docker: binding directories []string{"/tmp/NomadClient898796928/12a23d76-0250-fdee-9e2b-37d20e576e58/alloc:/alloc", "/tmp/NomadClient898796928/12a23d76-0250-fdee-9e2b-37d20e576e58/server/local:/local", "/tmp/NomadClient898796928/12a23d76-0250-fdee-9e2b-37d20e576e58/server/secrets:/secrets"} for server
    2017/08/29 05:08:32.099799 [DEBUG] driver.docker: networking mode not specified; defaulting to bridge
    2017/08/29 05:08:32.099866 [DEBUG] driver.docker: No network interfaces are available
    2017/08/29 05:08:32.099945 [DEBUG] driver.docker: applied labels on the container: map[group:server]
    2017/08/29 05:08:32.100022 [DEBUG] driver.docker: setting container name to: server-12a23d76-0250-fdee-9e2b-37d20e576e58
    2017/08/29 05:08:32.120281 [INFO] driver.docker: created container 581d04c7540e8b2f468867d78e44e743be82c1c9d79f395438024bdb7a94460f
    2017/08/29 05:08:32.294352 [INFO] driver.docker: started container 581d04c7540e8b2f468867d78e44e743be82c1c9d79f395438024bdb7a94460f
    2017/08/29 05:08:32.305708 [WARN] client: error fetching stats of task server: stats collection hasn't started yet
    2017/08/29 05:08:32.392254 [DEBUG] driver.docker: error collecting stats from container 581d04c7540e8b2f468867d78e44e743be82c1c9d79f395438024bdb7a94460f: io: read/write on closed pipe
    2017/08/29 05:08:32.398763 [INFO] client: task "server" for alloc "12a23d76-0250-fdee-9e2b-37d20e576e58" failed: Wait returned exit code 137, signal 0, and error Docker container exited with non-zero exit code: 137
    2017/08/29 05:08:32.398788 [INFO] client: Restarting task "server" for alloc "12a23d76-0250-fdee-9e2b-37d20e576e58" in 15.884717748s

Nomad Client logs (if appropriate)

Job file (if appropriate)

job "metabase" {

    datacenters = ["dc1"]
    type = "service"

    group "metabase" {

        count = 1
        task "server" {
            driver = "docker"
            config {
                image = "metabase/metabase:latest"
                args = [
                    "-bind", "${NOMAD_PORT_http}"
                    ]

                labels {
                    group = "server"
                }
            }
            resources {
                # cpu = 500 # Mhz
                memory = 10 # MB
            }
            env {}
        }
    }
}

Notes

  • Running nomad in nomad agent -dev mode.
dadgar added a commit that referenced this issue Aug 29, 2017
This PR fixes an assignment to a nil map in the api pkg related to
accessing logs and the filesystem of allocations.

Fixes #3115
@pims
Copy link
Author

pims commented Aug 29, 2017

That was fast @dadgar !

@github-actions
Copy link

github-actions bot commented Dec 8, 2022

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 8, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant