"failed to configure task directory: No task directory exists" for exec driver

[sarahhodne]

I tried running a small test job with the exec driver, and got some errors when running it. The error in the agent logs is this one:

    2015/09/29 17:50:35 [ERR] client: failed to start task 'sleep' for alloc 'c0921b7a-44ef-38b0-7f24-285611ab30ab': failed to configure task directory: No task directory exists for sleep

Here's the full log output from sudo nomad agent -dev:

==> Starting Nomad agent...
2015/09/29 17:49:41 [ERR] fingerprint.env_aws: Error querying AWS Metadata URL, skipping
==> Nomad agent configuration:

                 Atlas: <disabled>
                Client: true
             Log Level: DEBUG
                Region: global (DC: dc1)
                Server: true

==> Nomad agent started! Log data will stream in below:

    2015/09/29 17:49:39 [INFO] serf: EventMemberJoin: nomad.global 127.0.0.1
    2015/09/29 17:49:39 [INFO] nomad: starting 4 scheduling worker(s) for [service batch _core]
    2015/09/29 17:49:39 [INFO] client: using alloc directory /tmp/NomadClient516107513
    2015/09/29 17:49:39 [INFO] raft: Node at 127.0.0.1:4647 [Follower] entering Follower state
    2015/09/29 17:49:39 [INFO] nomad: adding server nomad.global (Addr: 127.0.0.1:4647) (DC: dc1)
    2015/09/29 17:49:39 [WARN] fingerprint.network: Ethtool not found, checking /sys/net speed file
    2015/09/29 17:49:41 [WARN] raft: Heartbeat timeout reached, starting election
    2015/09/29 17:49:41 [INFO] raft: Node at 127.0.0.1:4647 [Candidate] entering Candidate state
    2015/09/29 17:49:41 [DEBUG] raft: Votes needed: 1
    2015/09/29 17:49:41 [DEBUG] raft: Vote granted. Tally: 1
    2015/09/29 17:49:41 [INFO] raft: Election won. Tally: 1
    2015/09/29 17:49:41 [INFO] raft: Node at 127.0.0.1:4647 [Leader] entering Leader state
    2015/09/29 17:49:41 [INFO] raft: Disabling EnableSingleNode (bootstrap)
    2015/09/29 17:49:41 [DEBUG] raft: Node 127.0.0.1:4647 updated peer set (2): [127.0.0.1:4647]
    2015/09/29 17:49:41 [INFO] nomad: cluster leadership acquired
    2015/09/29 17:49:41 [DEBUG] client: applied fingerprints [arch cpu host memory storage network]
    2015/09/29 17:49:41 [DEBUG] client: available drivers [exec docker]
    2015/09/29 17:49:41 [DEBUG] client: node registration complete
    2015/09/29 17:49:41 [DEBUG] client: updated allocations at index 1 (0 allocs)
    2015/09/29 17:49:41 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 0)
    2015/09/29 17:49:41 [DEBUG] client: state updated to ready
    2015/09/29 17:50:35 [DEBUG] http: Request /v1/jobs (2.696931ms)    2015/09/29 17:50:35 [DEBUG] worker: dequeued evaluation 005c9080-f0f7-52c6-f4ab-5a3350ec97bc
    2015/09/29 17:50:35 [DEBUG] sched: <Eval '005c9080-f0f7-52c6-f4ab-5a3350ec97bc' JobID: 'test-exec'>: allocs: (place 1) (update 0) (migrate 0) (stop 0) (ignore 0)
    2015/09/29 17:50:35 [DEBUG] client: updated allocations at index 7 (1 allocs)
    2015/09/29 17:50:35 [DEBUG] client: allocs: (added 1) (removed 0) (updated 0) (ignore 0)
    2015/09/29 17:50:35 [DEBUG] worker: submitted plan for evaluation 005c9080-f0f7-52c6-f4ab-5a3350ec97bc
    2015/09/29 17:50:35 [DEBUG] sched: <Eval '005c9080-f0f7-52c6-f4ab-5a3350ec97bc' JobID: 'test-exec'>: setting status to complete
    2015/09/29 17:50:35 [DEBUG] client: starting runner for alloc 'c0921b7a-44ef-38b0-7f24-285611ab30ab'
    2015/09/29 17:50:35 [DEBUG] client: starting task context for 'sleep' (alloc 'c0921b7a-44ef-38b0-7f24-285611ab30ab')
    2015/09/29 17:50:35 [ERR] client: failed to start task 'sleep' for alloc 'c0921b7a-44ef-38b0-7f24-285611ab30ab': failed to configure task directory: No task directory exists for sleep
    2015/09/29 17:50:35 [DEBUG] worker: updated evaluation <Eval '005c9080-f0f7-52c6-f4ab-5a3350ec97bc' JobID: 'test-exec'>
    2015/09/29 17:50:35 [DEBUG] worker: ack for evaluation 005c9080-f0f7-52c6-f4ab-5a3350ec97bc
    2015/09/29 17:50:35 [DEBUG] client: updated allocations at index 9 (1 allocs)
    2015/09/29 17:50:35 [DEBUG] client: allocs: (added 0) (removed 0) (updated 1) (ignore 0)

Here's the job file:

job "test-exec" {
    datacenters = ["dc1"]

    type = "service"

    constraint {
        attribute = "$attr.kernel.name"
        value = "linux"
    }

    update {
        stagger = "10s"
        max_parallel = 1
    }

    group "sleep" {
        count = 1

        task "sleep" {
            driver = "exec"

            config {
                command = "/bin/bash"
                args = "-c \"sleep 1; echo -n win > $NOMAD_ALLOC_DIR/output.txt\""
            }

            resources {
                cpu = 500 # 500 Mhz
                memory = 256 # 256MB
                network {
                    mbits = 10
                }
            }
        }
    }
}

This is running Nomad v0.1.0 using this Vagrantfile on Virtualbox/OS X.

[justenwalker]

👍 - I'm also experiencing this on dedicated linux machines (1 server, 1 client)

• Job Spec
• nomad run sleep.nomad => output

[dadgar]

Thanks fixed in d377731

You will want to vagrant destroy and then vagrant up

[jmitchell]

I saw this behavior with my own project in v0.1.1.

I tried to replicate it by building master in the standard Vagrant environment, but was blocked because master is broken (see issue #220 and pull request #222).

Anyone else seeing this?

[dadgar]

@jmitchell: we are aware of this. Trying to get a v0.1.2 release out to fix it. It is a cross-compiling issue.

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.