Error running java tasks on OS X

[conorgriffin]

Nomad version

Nomad v0.3.2

Operating system and Environment details

OSX Yosemite 10.10.5
nomad running in dev mode

Issue

The following error is produced:

2016/05/12 14:49:13 [ERR] client: failed to start task 'main' for alloc 'e43b1fb2-b244-2533-5bb4-4b5c142067a7': fork/exec ../../../../System/Library/Frameworks/JavaVM.framework/Versions/A/Commands/java: no such file or directory

However, /System/Library/Frameworks/JavaVM.framework/Versions/A/Commands/java does exist, ls output is below:

-rwxr-xr-x  1 root  wheel    49K Sep  9  2014 /System/Library/Frameworks/JavaVM.framework/Versions/A/Commands/java*

Here is my start command for the nomad instance: nohup ./nomad agent -dev >&logs/nomad.log

Reproduction steps

Run the job file below against the dev mode nomad instance on OS X with java 8 latest in /usr/bin

Nomad Client logs (if appropriate)

    No configuration files loaded
==> Starting Nomad agent...
==> Nomad agent configuration:

                 Atlas: <disabled>
                Client: true
             Log Level: DEBUG
                Region: global (DC: dc1)
                Server: true

==> Nomad agent started! Log data will stream in below:

    2016/05/12 14:49:08 [INFO] raft: Node at 127.0.0.1:4647 [Follower] entering Follower state
    2016/05/12 14:49:08 [INFO] serf: EventMemberJoin: C02R900ZG8WN.global 127.0.0.1
    2016/05/12 14:49:08 [INFO] nomad: starting 8 scheduling worker(s) for [service batch system _core]
    2016/05/12 14:49:08 [INFO] nomad: adding server C02R900ZG8WN.global (Addr: 127.0.0.1:4647) (DC: dc1)
    2016/05/12 14:49:08 [INFO] client: using state directory /tmp/NomadClient163449348
    2016/05/12 14:49:08 [INFO] client: using alloc directory /tmp/NomadClient210552723
    2016/05/12 14:49:08 [DEBUG] client: periodically fingerprinting cgroup at duration 15s
    2016/05/12 14:49:08 [DEBUG] client: periodically fingerprinting consul at duration 15s
    2016/05/12 14:49:09 [WARN] raft: Heartbeat timeout reached, starting election
    2016/05/12 14:49:09 [INFO] raft: Node at 127.0.0.1:4647 [Candidate] entering Candidate state
    2016/05/12 14:49:09 [DEBUG] raft: Votes needed: 1
    2016/05/12 14:49:09 [DEBUG] raft: Vote granted from 127.0.0.1:4647. Tally: 1
    2016/05/12 14:49:09 [INFO] raft: Election won. Tally: 1
    2016/05/12 14:49:09 [INFO] raft: Node at 127.0.0.1:4647 [Leader] entering Leader state
    2016/05/12 14:49:09 [INFO] raft: Disabling EnableSingleNode (bootstrap)
    2016/05/12 14:49:09 [DEBUG] raft: Node 127.0.0.1:4647 updated peer set (2): [127.0.0.1:4647]
    2016/05/12 14:49:09 [INFO] nomad: cluster leadership acquired
    2016/05/12 14:49:10 [DEBUG] fingerprint.env_aws: Error querying AWS Metadata URL, skipping
    2016/05/12 14:49:12 [WARN] fingerprint.env_gce: Could not read value for attribute "machine-type"
    2016/05/12 14:49:12 [DEBUG] fingerprint.env_gce: Error querying GCE Metadata URL, skipping
    2016/05/12 14:49:12 [DEBUG] fingerprint.network: Detected interface lo0  with IP 127.0.0.1 during fingerprinting
    2016/05/12 14:49:12 [WARN] fingerprint.network: Unable to read link speed from /sys/class/net/lo0/speed
    2016/05/12 14:49:12 [DEBUG] fingerprint.network: Unable to read link speed; setting to default 100
    2016/05/12 14:49:12 [DEBUG] client: applied fingerprints [arch cgroup cpu host memory network nomad storage]
    2016/05/12 14:49:12 [DEBUG] driver.docker: using client connection initialized from environment
    2016/05/12 14:49:12 [DEBUG] client: periodically fingerprinting docker at duration 15s
    2016/05/12 14:49:12 [DEBUG] client: periodically fingerprinting exec at duration 15s
    2016/05/12 14:49:12 [DEBUG] client: available drivers [raw_exec java]
    2016/05/12 14:49:12 [INFO] client: setting server address list: []
    2016/05/12 14:49:12 [DEBUG] client: updated allocations at index 1 (pulled 0) (filtered 0)
    2016/05/12 14:49:12 [DEBUG] client: node registration complete
    2016/05/12 14:49:12 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 0)
    2016/05/12 14:49:12 [DEBUG] client: periodically checking for node changes at duration 5s
    2016/05/12 14:49:12 [DEBUG] client: state updated to ready
    2016/05/12 14:49:13 [DEBUG] client: state changed, updating node.
    2016/05/12 14:49:13 [DEBUG] client: node registration complete
    2016/05/12 14:49:13 [DEBUG] http: Request /v1/jobs (751.619µs)
    2016/05/12 14:49:13 [DEBUG] worker: dequeued evaluation 1e408183-081b-8b86-9003-20cb3915dfbb
    2016/05/12 14:49:13 [DEBUG] sched: <Eval '1e408183-081b-8b86-9003-20cb3915dfbb' JobID: 'main'>: allocs: (place 1) (update 0) (migrate 0) (stop 0) (ignore 0)
    2016/05/12 14:49:13 [DEBUG] worker: submitted plan for evaluation 1e408183-081b-8b86-9003-20cb3915dfbb
    2016/05/12 14:49:13 [DEBUG] client: updated allocations at index 8 (pulled 1) (filtered 0)
    2016/05/12 14:49:13 [DEBUG] sched: <Eval '1e408183-081b-8b86-9003-20cb3915dfbb' JobID: 'main'>: setting status to complete
    2016/05/12 14:49:13 [DEBUG] client: allocs: (added 1) (removed 0) (updated 0) (ignore 0)
    2016/05/12 14:49:13 [DEBUG] worker: updated evaluation <Eval '1e408183-081b-8b86-9003-20cb3915dfbb' JobID: 'main'>
    2016/05/12 14:49:13 [DEBUG] worker: ack for evaluation 1e408183-081b-8b86-9003-20cb3915dfbb
    2016/05/12 14:49:13 [DEBUG] http: Request /v1/evaluation/1e408183-081b-8b86-9003-20cb3915dfbb (129.691µs)
    2016/05/12 14:49:13 [DEBUG] client: starting task runners for alloc 'e43b1fb2-b244-2533-5bb4-4b5c142067a7'
    2016/05/12 14:49:13 [DEBUG] client: starting task context for 'main' (alloc 'e43b1fb2-b244-2533-5bb4-4b5c142067a7')
    2016/05/12 14:49:13 [DEBUG] http: Request /v1/evaluation/1e408183-081b-8b86-9003-20cb3915dfbb/allocations (111.289µs)
    2016/05/12 14:49:13 [DEBUG] plugin: starting plugin: /Users/e048944/Developer/nomad/nomad []string{"/Users/e048944/Developer/nomad/nomad", "executor", "/tmp/NomadClient210552723/e43b1fb2-b244-2533-5bb4-4b5c142067a7/main/main-executor.out"}
    2016/05/12 14:49:13 [DEBUG] plugin: waiting for RPC address for: /Users/e048944/Developer/nomad/nomad
    2016/05/12 14:49:13 [DEBUG] plugin: nomad: 2016/05/12 14:49:13 [DEBUG] plugin: plugin address: unix /tmp/plugin821699640
    2016/05/12 14:49:13 [DEBUG] plugin: /Users/e048944/Developer/nomad/nomad: plugin process exited
    2016/05/12 14:49:13 [ERR] client: failed to start task 'main' for alloc 'e43b1fb2-b244-2533-5bb4-4b5c142067a7': fork/exec ../../../../System/Library/Frameworks/JavaVM.framework/Versions/A/Commands/java: no such file or directory
    2016/05/12 14:49:13 [INFO] client: Not restarting task: main for alloc: e43b1fb2-b244-2533-5bb4-4b5c142067a7
    2016/05/12 14:49:13 [DEBUG] client: updated allocations at index 10 (pulled 0) (filtered 1)
    2016/05/12 14:49:13 [DEBUG] client: allocs: (added 0) (removed 0) (updated 0) (ignore 1)
==> Caught signal: terminated
    2016/05/12 14:49:43 [DEBUG] http: Shutting down http server
    2016/05/12 14:49:43 [INFO] agent: requesting shutdown
    2016/05/12 14:49:43 [INFO] client: shutting down
    2016/05/12 14:49:43 [DEBUG] client: terminating runner for alloc 'e43b1fb2-b244-2533-5bb4-4b5c142067a7'
    2016/05/12 14:49:43 [INFO] nomad: shutting down server
    2016/05/12 14:49:43 [WARN] serf: Shutdown without a Leave
    2016/05/12 14:49:43 [INFO] client: shutting down consul sync
    2016/05/12 14:49:43 [INFO] agent: shutdown complete

I also obtained the following output with nomad fs cat from a more recent run with the same configuration

e048944@C02R900ZG8WN:~/Developer/nomad$ ./nomad fs cat 3c3a8b46 main/main-executor.out
2016/05/12 14:59:40 [DEBUG] executor: launching command /System/Library/Frameworks/JavaVM.framework/Versions/A/Commands/java -jar main.jar
2016/05/12 14:59:40 [DEBUG] executor: running command as nobody

Nomad Server logs (if appropriate)

==> Monitoring evaluation "1e408183"
    Evaluation triggered by job "main"
    Allocation "e43b1fb2" created: node "4a109636", group "conors-tasks"
    Evaluation status changed: "pending" -> "complete"
==> Evaluation "1e408183" finished with status "complete"

Job file (if appropriate)

job "main" {

    datacenters = ["dc1"]

    type = "service"

    update {
        stagger = "10s"
        max_parallel = 1
    }

    group "conors-tasks" {
        count = 1

        task "main" {
            driver = "java"

            config {
                jar_path = "main.jar"
            }

            logs {
                max_files = 5
                max_file_size = 10
            }

            artifact {
                source = "http://localhost:8080/main.jar"
            }

            resources {
                cpu = 500
                memory = 256
                network {
                    mbits = 10
                }
            }
        }
    }
}

[github-actions]

I'm going to lock this issue because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.