Tag latest not found in repository docker.io/mesos/storm

[dinoba]

Hi everyone,

I want to run Storm on Mesos with Marathon and Docker with this JSON

> {
>   "id": "storm-nimbus",
>   "cmd": "./bin/run-with-marathon.sh",
>   "cpus": 1.0,
>   "mem": 1024,
>   "ports": [0, 1],
>   "instances": 1,
>   "container": {
>     "type": "DOCKER",
>     "docker": {
>       "image": "mesos/storm",
>       "network": "HOST",
>       "forcePullImage":true
>     }
>   },
>   "healthChecks": [
>     {
>       "protocol": "HTTP",
>       "portIndex": 0,
>       "path": "/",
>       "gracePeriodSeconds": 120,
>       "intervalSeconds": 20,
>       "maxConsecutiveFailures": 3
>     }
>   ],
>   "constraints": [["hostname", "CLUSTER", "192.168.1.37"]]
> }

And I get this message ...

Oct 25 17:17:07 Slave7 mesos-slave[2245]: E1025 17:17:07.720140 2265 slave.cpp:3976] Container '28a93411-6d15-47d1-b550-4fb0f73d0265' for executor 'storm-nimbus.1605ce93-9ac6-11e6-b4f4-080027f5b709' of framework 4277b841-4a4a-4a48-889f-aaa1e3e5730a-0001 failed to start: Failed to run 'docker -H unix:///var/run/docker.sock pull mesos/storm': exited with status 1; stderr='Tag latest not found in repository docker.io/mesos/storm

Any ideas?

[erikdw]

@dinoba : here's where we upload the Docker images after they are built:

• https://hub.docker.com/r/mesos/storm/tags/

Let's break down a version tag's 5 different dimensions; e.g., 0.1.7-0.9.6-0.27.2-jdk8-onbuild:

• 0.1.7: storm-mesos version (this project's direct version)
• 0.9.6: storm version (the version of storm this image was built for)
• 0.27.2: mesos version (the version of mesos this image was built for)
• jdk8: Java version (the version of Java used to build this image)
• onbuild: if present, it indicates that the image is meant for "onbuild" use where you'd vary the contents of the docker image, perhaps useful to specialize the storm.yaml that is embedded in the image.

So I'm not even sure what "latest" should mean. Which of the many combinations should we point at? (I'm also not sure how "latest" gets set by DockerHub.)

I am not familiar with Marathon, but I think the docker.image field can include the version also (the docs do not say this explicitly, I think it's an implicit behavior based on mimicking docker's style). So you might wanna do something like:

  ...
  "docker": {
    "image": "mesos/storm:0.1.7-0.9.6-0.27.2-jdk8",
  ...

[dinoba]

I was using this example from doc https://github.com/mesos/storm#running-storm-on-marathon

[erikdw]

@dinoba : sure. As I pointed out in my comment above, and in the linked issue [1], I don't really know anything about Marathon (yet). Did my suggestion work?

[1] Ah, I noticed that I failed to link the other issue where someone is asking about the Marathon usage: #173

[dinoba]

It worked, thank you.

Can author update this doc?

[erikdw]

@dinoba : as you noted in the other issue you filed, the docs for marathon need to be updated. But, as I've mentioned repeatedly, my team hasn't ever used marathon. So we don't yet have the experience and knowledge to update the docs. You in fact have better knowledge at this point -- it's a collaborative project, so PRs for fixing docs are welcome. However, as I noted here, my team does plan to spend some time figuring out how to get this project running under marathon, so we do plan to update the docs soon.

[dinoba]

This marathon task runs well

{
  "id": "storm-nimbus",
  "cmd": "./bin/run-with-marathon.sh",
  "cpus": 1.0,
  "mem": 1024,
  "ports": [0, 1],
  "instances": 1,
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "mesos/storm:0.2.0-1.0.2-1.0.1-jdk8",
      "network": "HOST",
      "forcePullImage":true
    }
  },
  "healthChecks": [
    {
      "protocol": "HTTP",
      "portIndex": 0,
      "path": "/",
      "gracePeriodSeconds": 120,
      "intervalSeconds": 20,
      "maxConsecutiveFailures": 3
    }
  ],
  "constraints": [["hostname", "CLUSTER", "192.168.1.39"]],
  "env": {
      "MESOS_MASTER_ZK": "192.168.1.21",
      STORM_NIMBUS_OPTS="-c storm.local.dir=/opt/storm -c topology.mesos.worker.cpu=1"     
  }
}

[erikdw]

@dinoba : good to know! So the difference is simply the container.docker.image value, right?

[dinoba]

Yes. Also there is way to set MESOS_MASTER_ZK and STORM_NIMBUS_OPTS directly from Marathon,

"env": {
      "MESOS_MASTER_ZK": "192.168.1.21",
      STORM_NIMBUS_OPTS="-c storm.local.dir=/opt/storm -c topology.mesos.worker.cpu=1"     
  }

[erikdw]

Ok, thanks @dinoba, that is useful info for us to record.