Update docker instructions #295

begriffs · 2017-02-22T18:43:19Z

Fixes #292 (I will subsequently open a pull request to update the Citus readme as well).

Tested the Docker installation instructions under Ubuntu 16.04 and CentOS 7.3.

begriffs · 2017-02-22T18:44:16Z

Here you go, @samay-sharma, give it a try.

begriffs · 2017-02-22T18:52:49Z

installation/single_machine_docker.rst

+
+    # Linux ONLY
+
+    curl -L https://github.com/docker/compose/releases/download/1.11.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose


We could alternately link to the instructions in https://github.com/docker/compose/releases/latest

This has the advantage of not hardcoding a docker-compose version, but the disadvantage of being harder to follow than our reproducing the command here.

It's super annoying that we have to ask users to add a repo to their box (to download Engine) only to turn around and ask them to curl some binary into their path.

docker/compose#2235 is a request to add docker-compose to the repository. I have no clue why this isn't already the case, but dropping a reference here in order to register my displeasure with upstream.

I spinned up a linux box and tried these steps. I ran into two separate issues.

(1) These steps assume that I'm running as root on my box. This isn't the case and the following instructions worked.

sudo curl -L https://github.com/docker/compose/releases/download/1.11.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose

(2) Even after this, when I ran docker-compose -p citus up -d, I saw the following error message.

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.26/containers/json: dial unix /var/run/docker.sock: connect: permission denied

This was because I didn't run the Linux postinstall steps in here: https://docs.docker.com/engine/installation/linux/linux-postinstall/

samay-sharma · 2017-02-23T02:08:03Z

installation/single_machine_docker.rst


 **2. Start the Citus Cluster**

 Citus uses docker-compose to run and connect containers holding the database master node, workers, and a persistent data volume. To create a local cluster download our docker-compose configuration file and run it

 .. code-block:: bash

-  wget https://raw.githubusercontent.com/citusdata/docker/master/docker-compose.yml
+  curl -L https://raw.githubusercontent.com/citusdata/docker/master/docker-compose.yml > docker-compose.yml
  docker-compose -p citus up -d


After installing docker, I had to start docker before I was able to do docker-compose. Could we add a step to the instructions which tells users to do that?

Come to think of it, when I tested on Ubuntu and CentOS the installer also started the daemon automatically. It's only the Mac installation that requires a start step. So I added a "Mac Only" step to the list.

Super minor unrelated point for the note below (line 41).

If you have PostgreSQL running on your machine -> If you already have PostgreSQL running on your machine

ozgune · 2017-02-23T22:08:05Z

installation/single_machine_docker.rst

+
+    open /Applications/Docker.app
+
+* (Linux only) On most platforms the previous step will install docker-compose as well as Docker, but on Linux you'll need to install docker-compose manually by downloading the binary and putting it in your path:


Small rewording.

On Linux, you need to manually install docker-compose (you can find the latest docker-compose version here). For this, you need to download the binary and put it in your path:

begriffs · 2017-02-24T00:29:26Z

installation/single_machine_docker.rst

+
+  curl -sSL https://get.docker.com/ | sudo bash
+  sudo usermod -aG docker $USER && exec su -l $USER
+  sudo systemctl start docker


In my CentOS test Docker did not start automatically after non-root installation. Adding systemctrl start fixed that, and also causes no harm in Ubuntu where the service was already started.

The systemctl works for Linux distros that use systemd, which I think is recent versions of all major distros. @jasonmp85 does this seem like a good bet to you?

ozgune

@begriffs -- I had three higher level questions.

When I look at the "Single-Machine Cluster" menu, it looks different and more involved than installation steps for other platforms. Could we make this uniform? (preferably keep Docker steps as they were before)?

https://citus-doc-preview-pr-295.herokuapp.com/installation/development.html

How confident are we that curl, usermod, systemctl are going to reliably work across different Linux distros? -- don't know, just asking
I remember @jasonmp85 saying that the Docker installation steps didn't work on Windows. Do we know whether Docker runs on Windows or not? (If it doesn't work on Windows, happy to just nix the windows part.)

ozgune · 2017-02-23T22:45:45Z

installation/single_machine_docker.rst

+
+    # Linux ONLY
+
+    curl -L https://github.com/docker/compose/releases/download/1.11.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose


I spinned up a linux box and tried these steps. I ran into two separate issues.

(1) These steps assume that I'm running as root on my box. This isn't the case and the following instructions worked.

sudo curl -L https://github.com/docker/compose/releases/download/1.11.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose sudo chmod +x /usr/local/bin/docker-compose

(2) Even after this, when I ran docker-compose -p citus up -d, I saw the following error message.

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.26/containers/json: dial unix /var/run/docker.sock: connect: permission denied

This was because I didn't run the Linux postinstall steps in here: https://docs.docker.com/engine/installation/linux/linux-postinstall/

ozgune · 2017-02-24T19:24:00Z

installation/single_machine_docker.rst

+.. code-block:: bash
+
+  curl -sSL https://get.docker.com/ | sudo bash
+  sudo usermod -aG docker $USER && exec su -l $USER


I spinned up a Redhat instance on AWS and got stock at the exec su -l $USER step.

This is because I have sudo on AWS instances but not su.

begriffs · 2017-02-28T18:47:41Z

Waiting to test this on a few EC2 machines until the S3 outage (!) abates.

begriffs · 2017-03-01T00:33:16Z

@ozgune these instructions worked for me on Red Hat, Ubuntu and Debian on AWS. I was logged in as ec2-user, ubuntu, and admin for each system respectively (not root). The only thing that went wrong is that Debian on AWS did not come pre-bundled with curl and I had to install that. However I'm pretty confident that curl is a common enough tool that our users will either have it or know how to get it.

jasonmp85 · 2017-03-01T00:36:41Z

The only thing that went wrong is that Debian on AWS did not come pre-bundled with curl and I had to install that.

This is also true of the Debian-based Docker images themselves. They're pretty barebones and don't include man or curl or which or many other common things. Weird that the AWS image itself doesn't have any curl, but not something I haven't come across frequently.

I don't think it's e.g. worth investigating wget's availability as an alternative, since if that's the sole place you ran into this we're probably safe, and it's probably a common enough annoyance that users of that platform know where and how to get curl. (i.e. the colloquial name for the idiom is curl | bash, not wget | bash).

begriffs · 2017-03-01T00:38:08Z

The debian one was actually a community AMI. (ami-0343ae47)

ozgune · 2017-03-01T00:46:22Z

@begriffs -- Sounds good, let's 🚀

I had two side notes. First, we'll need to resolve a minor conflict with the tutorial changes before merging this.

Second, I played around with a few AMIs and things looked good. On SUSE 11, I ran into the following issue. I think that's perfectly acceptable since Docker seems to only support SUSE 12+. The error message here was also pretty descriptive. JFYI:

ec2-user@ip-10-203-133-221:~> curl -sSL https://get.docker.com/ | sudo bash

  Either your platform is not easily detectable, is not supported by this
  installer script (yet - PRs welcome! [hack/install.sh]), or does not yet have
  a package for Docker.  Please visit the following URL for more detailed
  installation instructions:

    https://docs.docker.com/engine/installation/

Differs from mac enough now that I made subsections

CentOS did not auto-start it on install in my test

craigkerstiens temporarily deployed to citus-doc-preview-pr-295 February 22, 2017 18:43 Inactive

begriffs added needs review in progress and removed needs review labels Feb 22, 2017

craigkerstiens temporarily deployed to citus-doc-preview-pr-295 February 22, 2017 18:52 Inactive

begriffs commented Feb 22, 2017

View reviewed changes

samay-sharma reviewed Feb 23, 2017

View reviewed changes

craigkerstiens temporarily deployed to citus-doc-preview-pr-295 February 23, 2017 02:15 Inactive

ozgune reviewed Feb 23, 2017

View reviewed changes

craigkerstiens temporarily deployed to citus-doc-preview-pr-295 February 23, 2017 23:24 Inactive

begriffs force-pushed the new-docker branch from 778dca8 to 3af179d Compare February 23, 2017 23:26

craigkerstiens temporarily deployed to citus-doc-preview-pr-295 February 23, 2017 23:26 Inactive

craigkerstiens deployed to citus-doc-preview-pr-295 February 24, 2017 00:21 View deployment

begriffs commented Feb 24, 2017

View reviewed changes

ozgune suggested changes Feb 24, 2017

View reviewed changes

ozgune reviewed Feb 24, 2017

View reviewed changes

sumedhpathak assigned ozgune Feb 28, 2017

craigkerstiens had a problem deploying to citus-doc-preview-pr-295 February 28, 2017 23:35 Failure

begriffs added 6 commits February 28, 2017 16:50

Link to newest Docker installation method

6e54f9e

Use curl throughout for consistency

a6dc3b7

Make it more obvious that the docker-compose install is for linux only

c0cdc9e

Remember to start Docker on mac

b05eeb8

Linux instructions for non-root too

810ddd0

Differs from mac enough now that I made subsections

Make sure Docker is started on all linux distros

206235e

CentOS did not auto-start it on install in my test

begriffs added 2 commits February 28, 2017 16:54

Remove some words

0a08052

Use newgrp for platforms lacking su

4b2f329

begriffs force-pushed the new-docker branch from d4e7908 to 4b2f329 Compare March 1, 2017 00:55

begriffs merged commit be89430 into master Mar 1, 2017

begriffs removed the in progress label Mar 1, 2017

begriffs added a commit that referenced this pull request Mar 2, 2017

Update docker instructions (#295)

566ae14

ozgune mentioned this pull request Mar 3, 2017

Docker installation steps -- make headers uniform with other installation instructions #306

Closed

sumedhpathak deleted the new-docker branch July 19, 2017 18:17

naisila pushed a commit that referenced this pull request Jul 18, 2018

Update docker instructions (#295)

e6ba9de

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update docker instructions #295

Update docker instructions #295

begriffs commented Feb 22, 2017

begriffs commented Feb 22, 2017

begriffs Feb 22, 2017

jasonmp85 Feb 23, 2017

ozgune Feb 23, 2017

samay-sharma Feb 23, 2017

begriffs Feb 23, 2017

ozgune Feb 23, 2017

ozgune Feb 23, 2017

begriffs Feb 24, 2017

ozgune left a comment

ozgune Feb 23, 2017

ozgune Feb 24, 2017

begriffs commented Feb 28, 2017

begriffs commented Mar 1, 2017

jasonmp85 commented Mar 1, 2017

begriffs commented Mar 1, 2017

ozgune commented Mar 1, 2017


		# Linux ONLY

		curl -L https://github.com/docker/compose/releases/download/1.11.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose


		open /Applications/Docker.app

		* (Linux only) On most platforms the previous step will install docker-compose as well as Docker, but on Linux you'll need to install docker-compose manually by downloading the binary and putting it in your path:

Update docker instructions #295

Update docker instructions #295

Conversation

begriffs commented Feb 22, 2017

begriffs commented Feb 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ozgune left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

begriffs commented Feb 28, 2017

begriffs commented Mar 1, 2017

jasonmp85 commented Mar 1, 2017

begriffs commented Mar 1, 2017

ozgune commented Mar 1, 2017