Add production Alpine and Debian Stretch Docker images #225

kaiyou · 2018-01-21T12:45:07Z

This PR is currently for reference mostly, as it will conflict with #213 anyway. It implements a production Alpine-based image with the following concepts in mind:

the image is built from source (see Add config for running with Docker Compose and on Docker Swarm #213 (comment) for the discussion)
the image only contains PeerTube and will run a single process, no caching webserver or frontend
configuration is committed to support/docker/config to avoid duplicating a common configuration file
as few layers as possible while maintaining readability

Uses an existing traefik server as a https reverse proxy.

(The base image already contains yarn.)

djmaze · 2018-01-21T13:04:30Z

support/docker/config/production.yaml

+
+# From the project root directory
+storage:
+  avatars: '../data/avatars/'


So this is /usr/src/data? You need to adjust the VOLUME directive in the Dockerfile accordingly. (Yes, I made a similar mistake in my PR ;))

This is relative to /app, so it is actually /data/avatars.

Ah, sorry for the noise.

djmaze · 2018-01-21T13:26:09Z

I like this. Maybe we should go with this PR for now and try to get automated builds working (probably via Travis?). We can add sample Compose / Swarm configs later.

Dryusdan · 2018-01-21T13:41:43Z

You haven't any problem with npm build ?

There are too many layer but it's cool :D

kaiyou · 2018-01-21T13:43:27Z

@Dryusdan npm build uses scripty, which fails with the default Alpine shell, hence the bash -c call. Also, I am open to suggestions as to limiting the number of layers without impacting readability.

Dryusdan · 2018-01-21T13:48:31Z

@kaiyou yeah ! You unlock me o/

Actually I make this image (github mirror repo) : https://github.com/Dryusdan/docker-peertube/blob/master/Dockerfile.alpine

Dryusdan · 2018-01-21T13:49:36Z

So you can inspired if you would :)

djmaze · 2018-01-21T13:52:06Z

I have to say I really don't get the obsession of people wanting to minimize the amount of layers at all costs. With today's docker graph drivers, I believe the overhead is minimal.

I'd rather optimize for better layer caching, which allows faster updates in production. (See my PR.)

Dryusdan · 2018-01-21T13:59:05Z

If you build your image on the same host, minimal layer is usefull...
In CI build docker in docker, the multiple layer takes longer to build than little layer because cache doesn't exist for this method

djmaze · 2018-01-21T14:36:14Z

@Dryusdan

If you build your image on the same host, minimal layer is usefull...

I suppose you mean "minimal number of layers". So why should it be better when building on the same host? A multi-layer approach will probably be much faster since you can skip building many of the base layers if only the application code was changed.

In CI build docker in docker, the multiple layer takes longer to build than little layer because cache doesn't exist for this method

Right, with Travis there is no Docker caching currently, although there is an improvement with --cache-from.

But does it really matter if a CI build takes 5 or 7 minutes? This is not something you impatiently wait for, do you?

Dryusdan · 2018-01-21T15:05:49Z

Use cache is can create some problem with version of paquet, because cache "create" outdated packet (ouch, bad english). The base image first.
Then, in production use image not build image not create or build by trust-worthy machine is not very secure

Fr translation : Le cache Docker est un soucis plus qu'un avantage en production, il utilise des images pas forcément à jour, et des paquets pareil. Cela pose un soucis de sécurité.
De plus, faire confiance à une machine tierce pour le build d'une image destiné à la production n'est pas forcément une bonne chose, surtout quand l'application tourne sur du Docker qui a les droits root.

Chocobozzz · 2018-01-22T15:35:38Z

I like this. Maybe we should go with this PR for now and try to get automated builds working (probably via Travis?). We can add sample Compose / Swarm configs later.

Maybe @kaiyou @Dryusdan @djmaze you could merge your different work in one repository, and make only one PR? For example this alpine image with @djmaze docker compose or whatever.

kaiyou · 2018-01-22T15:46:09Z

I'll try and merge both tonight. I'll update if I cannot deliver in time.

Dryusdan · 2018-01-23T08:58:24Z

My work is really different and I don't think it's that well accepted. ^^

kaiyou · 2018-01-23T12:24:40Z

I have just merged @djmaze commits into the directory structure of this PR. I have not tested yet, so it probably doesn't event build.

…ich are not strictly required

kaiyou · 2018-01-23T19:22:27Z

I just tested, it seems both images build successfully. Now some live testing :)

kaiyou · 2018-01-26T07:27:01Z

Quick update: the Debian image (stretch based) works fine and has been running in production for two days now. The Alpine image still segfaults and I need time or help to fix it (I have been stracing inside the containers for hours now, no success).

I suggest we merge but only configure builds for the Debian image at this point.

Chocobozzz · 2018-01-26T08:44:59Z

@kaiyou Please put your files inside a /support/docker/production directory and I'll merge. Thanks for your work! :)

djmaze

Apart from some minor typos, this LGTM! 👍

djmaze · 2018-01-26T14:12:21Z

support/docker/Dockerfile.alpine

@@ -0,0 +1,21 @@
+FROM node:8-alpine
+
+# Install dpeendencies


djmaze · 2018-01-26T14:12:39Z

support/docker/Dockerfile.stretch

@@ -0,0 +1,25 @@
+FROM node:8-stretch
+
+# Install dpeendencies


djmaze · 2018-01-26T14:12:59Z

support/docker/Dockerfile.stretch

+COPY . ./
+RUN yarn install --pure-lockfile && npm run build
+
+# Configuration the application


Configuring?

… develop

kaiyou · 2018-01-28T00:00:47Z

@Chocobozzz I did, thank you @djmaze for the review, I fixed the typos, merged the latest develop, added redis settings accordingly.

Chocobozzz · 2018-01-29T07:52:07Z

Thanks @kaiyou for your awesome work!

1000i100 · 2018-03-11T13:24:02Z

I've just test.
This case :

# docker-compose up app
ERROR: No such service: app

This other case :

 # docker-compose up
Starting peertube_db_1 ... done
Starting peertube_peertube_1 ... done
Attaching to peertube_db_1, peertube_peertube_1
db_1        | 2018-03-11 13:19:57.272 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1        | 2018-03-11 13:19:57.272 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db_1        | 2018-03-11 13:19:57.338 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1        | 2018-03-11 13:19:57.387 UTC [22] LOG:  database system was shut down at 2018-03-11 13:18:36 UTC
db_1        | 2018-03-11 13:19:57.401 UTC [1] LOG:  database system is ready to accept connections
peertube_1  | npm ERR! path /app/package.json
peertube_1  | npm ERR! code EACCES
peertube_1  | npm ERR! errno -13
peertube_1  | npm ERR! syscall open
peertube_1  | npm ERR! Error: EACCES: permission denied, open '/app/package.json'
peertube_1  | npm ERR!  { Error: EACCES: permission denied, open '/app/package.json'
peertube_1  | npm ERR!   stack: 'Error: EACCES: permission denied, open \'/app/package.json\'',
peertube_1  | npm ERR!   errno: -13,
peertube_1  | npm ERR!   code: 'EACCES',
peertube_1  | npm ERR!   syscall: 'open',
peertube_1  | npm ERR!   path: '/app/package.json' }
peertube_1  | npm ERR!
peertube_1  | npm ERR! Please try running this command again as root/Administrator.
peertube_1  |
peertube_1  | npm ERR! A complete log of this run can be found in:
peertube_1  | npm ERR!     /data/.npm/_logs/2018-03-11T13_19_58_817Z-debug.log
peertube_peertube_1 exited with code 243

PS : for succefull docker build, i've changed Dockerfile.stretch in Dockerfile and the COPY line in :

COPY src ./

I've git cloned in src peertube for package.json and other yarn.lock dependency index availability in building process.

kaiyou · 2018-03-11T13:29:07Z

As-tu essayé de build depuis la racine du projet ?

docker build -t peertube:stretch -f support/docker/production/Dockerfile.stretch .

1000i100 · 2018-03-11T13:37:54Z

nop, i will try that now.

kaiyou · 2018-03-11T13:43:28Z

In english for the community to be able to read: the docker image seriously lacks documentation. I'll try to remedy that.

1000i100 · 2018-03-11T13:52:20Z

AS: I've just edited my messages to translate them in english.

docker build -t peertube:stretch -f support/docker/production/Dockerfile.stretch .

works, but it dosent change docker-compose behavior.

kaiyou · 2018-03-11T15:43:26Z

I have just updated the compose file in PR #332. Maybe its best we follow up there and make sure the resulting compose example is working properly

djmaze and others added 7 commits January 18, 2018 00:04

First pass at a (swarm-compatible) docker image

cc075c5

Uses an existing traefik server as a https reverse proxy.

Add example config for a Docker swarm deployment

ff4975c

Point to traefik config for docker compose setup

b7c53df

Clarify that traefik is needed for the example config

fcf55bb

Use node:8-stretch base image and don't install yarn

5e7784e

(The base image already contains yarn.)

Initial commit for an Alpine Docker image

72dc0ad

Fix docker volume path

712add1

djmaze reviewed Jan 21, 2018

View reviewed changes

kaiyou added 3 commits January 23, 2018 11:09

Merge remote-tracking branch 'upstream/develop' into develop

1618207

Merge remote-tracking branch 'djmaze/docker' into develop

755b5de

Merge Chocobozzz#213 and Chocobozzz#225 and move files around

e7f0065

kaiyou added 6 commits January 23, 2018 17:00

Remove unnecessary dependencies from the alpine build

b219b7c

Update Dockerfiles to match install path, config path, etc.

5398e8a

Update the configuration in the example compose file

6c5c511

Update the configuration in the example swarm file

b614f2c

Remove the declared networks and volumes from the compose example, wh…

6c98ebe

…ich are not strictly required

Update attachment path in the documentation

4535a1d

Display traefik as a suggestion and not a required dependency

6b75103

Chocobozzz mentioned this pull request Jan 23, 2018

Add config for running with Docker Compose and on Docker Swarm #213

Closed

djmaze approved these changes Jan 26, 2018

View reviewed changes

kaiyou added 7 commits January 26, 2018 23:48

Update the Docker ignored files

a56851b

Fix typos reported in Chocobozzz#225

988c505

Move production Dockerfiles to a production directory

e7edbe7

Merge branch 'develop' of https://github.com/Chocobozzz/PeerTube into…

c7a642b

… develop

Add the redis configuration settings

987515f

Add Docker files to the dockerignore

cf99dd8

Make the signup limit configurable

6b62b83

kaiyou changed the title ~~Add a production Alpine Linux image~~ Add production Alpine and Debian Stretch Docker images Jan 28, 2018

Chocobozzz merged commit e8395f0 into Chocobozzz:develop Jan 29, 2018

ar9708 added a commit to OwnTube-tv/peertube-runner that referenced this pull request Mar 16, 2024

Add Docker baseline from Chocobozzz/PeerTube#225

b682956

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add production Alpine and Debian Stretch Docker images #225

Add production Alpine and Debian Stretch Docker images #225

kaiyou commented Jan 21, 2018

djmaze Jan 21, 2018

kaiyou Jan 21, 2018

djmaze Jan 21, 2018

djmaze commented Jan 21, 2018

Dryusdan commented Jan 21, 2018

kaiyou commented Jan 21, 2018

Dryusdan commented Jan 21, 2018

Dryusdan commented Jan 21, 2018

djmaze commented Jan 21, 2018

Dryusdan commented Jan 21, 2018

djmaze commented Jan 21, 2018

Dryusdan commented Jan 21, 2018 •

edited

Loading

Chocobozzz commented Jan 22, 2018

kaiyou commented Jan 22, 2018

Dryusdan commented Jan 23, 2018

kaiyou commented Jan 23, 2018 •

edited

Loading

kaiyou commented Jan 23, 2018 •

edited

Loading

kaiyou commented Jan 26, 2018 •

edited

Loading

Chocobozzz commented Jan 26, 2018

djmaze left a comment

djmaze Jan 26, 2018

djmaze Jan 26, 2018

djmaze Jan 26, 2018

kaiyou commented Jan 28, 2018

Chocobozzz commented Jan 29, 2018

1000i100 commented Mar 11, 2018 •

edited

Loading

kaiyou commented Mar 11, 2018

1000i100 commented Mar 11, 2018 •

edited

Loading

kaiyou commented Mar 11, 2018

1000i100 commented Mar 11, 2018 •

edited

Loading

kaiyou commented Mar 11, 2018

Add production Alpine and Debian Stretch Docker images #225

Add production Alpine and Debian Stretch Docker images #225

Conversation

kaiyou commented Jan 21, 2018

djmaze Jan 21, 2018

Choose a reason for hiding this comment

kaiyou Jan 21, 2018

Choose a reason for hiding this comment

djmaze Jan 21, 2018

Choose a reason for hiding this comment

djmaze commented Jan 21, 2018

Dryusdan commented Jan 21, 2018

kaiyou commented Jan 21, 2018

Dryusdan commented Jan 21, 2018

Dryusdan commented Jan 21, 2018

djmaze commented Jan 21, 2018

Dryusdan commented Jan 21, 2018

djmaze commented Jan 21, 2018

Dryusdan commented Jan 21, 2018 • edited Loading

Chocobozzz commented Jan 22, 2018

kaiyou commented Jan 22, 2018

Dryusdan commented Jan 23, 2018

kaiyou commented Jan 23, 2018 • edited Loading

kaiyou commented Jan 23, 2018 • edited Loading

kaiyou commented Jan 26, 2018 • edited Loading

Chocobozzz commented Jan 26, 2018

djmaze left a comment

Choose a reason for hiding this comment

djmaze Jan 26, 2018

Choose a reason for hiding this comment

djmaze Jan 26, 2018

Choose a reason for hiding this comment

djmaze Jan 26, 2018

Choose a reason for hiding this comment

kaiyou commented Jan 28, 2018

Chocobozzz commented Jan 29, 2018

1000i100 commented Mar 11, 2018 • edited Loading

kaiyou commented Mar 11, 2018

1000i100 commented Mar 11, 2018 • edited Loading

kaiyou commented Mar 11, 2018

1000i100 commented Mar 11, 2018 • edited Loading

kaiyou commented Mar 11, 2018

Dryusdan commented Jan 21, 2018 •

edited

Loading

kaiyou commented Jan 23, 2018 •

edited

Loading

kaiyou commented Jan 23, 2018 •

edited

Loading

kaiyou commented Jan 26, 2018 •

edited

Loading

1000i100 commented Mar 11, 2018 •

edited

Loading

1000i100 commented Mar 11, 2018 •

edited

Loading

1000i100 commented Mar 11, 2018 •

edited

Loading