Support `cp' sub command

[arnisoph]

For now, docker-compose cp is missing. :)

$ docker-compose -v
docker-compose version 1.7.1, build 0a9ab35

$ docker-compose -h
Define and run multi-container applications with Docker.

Usage:
  docker-compose [-f <arg>...] [options] [COMMAND] [ARGS...]
  docker-compose -h|--help

Options:
  -f, --file FILE             Specify an alternate compose file (default: docker-compose.yml)
  -p, --project-name NAME     Specify an alternate project name (default: directory name)
  --verbose                   Show more output
  -v, --version               Print version and exit
  -H, --host HOST             Daemon socket to connect to

  --tls                       Use TLS; implied by --tlsverify
  --tlscacert CA_PATH         Trust certs signed only by this CA
  --tlscert CLIENT_CERT_PATH  Path to TLS certificate file
  --tlskey TLS_KEY_PATH       Path to TLS key file
  --tlsverify                 Use TLS and verify the remote
  --skip-hostname-check       Don't check the daemon's hostname against the name specified
                              in the client certificate (for example if your docker host
                              is an IP address)

Commands:
  build              Build or rebuild services
  config             Validate and view the compose file
  create             Create services
  down               Stop and remove containers, networks, images, and volumes
  events             Receive real time events from containers
  exec               Execute a command in a running container
  help               Get help on a command
  kill               Kill containers
  logs               View output from containers
  pause              Pause services
  port               Print the public port for a port binding
  ps                 List containers
  pull               Pulls service images
  restart            Restart services
  rm                 Remove stopped containers
  run                Run a one-off command
  scale              Set number of containers for a service
  start              Start services
  stop               Stop services
  unpause            Unpause services
  up                 Create and start containers
  version            Show the Docker-Compose version information

[ocramz]

@bechtoldt What should cp do?

[ain]

docker-compose cp <container> <path> <local path>, e.g.

$ docker-compose cp db /var/lib/postgresql ~/Downloads/database_backup

[arnisoph]

@ocramz ^

[ain]

@bechtoldt @ocramz I also filed #3609 earlier, would offer great value when working with data across containers.

[ColinHebert]

That would be an awesome feature.

On top of that a support of copy in the configuration file (#1664) would be perfect

[ghost]

+1 for this feature. need copy some files into running container

[dorgan]

+1.... having to build a container to get a large file or 2 updates is a pain.

[alberto56]

As a workaround, you can do:

docker cp /path/to/my-local-file.sql "$(docker-compose ps -q mycontainer)":/file-on-container.sql

Now /file-on-container.sql will exist on your running container:

docker-compose exec mycontainer /bin/bash -c 'ls -lah /file-on-container.sql'
-rw-r--r-- 1 root root 244M Nov 14 19:48 /file-on-container.sql

[harobed]

I use this bash alias:

alias docker-compose-cp="function _f(){ docker cp \$2 \`docker-compose ps -q \$1\`:\$3;};_f"

:)

[johnmarcou]

With a "files" option in Compose which can copy local file to remote container between creation and starting process, it could be amazing to inject config files and secrets 👍

With local docker daemon, you can use "volumes" to map local file in a container, but it is not possible with remote docker daemon.

version: '2'
services:
  service1:
    image: nginx
    files:
      - nginx.conf:/etc/nginx/conf.d/nginx.conf
      - cert.crt:/etc/nginx/certs/cert.crt

[johanbrandhorst]

The workarounds suggested here are not bad, but the prime reason to want this is so that the cp is guaranteed to be executed before the command, which like @johnmarcou points out allows docker-compose to interact much better with remote docker daemons (i.e. ANY cloud based CI service). This should be a high priority feature imo.

[bperel]

Wouldn't it make more sense to use the service name rather than the container name ?
docker-compose cp [OPTIONS] SERVICE SRC_PATH DEST_PATH

So that we're in sync with the syntax of docker-compose exec which is:

$ docker-compose exec
Execute a command in a running container

Usage: exec [options] SERVICE COMMAND [ARGS...]

[ilinum]

How would it work if there are multiple instances of the same service?
If there are multiple containers, which one should it copy to?

[keithmattix]

Was there any resolution to this? I think this would be a very nice feature!

[ain]

Currently not. I'm using docker cp still. Somewhat inconvenient since you always need to know the ID or the name of the container.

[TETRA2000]

I think this feature will be great!!
I'll try to implement this command.

[barat]

Is this topic still alive? It would be great to be able to use 'cp' with service tags instead of container name (which can be dynamic).

[TETRA2000]

Sorry. I was doing other things. 😅
I restarted implementing recently.
If anyone else is trying this, I'll respect them.

[MitchStephan]

Any update on this? Ran into a use case where this would have been helpful.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tnguyen14]

This is not stale, still relevant and necessary

[stale]

This issue has been automatically marked as not stale anymore due to the recent activity.

[ianfixes]

See also #5523 (and the comment containing my workaround)

As demonstrated there, the docker-compose ps -q workaround doesn't work for the use case of copying files out of a container because you get the ID of the container before any artifacts were created.

[bryanpedini]

2nd of June 2020 now, (almost) two entire years after the first proposal...

Any news about replacing "a whack ton of time and space to build one image per configuration file and deploying 27000 different images" with "just a copy command in your compose yaml"?

I would need to deploy a demo of a service, with that scope in mind, I need to use a sqlite3 file instead of keeping everything on memory to allow me to use known admin credentials and not let people administer whatever they want (even in the demo image).
That poses a challenge: how I reset the database every X hours if not by keeping a local copy of the base image with just the database file as difference, and using the local build (or image) in docker-compose when I just had to change that file (perfect scope for docker-compose cp on the base image after up, and just reload the service to pick up the new config and db files)?

[TrentonAdams]

@bryanpedini It's my understanding that your use case is a docker anti-pattern. Docker containers are intended to be ephemeral. In other words, configuration management should be managed at either startup, as part of the entry point scripts, or done real time (with caching) with APIs to a configuration management services (AWS SSM parameter store, spring configuration server, etc, etc).

[danieldjewell]

@TrentonAdams while I agree that @bryanpedini 's particular case isn't 100% "docker-esque" ... there are still use cases where this would be convenient... For example, copying a small number of small files (e.g. crypto keys) into the image at startup. Sure, could use a volume or even build a custom image but those options (while certainly docker-esque) seem like such overkill for 2 or 3 ~5kb files. Especially if you want to customize on a per-image basis.

Further, with docker-compose, if you're copying in the files at each start in the yaml... that's still ephemeral. (I guess it's kinda like a "mini" layer at the end of the container image at that point...)

@bryanpedini Perhaps you can "docker cp" into the containers created by docker-compose?

[TrentonAdams]

@danieldjewell That sounds like the job of a startup script. e.g. enabling an env var that has data that gives permission to a crypto service, then connecting to said crypto service and grabbing keys for your needs. In the case of AWS for example, your container would receive an IAM access key for a role, which has permissions to use AWS KMS.

Or, you can put those keys (text only) themselves into ENV vars (not secure), if you're not using such services. Or mount the files. Or if you're using pure docker, it has facilities to share secret data. Mounting files, using docker's secrets, or using a configuration service at startup, is usually the best option.

[bryanpedini]

@TrentonAdams My understanding (which can be totally biased/wrong, not native Eng speaker here) of the situation is that Docker was not "meant" for this type of thing, tho IMO it's so much convenient to administer an image/container instead of an entire service on the host machine.
That said, I resolved my demo "issue" by having a custom image with just config file and SQLite3 DB file with credentials (otherwise empty) on top of the base image; then the docker-compose YAML uses the custom img instead of the baseline; and finally a cronjob that resets the compose stack (literally one container with one custom image).
That, if more "Docker-esque", is quite less convenient than just a cronjob with cd /opt/<service>; docker-compose cp <srvc_name> <file>; docker-compose restart <srvc_name>...
That's all 😅

[TrentonAdams]

Docker was not "meant" for this type of thing

Not meant for what type of thing?

[erithmetic]

There's a really good use case for this. In a CI system, I want to run tests in a docker container and get a coverage report. The actual docker host is a remote machine so I can't use simple volume mounts. Therefore, the only way to get the coverage report is to docker cp the file from the container onto the CI node.

[ndeloof]

docker compose cp is implemented by Compose v2

[bryanpedini]

sorry, I'm not all informed on the current news about everything in IT, but from my understanding what is now docker-compose as a separate binary and set of tools that has to be installed/updated/managed separately, will in the future be a docker <subcommand> like ps or rmi, integrated and installed/updated/managed with the single docker binary itself?

[ndeloof]

compose v2 is indeed a docker - actually a CLI plugin (separated binary) but will be distributed with the docker CLI. For now it's still beta so need to be installed in addition to the docker CLI, see https://github.com/docker/compose-cli#compose-v2-aka-local-docker-compose