Skip to content
/ dvsync Public

🐳️ Easy and secure way to copy data between Docker volumes, even across data centers

Notifications You must be signed in to change notification settings

suda/dvsync

Repository files navigation


dvsync


Easy and secure way to copy data between Docker volumes, even across VPCs or data centers.

dvsync is a set of two containers running OpenSSH, ngrok and rsync that automatically create a secure, encrypted channel between each other and enable easy way to migrate data stored in Docker volumes.


Running

⚠️ To establish a secure channel, dvsync uses ngrok and you need to pass your NGROK_AUTHTOKEN which can be found in ngrok dashboard.

dvsync will synchronize contents of its /data directory, therefore whatever data you want to sync, should be mounted under it.

Here are example ways to run it using Docker CLI, Docker Compose / Swarm or Kubernetes. Note that you can mix those, making data migration much easier.

Docker CLI

  1. Start a server where you want to copy data from:
docker run --rm -e NGROK_AUTHTOKEN="$NGROK_AUTHTOKEN" \
  --mount source=MY_SOURCE_VOLUME,target=/data,readonly \
  ghcr.io/suda/dvsync-server
  1. Once the server started, look into the logs and copy the DVSYNC_TOKEN
  2. Start the client where you want tot copy the data to:
docker run --rm -e DVSYNC_TOKEN="$DVSYNC_TOKEN" \
  --mount source=MY_TARGET_VOLUME,target=/data \
  ghcr.io/suda/dvsync-client

Using local source/target

Alternatively, if you want to copy this data to your local machine, you can mount a host directory as well:

docker run --rm -e DVSYNC_TOKEN="$DVSYNC_TOKEN" \
  -v $PWD:/data \
  ghcr.io/suda/dvsync-client

This can also be done other way around, when you start the server locally if you need to copy local data into the data center.

Docker Compose / Swarm

  1. Start a server where you want to copy data from:
version: '3.6'
services:
  dvsync-server:
    image: 'ghcr.io/suda/dvsync-server'
    environment:
      NGROK_AUTHTOKEN: ${NGROK_AUTHTOKEN}
    volumes:
      - type: volume
        source: MY_SOURCE_VOLUME
        target: /data
        read_only: true
volumes:
  MY_SOURCE_VOLUME:
  1. Once the server started, look into the logs and copy the DVSYNC_TOKEN
  2. Start the client where you want tot copy the data to:
version: '3.6'
services:
  dvsync-server:
    image: 'ghcr.io/suda/dvsync-client'
    environment:
      DVSYNC_TOKEN: ${DVSYNC_TOKEN}
    volumes:
      - type: volume
        source: MY_TARGET_VOLUME
        target: /data
volumes:
  MY_SOURCE_VOLUME:

Kubernetes

  1. Start a server where you want to copy data from:
apiVersion: v1
kind: Pod
metadata:
  name: dvsync-server
spec:
  containers:
  - image: ghcr.io/suda/dvsync-server
    name: dvsync-server
    env:
    - name: NGROK_AUTHTOKEN
      value: "REPLACE WITH YOUR NGROK_AUTHTOKEN"
    volumeMounts:
    - mountPath: /data
      name: MY_SOURCE_VOLUME
  volumes:
  - name: MY_SOURCE_VOLUME
  1. Once the server started, look into the logs and copy the DVSYNC_TOKEN
  2. Start the client where you want tot copy the data to:
apiVersion: v1
kind: Pod
metadata:
  name: dvsync-client
spec:
  containers:
  - image: ghcr.io/suda/dvsync-client
    name: dvsync-client
    env:
    - name: DVSYNC_TOKEN
      value: "REPLACE WITH YOUR DVSYNC_TOKEN"
    volumeMounts:
    - mountPath: /data
      name: MY_TARGET_VOLUME
  volumes:
  - name: MY_TARGET_VOLUME

Contributing

All contributions (no matter if small) are always welcome.

To see how you can help and where to start see Contributing file.

About

🐳️ Easy and secure way to copy data between Docker volumes, even across data centers

Topics

Resources

Code of conduct

Stars

Watchers

Forks

Packages

 
 
 

Languages