Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor/use docker swarm #114

Merged
merged 44 commits into from
Dec 22, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
bd4e202
🚧 WIP
Fredkiss3 Dec 20, 2023
7b344fd
🚧 WIP
Fredkiss3 Dec 20, 2023
3a99ce3
🚧 Deploy to swarm local working
Fredkiss3 Dec 21, 2023
c517fbe
Merge branch 'main' into refactor/docker-swarm
Fredkiss3 Dec 21, 2023
97fddc1
🔥 remove unused component
Fredkiss3 Dec 21, 2023
10fc586
🚧 wip
Fredkiss3 Dec 21, 2023
b3ddb8a
🔧 update docker stack scripts
Fredkiss3 Dec 21, 2023
aa7750f
🚧 docker + redis shared cache, but sloooow
Fredkiss3 Dec 21, 2023
3d96237
🐛 remove docker x86 architecture dependecy
Fredkiss3 Dec 21, 2023
317619e
🚧 some changes
Fredkiss3 Dec 21, 2023
e2bbbbd
🚧 starting docker
Fredkiss3 Dec 22, 2023
217fbe5
🐛 some fixes
Fredkiss3 Dec 22, 2023
75f008c
💚 remove quotes around docker credentials
Fredkiss3 Dec 22, 2023
c5fcc0c
♻️ refactor
Fredkiss3 Dec 22, 2023
e4ad9cc
👷 use KV_PREFIX
Fredkiss3 Dec 22, 2023
0108168
👷 update
Fredkiss3 Dec 22, 2023
64f3736
👷 cache & restore data from nextjs build cache
Fredkiss3 Dec 22, 2023
e90feae
👷 use proper build cache key
Fredkiss3 Dec 22, 2023
58329c8
👷 some fixes
Fredkiss3 Dec 22, 2023
7a2771f
💚 put cmd in quotes
Fredkiss3 Dec 22, 2023
927a5e7
💚 remove ls
Fredkiss3 Dec 22, 2023
a4b7375
💚 separate copy cache output into its own step
Fredkiss3 Dec 22, 2023
f4ac03b
💚 Ignore error on ls
Fredkiss3 Dec 22, 2023
1735c40
💚 fix ci build
Fredkiss3 Dec 22, 2023
de90506
💚 copy to .next folder
Fredkiss3 Dec 22, 2023
ed7a905
👷 some changes
Fredkiss3 Dec 22, 2023
34433f7
👷 copy the build output to the correct path
Fredkiss3 Dec 22, 2023
96d6f91
👷 fix ?
Fredkiss3 Dec 22, 2023
f0632bc
💚 copy to .next
Fredkiss3 Dec 22, 2023
457cf61
👷 some changes
Fredkiss3 Dec 22, 2023
2e4b345
👷 add prod script & change env variables for dev
Fredkiss3 Dec 22, 2023
83e26eb
👷 auth with registry when deploying to docker swarm
Fredkiss3 Dec 22, 2023
b1a87c9
👷 disable previous CI (temp)
Fredkiss3 Dec 22, 2023
498fdb2
♻️ update local docker config
Fredkiss3 Dec 22, 2023
9b081dc
👷 embed webdis
Fredkiss3 Dec 22, 2023
addf4de
💚 fix ci errors
Fredkiss3 Dec 22, 2023
a2af4b6
🚧 some changes
Fredkiss3 Dec 22, 2023
e1898e3
🔊 log webdis error
Fredkiss3 Dec 22, 2023
89ead20
💚 pass default webdis URL to docker
Fredkiss3 Dec 22, 2023
6929403
🔨 some changes
Fredkiss3 Dec 22, 2023
bd45b1f
⚗️ test building production
Fredkiss3 Dec 22, 2023
154f08c
Revert "⚗️ test building production"
Fredkiss3 Dec 22, 2023
5ce4115
🐛 fix dev URL
Fredkiss3 Dec 22, 2023
b20232b
🔥 remove unused caddy file
Fredkiss3 Dec 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
.env
.env.local
Dockerfile
.dockerignore
node_modules
npm-debug.log
README.md
.git
6 changes: 3 additions & 3 deletions .github/workflows/deploy-to-vps.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ name: CI/CD For the Deployment
# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches:
- main
# push:
# branches:
# - main

# Allows you to run this workflow manually from the Actions tab+1
workflow_dispatch:
Expand Down
93 changes: 93 additions & 0 deletions .github/workflows/docker-deploy-dev.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# This is a basic workflow to help you get started with Actions

name: CI/CD For docker deploy

# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
pull_request:
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
build-push-docker:
runs-on: ubuntu-latest
environment: dev

steps:
- uses: actions/checkout@v2

- name: Get github branch name
id: get-github-ref
run: |
echo "github_ref=${{ github.head_ref }}" >> $GITHUB_OUTPUT
shell: bash

- name: Cache build output
id: next-build-cache
uses: actions/cache@v3
with:
path: .next
key: ${{ runner.os }}-next-build-cache-${{ steps.get-github-ref.outputs.github_ref }}
restore-keys: |
${{ runner.os }}-next-build-cache-${{ steps.get-github-ref.outputs.github_ref }}

- uses: whoan/docker-build-with-cache-action@v5
with:
username: fredkiss3
password: ${{ secrets.DCR_PASSWD }}
image_name: gh-next
image_tag: dev
push_git_tag: true
registry: dcr.fredkiss.dev
dockerfile: docker/Dockerfile.prod
context: .
build_extra_args: "--build-arg NEXT_PUBLIC_VERCEL_URL=gh-dev.fredkiss.dev --build-arg GITHUB_REDIRECT_URI=https://gh-dev.fredkiss.dev/api/auth/callback --build-arg SESSION_SECRET=${{ secrets.SESSION_SECRET }} --build-arg DATABASE_URL=${{ secrets.POSTGRES_DB_URL }} --build-arg GITHUB_CLIENT_ID=${{ secrets.GH_CLIENT_ID }} --build-arg GITHUB_SECRET=${{ secrets.GH_SECRET }} --build-arg GITHUB_PERSONAL_ACCESS_TOKEN=${{ secrets.GH_PERSONAL_ACCESS_TOKEN }} --build-arg REDIS_HTTP_USERNAME=${{ secrets.REDIS_HTTP_USERNAME }} --build-arg REDIS_HTTP_PASSWORD=${{ secrets.REDIS_HTTP_PASSWORD }} --build-arg KV_PREFIX=__gh_next__cache_dev_"

- name: copy build cache
run: |
ls -l ./.next/cache 2> /dev/null || true
echo ${{ secrets.DCR_PASSWD }} | docker login --username=fredkiss3 --password-stdin dcr.fredkiss.dev
docker pull dcr.fredkiss.dev/gh-next:dev
CONTAINER_ID=$(docker create dcr.fredkiss.dev/gh-next:dev)
mkdir -p .next/
docker cp ${CONTAINER_ID}:/app/.next/cache .next/
docker rm ${CONTAINER_ID}
ls -l ./.next/cache

deploy:
# The type of runner that the job will run on
needs:
- build-push-docker
runs-on: ubuntu-latest
environment: dev

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- uses: actions/checkout@v2

- name: Install SSH key
uses: shimataro/ssh-key-action@v2
with:
key: ${{ secrets.SSH_SERVER_KEY }}
name: id_rsa # optional
known_hosts: ${{ secrets.KNOWN_HOSTS }}

- name: Restart docker-stack
run: |
scp -P $DEPLOY_PORT ./docker/docker-stack.dev.yaml $DEPLOY_USER@$DEPLOY_DOMAIN:$DCR_DEPLOY_DIR/docker-stack.dev.yaml
ssh -p $DEPLOY_PORT $DEPLOY_USER@$DEPLOY_DOMAIN "
cd $DCR_DEPLOY_DIR
echo updating docker services for dev environment...
echo ${{ secrets.DCR_PASSWD }} | docker login --username=fredkiss3 --password-stdin dcr.fredkiss.dev
docker stack deploy --with-registry-auth --compose-file ./docker-stack.dev.yaml gh-stack-dev
echo services updated succesfully ✅
"
env:
DCR_DEPLOY_DIR: ${{ secrets.DCR_DEPLOY_DIR }}
DEPLOY_PORT: ${{ secrets.DEPLOY_PORT }}
DEPLOY_DOMAIN: ${{ secrets.DEPLOY_DOMAIN }}
DEPLOY_USER: ${{ secrets.DEPLOY_USER }}
DCR_PASSWD: ${{ secrets.DCR_PASSWD }}
DCR_USER: ${{ secrets.DCR_USER }}
95 changes: 95 additions & 0 deletions .github/workflows/docker-deploy-prod.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# This is a basic workflow to help you get started with Actions

name: CI/CD For docker deploy

# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches:
- main
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
build-push-docker:
runs-on: ubuntu-latest
environment: production

steps:
- uses: actions/checkout@v2

- name: Get github branch name
id: get-github-ref
run: |
echo "github_ref=${{ github.head_ref }}" >> $GITHUB_OUTPUT
shell: bash

- name: Cache build output
id: next-build-cache
uses: actions/cache@v3
with:
path: .next
key: ${{ runner.os }}-next-build-cache-${{ steps.get-github-ref.outputs.github_ref }}
restore-keys: |
${{ runner.os }}-next-build-cache-${{ steps.get-github-ref.outputs.github_ref }}

- uses: whoan/docker-build-with-cache-action@v5
with:
username: fredkiss3
password: ${{ secrets.DCR_PASSWD }}
image_name: gh-next
image_tag: latest
push_git_tag: true
registry: dcr.fredkiss.dev
dockerfile: docker/Dockerfile.prod
context: .
build_extra_args: "--build-arg NEXT_PUBLIC_VERCEL_URL=gh.fredkiss.dev --build-arg GITHUB_REDIRECT_URI=https://gh.fredkiss.dev/api/auth/callback --build-arg SESSION_SECRET=${{ secrets.SESSION_SECRET }} --build-arg DATABASE_URL=${{ secrets.POSTGRES_DB_URL }} --build-arg GITHUB_CLIENT_ID=${{ secrets.GH_CLIENT_ID }} --build-arg GITHUB_SECRET=${{ secrets.GH_SECRET }} --build-arg GITHUB_PERSONAL_ACCESS_TOKEN=${{ secrets.GH_PERSONAL_ACCESS_TOKEN }} --build-arg REDIS_HTTP_USERNAME=${{ secrets.REDIS_HTTP_USERNAME }} --build-arg REDIS_HTTP_PASSWORD=${{ secrets.REDIS_HTTP_PASSWORD }} --build-arg KV_PREFIX=__gh_next__cache_prod_"

- name: copy build cache
run: |
ls -l ./.next/cache 2> /dev/null || true
echo ${{ secrets.DCR_PASSWD }} | docker login --username=fredkiss3 --password-stdin dcr.fredkiss.dev
docker pull dcr.fredkiss.dev/gh-next:latest
CONTAINER_ID=$(docker create dcr.fredkiss.dev/gh-next:latest)
mkdir -p .next/
docker cp ${CONTAINER_ID}:/app/.next/cache .next/
docker rm ${CONTAINER_ID}
ls -l ./.next/cache

deploy:
# The type of runner that the job will run on
needs:
- build-push-docker
runs-on: ubuntu-latest
environment: dev

# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- uses: actions/checkout@v2

- name: Install SSH key
uses: shimataro/ssh-key-action@v2
with:
key: ${{ secrets.SSH_SERVER_KEY }}
name: id_rsa # optional
known_hosts: ${{ secrets.KNOWN_HOSTS }}

- name: Restart docker-stack
run: |
scp -P $DEPLOY_PORT ./docker/docker-stack.prod.yaml $DEPLOY_USER@$DEPLOY_DOMAIN:$DCR_DEPLOY_DIR/docker-stack.prod.yaml
ssh -p $DEPLOY_PORT $DEPLOY_USER@$DEPLOY_DOMAIN "
cd $DCR_DEPLOY_DIR
echo updating docker services for dev environment...
echo ${{ secrets.DCR_PASSWD }} | docker login --username=fredkiss3 --password-stdin dcr.fredkiss.dev
docker stack deploy --with-registry-auth --compose-file ./docker-stack.prod.yaml gh-stack-prod
echo services updated succesfully ✅
"
env:
DCR_DEPLOY_DIR: ${{ secrets.DCR_DEPLOY_DIR }}
DEPLOY_PORT: ${{ secrets.DEPLOY_PORT }}
DEPLOY_DOMAIN: ${{ secrets.DEPLOY_DOMAIN }}
DEPLOY_USER: ${{ secrets.DEPLOY_USER }}
DCR_PASSWD: ${{ secrets.DCR_PASSWD }}
DCR_USER: ${{ secrets.DCR_USER }}
5 changes: 4 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
{
"typescript.preferences.importModuleSpecifier": "non-relative",
"typescript.tsdk": "node_modules/typescript/lib",
"editor.defaultFormatter": "biomejs.biome"
"editor.defaultFormatter": "biomejs.biome",
"[typescript]": {
"editor.defaultFormatter": "biomejs.biome"
}
}
4 changes: 4 additions & 0 deletions dc-build-local.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
while read -r line; do
build_args="$build_args --build-arg $line"
done < .env.docker.local
docker build $build_args -t dcr.fredkiss.dev/gh-next:latest -f docker/Dockerfile.dev .
110 changes: 110 additions & 0 deletions docker/Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
##### DEPENDENCIES

FROM node:20-alpine3.19 AS deps
RUN apk add --no-cache libc6-compat
RUN apk update && apk upgrade openssl
WORKDIR /app

# Install dependencies based on the preferred package manager

COPY package.json yarn.lock* package-lock.json* pnpm-lock.yaml\* ./
COPY ./patches ./patches

RUN \
if [ -f yarn.lock ]; then yarn --frozen-lockfile; \
elif [ -f package-lock.json ]; then npm ci; \
elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && pnpm install --shamefully-hoist --strict-peer-dependencies=false --frozen-lockfile; \
else echo "Lockfile not found." && exit 1; \
fi

##### BUILDER

FROM node:20-alpine3.19 AS builder

ARG SESSION_SECRET
ARG DATABASE_URL
ARG GITHUB_CLIENT_ID
ARG GITHUB_SECRET
ARG GITHUB_PERSONAL_ACCESS_TOKEN
ARG KV_PREFIX
ARG NEXT_PUBLIC_VERCEL_URL="localhost:3000"
ARG REDIS_HTTP_URL="http://webdis:7379"
ARG REDIS_HTTP_USERNAME="user"
ARG REDIS_HTTP_PASSWORD="password"
ARG GITHUB_REDIRECT_URI="http://localhost:3000/api/auth/callback"

ENV NEXT_PUBLIC_VERCEL_URL=$NEXT_PUBLIC_VERCEL_URL
ENV SESSION_SECRET=$SESSION_SECRET
ENV DATABASE_URL=$DATABASE_URL
ENV GITHUB_CLIENT_ID=$GITHUB_CLIENT_ID
ENV GITHUB_REDIRECT_URI=$GITHUB_REDIRECT_URI
ENV GITHUB_SECRET=$GITHUB_SECRET
ENV GITHUB_PERSONAL_ACCESS_TOKEN=$GITHUB_PERSONAL_ACCESS_TOKEN
ENV REDIS_HTTP_URL=$REDIS_HTTP_URL
ENV REDIS_HTTP_USERNAME=$REDIS_HTTP_USERNAME
ENV REDIS_HTTP_PASSWORD=$REDIS_HTTP_PASSWORD
ENV KV_PREFIX=$KV_PREFIX

WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

ENV NEXT_TELEMETRY_DISABLED 1

RUN \
if [ -f yarn.lock ]; then yarn build; \
elif [ -f package-lock.json ]; then npm run build; \
elif [ -f pnpm-lock.yaml ]; then yarn global add pnpm && pnpm run build; \
else echo "Lockfile not found." && exit 1; \
fi

##### RUNNER

FROM node:20-alpine3.19 AS runner
WORKDIR /app

ARG SESSION_SECRET
ARG DATABASE_URL
ARG GITHUB_CLIENT_ID
ARG GITHUB_SECRET
ARG GITHUB_PERSONAL_ACCESS_TOKEN
ARG KV_PREFIX
ARG NEXT_PUBLIC_VERCEL_URL="localhost:3000"
ARG REDIS_HTTP_URL="http://webdis:7379"
ARG REDIS_HTTP_USERNAME="user"
ARG REDIS_HTTP_PASSWORD="password"
ARG GITHUB_REDIRECT_URI="http://localhost:3000/api/auth/callback"

ENV NEXT_PUBLIC_VERCEL_URL=$NEXT_PUBLIC_VERCEL_URL
ENV SESSION_SECRET=$SESSION_SECRET
ENV DATABASE_URL=$DATABASE_URL
ENV GITHUB_CLIENT_ID=$GITHUB_CLIENT_ID
ENV GITHUB_REDIRECT_URI=$GITHUB_REDIRECT_URI
ENV GITHUB_SECRET=$GITHUB_SECRET
ENV GITHUB_PERSONAL_ACCESS_TOKEN=$GITHUB_PERSONAL_ACCESS_TOKEN
ENV REDIS_HTTP_URL=$REDIS_HTTP_URL
ENV REDIS_HTTP_USERNAME=$REDIS_HTTP_USERNAME
ENV REDIS_HTTP_PASSWORD=$REDIS_HTTP_PASSWORD
ENV KV_PREFIX=$KV_PREFIX

ENV NODE_ENV production

ENV NEXT_TELEMETRY_DISABLED 1

RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

COPY --from=builder /app/next.config.js ./
COPY --from=builder /app/public ./public
COPY --from=builder /app/package.json ./package.json

COPY --from=builder --chown=nextjs:nodejs /app/.next/standalone ./
COPY --from=builder --chown=nextjs:nodejs /app/.next/static ./.next/static

USER nextjs
EXPOSE 3000
ENV PORT 3000
ENV NODE_ENV=production
ENV HOSTNAME 0.0.0.0

CMD [ "node", "server.js"]
Loading
Loading