-
Notifications
You must be signed in to change notification settings - Fork 981
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat(build): debian package building
Signed-off-by: Boaz Sade <boaz@dragonflydb.io>
- Loading branch information
Showing
13 changed files
with
321 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
# Installation Packages | ||
|
||
## Overview | ||
This directory includes a set of files and scripts to build installation package for various Linux distributions. | ||
|
||
## Debian | ||
The file to build the Debian package all located under "debian" directory. | ||
The resulting package will install the binary of Dragonfly as well as generate a new service entry for dragonfly, | ||
that can be controlled with "systemctl" command, to start, stop and check status of. | ||
### Building | ||
To build the package, you have a script called "generate_debian_package.sh". This script accepts the following parameters: | ||
* Optional binary path - the location from which to take the binary for the installation. The default for this is "repo path/build-opt". | ||
The location to which the resulting package is writing is at the location from which the script is executed. | ||
This script is depends on the following packages: | ||
* git | ||
* moreutils | ||
* debhelper | ||
* dpkg-dev | ||
|
||
To build: | ||
``` | ||
/path/to/dragonfly/tools/packaging/generate_debian_package.sh [/path/to/dragonfly-binary-file] | ||
``` | ||
|
||
This can only be run on Debian based hosts. | ||
You can use the flowing docker file to generate this package: | ||
``` | ||
FROM ubuntu:20.04 | ||
ARG DEBIAN_FRONTEND=noninteractive | ||
RUN apt update -y && apt-get install -y gcc dpkg-dev gpg vim wget git moreutils debhelper | ||
``` | ||
Build the above docker and then run it with your dragonfly source code path mount as volume for the build: | ||
``` | ||
docker build -t ubuntu-package . | ||
docker run --rm -ti -v /path/to/dragonfly-repo:/mydocker-path ubuntu-package bash | ||
``` | ||
Again note that you need to be at "main" branch to successfully build this package. | ||
Note: If at the end of the installation you see a message "/usr/bin/deb-systemd-helper: error: systemctl preset failed on dragonfly.service: No such file or directory", | ||
you can ignore it, this seem to be related to [the following issue](https://groups.google.com/g/linux.debian.bugs.dist/c/m6xGZ82TdvM). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
11 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,12 @@ | ||
Source: dragonfly | ||
Maintainer: DragonflyDB authors <dragonfly@dragonflydb.io> | ||
Standards-Version: 4.2.1 | ||
Priority: optional | ||
Section: database | ||
Vcs-Git: https://github.com/dragonflydb/dragonfly | ||
|
||
Package: dragonfly | ||
Architecture: amd64 arm64 | ||
Depends: libc6, openssl, adduser, systemctl | ||
Homepage: https://dragonflydb.io | ||
Description: A fast in-memory store that is fully compatible with Redis™* and Memcached. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--pidfile=/var/run/dragonfly/dragonfly.pid | ||
--log_dir=/var/log/dragonfly | ||
--dir=/var/run/dragonfly | ||
--version_check=true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
debian/dragonfly.service /lib/systemd/system | ||
debian/dragonfly.conf /etc/dragonfly | ||
debian/bin/dragonfly /usr/bin |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
/var/log/dragonfly/dragonfly* { | ||
weekly | ||
missingok | ||
rotate 12 | ||
compress | ||
notifempty | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
#!/bin/sh | ||
|
||
# Script to run at the end of the installation | ||
set -eu | ||
|
||
USER="dfly" | ||
DIR_NAME="dragonfly" | ||
GROUP="$USER" | ||
CONFFILE="/etc/${DIR_NAME}/${DIR_NAME}.conf" | ||
|
||
if [ "$1" = "configure" ] | ||
then | ||
if ! dpkg-statoverride --list ${CONFFILE} >/dev/null 2>&1 | ||
then | ||
dpkg-statoverride --update --add ${USER} ${GROUP} 640 ${CONFFILE} | ||
fi | ||
fi | ||
|
||
#DEBHELPER# | ||
|
||
if [ "$1" = "configure" ] | ||
then | ||
find /etc/${DIR_NAME} -maxdepth 1 -type d -name '${DIR_NAME}.*.d' -empty -delete | ||
fi | ||
|
||
exit 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
#!/bin/sh | ||
# Script to run at the end of remove | ||
set -eu | ||
DIR_NAME="dragonfly" | ||
USER_NAME="dfly" | ||
CONFFILE="/etc/${DIR_NAME}/${DIR_NAME}.conf" | ||
|
||
# When purging the package, remove all trances | ||
if [ "${1}" = "purge" ] | ||
then | ||
userdel ${USER_NAME} || true | ||
rm -rf /var/lib/${DIR_NAME} /var/log/${DIR_NAME} /etc/${DIR_NAME} /var/run/${DIR_NAME} | ||
dpkg-statoverride --remove ${CONFFILE} || test $? -eq 2 | ||
fi | ||
|
||
#DEBHELPER# | ||
|
||
exit 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
#!/bin/sh | ||
|
||
set -eu | ||
# Script to run before the installation starts. | ||
# We are creating a user "dragonfly", and the directories that | ||
# would be used by the application | ||
|
||
USER="dfly" | ||
DIR_NAME="dragonfly" | ||
|
||
setup_dir () { | ||
DIR="${1}" | ||
MODE="${2}" | ||
GROUP="${3}" | ||
|
||
mkdir -p ${DIR} || { | ||
echo "failed to create dir ${DIR}" | ||
return 1 | ||
} | ||
|
||
if ! dpkg-statoverride --list ${DIR} >/dev/null 2>&1 | ||
then | ||
echo "changing owner for ${DIR} to user ${USER}" | ||
chown ${USER}:${GROUP} ${DIR} | ||
chmod ${MODE} ${DIR} | ||
fi | ||
} | ||
|
||
if [ "$1" = "install" ]; then | ||
if ! id ${USER} >/dev/null 2>&1 ; then | ||
echo "trying to create user ${USER}" | ||
adduser \ | ||
--system \ | ||
--home /var/lib/${DIR_NAME} \ | ||
--quiet \ | ||
--group \ | ||
${USER} || { | ||
echo "failed to add user ${USER}" | ||
exit 1 | ||
} | ||
|
||
setup_dir /var/log/${DIR_NAME} 2755 adm | ||
setup_dir /var/lib/${DIR_NAME} 755 ${USER} | ||
setup_dir /var/run/${DIR_NAME} 755 ${USER} | ||
setup_dir /etc/${DIR_NAME} 2775 ${USER} | ||
fi | ||
fi | ||
#DEBHELPER# | ||
|
||
exit 0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,42 @@ | ||
[Unit] | ||
Description=Modern and fast key-value store | ||
After=network.target | ||
Documentation= | ||
|
||
[Service] | ||
Type=simple | ||
ExecStart=/usr/bin/dragonfly --flagfile=/etc/dragonfly/dragonfly.conf | ||
PIDFile=/var/run/dragonfly/dragonfly.pid | ||
TimeoutStopSec=0 | ||
Restart=always | ||
User=dfly | ||
Group=dfly | ||
RuntimeDirectory=dragonfly | ||
RuntimeDirectoryMode=2755 | ||
|
||
UMask=007 | ||
PrivateTmp=yes | ||
LimitNOFILE=65535 | ||
PrivateDevices=yes | ||
ProtectHome=yes | ||
ReadOnlyDirectories=/ | ||
ReadWritePaths=-/var/lib/dragonfly | ||
ReadWritePaths=-/var/log/dragonfly | ||
ReadWritePaths=-/var/run/dragonfly | ||
|
||
NoNewPrivileges=true | ||
CapabilityBoundingSet=CAP_SETGID CAP_SETUID CAP_SYS_RESOURCE | ||
MemoryDenyWriteExecute=true | ||
ProtectKernelModules=true | ||
ProtectKernelTunables=true | ||
ProtectControlGroups=true | ||
RestrictRealtime=true | ||
RestrictNamespaces=true | ||
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX | ||
|
||
ProtectSystem=true | ||
ReadWriteDirectories=-/etc/dragonfly | ||
|
||
[Install] | ||
WantedBy=multi-user.target | ||
Alias=dragonfly.service |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
#!/usr/bin/make -f | ||
|
||
%: | ||
dh $@ | ||
|
||
|
||
override_dh_auto_build: | ||
@echo "no build is done here" | ||
|
||
override_dh_installchangelogs: | ||
@echo "no change long installation" | ||
|
||
override_dh_auto_test: | ||
@echo "no testing" | ||
|
||
override_dh_auto_clean: | ||
dh_auto_clean |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,30 @@ | ||
#!/usr/bin/env bash | ||
|
||
# This would generate a change log required for build Debian installation package. | ||
# Don't run this script on your local machine, run this inside docker | ||
# you would need to install git client as well as moreutils | ||
# apt install -y git moreutils | ||
# note: This script should run on branch "main". | ||
|
||
set -eu | ||
|
||
if [ $# -ne 1 ]; then | ||
echo "usage: <git repo path> <target path>" | ||
exit 1 | ||
fi | ||
SCRIPT_ABS_PATH=$(realpath $0) | ||
|
||
THIS_DIR=$(dirname ${SCRIPT_ABS_PATH}) | ||
GIT_DIR=$1 | ||
PACKGE_DIR=${THIS_DIR}/debian | ||
CHANGE_LOG=${PACKGE_DIR}/changelog | ||
|
||
cd ${GIT_DIR} | ||
git config --global --add safe.directory ${GIT_DIR} | ||
>${CHANGE_LOG} | ||
prevtag=v0.2.0 | ||
pkgname=`cat ${PACKGE_DIR}/control | grep '^Package: ' | sed 's/^Package: //'` | ||
git tag -l v* | sort -V | while read tag; do | ||
(echo "$pkgname (${tag#v}) unstable; urgency=low"; git log --pretty=format:' * %s' $prevtag..$tag; git log --pretty='format:%n%n -- %aN <%aE> %aD%n%n' $tag^..$tag) | cat - ${CHANGE_LOG} | sponge ${CHANGE_LOG} | ||
prevtag=$tag | ||
done |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
#!/usr/bin/env bash | ||
|
||
# Generate a debian package from a pre-build dragonfly bianry and set of files as well as generating change log from git history. | ||
# The result is debian install package file (.deb file). | ||
# This script accept 2 parameters: | ||
# 1. Optioanl path to the location at which the binary file is located. | ||
# this depends on | ||
# * git | ||
# * moreutils | ||
# * debhelper | ||
# e.g. apt update -y && apt install -y git moreutils debhelper | ||
# Please note that is must run from main branch. | ||
# Best running this from inside a container. | ||
# The result are writing to the location from which you would execute the script (not where the script is located). | ||
# Version number is the tag number. Currently this would only generate a package for amd64. | ||
|
||
|
||
set -eu | ||
|
||
SCRIPT_ABS_PATH=$(realpath $0) | ||
SCRIPT_PATH=$(dirname ${SCRIPT_ABS_PATH}) | ||
PACKAGES_PATH=${SCRIPT_PATH}/debian | ||
CHANGELOG_SCRIPT=generate_changelog.sh | ||
BUILD_DIR=build-opt | ||
ROOT_ABS_PATH=$(cd ${SCRIPT_PATH}; while [ ! -d ${BUILD_DIR} ]; do cd ..; done ; pwd) | ||
REPO_PATH=${ROOT_ABS_PATH} | ||
TEMP_WORK_DIR=$(mktemp -d) | ||
BASE_DIR=${TEMP_WORK_DIR}/packges | ||
ARCH_VAL=amd64 | ||
BASE_PATH=${BASE_DIR}/dragonfly_${ARCH_VAL} | ||
BINARY_TARGET_DIR=${BASE_PATH}/debian/bin | ||
|
||
function cleanup { | ||
echo $@ | ||
rm -rf ${TEMP_WORK_DIR} | ||
exit 1 | ||
} | ||
|
||
if [ $# -ge 1 ]; then | ||
VERSION_FILE=$1 | ||
else | ||
if ! [ -f ${ROOT_ABS_PATH}/${BUILD_DIR}/dragonfly ]; then | ||
cleanup "no dragonfly binary found at ${ROOT_ABS_PATH}/${BUILD_DIR}" | ||
else | ||
VERSION_FILE=${ROOT_ABS_PATH}/${BUILD_DIR}/dragonfly | ||
fi | ||
fi | ||
|
||
mkdir -p ${BASE_PATH} || cleanup "failed to create working directory for building the package" | ||
|
||
cp -r ${PACKAGES_PATH} ${BASE_PATH} || cleanup "failed to copy required for the package build from ${PACKAGES_PATH}" | ||
|
||
cp ${SCRIPT_PATH}/${CHANGELOG_SCRIPT} ${BASE_PATH} || cleanup "failed to copy changelog script to ${BASE_PATH}" | ||
|
||
mkdir -p ${BINARY_TARGET_DIR} || cleanup "failed to create install directory for building the package" | ||
|
||
cp ${VERSION_FILE} ${BINARY_TARGET_DIR}/dragonfly || cleanup "failed to copy binary to target dir" | ||
|
||
${BASE_PATH}/${CHANGELOG_SCRIPT} ${REPO_PATH} || cleanup "failed to generate changelog for package" | ||
|
||
MY_DIR=${PWD} | ||
cd ${BASE_PATH} && dpkg-buildpackage --build=binary || cleanup "failed to generate the package" | ||
TEMP_RESULT_FILE=$(ls ../*.deb) | ||
|
||
mv ${TEMP_RESULT_FILE} ${MY_DIR} && cd ${MY_DIR} | ||
RESULT_FILE=$(ls *.deb 2>/dev/null) | ||
if [ "$RESULT_FILE" = "" ]; then | ||
cleanup "failed to find build result file" | ||
fi | ||
echo "successfully build the install package at ${MY_DIR}/${RESULT_FILE}" | ||
rm -rf ${TEMP_WORK_DIR} | ||
exit 0 |