Skip to content

Commit

Permalink
Dev to main - 04092024 (#5)
Browse files Browse the repository at this point in the history
* Added backup feature via cron
* Added automated tuning mysql params based on deployed VM
* Fix for deployment without warp, earlier xray config was not valid.
* Doc
  • Loading branch information
v-kamerdinerov authored Sep 4, 2024
1 parent 5974bf7 commit a5be092
Show file tree
Hide file tree
Showing 14 changed files with 349 additions and 13 deletions.
2 changes: 1 addition & 1 deletion README-RU.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ ansible-playbook marzban-deploy.yml
* ~~Добавить возможность использования отдельного инстанса СУБД MySQL/MariaDB~~
* Добавить возможность автоматической генерации сертификата Lets Encrypt при его отсутствии в инвентори
* Добавить новые вариативные инбауды, Trojan, Vmess и так далее
* Добавить автоматические скрипты бекапа
* ~~Добавить автоматические скрипты бекапа~~
* Добавить автоматический поиск наилучшего SNI в зависимости от региона
* Поправить известные проблемы :)

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ ansible-playbook marzban-deploy.yml
* ~~Adding the ability to use a separate MySQL/MariaDB DB instance~~
* Adding possibility to automatically generate Lets Encrypt certificate if it is not present in the inventory
* Adding new variate inbound like Trojan, Vmess etc
* Adding automatic backup scripts
* ~~Adding automatic backup scripts~~
* Add automatic search for the best SNI depending on the region
* Fix known issues :)

Expand Down
52 changes: 51 additions & 1 deletion doc/FAQ-RU.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,4 +70,54 @@ marzban_warp_domains:
- "domain:iplocation.net"
- "domain:spotify.com"
- "domain:linkedin.com"
```
```

## Я хочу использовать отдельную базу Mysql, как мне это сделать?

По дефолту в инвентори уже включена такая возможность `marzban_mysql_instance: true`

```
root@main:/opt/marzban# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9cac00a5a838 haproxy:2.4.25 "docker-entrypoint.s…" 23 minutes ago Up 23 minutes marzban-haproxy-1
8623ba69c221 gozargah/marzban:latest "bash -c 'alembic up…" 23 minutes ago Up 23 minutes marzban-marzban-1
63a539f0ee62 mariadb:lts "docker-entrypoint.s…" 23 minutes ago Up 23 minutes (healthy) marzban-mariadb-1
```

## Я хочу бекапировать все данные marzban, как мне это сделать?

По дефолту в инвентори уже включена такая возможность `marzban_backup: true`.

Периодичность сохранения конфигурируется в переменной `marzban_backup_cron`.

```
/var/lib/marzban_backups/backup_${DATE}/
├── lib
│ ├── access.log
│ ├── certs
│ │ ├── example-domain.com.cert
│ │ └── example-domain.com.key
│ ├── error.log
│ └── xray_config.json
│ └── db.sqlite3
└── opt
├── credentials
│ ├── pass_marzban_mysql_root_password
│ ├── pass_marzban_mysql_user_password
│ └── x25519_key
└── docker-compose.yml
```
Крон джоба каждый день сохраняет все ваши данные по пути в:
```/var/lib/marzban_backups/```

Если у вас есть отдельны инстанс СУБД, джоба так же сделает дамп данной БД:

```
/var/lib/marzban_backups/backup_${date}/db-backup/
└── marzban.sql
```

Так же, если вы заполните переменные `marzban_backup_telegram_bot_token` и `marzban_backup_telegram_chat_id` бекап будет отправлен вам в чат.

<img src="./images/backup_telegram.png" width="400">
52 changes: 51 additions & 1 deletion doc/FAQ.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,4 +70,54 @@ marzban_warp_domains:
- "domain:iplocation.net"
- "domain:spotify.com"
- "domain:linkedin.com"
```
```

## I want to use a separate Mysql database, how do I do that?

By default, `marzban_mysql_instance: true` is already enabled in the inventory.

```
root@main:/opt/marzban# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9cac00a5a838 haproxy:2.4.25 "docker-entrypoint.s…" 23 minutes ago Up 23 minutes marzban-haproxy-1
8623ba69c221 gozargah/marzban:latest "bash -c 'alembic up…" 23 minutes ago Up 23 minutes marzban-marzban-1
63a539f0ee62 mariadb:lts "docker-entrypoint.s…" 23 minutes ago Up 23 minutes (healthy) marzban-mariadb-1
```

## I want backup all my marzban data, how do I do this?

By default, `marzban_backup: true` is already enabled in the inventory.

The frequency of saving is configured in the `marzban_backup_cron` variable.

```
/var/lib/marzban_backups/backup_${DATE}/
├── lib
│ ├── access.log
│ ├── certs
│ │ ├── example-domain.com.cert
│ │ └── example-domain.com.key
│ ├── error.log
│ └── xray_config.json
│ └── db.sqlite3
└── opt
├── credentials
│ ├── pass_marzban_mysql_root_password
│ ├── pass_marzban_mysql_user_password
│ └── x25519_key
└── docker-compose.yml
```
The cron job saves all your data every day on the path:
```/var/lib/marzban_backups/```

If you have a optional DB instance, the job will also dump this database:

```
/var/lib/marzban_backups/backup_${date}/db-backup/
└─── marzban.sql
```

Also, if you fill in the variables `marzban_backup_telegram_bot_token` and `marzban_backup_telegram_chat_id` the backup will be sent to your chat.

<img src="./images/backup_telegram.png" width="400">
Binary file added doc/images/backup_telegram.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions group_vars/marzban/marzban.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,15 @@ marzban_docs: false
# Flag for using mysql instance instead of sql-lite
marzban_mysql_instance: true

# Flag for enabling automatic backup
marzban_backup: true
marzban_backup_cron: { minute: "0", hour: "2", day: "*", month: "*", weekday: "*" }

# Fill if you want sent you backups to Telegram. Stay null if not needed
marzban_backup_telegram_bot_token: ""
# If chat_id is null - role try to get it automatically. But first, start chat with you bot.
marzban_backup_telegram_chat_id: ""

# List of DNS servers used
marzban_dns_servers:
- "8.8.8.8"
Expand Down
8 changes: 7 additions & 1 deletion roles/marzban/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,9 +90,15 @@ marzban_mysql_root_password: ""

# tuning
marzban_mysql_innodb_open_files: "1024"
marzban_mysql_innodb_buffer_pool_size: "268435456"
marzban_mysql_innodb_buffer_pool_size: ""
marzban_mysql_binlog_expire_logs_seconds: "5184000" # 60 days

### Backup
marzban_backup: false
marzban_backup_telegram_bot_token: ""
marzban_backup_telegram_chat_id: ""
marzban_backup_cron: { minute: "0", hour: "2", day: "*", month: "*", weekday: "*" }

### Node configuration

marzban_ssl_node_path: "/var/lib/marzban-node/ssl_client_cert.pem"
Expand Down
139 changes: 139 additions & 0 deletions roles/marzban/files/marzban_backup.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
#!/bin/bash

# Log file
LOG_FILE="/var/log/marzban-backuper.log"

# Colors for logging
FontColor_Red="\033[31m"
FontColor_Green="\033[32m"
FontColor_Yellow="\033[33m"
FontColor_Suffix="\033[0m"

# Directories to backup
LIB_SRC_DIR="/var/lib/marzban/*"
OPT_SRC_DIR="/opt/marzban/*"

# Backup directory
BACKUP_DIR="/var/lib/marzban_backups"

# Environment file
ENV_FILE="/opt/marzban/.env"

# Get current date
CURRENT_DATE=$(date +"%Y-%m-%d_%H-%M-%S")
HOSTNAME_NODE=$(hostname)

# Logging function with levels and colors
log() {
LEVEL="$1"
MSG="$2"
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
case "${LEVEL}" in
INFO)
printf "[${TIMESTAMP}] [${FontColor_Green}${LEVEL}${FontColor_Suffix}] %s\n" "${MSG}" | tee -a "$LOG_FILE"
;;
WARN)
printf "[${TIMESTAMP}] [${FontColor_Yellow}${LEVEL}${FontColor_Suffix}] %s\n" "${MSG}" | tee -a "$LOG_FILE"
;;
ERROR)
printf "[${TIMESTAMP}] [${FontColor_Red}${LEVEL}${FontColor_Suffix}] %s\n" "${MSG}" | tee -a "$LOG_FILE" >&2
;;
*)
printf "[${TIMESTAMP}] %s\n" "${MSG}" | tee -a "$LOG_FILE"
;;
esac
}

# Function to create backup directory
create_backup_dir() {
if [ ! -d "$BACKUP_DIR" ]; then
log INFO "Creating backup directory: $BACKUP_DIR"
mkdir -p "$BACKUP_DIR"
fi
}

# Function to backup directories
backup_directories() {
create_backup_dir
BACKUP_PATH="$BACKUP_DIR/backup_$CURRENT_DATE"
log INFO "Backing up directories to $BACKUP_PATH"

# Create directory structure for backup
mkdir -p "$BACKUP_PATH/lib"
mkdir -p "$BACKUP_PATH/opt"

# Backup contents
cp -r $LIB_SRC_DIR "$BACKUP_PATH/lib/" || log WARN "Failed to backup lib directory."
cp -r $OPT_SRC_DIR "$BACKUP_PATH/opt/" || log WARN "Failed to backup opt directory."

log INFO "Directories backed up successfully."
}

# Function to backup MySQL databases
backup_databases() {
if grep -q "SQLALCHEMY_DATABASE_URL" "$ENV_FILE" && ! grep -q "sqlite" "$ENV_FILE"; then
# Fixed user root
DB_USER="root"

# Extract root password from environment file
DB_PASSWORD=$(grep -oP '(?<=ROOT_PASSWORD=).*' "$ENV_FILE")

# Install mysqldump if not installed
if ! command -v mysqldump &> /dev/null; then
log WARN "mysqldump not found, installing..."
apt-get update && apt-get install -y mysql-client || log ERROR "Failed to install mysql-client."
fi

# Dump databases
DB_BACKUP_PATH="$BACKUP_PATH/db-backup"
mkdir -p "$DB_BACKUP_PATH"

# Get the list of databases and exclude system databases
databases=$(mysql -h 127.0.0.1 --user="$DB_USER" --password="$DB_PASSWORD" -e "SHOW DATABASES;" | tr -d "| " | grep -v -E 'Database|information_schema|mysql|performance_schema|sys')

for db in $databases; do
log INFO "Dumping database: $db"
# Use additional parameters to avoid errors
mysqldump -h 127.0.0.1 --force --opt --user="$DB_USER" --password="$DB_PASSWORD" --single-transaction --skip-lock-tables --no-tablespaces --databases "$db" > "$DB_BACKUP_PATH/$db.sql" || log ERROR "Getting some error while dump database: $db"
done
log INFO "Databases dumped successfully."
else
log INFO "SQLite detected or SQLALCHEMY_DATABASE_URL not found. No MySQL databases to backup."
fi
}

# Function to send backup to Telegram
send_to_telegram() {
# Extract Telegram bot token and chat ID from environment file
TELEGRAM_BOT_TOKEN=$(grep -oP '(?<=TELEGRAM_BOT_TOKEN=).*' "$ENV_FILE")
TELEGRAM_CHAT_ID=$(grep -oP '(?<=TELEGRAM_CHAT_ID=).*' "$ENV_FILE")

if [ -n "$TELEGRAM_BOT_TOKEN" ] && [ -n "$TELEGRAM_CHAT_ID" ]; then
TAR_FILE="$BACKUP_DIR/backup_$CURRENT_DATE.tar.gz"
log INFO "Creating tarball $TAR_FILE"
tar -czf "$TAR_FILE" -C "$BACKUP_DIR" "backup_$CURRENT_DATE" || log ERROR "Failed to create tarball."

log INFO "Sending backup to Telegram chat ID $TELEGRAM_CHAT_ID"
curl -F chat_id="${TELEGRAM_CHAT_ID}" \
-F caption="Backup from $HOSTNAME_NODE at $CURRENT_DATE" \
-F parse_mode="HTML" \
-F document=@"$TAR_FILE" \
https://api.telegram.org/bot${TELEGRAM_BOT_TOKEN}/sendDocument || log ERROR "Failed to send backup to Telegram."

log INFO "Backup sent to Telegram successfully."
else
log WARN "Telegram BOT token or chat ID not provided. Skipping upload."
fi
}

# Main backup process
main() {
log INFO "Starting backup process."
backup_directories
backup_databases
send_to_telegram
log INFO "Backup process completed."
}

# Run the main process
main
17 changes: 17 additions & 0 deletions roles/marzban/tasks/backup.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---

- name: "Marzban | Backup | Add backup script"
copy:
src: "marzban_backup.sh"
dest: "/usr/local/bin/marzban_backup.sh"
mode: "0755"

- name: "Marzban | Backup | Add cronjob"
cron:
name: "MARZBAN backup"
minute: "{{ marzban_backup_cron.minute | default(omit) }}"
hour: "{{ marzban_backup_cron.hour | default(omit) }}"
day: "{{ marzban_backup_cron.day | default(omit) }}"
month: "{{ marzban_backup_cron.month | default(omit) }}"
weekday: "{{ marzban_backup_cron.weekday | default(omit) }}"
job: "/usr/local/bin/marzban_backup.sh"
8 changes: 8 additions & 0 deletions roles/marzban/tasks/conf/panel.yml
Original file line number Diff line number Diff line change
Expand Up @@ -153,3 +153,11 @@
register: panel_host_info_change
until: panel_host_info_change is not failed
tags: bootstrap

- name: "Marzban | Configure {{ role | upper() }} | Post-deploy"
include_tasks: ../post-deploy.yml

- name: "Marzban | Configure {{ role | upper() }} | Backup"
include_tasks: ../backup.yml
when:
- marzban_backup is defined and marzban_backup
4 changes: 0 additions & 4 deletions roles/marzban/tasks/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,7 @@
tags:
- configure


- name: "Marzban | Add nodes"
include_tasks: add-node.yml
when:
- marzban_nodes_group_name in groups

- name: "Marzban | Post-deploy"
include_tasks: post-deploy.yml
Loading

0 comments on commit a5be092

Please sign in to comment.