Improve (#6)

* sync

* Tested clamav for every supported distribution

* Add firewall rules; Sync

* Improve hcloud interface

.github/workflows/gitops.yaml

@@ -39,8 +39,8 @@ jobs:
 
           echo "${{ secrets.ANSIBLE_KEYS_ALL }}" > .keys/all
 
-          git submodule update
-          ansible-galaxy collection install hetzner.hcloud
+          ansible-galaxy collection install hetzner.hcloud community.general ansible.posix
+          ansible-galaxy role install geerlingguy.postgresql geerlingguy.redis
           /opt/pipx/venvs/ansible-core/bin/python -m pip install -r requirements.txt
 
           # not quiet sure why, but we need this package in this python environment

.gitignore

@@ -358,3 +358,5 @@ $RECYCLE.BIN/
 
 .keys/*
 !.keys/.gitkeep
+
+_examples.tar.gz

.gitmodules

@@ -1,15 +0,0 @@
-[submodule "roles/ansible-role-fail2ban"]
-	path = roles/ansible-role-fail2ban
-	url = https://github.com/secshellnet/ansible-role-fail2ban
-[submodule "roles/ansible-role-sshd"]
-	path = roles/ansible-role-sshd
-	url = https://github.com/secshellnet/ansible-role-sshd
-[submodule "roles/ansible-role-nginx"]
-	path = roles/ansible-role-nginx
-	url = https://github.com/secshellnet/ansible-role-nginx
-[submodule "roles/ansible-role-postgresql"]
-	path = roles/ansible-role-postgresql
-	url = https://github.com/geerlingguy/ansible-role-postgresql
-[submodule "roles/ansible-role-redis"]
-	path = roles/ansible-role-redis
-	url = https://github.com/geerlingguy/ansible-role-redis

README.md

@@ -2,62 +2,77 @@
 
 This repository template provides an ansible inventory to manage cloud server in 
 hetzner cloud (hcloud). It performes some basic linux hardening (unattended upgrades, 
-ssh, fail2ban) and can be extended by roles or tasks to perform whatever you need.
+ssh, fail2ban, ...) and can be extended by roles or tasks to perform whatever you need.
 
 ## Getting started
-1. Create a reporitory from this template repository and clone it:
-   ```shell
-   git clone git@github.com:YOUR-USERNAME/hcloud-ansible.git
-   ```
-2. Create account on [hetzner.cloud](https://console.hetzner.cloud/) and create a new cloud project
-3. Create an api token inside this cloud project
-   ![Creating an api token in the hetzner cloud console](./img/hetzner-create-api-token.png)
-4. Generate a new secret for the ansible vault file (you can use any password generated, store it in `.keys/all`)
-   ```shell
-   cat /dev/urandom | tr -dc A-Za-z0-9 | fold -w 20 | head -n 1 > .keys/all
-   ```
-5. Create a new ansible vault
-   ```shell
-   ansible-vault create group_vars/all/vault
-   ```
-   with the following content:
-   ```yaml
-   ---
-   hcloud_api_token: "__YOUR_API_TOKEN__"
-   worker_password: "__RANDOM_SECRET_PASSWORD__"
-   ```
-6. Extend the [`inventory.yaml`](./inventory.yaml), it should look for example like this:
-   ```yaml
-   ---
-   all:
-     hosts:
-       server1:   # default settings if no configuration given
-         server_type: cx11
-         location: hel1
-         image: ubuntu-22.04
-         enable_ipv4: false
-         enable_ipv6: true
-       server2:
-         server_type: cx21
-         location: fsn1
-         image: debian-12
-         enable_ipv4: true
-         enable_ipv6: true
-   ```
-
-7. Install the required ansible and python modules:
-   ```shell
-   ansible-galaxy collection install hetzner.hcloud
-   pip3 install -r requirements.txt
-   ```
-8. Use the ansible inventory:
-   ```shell
-   ansible-playbook playbook.yaml
-   ```
-9. Create a backup of the [`.keys`](./keys/) directory. It contains the key to your vault 
-   and the ssh key ansible uses to connect to the cloud servers. For security reasons this 
-   directory is excluded from git operations (see [`.gitignore`](./.gitignore)), so by 
-   default it will not be pushed to your git repository!
+To use this template, start by creating a repository that inherits from this template. Next 
+create an account and a new cloud project on [hetzner.cloud](https://console.hetzner.cloud/).
+Generate a password for the ansible vault and store it in `.keys/all`. Afterwards create a new
+ansible vault with this password and add the following to it.
+```shell
+# create random password
+cat /dev/urandom | tr -dc A-Za-z0-9 | fold -w 20 | head -n 1 > .keys/all
+
+# create ansible vault using predefined password
+ansible-vault create group_vars/all/vault
+```
+
+Add a random password for the worker user on the machine and the api token (see image) to the
+vault in the following format.
+![Creating an api token in the hetzner cloud console](./img/hetzner-create-api-token.png)
+
+```yaml
+---
+hcloud_api_token: "__YOUR_API_TOKEN__"
+worker_password: "__RANDOM_SECRET_PASSWORD__"
+```
+
+Next you need to extend your inventory, for example like this:
+```yaml
+---
+all:
+   hosts:
+      server1:   # default settings if no configuration given
+      server_type: cx11
+      location: hel1
+      image: ubuntu-22.04
+      enable_ipv4: false
+      enable_ipv6: true
+      server2:
+      server_type: cx21
+      location: fsn1
+      image: debian-12
+      enable_ipv4: true
+      enable_ipv6: true
+```
+
+After you installed the required ansible and python modules you should be able to use the inventory.
+```shell
+pip3 install ansible ansible-lint
+pip3 install -r requirements.txt
+ansible-galaxy collection install -r requirements.yaml
+ansible-galaxy role install -r requirements.yaml
+
+ansible-playbook playbook.yaml
+
+# you can also limit the playbook to one of your hosts
+ansible-playbook playbook.yaml --limit alpha
+
+# tags can be used to run only specific parts of a playbook
+ansible-playbook playbook.yaml --list-tasks
+ansible-playbook playbook.yaml --limit alpha --tags redis
+
+# to check wether the system picks up the correct variables you can run
+ansible-inventory --vars --graph
+
+# make sure to lint your inventory regulary
+ansible-lint
+```
+
+Make sure to create a backup of the [`.keys`](./keys/) directory. It contains the key to your
+vault and the ssh key ansible uses to connect to the cloud servers. For security reasons this
+directory is excluded from git operations (see [`.gitignore`](./.gitignore)), so by default it
+will not be pushed to your git repository!
 
 ## What about GitOps?
 I've tried integrating git ops, but there is one problem: the GitHub actions runner does 
@@ -70,6 +85,8 @@ sure to commit and push both your `inventory.yaml` and `group_vars/all/vault` fi
 - Secret: `ANSIBLE_KEYS_ALL` to the content of [`.keys/all`](./keys/all)
 
 ## Structure
+
+Even though the structure is self explaining here are some comments:
 ```shell
 hcloud-ansible
 ├── .github                      # github actions workflows
@@ -82,6 +99,7 @@ hcloud-ansible
 ├── ansible.cfg
 ├── group_vars
 │   └── all
+│       ├── defaults.yaml
 │       ├── vars.yaml            # plaintext global variables
 │       └── vault                # encrypted global variables (e. g. hetzner cloud api token)
 ├── inventory.yaml
@@ -91,59 +109,36 @@ hcloud-ansible
 │   ├── ansible-role-nginx       # our role to install nginx with acme.sh and cf dns integration
 │   ├── ansible-role-postgresql  # role to install a postgresql database server
 │   └── ansible-role-sshd
+├── ssh
+│   └── nicof2000.pub
 ├── tasks
-│   ├── auto-update.yaml
-│   ├── create-worker-user.yaml
-│   └── hetzner-cloud.yaml       # task to manage cloud servers and aquire information to connect
+│   ├── linux
+│   │   ├── audit-lynis.yaml
+│   │   ├── audit-openscap.yaml
+│   │   ├── create-users.yaml
+│   │   ├── create-worker-user.yaml
+│   │   ├── setup-auditd.yaml
+│   │   ├── setup-auto-update.yaml
+│   │   ├── setup-clamav.yaml
+│   │   ├── setup-fail2ban.yaml
+│   │   ├── setup-iptables.yaml
+│   │   ├── setup-rkhunter.yaml
+│   │   └── setup-sshd.yaml
+│   └── local
+│       ├── ensure-keys-exists.yaml
+│       └── hetzner-cloud.yaml   # task to manage cloud servers and aquire information to connect
 └── templates
-    └── dnf-automatic.conf.j2
+    ├── auditd.rules.j2
+    ├── dnf-automatic.conf.j2
+    └── fail2ban-sshd.conf.j2
 ```
 
-### [ansible-role-sshd](https://github.com/secshellnet/ansible-role-sshd)
-
-### [ansible-role-fail2ban](https://github.com/secshellnet/ansible-role-fail2ban)
-
-### [ansible-role-nginx](https://github.com/secshellnet/ansible-role-nginx)
+## [Examples](./docs/EXAMPLES.md)
 
-### [ansible-role-redis](https://github.com/geerlingguy/ansible-role-redis)
-
-### [ansible-role-postgresql](https://github.com/geerlingguy/ansible-role-postgresql)
-```yaml
-# host_vars/<hostname>/vars.yaml
----
-postgresql_databases:
-  - name: nextcloud
-    state: present
-
-  # synapse requires lc_collate and lc_ctype to be set to C
-  - name: synapse
-    lc_collate: C
-    lc_ctype: C
-    state: present
-
-postgresql_users_u:
-  - name: nextcloud
-    db: nextcloud
-    state: present
-
-  - name: synapse
-    db: synapse
-    state: present
+A full list of available system images can be aquired using the following command:
 ```
-
-```yaml
-# host_vars/<hostname>/vault
----
-postgresql_users_e:
-  - name: nextcloud
-    password: s3cr3t-p4ssw0rd
-
-  - name: synapse
-    password: s3cr3t-p4ssw0rd
+curl -s -H "Authorization: Bearer $TOKEN" \
+  "https://api.hetzner.cloud/v1/images" \
+  | jq -r '.images[] | select(.type == "system") | .name'
 ```
-
-- You can spawn a postgres shell using: `sudo -u postgres psql`.
-- Use `\l` to list databases, `\du` to list users and `\dt` to list tables.
-- Use `\c <database>` to connect to a database
-- You can also connect using tcp (like any other application): 
-  `psql -h 127.0.0.1 -U <user> <database>`
+TODO: For whatever reason neighter fedora nor alma linux is available in this list

ansible.cfg

@@ -1,6 +1,7 @@
 [defaults]
 inventory = inventory.yaml
-interpreter_python = /usr/bin/python3
-host_key_checking = False
 private_key_file = .keys/id_ecdsa
-vault_identity_list = all@.keys/all
+vault_identity_list = all@.keys/all
+
+# only during development
+host_key_checking = False

docs/EXAMPLES.md

@@ -0,0 +1,42 @@
+## Example configuration
+
+### [ansible-role-postgresql](https://github.com/geerlingguy/ansible-role-postgresql)
+```yaml
+# host_vars/<hostname>/vars.yaml
+---
+postgresql_databases:
+  - name: nextcloud
+    state: present
+
+  # synapse requires lc_collate and lc_ctype to be set to C
+  - name: synapse
+    lc_collate: C
+    lc_ctype: C
+    state: present
+
+postgresql_users_u:
+  - name: nextcloud
+    db: nextcloud
+    state: present
+
+  - name: synapse
+    db: synapse
+    state: present
+```
+
+```yaml
+# host_vars/<hostname>/vault
+---
+postgresql_users_e:
+  - name: nextcloud
+    password: s3cr3t-p4ssw0rd
+
+  - name: synapse
+    password: s3cr3t-p4ssw0rd
+```
+
+- You can spawn a postgres shell using: `sudo -u postgres psql`.
+- Use `\l` to list databases, `\du` to list users and `\dt` to list tables.
+- Use `\c <database>` to connect to a database
+- You can also connect using tcp (like any other application): 
+  `psql -h 127.0.0.1 -U <user> <database>`

group_vars/all/defaults.yaml

@@ -0,0 +1,42 @@
+---
+worker_user: worker
+
+# hcloud defaults for cloud server
+server_type: cax11  # this is arm64
+location: hel1
+image: ubuntu-22.04
+enable_ipv4: false
+enable_ipv6: true
+
+# ansible-role-postgresql host based authentication defaults to md5
+postgresql_auth_method: scram-sha-256
+
+# features / roles to install
+install_nginx: false
+install_postgresql: false
+install_redis: false
+
+# commands that should trigger auditd
+auditd_commands:
+  - /usr/bin/passwd
+  - /usr/bin/wget
+  - /usr/bin/curl
+  - /usr/bin/base64
+  - /bin/nc
+  - /bin/ncat
+  - /usr/bin/ssh
+  - /usr/bin/scap
+  - /usr/bin/sftp
+  - /usr/bin/ftp
+  - /usr/bin/socat
+  - /usr/sbin/tcpdump
+  - /usr/sbin/wireshark
+  - /usr/sbin/tshark
+  - /usr/sbin/rawshark
+  - /usr/bin/rdesktop
+  - /usr/bin/nmap
+  - /usr/bin/mknod
+  - /sbin/modprobe
+  - /sbin/insmod
+  - /sbin/lsmod
+  - /sbin/rmmod