This ansible role installs a Apache Airflow server in a Debian/Ubuntu environment.
These instructions will get you a copy of the role for your ansible playbook. Once launched, it will install Apache Airflow in a Debian or Ubuntu system.
Ansible 2.9.9 version installed. Inventory destination should be a Debian (preferable Debian 10 Buster ) or Ubuntu environment.
ℹ️ This role should work with older versions of Debian but you need to know that due to Airflow minimum requirements you should check that 🐍 Python 3.6 (or higher) is installed before (👉 See: Airflow prerequisites).
ℹ️ By default this role use the predefined installation of Python that comes with the distro.
For testing purposes, Molecule with Docker as driver.
Create or add to your roles dependency file (e.g requirements.yml) from GitHub:
- src: http://github.com/idealista/airflow-role.git
scm: git
version: 2.0.0
name: airflow
or using Ansible Galaxy as origin if you prefer:
- src: idealista.airflow-role
version: 2.0.0
name: airflow
Install the role with ansible-galaxy command:
ansible-galaxy install -p roles -r requirements.yml -f
Use in a playbook:
---
- hosts: someserver
roles:
- { role: airflow }
Look to the defaults properties files to see the possible configuration properties, take a look for them:
main.yml
for airflow general purpose.airflow-cfg.yml
for all the related airflow.cfg config parameters.webserver-config-py.yml
for all the related webserver_config.py config parameters.
👉 Don't forget :
- 🦸 To set your Admin user.
- 🔑 To set Fernet key.
- 🔑 To set webserver secret key.
- 📝 To set your AIRFLOW_HOME and AIRFLOW_CONFIG at your own discretion.
- 📝 To set your installation and config skelton paths at your own discretion.
- 👉 See
airflow_skeleton_paths
inmain.yml
- 👉 See
- 🐍 Python and pip version.
- 📦 Extra packages if you need additional operators, hooks, sensors...
- 📦 Required Python packages with version specific like SQLAlchemy for example (to avoid known Airflow bugs❗️) like below or because are necessary
⚠️ With Airflow v1.10.0, PyPi packagepyasn1
v0.4.4 is needed. See examples below
airflow_required_python_packages
should be a list following this format:
airflow_required_python_packages:
- { name: SQLAlchemy, version: 1.3.23 }
- { name: psycopg2 }
- {name: pyasn1, version: 0.4.4}
airflow_extra_packages
should be a list following this format:
airflow_extra_packages:
- apache.atlas
- celery
- ssh
👉 For more info about this extra packages see: Airflow extra packages
pipenv install -r test-requirements.txt --python 3.7
# Optional
pipenv shell # if in shell just use `molecule COMMAND`
pipenv run molecule test # To run role test
# or
pipenv run molecule converge # To run play with the role
For the versions available, see the tags on this repository.
Additionally you can see what change in each version in the CHANGELOG.md file.
- Idealista - Work with - idealista
See also the list of contributors who participated in this project.
This project is licensed under the Apache 2.0 license - see the LICENSE file for details.
Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.