This project aims to extract hiking routes data from the websites SAC, Schweizmobil, and Komoot. The data includes route descriptions, difficulty, distance, elevation change, and other relevant information.
To run the project, you need to have Python 3 installed on your machine, as well as the following packages:
- selenium
- beautifulsoup4
- pandas
- mariadb
- plus a few others... (see
requirements.txt
)
We recommend using virtualenv to manage dependencies.
- Clone the repository to your local machine.
- Create a virtual environment using virtualenv.
- Activate the virtual environment.
- Install the dependencies using
pip install -r requirements.txt
. - Run the code.
The repository is organized as follows:
docs
: documentation filessrc
: source code filesdatabase
: database-related codeextractors
: website-specific extraction codeKomootExtractor.py
: extractor for Komoot websitesac
: extractor for SAC websiteschweizmobil
: extractor for Schweizmobil websiteSeleniumUtil.py
: utility functions for Selenium web driver
loaders
: code to load data into databasemodels
: data modelsoutput
: output filestransformers
: code to transform datamain.py
: main script to run the data extraction process
Makefile
: makefile with common tasksrequirements.txt
: list of required packagesREADME.md
: this file
If you find any issues or have suggestions for improvement, please feel free to submit a pull request or open an issue.
This project is licensed under the MIT License. See the LICENSE file for details.