PietSmiet Aggregator is a selfhostable web application that aggregates all the videos, streams and additional content from PietSmiet and displays them in a nice overview.
⭐ Dashboard - Display all PietSmiet videos and streams in a nice overview.
⭐ API - Free and Public JSON HTTP API.
⭐ PietSmietDE Import - Import all videos and news from the PietSmiet website.
⭐ YouTube Import - Import all community posts from the PietSmiet YouTube channel.
⭐ Reddit Import - Import trending posts from r/pietsmiet.
⭐ Instagram Import - Import all posts.
⭐ Streamingplan - Analyze the Streamingplan with OpenAI Vision and import scheduled streams.
⭐ Full Control - Selfhostable, Open Source and Dockerized.
First, download and install Docker.
Create a .env
file in the root directory of the project. You can use the .env.example
file as a template.
If you do not want to use certain features (for example the OpenAI Streamingplan Analysis), you can skip the corresponding API keys.
A docker-compose.yml
file is provided to run the application. It will start the application and a MySQL database.
docker-compose up -d
👀 You can view the application at http://localhost:5650
.
You can view the docker logs with docker-compose logs -f
.
Keep in mind that some data imports might take a while. You can view the progress of the data imports in the docker logs.
Most data imports are executed periodically. Wait at least an hour before you start to worry about missing data.
That's all you need to know to start! 🎉
Feel free to contribute to this project. Please read the contribution guidelines before submitting a pull request.
This project has four main components:
- Frontend: The frontend is a SvelteKit application. It is located in the
src/psaggregator
directory. - Dataimporter: The dataimporter is a small project containing multiple python scripts. These scripts are used to import data from the PietSmiet website and various other APIs. The scripts are located in the
src/dataimporter
directory. - NGINX: NGINX is used as a reverse proxy for the frontend. The configuration is located in the
src/nginx
directory. - Database: The database is a MySQL database. The schema and data are located in the frontend project
- YouTubeOperationalAPI: The YouTubeOperationalAPI is a Open Source Solution by Benjamin Loison. It supports more features than the official public YouTube API.
Depending on the component you want to work on, you will need the following tools:
- Docker: Docker
>= 24.0.0
and docker-compose>= 2.23.0
. - Frontend: Node.js
>= 20.0.0
and npm>= 9.6.0
. - Dataimporter: Python
>= 3.10.0
.
First, clone the repository:
git clone https://github.com/zaanposni/psaggregator.git
Then, create a .env
file in the root directory of the project. You can use the .env.example
file as a template.
If you do not want to use certain features (for example the OpenAI Streamingplan Analysis), you can skip the corresponding API keys.
Start the MySQL database and YouTubeOperationalAPI:
docker-compose -f docker-compose.dev.yml up -d
Setup the frontend
cd src/psaggregator
npm install
npm run prismagenerate
npm run prismamigrate # might be needed if the database is not initialized yet
npm run dev
Setup the dataimporter by setting up the environment variables like in the .env.example
file.
The dataimporter uses numerous APIs to import data.
You might want to create API keys for the APIs you want to use and skip the ones you don't want to use.
Start the dataimport scripts (the process is the same for all kind of imports)
cd src/dataimporter
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python3 pietsmietdeuploadplan.py # uploadplan does not need any API key. However - for example - reddit.py does
If you want to say thank you or/and support active development of psaggregator
:
- Add a GitHub Star to the project.
- Support me on Ko-fi.
And now, I invite you to participate in this project! Let's work together to create the most useful tool for all PietSmiet Enjoyers.
- Issues: ask questions and submit your features.
- Pull requests: send your improvements to the current.
- Mail: send your ideas for the project.
- Discord: add me as a friend on Discord: @zaanposni
Together, we can make this project better every day!
- discord-masz - MASZ is a selfhostable highly sophisticated moderation bot for Discord. Includes a web dashboard and a discord bot.
psaggregator
is free and open-source software licensed under
the GPL-3.0 License.