Project structure:
-
Custom NodeJS calendar code, which retrieves info from RSS, APIs and/or scraping (With Axios, Cheerio and Puppeteer), stores in
DuckDBSQLite for further analysis. Deploys a simple HTML file into the CMS assets folder via simple FTP connection.Be aware: DuckDB does not work in Musl systems, like Alpine Linux.
- Switched from DuckDB to SQLite as the former still requires some work to use with DBveaber and other tools. DuckDB can work with SQLite as an OLAP layer on top, if I want that capability in the future.
-
PicoCMS for managing content. Very low complexity PHP CMS that runs fast in my already purchased shared hosting. Reads Markdown and HTML, so no database. No automatic updates, no backoffice, etc. Very low surface of attack --> Peace of mind.
-
Asumes you're using Debian 12+. Get your version with
cat /etc/os-release
. -
Assumes you have Git 2.38+, NodeJS 18.14+ and npm 9+ installed (check with
packageName --version
or install withapt install packageName
). -
Assumes you have Chromium installed.
-
Assumes you already installed the inference server (
../inference/readme.md
)
-
Create a directory
cometocoruna
withmkdir ./cometocoruna
->cd cometocoruna
. -
Clone the repo with
git clone https://github.com/iagovar/cometocoruna.git ./
. -
Go to the calendar directory with
cd ./calendar
-
Install all dependencies with
npm install
. -
You'll need to configure the
authentication.config.json
file. An example is provided asauthentication.config.json.template
as I can't upload my credentials to a public repo for obvious reasons. -
Try running it with
npm start
. There should be no errors in the first run (it may be in subsequent errors because the Database won't allow duplicate primary keys, but the script will continue doing its task). -
Time to make it execute periodically with CRON.
-
Type
crontab -e
to open the crontab editor (likely in vi). -
Enter the vi editor mode pressing
i
. -
Paste this lines to run the script every day on 6AM:
# Run the cometocoruna.com calendar every day on 6AM 0 6 * * * cd ~/cometocoruna/calendar/ && npm start >> debug.log 2>&1
-
Exit the vi editor mode with
esc
and write changes with:wq
. -
If you write in the console
crontab -l
it should list your crontab with the changes you just introduced.
-
Here's a good resource for understaing CRONTAB: Link.
-
There's also a very useful Crontab generator.
-
-
Any update in the script will require a
git pull
in~/cometocoruna
to bring it to local.
It should run in pretty much any standard php shared hosting, really. I'm actually running it in a low-budget one, so, in general, just upload by FTP, as PicoCMS requires no installation.