Skip to content

This is a worker service for the TimeLiner application. It collects, and filters the data from external API service and store it in the database as well as manages it and serves as first level source/data provider on which the TimeLiner GraphQL API depends.

License

Notifications You must be signed in to change notification settings

happy-mammal/timeliner-worker-service

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Heroku

timeliner-worker-service

This is a worker service for the TimeLiner application. It collects, and filters the data from external API service and store it in the database as well as manages it and serves as first level source/data provider on which the TimeLiner GraphQL API depends.

Functions / Tasks

  • Executes the script based on cron scheduling.
  • Fetches the data for specified category from Gnews API.
  • Filters the data as per the custom requirements.
  • Restructures the json object with custom data added to it.
  • Manages the firestore document size and pushes the indexes(helful in serching) of the data in the document(termed as store).
  • Pushes the indexed data to the firebase realtime database.
  • Automatiic creation and updation of new firestore document(termed as store) when no enough space is avaiable.

Implementations

  • Firebase Admin SDK (firebase-admin)
  • HTTP Requests (node-fetch)
  • Cron Job /Scheduling (node-cron)
  • Firestore-Size (firestore-size)
  • Used Enviroment Variables where ever possible (dotenv)

Where to use ?

You can use this if you want to:

  • Fetch and store the data from an external API to Firebase Realtime Database / Firestore (Using Admin SDK).
  • Modifiy the data and restructure the json object as per your need.
  • Reduce the firestore read/writes/deletes.
  • Manage and use firestore efficiently for a longer period.
  • Execute the scripts/functions at scheduled intervals.

Folder Structure

.
├── config
│   └── config.js
├── utils
│   ├── keywordsGenerator.js
│   └── queryDateGenerator.js
├── .env
├── .gitignore
├── app.js
├── LICENSE
├── package-lock.json
├── package.json
├── Procfile
├── README.md
└── script.js

Techs Used

  • Firesbase
  • Node.js
  • Heroku
  • GNews API

License

About

This is a worker service for the TimeLiner application. It collects, and filters the data from external API service and store it in the database as well as manages it and serves as first level source/data provider on which the TimeLiner GraphQL API depends.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published