Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a backend service for accelerating Channel discovery responsiveness #13

Open
6 tasks
mblomdahl opened this issue Aug 29, 2024 · 0 comments
Open
6 tasks
Assignees
Labels
boost Boosting Lizo'nikah

Comments

@mblomdahl
Copy link

mblomdahl commented Aug 29, 2024

As an alternative approach to waiting for acceptance and roll-out of changes in #11 and #12 (and then wait for all the video site maintainers to actually upgrade their servers with newer PeerTube software), we can deploy a backend service that queries a PeerTube server for all channels and return the subset that matches our criteria, i.e. 1) is local and 2) includes 1 or more public video. The workaround could also be applicable for #9 and #10 in the future, if we struggle with getting enhancements adopted by the PeerTube maintainers.

Functional requirements:

  1. Send a GET <owntube-backend>/api/peertube/<instance_hostname>/api/v1/video-channels?isLocal=true&hasPublicVideos=true
  2. Expect a response that includes all local channels with 1 or more public video, using same data structure as the native /api/v1/video-channels API endpoint

Non-functional requirements:

  • Implemented in Node 22 with NestJS and Typescript
  • No robust database persistence in initial version, use a container-local SQLite database
  • Channel discovery results for a host should be cached and reused between requests, i.e.
    1. in request handler; request received → emit a backend-local "check cache for instance_hostname Channels" event → return the cached results
    2. in backend-local subscriber for the "check cache for instance_hostname Channels" event; a) check when was the site last queried for channels and if it was more than 1 week ago or b) send a request and check the first page of results ordered by updatedAt (descending); then if any of a) or b) are true emit a backend-local "update cache for instance_hostname Channels" event
    3. in backend-local subscriber for the "update cache for instance_hostname Channels" event, run the Channel discovery on the target site and update the cache with new results and new cache update timestamp
  • If no Channel discovery result is available in the backend-local storage, it should run the Channel discovery on the target site synchronously, cache it, and then return the results to the client
  • Rate limits on the PeerTube API must be respected (dynamically, subject to rate limiting response headers)
  • GitHub Actions CI/CD pipeline for deploying it to OwnTube.tv Kubernetes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
boost Boosting Lizo'nikah
Projects
Development

No branches or pull requests

2 participants