Add series rss feed to share button (#959)

This adds RSS feeds for series in tobira as outlined in #791.
It is still missing the caching that @LukasKalbertodt mentions in the
issue, which is to be done in a follow up PR.

~~- [ ] This is also missing a mechanism for updating the feed when new
events are added, which I realize makes these feeds kind of useless at
the moment. At least that should still be added in this PR.~~
~~- [ ] I don't know exactly yet how we can tell feed readers that the
feed has new content, or if that is even necessary.~~
Edit 2: My limited knowledge about RSS led to the above assumptions, but
after doing some testing as described below it turns out we don't need
to implement anything like that.
- [x] And I also still need to look into some specific tags like the
`<itunes:...>` ones that are present in the feeds of the current ETH
video portal. They aren't part of the current (but pretty much ancient)
RSS specs but might be a worthwhile, if not mandatory addition (even if
they are only necessary for the feeds to work with itunes).

We need to test how the generated xml works in different RSS readers
(they all appear to be doing their own thing), and probably adjust a
couple of things based on these tests.
- Edit: I tested different desktop/web ("Feedly", "Inoreader", "The Old
Reader", "CommaFeed" and mobile (android "RSS Reader", "Feeder") feed
readers. Some findings:
- the desktop readers don't show thumbnails, which kinda sucks (mobile
readers do tho). There is a workaround in adding an image to the
`description`, but that would also clutter the entry itself. I also
considered adding an embedded video to each item, but besides the
cluttering argument, the items a) link to the videos, b) some readers
already embed the videos automatically and c) all readers also feature a
download option (not sure if that can be disabled, which might be an
issue seeing as some institutions don't want their videos downloadable).
- when adding a video to a subscribed series in tobira, the mobile feed
readers update automatically. Desktop readers "Inoreader" and "The Old
Reader" also pick up on the updated feed, but updating has to be done
manually. "CommaFeed" has an option to "fetch all feeds" or sth, which
also updates the feed. "Feedly" however ~~does not update the feeds,
even when using their "refresh" button. So this might still need to be
imlemented on our side (sigh...)~~ updates automatically, but less
regularly, it appears (I have found no way of configuring this, might be
behind a paywall). Actually, now I suspect that every reader updates
automatically after a certain time period. Again - total RSS noob here.

.deployment/templates/config.toml

@@ -1,5 +1,6 @@
 [general]
 site_title.en = "Tobira Test Deployment"
+tobira_url = "https://{% if id != 'master' %}{{id}}.{% endif %}tobira.opencast.org"
 
 [general.metadata]
 dcterms.source = "builtin:source"

backend/Cargo.toml

@@ -44,6 +44,7 @@ libz-sys = { version = "1", features = ["static"] }
 log = { version = "0.4", features = ["serde", "std"] }
 meilisearch-sdk = "0.23.0"
 mime_guess = { version = "2", default-features = false }
+ogrim = "0.1.1"
 once_cell = "1.5"
 p256 = { version = "0.13.2", features = ["jwk"] }
 p384 = { version = "0.13.0", features = ["jwk"] }

backend/src/config/general.rs

@@ -1,5 +1,7 @@
 use std::collections::HashMap;
 
+use crate::util::HttpHost;
+
 use super::TranslatedString;
 
 
@@ -10,6 +12,13 @@ pub(crate) struct GeneralConfig {
     // TODO: fix automatically generated `site_title =` template output.
     pub(crate) site_title: TranslatedString,
 
+    /// Public URL to Tobira (without path).
+    /// Used for RSS feeds, as those require specifying absolute URLs to resources.
+    /// 
+    /// Example: "https://tobira.my-uni.edu".
+    pub(crate) tobira_url: HttpHost,
+
+
     /// Whether or not to show a download button on the video page.
     #[config(default = true)]
     pub show_download_button: bool,

backend/src/db/types.rs

@@ -50,7 +50,7 @@ impl fmt::Debug for Key {
 
 
 /// Represents the `event_track` type defined in `5-events.sql`.
-#[derive(Debug, FromSql, ToSql)]
+#[derive(Debug, FromSql, ToSql, Clone)]
 #[postgres(name = "event_track")]
 pub struct EventTrack {
     pub uri: String,

backend/src/http/handlers.rs

@@ -15,7 +15,7 @@ use crate::{
     db::{self, Transaction},
     http::response::bad_request,
     metrics::HttpReqCategory,
-    prelude::*,
+    prelude::*, rss,
 };
 use super::{Context, Request, Response, response};
 
@@ -90,6 +90,11 @@ pub(super) async fn handle(req: Request<Body>, ctx: Arc<Context>) -> Response {
                 .unwrap()
         },
 
+        path if path.starts_with("/~rss") => {
+            register_req!(HttpReqCategory::Other);
+            handle_rss_request(path, &ctx).await.unwrap_or_else(|r| r)
+        }
+
         // Assets (JS files, fonts, ...)
         path if path.starts_with(ASSET_PREFIX) => {
             register_req!(HttpReqCategory::Assets);
@@ -105,10 +110,7 @@ pub(super) async fn handle(req: Request<Body>, ctx: Arc<Context>) -> Response {
         // (and without our frontend!).
         "/favicon.ico" => {
             register_req!(HttpReqCategory::Other);
-            Response::builder()
-                .status(StatusCode::NOT_FOUND)
-                .body("Not found".into())
-                .unwrap()
+            response::not_found()
         }
 
         // ----- Special, internal routes, starting with `/~` ----------------------------------
@@ -176,6 +178,19 @@ pub(super) async fn reply_404(ctx: &Context, method: &Method, path: &str) -> Res
     ctx.assets.serve_index(StatusCode::NOT_FOUND, &ctx.config).await
 }
 
+async fn handle_rss_request(path: &str, ctx: &Arc<Context>) -> Result<Response, Response> {
+    let Some(series_id) = path.strip_prefix("/~rss/series/") else {
+        return Ok(response::not_found());
+    };
+
+    rss::generate_feed(&ctx, series_id).await.map(|rss_content| {
+        Response::builder()
+            .header("Content-Type", "application/rss+xml")
+            .body(Body::from(rss_content))
+            .unwrap()
+    })
+}
+
 /// Handles a request to `/graphql`. Method has to be POST.
 async fn handle_api(req: Request<Body>, ctx: &Context) -> Result<Response, Response> {
     // TODO: With Juniper 0.16, this function can likely be simplified!

backend/src/http/response.rs

@@ -27,3 +27,11 @@ pub(crate) fn internal_server_error() -> Response {
         .body("Internal server error".into())
         .unwrap()
 }
+
+pub(crate) fn not_found() -> Response {
+    Response::builder()
+        .status(StatusCode::NOT_FOUND)
+        .body("Not found".into())
+        .unwrap()
+}
+

backend/src/main.rs

@@ -29,6 +29,7 @@ mod search;
 mod sync;
 mod util;
 mod version;
+mod rss;
 
 
 #[tokio::main]

backend/src/rss.rs

@@ -0,0 +1,215 @@
+use std::{sync::Arc, future, collections::HashMap};
+use chrono::{DateTime, Utc};
+use deadpool_postgres::{GenericClient, Client};
+use anyhow::{Error, Result};
+use futures::TryStreamExt;
+use ogrim::xml;
+
+use crate::{
+    db::{types::{EventTrack, Key}, self, util::{impl_from_db, FromDb, dbargs}},
+    http::{Context, response::{bad_request, self, not_found}, Response},
+    util::HttpHost,
+    prelude::*,
+};
+
+#[derive(Debug)]
+struct Event {
+    id: Key,
+    title: String,
+    description: Option<String>,
+    created: DateTime<Utc>,
+    creators: Vec<String>,
+    thumbnail_url: Option<String>, 
+    tracks: Vec<EventTrack>,
+}
+
+impl_from_db!(
+    Event,
+    select: {
+        events.{ id, title, description, creators, thumbnail, created, tracks },
+    },
+    |row| {
+        Self {
+            id: row.id(),
+            title: row.title(),
+            description: row.description(),
+            created: row.created(),
+            creators: row.creators(),
+            thumbnail_url: row.thumbnail(),
+            tracks: row.tracks(),
+        }
+    }
+);
+
+/// Generates the xml for an RSS feed of a series in Tobira.
+pub(crate) async fn generate_feed(context: &Arc<Context>, id: &str) -> Result<String, Response> {
+    let db_pool = &context.db_pool;
+    let tobira_url = &context.config.general.tobira_url;
+    let series_link = format!("{tobira_url}/!s/{id}");
+    let rss_link = format!("{tobira_url}/~rss/series/{id}");
+
+    let Some(series_id) = Key::from_base64(id) else {
+        return Err(bad_request("invalid series ID"));
+    };
+
+    let db = db::get_conn_or_service_unavailable(db_pool).await?;
+
+    let query = "select opencast_id, title, description from series where id = $1";
+    let series_data = match db.query_opt(query, &[&series_id]).await {
+        Ok(Some(data)) => data,
+        Ok(None) => return Err(not_found()),
+        Err(e) => {
+            error!("DB error querying series data for RSS: {e}");
+            return Err(bad_request("DB error querying series data"));
+        }
+    };
+
+    let series_oc_id = series_data.get::<_, String>("opencast_id");
+    let series_title = series_data.get::<_, String>("title");
+    let series_description = series_data
+        .get::<_, Option<String>>("description")
+        .unwrap_or_default();
+
+    let format = if cfg!(debug_assertions) {
+        ogrim::Format::Pretty { indentation: "  " }
+    } else {
+        ogrim::Format::Terse
+    };
+
+    let buf = xml!(
+        #[format = format]
+        <?xml version="1.0" encoding="UTF-8"?>
+        <rss
+            version="2.0"
+            xmlns:dc="http://purl.org/dc/elements/1.1/"
+            xmlns:content="http://purl.org/rss/1.0/modules/content/"
+            xmlns:atom="http://www.w3.org/2005/Atom"
+            xmlns:media="http://search.yahoo.com/mrss/"
+            xmlns:itunes="http://www.itunes.com/dtds/podcast-1.0.dtd"
+        >
+            <channel>
+                <title>{series_title}</title>
+                <link>{series_link}</link>
+                <description>{series_description}</description>
+                <language>"und"</language>
+                <itunes:category text="Education" />
+                <itunes:explicit>"true"</itunes:explicit>
+                <itunes:image href={format!("{tobira_url}/~assets/logo-small.svg")} />
+                <atom:link href={rss_link} rel="self" type="application/rss+xml" />
+                {|buf| {
+                    video_items(buf, &db, &series_oc_id, &series_title, &rss_link, &tobira_url)
+                        .await
+                        .map_err(|e| {
+                            error!("Could not retrieve videos for RSS: {e}");
+                            response::internal_server_error()
+                        })?
+                }}
+            </channel>
+        </rss>
+    );
+
+    Ok(buf.into_string())
+}
+
+/// Generates the single video items of a series in Tobira for inclusion in an RSS feed.
+async fn video_items(
+    doc: &mut ogrim::Document,
+    db: &Client,
+    series_oc_id: &str,
+    series_title: &str,
+    rss_link: &str,
+    tobira_url: &HttpHost,
+) -> Result<(), Error> {
+    let selection = Event::select();
+    let query = format!("select {selection} from events where part_of = $1");
+    let rows = db.query_raw(&query, dbargs![&series_oc_id]).await?;
+
+    fn map_tracks(tracks: &[EventTrack], doc: &mut ogrim::Document) {
+        xml!(doc,
+            <media:group>
+                {|doc| for track in tracks {
+                    xml!(doc,
+                        <media:content
+                            url={track.uri}
+                            {..track.mimetype.as_ref().map(|t| ("type", t))}
+                            {..track.resolution.into_iter().flat_map(|[w, h]| [("width", w), ("height", h)])}
+                        />
+                    )}
+                }
+            </media:group>
+        )
+    }
+
+    rows.try_for_each(|row| {
+        let event = Event::from_row_start(&row);
+
+        let mut buf = [0; 11];
+        let tobira_event_id = event.id.to_base64(&mut buf);
+        let event_link = format!("{tobira_url}/!v/{tobira_event_id}");
+        let thumbnail = &event.thumbnail_url.unwrap_or_default();
+        let (enclosure_track, track_groups) = preferred_tracks(event.tracks);
+
+        xml!(doc,
+            <item>
+                <title>{event.title}</title>
+                <link>{event_link}</link>
+                <description>{event.description.unwrap_or_default()}</description>
+                <dc:creator>{event.creators.join(", ")}</dc:creator>
+                <pubDate>{event.created.to_rfc2822()}</pubDate>
+                <guid>{event_link}</guid>
+                <media:thumbnail url={thumbnail} />
+                <itunes:image href={thumbnail} />
+                <enclosure
+                    url={&enclosure_track.uri}
+                    type={&enclosure_track.mimetype.unwrap_or_default()}
+                    length="0"
+                />
+                <source url={rss_link}>{series_title}</source>
+                {|doc| for (_, tracks) in &track_groups {
+                    map_tracks(tracks, doc)
+                }}
+            </item>
+        );
+
+        future::ready(Ok(()))
+    }).await?;
+
+    Ok(())
+}
+
+
+
+/// This returns a single track for use in the enclosure, that:
+/// a) is a `presentation` track.
+/// b) has a resolution that is closest to full hd.
+/// Defaults to any track meeting the b) criteria if there is no `presentation` track.
+/// It also returns a hashmap of all tracks grouped by their flavor.
+fn preferred_tracks(tracks: Vec<EventTrack>) -> (EventTrack, HashMap<String, Vec<EventTrack>>) {
+    let target_resolution = 1920 * 1080;
+
+    let mut preferred_tracks: Vec<EventTrack> = tracks
+        .iter()
+        .filter(|track| track.flavor.contains("presentation"))
+        .cloned()
+        .collect();
+
+    if preferred_tracks.is_empty() {
+        preferred_tracks = tracks.clone();
+    }
+
+    let enclosure_track = preferred_tracks.iter().min_by_key(|&track| {
+        let [w, h] = track.resolution.unwrap_or_default();
+        (w * h - target_resolution).abs()
+    }).expect("event without tracks");
+
+    let mut track_groups: HashMap<String, Vec<EventTrack>> = HashMap::new();
+
+    for track in &tracks {
+        let flavor = track.flavor.clone();
+        let entry = track_groups.entry(flavor).or_insert(Vec::new());
+        entry.push(track.clone());
+    }
+
+    (enclosure_track.clone(), track_groups)
+}
+