Use FreeBusy for schedule availability

[MelissaAutumn]

Fixes #662

Requesting caldav testers from: @devmount

This uses the freebusy api for Google and tries to use the freebusy api for CalDav. If the CalDav request fails it will use the old way of querying all of the events. This might give differing results since we can't calculate busyness based on attendance for caldav with the querying events method.

I need to cover the error conditions on the google side, and maybe look into re-architecture how the connectors work (Maybe steal how caldav has principal calendars and group them like that, but dunno yet.) And also add tests.

Why do we want to use this api instead of just calculating the availability based on the list of events? Because it's faster, and we can batch up the requests on the Google side for a significant speed up.

Caldav Ref:
https://caldav.readthedocs.io/en/latest/caldav/objects.html#caldav.objects.Calendar.freebusy_request

ICalendar Object:
https://icalendar.readthedocs.io/en/latest/api.html#icalendar.cal.FreeBusy

Google API Ref:
https://developers.google.com/calendar/api/v3/reference/freebusy/query

[devmount]

First of all: The implementation works. I tested this with my Google calendar and with a custom Nextcloud CalDAV calendar. The results are surprising. Numbers in milliseconds. Measured waiting time on the schedule/public/availability call on page reload without cache. Averaged over 15 measurements each.

CalDAV freebusy	CalDAV legacy	Google freebusy	Google legacy
915	61	447	68

The freebusy implementation is 93% slower for CalDAV and 85% slower for Google on my end. I feel I might have been made a mistake somewhere? Maybe this is depending on the event count in the availability timeframe? I have around 30 events in the requested time frame on each calendar.

I'm happy to do more testing if we need it.

[MelissaAutumn]

Oh the old method caches for 15 minutes after the first load. Try removing the redis url from env 😄

[devmount]

Hahaha well, this is a bit emparrassing. I mistook the @limiter rule (which I commented out) on the endpoint for caching instead of rate limiting 🙈 Totally my bad 😂 I took the measurements for the legacy cases again, this time without redis 😅 But turns out the freebusy API doesn't have any significant effect not on Google nor on CalDAV on my end:

CalDAV freebusy	CalDAV legacy	Google freebusy	Google legacy
915	918	447	451

I'm using a Nextcloud CalDAV calendar with around 30 events per month. Happy to do more testing if needed!

[MelissaAutumn]

Hahaha well, this is a bit emparrassing. I mistook the @limiter rule (which I commented out) on the endpoint for caching instead of rate limiting 🙈 Totally my bad 😂 I took the measurements for the legacy cases again, this time without redis 😅 But turns out the freebusy API doesn't have any significant effect not on Google nor on CalDAV on my end:
CalDAV freebusy CalDAV legacy Google freebusy Google legacy
915 918 447 451

I'm using a Nextcloud CalDAV calendar with around 30 events per month. Happy to do more testing if needed!

How many calendars are you testing? The efficiency gain here is batching calendars together for availability. (Which I found with 4 google calendars)

And no worries, we should probably add a decorator for caching at route level sometime soon

[devmount]

One calendar each. I'll repeat the tests with more calendars then! Does it matter, if CalDAV calendars are from the same principal?

[MelissaAutumn]

One calendar each. I'll repeat the tests with more calendars then! Does it matter, if CalDAV calendars are from the same principal?

Yeah they have to be from the same user/pass/url in order for that to work, although I don't think batched CalDAV calendars are supported yet. (They should be with a little extra work once the CalDAV PR is merged) The main issue was that we're calling each calendar individually to get their events and then we calculate the busy time ourselves. This should basically tells the server to do this for us.

[devmount]

I repeated CalDAV tests with 5 separate CalDAV calendars (from the same domain) and ~20 events in the requested time frame per calendar. This is the result:

CalDAV freebusy	CalDAV legacy
5502	6135

So, there is an improvement of ca. 630ms in this scenario, which is good. We might gain more if we support batched CalDAV calendars though.