iCalendar support

[towo]

Having the ability to view and accept (including sending a reply (!)) meetings on mobile would be a pretty great option.

Steps required for this:

1. ical/vcalendar parsing (copy and paste logic from e.g. GNOME Evolution (https://git.gnome.org/browse/evolution/)
2. Displaying event information in mail.
3. Add parsed event to calendar.
4. Offer to send response to organizer/all attendees.
5. Create response (yet another vcalendar) and send it, if requested.

Step four should probably not forget the security signing hook (e.g. openkeychain).

[AHoros]

From the duplicated issue #1661 I would escalate one additional simple sub-feature:

• filter inbox by emails containing attachment with *.ics name (sometimes there is more than one attachments in one email, but if at least one of them is calendar invite that we want to highlight this email)

NOTE: development could be separated into two stages: basic & advanced
with basic we can get big improvement in usability with these three sub-features:

1. ical/vcalendar parsing
2. Displaying event information in mail.
3. Filter inbox by emails containing attachment with ical/vcalendar files.

And for advanced and more complex development:
4. Offer to send response to organizer/all attendees.
5. Create response (yet another vcalendar) and send it, if requested.
6. the security signing hook (e.g. openkeychain).

[philipwhiuk]

In terms of quick off-hand estimation

• Basic is likely to be 1 & 2. (without 2, 1 is pointless). We don't really do huge feature branches here (we did one for PGP-MIME but that's an exceptional case) - this will likely be a single PR.
• 3 is complex.
• 4 is probably fairly simple
• 5 is probably fairly simple
• 6 I have no idea on.

The ordering of 3, 4, 5 and 6 will depend on usage. Personally, sending a response is more useful to me than filtering email by attachment (which is a pre-requisite for 3 I don't think we currently support) so I'd be inclined to do 4 and 5 first. But I'm not picking this up as there's lots of higher priority things to fix for me.

[philipwhiuk]

Potential library for iCal parsing: https://github.com/mangstadt/biweekly

[philipwhiuk]

Seeing noname.ics in my email inbox keeps bugging me so I started looking into this.

I've done some bits of work to K-9 for a while now, but the handling of Part's actual data still confuses me. I'm not quite sure yet where the parsing of the ICS file ought to take place yet.

What I do know is that the ICS file has a lot of complexity to it! There's lots of properties that probably most people never use. And there's a fair bit people do use.

I suspect a first PR will add basic support for simple events with participants.

We certainly don't need to deal with every field. Firstly because it's an email app, so stuff like Alarms is obviously out of scope. Secondly because supporting endless complexity is burdensome. And finally because it's a mobile client, so trying to fit in lots of minor information is going to use up valuable real estate.

Given ICS supports attachments inline there will need to be tests done on iCalendar files with large attachments to make sure we don't crash K-9 trying to parse an ICS file.

[Valodim]

Parsing this is probably best implemented by creating a new type of Viewable, similar to how mail headers are displayed in a special manner

[philipwhiuk]

So I didn't go down the Viewable route. It is a new Part type but in several places we make the assumption that viewable actually means HTML / TEXT / WebView capable. So instead I added some extra arguments.

Someone familiar with the database/LocalPart code will probably tell me I'm doing stuff in the wrong place. I know the constructor of ICalPart is doing stuff it really shouldn't. What I don't know is quite where that logic does belong.

Anyway: spike development: https://github.com/philipwhiuk/k-9/commits/ICSsupport

And fancy image:

Note that the names are fine in reality - I just blanked them except for mine.
That's an actual Outlook iCalendar attachment sent to another of my accounts, forwarded from me to me, being parsed and displayed.

I still need to do lots of work (in addition to feedback on LocalParts etc):

• I need to add a way to download the ICS file again. It no longer shows as an attachment.
• It would be nice to get the usernames displayed in a way that you can send emails to them.
• I appear to have broken Textual parts (I noticed this as wrote this comment)...
• I need to make sure forwarding sends the ICS file as an attachment.
• I should add the 'Organizer' field if it's not the sender.

[philipwhiuk]

Assigned to me as I'm actively developing this.

I fixed Textual parts mostly. However:

• Device crashing on sending e-mail (might be unrelated)
• NPE for HTML emails - definitely related - source below.

com.fsck.k9.mailstore.LockableDatabase$WrappedException: com.fsck.k9.mail.MessagingException: Error appending message: **subject**
    at com.fsck.k9.mailstore.LocalFolder$12.doDbWork(LocalFolder.java:1090)
    at com.fsck.k9.mailstore.LocalFolder$12.doDbWork(LocalFolder.java:1078)
    at com.fsck.k9.mailstore.LockableDatabase.execute(LockableDatabase.java:275)
    at com.fsck.k9.mailstore.LocalFolder.storeSmallMessage(LocalFolder.java:1078)
    at com.fsck.k9.controller.MessagingController$12.messageFinished(MessagingController.java:1406)
    at com.fsck.k9.mail.store.imap.ImapFolder.fetch(ImapFolder.java:770)
    at com.fsck.k9.controller.MessagingController.downloadSmallMessages(MessagingController.java:1393)
    at com.fsck.k9.controller.MessagingController.downloadMessages(MessagingController.java:1196)
    at com.fsck.k9.controller.MessagingController.synchronizeMailboxSynchronous(MessagingController.java:953)
    at com.fsck.k9.controller.MessagingController$9.run(MessagingController.java:739)
    at com.fsck.k9.controller.MessagingController.runInBackground(MessagingController.java:204)
    at com.fsck.k9.controller.MessagingController.access$000(MessagingController.java:115)
    at com.fsck.k9.controller.MessagingController$1.run(MessagingController.java:173)
    at java.lang.Thread.run(Thread.java:761)
Caused by: com.fsck.k9.mail.MessagingException: Error appending message: **subject**
    at com.fsck.k9.mailstore.LocalFolder.saveMessage(LocalFolder.java:1326)
    at com.fsck.k9.mailstore.LocalFolder$14.doDbWork(LocalFolder.java:1187)
    at com.fsck.k9.mailstore.LocalFolder$14.doDbWork(LocalFolder.java:1182)
    at com.fsck.k9.mailstore.LockableDatabase.execute(LockableDatabase.java:275)
    at com.fsck.k9.mailstore.LocalFolder.appendMessages(LocalFolder.java:1182)
    at com.fsck.k9.mailstore.LocalFolder.appendMessages(LocalFolder.java:1109)
    at com.fsck.k9.mailstore.LocalFolder$12.doDbWork(LocalFolder.java:1082)
    ... 13 more
Caused by: java.lang.NullPointerException: Attempt to invoke interface method 'boolean java.util.List.add(java.lang.Object)' on a null object reference
    at com.fsck.k9.mail.internet.MessageExtractor.findHtmlPart(MessageExtractor.java:385)
    at com.fsck.k9.mail.internet.MessageExtractor.findViewablesAndAttachments(MessageExtractor.java:159)
    at com.fsck.k9.message.extractors.AttachmentCounter.getAttachmentCount(AttachmentCounter.java:33)
    at com.fsck.k9.mailstore.LocalFolder.saveMessage(LocalFolder.java:1259)
    ... 19 more

[xenithorb]

I assume none of this has shipped yet in 5.115 on F-Droid?

The behavior I'm noticing is that K9 recognizes that there's an embedded .ics file but is always zero-byte and crashes the calendar handler.

However, for actual ics attachments, as such from gmail and friends, the handler correctly passes off the attachment to the calendar app and the appointment is correctly added.

So I assume this is all about parsing the embedded "non-attached" ics?

[philipwhiuk]

Correct - it's not been merged. It won't be merged until after the next stable release (5.2x) because I really want us to do a release, we are way overdue, and I don't create new blockers.

I'm not exactly sure what you mean by embedded - Outlook for example still creates attachments, it's just the UI presents it nicely. K-9 certainly should be passing off the ICS files correctly to the handler in 5.115

[brad-x]

Another mechanism in use is to simply put the vcalendar contents into the body of an email message (this is not Outlook compatible but seems to be in adoption among other mail clients/systems).

It looks like K-9 represents both the inline BEGIN:VCALENDAR and the MIME attachment as two separate attachments. So I think that's in the realm of what he means.

[Anutrix]

Is this planned before the name change to Thunderbird this year(2024)?

[Kevinf63]

Can't say I can contribute to the technical discussion but I can outline the problem I'd love solved by this.

I'm greatly missing this key functionality and actively looking for alternative mail apps for the interim. I tend to get a number of emails during the day and it's very easy to miss an appointment or meeting as I haven't manually added the attached ICS file to my calendar app of choice.

Also, typically when you add ICS files like this, the wider context is not held in memory by the external calendar app for good reason, so if updates are made to the event, there will be duplicates in your calendar when attached and careful, manual remediation is required.