Bi-directional kobo notes and highlights sync

[atrauzzi]

Is your feature request related to a problem? Please describe.
I'd really love to be able to have a desktop ebook utility that would sync down any content I author on my Kobo libra colour. While free e-ink devices are starting to creep into the market, it'll likely be a while before they reach the level of polish that the more popular manufacturers are offering today. Particularly as it pertains to things like stylus support.

Describe the solution you'd like
When I connect my Kobo libra colour to my machine, I'd like foliate to sync highlights and note data down from the device. The following list of details are probably worth considering:

• An internal format for highlights, markings and standalone notes
  • Power feature: Support syncing out to external tools like miro
• External device support
• Maintaining association between highlights and content
• Establishing and preserving uniqueness of content from devices
• Determining when content has changed, either through something like an etag, or by a hash
• Rendering highlights and markings from within the foliate experience
• Allowing for copy/paste functionality from within foliate
  • Power feature: If I can copy pen strokes from foliate and paste them into miro? 🤯
• Power feature: Editing from within foliate and syncing back to the device
  • This is less important than simply ensuring I can easily get data off the device
• Power feature: Writing some kind of application to run on-device to push data to some kind of server or service that foliate could then pull from

Describe alternatives you've considered
Nothing.

[johnfactotum]

It would probably work better as a separate program. One can just read and write to Foliate's JSON files (there's a bug since v3 where if you've edited the JSON files manually, it won't automatically reload; so one must make sure to sync only when the books are closed). There's some info on how Kobo highlights work here: https://github.com/valeriangalliat/kobo-highlights-to-calibre. The main thing is that Kobo calculates offsets differently. Other than that it seems to be relatively straightforward.

But the above only applies to simple highlights and notes. From what you wrote I assume you're talking about freehand drawings. According to https://www.reddit.com/r/kobo/comments/1cixff9/exporting_handwritten_annotations_markup_on_kepub/, those markings are applied to screenshots of the book. If so, it wouldn't really be possible to render them in Foliate. Well, I suppose nothing is really impossible if one tries hard enough...

• Making the drawings reflowable can probably be done with the level of AI technologies today. The key is that the drawings must be transformed from static images into programs, or functions, that output images. Then one just have to make Folaite evaluate these programs. Needless to say, this is quite involved. To minimize security risks, one might even have to invent a DSL, so as not to require evaluating arbitrary JavaScript functions.
• Or, as a simpler alternative, one can convert the hand-drawn notes into normal highlights. This would still require very sophisticated OCR, though.