Added option in order to synchronize only selected shelves on Kobo device

[alfred82santa]

• Added a flag to shelves in order to set whether they must be synchronized with Kobo device.
• Only books on kobo shelves are synchronized.
• Shelves modified after last sync with no kobo sync flag are remove from Kobo device.
• Update db on startup: kobo_sync field to shelf table.
• In order to keep old behavior, if no shelves are marked to kobo sync, all books and shelves are synchronized.

Maybe, it solves/helps: #1276

[ghost]

I was just thinking how this would be a perfect solution to multi user Kobo sync

[pulitux]

I like this aproach, almost perfect solution to large libraries sync

[alfred82santa]

@OzzieIsaacs could you check it?

[pulitux]

I've not so much spare time, but i'll try. i'm running a docker container but I'll try to make an image

[rolfberkenbosch]

I have tested your code, nice work!!!

I found a small flaw, when you have removed books from the shelves and from your kobo. The cps keep syncing some information about the removed books from the shelves. That kind of information will eventually overwhelming the the sync duration/time and gives timeouts. I have tested that with alot of books and can confirm that. For each book that have been deleted from the self you will see the following information in the sync output:

   {
      "ChangedReadingState":{
         "ReadingState":{
            "Created":"2021-01-18T20:03:46Z",
            "CurrentBookmark":{
               "LastModified":"2021-03-12T21:01:19Z"
            },
            "EntitlementId":"67b43ab5-ebad-4dae-8aed-ffa3ac26a3f8",
            "LastModified":"2021-03-12T21:01:19Z",
            "PriorityTimestamp":"2021-03-12T21:01:19Z",
            "Statistics":{
               "LastModified":"2021-03-12T21:01:19Z"
            },
            "StatusInfo":{
               "LastModified":"2021-03-12T21:01:19Z",
               "Status":"ReadyToRead",
               "TimesStartedReading":0
            }
         }
      }
   },

Can you remove that kind of information so that the sync keeps clean and fast only for the books that are on the shelves.

[alfred82santa]

Could you try it again, please? I think I've solved it.

[mlaccetti]

Any chance this is going to get merged soon? Super excited to finally be able to sync to my Kobo!

[OzzieIsaacs]

I have to finish some test on the development branch and then I’m ready for this one

[rolfberkenbosch]

I have tested this version and looks ok for me

[OzzieIsaacs]

I started working on it. My findings so far:

• you removed new_tags_last_modified = max(shelf.last_modified, new_tags_last_modified) from line 663 which causes deleted Tags to be synced every time unless you do another shelf operation
• I replaced the join in line 690 for shelf in ub.session.query(ub.Shelf).join(ub.BookShelf).filter( with an outer join, otherwise a new empty shelf isn't synced.

There is nothing to do for you right now, I fiexed both and will now start testing the new functionality

[alfred82santa]

Well, about the first issue. I decided to remove all shelves because there is no way to know which shelves had have the flag in previous sync, and not in current sync. I thought it was low price to allow to remove synchronised shelves.
About second issues. Do you think we need to sync empty shelves?

[OzzieIsaacs]

Second issue: it‘s a change to the previous behavior, I‘m running automated tests for it, I think I checked it with the real sync for how it should work

[OzzieIsaacs]

Regarding first issue, I think we might misunderstood each other. My use case is the following:
I'm having a shelf and no shelf is ticked "synchronize to Kobo", in this "mode" all shelfs and all books are getting synchronized. During sync this shelf will get transferred to Kobo (correct). Now I delete the shelf and sync again. The shelf will be synced as "deleted" shelf (correct). If I now do nothing but another sync, nothing should be synced (behaviour before) and from my point of view the right way. Because you deleted the above line kobo doesn't update the sync time for shelfs and the deleted shelf is again synced. Be putting back the line into the code, the shelf will be reported as deleted during one sync and nothing more. This has nothing to do with your changes, it's just the old functionality.

[OzzieIsaacs]

One more thing:
Before the code change we had one use case:
All books and shelfs are synced to Kobo.

Now we are having 3 use cases:
All books are synced to Kobo, because there is no shelf
All books are synced to Kobo, because no shelf has a set the option Synchronize with Kobo device
Only books in shelfs with option Synchronize with Kobo device are synced ot Kobo.

If no Shelf has set the option Synchronize with Kobo device the user sees the following screen during editing of Shelf propertiers:

But the message is wrong, because this shelf is going to be synchronized, as long there is no other shelf having the Synchronize with Kobo device option set.
I think this could confuse users. There is no place where you can see which of the above use cases you are currently in, you have to go through every Shelf to find out what's going to be synced.

What do you think?
To avoid this I split the message into 3, depending on the current status:
Sync with Kobo device - If this option is turned off, all books and shelves will be synced with Kobo -> displayed if currently displayed shelf it the only one to be synced
Only sync books from selected shelves to Kobo instead of all books and shelves -> displayed if no shelf has the kobo_sync option enabled
Sync with Kobo device -> in all other cases (more than one shelf is going to be synced, or one, but not the currently displayed shelf is synced

[alfred82santa]

At this point, maybe we should change the main flow. I mean, we should add a new flag in general config in order to choose if user wants to sync all books and shelves or just selected ones. If flag is false the behaviour is the old one and the flag on shelves must disappear. If flag is true, the flag on shelves appears and just shelves with this flag set will be sync.

Just changing this line: https://github.com/janeczku/calibre-web/pull/1852/files#diff-07a86a0f46bee91e3fe947bec8a613581aaa97fb5d1430199bc784844701c10cR156

only_kobo_shelves = config.config_kobo_shelves_sync # New config flag

[edillmann]

Hello, any news ? This seem's perfect for my usage.

[Cloudworks99]

Am I correct in assuming that this change is now available in v0.6.12 Pilar?

If so I tried testing it through the linuxserver.io docker build. I can switch the Kobo sync on under settings and I can generate a token for a user but I do not see the "Synchronize with Kobo device" switch on the Shelf edit screen. I only see "Title" and "Share with Everyone". Any idea what I am missing to make this work?

[mlaccetti]

If I remember correctly, I had to first create a new shelf and enable the "sync to Kobo" flag on the shelf. I could then add books to it and have them sync.

Edit: but I'm also running nightly, not a tagged build.

[quamok]

Am I correct in assuming that this change is now available in v0.6.12 Pilar?

If so I tried testing it through the linuxserver.io docker build. I can switch the Kobo sync on under settings and I can generate a token for a user but I do not see the "Synchronize with Kobo device" switch on the Shelf edit screen. I only see "Title" and "Share with Everyone". Any idea what I am missing to make this work?

Were you able to make it work? I'm having the same issue, running v0.6.12. Maybe it's just available in the nightly builds for now? Are they stable enough for daily usage?

[Cloudworks99]

Were you able to make it work? I'm having the same issue, running v0.6.12. Maybe it's just available in the nightly builds for now? Are they stable enough for daily usage?

I was not able to get it to work with the "latest" version of the linuxserver.io image. But using the nightly version of that image I got it working. The code for it is in master but not in the v0.6.12 release.

[quamok]

Were you able to make it work? I'm having the same issue, running v0.6.12. Maybe it's just available in the nightly builds for now? Are they stable enough for daily usage?

I was not able to get it to work with the "latest" version of the linuxserver.io image. But using the nightly version of that image I got it working. The code for it is in master but not in the v0.6.12 release.

Weird, I switched to nightly but still can't see the option...

[OzzieIsaacs]

Please enable the feature first

[quamok]

Please enable the feature first

Thank you, that is obviously the answer.

Is it supposed to delete books from the Kobo when a book is removed from the shelf? It seems to sync new books from the shelf to the Kobo device, but not removing books from Kobo that were removed from the shelf.

[Stvn21]

Please enable the feature first

Thank you, that is obviously the answer.

Is it supposed to delete books from the Kobo when a book is removed from the shelf? It seems to sync new books from the shelf to the Kobo device, but not removing books from Kobo that were removed from the shelf.

I encounter the same -> after removing books from the shelf it doesn't get removed from the kobo device. not sure if this is expected behavior.