/messages?dir=f from a historical point to a /sync prev_batch returns a duplicate event (SYN-775)

[matrixbot]

If you have a batch of messages for a given room from /sync beginning at a certain prev_batch, then obtain an earlier token by requesting a previous batch using /messages?dir=b, and then request events from the resulting batch token back to the prev_batch from /sync using /messages?dir=f, you should receive the same set of events, assuming nothing weird got added to the event graph in the intervening time. Instead, the final event in the second set of events consistently duplicates the first event in the sync response.

(Imported from https://matrix.org/jira/browse/SYN-775)

(Reported by Benjamin Saunders)

[richvdh]

this feels like a thing that has probably been fixed somewhere in the last 3.5 years. if not, we can reopen.

[deepbluev7]

I actually just ran into this, I think. If you do an incremental sync, trying to paginate back using /messages from that incremental syncs prev_batch seems to return the events from that incremental sync too.

Steps to reproduce:

1. Do an initial sync
2. Send a message
3. Do an incremental sync
4. Call /messages with the prev_batch from 3.

4. returns messages from 3., although it should return messages from 1..

[deepbluev7]

After some more investigation, it seems like on incremental syncs, synapse returns the previous sync token as the prev_batch token. While synapse allows using sync tokens as batch tokens for /messages, that token doesn't paginate from the correct position. This may work fine if you only have only a single room, but probably fails as soon as you receive a typing notification in that room or a presence update.

[deepbluev7]

I can confirm that #8486 seems to have fixed this.