Use a new token format for sliding sync

[erikjohnston]

This is in preparation for adding per-connection state.

[MadLittleMods]

Instead of making an initial request to get a token, I think we should just generate one how we were doing before with a 0 connection_token. This way when we start to actually diff what the user has already seen, we don't mess with these tests.

I think we should just have another set of tests that stresses the diffing functionality. test_rooms_connection_tracking_xxx_etc

[erikjohnston]

I disagree, I think its more useful to test what we want the APIs to actually do? Yes, we need to change one or two tests when we add the connection tracking, but my view is that those tests are wrong as they are checking for things that the API should not be returning. We're essentially testing behaviour that can never be encountered through real world usage, and so what benefit do those tests/assertions have?

[MadLittleMods]

I want to treat the diff'ing as an extra thing (since it is)

And it's as if there is no cache built up.

[erikjohnston]

Hmm, right. I think that would be better expressed as what happens when the server-side cache gets cleared? We can add a .clear() to the store to simulate that?

[MadLittleMods]

🤷 That or just a comment like this:

def test_xxx(...) -> None:
    """
    ...
    Make sure we're doing the right thing during an incremental sync without any cache.
    """

    # Doing an incremental sync with a fresh token from `0` just simulates a request when
    # there is no cache built up.
    from_token = SlidingSyncStreamToken(self.event_sources.get_current_token(), 0)

[erikjohnston]

Quick thought: we probably do want to match, and that might be easiest if we had a helper function, something like:

sync_request_body = {
    "lists": [ ... ],
}

body = self.do_sync(sync_request_body, tok=user1_tok)
from_token = body["pos"]

body = self.do_sync(sync_request_body, since=from_token, tok=user1_tok)

Maybe? Or just have a from_token = self.do_sync_and_get_token(sync_request_body, tok=user1_tok)

[MadLittleMods]

@erikjohnston Looks alright 👍 I think we need access to the response body for assertions but could do response, from_token = ... as a nice shortcut for fetching the next token.

[erikjohnston]

Have updated the PR with a do_sync. I have only changed the calls we added to get the from_token, as otherwise this would touch all the tests.

[MadLittleMods]

👍 We can do the rest in a follow-up PR

[MadLittleMods]

Updated the rest of the tests to use do_sync(...) in #17482 ⏩

[MadLittleMods]

.

[MadLittleMods]

Aside: Could move def default_config(self) here