Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stream_ordering sort to Sliding Sync /sync #17293

Merged
merged 34 commits into from
Jun 17, 2024

Commits on Jun 10, 2024

  1. Sliding sync sort stub

    MadLittleMods committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    ce45cc1 View commit details
    Browse the repository at this point in the history
  2. Add changelog

    MadLittleMods committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    75b701f View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Prefer ? < a AND a <= ?

    See #17293 (comment)
    
    > FWIW I mildly prefer the styling `? < a AND a <= ?` as then my brain can more easily convert that to `a in (?...?]`
    MadLittleMods committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    c8a240f View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Fix get_last_event_in_room_before_stream_ordering(...) not finding …

    …the last event
    
    Previously, it would use the event with the lowest `stream_ordering` that was
    fetched (we want the highest `stream_ordering. `union` does not work how you
    might think at first and does not preserve the ordering of the operand queries.
    
    `union all` also doesn't necessarily have that gurantee but it does behave
    that way. See https://dba.stackexchange.com/questions/316818/are-results-from-union-all-clauses-always-appended-in-order/316835#316835
    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    2a82ac0 View commit details
    Browse the repository at this point in the history
  2. Add actual guranteed order for UNION

    We use `union all` because we don't need any of the deduplication logic
    (`union` is really a union + distinct). `UNION ALL`` does preserve the
    ordering of the operand queries but there is no actual gurantee that it
    has this behavior in all scenarios so we need the extra `ORDER BY` at the
    bottom.
    
    See https://dba.stackexchange.com/questions/316818/are-results-from-union-all-clauses-always-appended-in-order/316835#316835
    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    2e1d142 View commit details
    Browse the repository at this point in the history
  3. Some clean-up

    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    b1af992 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    901ce62 View commit details
    Browse the repository at this point in the history
  5. Fix get_last_event_in_room_before_stream_ordering(...) not finding …

    …the last event
    
    Previously, it would use the event with the lowest `stream_ordering` that was
    fetched (we want the highest `stream_ordering. `union` does not work how you
    might think at first and does not preserve the ordering of the operand queries.
    
    `union all` also doesn't necessarily have that gurantee but it does behave
    that way. See https://dba.stackexchange.com/questions/316818/are-results-from-union-all-clauses-always-appended-in-order/316835#316835
    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    87ad458 View commit details
    Browse the repository at this point in the history
  6. Add actual guranteed order for UNION

    We use `union all` because we don't need any of the deduplication logic
    (`union` is really a union + distinct). `UNION ALL`` does preserve the
    ordering of the operand queries but there is no actual gurantee that it
    has this behavior in all scenarios so we need the extra `ORDER BY` at the
    bottom.
    
    See https://dba.stackexchange.com/questions/316818/are-results-from-union-all-clauses-always-appended-in-order/316835#316835
    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    431b31e View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    d7f40ae View commit details
    Browse the repository at this point in the history
  8. Add changelog

    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    a8056ae View commit details
    Browse the repository at this point in the history
  9. Configuration menu
    Copy the full SHA
    3f317a9 View commit details
    Browse the repository at this point in the history
  10. Fix invalid syntax

    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    54bdc0c View commit details
    Browse the repository at this point in the history
  11. Merge branch 'develop' into madlittlemods/fix-and-tests-for-get_last_…

    …event_in_room_before_stream_ordering
    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    4d585b6 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    42f24de View commit details
    Browse the repository at this point in the history
  13. Merge branch 'madlittlemods/fix-and-tests-for-get_last_event_in_room_…

    …before_stream_ordering' into madlittlemods/msc3575-sliding-sync-sort
    
    Conflicts:
    	synapse/storage/databases/main/stream.py
    	tests/storage/test_stream.py
    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    03547b0 View commit details
    Browse the repository at this point in the history
  14. Add event.internal_metadata.instance_name and event position to `ge…

    …t_last_event_in_room_before_stream_ordering(...)`
    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    c94550d View commit details
    Browse the repository at this point in the history
  15. Configuration menu
    Copy the full SHA
    afb6627 View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    af60f7b View commit details
    Browse the repository at this point in the history
  17. Add some tests

    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    bd49c34 View commit details
    Browse the repository at this point in the history
  18. Fix newly_left not being added back if we returned early (when `membe…

    …rship_snapshot_token.is_before_or_eq(to_token.room_key)`)
    
    Wasn't caught in the tests because the test was wrong
    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    5060588 View commit details
    Browse the repository at this point in the history
  19. Fix ban test case

    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    5243a30 View commit details
    Browse the repository at this point in the history
  20. Adjust wording

    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    d5929f1 View commit details
    Browse the repository at this point in the history
  21. Fix lints

    MadLittleMods committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    8935c6c View commit details
    Browse the repository at this point in the history
  22. Configuration menu
    Copy the full SHA
    93aa4ff View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-sort

    Conflicts:
    	synapse/storage/databases/main/stream.py
    	tests/storage/test_stream.py
    MadLittleMods committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    185e0b5 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'develop' into madlittlemods/msc3575-sliding-sync-sort

    Conflicts:
    	synapse/handlers/sliding_sync.py
    	tests/handlers/test_sliding_sync.py
    MadLittleMods committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    8244b25 View commit details
    Browse the repository at this point in the history
  3. Fix filtering

    MadLittleMods committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    35808b3 View commit details
    Browse the repository at this point in the history
  4. No more sort option

    We always just sort by stream_ordering
    MadLittleMods committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    a917eda View commit details
    Browse the repository at this point in the history
  5. Add rest test

    MadLittleMods committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    84eaeea View commit details
    Browse the repository at this point in the history
  6. Update changelog

    MadLittleMods committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    99ed012 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. Configuration menu
    Copy the full SHA
    7d80418 View commit details
    Browse the repository at this point in the history
  2. Stable sort with just stream_ordering

    > FWIW `stream_ordering` is unique, so we don't need a tie break.
    
    See #17293 (comment)
    MadLittleMods committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    ef92f3c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    63ff8f9 View commit details
    Browse the repository at this point in the history