client-replication version A and B working in simple-case. Think abou…

…t entity spawn/despawn. Component insert/remove during prediction

NOTES.md

@@ -5,6 +5,21 @@
   - one server: 1 game room per core?
 
 
+- BUGS:
+  - client-replication: it seems like the updates are getting accumulated for a given entity while the client is not synced
+    - that's because we don't do the duplicate check anymore, so we keep adding new updates
+    - but our code relies on the assumption that finalize() is called every send_interval (to drain pending-actions/pending-updates) which is not the case anymore.
+    - we still want to accumulate updates early though (before client is synced)
+    - OPTION 1 (SELECTED):
+      - just try sending the updates (which fails because we don't send anything until client is connected). That means 
+        we might have a bit of delay to receive the updates at the very beginning (retry_delay).
+    - OPTION 2:
+      - have a more clever way of accumulating updates. Maybe get a HashMap<ComponentKind, latest-tick> for updates?
+      - For actions, we still want to send every update sequentially...
+  - input-events are cleared every fixed-udpate, but we might want to use them every frame. What happens if frames are more frequent
+    than fixed-update? we double-use an input.. I feel like we should just have inputs every frame?
+
+
 - CLIENT REPLICATION:
   - we want client to be able to send messages to specific other clients
     - send a message to server, who then retransmits it to other clients
@@ -40,13 +55,43 @@
     Client 1 spawns an entity C1, which gets replicated on the client as S1 (and can then be further replicated to other clients).
     S1 doesn't get replicated back to client 1, but only to other clients. For example, we want to replicate client 1's cursor
     position to other clients.
+    DONE! Server can just add Replicate when entity is spawned.
 
 
   - B: client spawning a Predicted entity.
     For example client 1 spawns a predicted entity P1 (a shared UI menu). Server receives that notification; spawns an
     entity S1 that gets replicated to client 1. Client 1 spawns a confirmed entity C1. But instead of spawning a new
     corresponding predicted entity, it re-uses the existing P1. From there on prediction with rollback can proceed as usual.
 
+    P1 spawns on client. [Sends ShouldBePredicted(predicted=P1) as metadata on the spawn?, just so that the server can send it back]
+    Server spawns S1. user adds a system to replicate S1 to other clients (including client 1).
+    Client1 spawns C1 (but re-uses C1). [Sends ShouldBePredicted(predicted=P1)]
+
+    - TODO: a client system, upon receiving ShouldBePredicted { client_id: not_none }, attaches Predicted to the entity.
+    - TODO: update the confirmed spawn system to handle ShouldBePredicted { not_none }
+
+- OPTION 1:
+  - re-use ShouldBePredicted to send the client_entity
+  - when we receive back ShouldBePredicted { client_entity } on confirmed.
+    - if the current_entity is the client_entity from ShouldBePredicted, ignore (we are the current entity)
+    - we use client_entity to find the predicted entity. We add Confirmed and predicted
+
+- OPTION 2: 
+  - create with PreSpawned::for_predicted, PreSpawned::for_confirmed. PreSpawned {confirmed: Option, predicted: Option}
+  - to start, let's just Prespawned(Entity) because it's easier...
+  - use a new component PreSpawned::new() [Prespawned(None)] that the client can add on the entity when it's spawned
+  - the server returns PreSpawned(Some(entity)) when it spawns the entity
+  - when we receive PreSpawned on client on the confirmed entity:
+    - if it's PreSpawned.predicted:
+      - if the entity has ShouldBePredicted, we use PreSpawned.predicted to find its predicted entity. We add Confirmed and Predicted.
+        - if PreSpawned.predicted is None, we spawn it? maybe not, think about this more... (how to spawn new entities that were despawned.)
+          maybe the best solution is to attach Predicted(None) right away to the Predicted entity.
+      - if the entity doesnt have ShouldBePredicted, it's an error.
+    - if it's PreSpawned.confirmed:
+      - when the client receives it; it makes sure not to spawn a new entity, but to re-use the existing one.
+      - it's confirmed; so if entity doesn't exist, we spawn it? 
+
+
   - C: client spawning a Confirmed entity.
     Client 1 spawns a confirmed entity C1. It gets replicated to server, which spawns S1. Then that entity can get
     replicated to other clients AND to client 1. When client 1 receives the replication of S1, it knows that it corresponds

examples/client_replication/README.md

@@ -1,11 +1,14 @@
 # Introduction
 
-A simple example that shows how to use lightyear for replication:
-- server-replication: the entity is spawned on the server and replicated to the clients
-- client-replication: the entity is spawned on the client and replicated to the server and then to other clients.
-  - with client-authority: the client is authoritative for the entity.
-  - with server-authority: after the initial spawn, the server is authoritative for the entity.
-    - can use client-prediction...
+A simple example that shows how to use lightyear for client-replication (the entity is spawned on the client and replicated to the server and then to other clients):
+  - with client-authority: the circle cursor is replicated to the server and to other clients. Any client updates are replicated to the server.
+    The client's cursor is replicated to the server; one just needs to add the `Replicate` component on the server to then
+    replicate the cursor to other clients. 
+
+  - with server-authority: when pressing space, a square is spawned on the client. That square is a 'pre-predicted' entity:
+    it will get replicated to the server. The server can replicate it back to all clients.
+    When the original client gets the square back, it will spawn a 'Confirmed' square on the client, and will recognize
+    that the original square spawned was a prediction. From there on it's normal replication.
 
 
 ## Running the example

examples/client_replication/client.rs

@@ -3,6 +3,7 @@ use crate::protocol::*;
 use crate::shared::{color_from_id, shared_config, shared_movement_behaviour};
 use crate::{Transports, KEY, PROTOCOL_ID};
 use bevy::prelude::*;
+use lightyear::_reexport::ShouldBePredicted;
 use lightyear::prelude::client::*;
 use lightyear::prelude::*;
 use std::net::{Ipv4Addr, SocketAddr};
@@ -69,6 +70,7 @@ impl Plugin for MyClientPlugin {
                 cursor_movement,
                 receive_message,
                 send_message,
+                spawn_player,
                 handle_predicted_spawn,
                 handle_interpolated_spawn,
             ),
@@ -125,7 +127,7 @@ pub(crate) fn buffer_input(mut client: ResMut<Client<MyProtocol>>, keypress: Res
         return client.add_input(Inputs::Direction(direction));
     }
     if keypress.pressed(KeyCode::Delete) {
-        // currently, directions is an enum and we can only add one direction per tick
+        // currently, directions is an enum and we can only add one input per tick
         return client.add_input(Inputs::Delete);
     }
     if keypress.pressed(KeyCode::Space) {
@@ -140,6 +142,7 @@ pub(crate) fn buffer_input(mut client: ResMut<Client<MyProtocol>>, keypress: Res
 fn player_movement(
     // TODO: maybe make prediction mode a separate component!!!
     mut position_query: Query<&mut PlayerPosition, With<Predicted>>,
+    // InputEvent is a special case: we get an event for every fixed-update system run instead of every frame!
     mut input_reader: EventReader<InputEvent<Inputs>>,
 ) {
     if PlayerPosition::mode() != ComponentSyncMode::Full {
@@ -154,9 +157,53 @@ fn player_movement(
     }
 }
 
+/// Spawn a player when the space command is pressed
+fn spawn_player(
+    mut commands: Commands,
+    mut input_reader: EventReader<InputEvent<Inputs>>,
+    plugin: Res<MyClientPlugin>,
+    players: Query<&PlayerId, With<PlayerPosition>>,
+) {
+    // do not spawn a new player if we already have one
+    for player_id in players.iter() {
+        if player_id.0 == plugin.client_id {
+            return;
+        }
+    }
+    for input in input_reader.read() {
+        if let Some(input) = input.input() {
+            match input {
+                Inputs::Spawn => {
+                    debug!("got spawn input");
+                    let mut entity = commands.spawn(PlayerBundle::new(
+                        plugin.client_id,
+                        Vec2::ZERO,
+                        color_from_id(plugin.client_id),
+                    ));
+                    let entity_id = entity.id();
+                    // IMPORTANT: this lets the server know that the entity is pre-predicted
+                    // when the server replicates this entity; we will get a Confirmed entity which will use this entity
+                    // as the Predicted version
+                    entity.insert(ShouldBePredicted {
+                        client_entity: Some(entity_id),
+                    });
+                }
+                _ => {}
+            }
+        }
+    }
+}
+
 // Adjust the movement of the cursor entity based on the mouse position
-fn cursor_movement(window_query: Query<&Window>, mut cursor_query: Query<&mut CursorPosition>) {
-    if let Ok(mut cursor_position) = cursor_query.get_single_mut() {
+fn cursor_movement(
+    plugin: Res<MyClientPlugin>,
+    window_query: Query<&Window>,
+    mut cursor_query: Query<(&mut CursorPosition, &PlayerId)>,
+) {
+    for (mut cursor_position, player_id) in cursor_query.iter_mut() {
+        if player_id.0 != plugin.client_id {
+            return;
+        }
         if let Ok(window) = window_query.get_single() {
             if let Some(mouse_position) = window_relative_mouse_position(window) {
                 cursor_position.0 = mouse_position;
@@ -203,7 +250,7 @@ pub(crate) fn send_message(mut client: ResMut<Client<MyProtocol>>, input: Res<In
 // - keep track of it in the Global resource
 pub(crate) fn handle_predicted_spawn(mut predicted: Query<&mut PlayerColor, Added<Predicted>>) {
     for mut color in predicted.iter_mut() {
-        color.0.set_s(0.3);
+        color.0.set_s(0.4);
     }
 }
 

examples/client_replication/protocol.rs

@@ -48,7 +48,7 @@ impl CursorBundle {
                 replication_target: NetworkTarget::All,
                 // prediction_target: NetworkTarget::None,
                 // prediction_target: NetworkTarget::Only(vec![id]),
-                // interpolation_target: NetworkTarget::AllExcept(vec![id]),
+                interpolation_target: NetworkTarget::AllExcept(vec![id]),
                 ..default()
             },
         }
@@ -58,7 +58,7 @@ impl CursorBundle {
 // Components
 
 #[derive(Component, Message, Serialize, Deserialize, Clone, Debug, PartialEq)]
-pub struct PlayerId(ClientId);
+pub struct PlayerId(pub ClientId);
 
 #[derive(
     Component, Message, Serialize, Deserialize, Clone, Debug, PartialEq, Deref, DerefMut, Add, Mul,

examples/client_replication/server.rs

@@ -42,22 +42,21 @@ impl Plugin for MyServerPlugin {
         let plugin_config = PluginConfig::new(config, io, protocol());
         app.add_plugins(server::ServerPlugin::new(plugin_config));
         app.add_plugins(shared::SharedPlugin);
-        app.init_resource::<Global>();
         app.add_systems(Startup, init);
         // the physics/FixedUpdates systems that consume inputs should be run in this set
         app.add_systems(FixedUpdate, movement.in_set(FixedUpdateSet::Main));
         app.add_systems(
             Update,
-            (handle_connections, replicate_cursors, send_message),
+            (
+                handle_disconnections,
+                replicate_cursors,
+                replicate_players,
+                send_message,
+            ),
         );
     }
 }
 
-#[derive(Resource, Default)]
-pub(crate) struct Global {
-    pub client_id_to_entity_id: HashMap<ClientId, Entity>,
-}
-
 pub(crate) fn init(mut commands: Commands) {
     commands.spawn(Camera2dBundle::default());
     commands.spawn(TextBundle::from_section(
@@ -70,40 +69,26 @@ pub(crate) fn init(mut commands: Commands) {
     ));
 }
 
-/// Server connection system, create a player upon connection
-pub(crate) fn handle_connections(
-    mut connections: EventReader<ConnectEvent>,
+/// Server disconnection system, delete all player entities upon disconnection
+pub(crate) fn handle_disconnections(
     mut disconnections: EventReader<DisconnectEvent>,
-    mut global: ResMut<Global>,
     mut commands: Commands,
+    player_entities: Query<(Entity, &PlayerId)>,
 ) {
-    for connection in connections.read() {
-        let client_id = connection.context();
-        // Generate pseudo random color from client id.
-
-        let entity = commands.spawn(PlayerBundle::new(
-            *client_id,
-            Vec2::ZERO,
-            color_from_id(*client_id),
-        ));
-        // Add a mapping from client id to entity id
-        global
-            .client_id_to_entity_id
-            .insert(*client_id, entity.id());
-    }
     for disconnection in disconnections.read() {
         let client_id = disconnection.context();
-        if let Some(entity) = global.client_id_to_entity_id.remove(client_id) {
-            commands.entity(entity).despawn();
+        for (entity, player_id) in player_entities.iter() {
+            if player_id.0 == *client_id {
+                commands.entity(entity).despawn();
+            }
         }
     }
 }
 
 /// Read client inputs and move players
 pub(crate) fn movement(
-    mut position_query: Query<&mut PlayerPosition>,
+    mut position_query: Query<(&mut PlayerPosition, &PlayerId)>,
     mut input_reader: EventReader<InputEvent<Inputs>>,
-    global: Res<Global>,
     server: Res<Server<MyProtocol>>,
 ) {
     for input in input_reader.read() {
@@ -115,15 +100,41 @@ pub(crate) fn movement(
                 client_id,
                 server.tick()
             );
-            if let Some(player_entity) = global.client_id_to_entity_id.get(client_id) {
-                if let Ok(mut position) = position_query.get_mut(*player_entity) {
+
+            for (mut position, player_id) in position_query.iter_mut() {
+                if player_id.0 == *client_id {
                     shared_movement_behaviour(&mut position, input);
                 }
             }
         }
     }
 }
 
+pub(crate) fn replicate_players(
+    mut commands: Commands,
+    mut player_spawn_reader: EventReader<ComponentInsertEvent<PlayerPosition>>,
+) {
+    for event in player_spawn_reader.read() {
+        info!("received player spawn event: {:?}", event);
+        let client_id = event.context();
+        let entity = event.entity();
+
+        // for all cursors we have received, add a Replicate component so that we can start replicating it
+        // to other clients
+        if let Some(mut e) = commands.get_entity(*entity) {
+            e.insert(Replicate {
+                // do not replicate back to the owning entity!
+                replication_target: NetworkTarget::All,
+                // NOTE: Be careful to not override the pre-spawned prediction! we do not need to enable prediction
+                //  because there is a pre-spawned predicted entity
+                // we want the other clients to apply interpolation for the player
+                interpolation_target: NetworkTarget::AllExcept(vec![*client_id]),
+                ..default()
+            });
+        }
+    }
+}
+
 pub(crate) fn replicate_cursors(
     mut commands: Commands,
     mut cursor_spawn_reader: EventReader<ComponentInsertEvent<CursorPosition>>,
@@ -137,7 +148,10 @@ pub(crate) fn replicate_cursors(
         // to other clients
         if let Some(mut e) = commands.get_entity(*entity) {
             e.insert(Replicate {
+                // do not replicate back to the owning entity!
                 replication_target: NetworkTarget::AllExcept(vec![*client_id]),
+                // we want the other clients to apply interpolation for the cursor
+                interpolation_target: NetworkTarget::AllExcept(vec![*client_id]),
                 ..default()
             });
         }

examples/client_replication/shared.rs

@@ -1,6 +1,7 @@
 use crate::protocol::*;
 use bevy::prelude::*;
 use bevy_inspector_egui::quick::WorldInspectorPlugin;
+use lightyear::prelude::client::Confirmed;
 use lightyear::prelude::*;
 use std::time::Duration;
 use tracing::Level;
@@ -33,8 +34,8 @@ impl Plugin for SharedPlugin {
 
 // Generate pseudo-random color from id
 pub(crate) fn color_from_id(client_id: ClientId) -> Color {
-    let h = (((client_id * 45) % 360) as f32) / 360.0;
-    let s = 0.8;
+    let h = ((client_id * 90) % 360) as f32;
+    let s = 1.0;
     let l = 0.5;
     Color::hsl(h, s, l)
 }
@@ -71,11 +72,11 @@ pub(crate) fn draw_elements(
         gizmos.rect_2d(
             Vec2::new(position.x, position.y),
             0.0,
-            Vec2::ONE * 50.0,
+            Vec2::ONE * 40.0,
             color.0,
         );
     }
     for (position, color) in &cursors {
-        gizmos.circle_2d(Vec2::new(position.x, position.y), 30.0, color.0);
+        gizmos.circle_2d(Vec2::new(position.x, position.y), 15.0, color.0);
     }
 }

lightyear/src/client/connection.rs

@@ -75,6 +75,10 @@ impl<P: Protocol> Connection<P> {
         }
     }
 
+    pub(crate) fn clear(&mut self) {
+        self.events.clear();
+    }
+
     /// Add an input for the given tick
     pub fn add_input(&mut self, input: P::Input, tick: Tick) {
         self.input_buffer.set(tick, Some(input));
@@ -110,6 +114,14 @@ impl<P: Protocol> Connection<P> {
     }
 
     pub fn buffer_replication_messages(&mut self, tick: Tick) -> Result<()> {
+        // NOTE: this doesn't work too well because then duplicate actions/updates are accumulated before the connection is synced
+        // if !self.sync_manager.is_synced() {
+        //
+        //
+        //     // // clear the duplicate component checker
+        //     // self.replication_sender.pending_unique_components.clear();
+        //     return Ok(());
+        // }
         self.replication_sender
             .finalize(tick)
             .into_iter()