Add bots to multiplayer games

[jakehsiao]

I am currently writing a game using this engine. Now this feature is required.

It's a 10-person game. In single player's mode, player will play with 9 AIs. When it's AI's turn, AI needs to operate automatically(not by player clicking "step" or "simulation").

Mixed players and AIs in online game is not required now(but maybe required in the future).

If this feature is WIP, please tell me where I can help. I'm willing to implement this and contribute.

[nicolodavis]

Hey! If all you need is a simple thing that activates the AI player in a non networked game, you can just call step() from your client automatically. I believe this is what @flamecoals does.

If you need bots for a multiplayer game, that's a bit more involved. For example, we probably need to support a few different configurations where the bot sits on a separate server / process and doesn't choke the game server when it is "thinking". The design for this is still WIP and feel free to get involved in this as well if you're interested.

However, for your use case, I don't think you actually need any changes to the framework (you can do everything on your client).

[jakehsiao]

Adding "step" to board really let AIs moves on their own, thanks!

However, when AI let player to discard a card(set action player to "0" which is player's ID but currentPlayer is still "1"), player cannot choose which card to discard that unless click "updatePlayerID" on debug interface. "UpdatePlayerID" only exists on debug interface, not on board.

Here's my solution in "client/react.js":

 if (board) {
        _board = React.createElement(board, {
          ...state,
          ...rest,
          isMultiplayer: multiplayer !== undefined,
          moves: this.client.moves,
          events: this.client.events,
          gameID: this.gameID,
          playerID: this.playerID,
          step: this.client.step,
          reset: this.client.reset,
          undo: this.client.undo,
          redo: this.client.redo,
          updatePlayerID: this.updatePlayerID, //Added
        });
      }

By adding "updatePlayerID" function to board's props, it supports AI let player make a choice in AI's turn.

[lukegothic]

Hey! If all you need is a simple thing that activates the AI player in a non networked game, you can just call step() from your client automatically. I believe this is what @flamecoals does.

[...]

However, for your use case, I don't think you actually need any changes to the framework (you can do everything on your client).

Two scenarios:

1. Two-player game where one of the players is an AI, and a single move per turn (e.g. TicTacToe)
2. Another game with one or many AI players and multiple moves per turn (e.g. Dominion)

Where should the .step() be called to tell the AI to begin playing? What checks should be performed to determine if the AI can begin to play in any of the scenarios?

To solve this issue in a general approach, it makes sense to me to invoke the AI play / step() at the beggining of an AI player's turn (onTurnBegin), but from what it is posted here I get that the only way to access the .step() function is on the Board Component, which AFAIK isn't able to listen to events (onTurnBegin, onTurnEnd, onMove). Plugging this.props.step() at the end of a correct player's move function works for the first scenario but not for the second one.

Thanks and please point me to any relevant code that I could look into.

[nicolodavis]

Sorry about not getting back to this thread sooner.

Sounds like we have a couple of things that we could do here:

Singleplayer

This is where you have the game running completely client-side with no server. You just need the bot to make a move at the appropriate time. A simple solution would be to automate calling step() at the right time (similar to the code here).

Another solution would be to use multiplayer: { local: true } and have the bot actually run as a separate client instance on the browser. This feels more natural to me.

Multiplayer (with server)

The trick here is to have the bot run on the server and not block the main thread (due to it being CPU intensive). Options include Worker Threads or a Child Process. Each has a different set of tradeoffs. I would recommend starting with a Worker Thread first.

How does the bot know when to play in both scenarios?

Rather than have the bot trigger from the user's code (inside onTurnBegin or somewhere else), we should either:

1. Put the code somewhere inside the framework at master.js, which will see all moves (even in the Singleplayer mode if multiplayer: { local: true } is set). Perhaps it can send a message to the bot saying "hey it's your turn" which triggers the bot to make a move.

2. If the bot is just another client (identical to a human player from the perspective of master.js), it can just watch the game and determine when to play on its own.

TL;DR I think step() is just a convenience function that should only be used while prototyping and we should build a better way for bots to play in actual games.

[andrscyv]

Hello! @nicolodavis @delucis

I'm really interested in using bots in a multiplayer game with a remote master server.

I've been going through the source code and also read the other comments in this issue. I want to build this feature and would appreciate your feedback and guidance.

My proposal is the following

I think we could build a new server (BotServer) that would:

1. Have a restful api to request that a bot joins a particular match
2. For each match requested, listen to the game updates and identify when it's the bot's turn, run the bot's code and submit a move to the master server

I think a proof of concept of this could be built without modifying the existing code: the botServer could instantiate a Client for each of the bots it handles and use it to interact with the master server. But in the long run it may be better to build a stripped down version of the Client class (a BotClient) that could listen to the events of multiple matches and handle multiple players (the bots) to make it more efficient

I also think that the BotServer could be customized by having a function ( something like runBot(state: State): Move ) in which devs could add arbitrary code to run their bots, they could even spawn other processes to run bots in other languages (the bot I'm implementing is written in python for example)

This is just a very high level overview of the implementation that I'm thinking about. What do you think ?

[delucis]

Hi @andrscyv! Thanks for looking into this — I’d be happy to see you build out a solution for this and am happy to answer any questions you might have.

A couple of thoughts:

1. In your diagram above you show Server/Lobby as a single entity, but these can be decoupled to run on two separate ports (or multiple hardware with a transport pubsub). I wonder if a proof of concept could work similarly: add the bot server functionality as an optional component when creating a boardgame.io server, to support simple one server set-ups as well as multi-server set-ups?

2. Instead of having a separate REST API, could the existing Lobby API contain a POST /games/{name}/{id}/add-bot endpoint? Might that help allow multiple bot server instances share the work? Maybe the lobby server could post a new bot slot as available and the servers could bid to take it based on how busy they are?

3. In general we like for the Local master to replicate the server behaviour as much as possible (even if it doesn’t currently support the lobby API). Do you think we could refactor and share some of the generic bot-running logic across your BotServer and there?

[andrscyv]

Thanks for the feedback @delucis !

I agree with the improvements you suggest but I don't understand completely your 3rd point. The LocalMaster in src/client/transport/local.ts already has logic for running bots doesn't it ? What kind of functionality you think that the botserver could share with LocalMaster?

[delucis]

I thought maybe we could share some of the logic regarding when a bot should play etc. I think that code is currently fairly basic and doesn't handle all cases well, so you don't need to use that specifically but if we'll need something like that, it would be cool to be able to use it both on the new server and on the local version. Does that make sense?

[gabrielecastellano]

Hello! Any update on this feature?

[JacobSyndeo]

+1… this would be awesome to have, but I'm not sure I currently have the knowledge to do it myself.

[nicolodavis]

Hi @gabrielecastellano @JacobSyndeo. Nobody is working on this at the moment.

[andrscyv]

@nicolodavis I never got any feedback on #1081