Merge pull request #31 from kanekoshoyu/feature/monitor_taskpool

Feature/monitor taskpool

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "kucoin_arbitrage"
-version = "0.0.13"
+version = "0.0.14"
 edition = "2021"
 authors = ["Sho Kaneko <kanekoshoyu@gmail.com>"]
 description = "Event-Driven Kucoin Arbitrage Framework in Async Rust"
@@ -14,7 +14,7 @@ futures = "0.3.28"
 tokio = { version = "1.33.0", features = ["full"] }
 tokio-signal = "0.2.9"
 # log
-log = "0.4"
+log = "0.4.20"
 fern = "0.6.2"
 env_logger = "0.10.0"
 # config

README.md

@@ -3,10 +3,12 @@
 [![](https://img.shields.io/docsrs/kucoin_arbitrage)](https://docs.rs/kucoin_arbitrage)
 [![](https://img.shields.io/github/license/kanekoshoyu/kucoin_arbitrage)](https://github.com/kanekoshoyu/kucoin_arbitrage/blob/master/LICENSE)  
 This is an async Rust project to implement zero-risk crypto trinagular arbitrage, explore technical feasiblity of generating passsive income (i.e. sleep to earn!).  
-## How the arbitrage works
-Say we hold USDT, it checks all the coins(e.g. ETH) that can trade against BTC and USDT, and compare the profit by either:  
+## How cyclic arbitrage works
+Say we hold USDT, it checks all the listed crypto (e.g. ETH) that can trade against BTC and USDT, and compare the profit by either:  
 - Buy-Sell-Sell: buy ETH (sell USDT), sell ETH (buy BTC), sell BTC (buy USDT)  
 - Buy-Buy-Sell: buy BTC (sell USDT), buy ETH (sell BTC), sell ETH (buy USDT)  
+
+The above is triangular arbitrage, which is the simplest form of cyclic arbitrage. We can make more complex say finding the most profitable routes, or placing order at maker price for reduced fees. 
 
 ## Previous mistakes in Python
 2 years ago I have made a python script that runs the triangular arbitrage in KuCoin, but it had several technical issues and ended up not following up.  
@@ -16,7 +18,7 @@ https://github.com/kanekoshoyu/Kucoin-Triangular-Arbitrage
 - It didn't count the actual size of the arbitrage order, which meant that the script kept buying some shitcoin and could not sell properly.
 - It simply took the mean price instead of best bid/ask, which means it took maker positions for each of three actions in one arbitrage, and did not execute arbitrage promptly.
 ## How to run example executables
-Copy/Rename config.ini.sample as config.ini and set the API key with your own KuCoin API credentials. Configure the event monitor interval and the USD budget per clyclic arbitrage.   
+1. Rename config_sample.toml as config.toml and set the API key with your own KuCoin API credentials. Configure the event monitor interval and the USD budget per each iteration of cyclic arbitrage.   
 ```
 [KuCoin Credentials]
 api_key="YOUR_API_KEY"
@@ -29,25 +31,50 @@ monitor_interval_sec=120
 # max amount of USD to use in a single cyclic arbitrage
 usd_cyclic_arbitrage=100
 ```
-At the root directory of the project, run the below command
+2. At the root directory of the project(kucoin_arbiitrage), run the command below:
 ```
 cargo run --bin event_triangular  
 ```
-`event_triangular` is one of the example executables. There are other executables in the `bin` directory.
+`event_triangular` is one of the example executables that implements [XXX-BTC, XXX-USDT, BTC-USDT] triangular arbitrage. There are other executables in the `bin` directory.
 
 ## Overview
+
+### Code Structure
 The project is split into these components:
-###### Example Executables
+
+##### Example Executables
 - `bin` contains example executable codes. Some of them are for network testing purpose.
-###### Internal Structure (Independent of Exchange APIs)
+
+##### Internal Structure (Independent of Exchange APIs)
 - `model` has internal generic data structures used for abstracted representations of markets. This should be independent of exchange APIs so that the the arbitrage strategy algorithm can be conducted across different exchanges.
 - `event` has the events used to pass states and data passed across different components. It uses the internal model for the same reason.
-- `strategy` has the implementations of arbitrage strategy algorithm. The algorithms are built upon internal model and event. 
-- `global` has the lazy_static globals that is used across the code. This may get replaced by passing Arc/Mutex to functions soon. I just wanted the code to be simple and readable with this way first.
+- `strategy` has the implementations of arbitrage strategy algorithm. The algorithms are built upon internal model and event.
+- `monitor` has the counter used to monitor MPS (message per seconds) for each broadcast channels, and a timers mapped globally by string for easy debug access.
 
-###### Link to Exchange APIs (e.g. KuCoin)
+##### Link to Exchange APIs (e.g. KuCoin)
 - `translator` has the conversion of exchange API objects into internal models and vice versa. It uses traits and the traits are implemented per API models.
 - `broker` has the tasks that runs API calls, and converts into internal data structure.
+
+### Event Pub/Sub with Tokio Broadcast
+Event broadcasts empowers the modularity of tasks. Each async task communicates with eachother using events, pub/sub via tokio's broadcast. Here is the exmaple for `event_triangular.rs`
+| Channel        | Publisher                         | Subscriber                                                  |
+| -------------- | --------------------------------- | ----------------------------------------------------------- |
+| orderbook      | task_pub_orderbook_event          | task_monitor_channel_mps, task_sync                         |
+| orderbook_best | task_sync                         | task_monitor_channel_mps, task_pub_chance_all_taker_btc_usd |
+| chance         | task_pub_chance_all_taker_btc_usd | task_monitor_channel_mps, task_gatekeep_chances             |
+| order          | task_gatekeep_chances             | task_monitor_channel_mps, task_place_order                  |
+| orderchange    | task_pub_orderchange_event        | task_monitor_channel_mps, task_gatekeep_chances             |
+
+### Task Pools with Tokio JoinSet
+Tasks are grouped and spawned using JoinSets. We can either await for all the tasks to end with `join!` or await until a single task ends with `select!` or `join_next`. This provides full control over how we want to control these tasks. Here is the exmaple for task pools declared in core function of `event_triangular.rs`
+| TaskPool                | Task                                                                                            |
+| ----------------------- | ----------------------------------------------------------------------------------------------- |
+| taskpool_infrastructure | task_sync_orderbook, task_pub_chance_all_taker_btc_usd, task_gatekeep_chances, task_place_order |
+| taskpool_subscription   | task_pub_orderbook_event, task_pub_orderchange_event                                            |
+| taskpool_monitor        | task_monitor_channel_mps, task_log_mps                                                          |
+
+When a task in taskpool returns, its result is received by `join_next`, which are received by core's `select!`. 
+When an external signal is received, or core returns error, it gets detected by `select!` at the main and terminates the program.
 
 ## Major Structural Improvements
 - Use compiled Rust code for neat, efficient and proper code
@@ -68,6 +95,9 @@ The project is split into these components:
 | Resort against limit order that could not get filled                                               | Pending   |
 | Full triangular arbitrage with the middle coin other than BTC (e.g. ETH-USD, ALT-ETH, ALT-USD)     | Pending   |
 
-
-## To-dos
-please refer to the [Discussions](https://github.com/kanekoshoyu/kucoin_arbitrage/discussions) channel in GitHub.
+## Deployment
+Please refer to my another repo implementing service-level wrappers: [chaiwala](https://github.com/kanekoshoyu/chaiwala)
+
+## Community
+- Rust Questions: [GitHub Repo Discussion](https://github.com/kanekoshoyu/kucoin_arbitrage/discussions)
+- Arbitrage Questions: [Discord Server](https://discord.gg/q3j5MYdwnm)