diff --git a/README.md b/README.md index a09a85b5..7839573d 100644 --- a/README.md +++ b/README.md @@ -26,15 +26,31 @@ It is free to use and modify for your own strategies. It provides the following: ## Getting Started 1. Create a [Testnet BitMEX Account](https://testnet.bitmex.com) and [deposit some TBTC](https://testnet.bitmex.com/app/deposit). -2. Install: `pip install bitmex-market-maker`. It is strongly recommeded to use a virtualenv. -3. Create a marketmaker project: run `marketmaker setup` + +### Non-developer instructions + +The below instructions are fine for a non-developer, but you will not be installing this fork of the Bitmex market-maker. They are fine for +getting your feet wet: + +1. Install: `pip install bitmex-market-maker`. It is strongly recommeded to use a virtualenv. +1. Create a marketmaker project: run `marketmaker setup` * This will create `settings.py` and `market_maker/` in the working directory. * Modify `settings.py` to tune parameters. -4. Edit settings.py to add your [BitMEX API Key and Secret](https://testnet.bitmex.com/app/apiKeys) and change bot parameters. + +### Developer instructions + +If you wish to follow my changes and/or make changes, you will need to do the following. + +1. `git clone` my repo +1. `cp ./market_maker/_settings_base.py settings.py` + +### Continuing, both non-developers and developers should: + +1. Edit settings.py to add your [BitMEX API Key and Secret](https://testnet.bitmex.com/app/apiKeys) and change bot parameters. * Note that user/password authentication is not supported. * Run with `DRY_RUN=True` to test cost and spread. -5. Run it: `marketmaker [symbol]` -6. Satisfied with your bot's performance? Create a [live API Key](https://www.bitmex.com/app/apiKeys) for your +1. Run it: `marketmaker [symbol]` +1. Satisfied with your bot's performance? Create a [live API Key](https://www.bitmex.com/app/apiKeys) for your BitMEX account, set the `BASE_URL` and start trading! ## Operation Overview @@ -45,8 +61,11 @@ This market maker works on the following principles: * Based on parameters set by the user, the bot creates a descriptions of orders it would like to place. - If `settings.MAINTAIN_SPREADS` is set, the bot will start inside the current spread and work outwards. - Otherwise, spread is determined by interval calculations. -* If the user specifies position limits, these are checked. If the current position is beyond a limit, - the bot stops quoting that side of the market. +* If the user specifies position limits,(via `CHECK_POSITION_LIMITS, MIN_POSITION` and +`MAX_POSITION` these are checked. If the current position is beyond a limit, + the bot stops quoting that side of the market. **IMPORTANT**: these parameters can save you + from buying/selling more contracts that your margin can sustain. If you dont use them, then + you may well margin yourself into a drained account. * These order descriptors are compared with what the bot has currently placed in the market. - If an existing order can be amended to the desired value, it is amended. - Otherwise, a new order is created. @@ -175,7 +194,6 @@ Common errors we've seen: * `TypeError: __init__() got an unexpected keyword argument 'json'` * This is caused by an outdated version of `requests`. Run `pip install -U requests` to update. - ## Compatibility This module supports Python 3.5 and later. @@ -184,3 +202,67 @@ This module supports Python 3.5 and later. BitMEX has a Python [REST client](https://github.com/BitMEX/api-connectors/tree/master/official-http/python-swaggerpy) and [websocket client.](https://github.com/BitMEX/api-connectors/tree/master/official-ws/python) + +### Related software + +* [Krypto](https://github.com/ctubio/Krypto-trading-bot) +* [Tribeca](https://github.com/michaelgrosner/tribeca) +* [Autoview](https://www.youtube.com/watch?v=1xXOc0y6wRg) + + +# THEORY and PRACTICE of Market-Making + +I recommend [this article](https://rados.io/how-profitable-is-market-making-on-different-exchanges/) if you are not familiar with market-making and its pitfalls. Reading that will make my discussion eaiser to follow. + +## Daily range + +The most important idea for you to grasp *now* about market-making is that you are setting +up a grid of buy and sell orders and you make money on the daily fluctuation of the market. + +As you can see from [this picture](http://take.ms/Q9kOY) having a grid of buy and sell orders +allows you to profit from the daily fluctuations of bitcoin regardless of whether the price +is going up or down. + +The caveat is: you need to need to be 100% certain that your buy/sell grid covers the range +of fluctuation **AND** that you have enough capital in your account to make the buys/sells +throughout an entire day of fluctuation. Check our TOTAL capital versus AVAILABLE credit +after setting up your buy/sell grid and make sure that only 50% of your capital is expended +in opening up these limit orders. + +An important concept is daily price range. You must know the daily price range of Bitcoin +and be certain that your buy/sell grid covers it. For instance, [here you can see that I have +a wide range of orders on the book](http://take.ms/nEm0N) and the daily range of Bitcoin is 4 times or more smaller than my grid. + +Reading an article like [this Bloomberg one](https://www.bloomberg.com/news/articles/2018-05-02/bitcoin-s-daily-trading-range-falls-from-4-700-to-124-chart) will help you understand +what range your grid should be prepared to cover on a daily basis. + +For me, all I do is look at the UPside and DOWNside contracts at Bitmex to get an idea of +the range that Bitcoin will trade in - this is because these future contracts are designed +to not pay back and the prices of these contracts are calculated to make sure that Bitcoin +does not rise or fall to those values within a week. + +For instance, at this moment, Bitcoin is trading at $7639.00 and the UP contract strike is +$8250 and the DOWN contract strike is $6750. So as long as my grid covers that price range, +I am good to go for a week. + +## Are you bullish on Bitcoin? + +If you are bullish on Bitcoin and don't want to get caught with sell orders when BTC +skyrockets to the moon without notice, then you may want to only offer buy-side market-making +instead of two-way market making. + +The way to do this is to enable `CHECK_POSITION_LIMITS` and then set `MIN_POSITION` to `-1` +so that you never hold more than 1 short contract. Set the `MAX_POSITION` to a number that +you feel comfortable with. e.g., if your initial buy contract is for 500 contracts and your +orders are spaced 1% apart and you want to be able to buy all the way down to BTC dropping +20% in value, then you need 20 * 500 as your `MAX_POSITION` and no more. + +## CHECK_POSITION_LIMITS should be True by default + +You do not want to wake (as I have) and find that a 0.6BTC account has been drained because there was no limit on total amount of contracts you could have open. + +## Monitoring your position + +Keep an eye on your TOTAL balance, AVAILABLE balance and the Liquidation price for your contracts, as [this picture shows](http://take.ms/sdNZO) then use +a chart to understand what the maximum trading range for your instrument (XBTUSD in most cases) is and make sure the difference between the current price and the liquidation price is +at least twice that. [This diagram](http://take.ms/p6ZX7) shows a simple study of daily trading range to give me a sense of assurance. diff --git a/batch/README.md b/batch/README.md new file mode 100644 index 00000000..c99bd543 --- /dev/null +++ b/batch/README.md @@ -0,0 +1,5 @@ +# + + shell> cd surgetrader/src + shell> mv nohup.out /tmp/nohup.out.$$ + shell> nohup ./batch/rerun & diff --git a/batch/killmm b/batch/killmm new file mode 100755 index 00000000..37ed776a --- /dev/null +++ b/batch/killmm @@ -0,0 +1,5 @@ +#!/bin/bash -x + +mv nohup.out /tmp/nohup.out.$$ +pkill -f runmm +pkill -f marketmaker diff --git a/batch/rerun b/batch/rerun new file mode 100755 index 00000000..0e422693 --- /dev/null +++ b/batch/rerun @@ -0,0 +1,6 @@ +#!/bin/bash -x + + +pkill -f runmm +#mv nohup.out /tmp/nohup.out.$$ +./batch/runmm diff --git a/batch/runmm b/batch/runmm new file mode 100755 index 00000000..cb6077fd --- /dev/null +++ b/batch/runmm @@ -0,0 +1,12 @@ +#!/bin/bash -x + + +# mv nohup.out /tmp/nohup.out.$$ + + +CMD="python ./marketmaker XBTUSD" + +until $CMD; do + echo "$CMD crashed with exit code $?. Respawning.." >&2 + sleep 1 +done diff --git a/market_maker/_settings_base.py b/market_maker/_settings_base.py index 11cf2bcf..20d75d5c 100644 --- a/market_maker/_settings_base.py +++ b/market_maker/_settings_base.py @@ -64,7 +64,7 @@ # Position limits - set to True to activate. Values are in contracts. # If you exceed a position limit, the bot will log and stop quoting that side. -CHECK_POSITION_LIMITS = False +CHECK_POSITION_LIMITS = True MIN_POSITION = -10000 MAX_POSITION = 10000 @@ -90,7 +90,15 @@ # Wait times between orders / errors API_REST_INTERVAL = 1 API_ERROR_INTERVAL = 10 -TIMEOUT = 7 + +# +TIMEOUT = 5 + +# Number of times to retry an errored call +RETRIES = 24 + +# Minutes Between retries +RETRY_DELAY = 5 # If we're doing a dry run, use these numbers for BTC balances DRY_BTC = 50 diff --git a/market_maker/bitmex.py b/market_maker/bitmex.py index d3cfe3ea..477dd01b 100644 --- a/market_maker/bitmex.py +++ b/market_maker/bitmex.py @@ -18,7 +18,9 @@ class BitMEX(object): """BitMEX API Connector.""" def __init__(self, base_url=None, symbol=None, apiKey=None, apiSecret=None, - orderIDPrefix='mm_bitmex_', shouldWSAuth=True, postOnly=False, timeout=7): + orderIDPrefix='mm_bitmex_', shouldWSAuth=True, postOnly=False, timeout=7, + retries=24, retry_delay=5 + ): """Init connector.""" self.logger = logging.getLogger('root') self.base_url = base_url @@ -47,6 +49,8 @@ def __init__(self, base_url=None, symbol=None, apiKey=None, apiSecret=None, self.ws.connect(base_url, symbol, shouldAuth=shouldWSAuth) self.timeout = timeout + self.max_retries = retries + self.retry_delay = retry_delay * 60 def __del__(self): self.exit() @@ -165,7 +169,7 @@ def place_order(self, quantity, price): def amend_bulk_orders(self, orders): """Amend multiple orders.""" # Note rethrow; if this fails, we want to catch it and re-tick - return self._curl_bitmex(path='order/bulk', postdict={'orders': orders}, verb='PUT', rethrow_errors=True) + return self._curl_bitmex(path='order/bulk', postdict={'orders': orders}, verb='PUT', rethrow_errors=True, max_retries=self.max_retries) @authentication_required def create_bulk_orders(self, orders): @@ -175,7 +179,7 @@ def create_bulk_orders(self, orders): order['symbol'] = self.symbol if self.postOnly: order['execInst'] = 'ParticipateDoNotInitiate' - return self._curl_bitmex(path='order/bulk', postdict={'orders': orders}, verb='POST') + return self._curl_bitmex(path='order/bulk', postdict={'orders': orders}, verb='POST', max_retries=self.max_retries) @authentication_required def open_orders(self): @@ -235,7 +239,7 @@ def _curl_bitmex(self, path, query=None, postdict=None, timeout=None, verb=None, # or you could change the clOrdID (set {"clOrdID": "new", "origClOrdID": "old"}) so that an amend # can't erroneously be applied twice. if max_retries is None: - max_retries = 0 if verb in ['POST', 'PUT'] else 3 + max_retries = 0 if verb in ['POST', 'PUT'] else self.max_retries # Auth: API Key/Secret auth = APIKeyAuthWithExpires(self.apiKey, self.apiSecret) @@ -249,7 +253,8 @@ def exit_or_throw(e): def retry(): self.retries += 1 if self.retries > max_retries: - raise Exception("Max retries on %s (%s) hit, raising." % (path, json.dumps(postdict or ''))) + raise Exception("Max retry amount of {} on {} ({}) hit, raising.".format(max_retries, path, json.dumps(postdict or ''))) + time.sleep(self.retry_delay) return self._curl_bitmex(path, query, postdict, timeout, verb, rethrow_errors, max_retries) # Make the request diff --git a/market_maker/market_maker.py b/market_maker/market_maker.py index fcca4328..085a6630 100644 --- a/market_maker/market_maker.py +++ b/market_maker/market_maker.py @@ -33,7 +33,10 @@ def __init__(self, dry_run=False): self.bitmex = bitmex.BitMEX(base_url=settings.BASE_URL, symbol=self.symbol, apiKey=settings.API_KEY, apiSecret=settings.API_SECRET, orderIDPrefix=settings.ORDERID_PREFIX, postOnly=settings.POST_ONLY, - timeout=settings.TIMEOUT) + timeout=settings.TIMEOUT, + retries=settings.RETRIES, + retry_delay=settings.RETRY_DELAY + ) def cancel_order(self, order): tickLog = self.get_instrument()['tickLog'] diff --git a/market_maker/ws/ws_thread.py b/market_maker/ws/ws_thread.py index d65f9dc0..0dae80bf 100644 --- a/market_maker/ws/ws_thread.py +++ b/market_maker/ws/ws_thread.py @@ -164,7 +164,7 @@ def __connect(self, wsURL): self.logger.info("Started thread") # Wait for connect before continuing - conn_timeout = 5 + conn_timeout = settings.TIMEOUT while (not self.ws.sock or not self.ws.sock.connected) and conn_timeout and not self._error: sleep(1) conn_timeout -= 1 @@ -331,4 +331,3 @@ def findItemByKeys(keys, table, matchData): ws.connect("https://testnet.bitmex.com/api/v1") while(ws.ws.sock.connected): sleep(1) -