client/mm: Arb Market Maker #2530

martonp · 2023-09-20T09:41:10Z

This adds a market making strategy that is kind of a combination between the basic market maker and the simple arbitrage strategies. Based on a CEX order book, it places orders on the DEX order book, and when there is a match on the DEX, the opposite order on the CEX is immediately executed for a profit.

// ArbMarketMakerConfig is the configuration for a market maker that places
// orders on both sides of the DEX order book, at rates where there are
// profitable counter trades on a CEX order book. Whenever a DEX order is
// filled, the opposite trade will immediately be made on the CEX.
//
// Each placement in BuyPlacements and SellPlacements represents an order
// that will be made on the DEX order book. The first placement will be
// placed at a rate closest to the CEX mid-gap, and each subsequent one
// will get farther.
//
// The bot calculates the extrema rate on the CEX order book where it can
// buy or sell the quantity of lots specified in the placement multiplied
// by the multiplier amount. This will be the rate of the expected counter
// trade. The bot will then place an order on the DEX order book where if
// both trades are filled, the bot will earn the profit specified in the
// configuration.
//
// The multiplier is important because it ensures that even some of the
// trades closest to the mid-gap on the CEX order book are filled before
// the bot's orders on the DEX are matched, the bot will still be able to
// earn the expected profit.
//
// Consider the following example:
//
//	Market:
//		DCR/BTC, lot size = 10 DCR.
//
//	Sell Placements:
//		1. { Lots: 1, Multiplier: 1.5 }
//		2. { Lots 1, Multiplier: 1.0 }
//
//	 Profit:
//	   0.01 (1%)
//
//	CEX Asks:
//		1. 10 DCR @ .005 BTC/DCR
//		2. 10 DCR @ .006 BTC/DCR
//		3. 10 DCR @ .007 BTC/DCR
//
// For the first placement, the bot will find the rate at which it can
// buy 15 DCR (1 lot * 1.5 multiplier). This rate is .006 BTC/DCR. Therefore,
// it will place place a sell order at .00606 BTC/DCR (.006 BTC/DCR * 1.01).
//
// For the second placement, the bot will go deeper into the CEX order book
// and find the rate at which it can buy 25 DCR. This is the previous 15 DCR
// used for the first placement plus the Quantity * Multiplier of the second
// placement. This rate is .007 BTC/DCR. Therefore it will place a sell order
// at .00707 BTC/DCR (.007 BTC/DCR * 1.01).

buck54321 · 2023-09-29T11:41:50Z

Based on a CEX order book, it places orders on the DEX order book, and when there is a match on the DEX, the opposite order on the DEX is immediately executed for a profit.

... order on the CEX is immediately... ?

buck54321

Concept ACK, but looking for some clarification on the multiplier setting. Some general observations for now.

There's a fair amount of duplication. The code could potentially benefit from some base type that both basicMarketMaker and arbMarketMaker embed. I played with the idea for a minute and I was able dedup maybe 40 lines, so not out-of-the-park improvement, but something to think about.
More inline comments would be helpful.

buck54321 · 2023-09-30T01:28:59Z

client/mm/mm_arb_market_maker.go

+// The first placement will be closest to the mid gap, and each subsequent one
+// will be farther from the mid-gap. The multiplier ensures that even if some
+// of the orders closest to the mid-gap are filled, there will still be enough
+// on the CEX orderbook for a profitable trade.


Kinda seems like the multiplier is just extra padding that could be accomplished to a large degree by choosing a larger profit instead. Can you help me understand the multiplier knob better?

To point of the multiplier is to avoid the situation where the trades on the CEX are matched before the bot can get around to making its counter-trade. multiplier gives the user more control over the size of the buffer than profit. Depending on how sparse the CEX orderbook is near the mid-gap, changing the profit could give you much less or much more buffer than you need. The point is to be able to lock in a certain amount of profit and have precise control over the probability you will be able to fill the counter-trade.

martonp · 2023-10-02T04:49:53Z

Concept ACK, but looking for some clarification on the multiplier setting. Some general observations for now.

There's a fair amount of duplication. The code could potentially benefit from some base type that both basicMarketMaker and arbMarketMaker embed. I played with the idea for a minute and I was able dedup maybe 40 lines, so not out-of-the-park improvement, but something to think about.

More inline comments would be helpful.

I agree on the duplication.. I'll see if I can think of something that makes sense. I've updated the comments on the ArbMarketMakingConfig, let me know if it still doesn't make sense.

Testing this using the Binance Testnet is currently not really possible. There are no updates being sent on the testnet market data stream. Our code is fine though, they are coming through on the mainnet API. However, there will need to be updates made to the Binance code. Only relying on the streams will likely not be sufficient as only the first 20 bids/asks closest to the mid-gap come through. This arb market maker will need more.

This adds a market making strategy that is kind of a combination between the basic market maker and the simple arbitrage strategies. Based on a CEX order book, it places orders on the DEX order book, and when there is a match on the DEX, the opposite order on the DEX is immediately executed for a profit.

client/mm/mm_arb_market_maker.go

JoeGruffins · 2023-10-02T07:15:32Z

client/mm/mm_arb_market_maker.go

+	a.ordMtx.Lock()
+	o, found := a.ords[oid]
+	if !found {
+		a.ordMtx.Unlock()
+		return
+	}
+	a.ords[oid] = note.Order
+	a.ordMtx.Unlock()


Suggested change

a.ordMtx.Lock()

o, found := a.ords[oid]

if !found {

a.ordMtx.Unlock()

return

}

a.ords[oid] = note.Order

a.ordMtx.Unlock()

a.ordMtx.RLock()

o, found := a.ords[oid]

a.ordMtx.RUnlock()

if !found {

return

}

a.ords[oid] = note.Order

But then a.ords won't be guarded when doing a.ords[oid] = note.Order.

JoeGruffins · 2023-10-02T07:25:40Z

client/mm/mm_arb_market_maker.go

+	for i, ord := range orders {
+		var oid order.OrderID
+		copy(oid[:], ord.ID)
+		a.ords[oid] = ord
+		a.oidToPlacement[oid] = placements[i].placementIndex
+	}


MultiTrade in core can fail one trade and keep going, possibly causing this index to be off if an order is not appended.

MultiTrade always returns the trades in the same order as they were passed in. If second one fails, then it won't try the third.

client/mm/mm_arb_market_maker.go

JoeGruffins · 2023-10-02T07:44:00Z

client/mm/mm_arb_market_maker.go

+			if requiredOnDEX > remainingDEXBalance {
+				log.Debugf("not enough DEX balance to place %d lots", lotsToPlace)
+				continue
+			}
+			if requiredOnCEX > remainingCEXBalance {
+				log.Debugf("not enough CEX balance to place %d lots", lotsToPlace)
+				continue
+			}


Can also log the balances we have and needed? Is this not a warning? Does the user always to expect to have cfgPlacement.Lots placed?

I think that's OK. If the balance has shifted to one side of the market then this is to be expected.

buck54321 · 2023-10-28T17:36:44Z

@martonp Please consider this (completely untested) alternative approach. Instead of requiring the user to define preset "placements", this approach uses a binary search to find the maximum number of lots to place that satisfy the user's profit demands. This avoids complex configuration and prevents the user from shooting themselves in the foot and missing out on profits. You still have an option to include a user-defined quantity multiplier, though I'm still convinced that it's necessary or prudent.

func (a *arbMarketMaker) rebalanceV1(epoch uint64) {
	if !a.rebalanceRunning.CompareAndSwap(false, true) {
		return
	}
	defer a.rebalanceRunning.Store(false)

	type dexBin struct {
		ords []*core.Order
		rate uint64
		lots uint64
	}

	findBin := func(rate uint64, bins []*dexBin) ([]*dexBin, *dexBin) {
		for _, bin := range bins {
			if bin.rate == rate {
				return bins, bin
			}
		}
		bin := &dexBin{
			rate: rate,
		}
		bins = append(bins, bin)
		return bins, bin
	}

	var dexBuys, dexSells []*dexBin
	a.ordMtx.RLock()
	for _, ord := range a.ords {
		var bin *dexBin
		if ord.Sell {
			dexSells, bin = findBin(ord.Rate, dexSells)
		} else {
			dexBuys, bin = findBin(ord.Rate, dexBuys)
		}
		bin.ords = append(bin.ords, ord)
		bin.lots += (ord.Qty - ord.Filled) / a.mkt.LotSize
	}
	a.ordMtx.RUnlock()
	sort.Slice(dexSells, func(i, j int) bool { return dexSells[i].rate < dexSells[j].rate })
	sort.Slice(dexBuys, func(i, j int) bool { return dexBuys[i].rate > dexBuys[j].rate })

	const qtyBuffer = 1.5 // like ArbMarketMakingPlacement.Multiplier

	calcProfit := func(lots, rate uint64, cexSell bool) float64 {
		vwap, _, filled, err := a.vwap(cexSell, uint64(float64(a.mkt.LotSize)*qtyBuffer))
		if err != nil {
			a.log.Errorf("Error calculating VWAP for %s: %w", a.mkt.Name, err)
			return -1
		}
		if !filled {
			return -1
		}
		dexQuoteQty := calc.BaseToQuote(rate, a.mkt.LotSize)
		cexQuoteQty := calc.BaseToQuote(vwap, a.mkt.LotSize)
		if cexSell {
			// Buying on DEX and selling on CEX. We want our CEX quote qty to
			// be larger than our DEX quote qty.
			return (float64(cexQuoteQty) - float64(dexQuoteQty)) / float64(dexQuoteQty)
		}
		// if we're buying on CEX and selling on DEX, we want our DEX quote qty
		// to be larger than our CEX quote qty.
		return (float64(dexQuoteQty) - float64(cexQuoteQty)) / float64(cexQuoteQty)
	}

	belowProfitThreshold := func(lots, rate uint64, cexSell bool) bool {
		return calcProfit(lots, rate, cexSell) < a.cfg.Profit
	}

	calcProfitableLots := func(rate, maxLots uint64, cexSell bool) uint64 {
		// Check for a single lot. No need to search if this doesn't produce
		// a profit.
		if belowProfitThreshold(1, rate, cexSell) {
			return 0
		}
		return uint64(sort.Search(int(maxLots), func(n int) bool {
			return belowProfitThreshold(uint64(n+1), rate, cexSell)
		}))
	}

	var dexSellLots, dexSellRate uint64
	for _, buyBin := range dexBuys {
		cexSell := false // selling on DEX, buying on CEX
		profitableLots := calcProfitableLots(buyBin.rate, buyBin.lots+dexSellLots, cexSell)
		if profitableLots == 0 {
			break
		}
		dexSellRate = buyBin.rate
		if profitableLots == buyBin.lots {
			// All lots are profitable.
			dexSellLots += buyBin.lots
			continue
		}
		break
	}

	if dexSellLots > 0 {
		// place sell order on DEX...
		return
	}

	var dexBuyLots, dexBuyRate uint64
	for _, sellBin := range dexSells {
		cexSell := true // buying on DEX, selling on CEX
		profitableLots := calcProfitableLots(sellBin.rate, sellBin.lots+dexBuyLots, cexSell)
		if profitableLots == 0 {
			break
		}
		dexBuyRate = sellBin.rate
		if profitableLots == sellBin.lots {
			// All lots are profitable.
			dexBuyLots += sellBin.lots
			continue
		}
		break
	}

	if dexSellLots > 0 {
		// place sell order on DEX...
		return
	}
}

martonp · 2023-11-02T05:55:08Z

@buck54321 I'm having trouble understanding what this is doing. It seems like you're placing orders just based on orders that were already placed by this bot. dexBuys and dexSells are getting populated based on a.ords.. so seems like when the bot starts up and a.ords is empty, no trades will ever be placed.

Why do you want a binary search?

buck54321 · 2023-11-02T11:09:40Z

seems like you're placing orders just based on orders that were already placed by this bot. dexBuys and dexSells are getting populated based on a.ords..

Oh right. Those should be from the order book. My bad.

Why do you want a binary search?

Lot sizes for non- BTC, non-ETH markets could be thousands of times smaller smaller than what we're used to working with. This introduces a couple of new challenges. 1) Users can very easily screw up configuration if parameters are based on lot size, and 2) searching for the right number of lots to satisfy our parameters should not be done linearly. (this is also a problem for our current maxOrder implementation,which does a loop with lots-- to find the exact amount).

buck54321 · 2023-11-02T17:53:29Z

client/mm/mm_arb_market_maker.go

+		a.matchesMtx.Unlock()
+		return
+	}
+	a.matchesSeen[matchID] = true


Should find a place to clean up matchesSeen once the match settles.

buck54321 · 2023-11-02T17:57:14Z

client/mm/mm_arb_market_maker.go

+		a.ordMtx.Lock()
+		delete(a.ords, oid)
+		delete(a.oidToPlacement, oid)
+		a.ordMtx.Unlock()


Maybe this is where you can clean up matches?

buck54321 · 2023-11-02T18:06:49Z

client/mm/mm_arb_market_maker.go

+	mkt, err := c.ExchangeMarket(cfg.Host, cfg.BaseAsset, cfg.QuoteAsset)
+	if err != nil {
+		log.Errorf("Failed to get market: %v", err)
+		return
+	}
+


I wonder if we need to somehow account for the ability of the exchange operator to change market parameters, e.g. max fee rate or lot size.

Yeah I think so. Core could send a notification regarding this. I'll make an issue.

buck54321 · 2023-11-02T18:08:34Z

client/mm/mm_arb_market_maker_test.go

+
+	tests := []*test{
+		{
+			name: "one buy and one cell match notifications",


client/mm/mm_arb_market_maker.go

buck54321 reviewed Sep 30, 2023

View reviewed changes

martonp force-pushed the arbMarketMaker branch from 974ef4e to 3a5cad9 Compare October 2, 2023 04:47

martonp force-pushed the arbMarketMaker branch from 3a5cad9 to cc8e827 Compare October 2, 2023 05:06

JoeGruffins approved these changes Oct 2, 2023

View reviewed changes

Joe review fixes.

9b8ca86

buck54321 reviewed Nov 2, 2023

View reviewed changes

martonp mentioned this pull request Nov 10, 2023

Handle updates to market parameters in market making #2600

Open

buck54321 reviewed Nov 10, 2023

View reviewed changes

client/mm/mm_arb_market_maker.go Outdated Show resolved Hide resolved

Buck review updates

cc6eeb5

martonp force-pushed the arbMarketMaker branch from 6b6a544 to cc6eeb5 Compare November 11, 2023 02:32

buck54321 approved these changes Nov 15, 2023

View reviewed changes

buck54321 merged commit cb851d1 into decred:master Nov 15, 2023
5 checks passed

martonp deleted the arbMarketMaker branch January 20, 2024 13:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

client/mm: Arb Market Maker #2530

client/mm: Arb Market Maker #2530

martonp commented Sep 20, 2023 •

edited

Loading

buck54321 commented Sep 29, 2023

buck54321 left a comment

buck54321 Sep 30, 2023

martonp Oct 1, 2023

martonp commented Oct 2, 2023 •

edited

Loading

JoeGruffins Oct 2, 2023

martonp Oct 6, 2023

JoeGruffins Oct 2, 2023

martonp Oct 6, 2023

JoeGruffins Oct 2, 2023

martonp Oct 6, 2023

buck54321 commented Oct 28, 2023 •

edited

Loading

martonp commented Nov 2, 2023

buck54321 commented Nov 2, 2023 •

edited

Loading

buck54321 Nov 2, 2023

buck54321 Nov 2, 2023

buck54321 Nov 2, 2023

martonp Nov 10, 2023

buck54321 Nov 2, 2023

client/mm: Arb Market Maker #2530

client/mm: Arb Market Maker #2530

Conversation

martonp commented Sep 20, 2023 • edited Loading

buck54321 commented Sep 29, 2023

buck54321 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martonp commented Oct 2, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

buck54321 commented Oct 28, 2023 • edited Loading

martonp commented Nov 2, 2023

buck54321 commented Nov 2, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martonp commented Sep 20, 2023 •

edited

Loading

martonp commented Oct 2, 2023 •

edited

Loading

buck54321 commented Oct 28, 2023 •

edited

Loading

buck54321 commented Nov 2, 2023 •

edited

Loading