xgboost and LightGBM GPU requirements

[Laurae2]

GPU RAM behavior information:

• xgboost GPU RAM requirement sampling is unreliable, as there is an initialization peak which is difficult to catch. What you see in nvidia-smi is not what you need in practice for xgboost.
• LightGBM GPU RAM requirement never grows up, what you see in nvidia-smi is really what you need

Therefore, xgboost requires trial and error sampling (using different number of parallel workers) until crash, while LightGBM is easily predictable with only 1 sample.

Notes:

• The GPU requirements also depend on the hyperparameters you are using for your model
• For reference, the hyperparameters are the following: depth = 6, leaves = 63, bins = 255

Table for GPU requirements:

Dataset	Observations	xgboost	LightGBM
Airline	100,000	"nearly" 1GB (fits 4 on 4GB)	about 67 MB
Airline	1,000,000	"nearly" 1GB (fits only 3 on 4GB)	about 93 MB
Airline	10,000,000	"nearly" 1.1GB (fits only 2 on 4GB)	about 333 MB

To be updated with more datasets later.

Edit 05/31/2019:

• Airline 1M 4x sometimes crashes on xgboost with 4GB GPU. Reduced to 3x.
• Airline 10M 3x sometimes crashes on xgboost with 4GB GPU. Reduced to 2x.

[Laurae2]

LightGBM, 10M Airline x12 on same GPU on 4GB GPU:

[Laurae2]

xgboost, 10M Airline x3 on same GPU on 4GB GPU: