Loss suddenly becoming very large

[CM-913]

2024-07-11 17:07:28,476 - INFO - workflow: [('train', 1)], max: 4 epochs
2024-07-11 17:08:35,687 - INFO - Epoch [1][50/2766] lr: 0.01000, eta: 4:06:41, time: 1.344, data_time: 0.074, memory: 3611, loss_rpn_cls: 0.2045, loss_rpn_bbox: 0.0280, loss_cls: 0.5364, acc: 94.1191, loss_bbox: 0.0785, loss: 0.8474
2024-07-11 17:09:35,933 - INFO - Epoch [1][100/2766] lr: 0.01000, eta: 3:52:52, time: 1.205, data_time: 0.002, memory: 3611, loss_rpn_cls: 0.2257, loss_rpn_bbox: 0.1235, loss_cls: 1.8755, acc: 90.9961, loss_bbox: 0.2189, loss: 2.4435
2024-07-11 17:10:35,892 - INFO - Epoch [1][150/2766] lr: 0.01000, eta: 3:47:14, time: 1.199, data_time: 0.002, memory: 3611, loss_rpn_cls: 298.0166, loss_rpn_bbox: 359.4627, loss_cls: 36350.0296, acc: 91.0960, loss_bbox: 26111.0423, loss: 63118.5509
2024-07-11 17:11:34,803 - INFO - Epoch [1][200/2766] lr: 0.01000, eta: 3:42:59, time: 1.178, data_time: 0.002, memory: 3611, loss_rpn_cls: 317872648288.5507, loss_rpn_bbox: 719705513238.4994, loss_cls: 5641620281320.8613, acc: 72.2567, loss_bbox: 4112043418927.8037, loss: 10791241863295.7031
2024-07-11 17:12:31,443 - INFO - Epoch [1][250/2766] lr: 0.01000, eta: 3:38:24, time: 1.133, data_time: 0.002, memory: 3611, loss_rpn_cls: 1342402719373821617373184.0000, loss_rpn_bbox: 445532799740254198169600.0000, loss_cls: 20808144155606727859896320.0000, acc: 86.6701, loss_bbox: 39000440132089944379228160.0000, loss: 61596520540968801825456128.0000
2024-07-11 17:13:27,763 - INFO - Epoch [1][300/2766] lr: 0.01000, eta: 3:34:49, time: 1.126, data_time: 0.002, memory: 3611, loss_rpn_cls: 47490214674480742442991616.0000, loss_rpn_bbox: 14334860135127650800762880.0000, loss_cls: 612401653169420411301003264.0000, acc: 90.2228, loss_bbox: 1563048999675987437012647936.0000, loss: 2237275709971362084672241664.0000
2024-07-11 17:14:26,721 - INFO - Epoch [1][350/2766] lr: 0.01000, eta: 3:33:22, time: 1.179, data_time: 0.002, memory: 3611, loss_rpn_cls: 61938456058247602409308160.0000, loss_rpn_bbox: 17071254521314804517306368.0000, loss_cls: 726978391671261779812417536.0000, acc: 87.8476, loss_bbox: 1770044873227421734154534912.0000, loss: 2576032968830212433080483840.0000