About the positional encoding

[Yang-L1]

Hi, thanks for sharing this great work.
I wonder if this an implementation mistake in the position encoding part in line 21 of position_encoding.py.
div_term = torch.exp(torch.arange(0, d_model//2, 2).float() * (-math.log(10000.0) / d_model//2))
Should we change (-math.log(10000.0) / d_model // 2) to (-math.log(10000.0) / (d_model // 2) )? Otherwise this part ends up with -1.0 which means very high temperature.

[zehongs]

Hi, sorry for the late reply. And thanks for pointing out this implementation mistake! 👍 We'll fix this in this week!

[angshine]

We have trained a new model on ScanNet with this bug fixed, and it leads to a minor performance increase as shown below:

model	auc@5	auc@10	auc@20
paper	22.06	40.8	57.62
bug-fix	22.5	41.23	57.77

I would conclude that this bug does not affect the results of experiments in our paper. However, thanks again for pointing this out!