We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
if pooling == 'first_last_avg': output_hidden_state = (hidden_states[-1] + hidden_states[1]).mean(dim=1) if pooling == 'first_last_avg': hidden = (hidden_states[-1] + hidden_states[1]) / 2 output_hidden_state = (hidden * att_mask.unsqueeze(-1)).sum(1) / att_mask.sum(-1).unsqueeze(-1)
The text was updated successfully, but these errors were encountered:
由all_utils.py的sent_to_vec可知,数据是一句一句单独地进行tokenization,没有对齐的需要,所以是没有padding token的。 不过这么做计算很慢,如果要改成整个batch一起tokenization然后编码的话,还是要用attention mask来过滤padding的。亲测前后结果差距很大。
Sorry, something went wrong.
No branches or pull requests
The text was updated successfully, but these errors were encountered: