Skip to content

Commit

Permalink
Core: Fix copies on main (huggingface#29624)
Browse files Browse the repository at this point in the history
fix fix copies
  • Loading branch information
younesbelkada authored Mar 13, 2024
1 parent be3fd8a commit 9acce7d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/transformers/models/gptj/modeling_gptj.py
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ def _flash_attention_forward(
attention_mask (`torch.Tensor`):
The padding mask - corresponds to a tensor of size `(batch_size, seq_len)` where 0 stands for the
position of padding tokens and 1 for the position of non-padding tokens.
dropout (`int`, *optional*):
dropout (`float`):
Attention dropout
softmax_scale (`float`, *optional*):
The scaling of QK^T before applying softmax. Default to 1 / sqrt(head_dim)
Expand Down

0 comments on commit 9acce7d

Please sign in to comment.