Skip to content

Commit

Permalink
fix backward process of rope
Browse files Browse the repository at this point in the history
rope backward process was broken after YaRN RoPE (ggerganov#2268) implementation, due to missing changes in backward functions.

the code for the backward process is nearly identically to the forward process:
the only difference is the sign of the sin-values.

to avoid future regressions remove the near-duplicate backward functions and reuse the forward code:

for this a new function argument `bool forward` was added to `ggml_compute_forward_rope_f32` and `ggml_compute_forward_rope_f16`.
the sin-values will be negated when forward is false.
  • Loading branch information
xaedes committed Nov 6, 2023
1 parent 2833a6f commit ff0a364
Show file tree
Hide file tree
Showing 2 changed files with 89 additions and 254 deletions.
Loading

0 comments on commit ff0a364

Please sign in to comment.