Skip to content

Commit

Permalink
ggml : fix backward rope after YaRN (ggerganov#3974)
Browse files Browse the repository at this point in the history
* fix backward process of rope

rope backward process was broken after YaRN RoPE (ggerganov#2268) implementation, due to missing changes in backward functions.

the code for the backward process is nearly identically to the forward process:
the only difference is the sign of the sin-values.

to avoid future regressions remove the near-duplicate backward functions and reuse the forward code:

for this a new function argument `bool forward` was added to `ggml_compute_forward_rope_f32` and `ggml_compute_forward_rope_f16`.
the sin-values will be negated when forward is false.

* fix finetune rope call to use correct default attn_factor of 1.0f

* remove unused `ggml_rope_xpos_back`

it is better to have only one `ggml_rope_back` function that accepts all rope parameters, so that `ggml_compute_backward` can propagate all parameters without having to switch between different rope_back variants.

* fix comments explaining the sinus sign in ggml_forward_rope

* add missing function arguments in declaration

* fix function argument type in declaration
  • Loading branch information
xaedes authored and olexiyb committed Nov 23, 2023
1 parent d3e96f8 commit 8a7252d
Show file tree
Hide file tree
Showing 3 changed files with 84 additions and 253 deletions.
2 changes: 1 addition & 1 deletion examples/finetune/finetune.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -643,7 +643,7 @@ static struct ggml_tensor * llama_build_lora_finetune_graphs(

return ggml_rope_custom(ctx,
t, KQ_pos, n_rot, rope_mode, n_ctx, 0,
rope_freq_base, rope_freq_scale, 0.0f, 0.0f, 0.0f, 0.0f
rope_freq_base, rope_freq_scale, 0.0f, 1.0f, 0.0f, 0.0f
);
};

Expand Down
Loading

0 comments on commit 8a7252d

Please sign in to comment.