Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix backward rope after YaRN #3974

Merged
merged 6 commits into from
Nov 7, 2023
Merged

Fix backward rope after YaRN #3974

merged 6 commits into from
Nov 7, 2023

Commits on Nov 6, 2023

  1. fix backward process of rope 

    rope backward process was broken after YaRN RoPE (ggerganov#2268) implementation, due to missing changes in backward functions.

the code for the backward process is nearly identically to the forward process:
the only difference is the sign of the sin-values.

to avoid future regressions remove the near-duplicate backward functions and reuse the forward code:

for this a new function argument `bool forward` was added to `ggml_compute_forward_rope_f32` and `ggml_compute_forward_rope_f16`.
the sin-values will be negated when forward is false.
    @xaedes
    xaedes committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    ff0a364 View commit details
    Browse the repository at this point in the history

  2. fix finetune rope call to use correct default attn_factor of 1.0f

    @xaedes
    xaedes committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    93e65d3 View commit details
    Browse the repository at this point in the history

  3. remove unused ggml_rope_xpos_back 

    it is better to have only one `ggml_rope_back` function that accepts all rope parameters, so that `ggml_compute_backward` can propagate all parameters without having to switch between different rope_back variants.
    @xaedes
    xaedes committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    926930b View commit details
    Browse the repository at this point in the history

  4. fix comments explaining the sinus sign in ggml_forward_rope

    @xaedes
    xaedes committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    3490a17 View commit details
    Browse the repository at this point in the history

  5. add missing function arguments in declaration

    @xaedes
    xaedes committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    ad83e41 View commit details
    Browse the repository at this point in the history

  6. fix function argument type in declaration

    @xaedes
    xaedes committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    a352d40 View commit details
    Browse the repository at this point in the history