Skip to content

Commit

Permalink
[HGRN2] Allow passing both num_heads and expand_ratio
Browse files Browse the repository at this point in the history
  • Loading branch information
yzhangcs authored Jul 1, 2024
1 parent 4351b6f commit 47f687d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion fla/layers/hgrn2.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ def __init__(
expand_ratio = hidden_size // num_heads
elif expand_ratio is not None and num_heads is None:
num_heads = hidden_size // expand_ratio
else:
elif expand_ratio is None and num_heads is None:
raise RuntimeError("One of `expand_ratio` or `num_heads` should be provided.")
self.num_heads = num_heads
self.expand_ratio = expand_ratio
Expand Down

0 comments on commit 47f687d

Please sign in to comment.