Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

model_runner simplify #329

Merged
merged 1 commit into from
Mar 24, 2024
Merged

model_runner simplify #329

merged 1 commit into from
Mar 24, 2024

Conversation

hnyls2002
Copy link
Collaborator

dispatch different forward modes' argument only inside the function, remove the **kwargs for better simplicity.

def forward(self, batch: Batch, forward_mode: ForwardMode):
if self.is_multimodal_model and forward_mode == ForwardMode.EXTEND:
return self.forward_extend_multi_modal(batch)
elif forward_mode == ForwardMode.DECODE:
return self.forward_decode(batch)
elif forward_mode == ForwardMode.EXTEND:
return self.forward_extend(batch)
elif forward_mode == ForwardMode.PREFILL:
return self.forward_prefill(batch)
else:
raise ValueError(f"Invaid forward mode: {forward_mode}")

@hnyls2002 hnyls2002 merged commit 24e59f5 into main Mar 24, 2024
@hnyls2002 hnyls2002 deleted the model-runner branch March 24, 2024 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant