Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[model] Support Audio #6701

Merged
merged 9 commits into from
Feb 4, 2025
Merged

[model] Support Audio #6701

merged 9 commits into from
Feb 4, 2025
+735 −212

Conversation

BUAADreamer
Copy link
Collaborator

@BUAADreamer BUAADreamer commented Jan 18, 2025 •
edited by hiyouga
Loading

What does this PR do?

Support audio-text to text finetuning and inference of MiniCPM-o-2.6/qwen2-audio

Fixes #5756
Fixes #6738

Before submitting

hiyouga
hiyouga reviewed Jan 31, 2025
View reviewed changes
setup.py Outdated Show resolved Hide resolved
hiyouga
hiyouga reviewed Jan 31, 2025
View reviewed changes
src/llamafactory/data/collator.py Outdated Show resolved Hide resolved
src/llamafactory/data/mm_plugin.py Outdated Show resolved Hide resolved
src/llamafactory/data/mm_plugin.py Outdated Show resolved Hide resolved
@BUAADreamer BUAADreamer force-pushed the qwen2_audio branch 3 times, most recently from 5455ce9 to b9dec36 Compare February 1, 2025 01:32
@BUAADreamer BUAADreamer force-pushed the qwen2_audio branch 2 times, most recently from fdc002f to caf80f0 Compare February 1, 2025 06:38
@BUAADreamer BUAADreamer requested a review from hiyouga February 4, 2025 04:22
hiyouga
hiyouga approved these changes Feb 4, 2025
View reviewed changes
Copy link
Owner

@hiyouga hiyouga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hiyouga hiyouga added solved This problem has been already solved and removed pending This problem is yet to be addressed labels Feb 4, 2025
@hiyouga hiyouga merged commit 24c7842 into hiyouga:main Feb 4, 2025
11 of 12 checks passed
This was referenced Feb 5, 2025
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hiyouga hiyouga hiyouga approved these changes

Assignees
No one assigned
Labels
solved This problem has been already solved
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

请求支持 qwen2-audio 有计划支持whisper、qwen2-audio之类的asr模型微调吗？
3 participants
@BUAADreamer @Lingeng56 @hiyouga