Fix MMMU-Pro evaluation #296
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Luodian
approved these changes
Oct 5, 2024
pufanyi
added a commit
that referenced
this pull request
Oct 9, 2024
commit 581a672 Author: Yinan He <43169235+yinanhe@users.noreply.github.com> Date: Wed Oct 9 06:11:24 2024 +0200 [Fix] Fix cache_dir issue where MVBench cannot be found (#306) * [add] add internvideo2 support && change mvbench to video branch * [add] answer_prompt of internvideo2 * [add] change video type of internvideo2 * [fix] update template of mvbench * [reformat] * [fix] generate_until_multi_round * [Feat] videochat2 support * [feat] Link cache_path to cache_dir if no unzip or untar * [feat] new variable in dataset kwargs: create_link --------- Co-authored-by: heyinan <heyinan@pjlab.org.cn> commit 99fcd39 Author: GIO PAIK <giopaik0@gmail.com> Date: Tue Oct 8 15:56:53 2024 +0900 fix: Invalid group in mmsearch.yaml (#305) commit 6e1d747 Author: Li Bo <drluodian@gmail.com> Date: Tue Oct 8 08:42:20 2024 +0800 [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303) commit b838ff6 Author: Pu Fanyi <FPU001@e.ntu.edu.sg> Date: Tue Oct 8 01:25:35 2024 +0800 [Feat] LiveBench 2409 (#304) * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb * Refactor DefaultWebsite class in website.py * Refactor dataset name and limit processed images to 2 * Refactor extract_infomation.py and update prompts * update * Refactor live_bench_2409.yaml and live_bench.yaml commit 09d15c3 Author: Zhijian Liu <5782437+zhijian-liu@users.noreply.github.com> Date: Sat Oct 5 11:46:44 2024 -0400 Fix MMMU-Pro evaluation (#296)
kcz358
pushed a commit
that referenced
this pull request
Oct 9, 2024
* Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb * Refactor DefaultWebsite class in website.py * Refactor dataset name and limit processed images to 2 * Refactor extract_infomation.py and update prompts * update * Refactor live_bench_2409.yaml and live_bench.yaml * Refactor live_bench/utils_v2.py and question_finalizer.py * Squashed commit of the following: commit 581a672 Author: Yinan He <43169235+yinanhe@users.noreply.github.com> Date: Wed Oct 9 06:11:24 2024 +0200 [Fix] Fix cache_dir issue where MVBench cannot be found (#306) * [add] add internvideo2 support && change mvbench to video branch * [add] answer_prompt of internvideo2 * [add] change video type of internvideo2 * [fix] update template of mvbench * [reformat] * [fix] generate_until_multi_round * [Feat] videochat2 support * [feat] Link cache_path to cache_dir if no unzip or untar * [feat] new variable in dataset kwargs: create_link --------- Co-authored-by: heyinan <heyinan@pjlab.org.cn> commit 99fcd39 Author: GIO PAIK <giopaik0@gmail.com> Date: Tue Oct 8 15:56:53 2024 +0900 fix: Invalid group in mmsearch.yaml (#305) commit 6e1d747 Author: Li Bo <drluodian@gmail.com> Date: Tue Oct 8 08:42:20 2024 +0800 [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303) commit b838ff6 Author: Pu Fanyi <FPU001@e.ntu.edu.sg> Date: Tue Oct 8 01:25:35 2024 +0800 [Feat] LiveBench 2409 (#304) * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb * Refactor DefaultWebsite class in website.py * Refactor dataset name and limit processed images to 2 * Refactor extract_infomation.py and update prompts * update * Refactor live_bench_2409.yaml and live_bench.yaml commit 09d15c3 Author: Zhijian Liu <5782437+zhijian-liu@users.noreply.github.com> Date: Sat Oct 5 11:46:44 2024 -0400 Fix MMMU-Pro evaluation (#296) * fix
KairuiHu
pushed a commit
that referenced
this pull request
Oct 24, 2024
KairuiHu
pushed a commit
that referenced
this pull request
Oct 24, 2024
* Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb * Refactor DefaultWebsite class in website.py * Refactor dataset name and limit processed images to 2 * Refactor extract_infomation.py and update prompts * update * Refactor live_bench_2409.yaml and live_bench.yaml * Refactor live_bench/utils_v2.py and question_finalizer.py * Squashed commit of the following: commit 581a672 Author: Yinan He <43169235+yinanhe@users.noreply.github.com> Date: Wed Oct 9 06:11:24 2024 +0200 [Fix] Fix cache_dir issue where MVBench cannot be found (#306) * [add] add internvideo2 support && change mvbench to video branch * [add] answer_prompt of internvideo2 * [add] change video type of internvideo2 * [fix] update template of mvbench * [reformat] * [fix] generate_until_multi_round * [Feat] videochat2 support * [feat] Link cache_path to cache_dir if no unzip or untar * [feat] new variable in dataset kwargs: create_link --------- Co-authored-by: heyinan <heyinan@pjlab.org.cn> commit 99fcd39 Author: GIO PAIK <giopaik0@gmail.com> Date: Tue Oct 8 15:56:53 2024 +0900 fix: Invalid group in mmsearch.yaml (#305) commit 6e1d747 Author: Li Bo <drluodian@gmail.com> Date: Tue Oct 8 08:42:20 2024 +0800 [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303) commit b838ff6 Author: Pu Fanyi <FPU001@e.ntu.edu.sg> Date: Tue Oct 8 01:25:35 2024 +0800 [Feat] LiveBench 2409 (#304) * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb * Refactor DefaultWebsite class in website.py * Refactor dataset name and limit processed images to 2 * Refactor extract_infomation.py and update prompts * update * Refactor live_bench_2409.yaml and live_bench.yaml commit 09d15c3 Author: Zhijian Liu <5782437+zhijian-liu@users.noreply.github.com> Date: Sat Oct 5 11:46:44 2024 -0400 Fix MMMU-Pro evaluation (#296) * fix
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.