Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fix] Fix cache_dir issue where MVBench cannot be found #306

Merged
merged 15 commits into from
Oct 9, 2024
Merged

[Fix] Fix cache_dir issue where MVBench cannot be found #306

merged 15 commits into from
Oct 9, 2024
+15 −0

Conversation

yinanhe
Copy link
Contributor

@yinanhe yinanhe commented Oct 8, 2024

[Fix] Fix cache_dir issue where MVBench cannot be found
- Added Line 1004-1012 in lmms_eval/api/task.py. If the downloaded file does not need to be unzipped (unzip or tar), it will directly create the symbolic link the cache path of huggingface to the cache_dir, and the link will be updated after each pull. This operation also applies to dataset-repo that does not need to be unzipped.

heyinan and others added 15 commits September 23, 2024 18:43
[add] add internvideo2 support && change mvbench to video branch
86f1a83
Merge branch 'main' of https://github.com/yinanhe/lmms-eval into main
3e7cb78
[add] answer_prompt of internvideo2
f4cfe9b
[add] change video type of internvideo2
d368177
[fix] update template of mvbench
6439c30
@yinanhe
Merge branch 'EvolvingLMMs-Lab:main' into main
bdd132e
[reformat]
f6548f5
Merge branch 'main' of https://github.com/yinanhe/lmms-eval into main
80369c7
[fix] generate_until_multi_round
5a9d47a
@yinanhe
Merge branch 'EvolvingLMMs-Lab:main' into main
7ac170d
[Feat] videochat2 support
1ab9504
@yinanhe
Merge branch 'EvolvingLMMs-Lab:main' into main
20a9c9c
[feat] Link cache_path to cache_dir if no unzip or untar
3770abe
@yinanhe
Merge branch 'EvolvingLMMs-Lab:main' into main
66492f8
[feat] new variable in dataset kwargs: create_link
6d35107
@yinanhe
Copy link
Contributor Author

yinanhe commented Oct 8, 2024

  • [feat] New variable in dataset kwargs: create_link
    If create_link exists in dataset_kwargs, a symbolic link will be created to the data repository in cache_dir.
  • [fix] Modify the print function in the previous code to use logger

@Luodian Luodian merged commit 581a672 into EvolvingLMMs-Lab:main Oct 9, 2024
1 check passed
pufanyi added a commit that referenced this pull request Oct 9, 2024
@pufanyi
Squashed commit of the following:
1dd348a 
commit 581a672
Author: Yinan He <43169235+yinanhe@users.noreply.github.com>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <heyinan@pjlab.org.cn>

commit 99fcd39
Author: GIO PAIK <giopaik0@gmail.com>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (#305)

commit 6e1d747
Author: Li Bo <drluodian@gmail.com>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303)

commit b838ff6
Author: Pu Fanyi <FPU001@e.ntu.edu.sg>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit 09d15c3
Author: Zhijian Liu <5782437+zhijian-liu@users.noreply.github.com>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (#296)
kcz358 pushed a commit that referenced this pull request Oct 9, 2024
@pufanyi
[Fix] LiveBench 2409 (#308)
39e1586 
* Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

* Refactor DefaultWebsite class in website.py

* Refactor dataset name and limit processed images to 2

* Refactor extract_infomation.py and update prompts

* update

* Refactor live_bench_2409.yaml and live_bench.yaml

* Refactor live_bench/utils_v2.py and question_finalizer.py

* Squashed commit of the following:

commit 581a672
Author: Yinan He <43169235+yinanhe@users.noreply.github.com>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <heyinan@pjlab.org.cn>

commit 99fcd39
Author: GIO PAIK <giopaik0@gmail.com>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (#305)

commit 6e1d747
Author: Li Bo <drluodian@gmail.com>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303)

commit b838ff6
Author: Pu Fanyi <FPU001@e.ntu.edu.sg>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit 09d15c3
Author: Zhijian Liu <5782437+zhijian-liu@users.noreply.github.com>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (#296)

* fix
@kcz358 kcz358 mentioned this pull request Oct 11, 2024
Open
KairuiHu pushed a commit that referenced this pull request Oct 24, 2024
@yinanhe @KairuiHu
[Fix] Fix cache_dir issue where MVBench cannot be found (#306)
4ddf9e6 
* [add] add internvideo2 support && change mvbench to video branch

* [add] answer_prompt of internvideo2

* [add] change video type of internvideo2

* [fix] update template of mvbench

* [reformat]

* [fix] generate_until_multi_round

* [Feat] videochat2 support

* [feat] Link cache_path to cache_dir if no unzip or untar

* [feat] new variable in dataset kwargs: create_link

---------

Co-authored-by: heyinan <heyinan@pjlab.org.cn>
KairuiHu pushed a commit that referenced this pull request Oct 24, 2024
@pufanyi @KairuiHu
[Fix] LiveBench 2409 (#308)
3eefe54 
* Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

* Refactor DefaultWebsite class in website.py

* Refactor dataset name and limit processed images to 2

* Refactor extract_infomation.py and update prompts

* update

* Refactor live_bench_2409.yaml and live_bench.yaml

* Refactor live_bench/utils_v2.py and question_finalizer.py

* Squashed commit of the following:

commit 581a672
Author: Yinan He <43169235+yinanhe@users.noreply.github.com>
Date:   Wed Oct 9 06:11:24 2024 +0200

    [Fix] Fix cache_dir issue where MVBench cannot be found (#306)

    * [add] add internvideo2 support && change mvbench to video branch

    * [add] answer_prompt of internvideo2

    * [add] change video type of internvideo2

    * [fix] update template of mvbench

    * [reformat]

    * [fix] generate_until_multi_round

    * [Feat] videochat2 support

    * [feat] Link cache_path to cache_dir if no unzip or untar

    * [feat] new variable in dataset kwargs: create_link

    ---------

    Co-authored-by: heyinan <heyinan@pjlab.org.cn>

commit 99fcd39
Author: GIO PAIK <giopaik0@gmail.com>
Date:   Tue Oct 8 15:56:53 2024 +0900

    fix: Invalid group in mmsearch.yaml (#305)

commit 6e1d747
Author: Li Bo <drluodian@gmail.com>
Date:   Tue Oct 8 08:42:20 2024 +0800

    [Doc] add more detailed task guide to explain the variables in yaml configuration file (#303)

commit b838ff6
Author: Pu Fanyi <FPU001@e.ntu.edu.sg>
Date:   Tue Oct 8 01:25:35 2024 +0800

    [Feat] LiveBench 2409 (#304)

    * Refactor code formatting in setup.py, .gitignore, __init__.py, and data_summary.ipynb

    * Refactor DefaultWebsite class in website.py

    * Refactor dataset name and limit processed images to 2

    * Refactor extract_infomation.py and update prompts

    * update

    * Refactor live_bench_2409.yaml and live_bench.yaml

commit 09d15c3
Author: Zhijian Liu <5782437+zhijian-liu@users.noreply.github.com>
Date:   Sat Oct 5 11:46:44 2024 -0400

    Fix MMMU-Pro evaluation (#296)

* fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Luodian Luodian Luodian approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@yinanhe @Luodian