Release v0.5.2

Highlight features

• Support Multi-modal models evaluation (VLM Eval)
• Transform the synchronous API to asynchronous for OpenAI's API format, speed up the evaluation process up to 10x .
• Support installation with format: pip install evalscope[opencompass] or pip install evalscope[vlmeval]

Breaking Changes

None

What's Changed

1. Support Multi-modal models evaluation (VLM Eval)
2. Transform the synchronous API to asynchronous for OpenAI's API format, speed up the evaluation process up to 10x .
3. Support installation with format: pip install evalscope[opencompass] or pip install evalscope[vlmeval]
4. Update README
5. Add UT cases for VLM eval
6. Update examples for OpenCompass and VLMEval eval backends
7. Update version restrictions for ms-opencompass and ms-vlmeval dependencies.

Release v0.4.3

1. Support async client infer for OpenAI API format evaluation
2. Support mulati-modal evaluation with VLMEvalKit as a eval-backend
3. Refactor setup, support pip install llmuses[opencompass], pip install llmuses[vlmeval], pip install llmuses[all]
4. Fix some bugs

Release v0.2.8

1. Fix local dir eval
2. Add fuzzy match for templates
3. Add local models with templates

Release v0.2.6

1. Support loading cmmlu from local disk

Release v0.2.5

Init release version: 0.2.5