qwen1.5 vs qwen2的一些评测数据对比

[Valdanitooooo]

qwen1.5 vs qwen2

Model	Quantification	Non-Emb Params	MMLU	C-Eval	IFEval	GSM8K	MATH	HumanEval	MBPP	BBH	CMMLU
GPT-4	BF16	-	86.4	69.9	-	92.0	45.8	67.0	61.8	86.7	71.0
Llama2-7B	BF16	-	46.8	32.5	-	16.7	3.3	12.8	20.8	38.2	31.8
Llama2-13B	BF16	-	55.0	41.4	-	29.6	5.0	18.9	30.3	45.6	38.4
Llama2-34B	BF16	-	62.6	-	-	42.2	6.2	22.6	33.0	44.1	-
Llama2-70B	BF16	-	69.8	50.1	-	54.4	10.6	23.7	37.7	58.4	53.6
Mistral-7B	BF16	-	64.1	47.4	-	47.5	11.3	27.4	38.6	56.7	44.7
Mixtral-8x7B	BF16	-	70.6	-	-	74.4	28.4	40.2	60.7	-	-
Qwen1.5-0.5B	BF16	0.3B	39.2	50.5	-	22.0	3.1	12.2	6.8	18.3	46.6
Qwen1.5-1.8B	BF16	1.2B	46.8	59.7	-	38.4	10.1	20.1	18.0	24.2	57.8
Qwen1.5-4B	BF16	3.1B	56.1	67.6	-	57.0	10.0	25.6	29.2	32.5	66.7
Qwen1.5-MoE-A2.7B	BF16	2.0B	62.5	79.2	-	61.5	21.9	34.2	36.6	39.1	79.2
Qwen1.5-7B	BF16	6.5B	61.0	74.1	-	62.5	20.3	36.0	37.4	40.2	73.1
Qwen1.5-14B	BF16	-	67.6	78.7	-	70.1	29.2	37.8	44.0	53.7	77.6
Qwen1.5-32B	BF16	-	73.4	83.5	-	77.4	36.1	37.2	49.4	66.8	82.3
Qwen1.5-72B	BF16	-	77.5	84.1	-	79.5	34.1	41.5	53.4	65.5	83.5
Qwen2-0.5B-Instruct	BF16	0.35B	37.9	45.2	20.0	-	-	-	-	-	-
Qwen2-0.5B-Instruct	GPTQ-Int8	-	35.6	43.9	18.1	-	-	-	-	-	-
Qwen2-0.5B-Instruct	GPTQ-Int4	-	33.0	39.2	16.8	-	-	-	-	-	-
Qwen2-0.5B-Instruct	AWQ	-	34.4	42.1	16.7	-	-	-	-	-	-
Qwen2-1.5B-Instruct	BF16	1.31B	52.4	63.8	29.0	-	-	-	-	-	-
Qwen2-1.5B-Instruct	GPTQ-Int8	-	53.0	62.5	28.8	-	-	-	-	-	-
Qwen2-1.5B-Instruct	GPTQ-Int4	-	50.7	57.4	27.0	-	-	-	-	-	-
Qwen2-1.5B-Instruct	AWQ	-	46.5	58.1	29.9	-	-	-	-	-	-
Qwen2-7B-Instruct	BF16	5.98B	70.5	77.2	53.1	-	-	-	-	-	-
Qwen2-7B-Instruct	GPTQ-Int8	-	69.1	76.7	52.9	-	-	-	-	-	-
Qwen2-7B-Instruct	GPTQ-Int4	-	67.8	75.2	49.4	-	-	-	-	-	-
Qwen2-7B-Instruct	AWQ	-	67.4	73.6	51.4	-	-	-	-	-	-
Qwen2-57B-A14B-Instruct	BF16	56.32B	75.4	80.5	-	-	-	-	-	-
Qwen2-72B-Instruct	BF16	70.21B	82.3	83.8	77.6	-	-	-	-	-	-
Qwen2-72B-Instruct	GPTQ-Int8	-	81.3	83.4	77.5	-	-	-	-	-	-
Qwen2-72B-Instruct	GPTQ-Int4	-	80.8	83.9	78.9	-	-	-	-	-	-
Qwen2-72B-Instruct	AWQ	-	80.5	83.9	76.9	-	-	-	-	-	-

数据源

• https://qwenlm.github.io/blog/qwen1.5/
• https://qwenlm.github.io/blog/qwen2/