Support WikiText & Lambada (PaddlePaddle#7079)

* Support WikiText & Lambada * Support WikiText & Lambada * update
lugimzzz · Sep 20, 2023 · d2524ab · d2524ab
1 parent f18dcc8
commit d2524ab
Show file tree

Hide file tree

Showing 2 changed files with 416 additions and 0 deletions.
diff --git a/examples/benchmark/wiki_lambada/README.md b/examples/benchmark/wiki_lambada/README.md
@@ -0,0 +1,32 @@
+
+## 数据准备
+```
+wget https://paddlenlp.bj.bcebos.com/data/benchmark/lambada_test.jsonl
+```
+```
+wget https://paddlenlp.bj.bcebos.com/data/benchmark/wikitext-103.tar.gz
+```
+```
+wget https://paddlenlp.bj.bcebos.com/data/benchmark/wikitext-2.tar.gz
+```
+
+## 运行预测脚本
+
+验证Lambada数据集，运行以下脚本：
+```
+python eval.py \
+--model_name_or_path /path/to/your/model \
+--batch_size 4 \
+--eval_path /path/to/your/dataset/lambada_test.jsonl \
+--tensor_parallel_degree 1 \
+--cloze_eval
+```
+
+验证WikiText数据集，运行以下脚本：
+```
+python eval.py \
+--model_name_or_path /path/to/your/model \
+--batch_size 4 \
+--eval_path /path/to/your/dataset/wikitext-103/wiki.valid.tokens \
+--tensor_parallel_degree 1
+```