Shared-Task-7 Chinese Essay Discourse Coherence Evaluation

最新消息

时间	消息
5月23日	track4的训练集和验证集有更新，训练集增加250条数据，验证集增加10条数据
5月20日	测试集已发布，结果提交链接已发送至各组队长邮箱，提交结果榜单详见各赛道Github主页
5月8日	已通过邮件的方式将官方微信群二维码发送给5月5日之前报名的队伍

排行榜

  结果统计截止至2023年5月31日，榜单更新时间：2023年5月31日。

Track 1 (每队历史最好成绩排行榜)

Team Name	Organization	Precision	Recall	Macro-F1	Accuracy
EssayFlow	北京大学	38.50	43.54	32.54	43.99
Evay Info AI Team	山东省计算机科学中心	35.64	35.70	35.61	36.05
ouchnai	国家开放大学	36.38	41.32	33.22	34.92
CLsuper	广东外语外贸大学	34.13	34.28	32.80	32.88

Track 1 (2023年5月31日提交结果，未排名)

Team Name	Email	Precision	Recall	Macro-F1	Accuracy
EssayFlow	210***@stu.pku.edu.cn	35.83	35.78	33.05	40.14
Evay Info AI Team	921***@qq.com	35.64	35.70	35.61	36.05

Track 2(每队历史最好成绩排行榜)

Team Name	Organization	Paragraph Accuracy	Full Accuracy	Final Accuracy	Paragraph Similarity	Full Similarity
wuwuwu	上海交通大学	61.27	34.92	42.82	87.34	80.37
ouchnai	国家开放大学	62.61	33.33	42.12	85.20	79.16

Track 2 (2023年5月31日提交结果，未排名)

Team Name	Email	Paragraph Accuracy	Full Accuracy	Final Accuracy	Paragraph Similarity	Full Similarity
ouchnai	zhe***@ouchn.edu.cn	62.61	33.33	42.12	85.20	79.16

Track 3 (每队历史最好成绩排行榜)

Team Name	Organization	Precision	Recall	Macro-F1	Accuracy
ouchnai	国家开放大学	54.66	52.45	52.16	71.03
wuwuwu	上海交通大学	29.26	28.98	28.77	46.97
Lrt123	北京师范大学	28.19	30.26	27.54	48.81
BLCU_teamworkers	北京语言大学	27.17	27.65	25.95	48.73

Track 3 (2023年5月31日提交结果，未排名)

Team Name	Email	Precision	Recall	Macro-F1	Accuracy
BLCU_teamworkers	sol***@163.com	24.91	25.56	19.58	45.09
wuwuwu	don***@sjtu.edu.cn	25.75	16.25	4.88	7.53
Lrt123	sun***@mail.bnu.edu.cn	28.19	30.26	27.54	48.81

Track 4 （每队历史最好成绩排行榜）

Team Name	Organization	Precision	Recall	Macro-F1	Accuracy
ouchnai	国家开放大学	36.63	36.36	34.38	53.95
wuwuwu	上海交通大学	23.49	25.37	23.67	39.94
BLCU_teamworkers	北京语言大学	7.55	6.30	6.32	18.35

Track 4 (2023年5月31日提交结果，未排名)

Team Name	Email	Precision	Recall	Macro-F1	Accuracy
wuwuwu	don***@sjtu.edu.cn	23.49	25.37	23.67	39.94
ouchnai	zhe***@ouchn.edu.cn	35.86	39.22	32.68	55.90
BLCU_teamworkers	sol***@163.com	7.55	6.30	6.32	18.35

Introduction

In the scoring of the Chinese National College Entrance Examination (NCEE) and the Senior High School Entrance Examination, essay assessment is the most time-consuming and controversial task. While existing research has focused on language factors such as characters, words, and sentences, it has not explored the relationship between discourse coherence and text quality. The logical structure and coherence within an essay are essential for evaluation, but the lack of large-scale, high-quality discourse coherence evaluation data resources has hindered the development of AI essay grading. To address this issue, the CubeNLP laboratory of East China Normal University and Microsoft have constructed a Chinese essay coherence evaluation dataset called LEssay, which provides high-quality data resources and is significant for the development of automatic essay evaluation.

This shared task includes four tracks:
Track 1. Coherence Evaluation (CE). Given a middle school student essay, annotators will assess its coherence on a three-level scale of excellent, moderate, and poor. A score of 2 indicates excellent coherence, 1 indicates moderate coherence, and 0 indicates incoherence.
Track 2. Text Topic Extraction (TTE). Given a middle school student essay, annotators need to identify the topic sentence for each paragraph and one main topic sentence for the whole essay.
Track 3. Paragraph Logical Relation Recognition (PLRR). Given two paragraphs sorted in order from a composition, the annotator needs to determine the logical relationship between the two paragraphs based on the given definitions and examples of logical relationships.
Track 4. Sentence Logical Relation Recognition (SLRR). Given two sentences from an essay that are ordered sequentially, the annotator needs to determine what type of logical relation exists between them based on given definitions and examples.

The detailed content of the guideline can be found in the file Chinese Essay Discourse Coherence Evaluation.pdf