网上同行评估数据集（Computers and Society）
- 2020 年 1 月 2 日
- 筆記
同行评估实验在特伦托大学一年级和二年级学生中进行。这些实验历时一整个学期并在2013年至2016年期间进行了五门计算机科学课程。同行评估任务包括问题和答案提交以及答案评估任务。同行评估数据集由每个课程的参与学生的最终分数来补充。老师们每周都会过滤学生提交的问题。然后被选中的问题会被用于随后的同行评估任务。然而，专家评级不包括在数据集中。做出这一决定的一个主要原因是，同行评估的任务在设计时考虑的是只能有少量教师监督。同时支持这种方法的论据也被提出。数据集的设计方式使他们被允许在各种实验中使用它们。它们被报告为可分析的数据结构，通过中间处理，可以被模压成NLP或ML-ready数据集。潜在的应用包括性能预测和文本相似度任务。
原文标题：Computers and Society：Online Peer-Assessment Datasets
原文：
Peer-assessment experiments were conducted among first and second year students at the University of Trento. The experiments spanned an entire semester and were conducted in five computer science courses between 2013 and 2016. Peer-assessment tasks included question and answer submission as well as answer evaluation tasks. The peer-assessment datasets are complimented by the final scores of participating students for each course. Teachers were involved in filtering out questions submitted by students on a weekly basis. Selected questions were then used in subsequent peer-assessment tasks. However, expert ratings are not included in the dataset. A major reason for this decision was that peer-assessment tasks were designed with minimal teacher supervision in mind. Arguments in favour of this approach are presented. The datasets are designed in a manner that would allow their utilization in a variety of experiments. They are reported as parsable data structures that, with intermediate processing, can be moulded into NLP or ML-ready datasets. Potential applications of interest include performance prediction and text similarity tasks.
原文作者：Michael Mogessie Ashenafi
原文链接：https://arxiv.org/abs/1912.13050
