網上同行評估數據集（Computers and Society）
- 2020 年 1 月 2 日
- 筆記
同行評估實驗在特倫託大學一年級和二年級學生中進行。這些實驗歷時一整個學期並在2013年至2016年期間進行了五門電腦科學課程。同行評估任務包括問題和答案提交以及答案評估任務。同行評估數據集由每個課程的參與學生的最終分數來補充。老師們每周都會過濾學生提交的問題。然後被選中的問題會被用於隨後的同行評估任務。然而，專家評級不包括在數據集中。做出這一決定的一個主要原因是，同行評估的任務在設計時考慮的是只能有少量教師監督。同時支援這種方法的論據也被提出。數據集的設計方式使他們被允許在各種實驗中使用它們。它們被報告為可分析的數據結構，通過中間處理，可以被模壓成NLP或ML-ready數據集。潛在的應用包括性能預測和文本相似度任務。
原文標題：Computers and Society：Online Peer-Assessment Datasets
原文：
Peer-assessment experiments were conducted among first and second year students at the University of Trento. The experiments spanned an entire semester and were conducted in five computer science courses between 2013 and 2016. Peer-assessment tasks included question and answer submission as well as answer evaluation tasks. The peer-assessment datasets are complimented by the final scores of participating students for each course. Teachers were involved in filtering out questions submitted by students on a weekly basis. Selected questions were then used in subsequent peer-assessment tasks. However, expert ratings are not included in the dataset. A major reason for this decision was that peer-assessment tasks were designed with minimal teacher supervision in mind. Arguments in favour of this approach are presented. The datasets are designed in a manner that would allow their utilization in a variety of experiments. They are reported as parsable data structures that, with intermediate processing, can be moulded into NLP or ML-ready datasets. Potential applications of interest include performance prediction and text similarity tasks.
原文作者：Michael Mogessie Ashenafi
原文鏈接：https://arxiv.org/abs/1912.13050
