Statistical analysis of peer grading: a latent variable approach

Peer grading is an educational system in which students assess each other’s work. It is commonly applied under Massive Open Online Course (MOOC) and offline classroom settings. Peer grading data have a complex network structure, where each student is a vertex of the network, and each peer grade serves as an edge connecting one student as a grader to another student as an examinee. We introduce a latent variable model framework for analyzing peer grading data and develop a fully Bayesian procedure for its statistical inference. The proposed approach produces more accurate aggregated grades by modelling the heterogeneous grading behaviour with latent variables and provides a way to assess each student’s performance as a grader. It may be used to identify a pool of reliable graders or generate feedback to help students improve their grading. Thanks to the Bayesian approach, uncertainty quantification is straightforward when inferring the student-specific latent variables as well as the structural parameters of the model. The proposed method is applied to a real-world dataset.