Evaluating the evaluators: A comparative study of AI and teacher assessments in Higher Education
[ X ]
Tarih
2024
Yazarlar
Dergi Başlığı
Dergi ISSN
Cilt Başlığı
Yayıncı
Univ Barcelona, Res Group Educ & Virtual Learning, Digital Educ Observatory
Erişim Hakkı
info:eu-repo/semantics/closedAccess
Özet
This study aims to examine the potential differences between teacher evaluations and artificial intelligence (AI) tool-based assessment systems in university examinations. The research has evaluated a wide spectrum of exams including numerical and verbal course exams, exams with different assessment styles (project, test exam, traditional exam), and both theoretical and practical course exams. These exams were selected using a criterion sampling method and were analyzed using BlandAltman Analysis and Intraclass Correlation Coefficient (ICC) analyses to assess how AI and teacher evaluations performed across a broad range. The research findings indicate that while there is a high level of proficiency between the total exam scores assessed by artificial intelligence and teacher evaluations; medium consistency was found in the evaluation of visually based exams, low consistency in video exams, high consistency in test exams, and low consistency in traditional exams. This research is crucial as it helps to identify specific areas where artificial intelligence can either complement or needs improvement in educational assessment, guiding the development of more accurate and fair evaluation tools.
Açıklama
Anahtar Kelimeler
Artificial intelligence tool-based assessment systems, teacher evaluation, assessments in higher education
Kaynak
Digital Education Review
WoS Q Değeri
N/A
Scopus Q Değeri
Q2
Cilt
Sayı
45
