Korea Business Review

Korea Business Review

Edited by: Hyoung-Goo Kang Print ISSN: 2951-3596 Online ISSN: 2951-360X © THE KOREAN ACADEMIC SOCIETY OF BUSINESS ADMINISTRATION. Frequency: Quarterly (four times per year): February 28, May 31, August 31, and November 30
Cover
Latest Issue Vol. 60, No. 5 2025

Article Info

How Teacher-educators and ChatGPT Evaluate Pre-service Teachers’ Writing? : Focused on Newspaper Editorials

Ahn Sanghee Kim,Kyoung Hwa Kim, Youngran

전북대학교 청주대학교 강원대학교

Korea Business Review 60Vol. 5No. pp.5-39 (2025)

DOI: 10.20880/kler.2025.60.5.5

Abstract

This study explores how teacher educators (TEs) and ChatGPT construct and apply evaluation criteria in assessing pre-service teachers’ (PTs) writing, focusing on university newspaper editorials. Nineteen PTs majoring in Korean language education participated in the study. Three TEs collaboratively developed evaluation criteria and a rubric to rate the texts, while ChatGPT (Plus version) was instructed to create its own rubric and later assess the same texts using the TEs’ rubric. Both TEs and ChatGPT emphasized logical reasoning and persuasiveness as core elements of the editorial genre. However, TEs developed detailed, content-oriented criteria highlighting argumentative depth and reader awareness, whereas ChatGPT presented integrated, form-oriented criteria emphasizing clarity, structure, and media adaptability. When applying the same rubric, TEs exhibited a wider scoring range with greater qualitative sensitivity, while ChatGPT showed a convergent mid-range scoring pattern. These results indicate that ChatGPT can apply teacher-developed rubrics consistently but lacks contextual and interpretive nuance. The scoring results of ChatGPT also showed low discriminatory power of the scores. It may serve as a supplementary tool that helps teachers refine criteria & rubric design or cross-check their practice of evaluation, suggesting potential for AI–human collaboration in writing evaluation.

Keywords

AI-assisted writing evaluationteacher educatorsgenerative AIChatGPTrubric development

References

  1. [1] [학술지(정기간행물)] AlAfnan M. A./Chatgpt as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses/Journal of Artificial Intelligence and Technology/2023/3(2)/60~68//
  2. [2] [학술대회논문] Allam H./Artificial intelligence in education: an argument of Chat-GPT use in education/2023 9th International Conference on Information Technology Trends (ITT)/2023//151~156/IEEE
  3. [3] [학술지(정기간행물)] Awidi, I. T./Comparing expert tutor evaluation of reflective essays with marking by generative artificial intelligence (AI) tool/Computers and Education: Artificial Intelligence/2024/6/100226~//
  4. [4] [학술지(정기간행물)] Cha J. W./A Comparative Study of AI-Based Writing Assessment Tools and Human Evaluators - Focusing on book report texts -/Korean Language and Literature in International Context/2024/(102)/681~705//
  5. [5] [학술지(정기간행물)] Choi J. Y./Calibrating human and machine: Developing an AI-assisted rater discussion model for automated writing evaluation(AWE)/Journal of Korean Association for Educational Information and Media/2025/31(4)/1297~1329//
  6. [6] [학술지(정기간행물)] Choi, S. G./Development and Validation of Quality Assessment Tools for AI-Powered Automated Writing Feedback/Journal of CheongRam Korean Language Education/2025/103/227~263//
  7. [7] [학술지(정기간행물)] Herbold S./A large-scale comparison of human-written versus ChatGPT-generated essays/Scientific reports/2023/13(1)/18617~//
  8. [8] [학술지(정기간행물)] Imran M./Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature/Contemporary Educational Technology/2023/15(4)/ep464~//
  9. [9] [학술지(정기간행물)] Jang, S. M./ChatGPT has Changed the Future of Writing Education - Focusing on the response of writing education in the era of artificial intelligence -/Research on Writing/2023/56/7~34//
  10. [10] [학술지(정기간행물)] Jauhiainen, J. S./Generative AI in education: ChatGPT-4 in evaluating students’ written responses/Innovations in Education and Teaching International/2025/62(4)/1377~1394//
  11. [11] [학술지(정기간행물)] Kim, J. Y./The Effects and Limitations of Using AI-Based Automatic Writing Feedback Programs in Writing Classes - Focusing on the Case of Using KEEwi in Reading & Expression Classes -/The Journal of Yeongju Language & Literature/2024/58/355~386//
  12. [12] [학술지(정기간행물)] Kolade, O./Is AI changing learning and assessment as we know it? Evidence from a ChatGPT experiment and a conceptual framework/Heliyon/2024/10(4)/~//
  13. [13] [학술지(정기간행물)] Lee, Y. B./A Comparative Study on the Characteristics and Validity of Instructor, Peer, and AI Feedback in College Writing Education : Focusing on Common Issues in Students’ Argumentative Texts/Korean Journal of General Education/2025/19(3)/19~33//
  14. [14] [학술지(정기간행물)] Lee, Y. B./Reframing College Writing Education in the Era of Generative AI － An Integrated ‘AI-Assisted Collaborative Writing Pedagogy’ Combining Process-Based and Genre-Based Instruction with AI Literacy/The Korean Journal of Literacy Research/2025/16(4)/175~207//
  15. [15] [학술지(정기간행물)] Mao, J./Generative artificial intelligence in education and its implications for assessment/TechTrends/2024/68(1)/58~66//
  16. [16] [학술지(정기간행물)] Megawati R./The role of GPT chat in writing scientific articles: A systematic literature review/Jurnal Penelitian Pendidikan IPA/2023/9(11)/1078~1084//
  17. [17] [학술지(정기간행물)] Park S. Y./Exploring the Potential of AI Tools in University Writing Assessment: Comparing Evaluation Criteria between Humans and Generative AI/Journal of practical engineering education/2024/16(5)/663~676//
  18. [18] [학술지(정기간행물)] Rudolph J./ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?/Journal of applied learning and teaching/2023/6(1)/342~363//
  19. [19] [학술대회논문] Shidiq, M./The use of artificial intelligence-based chat-gpt and its challenges for the world of education; from the viewpoint of the development of creative writing skills/Proceeding of international conference on education, society and humanity/2023/1(1)/353~357/
  20. [20] [학술대회논문] Smolansky, A./Educator and student perspectives on the impact of generative AI on assessments in higher education/Proceedings of the tenth ACM conference on Learning@ Scale/2023//378~382/
  21. [21] [학술지(정기간행물)] Son, H. J./Perceptions and attitudes of English learners towards the utilization of AI-based Automated Writing Evaluation (AWE) programs and instructor feedback/English Language & Literature Teaching/2023/29(3)/91~112//
  22. [22] [학술지(정기간행물)] Son, H. S./Studying the Potential Use of Generative AI in College Writing/The Journal of General Education/2025/(31)/41~72//
  23. [23] [학술지(정기간행물)] Steiss, J./Comparing the quality of human and ChatGPT feedback of students’ writing/Learning and Instruction/2024/91/101894~//
  24. [24] [학술지(정기간행물)] /////~//
  25. [25] [학술지(정기간행물)] Wang, J./Evaluating rater accuracy and perception for integrated writing assessments using a mixed-methods approach/Assessing Writing/2017/33/36~47//
  26. [26] [학술지(정기간행물)] Yan D/Detection of AI-generated essays in writing assessments/Psychological Test and Assessment Modeling/2023/65(1)/125~144//

Korea Business Review · THE KOREAN ACADEMIC SOCIETY OF BUSINESS ADMINISTRATION.

Print ISSN 2951-3596 · Online ISSN 2951-360X

Privacy PolicyTerms of UseContact UsKMA Homepage

TEL : 010-2672-3919 EMAIL : kbr2editor@gmail.com 계좌번호 : 신한 140-009-175743 예금주 : (사)한국경영학회 COPYRIGHT ⓒ 2023 THE KOREAN ACADEMIC SOCIETY OF BUSINESS ADMINISTRATION. All rights reserved