Korea Business Review

Korea Business Review

편집인: Hyoung-Goo Kang Print ISSN: 2951-3596 Online ISSN: 2951-360X © 한국경영학회 발행주기: 연4회 (02월 28일, 05월 31일, 08월 31일, 11월 30일)
Cover
최신호 60권 5호 2025

논문 정보

교사교육자와 생성형 AI의 쓰기 평가 비교 연구 — 대학 신문 사설을 중심으로

안상희 김경화 김영란

전북대학교 청주대학교 강원대학교

Korea Business Review 60권 5호 5-39 (2025)

DOI: 10.20880/kler.2025.60.5.5

초록

본 연구는 예비교사의 글쓰기를 평가하는 과정에서 교사교육자(TEs)와 ChatGPT가 어떠한 평가 기준을 구성하여 채점하는지를 대학 신문 사설을 중심으로 탐구하였다. 이 연구에는 국어교육 전공 예비교사 19명이 참여하였다. 교사교육자 3인은 협의를 통해 평가 기준과 루브릭을 개발하여 텍스트를 평가하였으며, ChatGPT는 자체적으로 평가 기준과 루브릭을 생성한 뒤, 교사교육자가 개발한 루브릭을 적용하여 텍스트를 평가하였다. 교사교육자와 ChatGPT는 모두 사설 장르의 핵심 요소로 논리성과 설득력을 중시하였다. 그러나 교사교육자는 논증의 깊이와 독자 고려를 강조하는 내용 중심의 세분화된 평가 기준을 구성한 반면, ChatGPT는 명확성, 구조, 매체 적응성을 강조하는 통합적이고 형식 중심의 평가 기준을 제시하였다. 동일한 루브릭을 적용하여 채점한 결과, 교사교육자는 질적 민감성이 반영된 폭넓은 점수 분포를 보인 반면, ChatGPT는 중간 점수대에 수렴하는 점수 분포 양상을 보였다. 이러한 결과는 ChatGPT가 교사교육자가 개발한 루브릭을 비교적 일관되게 적용할 수 있으나, 맥락 고려나 해석에 따른 세밀한 의미 변화를 충분히 반영하는 데에는 한계가 있음을 시사한다. 또한 ChatGPT의 채점 결과는 점수의 변별력이 낮은 경향을 보였다. 그럼에도 불구하고 ChatGPT는 교사가 평가 기준과 루브릭 설계를 점검하거나 자신의 평가 실천을 검토할 때 보조적 도구로 활용될 가능성이 있어, 이는 글쓰기 평가에서 인간과 생성형 AI 간 협업 가능성을 시사한다.

키워드

AI-assisted writing evaluationteacher educatorsgenerative AIChatGPTrubric development

참고문헌

  1. [1] [학술지(정기간행물)] AlAfnan M. A./Chatgpt as an educational tool: Opportunities, challenges, and recommendations for communication, business writing, and composition courses/Journal of Artificial Intelligence and Technology/2023/3(2)/60~68//
  2. [2] [학술대회논문] Allam H./Artificial intelligence in education: an argument of Chat-GPT use in education/2023 9th International Conference on Information Technology Trends (ITT)/2023//151~156/IEEE
  3. [3] [학술지(정기간행물)] Awidi, I. T./Comparing expert tutor evaluation of reflective essays with marking by generative artificial intelligence (AI) tool/Computers and Education: Artificial Intelligence/2024/6/100226~//
  4. [4] [학술지(정기간행물)] Cha J. W./A Comparative Study of AI-Based Writing Assessment Tools and Human Evaluators - Focusing on book report texts -/Korean Language and Literature in International Context/2024/(102)/681~705//
  5. [5] [학술지(정기간행물)] Choi J. Y./Calibrating human and machine: Developing an AI-assisted rater discussion model for automated writing evaluation(AWE)/Journal of Korean Association for Educational Information and Media/2025/31(4)/1297~1329//
  6. [6] [학술지(정기간행물)] Choi, S. G./Development and Validation of Quality Assessment Tools for AI-Powered Automated Writing Feedback/Journal of CheongRam Korean Language Education/2025/103/227~263//
  7. [7] [학술지(정기간행물)] Herbold S./A large-scale comparison of human-written versus ChatGPT-generated essays/Scientific reports/2023/13(1)/18617~//
  8. [8] [학술지(정기간행물)] Imran M./Analyzing the role of ChatGPT as a writing assistant at higher education level: A systematic review of the literature/Contemporary Educational Technology/2023/15(4)/ep464~//
  9. [9] [학술지(정기간행물)] Jang, S. M./ChatGPT has Changed the Future of Writing Education - Focusing on the response of writing education in the era of artificial intelligence -/Research on Writing/2023/56/7~34//
  10. [10] [학술지(정기간행물)] Jauhiainen, J. S./Generative AI in education: ChatGPT-4 in evaluating students’ written responses/Innovations in Education and Teaching International/2025/62(4)/1377~1394//
  11. [11] [학술지(정기간행물)] Kim, J. Y./The Effects and Limitations of Using AI-Based Automatic Writing Feedback Programs in Writing Classes - Focusing on the Case of Using KEEwi in Reading & Expression Classes -/The Journal of Yeongju Language & Literature/2024/58/355~386//
  12. [12] [학술지(정기간행물)] Kolade, O./Is AI changing learning and assessment as we know it? Evidence from a ChatGPT experiment and a conceptual framework/Heliyon/2024/10(4)/~//
  13. [13] [학술지(정기간행물)] Lee, Y. B./A Comparative Study on the Characteristics and Validity of Instructor, Peer, and AI Feedback in College Writing Education : Focusing on Common Issues in Students’ Argumentative Texts/Korean Journal of General Education/2025/19(3)/19~33//
  14. [14] [학술지(정기간행물)] Lee, Y. B./Reframing College Writing Education in the Era of Generative AI － An Integrated ‘AI-Assisted Collaborative Writing Pedagogy’ Combining Process-Based and Genre-Based Instruction with AI Literacy/The Korean Journal of Literacy Research/2025/16(4)/175~207//
  15. [15] [학술지(정기간행물)] Mao, J./Generative artificial intelligence in education and its implications for assessment/TechTrends/2024/68(1)/58~66//
  16. [16] [학술지(정기간행물)] Megawati R./The role of GPT chat in writing scientific articles: A systematic literature review/Jurnal Penelitian Pendidikan IPA/2023/9(11)/1078~1084//
  17. [17] [학술지(정기간행물)] Park S. Y./Exploring the Potential of AI Tools in University Writing Assessment: Comparing Evaluation Criteria between Humans and Generative AI/Journal of practical engineering education/2024/16(5)/663~676//
  18. [18] [학술지(정기간행물)] Rudolph J./ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?/Journal of applied learning and teaching/2023/6(1)/342~363//
  19. [19] [학술대회논문] Shidiq, M./The use of artificial intelligence-based chat-gpt and its challenges for the world of education; from the viewpoint of the development of creative writing skills/Proceeding of international conference on education, society and humanity/2023/1(1)/353~357/
  20. [20] [학술대회논문] Smolansky, A./Educator and student perspectives on the impact of generative AI on assessments in higher education/Proceedings of the tenth ACM conference on Learning@ Scale/2023//378~382/
  21. [21] [학술지(정기간행물)] Son, H. J./Perceptions and attitudes of English learners towards the utilization of AI-based Automated Writing Evaluation (AWE) programs and instructor feedback/English Language & Literature Teaching/2023/29(3)/91~112//
  22. [22] [학술지(정기간행물)] Son, H. S./Studying the Potential Use of Generative AI in College Writing/The Journal of General Education/2025/(31)/41~72//
  23. [23] [학술지(정기간행물)] Steiss, J./Comparing the quality of human and ChatGPT feedback of students’ writing/Learning and Instruction/2024/91/101894~//
  24. [24] [학술지(정기간행물)] /////~//
  25. [25] [학술지(정기간행물)] Wang, J./Evaluating rater accuracy and perception for integrated writing assessments using a mixed-methods approach/Assessing Writing/2017/33/36~47//
  26. [26] [학술지(정기간행물)] Yan D/Detection of AI-generated essays in writing assessments/Psychological Test and Assessment Modeling/2023/65(1)/125~144//

Korea Business Review · 한국경영학회

Print ISSN 2951-3596 · Online ISSN 2951-360X

Privacy PolicyTerms of UseContact UsKMA Homepage

TEL : 010-2672-3919 EMAIL : kbr2editor@gmail.com 계좌번호 : 신한 140-009-175743 예금주 : (사)한국경영학회 COPYRIGHT ⓒ 2023 THE KOREAN ACADEMIC SOCIETY OF BUSINESS ADMINISTRATION. All rights reserved