Home Articles References
Research Article

GenAI-HITL Development and Validity Review of Reading -Writing Constructed -Response Tasks

Park Goun

한국교원대학교

Published: January 2025 · Vol. 60, No. 4 · pp. 129-174

DOI: https://doi.org/10.20880/kler.2025.60.4.129

Full Text PDF

Abstract

This study developed constructed-response tasks aligned with the 2022 revised Korean Language Arts (“Reading and Writing”) achievement standards using a Generative AI-Human-in-the-Loop (HITL) approach and reported preliminary validity evidence from expert-criterion ratings. A three-stage protocol integrating context engineering and Chain-of- Thought generated a linked package of texts, prompts, rubrics, and explanations. Eighteen in-service Korean language teachers from 13 regions rated the outputs on 12 items across three domains. The overall mean was high (M = 4.32), with the strongest ratings for standard alignment and structural coherence; internal consistency and inter-rater agreement were acceptable. Learner-level appropriateness was lower, indicating limits in capturing non-formal factors (developmental stage, classroom context) despite effective operationalisation of formalised curriculum elements. The findings suggest AI outputs can serve as teacher-adjustable drafts, and that human-AI collaboration may strengthen the balance between formal and substantive validity.
Keywords: 자동 문항 생성생성형 AI인간-AI 협력서술형 평가독서와 작문2022 개정 교육과정