Research Article
GenAI-HITL Development and Validity Review of Reading -Writing Constructed -Response Tasks
한국교원대학교
Published: January 2025 · Vol. 60, No. 4 · pp. 129-174
DOI: https://doi.org/10.20880/kler.2025.60.4.129
Full Text PDF
Abstract
This study developed constructed-response tasks aligned with the 2022 revised Korean Language Arts (“Reading and Writing”) achievement standards using a Generative AI-Human-in-the-Loop (HITL) approach and reported preliminary validity evidence from expert-criterion ratings. A three-stage protocol integrating context engineering and Chain-of- Thought generated a linked package of texts, prompts, rubrics, and explanations. Eighteen in-service Korean language teachers from 13 regions rated the outputs on 12 items across three domains. The overall mean was high (M = 4.32), with the strongest ratings for standard alignment and structural coherence; internal consistency and inter-rater agreement were acceptable. Learner-level appropriateness was lower, indicating limits in capturing non-formal factors (developmental stage, classroom context) despite effective operationalisation of formalised curriculum elements. The findings suggest AI outputs can serve as teacher-adjustable drafts, and that human-AI collaboration may strengthen the balance between formal and substantive validity.
