.jpeg)
Reliability Analysis of Scoring-Rescoring Depending on the Consistency Type of Korean Language Teachers’ the Writing Assessment
Yoon, Keumjoon Park, Young-min
청명고등학교 한국교원대학교
Korea Business Review 56Vol. 1No. pp.151-183 (2021)
Abstract
The purpose of this study is to analyze the reliability of raters based on the results of the writing assessment scoring-rescoring of Korean language teachers. For this purpose, 36 Korean language teachers scored and re-scored 20 students’ writings, and they were categorized into groups based on the type of consistency according to the results of the multi-faceted Rasch analysis. As a result, 25.00% of raters showed consistency in both scoring and rescoring and 52.78% of raters showed consistency only once. In addition, 22.22% of the raters were found to have failed to maintain consistency. Looking at the reliability of scoring and rescoring, the scoring-rescoring reliability tends to be high when consistency is suitable. In case of over-fitting, the reliability of scoring and rescoring was high, but the mean difference was significant because of the difference of the severity in scoring and rescoring. In case of mis-fitting, the reliability of scoring-rescoring was the lowest. This suggests that various measures should be sought to ensure the reliability of the writing assessment.
Keywords
국어교사쓰기평가평가자 일관성채점-재채점 신뢰도다국면 Rasch 모형
References
- [1] [학위논문] 가은아/쓰기 발달의 양상과 특성 연구/박사/한국교원대학교 대학원/국어교육/2011
- [2] [단행본] 신뢰도/강승호/교육과학사/2004/~///
- [3] [단행본] 교육평가/권대훈/학지사/2008/~///
- [4] [학위논문] 권태현/쓰기 성취기준에 따른 학생 예시문 선정에 관한 연구/박사/한국교원대학교 대학원/국어교육전공/2014
- [5] [단행본] 교육평 : 이해와 적용/박도순/교육과학사/2012/~///
- [6] [단행본] 작문교육론/박영민/역락/2016/~///
- [7] [학술지(정기간행물)] 서수현/쓰기 평가 협의 과정에 나타난 쓰기 평가자의 인식 연구/국어교육학연구/2012/(44)/335~367/국어교육학회/10.20880/kler.2012..44.335
- [8] [단행본] 국어학과 국어교육학/성낙수/채륜/2015/~///
- [9] [단행본] 타당도와 신뢰도/성태제/학지사/2002/~///
- [10] [학술지(정기간행물)] 이재창/조현병 환자가 시행한 주의력 네트워크 검사 점수의 검사-재검사 신뢰도/정신신체의학/2017/25(2)/210~217/한국정신신체의학회/
- [11] [학술지(정기간행물)] 이혜원/학령전기용 어음청각검사 어표의 검사-재검사 신뢰도/Audiology and Speech Research/2014/10(1)/25~34/한국청각언어재활학회/
- [12] [단행본] 언어교육평가 연구를 위한 FACETS 프로그램/장소영/글로벌콘텐츠/2009/~///
- [13] [학위논문] 장은주/채점의 일관성 유형에 따른 국어교사의 쓰기 평가 특성 분석/박사/한국교원대학교//2015
- [14] [학위논문] 허소희/한국어 이명주요기능설문지의 검사-재검사 신뢰도 검증 및 간소화 버전 제안/석사/한림대학교//2019
- [15] [학술지(정기간행물)] Barrett, S./The impact of training on rater variability/International Education Journal/2001/2/49~58//
- [16] [학술지(정기간행물)] Eckes, T./Rater types in writing performance assessments : A classification approach to rater variability/Language Testing/2008/25(2)/155~185//
- [17] [단행본] Handbook of Inter-Rater Reliability/Gwet, K. L./Advanced Analytics, LLC/2012/~///
- [18] [단행본] Many-facet Rasch Measurement/Linacre, J. M./MESA Press/1989/~///
- [19] [학술지(정기간행물)] Linacre, J. M./Rating, judges, and fairness/Rasch Measurement Transactions/1998/12(2)/630~631//
- [20] [단행본] FACETS/Linacre, J. M./MESA Press/1990/~///
- [21] [학술지(정기간행물)] Lumley, T./Rater characteristics and rater bias : Implications for training/Language Testing/1995/12(1)/54~71//
- [22] [단행본] Measuring Second Language Performance/McNamara, T. F./Longman/1996/~///
- [23] [학술지(정기간행물)] Mukaka, M. M./A guide to appropriate use of correlation coefficient in medical research/Malawi Medical Journal/2012/24(3)/69~71//
- [24] [학술지(정기간행물)] Myford, C. M./Detecting and Measuring Rater Effects Using Many-Facet Rasch Measurement : Part Ⅰ/Journal of Applied Measurement/2003/4(4)/386~422//
- [25] [단행본] Designing writing tasks for the assessment of writing/Ruth, L./Ablex Publishing Corporation/1988/~///
- [26] [학술지(정기간행물)] White, E. M./Holisticism/College Composition and Communication/1984/35(4)/400~409//