A fogalmazásértékelés megbízhatóságának empirikus vizsgálata két értékelési modell összehasonlításával

Abstract

This study examines the reliability of writing assessment in the context of the Hungarian language
and literature intermediate-level school-leaving examination, drawing on empirical data. The
aim of the research was to explore the extent to which the marking and assessment guidelines
introduced in 2024 provide an objective, consistent, and uniformly interpretable scoring framework
for evaluating students’ written texts, and to examine whether applying an alternative analytic
assessment rubric can enhance inter-rater agreement. The empirical investigation involved forty
Hungarian language and literature teachers with experience in administering the school-leaving
examination, who assessed the same student’s literary analytical essay using two different
assessment systems. Interrater agreement and the internal consistency of the measurement
instruments were analysed using multiple reliability indices, alongside an examination of the
discriminative power of individual assessment criteria and differences in raters’ scoring patterns.
The findings indicate that within the assessment framework currently applied in the examination,
inter-rater agreement is relatively low, and the interpretation of assessment criteria shows
considerable variability. By contrast, the alternative analytic assessment model yields higher
levels of agreement and a more stable assessment structure. The detailed performance
Anyanyelv-pedagógia XIX. évfolyam, 2026/1.
38
descriptors of the analytic rubric contribute to greater consistency in scoring decisions; however,
the results also suggest that further refinement of the assessment dimensions would be justified.
Overall, the study underscores the need for clearer specification of assessment criteria and the
development of a more standardised approach to writing assessment.