Diagnostic Assessment in Physics: Developing and Validating a Two-Tier Instrument on Heat Concepts Using the Rasch Model

Abstract

This study aims to develop and analyze a two-tier diagnostic test instrument designed to identify students’ misconceptions regarding heat. The instrument includes 20 multiple-choice items (10 for content and 10 for justification) aimed at measuring students’ understanding and reasoning. The data were collected from 57 preservice teachers and analyzed using the Rasch model. The results showed an item reliability value of 0.92 and an item separation index of 3.47, indicating a high level of internal consistency and the instrument's capacity to differentiate item difficulty levels. The person-item map showed a moderate match between student ability and item difficulty, but gaps were identified at the lower and upper ends of the ability distribution. Infit and Outfit statistics revealed three items-S8, A4, and S4-that did not fit the Rasch model and were considered ineffective in identifying students’ misconceptions. These findings suggest that the Rasch model is a robust analytical tool for evaluating diagnostic test quality and support the need for well-calibrated items in order to accurately reveal students’ conceptual understanding and misconceptions in physics learning.