Selecting an Appropriate Cognitive Diagnostic Model for Eighth Grade Math Questions: TIMSS 2019 Study

Abstract

Background and Objective: Traditional assessment methods have been criticized for neglecting the cognitive processes required by test-takers to provide correct responses. To address these issues, cognitive diagnostic models have been introduced by researchers. This study aimed to explore and apply cognitive diagnostic models to estimate the psychometric properties of published eighth-grade math questions from TIMSS 2019.
 
Methods and Materials: The research method employed in this study is mixed-methods, conducted with a retrofitting approach to existing tests. The statistical population of this study consisted of all Iranian eighth-grade students who participated in the TIMSS 2019 study. The population size included 1,095,026 students from 23,895 schools. The sample consisted of 5,980 Iranian eighth-grade students from 220 schools, selected through a two-stage cluster sampling method, with an average age reported as 14.1 years.
 
Findings: Using a qualitative method, 16 skills across four content domains—knowledge, application, and reasoning—were identified and the Q-matrix was developed. Then, in the quantitative section, the fit of the DINA model as a compensatory model, the DINO model as a non-compensatory model, and the G-DINA model as a general model were examined with data from students' responses to math questions in blocks 1, 2, 3, and 5. The findings showed that in terms of model fit, the G-DINA model had the weakest fit, while the DINA model had the best fit. Absolute fit indices of the models also indicated that the DINA model was better, and the DINO model did not fit. Relative fit indices of the models showed no significant difference between the G-DINA model and the other two models, and overall, the non-compensatory DINA model demonstrated the best fit.
 
Conclusion: The results indicated that the DINA model had the best fit with the data and can be considered the optimal model for analyzing the eighth-grade mathematics questions in TIMSS 2019.