Developing and validating a listening mindset questionnaire through Think-Aloud Protocol

Abstract

Purpose: This study aims to develop and validate the Listening Mindsets Questionnaire (LMQ) to assess EFL learners' incremental (growth) and entity (fixed) mindsets regarding English listening skills.

Methodology: The research employed a mixed-methods approach, specifically a convergent parallel design. The Listening Mindsets Questionnaire (LMQ) was developed based on the framework of Lou and Noels (2017) and consisted of 18 items. It was administered to 289 English language learners at B1, B2, and C1 levels according to the Common European Framework of Reference (CEFR), studying at various English institutes in Zanjan, Iran. Additionally, 10 learners were randomly selected to participate in Think-Aloud Protocols for qualitative validation. The validity of the questionnaire was assessed through factor analysis and Think-Aloud Protocols, while reliability was measured using Cronbach’s Alpha. The qualitative data obtained from Think-Aloud Protocols were analyzed through coding in MAXQDA software.

Findings: The results from both the factor analysis and the qualitative coding of Think-Aloud Protocols confirmed the validity of the newly developed questionnaire. The LMQ demonstrated a clear factor structure that aligned with the theoretical framework, ensuring its appropriateness for measuring listening mindsets. Additionally, Think-Aloud Protocols supported the questionnaire's validity but highlighted certain limitations in using verbal reports as a validation method. The reliability analysis indicated that the LMQ had a high internal consistency, confirming its suitability for assessing EFL learners' mindsets toward listening.

Conclusion: The study provides empirical support for the validity and reliability of the Listening Mindsets Questionnaire (LMQ). The instrument can serve as a useful tool for researchers and educators to assess and understand learners’ perceptions of language learning ability in the context of listening skills. Future research should explore further refinements and applications of the questionnaire across diverse learner populations.