Guan, Yuanyuan (2019) EFL listening development through diagnosis: an assessment-based study of listening sub-skills using Rasch measurement. PhD thesis, James Cook University.

Preview

PDF (Thesis) Download (1MB) | Preview

Abstract

The lack of informed knowledge about listening subskills and their relationships has hindered the development of the diagnostic English language track assessment (DELTA) in three participating Hong Kong universities. This study investigates English as a foreign language (EFL) learners' listening proficiency development in understanding different spoken genres in the Hong Kong Chinese tertiary contexts. It aims to: i) identify the subskills and/or cognitive processes that underlie student performance on the DELTA listening component; ii) examine the difficulty levels of the DELTA listening subskills, and, consequentially, their hierarchical order; iii) investigate the impact of text type on difficulty level and the hierarchical order of the subskills; and iv) infer principles underlying the development of listening proficiency in the Hong Kong tertiary education contexts.

A multi-method approach was employed for data collection and analysis. The primary quantitative data were derived from the DELTA listening component items answered by 2830 Chinese ELF learners who studied in their first or second year in the DELTA participating universities in the 2013-14 academic year. The item pool included 207 multiple-choice questions (MCQ) from 33 texts of three text types – conversation, interview and lecture. Each MCQ is intended to measure a particular listening subskill, including: 1) identifying specific information (SSK1); 2) understanding main idea and supporting ideas (SSK2); 3) understanding information and making an inference (SSK3); 4) interpreting a word or phrase as used by the speaker (SSK4); 5) inferring the attitude or intention of the speaker (SSK5); and 6) inferring the speaker's reasoning (SSK6). By adopting inter-related Rasch analyses using Winsteps and Facets, all test items were calibrated and analysed to determine their difficulty measures and their respective difficulties across the three text types. Qualitative Stimulated Recall Protocol (SRP) discussions were then conducted with 62 examinees of varying estimated listening abilities one month later, in a simulated test situation, where the test-taking process was video-recorded and the participants were asked to recall and to verbalise their thought processes and strategies they used to answer each question.

The SRP results reveal an array of both cognitive processes and test-taking strategies in the listening comprehension and test-answering process. Firstly, various combinations of cognitive processes were utilised by both the high and low ability examinees to answer questions targeting the same listening sub-skill; however, the dominant cognitive process that was reported to have been used to answer each question corresponded with the particular listening subskill intended by DELTA item writers. Secondly, an array of test-taking strategies best identified as elimination, and guessing, were reported as used by examinees during the test. While this finding might not be surprising given the exam-oriented atmosphere prevailing in Hong Kong secondary school education, it alerted the researcher to scrutinise the validity of the DELTA listening component.

The most striking observation from the listening test analysis is that, the DELTA listening subskills are measurably separable from each other, and a hierarchical pattern is established. In terms of their interaction with text type, the results showed that SSK1 and SSK6 were, respectively, the easiest and the most difficult subskills, whereas the hierarchical orders of the other four subskills varied across the three text types. More generally, these findings provide empirical evidence for the proposition that EFL listening comprehension is composed of multiple listening subskills, which operate interactively and interdependently in the listening process. The results regarding the difficulty level and the hierarchy of listening subskills corroborate the findings of prior research that low-level processing, such as identifying specific information, poses less challenge than high-level processing, such as summarising and inferencing. Because of the complexity in the interaction between text type and listening subskills, it is difficult to identify an overarching hierarchical order of the six listening subskills across the three text types. A general pattern, however, is that the difficulty increased from SSK1, SSK2 to SSK6 irrespective of the text type, and this corresponds to the general subskill hierarchy.

The study will benefit teachers and students with diagnostic profiling and bridge the gap in diagnostic test design with targeted items of appropriate difficulty for predicting learners' listening development. It will extend second language acquisition theory with a hierarchical trajectory of listening proficiency growth. Limitations and future research recommendations are discussed.

Item ID:	63730
Item Type:	Thesis (PhD)
Keywords:	EFL, English as a foreign language, listening, Rasch, language acquisition
Copyright Information:	Copyright © 2019 Yuanyuan Guan.
Date Deposited:	09 Jul 2020 22:55
FoR Codes:	20 LANGUAGE, COMMUNICATION AND CULTURE > 2003 Language Studies > 200303 English as a Second Language @ 50% 17 PSYCHOLOGY AND COGNITIVE SCIENCES > 1702 Cognitive Science > 170204 Linguistic Processes (incl Speech Production and Comprehension) @ 50%
SEO Codes:	95 CULTURAL UNDERSTANDING > 9502 Communication > 950201 Communication Across Languages and Culture @ 40% 95 CULTURAL UNDERSTANDING > 9502 Communication > 950202 Languages and Literacy @ 40% 93 EDUCATION AND TRAINING > 9301 Learner and Learning > 930103 Learner Development @ 20%
Downloads:	Total: 705 Last 12 Months: 31
	More Statistics