Interaction in paired oral proficiency assessment in Spanish

Speaking tasks involving peer-to-peer candidate interaction are increasingly being incorporated into language proficiency assessments, in both large-scale international testing contexts, and in smaller-scale, e.g. course-related, ones. This growth in the popularity and use of paired and group orals has stimulated research, particularly into the types of discourse produced and the possible impact of candidate background factors on performance. The validation of tests of speaking involving paired candidate formats is increasingly focusing on ways in which interaction between candidates is sustained, as 'interaction' emerges as an important criterion for assessing candidate performance. However, despite the fact that the strongest argument for the validity of peer-to-peer assessment lies in the claim that such tasks allow for the assessment of a broader range of interactional skills than the more traditional interview-format tests do, there is surprisingly little research into the judgments that are made of such performances, the development of scales to rate interactional skills, and candidates' awareness of features of interaction in such tests. The thesis reports on the findings of a verbal protocol study of teacher-raters viewing the paired test discourse of seventeen beginner dyads in a university-based Spanish as a foreign language course. The fact that raters, and rating criteria, are in a crucial mediating position between output and outcomes warrants investigation into how raters construe the interaction in these tasks. The thesis also reports on the development of an evidence-based rating method to score peer L2 communicative interaction, based on experienced judges’ comments on videoed student samples filmed during operational paired candidate tests of beginner level Spanish. Six trained and experienced raters generated criteria for communicative interaction, which were incorporated into a tool for developing a discourse sample based rating procedure, the Empirically-based, Binary-choice, Boundary-definition (EBB) method (Turner & Upshur, 1996). The thesis examines the features of paired candidate interaction which raters used to define ii the boundary between performance levels. Three main criteria emerged as the boundaries used to define levels of interaction: non-verbal interpersonal communication, interactive listening and interactional management. These new notions are evidence of how peer interaction can be rated and they advance our understanding of the significant features of interaction in the rating context. Finally, this thesis also focuses on the previously unexplored area of candidates' awareness of features of interaction in such tests. It uses a retrospective Stimulated Verbal Recall methodology with video-taped test performances from 25 participants in the same beginners' level test of spoken Spanish. The participant reports are analysed in terms of student orientation to paired interaction. The analysis addresses non-verbal interpersonal communication, interactive listening and interactional management, the features of interaction identified a priori in the rater Verbal Protocol Study. Examples from the data are presented and discussed. The findings confirm that candidates are significantly oriented to the features of interaction which are salient to raters and which have been included in the devised rating procedure. The findings from the three parts of the thesis (the rater orientation study, the scoring procedure and the candidate awareness study) have implications for our understanding of the construct of effective interaction in paired candidate speaking tests, and for the development of appropriate rating procedures.