Considering Validity in Assessing Oral Presentation Skills

Abstract

Despite numerous studies on the reliability of peer marking of oral presentation skills, no empirical studies to date have addressed the validity issue of whether students use a different 'perspective' from teachers in making an overall assessment. This paper analyses peer and teacher assessment data from thesis presentations made by students enrolled in a fourth year subject 'Communications for Professional Engineers'. The study uses a novel application of the 'index of association' statistic to determine the extent to which the variance of marks by teachers and by peers for the overall quality of the presentations can be attributed to factors other than errors associated with unreliable measures. Results indicate that students had applied criteria which were quite different from those used by teachers. The discussion describes methodologies which can be employed to determine which skill criteria are most influential in the determination of a global mark for oral presentations, and outlines plans to identify the ways in which students and staff differ in the criteria they apply.