A collaborative comparison of Objective Structured Clinical Examination (OSCE) standard setting methods at Australian medical schools

Malau-Aduli, Bunmi S., Teague, Peta-Ann, D'Souza, Karen, Heal, Clare, Turner, Richard, Garne, David, and van der Vleuten, Cees (2017) A collaborative comparison of Objective Structured Clinical Examination (OSCE) standard setting methods at Australian medical schools. Medical Teacher. pp. 1-23. (In Press)

[img] PDF (Accepted Author Version) - Accepted Version
Restricted to Repository staff only

[img] PDF (Accepted Publisher Version) - Accepted Version
Restricted to Repository staff only

View at Publisher Website: https://doi.org/10.1080/0142159X.2017.13...
 
3


Abstract

Background: A key issue underpinning the usefulness of the OSCE assessment to medical education is standard-setting, but the majority of standard-setting methods remain challenging for performance assessment because they produce varying passing marks. Several studies have compared standard setting methods; however, most of these studies are limited by their experimental scope, or use data on examinee performance at a single OSCE station or from a single medical school. This collaborative study between ten Australian medical schools investigated the effect of standard-setting methods on OSCE cut scores and failure rates.

Methods: This research used 5,256 examinee scores from seven shared OSCE stations to calculate cut scores and failure rates using two different compromise standard-setting methods, namely the Borderline Regression and Cohen's methods.

Results: The results of this study indicate that Cohen's method yields similar outcomes to the Borderline Regression method, particularly for large examinee cohort sizes. However, with lower examinee numbers on a station, the Borderline Regression method resulted in higher cut scores and larger difference margins in the failure rates.

Conclusion: Cohen's method yields similar outcomes as the Borderline Regression method and its application for benchmarking purposes and in resource-limited settings is justifiable, particularly with large examinee numbers.

Item ID: 50008
Item Type: Article (Refereed Research - C1)
Keywords: standard setting, OSCE, clinical assessment, benchmarking
ISSN: 1466-187X
Date Deposited: 06 Sep 2017 01:27
FoR Codes: 11 MEDICAL AND HEALTH SCIENCES > 1199 Other Medical and Health Sciences > 119999 Medical and Health Sciences not elsewhere classified @ 50%
13 EDUCATION > 1301 Education Systems > 130103 Higher Education @ 50%
SEO Codes: 92 HEALTH > 9299 Other Health > 929999 Health not elsewhere classified @ 50%
93 EDUCATION AND TRAINING > 9301 Learner and Learning > 930101 Learner and Learning Achievement @ 50%
Downloads: Total: 3
Last 12 Months: 3
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page