The one minute mentor: a pilot study assessing medical students' and residents' professional behaviours through recordings of clinical preceptors' immediate feedback

Topps, D., Evans, R.J., Thistlethwaite, J.E., Nan Tie, R., and Ellaway, R.H. (2009) The one minute mentor: a pilot study assessing medical students' and residents' professional behaviours through recordings of clinical preceptors' immediate feedback. Education for Health: change in learning and practice, 22 (1). 189.

[img]
Preview
PDF (Published Version)
Download (315kB)
View at Publisher Website: http://www.educationforhealth.net/publis...
 
6
825


Abstract

Introduction: The assessment of professional development and behaviour is an important issue in the training of medical students and physicians. Several methods have been developed for doing so. What is still needed is a method that combines assessment of actual behaviour in the workplace with timely feedback to learners. Goal: We describe the development, piloting and evaluation of a method for assessing professional behaviour using digital audio recordings of clinical supervisors’ brief feedback. We evaluate the inter-rater reliability, acceptability and feasibility of this approach. Methods: Six medical students in Year 5 and three GP registrars (residents) took part in this pilot project. Each had a personal digital assistant (PDA) and approached their clinical supervisors to give approximately one minute of verbal feedback on professionalism-related behaviours they had observed in the registrar’s clinical encounters. The comments, both in transcribed text format and audio, were scored by five evaluators for competence (the learner’s performance) and confidence (how confident the evaluator was that the comment clearly described an observed behaviour or attribute that was relevant). Students and evaluators were surveyed for feedback on the process. Results: Study evaluators rated 29 comments from supervisors in text and audio format. There was good inter-rater reliability (Cronbach α around 0.8) on competence scores. There was good agreement (paired t-test) between scores across supervisors for assessments of comments in both written and audio formats. Students found the method helpful in providing feedback on professionalism. Evaluators liked having a relatively objective approach for judging behaviours and attributes but found scoring audio comments to be time-consuming. Discussion: This method of assessing learners’ professional behaviour shows potential for providing both formative and summative assessment in a way that is feasible and acceptable to students and evaluators. Initial data shows good reliability but to be valid, training of clinical supervisors is necessary to help them provide useful comments based on defined behaviours and attributes of students. In addition, the validity of the scoring method remains to be confirmed.

Item ID: 5591
Item Type: Article (Research - C1)
ISSN: 1357-6283
Keywords: professionalism, educational assessment, digital audio, handheld computers
Related URLs:
Date Deposited: 25 Nov 2009 03:40
FoR Codes: 11 MEDICAL AND HEALTH SCIENCES > 1199 Other Medical and Health Sciences > 119999 Medical and Health Sciences not elsewhere classified @ 20%
13 EDUCATION > 1301 Education Systems > 130103 Higher Education @ 40%
13 EDUCATION > 1302 Curriculum and Pedagogy > 130209 Medicine, Nursing and Health Curriculum and Pedagogy @ 40%
SEO Codes: 93 EDUCATION AND TRAINING > 9305 Education and Training Systems > 930599 Education and Training Systems not elsewhere classified @ 100%
Downloads: Total: 825
Last 12 Months: 85
More Statistics

Actions (Repository Staff Only)

Item Control Page Item Control Page