Intended for healthcare professionals

  1. Education
  2. Use of SPRAT for peer...
  3. Use of SPRAT for peer review of paediatricians in training
Learning In Practice

Use of SPRAT for peer review of paediatricians in training

BMJ 2005; 330 doi: https://doi.org/10.1136/bmj.38447.610451.8F (Published 26 May 2005) Cite this as: BMJ 2005;330:1251
  1. Julian C Archer, clinical research fellow1,
  2. John Norcini, president3,
  3. Helena A Davies, consultant in medical education (h.davies{at}sheffield.ac.uk)2
  1. 1 Academic Unit of Child Health, Sheffield Children's Hospital, Sheffield S10 2HT
  2. 2 Postgraduate Medical Education Department, Sheffield Children's Hospital
  3. 3 Foundation for the Advancement of International Medical Education Research, (FAIMER), 3624 Market Street, Philadelphia, PA 19104, USA
  1. Correspondence to: H A Davies
  • Accepted 1 April 2005

Abstract

Objective To determine whether a multisource feedback questionnaire, SPRAT (Sheffield peer review assessment tool), is a feasible and reliable assessment method to inform the record of in-training assessment for paediatric senior house officers and specialist registrars.

Design Trainees' clinical performance was evaluated using SPRAT sent to clinical colleagues of their choosing. Responses were analysed to determine variables that affected ratings and their measurement characteristics.

Setting Three tertiary hospitals and five secondary hospitals across a UK deanery.

Participants 112 paediatric senior house officers and middle grades.

Main outcome measures 95% confidence intervals for mean ratings; linear and hierarchical regression to explore potential biasing factors; time needed for the process per doctor.

Results 20 middle grades and 92 senior house officers were assessed using SPRAT to inform their record of in-training assessment; 921/1120 (82%) of their proposed raters completed a SPRAT form. As a group, specialist registrars (mean 5.22, SD 0.34) scored significantly higher (t = – 4.765) than did senior house officers (mean 4.81, SD 0.35) (P < 0.001). The grade of the doctor accounted for 7.6% of the variation in the mean ratings. The hierarchical regression showed that only 3.4% of the variation in the means could be additionally attributed to three main factors (occupation of rater, length of working relationship, and environment in which the relationship took place) when the doctor's grade was controlled for (significant F change < 0.001). 93 (83%) of the doctors in this study would have needed only four raters to achieve a reliable score if the intent was to determine if they were satisfactory. The mean time taken to complete the questionnaire by a rater was six minutes. Just over an hour of administrative time is needed for each doctor.

Conclusions SPRAT seems to be a valid way of assessing large numbers of doctors to support quality assurance procedures for training programmes. The feedback from SPRAT can also be used to inform personal development planning and focus quality improvements.

Footnotes

  • Embedded ImageThis is the abridged version of an article that was posted on bmj.com on 9 May 2005: http://bmj.com/cgi/doi/10.1136/bmj.38447.610451.8F

  • Contributors See bmj.com

  • Funding JCA's research fellowship is funded by cooperation between the Academic Unit of Child Health, University of Sheffield, and Bassetlaw District General Hospital, Worksop.

  • Competing interests None declared.

  • Ethical approval Not sought. SPRAT was implemented as part of the assessment programme in the South Yorkshire and South Humberside Deanery.

  • Accepted 1 April 2005
View Full Text
Back to top