Appraising appraisal

Appraisal vs assessment

The Guide to Specialist Registrar Training(1) distinguishes between assessment, a process “to measure progress against defined criteria based on relevant curricula,” and appraisal, which provides “a complementary or parallel approach focussing on the trainee and his or her personal and professional needs.”(1) The process will enable educational supervisors to offer trainees “feedback on performance and assistance in career progression” by means of confidential “constructive and regular dialogue.” Homage duly paid, the orange book declines to give any guidance on appraisal and instead moves swiftly on to discuss assessment. So what guidance is there?

In other sectors, such as the university sector in which I work, appraisal is widely used to enable us, as employees, to develop effectively in our work and careers. Here a distinction is drawn between reviewing an individual's potential, in the context of that person's own career history, and reviewing it in relation to fixed criteria established by the organisation - such as those related to regradings. So, at least in University College London, appraisal is seen as having no direct link with promotion or regrading, although these might be discussed. Similarly, appraisal is seen as having no direct link with external criteria related to assessment. So what is the real relation between appraisal and assessment?

Appraisal in assessment

Appraisal focuses on trainees, setting their achievements in their current post within the context of their own career development. In the assessment, when the annual Record of In-Training Assessment (RITA) form is completed, a trainee's achievements in the current post will be judged against the criteria set by the appropriate royal college. Focusing on trainees means enabling them to reflect on their own progress, to become aware of their strengths and weaknesses (or “development opportunities”), to set their own targets, and to address issues related to their professional development. Trainees working on this in partnership with their educational supervisor is what appraisal is about.

The process should also provide a learning opportunity for supervisors, who should be prepared to reflect on their role in facilitating trainees' achievements. Are there any obstacles to a trainee's progress which the supervisor could remove? Is the supervisor the obstacle, and, if so, is he or she prepared to do anything about it? These are uncomfortable questions for many of us.

The appraisal meeting

The meeting should, it goes without saying, be a private one between supervisor and trainee, and its confidentiality should be emphasised. The training agreement, which is the agreed outcome of the meeting and signed by both parties, is not confidential and becomes an admissible document in the assessment.

Appraisal is important because it allows trainees' self assessment and reflection to feed into the assessment process. Successful appraisal requires supervisors, as appraisers, first and foremost to relate effectively to trainees and to empathise with them in order to encourage this. Supervisors must be able to listen in an active way - to hear and understand the importance of what is said, and be aware of what is not said. They must have the ability to explore issues raised by trainees, and to see where they lead. This will mean resisting the temptation to jump to conclusions.

Most of us neglect giving positive feedback and are inclined to duck giving negative feedback. Since the Calman reforms, educational supervisors are less able to opt out of giving bad news to “difficult” trainees. It is good practice to give good news first, but, in any case, feedback needs to be given. After giving negative feedback, supervisors should be there to pick up the pieces and help their trainee move on by giving support and counsel. This may require supervisors to give advice that is in their trainee's best interests but not in their own. It may also mean supervisors having to recognise their limitations and, if appropriate, referring the trainee for help elsewhere.

Progress should be reviewed, looking backwards with the aim of moving forwards. Crucial to this is helping trainees to identify educational and training needs. Supervisors must remain focused on their trainee and the wider context of his or her professional development. Lateral thinking and diplomacy may be needed to persuade trainees to address shortcomings that they possibly do not recognise.

Most importantly, supervisors should help trainees to develop the skills of self appraisal and reflection. Supervisors who cannot look honestly at themselves are unlikely to be successful in encouraging this in their trainees.

Assessing the supervisors

We recently conducted some research on appraisal and assessment in association with North Thames Postgraduate Medicine and Dentistry, which is responsible for the higher specialist training of 5500 doctors and which has about 4,400 consultants acting as educational supervisors. We sent questionnaires to a stratified random sample of 352 educational supervisors and 318 specialist registrars from various specialties working in a range of hospitals in the North Thames region. The response rate was 50%.

The research identified a range of tasks involved in both assessment and appraisal, in the context of specialist medical training, and listed them in a random fashion. Our aim was to discover what importance educational supervisors attached to the various tasks and whether their trainees agreed with this, or had other expectations. We also wanted to find out how good the supervisors were at the various tasks, and how good their trainees thought they were. The results are illuminating.(2)

Both supervisors and trainees broadly agreed on the most important aspects of a supervisor's role, and these turned out to be more pertinent to appraisal than to assessment. Trainees thought that supervisors should discuss and explain potential training opportunities, and trainees' individual goals, and that they should ensure that trainees knew what was expected of them. All of these require supervisors to focus on the trainee and his or her professional development, and represent features of the appraisal meeting.

Supervisors' perceptions of their own proficiency at appraisal and assessment indicated that they thought they needed training, mostly in cognitive areas. For example, they wanted to learn more about different assessment methods and criteria. The trainees, however, attached greater importance to their supervisors concentrating on them - giving constructive feedback and helping trainees to assess themselves, neither of which they thought their supervisors were very good at. The supervisors themselves regarded communication skills as fairly irrelevant in helping them with the process. Yet it is the ability to establish rapport, to listen, and to empathise that is at the heart of the appraisal process.

Where now?

From our research, we identified the key features that a training programme for educational supervisors should have. Such a programme is now being delivered in North Thames region.

The training should recognise the constraints of time and funding under which consultants labour. It should therefore balance the demand for a professional course with the realistic constraints of hospital training budgets, and not be too expensive. (Our training is currently funded by the Deanery and so is, like the NHS, free at the point of delivery.) It should not require consultants to take too much time out from their clinical work, and so should be short: a day is ideal, and even better is a short day on-site which allows consultants clinical time at either end. The numbers attending should be restricted to facilitate practical work. In order to encourage an honest sharing of problems, and frank self-appraisal, the group should be restricted to consultants and should not include, for example, specialist registrars, unless they themselves are about to take up consultant appointments. The tutors should be familiar with appraisal, and with the context in which medical training takes place.

The training itself should not be too general (and a lot of general training on appraisal is available) but should focus on appraisal within the assessment process for specialist registrars. It should concentrate on giving consultants practice (via role-play) in the following key tasks: reviewing the trainee's progress against training objectives, and agreeing an individual development plan; recognising and managing the trainee's resistance, and attempting to gain his commitment to addressing problem areas; giving feedback to the trainee, and relating this to his broader learning objectives as well as work-related goals; clarifying the trainee's understanding of his strengths and development needs; exploring options, and gaining the trainee's commitment to a plan for development. Finally, any such training programme needs to recognise the development needs of the supervisors themselves, and to provide the opportunity for follow-up meetings, where specific issues can be explored in depth.

The wider context

Effective use of the appraisal process is an important tool in the professional development of tomorrow's doctors. Most of those now passing through the training grades will, unless they are completely put off by the process, be the consultants and educational supervisors of tomorrow. The more positive we can make their training experience, the more positive they are likely, in their turn, to make the experience for the doctors of the day after tomorrow.