Learning in practice

Learning needs assessment: assessing the need

Janet Grant, professor of education in medicine. 

Open University Centre for Education in Medicine, Milton Keynes MK7 6AA

j.r.grant{at}open.ac.uk

Learning needs assessment has a fundamental role in education and training, but care is needed to prevent it becoming a straitjacket

It might seem self evident that the need to learn should underpin any educational system. Indeed, the literature suggests that, at least in relation to continuing professional development, learning is more likely to lead to change in practice when needs assessment has been conducted, the education is linked to practice, personal incentive drives the educational effort, and there is some reinforcement of the learning.¹ Learning needs assessment is thus crucial in the educational process, but perhaps more of this already occurs in medical education than we suspect. The key lesson might be for those who design new systems of education and training: for example, the postgraduate education allowance system in general practice was felt to fail the profession because it did not include needs assessment and so led to ad hoc education to fulfil the time requirements of the system rather than the needs of individual doctors or the profession as a whole. On the other hand, basing learning in a profession entirely on the assessment of needs is a dangerous and limiting tactic. So a balance must be struck.

	Learning needs assessment in medicine

Top Learning needs assessment in... The definition of need Methods of needs assessment The difference between needs... Learning for needs References

In 1998 both individual and organisational needs assessment became part of government policy in relation to the continuing professional development and personal development plans of all healthcare professionals.² Thus, it has a role in the clinical governance of the service³ and is therefore much more than an educational undertaking. This integration of needs assessment, education, and quality assurance of the service was first made explicit in 1989 in relation to clinical audit, which would identify practices in need of improvement and ensure that educational and organisational interventions were made to address these needs.⁴ Accordingly, audit was described as "essentially educational" and the educational process surrounding it described.⁵

Long before these recent developments, needs assessment outside medicine was presented as an important part of managed education and learning contracts, which are the predecessors of the personal development plans to be developed for all NHS healthcare professionals.⁶ In his descriptions of adult learning Knowles assumed (he did not claim to have research evidence) that learners needed to feel a necessity to learn and that identifying one's own learning needs was an essential part of self directed learning.⁷ In medicine a doctor's motivation to learn would therefore derive from needs identified during his or her experience of clinical practice. So the pedigree and practice of learning needs assessment, if not the evidence, are well established.

	The definition of need

Top Learning needs assessment in... The definition of need Methods of needs assessment The difference between needs... Learning for needs References

As in most areas of education, for many years there has been intense debate about the definition, purpose, validity, and methods of learning needs assessment.⁸ It might be to help curriculum planning, diagnose individual problems, assess student learning, demonstrate accountability, improve practice and safety, or offer individual feedback and educational intervention. Published classifications include felt needs (what people say they need), expressed needs (expressed in action) normative needs (defined by experts), and comparative needs (group comparison).⁹ Other distinctions include individual versus organisational or group needs, clinical versus administrative needs, and subjective versus objectively measured needs.¹⁰ The defined purpose of the needs assessment should determine the method used and the use made of findings.

Furthermore, even though the concept of educational needs assessment is enshrined in practice, policy, and the educational canon, several factors indicate the need for careful planning and research in this subject (see boxes 1 and 2). Exclusive reliance on formal needs assessment in educational planning could render education an instrumental and narrow process rather than a creative, professional one. This is especially so in a profession where there is inherent unpredictability and uncertainty. Members of any profession require wide knowledge and depth of experiencethe relevance of some of which might not have been obvious at the time of learning. Certainly, learning needs can and should be identified on the basis of what has been experienced and of what more experienced members of the profession know to be relevant, but this must not deter other, more general or even speculative, learning that, at the time, seems to answer no specific need. Possibly no specific learning needs assessment would ever send a person to a large international conference on a generic subject (such as endocrinology, medical education, or management). It is, nevertheless, important that doctors attend such meetings and return with the unexpected and expected benefits that they accrue.

	Methods of needs assessment

Top Learning needs assessment in... The definition of need Methods of needs assessment The difference between needs... Learning for needs References

Although the literature generally reports only on the more formal methods of needs assessment, doctors use a wide range of informal ways of identifying learning needs as part of their ordinary practice. These should not be undervalued simply because they do not resemble research. Questionnaires and structured interviews seem to be the most commonly reported methods of needs assessment, but such methods are also used for evaluation, assessment, management, education, and now appraisal and revalidation.¹¹ Together, these formal and informal methods might make an effective battery where there is clarity of purpose. The Good CPD Guide details 46 formal and informal methods of self assessment (see box 3).¹²

The methods listed are both formal and informal, planned and opportunistic, showing that day to day work and encounters have the potential to generate needs as much as do formal methods. Formal needs assessment methods include critical incident techniques, gap analysis, objective knowledge and skills tests, observation, revalidation, self assessment, video assessment, and peer review. Such methods are often used to identify group needs. ^{14 15} Formal identification of needs can also arise from audit, morbidity patterns, adverse events, patient satisfaction surveys, and risk assessment. Most of these tools use quantitative methods that can generate computerised data and cover wider population ranges, but these are often unable to probe into the personal agendas and opinions of individuals.

Types of needs assessment
Methods of needs assessment can be classified into seven main types, each of which can take many different forms in practice.

Gap or discrepancy analysis
This formal method involves comparing performance with stated intended competenciesby self assessment, peer assessment, or objective testingand planning education accordingly. ^{9 16 17}

Reflection on action and reflection in action
Reflection on action is an aspect of experiential learning and involves thinking back to some performance, with or without triggers (such as videotape or audiotape), and identifying what was done well and what could have been done better. ^{18 19} The latter category indicates learning needs.

Self assessment by diaries, journals, log books, weekly reviews
This is an extension of reflection that involves keeping a diary or other account of experiences.²¹ However, practice might show that such documents tend to be written nearer the time of their review than the time of the activity being recorded.

Peer review
This is rapidly becoming a favourite method. It involves doctors assessing each other's practice and giving feedback and perhaps advice about possible education, training, or organisational strategies to improve performance. The Good CPD Guide describes five types of peer reviewinternal, external, informal, multidisciplinary, and physician assessment.¹¹ The last of these is the most formal, involving rating forms completed by nominated colleagues, and shows encouraging levels of validity, reliability, and acceptability. ^{22 23}

Observation
In more formal settings doctors can be observed performing specific tasks that can be rated by an observer, either according to known criteria or more informally. The results are discussed, and learning needs are identified. The observer can be a peer, a senior, or a disinterested person if the ratings are sufficiently objective or overlap with the observer's area of expertise (such as communication skills or management).

Critical incident review and significant event auditing
Although this technique is usually used to identify the competencies of a profession or for quality assurance, it can also be used on an individual basis to identify learning needs.²⁴ The method involves individuals identifying and recording, say, one incident each week in which they feel they should have performed better, analysing the incident by its setting, exactly what occurred, and the outcome and why it was ineffective.

Practice review
A routine review of notes, charts, prescribing, letters, requests, etc, can identify learning needs, especially if the format of looking at what is satisfactory and what leaves room for improvement is followed.

	The difference between needs assessment and assessment

Top Learning needs assessment in... The definition of need Methods of needs assessment The difference between needs... Learning for needs References

Needs assessment is not the same as assessment in the sense of examination of learning. Assessment systems that lead to academic or professional awards should show certain minimum characteristics, including measurement of performance against external criteria and standards, a decision on adequacy by an assessor, and standardised data gathering.²⁵ Needs assessment might sometimes have these characteristics, but it also might be based on practice, reflection, professional judgment, discussion, and informal data. Needs assessment methods that are limited by the standards of assessment will fall into the trap of assessing only a narrow range of needs.

	Learning for needs

Top Learning needs assessment in... The definition of need Methods of needs assessment The difference between needs... Learning for needs References

The main purpose of needs assessment must be to help educational planning, but this must not lead to too narrow a vision of learning. Learning in a profession is unlike any other kind of learning. Doctors live in a rich learning environment, constantly involved in and surrounded by professional interaction and conversation, educational events, information, and feedback. The search for the one best or "right" way of learning is a hopeless task,¹ especially if this is combined with attempting to "measure" observable learning. Research papers show, at best, the complexity of the process.

Multiple interventions targeted at specific behaviour result in positive change in that behaviour.²⁶ Exactly what those interventions are is less important than their multiplicity and targeted nature. On the other hand, different doctors use different learning methods to meet their individual needs. For example, in a study of 366 primary care doctors who identified recent clinical problems for which they needed more knowledge or skill to solve, 55 different learning methods were selected.²⁷ The type of problem turned out to be the major determinant of the learning method chosen, so there may not be one educational solution to identified needs.

Much of doctors' learning is integrated with their practice and arises from it. The style of integrated practice and learning ("situated learning") develops during the successive stages of medical education.²⁸ The components of apprenticeship learning in postgraduate training are made up of many activities that may be regarded as part of practice (see box 4).²⁹ Senior doctors might also recognise much of their learning in some of these elements and could certainly add moresuch as conversations with colleagues.

Thus, educational planning on the basis of identified needs faces real challenges in making learning appropriate to and integrated with professional style and practice. The first step in all of this is to recognise the needs assessment and learning that are a part of daily professional life in medicine and to formalise, highlight, and use these as the basis of future recorded needs assessment and subsequent planning and action, as well as integrating them with more formal methods of needs assessment to form a routine part of training, learning, and improving practice.

	Footnotes

   Competing interests: None declared.

	References

Top Learning needs assessment in... The definition of need Methods of needs assessment The difference between needs... Learning for needs References

1.	Grant J, Stanton F. The effectiveness of continuing professional development. Edinburgh: Association for the Study of Medical Education, 2000. (ASME medical education booklet.)
2.	Department of Health. A first class service: quality in the new NHS. London: HMSO, 1998.
3.	Department of Health. The new NHS: modern and dependable. London: HMSO, 1997. (Cm 3807.)
4.	Department of Health. Working for patients. London: HMSO, 1989. (Cm 555.)
5.	Batstone G, Edwards M. Management of learning and professional development through clinical audit. In: White T, ed. Textbook of management for doctors. London: Churchill Livingstone, 1996:341-358.
6.	Knowles MS. Using learning contracts: approaches to individualising and structuring learning. San Francisco: Jossey-Bass, 1986.
7.	Knowles MS. Self-directed learning: a guide for learners and teachers. New York: Association Press, 1975.
8.	Stufflebeam DL, McCormick CH, Brinkerhoff RO, Nelson CO. Conducting educational needs assessment. Boston: Kluwer Nijhoff, 1985.
9.	Gillam SJ, Murray SA. Needs assessment in general practice. London: Royal College of General Practitioners, 1996. (Occasional paper 73.)
10.	Lockyer J. Needs assessment: lessons learned. J Contin Educ Health Prof 1998; 18: 190-192[CrossRef].
11.	Myers P. The objective assessment of general practitioners' educational needs: an under-researched area? Br J Gen Pract 1999; 49: 303-307[ISI][Medline].
12.	Grant J, Chambers G, Jackson G, eds. The good CPD guide. Sutton: Reed Healthcare, 1999.
13.	Eve R. Learnng with PUNs and DENs: a method for determining educational needs and the evaluation of its use in primary care. Educ Gen Pract 2000; 11: 73-79.
14.	Glazebrook R, Chater B, Graham P. Rural and remote Australian general practitioners' educational needs in radiology. J Contin Educ Health Prof 2001; 21: 140-149[CrossRef][Medline].
15.	Westkaemper R, Doherty M, Woolf AD. Multiple choice question quiz: valid test for needs assessment in CME in rheumatology and for self-assessment. Ann Rheum Dis 2001; 60: 740-743[Abstract/Free Full Text].
16.	Knowles N. The adult learner: a neglected species. 4th ed. Houston: Gulf, 1990.
17.	Moore DE. Needs assessment in the new health-care environment: combining discrepancy analysis and outcomes to create more effective CME. J Contin Educ Health Prof 1998; 18: 133-141[CrossRef].
18.	Schon DA. Educating the reflective practitioner: towards a new design for teaching and learning in the professions. In: San Francisco: Jossey-Bass, 1987.
19.	Kolb DA, Rubin IM, McIntyre JM. Organizational psychology: an experiential approach. Englewood Cliffs, NJ: Prentice Hall, 1979.
20.	Parboosingh IJT, Thivierge RL. The maintenance of competence (MOCOMP) programme. Ann R Coll Physicians Surg Can 1993; 26: 512-517.
21.	Campbell C, Parboosingh J, Gondocz T, Babitskaya G. A study of the factors that influence physicians' commitments to change their practice using learning diaries. Acad Med 1999; 74: S34-S36[ISI][Medline].
22.	Ramsey PG, Carlene JD, Inui TS, Larson EB, LoGerfo JP, Wenrich MD. Predictive validity of certification by the American Board of Internal Medicine. Ann Int Med 1989; 110: 719-726.
23.	Paget NS, Newble DI, Saunders NS, Du J. Physician assessment pilot study for the Royal Australasian College of Physicians. J Contin Educ Health Prof 1996; 16: 103-111.
24.	Pringle M. Significant event auditing. In: van Zwanenberg T, Harrison J, eds. Clinical governance in primary care. Abingdon: Radcliffe Medical Press, 2000:105-115.
25.	Jolly B, Grant J. The good assessment guide: a practical guide to assessment and appraisal for higher specialist training. Milton Keynes: Open University Centre for Education in Medicine, 1997.
26.	Pyatt RS, Moore DE, Caldwell SC. Improving outcomes through innovative continuing medical education partnership. J Contin Educ Prof 1997; 17: 239-244.
27.	McClaren J, Franco E, Snell L. Type of clinical problem is a determinant of physicians' self-selected learning methods in their practice settings. J Contin Educ Health Prof 1998; 18: 107-118.
28.	Lave J, Wenger E. Situated learning: legitimate peripheral participation. Cambridge: Cambridge University Press, 1991.
29.	Macdonald MM. Craft knowledge in medicine: an interpretation of teaching and learning in apprenticeship [thesis]. In: Milton Keynes: Open University, 1998.

(Accepted 17 December 2001)