BMJ  2003;327:716-719 (27 September), doi:10.1136/bmj.327.7417.716

Clinical review

Understanding sensitivity and specificity with the right side of the brain

¹ Department of Community, Occupational, and Family Medicine, National University of Singapore, Singapore

Can you explain why a test with 95% sensitivity might identify only 1% of affected people in the general population? The visual approach in this article should make the reason clearer

Introduction

I first encountered sensitivity and specificity in medical school. That is, I remember my eyes glazing over on being told that "sensitivity = TP/TP+FN, where TP is the number of true positives and FN is the number of false negatives." As a doctor I continued to encounter sensitivity and specificity, and my bewilderment turned to frustration—these seemed such basic concepts; why were they so hard to grasp? Perhaps the left (logical) side of my brain was not up to the task of comprehending these ideas and needed some help from the right (visual) side. What follows are diagrams that were useful to me in attempting to better visualise sensitivity, specificity, and their cousins positive predictive value and negative predictive value.

Sensitivity and specificity

I will be using four symbols in these diagrams (fig 1). Let us start by looking at a hypothetical population (fig 2). The size of the population is 100 and the number of people with the disease is 30. The prevalence of the disease is therefore 30/100 = 30%.

View larger version (39K): [in this window] [in a new window]   Fig 1 Key to symbols

 

View larger version (22K): [in this window] [in a new window]   Fig 2 Hypothetical population

 

Now let us imagine applying a diagnostic test for the disease to this population and obtaining the results shown in figure 3. The test has correctly identified most, but not all of the people with the disease. It has also correctly labelled as disease free most, but not all, of the well people. Calculating sensitivity and specificity will allow us to quantify these statements.

View larger version (30K): [in this window] [in a new window]   Fig 3 Results of diagnostic test on hypothetical population

 

Sensitivity refers to how good a test is at correctly identifying people who have the disease. When calculating sensitivity we are therefore interested in only this group of people (fig 4). The test has correctly identified 24 out of the 30 people who have the disease. Therefore the sensitivity of this test is 24/30 = 80%.

View larger version (23K): [in this window] [in a new window]   Fig 4 Sensitivity of test

 

Specificity, on the other hand, is concerned with how good the test is at correctly identifying people who are well (fig 5). The test has correctly identified 56 out of 70 well people. The specificity of this test is therefore 56/70 = 80%.

View larger version (22K): [in this window] [in a new window]   Fig 5 Specificity of test

 

Having a high sensitivity is not necessarily a good thing, as we can see from figure 6. This test has achieved a sensitivity of 100% by using the simple strategy of always producing a positive result. Its specificity, however, clearly could not be worse, and the test is useless. By contrast, Figure 7 shows the result a perfect test would give us.

View larger version (28K): [in this window] [in a new window]   Fig 6 Test with 100% sensitivity

 

View larger version (29K): [in this window] [in a new window]   Fig 7 Perfect test

 

Predictive values

Now let us consider positive predictive value and negative predictive value. We will again use the population introduced in figure 3. Positive predictive value refers to the chance that a positive test result will be correct. That is, it looks at all the positive test results. Figure 8 shows that 24 out of 38 positive test results are correct. The positive predictive value of this test is therefore 24/38 = 63%.

View larger version (23K): [in this window] [in a new window]   Fig 8 Positive predictive value

 

On the other hand, negative predictive value is concerned only with negative test results (fig 9). In our example, 56 out of 62 negative test results are correct, giving a negative predictive value of 56/62 = 90%.

View larger version (23K): [in this window] [in a new window]   Fig 9 Negative predictive value

 

The interesting thing about positive and negative predictive values is that they change if the prevalence of the disease changes. Let's assume that the prevalence of disease in our population has fallen to 10%. If we were to use the same test as before, we would obtain the results in figure 10. The sensitivity and specificity have not changed (sensitivity = 8/10 = 80% and specificity = 72/90 = 80%), but the positive predictive value is now 8/26 = 31% (compared with 63% previously) and the negative predictive value is 72/74 = 97% (compared with 90% previously).

View larger version (36K): [in this window] [in a new window]   Fig 10 Results of testing population with disease prevalence of 10%

 

In fact, for any diagnostic test, the positive predictive value will fall as the prevalence of the disease falls while the negative predictive value will rise. This is not really so mystifying if we consider the prevalence to be the probability that a person has the disease before we do the test. A low prevalence simply means that the person we are testing is unlikely to have the disease and therefore, based on this fact alone, a negative test result is likely to be correct. The following real example should make this clearer.

A real example

So far we have been discussing hypothetical cases. Let us now take a look at the use of the antinuclear antibody test in the diagnosis of systemic lupus erythematosus. I have massaged the numbers slightly to make them easier to illustrate, but they are close to reported figures in both the United Kingdom and Singapore.^{1 2} The prevalence of systemic lupus erythematosus is 33 in 100 000, and the antinuclear antibody test has a sensitivity of 94% and a specificity of 97%. To visualise this we need to imagine 1000 of the 10 by 10 squares used in the earlier figures (fig 11). Only one of these squares contains some patients with the disease.

View larger version (72K): [in this window] [in a new window]   Fig 11 Prevalence of systemic lupus erythematosus

 

Figure 12 shows the result of applying the antinuclear antibody test to this population. There are many more true negative results than false negative results and many more false positive than true positive results. The test therefore has a superb negative predictive value of 99.99% and a depressingly low positive predictive value of about 1%. In practice, since most diseases have a low prevalence, even when the tests we use have apparently good sensitivity and specificity we may end up with dismal positive predictive values.

View larger version (87K): [in this window] [in a new window]   Fig 12 (top) Results of antibody nuclear test in systemic lupus erythematosus; (bottom) negative and positive predictive values

 

View larger version (55K): [in this window] [in a new window]

 

Knowing that the positive predictive value of this test is 1%, we may then ask: does a positive test result in a female patient with arthritis, malar rash, and proteinuria really mean that she has only a 1% chance of actually having systemic lupus? The answer is no. Look at it this way—the patient is not a member of the general population. She is from the population of people with symptoms of systemic lupus erythematosus, and in this population the prevalence is much higher than 33 in 100 000. Hence the positive predictive value of the test in her case is going to be much higher than 1%.

View larger version (49K): [in this window] [in a new window]

 

Using both sides of the brain

I hope that having worked through sensitivity and specificity from scratch you will be wondering why it initially seemed so confusing. It may be because of our dependence on the left (linguistic) side of the brain. When told that a test has a sensitivity of 94% and a positive predictive value of 1%, our left brain has difficulty grasping how a test can be 94% sensitive and yet be correct only 1% of the time. It is partly misled by the huge difference between prevalence, on the one hand, and sensitivity and specificity on the other. The prevalence of systemic lupus erythematosus is 0.033% while the sensitivity and specificity of the test are about 95%; this difference is of several orders of magnitude. If, for example, we developed a test with sensitivity and specificity of 99.999% rather than 95%, we would be able to boast of a positive predictive value of 97%.

---

Competing interests: None declared.

References

1. Johnson AE, Gordon C, Palmer RG, Bacon PA. The prevalence and incidence of SLE in Birmingham, England. Relationship to ethnicity and country of birth. Arthritis Rheum 1995;38: 551-8.[ISI][Medline]
2. Boey ML. Systemic lupus erythematosus. Singapore Med J 1992;33: 291-3.[Medline]

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

Relevant Articles

Do you ever mix up sensitivity and specificity?
BMJ 2003 327: 0. [Full Text]

Communicating risk: the main work of doctors

Richard Smith
BMJ 2003 327: 0. [Extract] [Full Text] [PDF]

This article has been cited by other articles:

• Scott, V., Votova, K., Scanlan, A., Close, J. (2007). Multifactorial and functional mobility assessment tools for fall risk among older adults in community, home-support, long-term and acute care settings. Age Ageing 36: 130-139 [Abstract] [Full text]  
• Grabe, D. W, Cerulli, J., Stroup, J. S, Kane, M. P (2006). Comparison of the Achilles Express Ultrasonometer with Central Dual-Energy X-Ray Absorptiometry. The Annals of Pharmacotherapy 40: 830-835 [Abstract] [Full text]  
• Vanagas, G. (2004). Receiver operating characteristic curves and comparison of cardiac surgery risk stratification systems. ICVTS 3: 319-322 [Abstract] [Full text]  
• (2003). Hit Parade. BMJ 327: 1234-1234 [Full text]  
• Peile, E. (2003). learning to stay vigilant about conditions that are rare but important. BMJ 327: 919-919 [Full text]  

Rapid Responses:

Read all Rapid Responses

It really can be confusing...

Graham B Byrnes
bmj.com, 26 Sep 2003 [Full text]

Best ever explanation of sensitivity and specificity

Nita G Forouhi
bmj.com, 26 Sep 2003 [Full text]

The question confuses rather than clarifies

Matthew McKenna
bmj.com, 26 Sep 2003 [Full text]

But it's got to get to the brain!

GH Hall
bmj.com, 26 Sep 2003 [Full text]

Confusing sensitivity for PPV

Antony R Goldstone, et al.
bmj.com, 26 Sep 2003 [Full text]

Don't confuse us further!

Angus Dobbie
bmj.com, 26 Sep 2003 [Full text]

Error in Calculations in Real Example

Susanne Haga
bmj.com, 27 Sep 2003 [Full text]

Mistake (left) in the last figure

Salvo Fedele
bmj.com, 27 Sep 2003 [Full text]

Precise language is still required for an accurate understanding of sensitivity and specificity.

Andrea Glassberg
bmj.com, 27 Sep 2003 [Full text]

But don't forget likelihood ratio!

Zeno Bisoffi, et al.
bmj.com, 27 Sep 2003 [Full text]

An excellent educational resource.

James H MacCabe
bmj.com, 27 Sep 2003 [Full text]

Oh dear

David JR Hutchon
bmj.com, 27 Sep 2003 [Full text]

Accuracy only 99%

Robert W Baker
bmj.com, 28 Sep 2003 [Full text]

Oops

Christopher J Martin, et al.
bmj.com, 28 Sep 2003 [Full text]

The over-riding importance of sensitivity

Richard G Fiddan-Green
bmj.com, 29 Sep 2003 [Full text]

Summary of responses so far

Tze-Wey Loong
bmj.com, 29 Sep 2003 [Full text]

Variable sensitivity and specificity

Christopher J Martin
bmj.com, 29 Sep 2003 [Full text]

Does sensitivity and specificity is REALLY not change?

Pisut Katavetin
bmj.com, 29 Sep 2003 [Full text]

One in a thousand?

Jennifer S Mindell
bmj.com, 29 Sep 2003 [Full text]

The left side of the brain needs to know what the right is doing

Colin C Geddes
bmj.com, 29 Sep 2003 [Full text]

A different approach to right brain thinking

Thomas F. Heston
bmj.com, 30 Sep 2003 [Full text]

Re: The over-riding importance of sensitivity

Sam Lewis
bmj.com, 30 Sep 2003 [Full text]

Teaching all this to medical students

Louis B. Jacques
bmj.com, 30 Sep 2003 [Full text]

Misunderstanding sensitivity

Sidney B Rosalki, et al.
bmj.com, 30 Sep 2003 [Full text]

Yes I can! but I think you meant something different

Chris D Evans
bmj.com, 1 Oct 2003 [Full text]

Clinical relevance of sensitivity and specificity

Nicola Petrucci
bmj.com, 1 Oct 2003 [Full text]

confused

adrienne j j garner
bmj.com, 2 Oct 2003 [Full text]

Clinical practice: not only percentage

Giulio Nati
bmj.com, 2 Oct 2003 [Full text]

Re: Clinical relevance of sensitivity and specificity

Pisut Katavetin
bmj.com, 2 Oct 2003 [Full text]

Congratulations

Sanjeev Sharma
bmj.com, 2 Oct 2003 [Full text]

Still confused?

Edward M Sivills
bmj.com, 3 Oct 2003 [Full text]

Lies damned lies and sub-editors

Paul G McIntyre
bmj.com, 3 Oct 2003 [Full text]

Quality control

Adam Jacobs
bmj.com, 5 Oct 2003 [Full text]

The right side of the brain wins in Italy too

Romolo M Dorizzi
bmj.com, 7 Oct 2003 [Full text]

Understanding Sensitivity & Specificity

Paul A Reeve
bmj.com, 7 Oct 2003 [Full text]

Confusing a clear cut concept

EAA Addou
bmj.com, 7 Oct 2003 [Full text]

What is a bad test?

Arturo Knol
bmj.com, 10 Oct 2003 [Full text]

Its true! The weird P comes to the rescue!

Taqi F Hashmi
bmj.com, 8 Oct 2003 [Full text]

Summary II; getting philosophical

Tze-Wey Loong
bmj.com, 14 Oct 2003 [Full text]

The Right Brain myth.

Neil Watson, MA, MD, FRCS
bmj.com, 14 Oct 2003 [Full text]

further comments

Fiona J Day
bmj.com, 17 Oct 2003 [Full text]

SpPIN or SnNOUT the diagnosis

Kit Byatt
bmj.com, 24 Oct 2003 [Full text]

Re: SpPIN or SnNOUT the diagnosis

Ian P Rodd
bmj.com, 21 Jan 2004 [Full text]

Communicating about sensitivity and specificity

Ben D Ewald
bmj.com, 11 May 2004 [Full text]

Keep it simple for the students

Anthony M. Benis, Sc.D., M.D.
bmj.com, 22 Mar 2006 [Full text]

Thank you!

Roxsie A Degen
bmj.com, 5 Dec 2007 [Full text]