Ruling a diagnosis in or out with “SpPIn” and “SnNOut”: a note of caution
BMJ 2004; 329 doi: https://doi.org/10.1136/bmj.329.7459.209 (Published 22 July 2004) Cite this as: BMJ 2004;329:209All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Evidence based medicine, which has become a religion in some parts of
the world, is fraught with some difficulties. The concept of using
likelihood ratios to rule in or rule out diagnoses with the aid of
nomograms, relies on a somewhat accurate assessment of pretest
probability. While more decision aids/clinical experience/epidemiological
data help in this assessment of pretest probability, the fact is that a
good deal of guesswork and clinical intuition is still involved. Applying
accurate likelihood ratios to inaccurate estimates of pretest probability
leads to inaccurate estimates of posttest probability.
Instead of trying to legitimize a quantitative approach that gives us
estimates that appear precise (i.e. a 50% pretest probability followed by
a negative test with a LR- of 0.1 gives a posttest probability of about
9%, while if we admit that our pretest probability has intervals between
40 and 60%, and out test likelihood ratios has intervals between 0.05 and
0.4, the posttest probability of a negative test could be anywhere from
about 3% to over 30%), but aren't precise, we should use this information
to know what is a good test to rule out a disease, and what is a good test
to rule in a disease sufficient so that the next step will be done. To
set arbitrary cut-offs such as "a 2% miss rate is clinically acceptable",
assumes that the confidence intervals of pretest probability and the
likelihood ratios are significantly compact enough.
Use likelihood ratios as a barometer of how good a test is, and the
confidence intervals of the likelihood ratio as a barometer of how well we
know how good the test is, not to calculate meaningless numbers.
In terms of the CAGE questionnaire, a real problem comes when studies
start to USE the CAGE questionnaire as the gold standard for alcoholism -
instead of DSM or other criteria. By the CAGE questionnaire, every
university student and medical student I know would be labelled as an
alcoholic - and while some undoubtedly are, the majority are not. I'm
going to go get a drink, and don't you dare criticize me for it!
Competing interests:
None declared
Competing interests: No competing interests
Pewsner et al (1) are correct in identifying the problems with overly
facile application the SpPIn and SnNOut approaches for implementing
clinical decision rules or making diagnoses. As appealing as the
principles of "sensitive test rules out" and "specific test rules in" are
when taken in isolation, this approach oversimplifies the information and
evidence behind tests and decision rules. This approach essentially
reduces medicine to "cook-book" application of simple axioms in the name
of evidence-based medicine, at the expense of attaining a more in-depth
understanding of the evidence that supports a particular decision rule.
Nearly a decade ago the basic approach behind evidence-based medicine
was outlined: (2)
i. Formulate a clear clinical question from a patient's
problem
ii. Search the literature for relevant clinical articles
iii. Evaluate (critically appraise) the evidence for its
validity and usefulness
iv. Implement useful findings in clinical practice
While it is true that the individual practitioner on a daily basis
may not be able to, or have the inclination to, search out the best
primary source articles on each clinical question that arises; it is also
true that if clinicians focus TOO much on the implemtation of evidence
that has been appraised by others (such as, overly simplified use of
decision rules), then these rules will be applied incorrectly and without
a proper understanding of their strenghts and limitations.
The practice of medicine demands clear and well-informed thinking -
that is, well-informed critical thinking. A vital part of critical
thinking includes a sound understanding of the primary sources of
information (in medicine, these primary sources are clincial trials and
other primary research). While the average clinician may do little actual
critical appraisal of primary sources, ALL clinicians must be educated in
the process so that their application of information derived from an
ostebsibly "evidence-based" approach to medicine will be well-informed,
and not simply blind application of an "EBM rule" rather than blind
applicatoin of what the local experts say.
Medicine demands critical thinking. Pewsner et al (1) demonstrate
what happens when the "critical thinking" aspect of evidence-based
medicine is abandoned for the sake of simplicity and ease of application.
In training clinicians, both our colleauges and our students, those of use
who espouse evidence-based medicine must strive to train truly well-
informed critical thinkers.
(1) Pewsner D, Battaglia M, Minder C, Marx A, Bucher HC, Egger M.
Ruling a diagnosis in or out with "SpPIn" and "SnNOut": a note of caution.
BMJ 2004; 329: 209-213.
(2) Rosenberg W, Donald A. Evidence based medicine: an approach to
clinical problem-solving. BMJ. 1995 Apr 29;310(6987):1122-6.
Competing interests:
None declared
Competing interests: No competing interests
EDITOR—We are totally in agreement with Pewsner et al. about the
deficiencies of ruling a diagnosis in or out with 'SpPIn' and 'SnNOut' 1.
They perfectly described the potential biases accompanying this approach
and the need for better tools (namely likelihood ratios).
Mathematical attitude in clinical encounters usually mandates us to
seek some clinical evidences to exterminate our uncertainty. Typically,
physicians want to rule out completely some particular diagnoses and
confidently rule in another. The most renowned rules for this purpose are
'SnNOut' and 'SpPIn'. However, this policy brings two major concerns.
Firstly, in real medicine, the tests with 100% sensitivity or specificity
are classically reference tests for a particular diagnosis and their
application in routine diagnostic approach has a variety of problems. For
instance, in a patient with thyroid nodule, we cannot perform total
thyroidectomy at the first steps just to rule out malignancy. Secondly, no
measurement in experimental sciences could be considered as an absolute
number. We know that even a quantity of 100% has a confidence interval.
Even a normal pathology report of thyroidectomy could not rule out
malignancy (because of possible errors in the process of surgery, section
preparation, and pathologists' agreements).
According to the Bayes' theorem, the main purpose of a clinical test
is to change the position of probability of a disease (ideally using
likelihood ratios) around a predefined threshold. The concept of threshold
is an indispensable part of clinical diagnosis, which Pewsner et al. did
not emphasize on it efficiently. For each disease, we can (and should)
assign a diagnostic or treatment threshold considering various parameters
like fatality, prognosis, prevalence, and treatability. In clinical
settings, we cannot expect to entirely eliminate our diagnostic errors;
however, by pulling down diagnostic thresholds (as the authors say, for
serious conditions), we would miss fewer patients. We should also consider
our unnecessary treatments and time and facilities spent for higher
numbers of misdiagnoses. The entire procedure seems to be a trade-off that
physicians play a great role on it.
References:
1) Pewsner D, Battaglia M, Minder C, Marx A, Bucher HC, Egger M. Ruling a
diagnosis in or out with "SpPIn" and "SnNOut": a note of caution. BMJ
2004;329:209-13
Competing interests:
None declared
Competing interests: No competing interests
The use of "SnNout and SpPin tests" do not rule out the worst
possible diagnoses in a particular patient.In today's clinical practice it
would be wiser for clinicians to consider the diagnoses that would have a
significant impact on the patient's life if not diagnosed early enough and
rule them out by appropiate investigations.
For example,in the example cited,if the patient had a ankle fracture
it would have stirred up medicolegal issues.
Cost does matter;prudence, protecting oneself and peace of mind also
does.
Competing interests:
None declared
Competing interests: No competing interests
Re: Ruling a diagnosis in or out with “SpPIn” and “SnNOut”: a note of caution
To whom it may concern,
The article is rather contradictory to definitions it uses.
If Sensitivity is about the rate of true positive results, then Sensitivity can only be used when the test result is positive, not when the test result is negative as the acronym SnNout suggests.
This is also the case for specificity. Specificity can be used when the test is negative as it is the rate of true negatives. Therefore the SpPout acronym is misleading because it assumes the test is positive.
Making conclusions using statistics that are not designed for their purpose will wreak havoc in inference.
I am ready to be corrected if I have made a mistake in my analysis.
Yours Sincerely,
Dr Mithilesh Dronavalli
MBBS BMedSc MBios MPhil GStat
www.dr-mit.com
Competing interests: No competing interests