Conclusions about why doctors change their practice were not supported by the data

Editor—Lynne A Allery and colleagues used qualitative research methods to address an important issue–namely, why doctors change their practice–but they drew conclusions that were not supported by their data.1

They interviewed 100 doctors using the critical incident technique. They asked them about instances in which they recalled having changed their practices and recorded their reasons for these changes. These methods thus seemed appropriate to explore possible reasons why doctors change their practices. The authors, however, did not inquire about instances in which the doctors could have made a change but did not. Nor did they interview doctors who did not recall any changes in their practice. Their study design thus seemed to be a case series of episodes of change in practice, lacking any control group.

The authors used their data to make causal inferences about the reasons why doctors change their practice: “nearly all changes in doctors' clinical behaviour are due to a combination of factors.” These inferences were unjustified. One should not draw conclusions about the causes of an outcome without studying subjects who did not experience it.2 Without a control group one is not even assured that the apparently causal factors were more common in subjects who experienced the outcome than in a control group who did not, much less that any apparent associations between the factors and the outcome were not due to bias. The implications for medical education that the authors give on the basis of these causal inferences should therefore be regarded as speculative.

There is increasing enthusiasm for qualitative research in medicine and health care.3 Such enthusiasm does not mean that such methods are magical. There are standards of rigour for qualitative research.4 Qualitative researchers should not be excused from recognising the limits of their methods and data.