Towards quality management of medical information on the internet: evaluation, labelling, and filtering of informationHallmarks for quality of informationQuality on the internetAssuring quality and relevance of internet information in the real world
BMJ 1998; 317 doi: https://doi.org/10.1136/bmj.317.7171.1496 (Published 28 November 1998) Cite this as: BMJ 1998;317:1496
All rapid responses
Rapid responses are electronic comments to the editor. They enable our users to debate issues raised in articles published on bmj.com. A rapid response is first posted online. If you need the URL (web address) of an individual response, simply click on the response headline and copy the URL from the browser window. A proportion of responses will, after editing, be published online and in the print journal as letters, which are indexed in PubMed. Rapid responses are not indexed in PubMed and they are not journal articles. The BMJ reserves the right to remove responses which are being wilfully misrepresented as published articles or when it is brought to our attention that a response spreads misinformation.
From March 2022, the word limit for rapid responses will be 600 words not including references and author details. We will no longer post responses that exceed this limit.
The word limit for letters selected from posted responses remains 300 words.
Dr Rashid is totally right in that there should be guidelines for
"online-doctoring" as well. We have pointed this out and discussed this in other
publications [1-3]. Several national guidelines already exist, but an international consensus
would be even more desirable, as patients easily cross borders to obtain health advice
[3]. A possible technical solution to establish trust again would be to distribute
metainformation on the qualification of the doctor, which we have called a cyberlicence
[2]. medPICS-labels may also contain digital signature and may therefore also serve as
electronic identity card, identifying health care professionals on the Internet, as
discussed in detail elsewhere [5]. This would at least enable consumers to verify the
medical qualification of a cyberdoctor.
Gunther Eysenbach, Researcher
Cybermedicine Unit, University of Heidelberg
References
[1] Eysenbach G, Diepgen TL (1998): Responses to unsolicited patient e-mail requests
for medical advice on the World Wide Web. JAMA.;280:1333-1335.
[2] Eysenbach G, Diepgen TL (1998). Evaluation of Cyberdocs. Lancet; 352 (9139):
1526
[3] Eysenbach G, Diepgen TL (1999). Patients Looking for Information on the Internet
and Seeking Teleadvice: Motivation, Expectations, and Misconceptions as Expressed in
E-mails Sent to Physicians. Arch Dermatol;135:151-156
[49 Eysenbach G, Diepgen TL (1999 – in press). Labeling and filtering of medical
information on the Internet. Meth Inf Med
Competing interests: No competing interests
Re: Quality control on the Internet: interpretation of current situation and likely future developme
Mr Whatling has charged us of being arrogant and naive. Arrogant, because we implied
that doctors can judge medical contents better than consumers. Naive, because Mr Whatling
is convinced that it is "very easy to disguise and falsify one's origin on the
internet" thus it would be impossible to create any system that could not be
undermined by anyone.
First of all, the statement "Patients have read and heard numerous reports of
magical cures and cancer healers for hundreds if not thousands of years in one form or
another and have given these the short shrift they deserve." is a document of
ignorance itself. To extrapolate from the middle ages where people not even had books,
through the emergence of the Internet, the most profound change in communication since
Gutenberg and to predict that all this will not change how people understand and digest
information is obviously a daring effort.
"The medium is the message". This well-known aphorism was coined by Marshall
McLuhan [1], who describes that any new medium or technology would impose a new
environment and that the modes of disseminating data will cause culture changes: "We
shape our tools and they in turn shape us," is the phrase McLuhan used to describe
this technological determinism. The Internet is the world's largest library, shopping
mall, business market, museum, university, health information provider and entertainment
vehicle. It facilitates the sharing of ideas, data and tools, creating an ethos of
co-operation and collaboration, but also creates an environment of chaos and information
overload. To assume that this will not have an impact on public health and change our
culture, change the way how we deal with and change the way we see and assess the things
around us is ignorance in its purest form. Content sometimes follows form, and the
insurgent technologies give rise to new structures of feeling and thought.
Secondly, I do indeed think that physicians can judge the reliability of medical
information better than consumers. I even think it is the job of physicians to do so, to
aid consumers through the information jungle. We are not talking about "quality"
of websites in respect to their design. We are talking about the reliability of health
information. It is hard to see why it is arrogant for someone who underwent years of
training in a certain discipline if he/she offers to make his knowledge and judgement
available to consumers. With the same argument it would be "arrogant" for
peer-reviewers to scientific journals to make a judgement on a submitted paper or
"arrogant" for a doctor to give health advice to a patient in the daily
practice.
Thirdly, the statement we would propose that "source filtering will confirm the
origin of information " is a fundamental misunderstanding. Raters could
"sign" their rating with digital signature labels [2] and the endusers software
can check the authenticity of this digital signature. The comparison with spam mail is an
inappropriate and misleading comparison, as these don't contain digital signatures. The
notion that the Internet is an insecure medium and everything on it can be falsified is
incorrect.
[1] McLuhan M (1994). Understanding Media - The Extensions of Man. Cambridge: MIT Press
[2] PICS Signed Labels (DSig) 1.0 Specification. http://www.w3.org/TR/REC-DSig-label/
Competing interests: No competing interests
Dr Wjst, please go back to the article and read it carefully again.
A word of clarification: We are not propagating any censorship. We are
propagating a culture of supplying consumers with meta-information, that
is information about information from third parties, to help them to put
information into context.
Besides, organisations such as HON (http://www.hon.ch) do not rate any
health information as you seem to understand (in fact, not a single
physician is employed at HON). The HON merely set up some ethical rules
and webauthors can display HON logos if they feel they stick to the HON
rules (but also, if they think it is a good advertisement). medPICS would
for example be an infrastructure where these HON labels could be
transported securely and reliably to the consumer without the webauthor
having the opportunity to influence or corrupt this information flow from
third parties.
Competing interests: No competing interests
Letter Response
The General Medical Council Needs to Set Guidelines for doctors
online
Eysenbach and Diepgen1 describe methods of internet regulation. Their
focus is on static web page information. Internet sites are available
that offer patients interactive consultation online with Physicians in
real time. The regulation of doctors on such sites may be more difficult.
In addition, anecdotal evidence suggests that patients may choose to use
this medium when trying to acquire drugs that are not available in their
host country2.
How are we to regulate these online consultations and what are the
rules when the consultation is between individuals from different
countries? Does the doctor have to work within guidelines for his home
country or the patient's country of residence. What are the medical
indemnity issues? Clearly there are a number of questions that need to be
answered while this technology is in its infancy.
Advances are being made in telemedicine that will allow the patient
and doctor to see each other through the internet. This will increase the
popularity of the World Wide Web for 'virtual' consultation. However
patients will need to have the computer hardware, as well as a 256K modem
bandwidth.
Just as the Government is piecing together a white paper on e-
commerce to protect consumers, the General Medical Council should aim to
set out guidelines for medicine on line. Doctors in turn must aim to abide
by the Hippocratic oath that has seen technological revolutions come and
go.
Dr Asrar Rashid
Specialist Paediatric Registrar
Hartlands Hospital,
West Midlands
Asrar_rashid@hotmail.com
[1] Eysenbach G, Diepgen TL, Gray JAM. Towards quality management of
medical information on the internet: evaluation, labelling, and filtering
of information. BMJ 1998;317: 1496-1502
[2] Ward C. 'Keep taking the tablets'. Net; 54: 46-52.
Competing interests: No competing interests
Editor--
Eysenbach and Diepgen identify many of the problems associated with
use of the Internet [1] however we believe that approach suggested will
merely add another layer of bureaucracy and not solve any of the
difficulties identified. Just as strategies exist among web-site
designers at present to ensure that their Internet site gains prominence
with the various search engines that might be used to look for them; so
medical web-site authors would be able to manipulate the "metadata"
supplied. We may not be any further forward.
Hall-marking [2] is suggested as an alternative. Hall-marking is
appropriate for items of enduring and increasing value. The value of most
information declines with time. It is not possible at the time of assay
to predict if the knowledge supplied will decline not at all, in a linear
or even exponential way. If it were we could create electronic "self-
decaying" electronic marks which decayed appropriately with time.
Instead it may be better to focus on the particular benefits Internet
technology offers and how to harness it. One of the advances has been the
ability to search across a number of different sites with Internet search
engines. However the problem with these search engines is that they often
produce too many responses, many of them not relevant to the context of
the individual searching. The challenge is to produce a search-engine
which provides the right volume of highly relevant, quality information.
Such a search engine can be produced in an "up-stream" filtering way by
searching across a few major evaluated sites. In the Doctors Desk Project3
we have tried to focus on EBM and have structured a Search Engine that
searches across such sites as the Cochrane Abstracts and Bandolier. We
feel this gives good coverage but allows the user to be presented with a
more select choice.
A development to this would be the addition of a "downstream" element
-sources of information that a user wishes to specify. For example a user
could specify a search across the MeReC site4 and the locally agreed
therapeutics guidelines . This would give the ability to the busy
clinician to search both national and local information sources
simultaneously.
References:
1 Eysenbach G, Diepgen TL. Towards quality management of medical
information on the internet: evaluation, labelling, and filtering of
information. BMJ 1998;317:1496-1500
2 Gray JAM. Hallmarks for quality of information. BMJ 1998;1500
3 Internet demonstration site: http://drsdesk.sghms.ac.uk
de Lusignan S. The Doctors Desk One vision of how to deliver the
future information and communication needs of General Practice. J. of
Informatics in Primary Care May 1998;19-22.
4 National Prescibing Centre MeReC http://www.npc.co.uk/
Competing interests: No competing interests
Postscard Ref 83862
Sir,
Eysenbach and Diepgen complain in their recent paper [1] the low quality
of internet based services like webpages, newsgroups, email communication
and mailing lists. They suggest a certification authority that should
censor web content in various ways ("upstream filtering", "downstream
filtering").
In my opinion there are several misunderstandings of the nature of the
internet. As already mentioned in the accompanying editorial [2] patients
have always asked their neighbours and relatives first before they went
for a physician consultation. Now they use a search engine, go to web
portals or ask their questions in newsgroups. It is not possible to
control access to this type of information and there are also no
convincing argument for such invasive control mechanisms. There have been
always inequalities of medical care, with different standards in different
countries. The authors have also not complained about the hundred of quack
articles published in the yellow press every week. In addition the
approach of the authors of faking emails to individuals is not an accepted
measurement for quality control.
Nevertheless there may be ways to achieve medical standards also in the
internet. There are several bodies that are at the forefront of the
medical knowledge, namely, the scientific associations, the journals and
initiatives like the Cochrane group or the HON initiative [3]. All have
extensive experience in rating medical information, all could work
together to develop common criteria for rating web sites. While two years
ago, most of the referring pages to a major medical web site came from
search engines like Altavista or Lycos, now the majority of links now come
from portal sites like Yahoo [4]. Why not creating a medical portal site
by all interested parties?
Finally, physicians and scientists should acknowledge that contributions
to web pages are worthwile and page hits are as relevant as impact factors
of printed articles. The best sites will attract most visitors and
qualified web pages by physicians will make quack pages more or less
unimportant. Also if we have no empirical data at the moment on the public
health impact of the internet [2], it is reasonable to assume that sites
with high traffic will influence individual patient behaviour world-wide.
These data are urgently required. However, we do not need medical police
officers in the internet. There are already lists of the most dubious
disease cure pages [5] and.the current internet users are not as
illiterate as Eysenbrinck and Diepgen suggest [1]. We should therefore not
go back to any censorship. Free speech for all has been driving force for
connecting computers to the internet [6] and will be essential also in the
future.
References:
[1] Eysenbach G, Diepgen TL. Towards quality managment of medical
information on the internet: evaluation, labelling and filtering of
information. BMJ 1998;317:1496-1502
[2] Coiera E. Information epidemics, economics and immunity on the
internet. BMJ 1998;317:1469-1470
[3] Health on the Net Foundation http://www.hon.ch [28.11.1998]
[4] Wjst M. An asthma information service in the internet. Am Rev Resp
Crit Care Med 1997;155:A73, http://www.mdnet.de/asthma [28.11.1998]
[5] „Weird & wacky asthma net links“
http://asthma.miningco.com/msub12.htm [28.11.1998]
[6] Blue Ribbon Campaign for online free speech
http://www.eff.org/blueribbon.html [28.11.1998]
Competing interests: No competing interests
Editor--
Eysenbach and Diepgen have summarised very clearly the problems of
quality control on the Internet but have failed in their interpretation of
the current situation and likely future developments.[1] There can be no
doubt that electronic communication is the way of the future and this
combined with advances in data transfer rates is increasing the world wide
availability of information in many areas, of which Medicine is only one.
The fundamental mistake that has been made by the authors in this
paper is to underestimate the discriminatory abilities of those who are
accessing the information. Patients have read and heard numerous reports
of magical cures and cancer healers for hundreds if not thousands of years
in one form or another and have given these the short shrift they deserve.
The internet may well increase access to this misinformation [2] but will
not change people’s interpretation of it. There is a seam of arrogance
running through all of the arguments on medical publication on the
internet which suggests that only Doctors can analyse the
quality of data produced and interpret accordingly.
The suggestion that any form of filtering will lead to a significant
degree of certainty as to the origin of the information implies a naivety
that could only be matched by the subscribers to some of the magical cures
offered online. It is still very easy to disguise and falsify one's origin
on the internet; how many readers have received "junk email" which appears
to be from an entirely reputable source but actually originates from
somewhere untraceable? The reality is that where money is involved, as in
the "cancer healer" Ryke Geerd Hamer [1], web site authors will not be
averse to falsifying certificates of authenticity.
Our role then as responsible medical practitioners has to be one of
leading by example, developing widely publicised and
authoritative websites with formal and open peer review of the material
published there. We must not dismiss the input that the non-medical reader
may provide, both with constructive criticism of methodology, but also
with the different, and often difficult to pre-judge, interpretation of an
end user. Perhaps a more free and open publication of data which can then
be openly assessed and criticised without the inordinate delays inherent
in paper publishing will reduce the publication bias for positive results
[3], encourage clinicians to develop their research in more relevant ways
and lead to an enhancement of the discussion
of those results. Specific medical discussion groups and noticeboards can
easily be passworded thus preventing inappropriate access where necessary.
The end aim must be to bring medical research to the forefront of
public awareness, to increase understanding and ultimately to increase
funding for further research. The internet is an open market and
additional regulation and red tape will strangle online presentation of
scientific and medical research before it starts to grow and sideline it
in favour of the exciting and lively sites run by the quacks of this
world.
Mr Paul Whatling
Specialist Registrar in Vascular Surgery
Nuffield Department of Surgery
John Radcliffe Hospital
Oxford
OX3 9DU
References:
1 Eysenbach G, Diepgen TL. Towards quality management of medical
information on the internet: evaluation, labelling, and filtering of
information. BMJ 1998;317:1496-1502
2 Arunachalam S. Assuring quality and relevance of internet
information in the real world. BMJ 1998;315:1501-1502
3 Stern JM, Simes RJ. Publication bias: evidence of delayed
publication in a cohort study of clinical research projects. BMJ
1997;315:640-645
Competing interests: No competing interests
Quality Standards for Information on the Internet
The large volume of health information resources available on the Internet has great potential to improve health, but it is increasingly difficult to discern which resources are accurate or appropriate for users. The problem is not too little information, but rather too much, which has the potential to be misleading, incomplete, or inaccurate. (1) I wholeheartedly agree with the authors that some form of internal or external policing needs to ensue.
Minimum bare bones criteria that have been offered for evaluation include (2):
1.Authorship: Authors and contributors, their affiliations, and relevant credentials should be provided.
2.Attribution: References and sources for all content should be listed clearly, and all relevant copyright information noted.
3.Disclosure: Web site "ownership" should be prominently and fully disclosed, as should sponsorship, advertising, underwriting, commercial funding interests or support or any potential conflicts of interest.
4.Currency: Dates that content was posted and updated should be indicated.
Several online organizations that have established themselves as "gatekeepers" of quality medical information on the web include:
Several online organizations that have established themselves as "gatekeepers" of quality medical information on the web include:
Each of these online organizations have their own criteria for quality sites, and given the evolving nature of the internet there is not yet any clear consensus on evaluating web sites for quality, although the minimum standards delimited above, as well as the standards of the above mentioned online organizations are a minimum starting point, for screening the quality of a web site for providers and patients.
For patient oriented information: An evidence based tool the DISCERN instrument has been developed to assess the quality of the information provided by web sites. This should be used as a screening tool prior to recommending a web site to a patient and including it on a web site. (3)
In conclusion, in this era of "information overload" (4) and "Caveant Lector" (2) If we don't self police the information on the website and train our patients to using the above suggestions. No one will. Either via internal tags or external ratings...quality needs to be addressed.
References:
1. Published criteria for evaluating health related sites: review: BMJ, Vol 318, 6 March, 1999 pp 647-649.
2. Silberg, William M: Assesing, Controlling and Asuring the Quality of Medical Information on the Internet: Caveant Lector et Viewor-Let the Reader and Viewer beware, ed. JAMA Vol 277(15) 16 April 1997 pp 1244-1245
3. Beyea: AORN J, Volume 72(5).November 2000.906-910 http://www.discern.org.uk/discern__instrument.htm
4. FUTURE SHOCK, Alvin Toffler Random House, NY, 1970
Competing interests:
None declared
Competing interests: No competing interests