Wikipedia:Identifying reliable sources (medicine)

See also: Wikipedia:Reliable sources, Wikipedia:No original research and Wikipedia:Manual of Style (medicine-related articles).

Wikipedia's articles, while not intended to provide medical advice, are nonetheless an important and widely-used source of health information. Therefore, it is vital that biomedical information in articles be based on reliable published sources and accurately reflect current medical knowledge. These guidelines supplement the general guidelines at Wikipedia:Reliable sources with specific attention to sources appropriate for the medical and health-related aspects of all articles. Ideal sources for these aspects include general or systematic reviews in reputable medical journals, widely recognised standard textbooks written by experts in a field, or medical guidelines and position statements from nationally or internationally reputable expert bodies. It is also useful to reference seminal papers on a subject to document its history and provide context for the experts' conclusions.

See Wikipedia:Reliable sources/Noticeboard for queries about the reliability of particular sources or ask at relevant Wikiprojects such as Wikipedia talk:WikiProject Medicine or Wikipedia talk:WikiProject Pharmacology.

Definitions

Further information: [[:Primary, secondary, and tertiary sources]]

• A primary source in medicine is one where the authors directly participated in the research or documented their personal experiences. They examined the patients, injected the rats or filled the test tubes, or at least supervised those who did. Many, but not all, papers published in medical journals are primary sources.
• A secondary source in medicine summarizes one or more primary or secondary sources, usually to provide an overview of the current understanding of a medical topic. Literature reviews, systematic review articles and specialist textbooks are examples of secondary sources, as are position statements and literature reviews by major health organizations. A good secondary source from a reputable publisher will be written by an expert in the field and be editorially or peer reviewed. Do not confuse a scientific review (the thing) with peer review (the activity).
• A tertiary source usually summarizes a range of secondary sources. General textbooks and encyclopedias are tertiary sources.

In general, medical information in Wikipedia articles should be based upon published, reliable secondary sources whenever possible. Reliable primary sources can add greatly to an article, but must be used with care because of the potential for misuse. For that reason, edits that rely on primary sources should only describe the conclusions of the source, and should describe these findings clearly so the edit can be checked by editors with no specialist knowledge. In particular, this description should follow closely to the interpretation of the data given by the authors, or by other reliable secondary sources. Primary sources should not be cited in support of a conclusion that is not clearly made by the authors or by reliable secondary sources, as defined above (see Wikipedia:No original research).

Basic advice

Respect secondary sources

Individual primary sources should not be cited or juxtaposed so as to "debunk" or contradict the conclusions of reliable secondary sources, unless the primary source itself directly makes such a claim (see Wikipedia:No original synthesis that advances a position). Controversies or areas of uncertainty in medicine should be illustrated with reliable secondary sources describing the varying viewpoints. The use and presentation of primary sources should also respect Wikipedia's policies on undue weight; that is, primary sources favoring a minority opinion should not be aggregated or presented devoid of context in such a way as to undermine proportionate representation of expert opinion in a field.

If an important scientific result is so new that no reliable reviews have been published on it, it may be helpful to cite the primary source that reported the result. Although popular-press news articles and press releases often tout the latest phase II clinical trial, such trials are rarely important enough to mention in an encyclopedia. Any such results should be described as being from a single study, for example:

"A 2009 U.S. study found the average age of formal autism spectrum diagnosis was 5.7 years." (citing PMID 19318992)

After enough time has passed for a review to be published in the area, the review should be cited in preference to the primary study. If no review is published in a reasonable amount of time, the primary source should be removed as not reporting an important result. When in doubt, omit mention of the primary study, as per WP:RECENTISM.

Summarize scientific consensus

Scientific journals are the best place to find primary source articles about experiments, including medical studies. Any serious scientific journal is peer-reviewed. Be careful of material in a journal that is not peer-reviewed reporting material in a different field. (See Marty Rimm and Sokal affair.)

The fact that a statement is published in a refereed journal does not make it true. Even a well-designed experiment or study can produce flawed results or fall victim to deliberate fraud. (See the Retracted article on neurotoxicity of ecstasy and the Schön affair.)

Neutrality and no original research policies demand that we present the prevailing medical or scientific consensus, which can be found in recent, authoritative review articles or textbooks and some forms of monographs. Although significant-minority views are welcome in Wikipedia, such views must be presented in the context of their acceptance by experts in the field. The views of tiny minorities need not be reported. (See Wikipedia:Neutral Point of View.)

Make readers aware of any uncertainty or controversy. A well-referenced article will point to specific journal articles or specific theories proposed by specific researchers.

Assess evidence quality

Several systems exist for assessing the quality of available evidence on medical subjects, and these should be kept in mind while assessing whether a particular viewpoint is a majority or minority one, and in deciding what constitutes evidence-based medicine. The best evidence comes from meta-analyses of randomised controlled trials (RCTs), and from systematic reviews of bodies of literature of overall good quality and consistency addressing the specific recommendation. Narrative reviews can help establish the context of evidence quality. Roughly in descending order of quality, lower-quality evidence in medical research comes from individual RCTs, other controlled studies, quasi-experimental studies, and non-experimental studies such as comparative, correlation, and case control studies. Although expert committee reports or opinions, along with clinical experience of respected authorities, are weaker evidence than the scientific studies themselves, they often provide helpful overviews of evidence quality. Case reports, whether in the popular press or a peer-reviewed medical journal, are a form of anecdote and generally fall below the minimum requirements of reliable medical sources.

Speculative proposals and early-stage research should not be cited in ways that suggest wide acceptance. For example, the results of an early-stage clinical trial are unlikely to be appropriate for inclusion in the Treatment section of an article about a disease, because a possible future treatment has little bearing on current treatment practice. However, the results might, in some cases, be appropriate for inclusion in an article dedicated to the treatment in question, or to the researchers or businesses involved in it. Such information, particularly if citing a secondary source, might also be appropriate for a well-documented section on research directions in an article about a disease. To prevent misunderstandings, the text should clearly identify the level of research cited, e.g., "first-in-human safety testing".

Use up-to-date evidence

Here are some rules of thumb for keeping an article up-to-date while maintaining the more-important goal of reliability. These guidelines are appropriate for actively-researched areas with many primary sources and several reviews, and may need to be relaxed in areas where little progress is being made and few reviews are being published.

• Look for reviews published in the last five years or so, preferably in the last two or three years. The range of reviews examined should be wide enough to catch at least one full review cycle, containing newer reviews written and published in the light of older ones and of more-recent primary studies.
• Within this range, things can be tricky. Although the most-recent reviews include later research results, do not automatically give more weight to the review that happens to have been published most recently, as this is recentism.
• Prefer recent reviews to older primary sources on the same topic. If recent reviews don't mention an older primary source, the older source is dubious. Conversely, an older primary source that is seminal, replicated, and often-cited in reviews is notable in its own right and can be mentioned in the main text in a context established by reviews. For example, Genetics might mention Darwin's 1859 book On the Origin of Species as part of a discussion supported by recent reviews.

These are just rules of thumb. There are exceptions:

• History sections often cite older work, for obvious reasons.
• Cochrane Library reviews are generally of high quality and are routinely maintained even if their initial publication dates fall outside the above window.

Use independent sources

Many medical claims lack reliable research about the efficacy and safety of proposed treatments, or about the legitimacy of statements made by proponents. In such cases, reliable sources may be much more difficult to find and unreliable sources can often be more readily available. Whenever writing about medical claims not supported by mainstream research, it is vital that third-party, independent sources be used. Sources written and reviewed by the advocates of such marginal ideas can be used to describe notable personal opinions, but extreme care should be taken when using such sources lest the more controversial aspects of their opinions be taken at face value or, worse, asserted as fact. If the independent sources discussing a medical subject are of low quality, then it is likely that the subject itself is not notable enough for inclusion.

Choosing sources

PLoS Medicine and other open access journals can be useful as sources for images in Wikipedia articles. Because the above image was published under the terms of a Creative Commons license, it can be uploaded to Wikimedia Commons and used in Wikipedia. Click on the above image to find its source.

A Wikipedia article should cite the best and most-reliable sources regardless of whether they require a fee or a subscription. When all else is equal, it is better to cite a source whose full text is freely readable, so that your readers can follow the link to the source. Some high-quality journals, such as JAMA, publish a few freely-readable articles even though most are not free. A few high-quality journals, such as PLoS Medicine, publish only freely-readable sources. Also, a few sources are in the public domain; these include many U.S. government publications, such as the Morbidity and Mortality Weekly Report of the Centers for Disease Control and Prevention.

When searching for biomedical sources, it is wise to skim-read everything available, including abstracts of papers you can't fully access, and use that to get a feel for what reliable sources are saying. However, when it comes to actually writing a Wikipedia article, it is generally not a good idea to cite a source after reading only its abstract, as the abstract necessarily presents a stripped-down version of the conclusions and omits the background that can be crucial for understanding exactly what the source says. You may need to visit a medical library to access the full text, or ask somebody at the WikiProject Resource Exchange or at WikiProject Medicine's talk page to either provide you with a copy or read the source for you and summarize what it says; if neither is possible, you may need to settle for using a lower-impact source or even just an abstract.

Biomedical journals

As mentioned above, the biomedical literature contains two major types of sources: primary publications describe novel research for the first time, and review articles summarize and integrate a topic of research into an overall view. In medicine, primary sources include clinical trials, which test new treatments; secondary sources include meta-analyses that bring together the results from many clinical trials and attempt to arrive at an overall view of how well a treatment works. It is usually best to use reviews and meta-analyses where possible, as these give a balanced and general perspective of a topic, and are usually easier to understand!

Peer-reviewed medical journals are a natural choice as a source for up-to-date medical information in Wikipedia articles. They contain a mixture of primary and secondary sources, as well as less technical material such as biographies. Although almost all such material will count as a reliable source, not all the material is equally useful. Journal articles come in many types: original research, reviews, editorials, advocacy pieces, speculation, book reviews, correspondence, biographies and eulogies. Research papers are primary sources; although they normally contain previous-work sections that are secondary sources, these sections are typically less reliable than reviews. A general narrative review of a subject by an expert in the field makes a good secondary source that can be used to cover various aspects of a subject within a Wikipedia article. Such reviews typically contain no original research but can make interpretations and draw conclusions from primary sources that no Wikipedia editor would be allowed to do. A systematic review uses a reproducible methodology to select primary studies meeting an explicit criteria in order to answer a specific question. Such reviews should be more reliable, accurate and less prone to bias than a narrative review. However, a systematic review's focus on answering one question limits its usage as a source on Wikipedia.

Some journals specialize in particular article types. A few, such as Evidence-based Dentistry (ISSN 1462-0049), publish third-party summaries of reviews and guidelines published elsewhere. If you have access to both the original source and the summary, and you find the summary helpful, it is good practice to cite both sources together (see Formatting citations for details). Others, such as Journal of Medical Biography, publish historical material that can be valuable for History sections but is rarely useful for current medicine. Still others, such as Medical Hypotheses, publish speculative proposals that are not reliable sources for biomedical topics.

The 2003 Brandon/Hill selected list includes 141 journals suitable for a small medical library. Although this list is no longer maintained, the listed journals are of high quality. The core general medical journals include the New England Journal of Medicine, The Lancet, Journal of the American Medical Association (JAMA), Annals of Internal Medicine, British Medical Journal (BMJ), and Canadian Medical Association Journal. Core basic science and biology journals include Science, Cell, and Nature.

Books

Medical textbooks published by academic publishers are often excellent secondary sources. If a book has as its declared target audience students, it may not be as complete as a monograph or chapter in a book intended for professionals or postgraduates. Ensure the book is up-to-date, unless a historical perspective is required. Doody's maintains a list of core health sciences books, which is available only to subscribers. Major academic publishers (Elsevier, Spinger Verlag, Wolters Kluwer, Informa to name a few) publish specialized medical book series with good editorial oversight; volumes in these series summarize the latest research in narrow areas usually in a more extensive format than journal reviews. Specialized biomedical encyclopaedias published by these established publishers are often of good quality but the information in there may be too terse for detailed articles.

Popular science and medicine books are useful tertiary sources, but there are exceptions. Most self-published books or books published by vanity presses undergo no independent fact-checking or peer review and consequently are not reliable sources.

Medical and scientific organizations

Statements and information from reputable major medical and scientific bodies may be valuable encyclopedic sources. These bodies include the U.S. National Academies (including the Institute of Medicine and the National Academy of Sciences), the British National Health Service, the U.S. National Institutes of Health (including the Centers for Disease Control and Prevention), and the World Health Organization. The reliability of these sources range from formal scientific reports, which can be the equal of the best reviews published in medical journals, through public guides and service announcements, which have the advantage of being freely readable but are generally less authoritative than the underlying medical literature.

Popular press

The popular press is generally not a reliable source for science and medicine information in articles. Most medical news articles fail to discuss important issues such as evidence quality, costs, and risks versus benefits, and news articles too often convey wrong or misleading information about health care. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new and experimental treatment as "the cure" for a disease, or an every-day substance as "the cause" of a disease. Newspapers and magazines may also publish articles about scientific results before those results have been published in a peer-reviewed journal, or reproduced by other experimenters. Such articles may be based uncritically on a press release, which can be a biased source even when issued by an academic medical center. News articles also tend neither to report adequately on the scientific methodology and the experimental error, nor to express risk in meaningful terms.

A news article should therefore not be used as a sole source for a medical fact or figure. Editors are encouraged to seek out the scholarly research behind the news story. One possibility is to cite a higher-quality source along with a more-accessible popular source, for example with the |laysummary= parameter of {{Cite journal}}.

On the other hand, the high-quality popular press can be a good source for social, biographical, current-affairs and historical information in a medical article. For example, popular science magazines such as New Scientist and Scientific American are not peer reviewed but sometimes feature articles that explain medical subjects in plain English. As the quality of press coverage of medicine ranges from excellent to irresponsible, use common sense, and see how well the source fits the verifiability policy, and the general reliable sources guideline. Sources for evaluating health-care media coverage include the review websites Behind the Headlines, Health News Review[1], and Media Doctor, along with specialized academic journals such as the Journal of Health Communication; reviews can also appear in the American Journal of Public Health, the Columbia Journalism Review, the Bad Science column in The Guardian, and others. Health News Review's criteria for rating news stories can help to get a general idea of the quality of a medical news article.

Other sources

Press releases, blogs, newsletters, advocacy and self-help publications, and other sources contain a wide range of biomedical information ranging from factual to fraudulent, with a high percentage being of low quality. Conference abstracts present incomplete and unpublished data and undergo varying levels of review; often they have no review at all and are self-published sources. Consequently, they are usually poor sources and should always be used with caution, never used to support surprising claims, and carefully identified in the text as preliminary work. Peer-reviewed medical information resources such as WebMD and UpToDate can be useful guides about the relevant medical literature and how much weight to give different sources; however, as much as possible Wikipedia articles should cite the literature directly.

Searching for sources

Search engines are commonly used to find biomedical sources. Each engine has quirks, advantages, and disadvantages, and may not return the results that you need unless used carefully. It typically takes experience and practice to recognize when a search has not been effective; even if you find useful sources, you may have missed other sources that would have been more useful, or you may generate pages and pages of less-than-useful material. A good strategy for avoiding sole reliance on search engines is to find a few recent high-quality sources and follow their citations to see what your search engine missed. It can also be helpful to perform a plain web search rather than one of scholarly articles only.

PubMed is an excellent starting point for locating peer-reviewed medical sources. It offers a free search engine for accessing the MEDLINE database of biomedical research articles offered by the National Library of Medicine at the U.S. National Institutes of Health. There are basic and advanced options for searching PubMed. Clicking on the "Review" tab will help you narrow your search to reviews. The "Limits" tab can let you further limit your search, for example, to meta-analyses or to freely-readable sources. Although PubMed is a comprehensive database, many of its indexed journals restrict online access. An additional site, PubMed Central, provides free access to full text; this is often not the official published version but is a peer-reviewed manuscript that is substantially the same but lacks minor copyediting by the publisher.

Other useful search engines include:

• The Cochrane Library contains a database of systematic reviews and meta-analyses, and is a key resource in evidence-based medicine. Its reviews are generally considered to be of very high quality.
• The TRIP Database is designed to help its users answer clinical medicine questions rapidly. It searches the Cochrane Library and many other databases containing various sources that include systematic reviews, medical guidelines, and evidence-based synopses. TRIP Answers, a related website, aims to answer clinical questions directly.
• Google Scholar covers all scholarly sources. It is useful not only as a sanity check for PubMed searches, but also to cover topics outside PubMed's core purview, such as the sociological or cultural aspects of medicine.
• Google Books can also be quite useful for medical searches: it can let readers peek at a few sentences in books even when full access is not granted, and can help editors find reliable sources quickly, either by looking at the book's references or by citing the book itself.
• EMBASE is a high-quality index that often generates better results than PubMed. Unfortunately, it is proprietary and requires a subscription.
• CINAHL is a proprietary index on nursing and allied health care.

Formatting citations

Citations should document precisely how to access sources. Normally, medical citations should contain a PubMed identifier (PMID). It is good practice to also supply a digital object identifier (DOI) if available. A common practice is to supply a uniform resource locator (URL) to a source if and only if its full text is freely readable. If the {{Cite journal}} template is used, all this information can be supplied with the |pmid=, |doi=, and |url= parameters, respectively. It is also helpful to mention whether a source is also available on PubMed Central; with {{Cite journal}} this can be done with the |pmc= parameter. For example:

{{cite journal |author=Bannen RM, Suresh V, Phillips GN Jr, Wright SJ, Mitchell JC |title=Optimal design of thermally stable proteins |journal=Bioinformatics |volume=24 |issue=20 |pages=2339–43 |year=2008 |pmid=18723523 |pmc=2562006 |doi=10.1093/bioinformatics/btn450 |url=http://bioinformatics.oxfordjournals.org/cgi/content/full/24/20/2339 }}

produces this:

Bannen RM, Suresh V, Phillips GN Jr, Wright SJ, Mitchell JC (2008). "Optimal design of thermally stable proteins". Bioinformatics. 24 (20): 2339–43. doi:10.1093/bioinformatics/btn450. PMC 2562006. PMID 18723523.{{cite journal}}: CS1 maint: multiple names: authors list (link)

If you are citing a source along with an expert summary, it is helpful to list them together, with the main source first to indicate that it is more authoritative. For example:

Griffin SO, Regnier E, Griffin PM, Huntley V (2007). "Effectiveness of fluoride in preventing caries in adults". J Dent Res. 86 (5): 410–5. doi:10.1177/154405910708600504. PMID 17452559.{{cite journal}}: CS1 maint: multiple names: authors list (link) Summary: Yeung CA (2007). "Fluoride prevents caries among adults of all ages". Evid Based Dent. 8 (3): 72–3. doi:10.1038/sj.ebd.6400506. PMID 17891121.

If a source is available in both HTML and some other form, normally the HTML form should be linked to, as it is more likely to work on a wide variety of browsers.

See also

• Dispatches: Sources in biology and medicine. The Wikipedia Signpost (2008-06-30)
• Reliable source examples in physical sciences, mathematics and medicine
• Users’ Guides to the Medical Literature
• {{MEDRS}} – adds:

Further reading

• Gray M (2009). Evidence-Based Health Care and Public Health: How to Make Decisions About Health Services and Public Health (3rd ed.). Edinburgh: Churchill Livingstone. ISBN 978-0-443-10123-6.
• Greenhalgh T (2006). How to Read a Paper: The Basics of Evidence-based Medicine (3rd ed.). BMJ Books. ISBN 1-4051-3976-5. The Greenhalgh citation in References is taken from an earlier version of this book, which was serialized in BMJ. Other parts of that serialization include:
  • Greenhalgh T (1997). "How to read a paper: Getting your bearings (deciding what the paper is about)". BMJ. 315 (7102): 243–6. PMC 2127173. PMID 9253275.
  • Greenhalgh T (1997). "How to read a paper: Assessing the methodological quality of published papers". BMJ. 315 (7103): 305–8. PMC 2127212. PMID 9274555.
  • Greenhalgh T (1997). "How to read a paper: Statistics for the non-statistician. I: Different types of data need different statistical tests". BMJ. 315 (7104): 364–6. PMC 2127256. PMID 9270463.
  • Greenhalgh T (1997). "How to read a paper: Statistics for the non-statistician. II: 'Significant' relations and their pitfalls". BMJ. 315 (7105): 422–5. PMC 2127270. PMID 9277611.
  • Greenhalgh T (1997). "How to read a paper: Papers that report drug trials". BMJ. 315 (7106): 480–3. PMC 2127321. PMID 9284672.
  • Greenhalgh T (1997). "How to read a paper: Papers that report diagnostic or screening tests". BMJ. 315 (7107): 540–3. PMC 2127365. PMID 9329312.
  • Greenhalgh T (1997). "How to read a paper: Papers that tell you what things cost (economic analyses)". BMJ. 315 (7108): 596–9. PMC 2127419. PMID 9302961.
  • Greenhalgh T, Taylor R (1997). "How to read a paper: Papers that go beyond numbers (qualitative research)". BMJ. 315 (7110): 740–3. PMC 2127518. PMID 9314762.
• Straus SE, Richardson WS, Glasziou P, Haynes RB (2005). Evidence-based Medicine: How to Practice and Teach EBM (3rd ed.). Edinburgh: Churchill Livingstone. ISBN 0-443-07444-5.{{cite book}}: CS1 maint: multiple names: authors list (link)
• "Users' guides to evidence-based practice". Centre for Health Evidence. 2001. Retrieved 2008-12-15. This is derived from a prepublication version of a series published in JAMA.