Skip to main content

Systematic Reviews

Different types of Reviews

It is important to clarify two different uses of the term ”systematic review” which may be confusing.

A systematic review in healthcare research refers to a specific research design which gathers, analyses and appraises the evidence about a clinical question according to a designated methodological template. Typically such studies investigate the outcomes of alternative clinical interventions by acquiring and analysing large quantities of data from clinical trials or primary studies and synthesizing the strength of the resultant evidence to make clinical recommendations. Systematic reviews are most often medium scale long term research projects involving multiple researchers. The Cochrane Database of Systematic Reviews is the global repository of such systematic reviews.

A different use of the term ”systematic review” is commonly used for a structured search for, and review of, published research papers. These are most often small scale projects undertaken by individual students as part of coursework for degrees at various levels and often involve specific guidelines from module co-ordinators.  The student explores and evaluates the literature in a defined way in order to discover the evidence for and about particular healthcare actions or issues. This type of review focusses on first exploring the best ways to search the journal databases such as PubMed Medline or CINAHL. Finding and applying search terms to produce the best results and reporting the search steps, the history and the results is the next step.  The final conclusions summarize the best evidence.  You may be told that you need to conduct a systematic review when in fact you just need to perform a literature search in a systematic manner.

Structured literature searching is common to both type of review and RCSI Library has created this comprehensive LibGuide which specifies all the main steps and stages.

Research Shorts: Conducting a Systematic Literature Review

There are various different types of structured reviews of published research.  Systematic reviews and meta-analyses of research evidence are well established and well known types of studies.  Scoping reviews and rapid review are two types of studies which emerged in the healthcare literature from 2010 and are now a recognized method of identifying and surveying  issues on both broad topics and specific clinical questions. 

What is a Systematic Review?

A systematic review identifies, appraises and synthesizes the evidence that meets pre-specified eligibility criteria to answer a given research question.

Explicit methodologies are used, which are aimed at minimizing bias and producing reliable findings.

The search strategy used in the review ideally should follow established guidelines, should be comprehensive, reproducible and documented.

The Cochrane Handbook (6.1.1.2) states:   Systematic reviews of interventions require a thorough, objective and reproducible search of a range of sources to identify as many relevant studies as possible. This is a major factor in distinguishing systematic reviews from traditional narrative reviews and helps to minimize bias and therefore assist in achieving reliable estimates of effects. [1]

Cochrane Website: What are systematic reviews?


[1] Lefebvre C, Manheimer E, Glanville J. Chapter 6: Searching for studies.  In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from  http://handbook-5-1.cochrane.org

What is a Scoping review?

Scoping reviews are frequently undertaken to determine the feasibility of more specific research questions. The purpose of scoping reviews has been presented as follows:

  • To examine the extent, range and nature of research activity
  • To determine the value of undertaking a full systematic review
  • To summarise and disseminate research findings
  • To identify research gaps in the existing literature[1]

A systematic review might typically focus on a well-defined question where appropriate study designs can be identified in advance;  a scoping study might discuss broader topics where many different study designs might be applicable. A scoping study is less likely to seek to address specific research questions in detail and will not usually assess all aspects of  the quality of data or evidence.

Approaches to scoping reviews

The steps in conducting a scoping study are similar to many review types while maintaining broad and general perspectives:

  • Identify the research questions: what domains and time periods need to be explored?
  • Find the relevant studies through a structured literature search of electronic databases, reference searching and grey literature.  This will involve using the standard systematic review search methods.
  • Scan, select and analyse  the relevant  studies.
  • Collate, summarize and report the results in a descriptive or narrative style.
  • Optional consultation with stakeholders and experts as appropriate.

Scoping reviews are not necessarily quick or rapid alternatives to systematic reviews.  In certain cases, as in BEME (Best Evidence Medical Education) reviews for instance, a scoping evaluation is a required preliminary to the main study.  Other factors to consider include:

  • The project timeline may be equivalent to a systematic review (12 months or more)
  • There may be a number of iterative phases
  • Multiple or broader questions may produce more citations and references for scanning
  • Reporting and write up requirement are as rigorous as any study type

To find examples of published scoping reviews in Medline simply search  “scoping review” and combine with any topic of your choice.

Resources to consult on scoping reviews.

  • Arksey H. and O'Malley L. Scoping Studies: towards a methodological framework.  International Journal of Social Research Methodology, 2005;8(1):19-32.  https://doi.org/10.1080/1364557032000119616

  • Colquhoun HL, Levac D, O'Brien KK et al  Scoping reviews: time for clarity in definition, methods, and reporting.  J Clin Epidemiol. 2014 Dec;67(12):1291-4. https://doi.org/10.1016/j.jclinepi.2014.03.013

  • Peters MDJ, Godfrey C, McInerney P, Baldini Soares C, Khalil H, Parker D. Chapter 11: Scoping Reviews. In: Aromataris E, Munn Z (Editors). Joanna Briggs Institute Reviewer's Manual. The Joanna Briggs Institute, 2017. Available from https://reviewersmanual.joannabriggs.org/

  • Munn Z, Peters M, Stern C, et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach.  BMC Medical Research Methodology. 2018;18:143. Available from: https://doi.org/10.1186/s12874-018-0611-x

  • Peters M, Godfrey C, Khalil H, et al. Guidance for Conducting Systematic Scoping Reviews.  Int J Evid Based Healthc. 2015;13:141-146. https://doi.org/10.1097/XEB.0000000000000050

  • Tricco AC, Lillie E, Zarin W, et al . PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018. https://doi.org/10.7326/M18-0850


[1] Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a methodological framework. International Journal of Social Research Methodology: Theory & Practice, 8(1), 19-32. Available from: http://eprints.whiterose.ac.uk/1618/1/Scopingstudies.pdf

What is a Rapid Review?

 “Rapid reviews are a form of evidence synthesis that may provide more timely information for decision making compared with standard systematic reviews. Systematic reviews are defined as “a review of a clearly formulated question(s) that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from the studies that are included in the review.”[1]

 “Rapid reviews are literature reviews that use methods to accelerate or streamline traditional systematic review processes in order to meet the needs and timelines of the end-users (e.g., government policymakers, health care institutions, health professionals, and patient associations) .”[2]

Rapid reviews target high quality and authoritative resources for time-critical decision-making or clinically urgent questions. Like a systematic reviews they aim to identify the key concepts, theories and resources in a field, and to survey the major research studies. Less time may be spent on critical appraisal as systematic reviews, evidence briefs and clinical guidelines are sought in preference to exhaustive coverage of primary studies. The objective is to apply systematic levels of search and appraisal but within shorter timeframes.

The methods of conducting rapid reviews varies widely, and are typically done in less than 5 weeks.  Often policy makers require a short deadline and a systematic review for synthesizing the evidence is not practical.  A rapid review speeds up the systematic review process by omitting stages of the systematic review making it less rigorous. 

Rapid reviews are best designed for: broader PICO questions, new or emerging research topics, updates of previous reviews, critical topics or to assess what is already known about a policy.
 

‚ÄčThe World Health Organization presents a wide ranging overview in the following guide:

To find examples of published rapid reviews search in Medline search “rapid review” and any subject term  -eg- “women’s health”
 

Resources to consult on rapid reviews

  • Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012). Evidence summaries: the evolution of a rapid review approach. Systematic reviews  1(1), 1. https://doi.org/10.1186/2046-4053-1-10
  • Hartling L, Guise JM, Hempel S, et al.  EPC Methods: AHRQ End-User Perspectives of Rapid Reviews. Rockville (MD): Agency for Healthcare Research and Quality (US); 2016 Apr.  Available from: https://www.ncbi.nlm.nih.gov/books/NBK362003/
  • Kelly, SE., Moher, D., Clifford, TJ.   Quality of conduct and reporting in rapid reviews: an exploration of compliance with PRISMA and AMSTAR guidelines. Systematic Reviews 2016 5:79  https://doi.org/10.1186/s13643-016-0258-9
  • Tricco AC, Antony J, Zarin W, et al. A scoping review of rapid review methods.  BMC Medicine. 2015; 13:224. 
  • https://doi.org/10.1186/s12916-015-0465-6

[1] Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions Version 5.1. 0. Cochrane Collaboration; 2012. [updated March 2011] Cochrane Collaboration.  Available from: https://handbook-5-1.cochrane.org/

[2] Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and implications of rapid reviews. Implementation Science. 2010; 5(1):56.   Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2914085/

Where to Find Guidance

Centre for Reviews and Dissemination (CRD), University of York.

CRD has produced a complete guide to writing up systematic reviews[1]. It covers core principles and methods including how to conduct a review, how to identify studies, the basics of searching databases and formulating search strategies and how to document and report searches. View and download the guide here:  https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf

Cochrane Collaboration

Essential if you are carrying out a Cochrane systematic review, and an excellent guideline for other reviewers.

Cochrane Handbook for Systematic Reviews of Interventions1 

Chapter 6: Searching for Studies

Lists of sources – including databases, trials registers

Search hints and tips

RCT filters for Medline

http://handbook-5-1.cochrane.org/

From September 2018 revised chapters of Version 6 of the Cochrane Handbook, due to be fully published in 2019, are freely available to all those who personally register for Cochrane Training accounts.  Go to https://training.cochrane.org/  to access the latest guides and the most recent handbook material.

The MECIR (Methodological Expectations of Cochrane Intervention Reviews) project[2]

The MECIR project has drawn up standards for the conduct and the reporting of Cochrane Intervention Reviews.  Very useful to follow even if you are not doing a Cochrane Review – use as a checklist. 

Each standard is either mandatory or highly desirable and the exclusion should be justified. A rationale and elaboration of each standard is included as well as a reference to the relevant section of the Cochrane Handbook. 

Methodological Expectations of Cochrane Intervention Reviews. Cochrane: London, Version 1.02. Updated Jan 2018

Items C24-C38 cover standards for searching for studies.   

Item C24              Searching key databases

Item C25              Searching specialist bibliographic databases

Item C26              Searching for different types of evidence

Item C27              Searching trials registers

Item C28              Searching for grey literature

Item C29              Searching within other reviews

Item C30              Searching reference lists

Item C31              Searching by contacting relevant individuals and organisations

Item C32              Structuring search strategies for bibliographic databases

Item C33              Developing search strategies for bibliographic databases

Item C34              Using search filters

Item C35              Restricting database searches

Item C36              Documenting the search process

Item C37              Rerunning searches

Item C38              Incorporating findings from rerun searches

The online version of the MECIR standards are continually updated and available within the revised Cochrane Community websites. https://community.cochrane.org/mecir-manual


[1] Systematic Reviews: CRD’s guidance for undertaking reviews in health care.  Centre for Reviews and Dissemination, University of York, 2008. Available from https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf

[2] Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews. Cochrane: London, Version 1.02. 2016. Updated Jan 2018.  Available from:  https://community.cochrane.org/sites/default/files/uploads/MECIR%20PRINTED%20BOOKLET%20FINAL%20v1.02.pdf