Systematic reviews

Developing a research question

Systematic reviews attempt to gather all relevant studies that address the same research question. Developing a specific research question along with inclusion and exclusion criteria is essential. Doing so will help identify key concepts that will be used in a search strategy. 

A research question should be open-ended, comprehensive and specific. It can include elements found in different frameworks and tools. PICO is a frequently used approach when devising research questions in the fields of health and medicine.

The following examples of research questions are then mapped out using PICO.

  • Example 1 = Is hand washing among healthcare workers effective in reducing hospital acquired infections?
  • Example 2 = Which form of sexual education program is most effective among high school students?
  Example 1 Example 2
Population or Problem hospital acquired infections high school students
Intervention hand washing sexual education programs
Comparison no hand washing, gloves, hand sanitizers no program, abstinence programs
Outcome reduced infection increased knowledge, attitude changes, rates of sexual activity, rates of teen pregnancy

In addition to a specific research question, a systematic review requires a list of inclusion and exclusion criteria. This list will be used when screening results. Criteria could include populations, types of publication, language, years, types of study, sample size and more.

Feel free to use a search strategy worksheet to start preparing your review. This worksheet helps identify the research question, relevant concepts along with useful synonyms.

Searching for systematic reviews

Searching for previously published systematic reviews serves two purposes. First, it verifies that a current and comprehensive review that addresses the same question has not been published recently. Second, it identifies search strategies that have been used previously. These strategies can provide useful information such as keywords, subject headings, operators, limits, databases and other resources that could be reused if appropriate.

If a published review focuses on only one of the key concepts of your research question, you can nevertheless draw on its strategies to assist in developing your own.

Resources for finding systematic reviews

Assembling a review group

Systematic reviews are often conducted by review teams that include:

  • Principal investigator - leads all phases of the project
  • Content expert - assists with the screening and reviewing process
  • Graduate student - can assist with searching, screening, evaluating
  • Subject matter expert - provides advice about search strategy, sources and key authors
  • Librarian - develops and runs searches, manages references, contributes to the methodology
  • Statistician - analyzes data gathered from included studies in a meta-analysis

Consideration should be given to establishing a team prior to conducting the review.

Writing a protocol

A systematic review protocol describes the rationale for the review and its objectives along with the research question it seeks to address. It also describes the methodology, including the search strategy, that will be used to locate, select and critically appraise studies. Finally, it explains how data from the included studies will be collected and analyzed. Writing a protocol keeps the review on track and helps reflect on any underlying assumptions about the topic while justifying and defending various decisions that were made.

Protocol Standards
The Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols, or PRISMA-P, was developed "to facilitate the development and reporting of systematic review protocols" in health and medicine. It includes a detailed, 17-point checklist that can be used when writing a protocol. Researchers producing health-related reviews can register their protocols using PROSPERO.

Biology and environmental studies have standards called RepOrting standards for Systematic Evidence Syntheses (ROSES), which include forms for protocols and systematic reviews

Here are a few examples of registered or published protocols:

Various templates are available and can be used to help prepare a protocol:

Registering a protocol

Protocol registration for systematic reviews is crucial because it enhances transparency, reduces the risk of bias, and promotes methodological rigor. By publicly documenting the review process and predefined methods before the review is conducted, it prevents selective reporting of outcomes and analytical approaches. This preemptive declaration of intentions ensures that the review adheres to a planned strategy, which fosters credibility and reproducibility. Additionally, protocol registration facilitates peer scrutiny and allows researchers to identify potential duplications in research efforts, ultimately contributing to a more efficient and trustworthy evidence base in the scientific community.

Registering a protocol for a systematic review involves several key steps:

  1. Choose a Registry: Select an appropriate registry for your protocol. Common registries include PROSPERO (International Prospective Register of Systematic Reviews) and the Open Science Framework (OSF). Each registry has specific requirements and guidelines.

  2. Prepare Your Protocol: Develop a detailed protocol that outlines your systematic review's objectives, methods, and analysis plans. Key elements to include are:

    • Title of the review
    • Background and rationale
    • Review question(s) and objectives
    • Eligibility criteria (e.g., types of studies, participants, interventions, and outcomes)
    • Information sources and search strategy
    • Data extraction and management plans
    • Risk of bias assessment
    • Data synthesis and analysis methods
  3. Create an Account: Register an account with the chosen registry if you do not already have one. This typically involves providing some personal information and agreeing to the terms of service.

  4. Submit Your Protocol: Follow the registry's submission process. This often involves filling out an online form with sections for each part of your protocol. Some registries may allow you to upload a document instead.

  5. Review and Approval: After submission, the registry may review your protocol for completeness and clarity. This step may involve communication with the registry staff to address any issues or questions.

  6. Receive Registration Number: Once approved, your protocol will be assigned a unique registration number and made publicly available. This number can be cited in your subsequent publications to link back to the registered protocol.

  7. Update as Necessary: If there are any changes to your review methods or scope during the course of your research, update the protocol on the registry to reflect these changes and maintain transparency.

By following these steps, you can ensure that your systematic review protocol is properly registered, promoting transparency and integrity in your research process.