Skip to main content

Table 1 Phase III item-reduced checklist assessed using the within-Task Force pilot

From: The REal Life EVidence AssessmeNt Tool (RELEVANT): development of a novel quality assurance asset to rate observational comparative effectiveness research studies

Primary or secondary

Items

Score 1 = ”yes”; 0 = ”no”

Reviewer comments

Background/relevance: 3 items (1 primary, 2 secondary)

Primary

1. Clear underlying hypotheses and specific research questions

  

Secondary

2. Relevant population and setting

  

Secondary

3. Relevant interventions and outcomes are included

  

Design 4 items: (2 primary, 2 secondary)

Primary

1. Evidence of a priori protocol, review of analyses, statistical analysis plan, and interpretation of results

  

Primary

2. Comparison groups justified

  

Secondary

3. Registration in a public repository with commitment to publish results

  

Secondary

4. Data sources that are sufficient to support the studya

  

Measures: 4 items (2 primary, 2 secondary)

Primary

1. Was exposure clearly defined, measured and (relevance) justifiedb

  

Primary

2. Primary outcomes defined, measured and (relevance) justifiedb

  

Secondary

3. Length of observation: Sufficient follow up duration to reliably assess outcomes of interest and long-term treatment effects

  

Secondary

4. Sample size: calculated based on clear a priori hypotheses regarding the occurrence of outcomes of interest and target effect of studied treatment versus comparator

  

Analyses 3 items (1 primary, 2 secondary)

Primary

1. Thorough assessment of and mitigation strategy for potential confounders

  

Secondary

2. Study groups are compared at baseline and analyses of subgroups or interaction effects reported

  

Secondary

3. Sensitivity analyses are performed to check the robustness of results and the effects of key assumptions on definitions or outcomes

  

Results/reporting: 6 items (2 primary, 4 secondary)

Primary

1. Extensive presentation of results/authors describe the key components of their statistical approachesa

  

Primary

2. Were confounder-adjusted estimates of treatment effects reportedb

  

Secondary

3. Flow chart explaining all exclusions and individuals screened or selected at each stage of defining the final sample

  

Secondary

4. Was follow-up similar or accounted for between groups

  

Secondary

5. Did the authors describe the statistical uncertainty of their findings

  

Secondary

6. Was the extent of missing data reported

  

Discussion/interpretation: 4 items (2 primary, 2 Secondary)

Primary

1. Results consistent with known information or if not, was an explanation provided

  

Primary

2. Are the observed treatment effects considered clinically meaningful

  

Secondary

3. Discussion of possible biases and confounding factors, especially related to the observational nature of the study

  

Secondary

4. Suggestions for future research to challenge, strengthen, or extend the study results

  

Conflict of interest: (1 primary item)

Primary

1. Potential conflicts of interest, including study funding, were stated

  
  1. aFor a retrospective design, answering “no” to this item may suggest a “fatal flaw” using the methodology developed by ISPOR14
  2. bFor any study design, answering “no” to this item may suggest a “fatal flaw” using the methodology developed by ISPOR14