Skip to main content

Table 1 Phase III item-reduced checklist assessed using the within-Task Force pilot

From: The REal Life EVidence AssessmeNt Tool (RELEVANT): development of a novel quality assurance asset to rate observational comparative effectiveness research studies

Primary or secondaryItemsScore 1 = ”yes”; 0 = ”no”Reviewer comments
Background/relevance: 3 items (1 primary, 2 secondary)
Primary1. Clear underlying hypotheses and specific research questions  
Secondary2. Relevant population and setting  
Secondary3. Relevant interventions and outcomes are included  
Design 4 items: (2 primary, 2 secondary)
Primary1. Evidence of a priori protocol, review of analyses, statistical analysis plan, and interpretation of results  
Primary2. Comparison groups justified  
Secondary3. Registration in a public repository with commitment to publish results  
Secondary4. Data sources that are sufficient to support the studya  
Measures: 4 items (2 primary, 2 secondary)
Primary1. Was exposure clearly defined, measured and (relevance) justifiedb  
Primary2. Primary outcomes defined, measured and (relevance) justifiedb  
Secondary3. Length of observation: Sufficient follow up duration to reliably assess outcomes of interest and long-term treatment effects  
Secondary4. Sample size: calculated based on clear a priori hypotheses regarding the occurrence of outcomes of interest and target effect of studied treatment versus comparator  
Analyses 3 items (1 primary, 2 secondary)
Primary1. Thorough assessment of and mitigation strategy for potential confounders  
Secondary2. Study groups are compared at baseline and analyses of subgroups or interaction effects reported  
Secondary3. Sensitivity analyses are performed to check the robustness of results and the effects of key assumptions on definitions or outcomes  
Results/reporting: 6 items (2 primary, 4 secondary)
Primary1. Extensive presentation of results/authors describe the key components of their statistical approachesa  
Primary2. Were confounder-adjusted estimates of treatment effects reportedb  
Secondary3. Flow chart explaining all exclusions and individuals screened or selected at each stage of defining the final sample  
Secondary4. Was follow-up similar or accounted for between groups  
Secondary5. Did the authors describe the statistical uncertainty of their findings  
Secondary6. Was the extent of missing data reported  
Discussion/interpretation: 4 items (2 primary, 2 Secondary)
Primary1. Results consistent with known information or if not, was an explanation provided  
Primary2. Are the observed treatment effects considered clinically meaningful  
Secondary3. Discussion of possible biases and confounding factors, especially related to the observational nature of the study  
Secondary4. Suggestions for future research to challenge, strengthen, or extend the study results  
Conflict of interest: (1 primary item)
Primary1. Potential conflicts of interest, including study funding, were stated  
  1. aFor a retrospective design, answering “no” to this item may suggest a “fatal flaw” using the methodology developed by ISPOR14
  2. bFor any study design, answering “no” to this item may suggest a “fatal flaw” using the methodology developed by ISPOR14