Skip to main content

Table 2 Extended Pilot Item level agreement summary

From: The REal Life EVidence AssessmeNt Tool (RELEVANT): development of a novel quality assurance asset to rate observational comparative effectiveness research studies

 Group A
Group B
Group C
a: Agreement across primary sub-items
1. Background1.1. Clearly stated research question7910086
2. Design2.1. Population defined and justified649471
2.2. Comparison groups defined and justified937179
2.3. Setting defined and justified9310093
3. Measures3.1. (If relevant), exposure is clearly defined937176
3.2. Primary outcomes clearly defined and measured718993
4. Analysis4.1. Potential confounders are considered and adjusted for in the analysis, and reported648171
4.2. Study groups are compared at baseline797979
5. Results5.1. Results are clearly presented for all primary and secondary endpoints as well as confounders799471
6. Discussion/interpretation6.1. Results consistent with known information or if not, an explanation is provided8610086
6.2. The clinical relevance of the results is discussed858893
7. Conflict of interests7.1. Potential conflicts of interest, including study funding, are stated7910093
b: Agreement across secondary quality sub-items
1. Background1.1. The research is based on a review of the background literature (ideal standard is a systematic review, but minimally citation of multiple [≥ 1] references in the introduction88100100
2. Design2.1. Clear written evidence of a priori protocol development and registration (e.g. in ENCePP or online registries) and a priori statistical analysis plan1007383
2.2. The data source (or database), as described, contains adequate exposures (if relevant) and outcome variables to answer the research question88100100
3. Measures3.1. Sample size justifies the inclusion criteria and follow-up period for the primary outcome887188
4. AnalysisNo secondary itemNANANA
5. Results5.1. Flow chart explaining all exclusions and individuals screened or selected at each stage of defining the final sample1008375
5.2. Was follow-up similar or accounted for between groups (i.e. no unexplained differential loss to follow up)1005558
5.3. The authors describe the statistical uncertainty of their findings (e.g. p-values, confidence intervals)10010083
5.4. The extent of missing data is reported7588100
6. Discussion/interpretation6.1. Possible biases and/or confounding factors described10010083
6.2. Suggestions for future research provided (e.g. to challenge, strengthen, or extend the study results)1006358
7. Conflict of interestsNo secondary itemNANANA