Table 2 Extended Pilot Item level agreement summary
Group A (%) | Group B (%) | Group C (%) | ||
|---|---|---|---|---|
a: Agreement across primary sub-items | ||||
1. Background | 1.1. Clearly stated research question | 79 | 100 | 86 |
2. Design | 2.1. Population defined and justified | 64 | 94 | 71 |
2.2. Comparison groups defined and justified | 93 | 71 | 79 | |
2.3. Setting defined and justified | 93 | 100 | 93 | |
3. Measures | 3.1. (If relevant), exposure is clearly defined | 93 | 71 | 76 |
3.2. Primary outcomes clearly defined and measured | 71 | 89 | 93 | |
4. Analysis | 4.1. Potential confounders are considered and adjusted for in the analysis, and reported | 64 | 81 | 71 |
4.2. Study groups are compared at baseline | 79 | 79 | 79 | |
5. Results | 5.1. Results are clearly presented for all primary and secondary endpoints as well as confounders | 79 | 94 | 71 |
6. Discussion/interpretation | 6.1. Results consistent with known information or if not, an explanation is provided | 86 | 100 | 86 |
6.2. The clinical relevance of the results is discussed | 85 | 88 | 93 | |
7. Conflict of interests | 7.1. Potential conflicts of interest, including study funding, are stated | 79 | 100 | 93 |
b: Agreement across secondary quality sub-items | ||||
1. Background | 1.1. The research is based on a review of the background literature (ideal standard is a systematic review, but minimally citation of multiple [≥ 1] references in the introduction | 88 | 100 | 100 |
2. Design | 2.1. Clear written evidence of a priori protocol development and registration (e.g. in ENCePP or Clinicaltrials.gov online registries) and a priori statistical analysis plan | 100 | 73 | 83 |
2.2. The data source (or database), as described, contains adequate exposures (if relevant) and outcome variables to answer the research question | 88 | 100 | 100 | |
3. Measures | 3.1. Sample size justifies the inclusion criteria and follow-up period for the primary outcome | 88 | 71 | 88 |
4. Analysis | No secondary item | NA | NA | NA |
5. Results | 5.1. Flow chart explaining all exclusions and individuals screened or selected at each stage of defining the final sample | 100 | 83 | 75 |
5.2. Was follow-up similar or accounted for between groups (i.e. no unexplained differential loss to follow up) | 100 | 55 | 58 | |
5.3. The authors describe the statistical uncertainty of their findings (e.g. p-values, confidence intervals) | 100 | 100 | 83 | |
5.4. The extent of missing data is reported | 75 | 88 | 100 | |
6. Discussion/interpretation | 6.1. Possible biases and/or confounding factors described | 100 | 100 | 83 |
6.2. Suggestions for future research provided (e.g. to challenge, strengthen, or extend the study results) | 100 | 63 | 58 | |
7. Conflict of interests | No secondary item | NA | NA | NA |