The state of asthma epidemiology: an overview of systematic reviews and their quality

Background Recently, we have published an overview of systematic reviews in allergy epidemiology and identified asthma as the most commonly reviewed allergic disease. Building on this work, we aimed to investigate the quality of systematic reviews in asthma using the AMSTAR checklist and to provide a reference for future, more in-depth assessment of the extent of previous knowledge. Methods We included all 307 systematic reviews indexed with asthma, including occupational asthma, and/or wheeze from our previous search in PubMed and EMBASE up to December 2014 for systematic reviews on epidemiological research on allergic diseases. Topics of the included systematic reviews were indexed and we applied the AMSTAR checklist for methodological quality to all. Statistical analyses include description of lower and upper bounds of AMSTAR scores and variation across publication time and topics. Results Of 43 topics catalogued, family history, birth weight, and feeding of formula were only covered once in systematic reviews published from 2011 onwards. Overall, at least one meta-analysis was conducted for all topics except for “social determinants”, “perinatal”, “birth weight”, and “climate”. AMSTAR quality scores were significantly higher in more recently published systematic reviews, in those with meta-analysis, and in Cochrane reviews. There was evidence of variation of quality across topics even, after accounting for these characteristics. Genetic factors in asthma development were often covered by systematic reviews with some evidence of unsubstantiated updates or repetition. Conclusions We present a comprehensive overview with an indexed database of published systematic reviews in asthma epidemiology including quality scores. We highlight some topics including active smoking and pets, which should be considered for future systematic reviews. We propose that our search strategy and database could be a basis for topic-specific overviews of systematic reviews in asthma epidemiology. Electronic supplementary material The online version of this article (doi:10.1186/s13601-017-0146-y) contains supplementary material, which is available to authorized users.


AMSTAR item liberal instruction conservative instruction 4. Was the status of publication (i.e. grey literature) used as an inclusion criterion?
The authors should state that they searched for reports regardless of their publication type. The authors should state whether or not they excluded any reports (from the systematic review), based on their publication status, language etc.
Note: If review indicates that there was a search for "grey literature" or "unpublished literature," indicate "yes." SINGLE database, dissertations, conference proceedings, and trial registries are all considered grey for this purpose. If searching a source that contains both grey and non-grey, must specify that they were searching for grey/unpublished lit. as originally described as originally described

Was a list of studies (included and excluded) provided?
A list of included and excluded studies should be provided.
Note: Acceptable if the excluded studies are referenced. If there is an electronic link to the list but the link is dead, select "no." sufficient if some excluded studies were referenced in an exemplary manner or if number of excluded articles per exclusion criterion were given only if excluded studies were referenced completely for at least some exclusion criteria

Were the characteristics of the included studies provided?
In an aggregated form such as a table, data from the original studies should be provided on the participants, interventions and outcomes. The ranges of characteristics in all the studies analyzed e.g., age, race, sex, relevant socioeconomic data, disease status, duration, severity, or other diseases should be reported.
Note: Acceptable if not in table format as long as they are described as above.
as originally described as originally described AMSTAR item liberal instruction conservative instruction 7. Was the scientific quality of the included studies assessed and documented? 'A priori' methods of assessment should be provided (e.g., for effectiveness studies if the author(s) chose to include only randomized, double-blind, placebo controlled studies, or allocation concealment as inclusion criteria); for other types of studies alternative items will be relevant.
Note: Can include use of a quality scoring tool or checklist, e.g., Jadad scale, risk of bias, sensitivity analysis, etc., or a description of quality items, with some kind of result for EACH study ("low" or "high" is fine, as long as it is clear which studies scored "low" and which scored "high"; a summary score/range for all studies is not acceptable).
sufficient if the quality of the included studies was assessed but not displayed for each study only if quality was assessed by a dedicated scale or restrictions or extracted items were defined for the purpose of quality assessment and if the results of quality assessment were displayed for each study

Was the scientific quality of the included studies used appropriately in formulating conclusions?
The results of the methodological rigor and scientific quality should be considered in the analysis and the conclusions of the review, and explicitly stated in formulating recommendations.
Note: Might say something such as "the results should be interpreted with caution due to poor quality of included studies." Cannot score "yes" for this question if scored "no" for question 7.
as originally described as originally described AMSTAR item liberal instruction conservative instruction 9. Were the methods used to combine the findings of studies appropriate? For the pooled results, a test should be done to ensure the studies were combinable, to assess their homogeneity (i.e., Chi-squared test for homogeneity, I2). If heterogeneity exists a random effects model should be used and/or the clinical appropriateness of combining should be taken into consideration (i.e., is it sensible to combine?).
Note: Indicate "yes" if they mention or describe heterogeneity, i.e., if they explain that they cannot pool because of heterogeneity/variability between interventions.
as originally described as originally described

Was the likelihood of publication bias assessed?
An assessment of publication bias should include a combination of graphical aids (e.g., funnel plot, other available tests) and/or statistical tests (e.g., Egger regression test, Hedges-Olken).
Note: If no test values or funnel plot included, score "no". Score "yes" if mentions that publication bias could not be assessed because there were fewer than 10 included studies.
as originally described as originally described

Was the conflict of interest included?
Potential sources of support should be clearly acknowledged in both the systematic review and the included studies.

Note: To get a "yes," must indicate source of funding or support for the systematic review AND for each of the included studies.
sufficient if conflict of interest or source of funding was stated for the systematic review only if (i) conflict of interest or source of funding was stated for the systematic review and (ii) conflict of interest or source of funding was stated within the systematic review for each included study the latter criterion was already satisfied if the risk of selective reporting was judged (e.g. in Cochrane reviews)  (8)) was defined to allow for meaningful visualisation along with the systematic review's AMSTAR quality scores (see Figure 3 and Figure S1).
3 Systematic reviews which present data on immunotherapy preventing the onset of subsequent allergic disease (n=2).
4 Systematic review on area-based and individual measures of social disadvantage (n=1).
5 Systematic reviews of articles on comorbidities were included as potential risk factors for allergic disease if they displayed data suggestive of the comorbid disease existing prior to the onset of the allergic disease.