- Open Access
Quality indicators for the acute and long-term management of anaphylaxis: a systematic review
Clinical and Translational Allergyvolume 7, Article number: 15 (2017)
The quality of acute and long-term anaphylaxis management is variable and this contributes to the poor outcomes experienced by many patients. Clinical practice guidelines have the potential to improve outcomes, but implementing guideline recommendations in routine practice is challenging. Quality indicators have the potential to support guideline implementation efforts.
To identify quality indicators to support the acute and long-term management of anaphylaxis.
We conducted a systematic review of the literature that involved searching Medline, EMBASE and CINAHL databases for peer-reviewed published literature for the period 1 January 2005–31 December 2015. Additionally we searched Google for grey and unpublished literature. The identified indicators were descriptively summarized against the most recent international anaphylaxis guidelines (i.e. those produced by the European Academy of Allergy and Clinical Immunology) and critically evaluated using the Agency for Healthcare Research and Quality’s criteria for indicator development.
Our searches revealed 830 publications, from which we identified five sources for 54 indicators addressing both acute (n = 27) and long-term (n = 27) management of anaphylaxis. The majority of indicators were developed through expert consensus with relatively few of these having been formally piloted or tested to demonstrate that they could discriminate between variations in practice and/or that they were sensitive to change.
There is a need for a comprehensive set of quality indicators for anaphylaxis management. We have however identified some indicators for the acute and long-term management of anaphylaxis that could with relatively little additional work support efforts to translate guideline recommendations into clinical care.
Anaphylaxis is a “severe, life-threatening generalized or systemic hypersensitivity reaction” [1, 2] that is responsible for considerable morbidity and, in some cases, mortality. The quality of emergency and ongoing care for patients experiencing and/or with a history of anaphylaxis is variable and this contributes to the poor outcomes (e.g. high risk of recurrent episodes of anaphylaxis) seen . In an attempt to standardize care, and thereby improve outcomes, a number of governments and professional bodies have developed clinical practice guidelines [4,5,6,7]. These aim to provide front-line clinicians with simple, concise, evidence-based recommendations for clinical care. Whilst undoubtedly a welcome development, there is a growing body of evidence demonstrating that guidelines often prove challenging to implement in routine clinical care . To support this implementation process, attention is increasingly focusing on the need to develop tools that can help clinicians implement key recommendations and monitor progress with implementation efforts .
Quality standards and indicators are potentially important tools designed to help clinicians and healthcare organisations assess the quality of care being provided against agreed evidence-based recommendations . These are now being used across a number of disease and clinical areas, but we are unaware of these currently being routinely used at scale in relation to anaphylaxis.
We are developing evidence-based tools to support translation of key anaphylaxis recommendations into clinical practice and in order to inform this process we undertook a systematic review to identify existing quality indicators for anaphylaxis and identify gaps where there is a need for further development.
Overview of methods, registration and reporting
We conducted a systematic review of the literature that involved searching for published and unpublished literature. It is registered in the PROSPERO database with registration number CRD42016035381. We reported findings using the principles advocated in the PRISMA guidelines  (Additional file 1).
We developed a highly sensitive search strategy to identify papers on standards and/or quality indicators for anaphylaxis. This involved searching Medline, EMBASE and CINAHL databases for peer-reviewed published literature, and the Google database for searching grey literature published during the period 1 January 2005–31 December 2015. No language restrictions were employed. Our search terms are detailed in the Appendix.
We were interested in publications reporting on indicators for measuring the quality of acute and long-term care of anaphylaxis in patients of any age. We did not specify any criteria on how these were developed and there was therefore no study filter employed in selecting papers.
Selection of indicators
Two reviewers independently selected manuscripts against the pre-specified inclusion criteria. Disagreements were resolved through discussion with arbitration by a third reviewer, where necessary.
Two reviewers independently extracted indicator data onto a customized data extraction sheet. Disagreements were resolved through discussion; a third reviewer arbitrated in instances where agreement could not be reached. Where available, we also extracted data on how these indicators were developed, whether they had been tested and if they had been used in experimental contexts to demonstrate that they could capture improvements in the quality of care.
Quality assessment of indicators
The quality of these indicators was then assessed against the criteria detailed using the four stage quality indicator process recommended by the Agency for Healthcare Research and Quality (AHRQ), namely:
Development: Identifying candidate indicators through a literature review and/or discussion with experts;
Implementation: Testing of candidate indicators, introducing them into software etc.;
Maintenance: Indicators need to be regularly checked and, if necessary, updated to keep abreast of latest developments; and
Retirement processes: Indicators need to be assessed at periodic intervals for relevance and in order to assess if they need to be discontinued .
We contacted the authors of these development tools for further clarification, if necessary.
We then mapped available indicators against the various recommendations in the most recent international anaphylaxis guidelines, namely those produced by the European Academy of Allergy and Clinical Immunology (EAACI) , identifying areas of overlap and gaps, and making an overall assessment of whether any particular indicator was considered appropriate for use in routine clinical practice. Available indicators were traffic-light color coded with green indicating that the indicators were suitable/nearly suitable for routine use as they had undergone the AHRQ process, amber indicating the need for some additional work, and red indicating the need for a substantial amount of additional underpinning work as most of the stages suggested by AHRQ had not been followed.
Characteristics of included studies
Our searches identified 830 studies, of which five satisfied our inclusion criteria (see Fig. 1) [12,13,14,15,16]. The five sources of indicators are detailed in Table 1. In total, 54 individual indicators were identified: 27 for the acute management of anaphylaxis and the remaining 27 for longer-term management. Indicators for the acute and longer-term management of anaphylaxis were identified by four of the five sources [12, 14,15,16]. Two sources of indicators only focused on children and young people [16, 17], and one focused solely on children attending Emergency Departments (ED) for the acute management of anaphylaxis .
Assessment of indicators against AHRQ criteria
Table 2 summarizes our assessment of the quality of the indicators against each of the four criteria stipulated by AHRQ.
1. Measure development
The EAACI indicators  were derived from clinical guidelines in relation to key recommendations. The Levy indicators  were developed through expert consensus. The National Institute of Health and Clinical Excellence (NICE) indicators were derived from relevant guideline recommendations . The Royal College of Paediatrics and Child Health (RCPCH) indicators were derived from a care pathway for children with suspected anaphylaxis . The Stang indicators  were the only ones that had been developed through the stages suggested by AHRQ, namely formal processes to identify and assess indicators; furthermore, these were developed using National Quality Framework (NQF) measure evaluation criteria .
The EAACI indicators  did not have any formal implementation assessment. The Levy indicators  are freely available for use from http://www.guideline-audit.com/adrenaline/audit_specification.php and had been successfully implemented in a number of UK general practices with the opportunity for benchmarking quality of care. NICE  had a generic implementation team and created a range of implementation tools, but it was unclear if the ability to implement these indicators in practice had been formally assessed. The RCPCH  give no mention of an implementation strategy. The Stang indicators were operationalized and tested in an ED setting .
None of the indicators had plans for formal maintenance checks.
There were no plans for retirement of indicators, although EAACI , NICE  and the RCPCH  stated that they had established processes for the periodic review of their clinical guidelines/pathways.
Mapping of indicators against guideline recommendations
The EAACI Guidelines  made 16 recommendations on the acute management of anaphylaxis and indicators were developed by EAACI for all of these recommendations (Table 3). Six of these recommendations also had indicators identified from the other sources.
For the longer-term management of anaphylaxis, EAACI made eight recommendations and indicators were developed by EAACI for all of these (Table 4). Additional indicators from other sources were identified for five of these recommendations.
Tables 3 and 4 have been colour coded according to their compliance with the AHRQ criteria to show which indicators are fit-for-purpose. Green identifies indicators that have been developed according to AHRQ principles and are ready to implement, red shows the indicators that need more developmental work before they can be implemented and amber falls between the two. These tables show that the Stang et al.  (coded green) and Levy  (coded amber) indicators could with relatively modest effort be rendered fit-for-purpose; gaps still however remained in relation to both acute and long-term management (coded red) where considerable development work is still required.
Statement of principal findings
This study has demonstrated that there are now candidate quality indicators covering many aspects of the acute and long-term management of anaphylaxis. Only a few of these have however undergone the four stages of development recommended by AHRQ, namely implementation and maintenance and none of them have considered decisions on the maintenance or retirement of quality indicators . Further work is therefore needed before any of these can be recommended for routine use in clinical practice . That said, the indicators developed by Stang et al.  for acute management of anaphylaxis and those developed by Levy  for long-term management could be rendered fit-for-purpose with relatively modest additional effort. EAACI should therefore consider undertaking this work and adopting these indicators. Other areas in relation to both acute and long-term management require much more development work and evaluation.
Strengths and limitations
The key strengths of this work are that we used systematic review methods to identify relevant literature, formally considered the appropriateness of the methods to develop and deploy these indicators using the four stage process recommended by the AHRQ  and then systematically mapped these against the recent EAACI anaphylaxis guidelines .
The limitations of this work also need to be considered. This includes the possibility that we failed to identify relevant literature and indicators, although we tried to minimize this risk by not having any restriction of languages on our searches, searching grey literature and by contacting a panel of experts. There may also have been experiences of using these indicators that have not yet found their way into the peer-reviewed or grey literature. This issue could be further investigated through, for example, contacting electronic health record and software vendors to see which if any have been computed and with what results.
Interpretation in the light of other published literature
Anaphylaxis, in comparison to other disease areas, is relatively undeveloped in terms of quality indicators . For example, NICE has developed indicators for a number of disorders—particularly long-term conditions—that have been used to incentivize improvements in care through the UK Quality and Outcomes Framework (QOF) [19, 20]. Examples of areas in which these have been used include asthma, atrial fibrillation, blood pressure and cancer care . Similarly, in the US indicators are in widespread use in hospital practice focusing, for example, on re-hospitalization of patients within 30 days of discharge, which can be used to penalize hospitals [22, 23]. By imposing financial penalties for those with the highest readmission rates and thus penalizing those with poor levels of care, the hope is to improve the quality of care delivered .
Implications for policy, practice and research
Indicator development, implementation testing, and maintenance and retirement considerations should be seen as integral to the process of producing guidelines as this will maximize the chances of translating guideline recommendations into routine clinical practice and thereby improve outcomes. Quality indicators can improve this translational process through associated financial incentives and penalties as noted above, but they can also be used in more subtle ways through, for example, benchmarking efforts, supporting audit cycles and quality improvement initiatives. These comparative processes, particularly if they involve financial incentives and fines or reputational damage, need to be undertaken with care and with appropriate case mix adjustment, if appropriate .
Key next steps are for a multi-stakeholder group to formally consider these existing candidate indicators, chose between existing indicators, propose alternative indicators where considered necessary, develop additional indicators to fill the recommendation gaps, and then undertake formal field work to support implementation efforts. In due course, plans also need to be put into place to consider indicator maintenance and retirement related issues. The AHRQ framework can prove useful to guide this process .
Indicators were identified for all of the recommendations made in the EAACI Anaphylaxis Guidelines, though none of these satisfied all four criteria specified by AHRQ. There are some indicators, particularly in relation to acute management, which would require relatively little effort to render them fit-for-purpose. We also identified some indicators, which may prove suitable in relation to assessing the quality of long-term anaphylaxis care. Other indicators, however, require much more developmental work. To progress this work, stakeholders now need to consider the findings from this review and then undertake additional formative work to ensure that there are a range of suitable indicators that have been both appropriately developed and demonstrated to work in practice to achieve the desired outcome, namely helping to assess the quality of anaphylaxis care delivered to patients.
Johansson SGO, Bieber T, Dahl R, Friedmann PS, Lanier B, Lockey RF, et al. A revised nomenclature for allergy for global use: report of the Nomenclature Review Committee of World Allergy Organization. J Allergy Clin Immunol. 2004;113:832–6.
Muraro A, Roberts G, Clark A, Eigenmann PA, Halken S, Lack G, et al. The management of anaphylaxis in childhood: position paper of the European academy of allergology and clinical immunology. Allergy. 2007;62(8):857–71.
Mullins RJ. Anaphylaxis: risk factors for recurrence. Clin Exp Allergy. 2003;33(8):1033–40.
Soar J, Pumphrey R, Cant A, Clarke S, Corbett A, Dawson P, et al. Emergency treatment of anaphylactic reactions—guidelines for healthcare providers. Resuscitation. 2008;77(2):157–69. doi:10.1016/j.resuscitation.2008.02.001.
Muraro A, Roberts G, Worm M, Bilò MB, Brockow K, Fernández Rivas M, et al. Anaphylaxis: guidelines from the European Academy of Allergy and Clinical Immunology. Allergy. 2014;69(8):1026–45. doi:10.1111/all.12437.
Simons E, Ardusso L, Beatrice Bilò M, El-Gamal Y, Ledford D, Ring J, et al. World Allergy Organization guidelines for the assessment and management of anaphylaxis. World Allergy Organ J. 2010;4(2):13–37.
Vale S, Smith J, Said M, Mullins R, Loh R. ASCIA guidelines for prevention of anaphylaxis in schools, pre-schools and childcare: 2015 update. J Paediatr Child Health. 2015. doi:10.1111/jpc.12962.
Shuttleworth A. A practical approach to implementing guidelines. Nursing Times, 30 Nov 2007. http://www.nursingtimes.net/a-practical-approach-to-implementing-guidelines/304506.fullarticle.
NICE quality standards and indicators. https://www.nice.org.uk/standards-and-indicators.
PRISMA transparent reporting of systematic reviews and meta-analyses. http://www.prisma-statement.org/PRISMAStatement/Default.aspx.
AHRQ quality indicators: quality indicator measure development, implementation, maintenance, and retirement. http://www.qualityindicators.ahrq.gov/Downloads/Resources/Publications/2011/QI_Measure_Development_Implementation_Maintenance_Retirement_Full_5-3-11.pdf.
Muraro A, Roberts G, Worm M, Bilò MB, Brockow K, Fernández Rivas M, et al. Anaphylaxis: guidelines from the European Academy of Allergy and Clinical Immunology. Allergy. 2014. doi:10.1111/all.12437.
Levy M. Audit of self-administered injectable adrenaline prescription in primary care. http://www.guideline-audit.com/adrenaline/audit_specification.php.
National Institute for Health and Clinical Excellence (NICE). Anaphylaxis clinical audit tool implementing NICE guidelines. 2011. https://www.nice.org.uk/guidance/cg134/resources.
Royal College of Paediatrics and Child Health (RCPCH). RCPCH Allergy Care Pathways Project Audit criteria, Apr 2011. http://www.rcpch.ac.uk/system/files/protected/page/2011_RCPCH_Allergy_Audit_v5_0.pdf.
Stang AS, Straus SE, Crotts J, Johnson DW, Guttmann A. Quality indicators for high acuity pediatric conditions Pediatrics. 2013;132(4). http://pediatrics.aappublications.org/content/132/4/752.long.
Gill PJ, O’Neill B, Rose P, Mant D, Harnden A. Primary care quality indicators for children: measuring quality in UK general practice. Br J Gen Pract. 2014;64(629):e752–7. doi:10.3399/bjgp14X682813.
Lee S, Stachler RJ, Ferguson BJ. Defining quality metrics and improving safety and outcome in allergy care. Int Forum Allergy Rhinol. 2014;4(4):284–91. doi:10.1002/alr.21284.
Roland M. Linking physicians’ pay to the quality of care—a major experiment in the United Kingdom. N Engl J Med. 2004;351:1448–54. doi:10.1056/NEJMhpr041294.
Roland M, Campbell S. Successes and failures of pay for performance in the United Kingdom. N Engl J Med. 2014;370:1944–9. doi:10.1056/NEJMhpr1316051.
The NICE Indicator Menu for the QOF National Institute for Health and Care Excellence. https://www.nice.org.uk/Standards-and-Indicators/QOFIndicators.
Joynt KE, Jha AK. Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA. 2013;309(4):342–3. doi:10.1001/jama.2012.94856.
Joynt KE, Jha AK. A path forward on Medicare readmissions. N Engl J Med. 2013;368:1175–7. doi:10.1056/NEJMp1300122.
Desai AS, Stevenson LW. Rehospitilisation for heart failure: predict or prevent? Circulation. 2012. doi:10.1161/CIRCULATIONAHA.112.125435.
Millett C, Majeed A, Saxena S, Laverty A, Alshamsan R, Lee J, et al. Impact of the 2004 General Practitioner Contract on health improvement and inequalities in cardiovascular disease and diabetes: findings from a systematic review and national and local quantitative studies. NIHR Service Delivery and Organisation Programme. Published June 2011. http://www.nets.nihr.ac.uk/__data/assets/pdf_file/0007/82393/ES-08-1716-209.pdf.
AS conceived this study, which was led by SD. AM, GR, SH, MFR and MW commented on an earlier draft of this manuscript. All authors read and approved the final manuscript.
We would like to thank U. Nurmatov for conducting searches and Zakariya Sheikh for technical support.
AS, GR, AM, GR, SH, MFR and MW are all members of the EAACI Anaphylaxis Guidelines and contributed to the development of the EAACI indicators. AS also contributed to the RCPCH indicators.
Availability of data and material
Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Search strategy 1: MEDLINE and EMBASE
((acute or severe or major or serious or life threatening or fatal*) and (allerg* or hypersensiti*)).mp.
exp food hypersensitivity/
exp drug hypersensitivity/
((food or egg? or nut? or peanut? or milk or wheat or drug? or respiratory or asthma* or sting* or venom*) adj3 (allerg* or hypersensiti*)).tw.
((allerg* or hypersensiti*) adj5 reaction*).tw.
quality indicators.mp. or exp Quality Indicators, Health Care/
“Process Assessment (Health Care)”/or clinical best practice.mp.
clinical audit.mp. or exp Clinical Audit/
(quality and outcomes framework).mp.
10 and 17
Search strategy 2: CINAHL
(anaphylaxis or anaphylaxis management) AND (quality indicators or quality standard or clinical audit or patient experience).
Search strategy 3: Google Scholar
Free key word search “anaphylaxis management and quality indicators 2005–2015.