Open Access

Proficiency testing of skin prick testers as part of a quality assurance system

Clinical and Translational Allergy20166:36

DOI: 10.1186/s13601-016-0126-7

Received: 8 June 2016

Accepted: 29 August 2016

Published: 21 September 2016

Abstract

Background

Skin prick test is an important diagnostic procedure in clinical allergy but documentation of the quality is often missing.

Methods

We describe a proficiency system to evaluate staff members in relation to the international recommended reproducibility in terms of coefficient of variation (CV < 40 %) and the linearity (coefficient of regression >0.85) based on blinded octuplicate histamine testing using histamine 3, 10, 30 and 100 mg/ml.

Results

Fourteen trained allergy nurses participated in the proficiency testing. More than 95 % of the nurses, generated coefficient of variation less than 40 %, and for around 35 % of testers the CV were below 20 % based on wheal area. Regarding the linearity (coefficient of regression), only two nurses produced tests with a value below 0.85. On the contrary, 79 % of testers demonstrated a coefficient of regression >0.95. Depending on the gentleness of the prick procedure, the inter-nurse variability in wheal area varied more than twofold corresponding to a 10-doubling of histamine concentration. This would never have been detected without using a proficiency testing system.

Conclusion

The described histamine testing provides an objective system for the evaluation of basic skin test quality assessment standards especially for documentation in scientific studies.

Keywords

Coefficient of regression Coefficient of variation Histamine skin prick test Reproducibility Quality assessment

Background

Skin prick test (SPT) is one of the diagnostic cornerstones in IgE-mediated allergic diseases [1]. It is widely accepted due to the safety, convenience and cost-effectiveness [2, 3]. It is, however, prone to variability that may fundamentally change the readout of the test. Standards for optimal performance of the SPT have been provided [2, 4, 5]. In the daily routine the SPT are normally performed by nurses or technicians. Based on the position as a “gold standard” in allergy diagnosis [3] it is important to bring the SPT proficiency up to a predefined minimal standard and to be able to document this. As quality assurance is becoming increasingly demanded, it is crucial also to bring a biological test into better alignment with other diagnostic tests where quality assurance is an integrated component of performing the test. A suggested proficiency testing protocol was presented in 2008 by The American Academy of Allergy, Asthma and Immunology (AAAAI) and the American College of Allergy, Asthma and Immunology (ACAAI) joint Update Practice Parameter [2]. In spite of that, a recent paper [6] calls for standards and consistency of the test due to a described gab between recommendations and daily practice. Especially in publication of clinical studies, proficiency testing data for SPT performance should be reported.

This study describes a quality assurance system to evaluate and to document the reproducibility of skin prick testers. Due to the extensiveness of this system, it is suggested to perform this test yearly or every second year.

Methods

Study population

All skin test were performed on only one individual (HJM) between 8 am and 4 pm over a period of 2 months with at least 1 day interval. The subject did not suffer from allergy or skin diseases and had not been treated with any substances interfering with the histamine induced skin reaction [5].

Skin prick test

The testing was performed double-blind using bottles numbered 1–32 which included 8 bottles of four concentrations of histamine hydrochloride (3, 10, 30 and 100 mg/ml) in 50 % glycerol applied in random order. Each test session consisted of 32 individual SPTs with 8 repeats of each of the four histamine concentrations. Negative controls in duplicate were performed with saline. The tests were applied on the forearms spaced 3 cm apart by the “puncture method” using the same type of a 1 mm lancet (device applied at 90° angle with downward pressure) (7). Devices were used once and then discarded [7]. The wheal reactions were read after 15 min, outlined and transferred by tape to paper. The area in mm2 was measured by PC-based planimetry witch previously has shown a documented CV of 2–3 % [8]. Positive skin test was defined as a wheal area ≥7 mm2 [2, 4, 5]. The consistency in producing identical wheal areas was expressed as coefficient of variation (CV) and was calculated for the octuplicate tests for each histamine concentration and testing personal. The linearity (dose–response relationship) was calculated as correlation coefficient in a log area vs log histamine concentration based on all tests per concentration [5]. All nurses employed at the Allergy Clinic, Copenhagen University Hospital Gentofte were included in the proficiency testing. Some nurses were newly employed with limited experience but carefully trained whereas others had more than 20 years of experience.

Results

Fourteen testers completed the proficiency study. The negative controls resulted for most operators in completely negative reactions and for all operators in wheal reactions less than 7 mm2. This “background” was not subtracted from the histamine reactions.

Figure 1 shows that the majority of operators were able to obtain a CV less than 40 % as 96.4 % of the tests came below this recommended maximum value. Around 35 % of operators produced a CV below 20 %. In order to accept the linearity in a log–log system the coefficient of regression must be >0.85. Two testers (14.3 %) failed to reach this goal. Positively, 79 % of testers demonstrated a coefficient of regression >0.95.
Fig. 1

The coefficient of variation of 8-double histamine wheal areas. a depicts the results of 3 mg/ml; b 10 mg/ml; c 30 mg/ml and d 100 mg/ml histamine. The solid line indicates the upper limit (CV < 40 %) for acceptance (4,5) while the dotted line represents the CV of 20 % (highly qualified tester)

The wheal area of the 4 different concentrations of histamine is shown in Fig. 2. The median area of histamine 3 mg/ml was 28 mm2, 41 mm2 for histamine 10 mg/ml, 49 mm2 for histamine 30 mg/ml and 63 mm2 for histamine 100 mg/ml. Some operators produced large skin reactions (like tester #3 and #7) with all concentrations tested, while other consequently produced small reactions (tester # 6). The difference between individual testers corresponded to more than a doubling of the area. The slope of the log–log dose response curves varied from 0.168 to 0.279 (median 0.217) equivalent to slightly less than a doubling of wheal area by 10-doubling the histamine concentration (Fig. 3).
Fig. 2

Mean wheal area (square millimetres) of the 8 repeats of each of the 4 different concentrations of histamine of the 14 nurses tested. Histamine concentrations as in Fig. 1

Fig. 3

Dose-response curves of wheal area vs histamine concentration of all 14 tested operators. The median slope is 0.217 corresponding to almost a doubling of the wheal area by increasing the histamine concentration by a factor 10

Discussion

In order to bring SPT into better alignment with areas of medicine where quality data are required, proficiency testing based on specific standards should be requested. Data from 2006 indicate a need as up to 90 % of practicing allergists did not assess the reproducibility of their testing staff’s skills [9]. The European standard for skin prick testing implies a coefficient of variation based on wheal diameter less than 20 % (40 % with wheal area) after histamine control applications and dose–response relationship calculated as correlation coefficient higher than 0.85 [5]. A coefficient of variation of less than 30 % (diameter) has been advised by the 2008 American Academy of Allergy, Asthma and Immunology and the American College of Allergy, Asthma and Immunology Joint Update Practice Parameter [2]. In the present study, the operators nicely fulfilled the requirements as regards reproducibility (CV %) and linearity (R2). In spite of minor deviations the increase in wheal area is constant over the range of histamine concentrations used (Fig. 3), but the operators were not able to produce inter-tester comparable wheal areas. The difference between different operators corresponds to at least a tenfold difference in histamine concentration. This substantial inter-tester variability does not seem to be related to variability in histamine responsiveness nor to the outlining of the wheal, but to the very performance of the SPT i.e., the gentleness of the prick procedure. The quality of the test was not convincingly related to the years of experience of the testers.

In daily clinical practice the skin response is either a “positive” or “negative” categorical assessment, but in scientific studies the results may often be interpreted quantitatively. In the scientific literature documentation of quality assurance standards being fulfilled is often missing [6]. Unless allergen sensitization is read as histamine equivalent reaction [10] it is important to produce wheal areas of a constant size and the force of the prick procedure seems to be crucial in this aspect. Producing small wheal reactions may imply that weakly sensitized patients may be missed due to a false negative skin test. To accurately interpret results of SPT in these settings, it is important to document a low-level inter-tester variability when performed by multiple operators [11]. The titrated histamine testing gives an easily assessable evaluation of the inter-tester and intra-tester reliability and quality in performing SPT.

Compared to the 2008 American Suggested Proficiency Testing and Quality Assurance Technique for SPT [2] the method suggested in this paper implies several advantages: (1) all tests are applied on only one individual and thereby ruling out intra-individual recipient differences as being the basis for observed differences between individual operators; (2) planimetry imply a more precise measurement of the wheal than using the mean diameter [8]; (3) the titrated test offers the possibility of evaluating the linearity (coefficient of correlation).

The histamine proficiency testing provides an objective system for making quality assurance evaluation without exposing allergic patients to unnecessary hazardous and unpleasant testing using allergen extracts. The results of histamine testing may not be directly transferable to testing with allergens, but it does give valid information how staff members performing SPT in reality meet basic quality assurance standards.

In centres not possessing equipment for planimetry, the mean diameter may be used, but we prefer planimetry due to a higher precision. For scientific purposes, SPT proficiency testing represents an objective standard to verify operator qualification and should be integrated in a centre’s standard quality assurance programme.

We are about to initiate a project focusing on self-training to be able to induce wheal areas within a pre-defined range (diameter 7 ± 1 mm; area 28–50 mm2) using histamine 10 mg/ml in order to be able to compare actual allergen induced wheal size between individual operators [12].

Conclusion

Documenting the quality of skin prick testing is essential especially in scientific studies. We describe an objective proficiency system using histamine testing for documenting basic skin test quality standards. Histamine testing imply not running the risk of inducing severe allergic reaction as might be the case with allergens and by using only one test person we minimize other variables, that might confound the outcome. We suggest performing proficiency testing annually or biannually for quality assurance.

Abbreviations

SPT: 

skin prick test

CV: 

coefficient of variation

R2

coefficient of regression

Declarations

Authors’ contributions

PAH, LSK and HJM designed the investigation and wrote the article. LKP managed data from the area measurements and critically reviewed the article. All authors read and approved the final manuscript.

Acknowledgements

None.

Competing interests

The authors declare that they have no competing interests.

Availability of data and materials

All relevant raw data on which this article is based are available from the author.

Ethical approval and consent to participate

According to Danish legislation standard quality assessment only involving staff members does not require ethical approval. In relation to the present manuscript, all tests were performed on one of the authors (HJM).

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
Allergy Clinic, Copenhagen University Hospital Gentofte

References

  1. Heinzerling L, Mari A, Bergmann K-C, et al. The skin prick test—European standards. Clin Transl Allergy. 2013;3:3.View ArticlePubMedPubMed CentralGoogle Scholar
  2. Bernstein IL, Li JT, Bernstein DI, et al. Allergy diagnostic testing: an update practice parameter. Ann Allergy Asthma Immunol. 2008;100(Suppl):3.Google Scholar
  3. Bousquet J, Heinzerling L, Bachert C, et al. Practical guide to skin prick tests in allergy to aeroallergens. Allergy. 2012;67:18–24.View ArticlePubMedGoogle Scholar
  4. Dreborg S. Skin tests used in type I allergy testing. EAACI position paper. Allergy. 1989;10(44):59.Google Scholar
  5. Dreborg S, Frew A. Allergen standardization and skin tests. EAACI position paper. Allergy. 1993;14(48):49–82.Google Scholar
  6. Fatteh S, Rekkerth DJ, Hadley JA. Skin prick/puncture testing in North America: a call for standards and consistency. Allergy Asthma Clin Immunol. 2014;10:44.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Malling H-J, Andersen CE, Boas M-B, Holgersen F, Munch EP, Weeke B. The allergy pricker. Qualitative aspects of skin testing using a precision needle. Allergy. 1982;37:563–7.View ArticlePubMedGoogle Scholar
  8. Poulsen LK, Liisberg C, Bindslev-Jensen C, Malling HJ. Precise area determination of skin-prick tests: validation of a scanning device and software for a personal computer. Clin Exp Allergy. 1993;23:61–8.View ArticlePubMedGoogle Scholar
  9. Oppenheim J, Nelson H. Skin testing: a survey of allergists. Ann Allergy Asthma Immunol. 2006;96:19–23.View ArticleGoogle Scholar
  10. Malling H-J. Diagnosis and immunotherapy of mould allergy. II. Reproducibility and relation between skin sensitivity estimated by end-point titration and histamine equivalent reaction using skin prick test and intradermal test. Allergy. 1985;40:354–62.View ArticlePubMedGoogle Scholar
  11. Werther RL, Choo S, Lee KJ, et al. Variability in skin prick test results performed by multiple operators depends on the device used. WAO J. 2012;5:200–4.Google Scholar
  12. Dreborg S. Allergen skin prick test should be adjusted by the histamine reactivity. Int Arch Allergy Immunol. 2015;166:77–80.View ArticlePubMedGoogle Scholar

Copyright

© The Author(s) 2016