Control of Allergic Rhinitis and Asthma Test (CARAT) can be used to assess individual patients over time
© Fonseca et al.; licensee BioMed Central Ltd. 2012
Received: 30 April 2012
Accepted: 11 July 2012
Published: 30 August 2012
The Control of Allergic Rhinitis and Asthma Test (CARAT10) has been proposed as the first tool to implement the Allergic Rhinitis and its Impact on Asthma initiative guidelines in clinical practice. To serve this purpose, it must have adequate properties to assess the control of an individual over time. This study aimed to prospectively assess the test-retest reliability, responsiveness and longitudinal validity of CARAT10.
Adults with asthma and allergic rhinitis were enrolled at 4 outpatient clinics of Portuguese central hospitals. At each of the two visits, 4 to 6 weeks apart, patients filled out CARAT10 and additional questionnaires, followed by a medical evaluation blinded to the questionnaires’ answers.
From the 62 patients included, 51 patients completely filled out CARAT10 at both visits. The test-retest reliability, computed as an intra-class correlation coefficient, was 0.82. Regarding responsiveness, a significant change (p = 0.002) of CARAT10 score in clinically unstable patients was observed (95%CI -5.08; -1.31) and the Guyatt’s responsiveness index was 1.54. As for the longitudinal validity assessment, the correlation coefficients of the changes of CARAT10 scores with those of ACQ5 and symptoms VAS ranged from 0.49 to 0.65, while with the physician assessment of control they ranged from 0.31 to 0.41.
CARAT10 has good test-retest reliability, responsiveness and longitudinal validity. It can be used to assess control of allergic rhinitis and asthma, both to compare groups in clinical studies and to evaluate individual patients in clinical practice.
KeywordsAsthma Allergic rhinitis Control Questionnaire
Rhinitis and asthma are highly prevalent diseases that are closely associated. A few questionnaires have been developed and validated to assess the control of rhinitis[1, 2] and asthma[3, 4]. For over a decade, the Allergic Rhinitis and its Impact on Asthma (ARIA) initiative has recommended the simultaneous assessment and management of these diseases[5, 6]. Recent observational studies reinforced the association of rhinitis and asthma. In a cross-sectional study of asthma patients from 85 primary care practices in the United Kingdom, self-reported rhinitis was identified as a major predictor of poor asthma control. Also, data from the West Sweden Asthma Study linked the degree of rhinitis with the risk of having multi-symptom asthma.
We have previously developed the Control of Allergic Rhinitis and Asthma Test – a 17-item version – employing a formal methodological approach to ensure its quality and content validity. Subsequently, we conducted a cross-sectional study, in which we performed factor analysis to reduce the questionnaire and assess its unidimensionality, resulting in a simple 10-item version (CARAT10). In that study, we described how 2 independent factors were extracted from the reduced version, matching the initial theoretical subdomains of asthma and allergic rhinitis. Nevertheless, we demonstrated that CARAT10 has good internal consistency, similar to that of widely used asthma control questionnaires, thus supporting the concept of assessing allergic rhinitis and asthma simultaneously (construct/structural validity). Finally, we also showed that CARAT10 has good discriminative properties and concurrent validity.
CARAT10 has been proposed as the first tool to implement ARIA guidelines in clinical practice. To serve this purpose, such tool must have adequate properties to assess asthma and allergic rhinitis control of an individual over time.
Thus, this study aimed to prospectively assess the test-retest reliability, responsiveness and longitudinal validity of CARAT10.
Study design and setting
This prospective observational study comprised two visits, 4 to 6 weeks apart, and was conducted in the first semester of 2009.
Patients were enrolled in 4 allergy outpatient clinics of central hospitals in three Portuguese regions – north (Porto and Gaia), south (Lisbon) and the Azores islands (Ponta Delgada).
The study was approved by the Hospital S. João review board (Comissão de Ética para a Saúde) and was given the project nº 120/08. Each patient gave his/her written informed consent.
All patients between 18 and 70 years of age, with a medical diagnosis of asthma and allergic rhinitis and at least 6 months of follow-up at the clinic were eligible. Only patients unable to fill the questionnaire were excluded.
At each visit, patients were asked to fill CARAT10 and additional tools: the Asthma Control Questionnaire (ACQ5) and three visual analogue scales (VAS) concerning all airways symptoms, bronchial/pulmonary symptoms and nasal symptoms as measures of self-perceived control. They also underwent lung function tests followed by a medical evaluation. The study design is described in the Additional file1: Figure S1.
Lung function and exhaled nitric oxide (Fe NO50) were measured according to the 2005 ATS/ERS position statements[13, 14]. Lung function variables included the forced vital capacity (FVC), the forced expiratory volume in 1 second (FEV1) and the peak expiratory flow (PEF). Fe NO50 was evaluated only in two of the four centres and expressed as parts per billion (ppb).
The medical evaluation was carried out by the attending allergy specialist, who was blinded to the questionnaires’ answers. The physician classified the patient’s rhinitis and asthma severity and rated the rhinitis and asthma control in two 10 cm VAS, according to his/her judgment and bearing in mind the classifications of ARIA and Global Initiative for Asthma (GINA). The treatment decisions were made by the attending physician, based on the clinical history, physical examination and lung function following ARIA and GINA guidelines. In addition, known allergies and current medication, as well as the decision regarding treatment plan (increase, maintain or decrease treatment) were registered. In the second visit, the same procedures were repeated and, additionally, the physician filled out two 10 cm VAS regarding the variation in asthma control and in rhinitis control (from 0 - greatest worsening, to 10 - greatest improvement).
The statistical analysis was carried out using SPSS 19.0 (SPSS Inc., Chicago, IL, USA). The level of significance was set at p < 0.05. The population was described using standard descriptive statistical techniques.
We derived dichotomous variables, classifying the patient as having controlled or not controlled asthma and controlled or not controlled rhinitis, from the physician’s control VAS. Patients were categorized as controlled if the physician filled out the VAS in the ]6,10]cm interval.
Patients who were graded between 5.0 and 6.0 in the second visit’s control variation VAS were considered to be clinically stable.
We classified the patients’ rhinitis severity according to the ARIA recommendations and the asthma control according to the GINA guidelines. The results from both visits were pooled and were plotted against the scores in CARAT10 and its factors.
Properties of CARAT10
The CARAT10 internal consistency analysis comprised the assessment of its internal consistency with Cronbach’s α.
The test-retest reliability was assessed using intra-class correlation coefficient (ICC) in clinically stable patients between visits.
The concurrent validity was studied using Spearman’s correlation coefficients between CARAT10/CARAT factors and 1) control assessment instruments or 2) physician’s assessment.
The longitudinal validity was assessed by Spearman’s correlation coefficients calculated between the variation of the CARAT10 score and the variation of the other measures.
For hypotheses testing, a priori predictions for the correlation coefficients were, based on previous studies as follows: (i) 0.6–0.8 with ACQ5; (ii) 0.6–0.8 with the symptoms VAS; (iii) 0.4–0.6 with the physician’s assessment[3, 10].
– Sample’s characteristics and results of physician assessment, control questionnaires and lung function tests
1stvisit n = 62
2ndvisit n = 61
Gender Female n (%)
Age mean (SD) years
Physician assessment n (%)
Only asthma controlled
Only rhinitis controlled
ACQ5 score median (P25-P75)
<0.5 n (%)
0.5–1.5 n (%)
>1.5 n (%)
VAS* median (P25-P75)
Lung function mean (sd) #
F e NO 50 ª
– Comparison of CARAT10 scores according to the patients’ self-perceived control
1 st visit
Rhinitis subscore #
Asthma subscore ª
2 nd visit
Rhinitis subscore #
Asthma subscore ª
Seventeen patients (28%) were classified as having clinically stable asthma and rhinitis. The unstable group included 30 patients with both unstable asthma and rhinitis, 9 only with unstable asthma and 5 only with unstable rhinitis.
Properties of CARAT10
– Spearman’s correlations of CARAT10 and its factors with external measures of rhinitis and asthma control in the first visit
Concerning responsiveness, we observed a significant within-patient change of CARAT10 scoring in clinically unstable patients (95% confidence interval [-5.08; -1.31], p = 0.002). The Guyatt’s responsiveness index was 1.54.
– Longitudinal validity – comparison of the variation of CARAT10 with the variation of external measures of control
The longitudinal assessment of the Control of Allergic Rhinitis and Asthma Test (CARAT10) showed that it has adequate properties to assess individual patients over time.
This is the third phase of a project that aimed to develop and validate a tool to simultaneously assess the control of rhinitis and asthma. We have previously shown that CARAT10 has good internal consistency and internal validity, supporting the concept of a single questionnaire assessing allergic rhinitis and asthma simultaneously, and showing it may be used to compare groups. We had observed good correlations between CARAT10 and ACQ5, symptoms VAS and the physician’s assessment of control, similar to that of other widely used control questionnaires for asthma. The new data confirmed our previous results. As for test-retest reliability, the ICC for CARAT10 scores between visits in stable patients was 0.82, better than the ICC previously reported for Asthma Control Test (ACT) (0.77). Also, the Guyatt’s responsiveness index observed is somewhat higher than that of ACQ7 (1.35). Regarding longitudinal validity, like ACT, CARAT10 correlates better with changes in ACQ than with the physician’s assessment; the correlation coefficient of CARAT10 with symptoms VAS was similar to the previously reported correlation of ACQ7 with asthma symptoms[3, 19]. The correlation coefficient for changes between visits of CARAT10 with the all symptoms VAS met the a priori predictions (Table 4). The correlation coefficients of CARAT10 subscores with external measures of rhinitis and asthma status were lower than predicted. These subscores were defined using standard procedures of exploratory factor analysis in a previous study. However, questions on waking up at night or increasing medication may be related to asthma and/or rhinitis in different patients. Therefore, a reassessment of CARAT10 structure in different populations seems to be necessary.
In this multicenter longitudinal study, it was necessary to categorize patients as clinically stable or unstable between visits. The criterion was the physician’s rating of control variation between visits, since there is no gold standard for the measurement of control of allergic rhinitis and asthma. The intrinsic insufficiencies of physician rating may undermine the results observed. However, the direction of this bias worsens the questionnaire’s properties. Therefore, the estimates for the evaluative and discriminative properties of CARAT10 are conservatively reported.
CARAT10 has been thoroughly studied in cross-sectional and prospective studies and now meets almost all requirements of COSMIN - COnsensus-based Standards for the selection of health Measurement Instruments. Further studies with larger datasets are needed to firmly establish the cut values for the CARAT10 global score and the rhinitis and asthma subscores. The existing data seems to suggest that a CARAT10 score over 24 identifies controlled patients, with a rhinitis subscore over 8 and an asthma subscore equal or greater than 16.
The availability of CARAT10 for clinical practice in multiple countries requires cross-cultural validation in other languages. A website to support an adequate process of translation was developed (http://www.caratnetwork.org) and researchers are invited to participate. Cross-cultural adaptation of CARAT has started in over 10 different countries. It was organized in 3 phases, following the Global Allergy and Asthma European Network (GA2LEN) network recommendations: forward translation, backward translation and patient testing; clinical validation studies for these languages should follow.
In summary, this study showed that CARAT10 has adequate test-retest reliability, responsiveness and longitudinal validity while confirming its high internal consistency and concurrent validity. Therefore, CARAT10 can be used both in clinical studies and in clinical practice, to compare groups and to evaluate individual patients over time.
Asthma control questionnaire
Asthma Control Test
Allergic Rhinitis and its Impact on Asthma
Control of Allergic Rhinitis and Asthma Test
COnsensus-based Standards for the selection of health Measurement Instruments
- Fe NO50:
Fraction of exhaled nitric oxide
Forced expiratory volume in 1 second
Forced vital capacity
Global Allergy and Asthma European Network
Global Initiative for Asthma
Guyatt’s responsiveness index
Intra-class correlation coefficient
Peak expiratory flow
Receiver operating characteristic
Visual analogue scales.
We thank the Portuguese Allergology and Clinical Immunology Society, the Portuguese Pneumology Society and the Portuguese Asthma Patients Association for their collaboration in the project. We thank the physicians participating in the data collection: Ferraz de Oliveira, J; Guilherme, A; Miranda, M; Lopes, I; Moreira da Silva, JP; Delgado, L; Moreira, A; Castro, E; Lopes, A; Fernandes, N; Costa, C. We thank the Directors of the centers for making possible the data collection and the patients for their time and help. Partially supported by an unrestricted grant from Merck Sharp & Dohme Portugal.
- Schatz M, Meltzer EO, Nathan R, Derebery MJ, Mintz M, Stanford RH: Psychometric validation of the rhinitis control assessment test: a brief patient-completed instrument for evaluating rhinitis symptom control. Ann Allergy Asthma Immunol. 2010, 104 (2): 118-24. 10.1016/j.anai.2009.11.063.View ArticlePubMedGoogle Scholar
- Demoly P, Jankowski R, Chassany O, Bessah Y, Allaert FA: Validation of a self-questionnaire for assessing the control of allergic rhinitis. Clin Exp Allergy. 2011, 41 (6): 860-8. 10.1111/j.1365-2222.2011.03734.x.View ArticlePubMedGoogle Scholar
- Juniper EF, O'Byrne PM, Guyatt GH, Ferrie PJ, King DR: Development and validation of a questionnaire to measure asthma control. Eur Respir J. 1999, 14 (4): 902-7. 10.1034/j.1399-3003.1999.14d29.x.View ArticlePubMedGoogle Scholar
- Nathan RA, Sorkness CA, Kosinski M, Schatz M, Li JT, Marcus P: Development of the Asthma Control Test: a survey for assessing asthma control. J Allergy Clin Immunol. 2004, 113 (1): 59-65. 10.1016/j.jaci.2003.09.008.View ArticlePubMedGoogle Scholar
- Bousquet J, Khaltaev N, Cruz AA, Denburg J, Fokkens WJ, Togias A: Allergic Rhinitis and its Impact on Asthma (ARIA) 2008 update (in collaboration with the World Health Organization, GA2LEN and AllerGen). Allergy: Eur J Allergy Clin Immunol. 2008, 63 (SUPPL. 86): 8-160.View ArticleGoogle Scholar
- Brozek JL, Bousquet J, Baena-Cagnani CE, Bonini S, Canonica GW, Casale TB: Allergic Rhinitis and its Impact on Asthma (ARIA) guidelines: 2010 revision. J Allergy Clin Immunol. 2010, 126 (3): 466-76. 10.1016/j.jaci.2010.06.047.View ArticlePubMedGoogle Scholar
- Clatworthy J, Price D, Ryan D, Haughney J, Horne R: The value of self-report assessment of adherence, rhinitis and smoking in relation to asthma control. Prim Care Respir J. 2009, 18 (4): 300-5. 10.4104/pcrj.2009.00037.View ArticlePubMedGoogle Scholar
- Lotvall J, Ekerljung L, Lundback B: Multi-symptom asthma is closely related to nasal blockage, rhinorrhea and symptoms of chronic rhinosinusitis - evidence from the West Sweden Asthma Study. Respir Res. 2010, 11 (1): 163-10.1186/1465-9921-11-163.PubMed CentralView ArticlePubMedGoogle Scholar
- Nogueira-Silva L, Martins SV, Cruz-Correia R, Azevedo LF, Morais-Almeida M, Bugalho-Almeida A: Control of allergic rhinitis and asthma test–a formal approach to the development of a measuring tool. Respir Res. 2009, 10 (52): 52-PubMed CentralView ArticlePubMedGoogle Scholar
- Van Oene CM, Van Reij EJF, Sprangers MAG, Fokkens WJ: Quality-assessment of disease-specific quality of life questionnaires for rhinitis and rhinosinusitis: a systematic review. Allergy: European Journal of Allergy and Clinical Immunology. 2007, 62 (12): 1359-71. 10.1111/j.1398-9995.2007.01482.x.View ArticlePubMedGoogle Scholar
- Fonseca JA, Nogueira-Silva L, Morais-Almeida M, Azevedo L, Sa-Sousa A, Branco-Ferreira M: Validation of a questionnaire (CARAT10) to assess rhinitis and asthma in patients with asthma. Allergy. 2010, 65 (8): 1042-8. 10.1111/j.1398-9995.2009.02310.x.View ArticlePubMedGoogle Scholar
- Bousquet J, Schunemann HJ, Zuberbier T, Bachert C, Baena-Cagnani CE, Bousquet PJ: Development and implementation of guidelines in allergic rhinitis - an ARIA-GA2LEN paper. Allergy. 2010, 65 (10): 1212-21. 10.1111/j.1398-9995.2010.02439.x.View ArticlePubMedGoogle Scholar
- Miller MR, Hankinson J, Brusasco V, Burgos F, Casaburi R, Coates A: Standardisation of spirometry. Eur Respir J. 2005, 26 (2): 319-38. 10.1183/09031936.05.00034805.View ArticlePubMedGoogle Scholar
- ATS/ERS: recommendations for standardized procedures for the online and offline measurement of exhaled lower respiratory nitric oxide and nasal nitric oxide. Am J Respir Crit Care Med. 2005, 171 (8): 912-930.View ArticleGoogle Scholar
- Bateman ED, Hurd SS, Barnes PJ, Bousquet J, Drazen JM, FitzGerald M: Global strategy for asthma management and prevention: GINA executive summary. Eur Respir J. 2008, 31 (1): 143-78. 10.1183/09031936.00138707.View ArticlePubMedGoogle Scholar
- Streiner DL, Norman GR: Health measurement scales: a practical guide to their development and use. 1995, Oxford; New York: Oxford University Press, 2Google Scholar
- Guyatt G, Walter S, Norman G: Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis. 1987, 40 (2): 171-8. 10.1016/0021-9681(87)90069-5.View ArticlePubMedGoogle Scholar
- Schmitt JS, Di Fabio RP: Reliable change and minimum important difference (MID) proportions facilitated group responsiveness comparisons using individual threshold criteria. J Clin Epidemiol. 2004, 57 (10): 1008-18. 10.1016/j.jclinepi.2004.02.007.View ArticlePubMedGoogle Scholar
- Schatz M, Sorkness CA, Li JT, Marcus P, Murray JJ, Nathan RA: Asthma Control Test: reliability, validity, and responsiveness in patients not previously followed by asthma specialists. J Allergy Clin Immunol. 2006, 117 (3): 549-56. 10.1016/j.jaci.2006.01.011.View ArticlePubMedGoogle Scholar
- Juniper EF, Chauhan A, Neville E, Chatterjee A, Svensson K, Mörk AC: Clinicians tend to overestimate improvements in asthma control: An unexpected observation. Prim Care Respir J. 2004, 13 (4): 181-4. 10.1016/j.pcrj.2004.04.003.View ArticlePubMedGoogle Scholar
- Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL: The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010, 19 (4): 539-49. 10.1007/s11136-010-9606-8.PubMed CentralView ArticlePubMedGoogle Scholar
- Juniper EF, Bousquet J, Abetz L, Bateman ED: Identifying 'well-controlled' and 'not well-controlled' asthma using the Asthma Control Questionnaire. Respir Med. 2006, 100 (4): 616-21. 10.1016/j.rmed.2005.08.012.View ArticlePubMedGoogle Scholar
- Cross-cultural adaptation of Control of Rhinitis and Asthma Test (CARAT). Edited by: Fonseca JA, Correia De Sousa J, Sá-Sousa A, Burnay E, Tsiligianni I, Nogueira-Silva L. 2011, Amsterdam, The Netherlands: 2nd IPCRG Scientific MeetingGoogle Scholar
- Braido F, Bousquet PJ, Brzoza Z, Canonica GW, Compalati E, Fiocchi A: Specific recommendations for PROs and HRQoL assessment in allergic rhinitis and/or asthma: a GA(2)LEN taskforce position paper. Allergy. 2010, 65 (8): 959-968. 10.1111/j.1398-9995.2010.02383.x. Annex 3View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.