Abstract
Objective. To determine the ability of the revised version of the Childhood Health Assessment Questionnaire (CHAQ), the VASCHAQ, to detect clinical change over time in pediatric patients with juvenile idiopathic arthritis (JIA). We studied the relative responsiveness of the VASCHAQ as compared to the original CHAQ-30 and revised CHAQ-38, as well as the parent-patient, physician-patient, and physician-parent concordance.
Methods. The CHAQ-38 and VASCHAQ were administered to 30 parents and patients (if older than 8 years) with any subtype of JIA before and after the start of a new treatment. The standardized response means (SRM) were calculated for the VASCHAQ, the original CHAQ-30, and the CHAQ-38. Comparisons of SRM were made using the relative SRM. Parent-patient, physician-patient, and physician-parent concordances were assessed by calculating a series of intraclass correlation coefficients.
Results. Twenty-seven parents and 21 patients completed questionnaires at both visits. All questionnaires demonstrated large responsiveness; however, the VASCHAQ was found to be about 25% more responsive than both the original CHAQ-30 and CHAQ-38.
Conclusion. The VASCHAQ was moderately more responsive than the CHAQ-30 and CHAQ-38 in both parent and patient groups and should be considered for use in studies evaluating change in function over time.
Juvenile idiopathic arthritis (JIA) is a chronic musculoskeletal condition that affects childhood functional ability. Decreased functional ability is caused by impairments such as a loss of joint motion, joint swelling, and pain1.
The Childhood Health Assessment Questionnaire-Disability Index (CHAQ) is a self-report questionnaire of patient functional ability in 8 domains of function: dressing and grooming, arising, eating, walking, hygiene, reach, grip, and activities.
The CHAQ has become a widely used tool for measuring functional ability in children with JIA and other childhood rheumatic conditions. The CHAQ demonstrated high validity, reliability, and good responsiveness in clinical trials and rehabilitative interventions2. These positive results have led to the widespread use of the CHAQ cross-culturally3. The CHAQ is used to assess functional ability in other childhood chronic conditions including systemic lupus erythematosus4 and juvenile idiopathic myopathies5, and has become a primary outcome tool in the management of juvenile dermatomyositis6.
However, the original CHAQ (CHAQ-30) exhibits a ceiling effect. Questionnaire scores are commonly clustered around 0, representing no disability7,8. Thus, the CHAQ-30 is insensitive to detecting clinical changes in mild cases of JIA; false-negative outcomes may occur in patients who are close to the ceiling.
The limitations of the CHAQ-30 warranted a revision to the questionnaire. Lam, et al proposed a revised CHAQ, the CHAQ-38, with an additional 8 more challenging questions9. Additionally, Lam, et al proposed 3 revised versions of the CHAQ with different rating scales: choice, categorical, and visual analog scale (VAS). It was determined that the VASCHAQ demonstrated the best ability to discriminate between patient and control groups, and scores differed the least from a normal distribution. In recent studies the addition of 8 more challenging questions and the removal of domain scoring reduced the ceiling effect in both patient and control groups10,11. However, revised categorical response options (questions asked in relation to the child’s peers) reduced the ceiling effect, but demonstrated lower discriminatory ability11.
The revised CHAQ, VASCHAQ, was modified from the CHAQ-38 by removing consideration for aids and devices or help, using response options where questions are asked in relation to the child’s peers, and having a 10-cm visual analog rating scale for each question. Similarly, the removal of aids and devices or help is consistent with research by Saad-Magalhães, et al, because the removal of these items in a methotrexate (MTX) trial did not alter the interpretation of disability at a group level12.
It is important to evaluate the child and parent-proxy agreement for the patient self-reported measures, because parents are the principal medical decision-makers in pediatric populations. In the preliminary study of the CHAQ-30 it was determined that parents can reliably report on their child’s functional abilities2. In other studies of patients with JIA, parents have been shown to be fair13, good8, and excellent14 proxy reporters for the CHAQ. Good patient-parent agreement is vital for a parent-proxy to effectively report on their child’s functional abilities, and thus assist clinicians in medical management.
Physicians are unable to assess day-to-day disease activity, and therefore it is imperative to evaluate the concordance between physician assessments at clinic visits and patient or proxy reports of daily functional abilities. Discordance was frequent between proxy-reported and physician objective assessment of functional ability in a study by Ravelli, et al15. Further, Armbrust, et al determined that parents tend to overestimate their child’s active joints16. However, agreement between patient-reported and physician-objective assessments on the number of total active joints was reasonable in a study by Dijkstra, et al17. Good concordance may result in more effective treatment regimens that reflect both the patients’ perceptions of their disease activity and the physician’s objective assessments.
It is necessary to measure the responsiveness of disease outcome instruments to validate the instrument’s use in assessing disease improvement and worsening18. The CHAQ-30 demonstrated adequate responsiveness in a study by Brown, et al19. A more responsive tool improves the power of study; it allows researchers to conduct studies with fewer subjects, making the studies less expensive and easier to perform.
The primary aim of our study was to determine the responsiveness of the self-reported and proxy-reported VASCHAQ. Therefore we asked whether the VASCHAQ is more responsive to a physician-implemented new intervention compared to the original CHAQ-30 in a population of pediatric patients with any subtype of JIA. We also examined the VASCHAQ’s responsiveness compared to the CHAQ-38, using its original response options. Secondary aims of our study were to evaluate the agreement between parent proxies and patient self-reports, between physician assessments and patient self-reports, and between physician assessments and parent reports using reports of total active joints and joints with a limited range of motion (ROM). It was hypothesized that the VASCHAQ would be more responsive than both the original CHAQ-30 and CHAQ-38.
MATERIALS AND METHODS
Study design
The Research Ethics Board at The Hospital for Sick Children (SickKids; Toronto, Canada) approved the study, and both parents and patients provided signed informed consent (or assent). Two questionnaires, the CHAQ (the original version plus the extra 8 questions to make the CHAQ-38) and VASCHAQ, were administered to parents and patients (if older than 8 years) using a repeated measures design. Questionnaires were administered at baseline and at a followup visit, 4–6 weeks after the implementation of a new therapeutic intervention. This timeline was chosen because it is the expected length of time to permit a change in a patient’s physical function. Researchers worked from an interview script at both baseline and followup visits to ensure clear instructions were given on how to complete the self-report questionnaires. Parents and patients completed the questionnaires separately at both visits. Researchers completed an interview quality recording sheet to assess the degree of difficulty the respondents had in answering the questions.
Subjects
Patients were recruited from rheumatology clinics and medical day care at SickKids during their normally scheduled clinic visits. Patients were included in the study if they had a confirmed diagnosis of any subtype of JIA and were about to start a new treatment. Parents and patients were excluded from the study if they were unable to complete the questionnaires in English. Basic patient demographic information was obtained at baseline, which included date of birth, sex, subtype, and duration of JIA. Basic information about the parent-proxy was also obtained at baseline including date of birth, sex, and educational level.
Sample size consideration
There is no generally accepted method of calculating sample size for a responsiveness study20. We used the formula for the CI around a standardized response mean21, with a 95% CI and effect size estimate of 1.0, to determine that at least 30 patients are needed to achieve adequate precision to distinguish a moderate standardized response mean effect size (> 0.5) from a large effect size.
Treatments
Patients started a new treatment that was expected on average to be associated with a change in function. These new treatments included intraarticular steroid injections (IAS), MTX, etanercept, oral corticosteroids, infliximab, and nonsteroidal antiinflammatory drugs, or a combination of these treatments.
Questionnaires
The same series of questionnaires were administered to parents and patients separately at both baseline and followup visits. The CHAQ-38 and VASCHAQ were administered in a computer-generated random order. Further, parents and patients completed a global assessment report to record the patient’s overall well-being during the past week. This measurement was recorded on a 10-cm VAS anchored by “excellent well-being” and “extremely poor well-being.” The physician completed a Global Assessment report of disease activity on a similar scale, anchored by “no disease” and “extremely active disease.” At the followup visit only, parents, patients, and physicians also answered a yes/no question: “Has [your, your child’s, this child’s] arthritis improved since starting the newest treatment?”
The original CHAQ contains 30 questions, each scored on a 4-point scale (0–3). Increasing scores represent decreasing functional ability. The 30 questions are separated into 8 domains of physical function, as above. The question with the greatest score in each domain is used as the score for that domain. If patients require aids and devices or help from others in any of the domains, a minimum score of 2 is assigned to that domain. Domain scoring (computing the average of all 8 domain scores) is used to calculate the Disability Index (DI) score for the original CHAQ-30. A “not applicable” option is available for each question to avoid developmental bias. Assessment of functional ability was the focus of our study; therefore the VAS scales for pain and disease effect were not assessed.
The CHAQ-38 contains the same items and response options as the original CHAQ-30, with an additional 8 more challenging questions in the Activities domain9. The summary score for the CHAQ-38 was calculated by averaging all question scores, with no consideration for the domain structure, as per Lam, et al9.
The VASCHAQ contains the same items as the CHAQ-38 but functional ability is answered relative to “other kids my age.” Each question is answered on a 10-cm VAS, with scores ranging from 0, or the worst functional ability, to 100, or the best functional ability, for each question. The VASCHAQ does not use the domain structure, and does not inquire about a child’s need for aids and devices or help from others. The average of the 38 questions is taken as the overall score for the VASCHAQ. If a question does not apply to the child, it is marked as “not applicable” and is excluded from the score calculation.
Joint counts
Self-reported and proxy-reported joint counts for tenderness or pain and ROM were completed at both the baseline and followup visits. The number of total active joints was assessed as joints reported to experience tenderness or pain and demonstrate a limited ROM.
Physicians assessed the number of total active joints at both the baseline and followup visits. The EPM-ROM scale22 was used at the baseline visit to measure the number of joints with a limited ROM. This scale has shown to be valid, reliable, and sensitive to change22,23.
Responsiveness
The correlation between measures of improvement was determined using the Pearson product-moment correlation. The standardized response mean (SRM) was calculated for the patient- and parent-completed CHAQ-30, CHAQ-38, and VASCHAQ. The 95% CI were calculated assuming normal distribution. According to Zou21, the SRM is the only necessary measure to quantify the responsiveness in a 2-timepoint (pre-post) or repeated measures study design. The results were interpreted as large if the SRM was > 0.8, as moderate (0.5–0.8), or as small (0.2–0.5)20. Comparisons of SRM were made using the relative SRM.
Agreement
To assess the concordance between patients, parent-proxies, and physician assessments, a series of intraclass correlation coefficients (ICC) were calculated. ICC were calculated for the VASCHAQ (patient-parent concordance only), the total active joint count, and for the number of joints with limited ROM. The ICC was considered to indicate poor agreement if the value was < 0, slight agreement at 0.00–0.20, fair agreement at 0.21–0.40, moderate agreement at 0.41–0.60, substantial agreement at 0.61–0.80, and almost perfect agreement at 0.81–1.0024.
RESULTS
Subjects
The study included 30 subjects. Thirty parents and 23 patients completed the questionnaires at the baseline visit only. At both visits, 27 parents and 21 patients completed the questionnaires. The average followup time between visits was 8.5 weeks. Patient demographics are presented in Table 1. The average age of the parent proxies was 41.6 years (SD 6.0), and the majority of the 30 parents were female (73.33%) and had attended either college or university (83.33%).
Patient demographics (n = 30): age, sex, JIA subtype, new interventions.
Treatments
The majority of patients received IAS (Table 1).
Questionnaires
The median questionnaire scores, parent and patient global assessment scores, and physician global assessment of disease activity scores are shown in Table 2. Most patients, parents, and physicians indicated improvement in the patient’s JIA after treatment; however, 7 patients had either parent, patient, or physician indicate “No” to improvement. Although the median scores indicated that our sample improved in the majority of our outcome variables, we observed worsening scores for several patients in several variables.
Scores: CHAQ-30, CHAQ-38, VASCHAQ, and joint counts at baseline and followup.
Joint counts
The results of the joint count assessments are presented in Table 2.
Responsiveness
Questionnaire scores and our other outcome variables indicated that some patients improved and some patients worsened. Pearson product-moment correlations were calculated (Supplementary Table 1, available online at jrheum.org), and it was determined that the change in questionnaire scores was correlated with the change in the other outcome variables such as proxy-reported total active joint counts and counts of joints with a limited ROM. This correlation justified using the absolute values of the questionnaire score differences from baseline to followup when calculating the SRM (Table 2). All questionnaires demonstrated moderate to large responsiveness in both parent and patient groups.
The relative SRM statistic was calculated using either the CHAQ-30 or CHAQ-38 as the standard of comparison (Table 3). The VASCHAQ was more responsive than the CHAQ-30 and the CHAQ-38 in both patient and parent groups. The CHAQ-30 and CHAQ-38 demonstrated about the same responsiveness in both patient and parent groups.
Comparing responsiveness of CHAQ-30, CHAQ-38, and VASCHAQ: relative standardized response means. Means were calculated by dividing SRM of VASCHAQ or CHAQ-38 by the standard of comparison (either CHAQ-30 or CHAQ-38).
To further evaluate the revised questionnaires compared to the original CHAQ-30, a regression of the difference in questionnaire scores from baseline to followup was completed for the CHAQ-38 (Supplementary Figure 1) and VASCHAQ (Supplementary Figure 2, both available online at jrheum.org).
Parent-child agreement
The concordance was measured for the CHAQ-30 (ICC 0.83, 95% CI 0.60–0.93), CHAQ-38 (ICC 0.84, 95% CI 0.63–0.93), VASCHAQ (ICC 0.49, 95% CI −0.18 to 0.78), total active joint count (ICC 0.74, 95% CI 0.39–0.89), and counts of joints with limited ROM (ICC 0.75, 95% CI 0.42–0.89).
Physician-child agreement
The concordance was measured for the total active joint count (ICC 0.58, 95% CI 0.033–0.82), and counts of joints with limited ROM (ICC 0.59, 95% CI 0.045–0.82).
Physician-parent agreement
The concordance was measured for the total active joint count (ICC 0.19, 95% CI −0.69 to 0.61), and counts of joints with limited ROM (ICC 0.16, 95% CI −0.75 to 0.6).
DISCUSSION
We found that all 3 questionnaires (CHAQ-30, CHAQ-38, VASCHAQ), both self-reported and proxy-reported, demonstrated moderate to large responsiveness. The CHAQ-30 and CHAQ-38 were equally responsive in both patient and parent groups. We found that parents are effective proxy-reporters, demonstrating substantial and almost perfect agreement with their child for the CHAQ-30, CHAQ-38, and reported joint counts. Overall, the VASCHAQ was more responsive than the CHAQ-30 and CHAQ-38 in both parent and patient groups and should be preferred for use in studies evaluating change in function over time.
Parent-child agreement was substantial or almost perfect for all measurements except for the VASCHAQ. This moderate agreement may be explained by the nature of the VASCHAQ’s response options. The VASCHAQ asks respondents to answer questions in relation to the child’s peers and record their response on a 10-cm VAS. Parents may have a different understanding of how their child relates to his or her peers. When the VASCHAQ is used, therefore, careful consideration should be made to choose the most appropriate respondent group.
Physician-child agreement was moderate or substantial for all measurements. Physician assessments are often regarded as the gold standard. Therefore, this finding suggests that self-report measures of health status, such as self-reported joint counts, are accurate assessment tools.
Physician-parent concordance demonstrated slight agreement for total active joint counts and counts of joints with a limited ROM, similar to the findings of Armbrust, et al16. Parents may overestimate their child’s active joint count by including joints with secondary symptoms of JIA, such as pain. Thus, it is important to educate parents on the difference between the symptoms of JIA, pain, and functional disability before they can be reliable proxy-reporters16.
Our results must be considered in light of several potential limitations. First, 3 of our subjects did not complete a followup visit; therefore we were missing some data and our sample size was somewhat lower than planned. However, 90% of enrolled subjects completed all necessary assessments. Additionally, researchers reported that all interviews were of high quality and easy to conduct, validating the quality of our measured variables. Also, despite fewer subjects than planned, our CI around the relative SRM were still quite tight, indicating an acceptable level of precision. Additionally, our study subjects received a variety of interventions, which may have led to reports of clinical worsening after the administration of a new intervention. This limited our ability to analyze the average responsiveness of the questionnaires. Despite this heterogeneity in our study population, the change in questionnaire scores correlated with the direction and change in our other outcome variables, which justified using the absolute value of the change in our SRM calculations.
We found that all 3 questionnaires demonstrated moderate to large responsiveness. The VASCHAQ was about 25% more responsive to a new treatment compared to both the CHAQ-30 and CHAQ-38. The VASCHAQ demonstrated moderate parent-child concordance, possibly because of the nature of the response options. The observed results suggest that the VASCHAQ is a better assessment of functional ability in populations of patients with JIA because it is moderately more responsive to change over time, and does not have a ceiling effect. The VASCHAQ should be considered for use in future trials because it may reduce the cost and time needed to determine effective treatments and may help guide better disease management.
ONLINE SUPPLEMENT
Supplementary data for this article are available online at jrheum.org.
- Accepted for publication June 20, 2016.
