Abstract
Objective. To explore the parent and child acceptable symptom state in juvenile arthritis (JA-PASS and JA-CASS, respectively) and estimate the JA-PASS and JA-CASS cutoff values for outcome measures.
Methods. Children with juvenile idiopathic arthritis (JIA) and their parents completed a multi-dimensional questionnaire that included parent-reported and child-reported outcomes and a question about whether they considered the disease state as satisfactory. Additional assessments included demographic data, physician-reported outcomes, and acute-phase reactant levels. Stepwise logistic regression was used to assess contributors to JA-PASS and JA-CASS. Cutoff values of outcome measures that defined JA-PASS and JA-CASS were determined using both 75th percentile and receiver-operating characteristic (ROC) curve methods. Testing procedures included evaluation of discriminative and construct validity of the satisfaction question and assessment of reliability of JA-PASS and JA-PASS cutoffs.
Results. Of 584 parents, 385 (65.9%) considered their child in JA-PASS. Of 343 children, 236 (68.8%) considered themselves in JA-CASS. Significant contributors to being in either JA-PASS or JA-CASS were absence of active joints, better rating of overall well-being, and better physical function or health. Cutoff values yielded by 75th percentile and ROC curve methods were similar. Parent, child, and physician global ratings yielded the lowest percentage of false-positive misclassification and the best tradeoff between sensitivity and specificity. The satisfaction question showed good discriminative and construct validity and the JA-PASS and JA-PASS cutoffs were found to be stable over time.
Conclusion. The acceptable symptom state is a relevant concept for children with JIA and their parents and constitutes a valid outcome measure that is potentially applicable in routine practice and clinical trials.
Over the last 20 years, there have been major advances in the treatment of juvenile idiopathic arthritis (JIA), which include the widespread use of methotrexate and intraarticular corticosteroids, the earlier introduction of these drugs, and more recently, the availability of the new biologic agents1,2,3,4,5. At the same time, the number of controlled clinical trials in JIA has grown sharply6. These advances have increased the potential to achieve disease remission or at least to lower levels of disease activity, and have consequently shifted therapeutic aims increasingly toward the attainment of inactive disease6,7,8,9. This has led to the notion that in JIA clinical trials it is important not only to know the magnitude of clinical improvement from baseline, but also to understand whether the observed change leads to an acceptable state according to the physician, the parent, or the child10,11,12,13.
In recent years there has been a great deal of effort to develop measures of disease activity in JIA, leading to the generation of definitions for clinical remission (CR)7 and minimal disease activity (MDA)12. However, CR criteria are based only on physician-reported outcomes and acute-phase reactant levels, while parent proxy-reported and child self-reported outcomes are neglected9. Although the parent global assessment is part of the MDA criteria for polyarthritis, no parent/child-reported measures are included in the MDA definition for oligoarthritis. Hence, definitions of both CR and MDA may not adequately reflect the parent’s and/or the child’s perception of the disease status. Physicians, parents, and children often disagree in assessing different aspects of disease activity, including remission, in JIA14,15,16,17.
In adult rheumatology, the need to know whether the observed change in a clinical trial leads to an acceptable state according to the patient has led to the concept of patient acceptable symptom state (PASS)11,18,19,20,21. The PASS has been defined as the symptom threshold beyond which patients consider their health status as satisfactory. It has been suggested that the PASS constitutes an ambitious target for disease management18.
At present, the definition of an acceptable symptom state does not exist for JIA. Therefore, the primary aim of our study was to identify the cutoff values of outcome measures that define the symptom state considered acceptable by children with JIA and their parents. A secondary aim was to examine the validity of the cutoffs and the acceptable symptom state concept.
MATERIALS AND METHODS
Patient selection
The study sample was composed of consecutive patients seen between March 2007 and December 2009 who were ≤ 18 years old and were diagnosed with JIA by the International League of Associations for Rheumatology (ILAR) criteria21. All parents/guardians provided written informed consent for participation in the study. The study was approved by the institutional review board of the Istituto G. Gaslini, Genoa, Italy.
Assessment of parent/child-reported outcomes
A parent (the mother, whenever possible) or legal guardian of each patient was asked to complete the Italian parent version of the Juvenile Arthritis Multidimensional Assessment Report (JAMAR)22. The child (if aged more than 7–8 years) was also asked to complete independently the Italian patient version of the JAMAR22. A researcher assisted parents and children if they had questions during questionnaire completion. However, no questionnaire was administered in the form of an interview. The JAMAR is a multidimensional questionnaire that incorporates 15 measures/items. For the purposes of our study, the following 7 measures were used: (1) assessment of functional ability, through the Juvenile Arthritis Functionality Scale (JAFS)23. The JAFS is a 15-item questionnaire in which the ability of the child to perform each task is scored as follows: 0 = without difficulty, 1 = with difficulty, 2 = unable to do. The total score ranges from 0 to 30; (2) assessment of health-related quality of life (HRQOL) through the Paediatric Rheumatology Quality of Life Scale (PRQL)24. The PRQL is a 10-item questionnaire that includes 2 subdimensions, each composed of 5 items. The total score ranges from 0 to 30, with higher scores indicating worse HRQOL. A separate score for the physical health and psychosocial health subscales (range 0–15) can be calculated; (3 to 5) rating of the child’s well-being, pain intensity, and level of disease activity on a 21-numbered circle visual analog scale (VAS; 0 = best; 10 = worst)25; (6) subjective rating of the disease status as remission, continued activity, or flare. To facilitate understanding of disease states by parents and children, remission was defined as “complete absence of symptoms,” continued disease activity as “continuing presence of symptoms,” and flare as “recurrence of symptoms after a period of complete well-being”; and (7) a question about satisfaction with the present symptom state. The question, “Considering all the ways the illness affects your child, would you be satisfied if his/her condition remained stable/unchanged for the next few months?” was to be answered “yes” or “no.” The wording of the question was taken from the definition of the concept of PASS suggested at the Outcome Measures in Rheumatology (OMERACT) 8 meeting26.
Assessment of physician-reported outcomes
The attending physician rated the overall level of disease activity on a 21-numbered circle VAS (0 = no activity; 10 = maximum activity)25, and assessed the count of joints with swelling, tenderness/pain on motion, restricted motion, and active disease27. The physician also rated subjectively the current disease status as remission, continued activity, or flare (defined as above). The physician was instructed to base this evaluation on his/her personal opinion about the current disease state and disease course since the previous visit.
Additional assessments
The following data were recorded for each patient: sex, onset age, ILAR category, age at visit, and disease duration. Acute-phase reactants included erythrocyte sedimentation rate and C-reactive protein.
Statistical analyses and validation procedures
Descriptive statistics were reported as medians and interquartile ranges (IQR) for continuous variables and as absolute frequencies and percentages for categorical variables. Comparison of quantitative data was made through the Mann-Whitney U test. Comparison of categorical variables was made by means of the chi-squared test or Fisher’s exact test, as appropriate.
The cutoff value for each outcome measure at which parents and children considered the symptom state as satisfactory (i.e., who answered “yes” to the satisfaction question) was estimated using 2 approaches. In the first, the parent acceptable symptom state in juvenile arthritis (JA-PASS) and the child acceptable symptom state in juvenile arthritis (JA-CASS) were defined as the 75th percentile of the cumulative distribution for each outcome for parents and children, respectively, who considered the symptom state as satisfactory18,20. In the second approach, we determined the JA-PASS/JA-CASS cutoffs by plotting receiver-operating characteristic (ROC) curves and identifying the cutoffs that yielded the smallest number of false positives and false negatives20.
To investigate the relative effect of covariates on JA-PASS/JA-CASS, a logistic regression analysis was performed, entering explanatory variables (Table 1) that showed significant results in univariate tests and JA-PASS or JA-CASS positivity (i.e., a positive answer to the satisfaction question) as the outcome variable. In the case of 2 or more variables being highly correlated (i.e., redundant), only the most relevant variable was included in the model. Before performing multivariate analysis, continuous variables were converted to binary variables using the cutoffs obtained through the ROC curve analysis. Cases with missing variables were excluded. Using a backward selection procedure, predictor variables that were significantly associated with the outcome were identified. The effect was expressed in terms of OR and 95% CI; statistical significance was tested by means of the likelihood-ratio test. The area under the ROC curve (AUC-ROC) of the best-fitting model was used as an indicator of its predictive ability.
Demographic and disease severity characteristics of children with juvenile idiopathic arthritis at first visit according to JA-PASS or JA-CASS. Values are the median (interquartile range) unless otherwise indicated.
Construct and discriminative validity of the satisfaction question were assessed by comparing the proportion of JA-PASS/JA-CASS–positive patients (i.e., patients who responded “yes” to the satisfaction question) and JA-PASS/JA-CASS–negative patients (i.e., patients who responded “no” to the satisfaction question) who met the criteria for inactive disease7 or minimal disease activity12 in JIA, or were judged subjectively as being in remission, continued activity, or disease flare by the parent, the child, or the physician. It was anticipated that for the satisfaction question to demonstrate good construct/discriminative validity there should be relatively more JA-PASS/JA-CASS–positive cases among patients with inactive disease, remission, or minimal disease activity, and relatively more JA-PASS/JA-CASS–negative cases among patients with continued disease activity or flare.
Reliability of the cutoff values of outcome measures that defined the JA-PASS and JA-CASS was determined by evaluating the stability of the cutoffs longitudinally at first, second, and third study visits. Differences in longitudinal cutoffs were interpreted qualitatively.
All statistical tests were 2-sided; a p value < 0.05 was considered statistically significant. The statistical packages used were Statistica (StatSoft Corp., Tulsa, OK, USA) and Stata release 11 (Stata Corp., College Station, TX, USA).
RESULTS
Patient characteristics
The parents of 584 children with JIA completed the JAMAR and responded to the question about satisfaction with illness outcome. The same assessments were performed independently by 343 children aged ≥ 7 years. The main demographic and clinical features as well as the values of parent-reported, child-reported, and physician-reported outcome measures and acute-phase reactant levels at study entry are presented in Table 1.
Assessment of JA-PASS and JA-CASS status
Of the 584 parents, 385 (65.9%) indicated that they considered their child in JA-PASS. Of the 343 children, 236 (68.8%) indicated that they considered themselves in JA-CASS. The percentage of missing data, that is, the percentage of the whole JAMAR completers (618 parents and 354 children) who did not respond to the question about satisfaction with illness outcome was 5.5% for JA-PASS and 3.1% for JA-CASS. The characteristics of JA-PASS/JA-CASS–positive and JA-PASS/JA-CASS–negative patients are shown in Table 1. Patients in JA-PASS had longer disease duration and were older than patients not in JA-PASS. There were no differences in the remaining demographic features between JA-PASS/JA-CASS–positive and –negative patients. Patients in JA-PASS or JA-CASS had lower (better) values for all parent/child-reported outcomes, physician-reported outcomes, and acute-phase reactant levels than did patients not in JA-PASS or JA-CASS.
Effect of covariates on the JA-PASS and JA-CASS
In stepwise logistic regression analyses (Table 2), being in JA-PASS was independently associated with absence of active joints, better rating of overall well-being, normal physical function, and longer disease duration. Significant contributors to JA-CASS included absence of active joints and better overall well-being and physical health. The AUC-ROC of the model was good (> 0.8) for both JA-PASS and JA-CASS. Notably, the pain rating was not included in multivariate models because it was highly correlated with the rating of overall well-being.
Best-fitting models obtained through logistic regression procedures with the presence of JA-PASS or JA-CASS status as dependent variables. The area under the receiver-operating characteristic curve of the model was 0.88 for JA-PASS and 0.83 for JA-CASS.
Cutoff values of outcome measures for being in JA-PASS or JA-CASS
The cutoff values of parent-reported, child-reported, and physician-reported outcome measures and acute-phase reactant levels, defined by the 75th percentile of the cumulative distribution and the ROC curve method, that corresponded with a symptom state considered satisfactory by parents or children are presented in Tables 3 and 4. The percentage of false-positive misclassification when applying the 75th percentile threshold and the sensitivity and specificity of ROC curve cutoffs are also reported. Overall, cutoffs were lower for JA-CASS than for JA-PASS. The percentage of patients with scores below the 75th percentile or ROC curve thresholds but still reporting not being in JA-PASS/JA-CASS, that is, the percentage of patients false-positively misclassified as being JA-PASS/JA-CASS–positive19, ranged from 10.1% to 24.2% for JA-PASS and from 13.6% to 26.6% for JA-CASS. The variable that provided the greatest percentage of false-positively misclassified patients for both JA-PASS and JA-CASS was assessment of psychosocial health.
JA-PASS cutoff values for parent-reported and physician-reported outcome measures and acute-phase reactant levels as defined by the 75th percentile of the cumulative distribution of each specific measure or ROC curve analysis, percentage of patients with scores below these cutoffs but still reported by their parents not being in JA-PASS (i.e., false positives), and sensitivity and specificity of ROC cutoffs.
JA-CASS cutoff values for parent-reported and physician-reported outcome measures and acute-phase reactant levels as defined by the 75th percentile of the cumulative distribution of each specific measure or ROC curve analysis, percentage of patients with scores below these cutoffs but still reported by their parents not being in JA-CASS (i.e., false positives), and sensitivity and specificity of ROC cutoffs.
The surprisingly low rate of misclassification yielded by physician global rating is worth mentioning. The best tradeoff between sensitivity and specificity was provided by parent-reported, child-reported, and (only for JA-PASS) physician-reported global ratings, whereas joint counts revealed good specificity but low sensitivity, and acute-phase reactant levels showed fair sensitivity but poor specificity. The poor performance of psychosocial health assessment was confirmed by its modest levels of sensitivity and specificity. JA-PASS and JA-CASS thresholds were higher for most variables in patients with systemic arthritis than in those with oligoarthritis and polyarthritis (results not shown).
Evaluation of construct and discriminative ability of the JA-PASS and JA-CASS
The proportion of JA-PASS/JA-CASS–positive and JA-PASS/JA-CASS–negative patients who met the criteria for inactive disease7 or minimal disease activity12 in JIA or were judged as being in remission, continued activity, or disease flare by the parent, the child, or the physician is reported in Table 5. As expected, there were relatively more JA-PASS/JA-CASS–positive cases among patients with inactive disease, remission, or minimal disease activity, and relatively more JA-PASS/JA-CASS–negative cases among patients with continued activity or disease flare.
Proportions of patients who had inactive disease or minimal disease activity according to established criteria or were judged subjectively as being in remission, continued disease activity, or disease flare by parents, children, and physicians, sorted by JA-PASS and JA-CASS status. Values are the number/total number (%).
Evaluation of reliability of the JA-PASS and JA-CASS
Reliability of cutoffs of outcome measures that defined the JA-PASS and JA-CASS was determined by evaluating their stability over time. This analysis was made by evaluating the JA-PASS or JA-CASS estimates, obtained with the 75th percentile approach, in 171 children who were reported as being in JA-PASS by their parents in 3 consecutive visits, and in 85 children who reported themselves as being in JA-CASS in 2 consecutive visits. The median (IQR) interval between first and second visit and between second and third visit was 5.8 months (range 3.3–7.3) and 6 months (range 3.6–8), respectively. As shown in Table 6, the cutoffs remained stable throughout visits.
JA-PASS and JA-CASS cutoff values for parent-reported, child-reported, and physician-reported outcome measures and acute-phase reactant levels at baseline, second, and third visit in children who were reported as being in JA-PASS by their parents in 3 consecutive visits or reported themselves as being in JA-CASS in 2 consecutive visits (75% percentile approach).
DISCUSSION
The purpose of our study was to use the question about the satisfaction with illness outcome to define the cutoff values of JIA outcome measures that corresponded with a disease state deemed acceptable by parents and children. The validity of the satisfaction question was tested by assessing its discriminative and construct validity. The reliability of cutoff values of outcome measures was examined by evaluating their stability over time.
In a cross-sectional population of patients with JIA seen in the context of standard clinical care, the majority of parents (65.9%) and children (68.9%) declared being satisfied if the child’s current condition remained stable/unchanged for the next few months. Multivariate analyses revealed that the major contributor to being in both JA-PASS and JA-CASS was the absence of active joints. This underscores the importance of joint symptoms in driving parents’ and children’s judgment regarding the acceptability of the disease status. Other disease-related covariates independently associated with both JA-PASS and JA-CASS were overall well-being and physical health status. That the independent contributors were similar among parents and children adds to the clinical relevance and reliability of the JA-PASS/JA-CASS concept. Notably, pain rating was not included in multivariate models because it was highly correlated with assessment of well-being. The association between being in JA-PASS and a longer disease duration may reflect parents’ adaptation to the disease burden over time or a greater likelihood of achievement of a satisfactory disease control in the later stages of the illness.
We estimated cutoff values for parent-reported, child-reported, and physician-reported outcome measures, and acute-phase reactant levels that defined the JA-PASS and JA-CASS, using both the 75th percentile method and the ROC curve analysis. Overall, JA-PASS and JA-CASS cutoffs were similar when obtained by either method, which ensures their robustness across statistical approaches. Threshold levels for JA-CASS were lower than those for JA-PASS, suggesting that children may require a more stringent disease control to feel satisfied. JA-PASS and JA-CASS cutoffs for parent/child global assessment of well-being or disease activity and physician’s global assessment of disease activity were similar. This observation may reflect a similar perspective and expectations of doctors, parents, and children regarding the achievement of satisfactory disease activity and health status. Notably, parent, child, and physician global ratings were the variables less affected by false-positive misclassification and revealed the best tradeoff between sensitivity and specificity. That the assessment of psychosocial health yielded the highest frequency of misclassification and had the poorest performances in terms of sensitivity and specificity is not surprising, because this measure is affected by many factors unrelated to the disease28,29. An alternative hypothesis is that parents and children tend to relate the burden of chronic arthritis to physical rather than psychosocial impairment. Neither the JA-PASS nor the JA-CASS cutoffs fit the criteria for inactive disease7, whereas both fell within the thresholds of MDA12. This is due to the greater stringency of the former criteria, which require the formal absence of clinical and laboratory evidence of disease activity. Threshold levels for both JA-PASS and JA-CASS were higher in systemic arthritis than in the other ILAR categories, reflecting the greater burden of the systemic JIA subset.
Construct and discriminative validity of the satisfaction question were established by showing that there were relatively more JA-PASS/JA-CASS–positive cases among patients with inactive disease, remission, or minimal disease activity, and relatively more JA-PASS/JA-CASS–negative cases among patients with continued activity or disease flare. Evidence of reliability of the cutoff values of outcome measures that defined the JA-PASS and JA-CASS was provided by the finding that cutoffs remained stable over time.
Several potential caveats should be taken into account in interpreting our study. The wording of the question about satisfaction with the symptom state was taken from wording suggested by the OMERACT group, with no previous validation in pediatric patients. We recognize that potential differences in the assessment of disease states of adults with chronic arthritis and the complex developmental situation of children with JIA and their parents were not considered. Further, the effects of age-related or development-related differences in the ability of children to understand the states of diseases were not taken into account. Because the concept of acceptable symptom state is new in pediatric rheumatology, the specific wording of the question and the timeframe to evaluate the present health state for the next several months should be explored further. It should also be investigated whether criteria other than the satisfaction question could be more suitable to define the acceptable symptom state in children with chronic arthritis. The study sample included few patients with psoriatic arthritis and enthesitis-related arthritis. Thus, our findings may be of limited value for children with these JIA subtypes. We did not examine the role of coping, self-efficacy, anxiety, depression, and treatment expectations. Because perceptions of the larger beneficial effects of treatment will influence parent and patient concepts of acceptable symptom state, it is possible that estimates may differ in patients who have experienced the benefits of anti-tumor necrosis factor therapies. This evaluation could not be performed, however, owing to the insufficient number of observations.
Our study showed that the JA-PASS/JA-CASS is a relevant concept for children with JIA and their parents and constitutes a valid outcome measure that is potentially applicable in routine practice as well as in clinical trials. Application of JA-PASS/JA-CASS may provide practicing pediatric rheumatologists with a feasible and valuable instrument that may help improve the quality of clinical care.
- Accepted for publication November 3, 2011.
