Abstract
Objective. We studied an inception cohort of children with juvenile idiopathic arthritis (JIA) to (1) identify distinct disease courses based on changes over 5 years in 5 variables prioritized by patients, parents, and clinicians; and (2) estimate the probability of a severe disease course for each child at diagnosis.
Methods. Assessments of quality of life, pain, medication requirements, patient-reported side effects, and active joint counts were scheduled at 0, 6, 12, 18, 24, 36, 48, and 60 months. Patients who attended at least 6 assessments were included. Multivariable cluster analysis, r2, and silhouette statistics were used to identify distinct disease courses. One hundred candidate prediction models were developed in random samples of 75% of the cohort; their reliability and accuracy were tested in the 25% not used in their development.
Results. Four distinct courses were identified in 609 subjects. They differed in prioritized variables, disability scores, and probabilities of attaining inactive disease and remission. We named them Mild (43.8% of children), Moderate (35.6%), Severe Controlled (9%), and Severe Persisting (11.5%). A logistic regression model using JIA category, active joint count, and pattern of joint involvement at enrollment best predicted a severe disease course (Controlled + Persisting, c-index = 0.87); 91% of children in the highest decile of risk actually experienced a severe disease course, compared to 5% of those in the lowest decile.
Conclusion. Children in this JIA cohort followed 1 of 4 disease courses and the probability of a severe disease course could be estimated with information available at diagnosis.
Shared decision making of physicians and parents is essential to ensure optimal outcomes for children with juvenile idiopathic arthritis (JIA). Sharing decisions requires effective communication of the expected disease course and the risks associated with treatment. The course of JIA is hard to explain to patients and their parents. Previous studies have concentrated on the number of affected joints1 or episodes of disease activity2 over time, and on predicting remission, functional status, or radiological evidence of joint damage at variable times after diagnosis3,4,5.
In a previous study we asked adolescents with JIA, parents of children with JIA, and clinicians who care for children with JIA what was most important to them in describing the clinical course of the disease, and what terms should be used to communicate disease course6. Five variables were chosen: participant-defined quality of life, pain, medication requirements, medication side effects, and the number of active joints. Plain descriptive terms that conveyed the overall severity of the disease course were preferred.
In our current study we used data from the Research in Arthritis in Canadian Children emphasizing Outcomes (ReACCh-Out) prospective inception cohort to (1) describe the clinical course of JIA based on changes in these 5 variables during the 5 years after diagnosis, and (2) develop a method to estimate the probability that an individual child will follow a severe disease course, using information available at diagnosis.
MATERIALS AND METHODS
Patients
From 2005 to 2010, the ReACCh-Out study recruited consecutive patients newly diagnosed with JIA at 16 pediatric rheumatology centers7,8. Study visits were scheduled at 0, 6, 12, 18, 24, 36, 48, and 60 months after enrollment to collect demographic and clinical information, juvenile arthritis core variables9, medications, and quality-of-life measures10,11. We included subjects who attended at least 6 of the 8 study visits by May 30, 2012, and had at least 1 value recorded for each of the 5 variables detailed below. This study was reviewed and approved by the University of British Columbia Children’s and Women’s Health Centre Research Ethics Board (number H14-01784).
Variables
Participant-defined quality of life was the answer of the child to the following question from the Quality of My Life Questionnaire11: “Considering my health, my life is…” from 0 = the worst, to 10 = the best. The child made a mark in a horizontal 10-cm visual analog scale (VAS), assisted by the parent as needed.
Pain during the past week was marked by the child or parents (depending on the child’s maturity) in a 10-cm horizontal VAS from 0 = no pain, to 10 = worst pain imaginable10.
Active joint count was the number of joints that were swollen or showed painful limitation of movement when examined by a pediatric rheumatologist (from 0 to 71)12.
Medication requirements and medication side effects are top priorities for patients, parents, and clinicians6, but there are no validated scales to measure these constructs in JIA. The authors developed draft scales based on data available in the ReACCh-Out cohort and convened focus groups of youth with JIA (n = 3), parents of children with JIA (n = 3), and clinicians (2 pediatric rheumatologists, 1 nurse) to evaluate them13. The scales were modified according to the feedback received, for use in our study (see Detail of statistical methods, Supplementary Material, Detail of Scales, available with the online version of this article).
Each current antirheumatic medication received at each visit was given a weight, and weights were added to obtain a medication requirements score: corticosteroid joint injection = 1, nonsteroidal antiinflammatory drugs = 1.5, systemic corticosteroid = 3, single disease-modifying antirheumatic drug (DMARD) = 3, multiple DMARD = 5, biologic = 8.
Ten symptoms reported by the patients/parents in the Juvenile Arthritis Quality of Life Questionnaire (JAQQ)10 as due to arthritis or its treatment were selected by the authors as possible side effects. The frequency of these symptoms was summarized into a 0-to-10 medication side effects scale, from 0 = no side effects, to 6 = 2 side effects occurring 50% or more of the time, and 10 = side effect resulting in death or disability.
Description of clinical course
Our statistical analyses and treatment of missing data are described in full in the Supplementary Material (available with the online version of this article). We summarize the important features here. Missing values for the child’s pain were imputed first, using linear regression based on the parent’s guess. Other missing values were imputed next. We tested several imputation methods by purposefully deleting values in patients with no missing values and selected the method that best approximated the deleted values; it was a type of regression imputation14,15,16.
After imputation, we used multivariable clustering to group subjects according to their degree of similarity in the 5 variables over 5 years, irrespective of other characteristics. We tried K-means, K-medoids, agglomerative clustering, and divisive clustering17,18,19. Silhouette coefficients and r2 statistics were compared within each method to measure the relative homogeneity of clusters in relation to others and help select the most appropriate number of clusters19. We then used silhouette coefficients to help select the best clustering method and the adjusted Rand index to assess stability of the clustering20. We used descriptive statistics and chi-square and Kruskal-Wallis tests to highlight differences across clusters and then assessed their perceived clinical utility. We wanted to determine whether the clusters reflected clinicians’ experiences of the course of JIA, and whether the clusters would be easily understood by patients and parents.
After reviewing clustering results, we conducted posthoc analyses to assess whether children with a severe persisting disease course were grouped that way because of an overlying chronic pain syndrome rather than persisting disease activity.
Prediction of a severe disease course
We ordered the clusters (disease courses) according to clinically perceived severity and explored different targets for prediction, e.g., predicting only the most severe course or the 2 most severe courses, etc. We considered explanatory variables (predictors) associated with disease outcomes in previous studies3,4,5 and gave preference to variables easily accessible at diagnosis. Logistic regression was used to screen explanatory variables using a modified version of backward elimination. “Full” models with all 52 explanatory variables were fit to each response outcome and the 28 variables with the greatest effects on prediction ability were retained.
We tried 5 binary classification methods: logistic regression, classification tree, random forest, neural networks, and k-nearest neighbor18, and different combinations of predictors. Subjects were split 75%/25% into “training” and “test” sets. Models were developed in the training set and their predictive ability was measured in the test set18 using 3 metrics: c-index, a maximum likelihood statistic, and a Pearson statistic based on optimal 3-group risk stratification. Higher values indicate better predictive ability. The process was repeated 50 times for each of 100 candidate models with random selection of the 75% and 25% subsets to assess model stability and compute 95% CI for the 3 metrics. The best model was refit to the full dataset to estimate the final c-index.
Ideally, variable selection and model comparison could be automated in a single round of data splits instead of using 2 separate rounds of splitting for each step. However, variable selection required some judgment and it was easier to implement in 2 steps. Sample-size considerations prevented us from using the 3-way split into training/validation/test samples that is sometimes recommended18. Our multiple 2-way splits allowed assessment of the stability of our performance metrics and served as a very good substitute.
RESULTS
Patients
Between January 2005 and December 2010, the ReACCh-Out study recruited 1497 children with JIA. As of May 30, 2012, 640 subjects had attended at least 6 study visits, of which 609 provided enough data (Figure 1, Table 1). Compared to excluded subjects, the included subjects were younger (median of 8.4 yrs vs 10.4), had a shorter time from onset to diagnosis (3.7 vs 4.6 mos), and a more severe disease at enrollment [physician’s global assessment (PGA) 2.8 vs 2.4 cm, 2 vs 1 active joints]. Polyarthritis and systemic JIA formed a larger proportion of included subjects (p = 0.05; chi-square test; Supplementary Table 1, available with the online version of this article).
Physician-reported data were available for > 95% of the 4144 study visits attended by these 609 patients, and patient-reported data for 82%–85% of the visits. Missing data at enrollment are reported in Table 1 and missing data at each subsequent visit are reported in Supplementary Table 2, available with the online version of this article. Our minimum requirement was 1 value for each of the 5 prioritized variables (5/40 possible values), but 99% of included subjects had at least 20 valid data points.
Description of clinical course
The values of silhouette coefficients, r2 statistics, and adjusted Rand index were best, and very similar, for the K-means 4-cluster and 5-cluster models. Based on examination of values of the 5 variables across clusters and discussions among the investigators, we chose the 4-cluster model because it was simpler, easier to convey to families, and more clinically meaningful.
Figure 2 presents lay-language descriptions for the 4 disease courses; Figure 3 presents Kaplan-Meier curves for attaining inactive disease and remission without medications, and flare-free survival for each course. Supplementary Figure 1 (available with the online version of this article) shows mean values and 95% CI for prioritized variables, disability scores21, juvenile arthritis disease activity scores22, sedimentation rate, PGA, and parent’s global assessments.
A mild disease course (n = 267, 43.8%, Figure 2) was characterized by minimal effect on quality of life throughout the 5 years after diagnosis. There was mild initial pain and disability that were quickly controlled. Children in this group had the lowest medication requirements and patient-reported medication side effects, and the greatest chance of attaining inactive disease and remission without medications, with the lowest chance of flares (Figure 3).
A moderate disease course (n = 217, 35.6%) was characterized by mild to moderate initial effect on quality of life, pain, and disability, with subsequent improvement. Children in this group had moderate treatment requirements and moderate side effects. Compared to the mild course, it took longer to attain inactive disease and remission without medications, with a higher probability of flares.
Children with a severe controlled disease course (n = 55, 9.0%) had levels of quality of life and pain similar to the moderate course, but very high active joint counts at presentation, and required intensive treatment. Levels of disability and PGA scores at presentation were high, but decreased quickly. The probabilities of attaining inactive disease and remission without medications were lower than for children in the mild and moderate disease courses.
Children with a severe persistent disease course (n = 70, 11.5%) experienced a moderate effect on their quality of life and persistent pain and disability, even though active joint counts were only moderate at presentation. These children were treated as intensively as those in the severe controlled course, but the effect on quality of life and pain remained problematic. They experienced the most side effects. The probabilities of attaining inactive disease and remission without medications were similar to those of the children with the severe controlled course.
Posthoc analyses showed that relative to subjects in the severe controlled disease course, the group of children with a severe persistent course (1) had similar probabilities of attaining inactive disease; (2) had a comparable proportion of subjects with at least 1 active joint at subsequent visits; (3) had comparable frequency of involvement of neck, wrist, hip, and ankle joints; (4) had a weaker overall correlation of pain intensity and active joint count (Spearman correlation of 0.30 vs 0.46); and (5) showed no differential effect on the Psychosocial and Symptoms domains of the JAQQ (see Posthoc analyses, Supplementary Material, available with the online version of this article).
As expected, children who followed an oligoarthritis extended course were more often classified as moderate or severe, relative to children who followed an oligoarthritis persistent course (data not shown).
As a sensitivity analysis, we repeated clustering in children with complete data. The results suggested our data imputation was not a source of bias on the clustering of subjects. Supplementary Figure 2 (available with the online version of this article) describes how the above disease courses relate to clusters based on juvenile arthritis core variables and to the alternative 5-cluster option.
Prediction of a severe disease course
Table 1 reports baseline characteristics of subjects in each disease course; Figure 4 shows the performance of the best prediction model; and Table 2 compares it with other candidate models.
The best performing model targeted severe controlled and severe persisting courses combined and was derived using logistic regression; 91% of children assigned to the highest decile of risk by this model actually had a severe course, compared to 5% of those assigned to the lowest decile. If instead of deciles of risk, the model was used to assign low, moderate, and high risk, this model assigned 436 subjects (71.5%) to a low-risk group with < 20% probability of a severe disease course, 100 subjects (16.4%) to a moderate-risk group with a 20% to 60% probability, and 73 subjects (12.0%) to a high-risk group with > 60% probability.
The signs of the coefficients in the final model shown in Figure 4 may not reflect the direction of the associations in Table 1, but they are correct in the context of the full model. The model was better at predicting the severe controlled course than the severe persisting course, but offered the best performance to predict them together. This final model had an in-sample c-index of 0.87, comparable to values computed on the data splits (mean c-index 0.85). Despite its apparent complexity, the necessary calculations can be handled by any modern smart phone; an online calculator can be found here: https://shiny.rcg.sfu.ca/jia-sdcc/. A model developed using random forest methods had similar performance statistics (Table 2). Models predicting the severe persisting course directly were not as robust and often missed severe controlled cases (Supplementary Figure 3, available with the online version of this article). Supplementary Tables 3 and 4 report associations of predictor variables with a severe disease course.
To assess the potential effect of missing data imputation on our prediction model, we conducted a sensitivity analysis including only subjects who had complete data for the predictor variables at enrollment. There were few missing data values for predictors at enrollment and the probabilities of a severe disease course calculated using the model derived from the complete dataset were very similar to those derived from the model based on the imputed dataset (see Supplementary Material, Detail of Statistical Methods, available with the online version of this article).
DISCUSSION
Motivated by the importance of shared decision making by physicians and parents, we undertook to describe the course of JIA using 5 variables prioritized by patients, parents, and clinicians6. We found that many children followed a mild disease course (43.8% of children) with minimal disease effect and minimal treatment requirements. About one-third of children (35.6%) followed a moderate course with initial mild to moderate pain and effect on quality of life that subsequently stabilized. There were 2 expressions of a severe disease course: a severe controlled course (9%) that presented with many active joints and improved with intensive treatment; and a severe persisting course that presented with some active joints but had persistent moderate pain and effect on quality of life despite treatment (11.5%).
We chose disease course names that were simple and informative for patients and parents. They are relative to each other, because there is no accepted definition of severe JIA or absolute yardstick to measure severity. Children in these 4 groups also differed in disability scores, PGA, and parent global assessments, and probabilities of attaining inactive disease and disease remission during the 5 years after diagnosis. We found little evidence that a chronic pain syndrome, rather than persistent disease activity, was the main reason for a severe persisting disease course.
We then set out to develop a system to estimate the probability of a severe disease course at diagnosis for each patient. A logistic regression model targeted at severe controlled and severe persisting courses together outperformed other candidate models and had a c-index of 0.87. The c-index varies from 0 to 1, where 0.5 corresponds to chance alone and 1.0 is perfect prediction. Values above 0.7 are considered helpful. For reference, the widely used Framingham Cardiovascular Risk Score has c-index values of 0.75–0.80, depending on the cohort23,24.
We considered the overall course of 5 patient-relevant variables over 5 years, instead of only the number of active joints1 or the number of cycles of active disease2. We chose these variables over the juvenile arthritis core variables because we wanted the resulting course groups to reflect the most important issues for patients and parents. We used the Quality of My Life scale as a measure of quality of life, instead of the JAQQ, parent global, or Childhood Health Assessment Questionnaire, because in our previous study these scales were seen as distinct and the Quality of My Life scale had the most face validity for patients and parents6. Our secondary analyses suggest that using the juvenile arthritis core variables may lead to different groups.
We considered many statistical methods and different groupings of explanatory variables to select the model that best predicted a severe JIA course. We measured performance in data not used for model development to ensure the model had the best predictive ability and did not merely overfit the training data. The selected model (and associated risk calculator) may be a good candidate for the “clinically usable prediction rule” called for by van Dijkhuizen and Wulffraat25.
The Juvenile Arthritis Disease Activity Score 3 (JADAS3), the sum of PGA, parent global assessment, and up to 10 active joints, has been increasingly used in JIA22. JADAS3 varied predictably across the 4 disease courses, but PGA, parent global, or the JADAS3 at enrollment added little predictive value to our prediction model.
It is important to consider 5 possible limitations. First, we excluded many subjects because they attended < 6 study visits. This was a calculated compromise, to include a large enough number of subjects who each had a sufficient followup period (at least 3 yrs). Included subjects were younger (perhaps owing to transferring of adolescents to adult care) and appeared to have a more severe disease at enrollment (perhaps subjects with milder disease missed more study visits). These differences, and the possibility that children with milder JIA may not be referred to a pediatric rheumatologist, suggest that the proportion of children with a severe disease course in the whole JIA population may be less than the reported 20.5%. As is the case in most JIA inception cohorts, the number of subjects with systemic, psoriatic, or rheumatoid factor–positive arthritis was small. It will be interesting to confirm findings in larger groups of children with these JIA categories.
Second, there were substantial missing data requiring imputation; we used a method of imputation that provided values demonstrably close to actual values that had been purposefully deleted, and our sensitivity analyses suggest data imputation was not a source of bias.
Third, it could be argued that using baseline values of a variable to predict a disease course partly defined by changes in that variable over the next 5 years is tautological. On the other hand, it would make no clinical sense to exclude from prediction models variables that are easily ascertained at diagnosis and predict the course. In the final model this potential tautology applied only to the number of active joints.
Fourth, although our medication requirements and side effects scales were assessed in focus groups with patients, parents, and clinicians13, they have not been formally validated. Rather than ignoring these important constructs, we chose to use the available scales. We did vary the weighting of scale items in secondary analyses, and it did not substantially change our findings.
Fifth, the described disease courses are inextricably linked to the treatments received by patients in the cohort8; different disease courses might have been observed if patients had been treated in a substantially different way.
The implication for practice of our study is that in practical terms, when counseling newly diagnosed families in Canada, a physician may describe the 4 JIA disease courses and give a reasonable estimate of the chance that the child will follow a severe course. This information can then be considered alongside the inconveniences and possible side effects of treatment to help arrive at a well-informed shared decision. Better knowledge of what to expect from the disease may decrease the uncertainty and anxiety of parents facing a diagnosis of JIA in their child. The implication for research is that because our cohort was similar to cohorts in other Western countries, our results may apply to children in those countries, but this will require confirmation in a local cohort. It will be important to apply our analysis methods to other JIA cohorts to understand differences across settings. Given the importance that families place on medication requirements and side effects, there is a need to validate scales for assessing them.
Based on changes over 5 years in 5 variables chosen by patients, parents, and clinicians, children with JIA in the ReACCh-Out cohort were grouped into 4 clinical courses. Most children followed a mild or a moderate course, but a minority followed 1 of 2 severe courses. One was controlled with treatment despite very high initial active joint counts, and 1 had a protracted effect on quality of life and pain, despite intensive treatment and moderately decreasing joint counts. A prediction model that combined JIA category and active joint count with other features usually known at diagnosis was able to estimate the risk of a severe JIA course in each patient with accuracy similar to methods for predicting cardiovascular disease used in current practice guidelines.
ONLINE SUPPLEMENT
Supplementary material accompanies the online version of this article.
APPENDIX 1
List of study collaborators. The ReACCh-Out Investigators. Here follow the additional investigators in the Research in Arthritis in Canadian Children emphasizing Outcomes Study: Roxana Bolaria, Katherine Gross, Stuart E. Turvey, David Cabral, Kristin Houghton, Kimberly Morishita, Ross Petty, University of British Columbia; Janet Ellsworth, the Stollery Children’s Hospital and University of Alberta, Edmonton; Nicole Johnson, Paivi Miettunen, Heinrike Schmeling, the Alberta Children’s Hospital and University of Calgary, Calgary; Alan M. Rosenberg, Royal University Hospital and University of Saskatchewan, Saskatoon; Kiem Oen, Winnipeg Children’s Hospital and University of Manitoba, Winnipeg; Maggie Larché, McMaster University, Hamilton, Ontario; Brian M. Feldman, Deborah M. Levy, Ronald M. Laxer, Debbie Feldman, Lynn Spiegel, Rayfel Schneider, Shirley M.L. Tse, Earl Silverman, Bonnie Cameron, Rae S.M. Yeung, Hospital for Sick Children and University of Toronto, Toronto, Ontario; Johannes Roth, Michele Gibbon, Karen Watanabe Duffy, Ciarán M. Duffy, Children’s Hospital of Eastern Ontario and University of Ottawa, Ottawa, Ontario; Anne-Laure Chetaille, Jean Dorval, Centre Hospitalier Universitaire de Laval and Université Laval, Quebec City; Gilles Boire, Alessandra Bruns, Centre Hospitalier Universitaire de Sherbrooke and Université de Sherbrooke, Sherbrooke; Rosie Scuccimarri, Sarah Campillo, Gaëlle Chédeville, Claire LeBlanc, McGill University Health Centre and McGill University, Montreal; Elie Haddad, Claire St. Cyr, CHU Ste. Justine and Université de Montréal, Montréal, Quebec; Bianca Lang, Suzanne E. Ramsey, Elizabeth Stringer, Adam M. Huber, IWK Health Centre and Dalhousie University, Halifax, Nova Scotia; Paul Dancey, Janeway Children’s Health and Rehabilitation Centre and Memorial University, Saint John’s, Newfoundland, Canada.
Footnotes
Funded by a grant from the Canadian Initiative for Outcomes in Rheumatology Care, Canadian Rheumatology Association. The Research in Arthritis in Canadian Children Emphasizing Outcomes (ReACCh-Out) cohort was funded by a New Emerging Team research grant from the Canadian Institutes of Health Research. Dr. Guzman was funded by a Clinical Investigator Award from the Child and Family Research Institute, Vancouver, British Columbia, Canada.
- Accepted for publication October 25, 2016.