Abstract
Objective. To date, there are no standardized disease activity tools for systemic juvenile idiopathic arthritis (sJIA). We developed a core set of disease activity measures for sJIA.
Methods. We conducted a validation study in patients with sJIA recruited from 3 Canadian institutions. Disease activity scores were based on questionnaires, clinical factors, and laboratory measures. The physician’s global assessment was our criterion standard. We determined the strength of association of each item with the criterion standard. We then surveyed international experts to determine the top 10 items. Finally, we used the experts’ responses to generate a proposed core set of disease activity measures.
Results. We enrolled 57 subjects — 26 with moderately or severely active disease, and 31 with mildly active or inactive disease. Items that most strongly correlated with the criterion standard were number of active joints (r = 0.79), parent’s global assessment of disease activity (r = 0.53), erythrocyte sedimentation rate (ESR; r = 0.62), and C-reactive protein (CRP; r = 0.61). The response rate from international experts was 82% (154/187). Items with the most votes, in descending order, were number of active joints, number of days with fever in the preceding 2 weeks, patient’s and parent’s global assessments of disease activity, sJIA rash, ESR, CRP, and hemoglobin level.
Conclusion. We propose a core set of items for measuring disease activity in sJIA. Future research should be aimed at further validation of this core set in the international context.
While the systemic category of juvenile idiopathic arthritis (sJIA) accounts for only 5–15% of JIA in North America and Europe1, it leads to disproportionate morbidity and mortality among children with rheumatic diseases. Although the longterm prognosis of sJIA is highly variable, about half the patients develop an unremitting disease course2,3. In a prospective study of disease activity, functional disability, and articular damage in JIA, children with sJIA had significantly higher progression of severe disability4.
Developing standardized tools to measure disease activity in pediatric rheumatic diseases is challenging but critical to patient care and clinical research5, particularly in determining the effect of disease and response to treatment.
To date, to our knowledge no validated, standardized tools for measuring disease activity in sJIA have been developed. Most randomized controlled trials in sJIA have measured disease response using the JIA American College of Rheumatology 30 (JIA ACR30) core set, which includes 6 core criteria: active joint count, limitation of motion joint count, physician’s global assessment (PGA), patient’s/parent’s global assessment, Childhood Health Assessment Questionnaire (CHAQ), and acute-phase reactant level6. Two randomized trials of canakinumab and 1 of tocilizumab (TCZ) in the treatment of sJIA used the JIA ACR30 in combination with resolution of fever to measure response7,8. Another trial of rilonacept in sJIA defined disease response by a composite of (1) improvement in the JIA ACR30 score, (2) absence of fever ≥ 38.5°C in the previous 2 weeks, and (3) at least 10% taper in systemic corticosteroids from baseline in patients taking corticosteroids9.
The Juvenile Arthritis Disease Activity Score (JADAS) is a tool developed in 2009 that includes PGA of disease activity, parent’s and patient’s global assessments (PtGA) of well-being, active joint count, and erythrocyte sedimentation rate (ESR)10. Several JIA studies have used various versions of the JADAS to measure disease activity, including a study on catch-up growth in patients with sJIA treated with TCZ11.
More recently, the ACR developed a set of preliminary criteria for defining clinically inactive disease in selected JIA categories. The provisional definition of inactive disease in sJIA requires that each of the following criteria be met: no joints with active arthritis; no fever, rash, serositis, splenomegaly, or generalized lymphadenopathy attributable to JIA; no active uveitis; ESR or C-reactive protein (CRP) within normal limits, or if elevated, not attributable to JIA; PGA of disease activity score of best possible on the scale used; and duration of morning stiffness ≤ 15 min12.
While the PGA, parent, and PtGA scores included in the above tools likely incorporate extraarticular manifestations, there is no tool that quantifies these systemic manifestations. Despite the widespread use of the JIA ACR30 and JADAS in sJIA studies, they have been validated for joint disease only and not for the systemic manifestations unique to sJIA5.
Our objective was to develop a tool to measure disease activity in sJIA. We defined disease activity as the reversible manifestations of disease. A multistep process was designed to develop the tool (Figure 1); its earlier steps have been published13,14. As the first step, 14 patients with sJIA followed at The Hospital for Sick Children were selected by purposive sampling. Interviews with these patients and their parents generated 292 items that were relevant to disease activity13. Next, we surveyed international experts using the Delphi method to determine the 29 most important indicators of disease activity in sJIA14.
In this report, we focused on the final steps of this process, which included (1) examining the measurement characteristics of the 29 items in a sample of patients, (2) re-surveying the experts with the inclusion of data from the validation study, and (3) synthesis of a proposed core set of disease activity measures.
MATERIALS AND METHODS
Validation study
Patients with sJIA were recruited from 3 tertiary care centers in Canada (The Hospital for Sick Children in Toronto, IWK Health Centre in Halifax, and Montreal Children’s Hospital in Montreal). Patients were approached on a consecutive basis at their followup appointments, which could have been at any time since disease onset. Informed consent was obtained for study participation. To meet inclusion criteria, patients had to meet the 2004 International League of Associations for Rheumatology classification criteria for sJIA. Patients could be of any age. Both patients and parents needed to be able to complete the questionnaires in English, and laboratory tests had to be completed within 1 week of completing questionnaires. Approval for the study was obtained through the Research Ethics Board at The Hospital for Sick Children (REB registration number 0020020051) as well as through the ethics boards at Montreal Children’s Hospital and the IWK Health Centre.
The tool used to score patients included 4 components (Supplementary Material, available with the online version of this article): (1) patient questionnaire, with 18 questions addressing pain, fatigue, physical activity level, disease activity, and quality of life; (2) parent questionnaire, with 11 questions regarding global assessment, fever, morning stiffness, sore throat, and chest pain; (3) CHAQ15; and (4) physician assessment, including PGA, plus 10 clinical and 12 laboratory items, and questions regarding changes to medications. Both patient and parent questionnaires were given to participants of all ages; for young patients (no specific age cutoff), parents were asked to assist their children with the patient questionnaire and the CHAQ. Clinical features were assessed by physical examination only, without the use of ultrasound or other imaging modalities. Demographic information was not collected because it was not necessary for inclusion in the data distributed to experts.
We used the PGA of disease activity, measured on a 10-cm visual analog scale (VAS), as our criterion standard. For some of our analyses, we divided patients into 2 groups using the PGA of disease activity score: “moderate or severe disease activity” and “mild disease activity or inactive disease.” Patients who scored ≥ 1.5 out of 10 were considered to have moderately to severely active disease, and those who scored < 1.5 were considered to have mildly active or inactive disease. This cutoff was chosen to be clinically sensible and to divide the patients into 2 roughly equal groups.
The strength of association of each item with the criterion standard was determined by a number of methods. For items measured on a continuous scale (for instance, number of active joints), we determined the strength of association with continuously scored global disease activity by Pearson correlation coefficient. Using the dichotomized criterion, we created receiver-operating characteristic curves (ROC) for each proposed item. We then used the ROC to determine optimal cutoff values that maximized sensitivity and specificity for each item, and the overall diagnostic value of each item by calculating the area under the ROC.
For items measured on a dichotomous scale (for instance, presence or absence of rash), we calculated the sensitivity and specificity when compared to the dichotomized criterion standard. We then used this information to calculate likelihood ratios for a positive test.
We determined that a sample size of 50 subjects would provide a 95% CI of ± 0.14 around a sensitivity of 0.5, and a CI of ± 0.08 around a specificity of 0.9.
Final survey of experts
The results of our validation study were shared with the original group of international experts surveyed in the earlier steps of this process. Experts in sJIA were nominated by the heads of various pediatric rheumatology organizations including the Association of Rheumatology Health Professionals, the British Pediatric Rheumatology Group (now the British Society for Paediatric and Adolescent Rheumatology), the Canadian Pediatric Rheumatology Association, the Pediatric Rheumatology Collaborative Study Group, the Pediatric Rheumatology European Society, and Pediatric Rheumatology International Trials Organization. We instructed the experts to use their own judgment, combined with our results as additional information, to determine their final top 10 items relevant to disease activity. PGA of disease activity was used as the criterion standard and was therefore excluded from the survey. We then tallied the number of votes for each outcome variable.
Synthesis of a proposed core set of items
To determine which items would be included in the core set, we performed Kruskal-Wallis 1-way ANOVA tests on all consecutive pairs of outcome variables, ordered from most to least number of votes by the experts. This analysis did not establish a statistically significant difference between any 2 consecutive items to determine a cutpoint for inclusion in the final set. We then used the data from our validation study to determine an optimal cutoff point, such that the outcome variables with the highest number of votes also needed to have a minimum Pearson correlation of 0.5 with our criterion standard to be included in the core set.
RESULTS
Validation study
Recruitment from the 3 study centers yielded a total of 57 patients with sJIA.
Five patient questionnaires were filled out by the patient and parent together; 9 patient questionnaires were filled out solely by the parent.
Using the cutoff value of 1.5 on the VAS for PGA of disease activity, 26 patients were determined to have moderately to severely active disease, and 31 to have mildly active or inactive disease. The distribution of PGA scores is shown in Figure 2.
The most prevalent manifestation of disease was arthritis, which was present in 33/57 patients (57.9%), with 29 patients having 2 or more active joints. In general, the prevalence of systemic manifestations was relatively low. Seven patients had sJIA rash by physical examination or parental history in the preceding 2 weeks. Chest pain, lymphadenopathy, splenomegaly, and hepatomegaly each occurred in only 1 patient (all different patients). Only 3 patients had fever for 2 or more days in the preceding 2 weeks. Two of them were in the group with moderately or severely active disease. Both of these patients had high disease activity on PtGA and parent’s global assessments, active arthritis (5 joints in 1 patient, 50 joints in the other patient), very elevated inflammatory markers, and anemia. One of these patients had rash on physical examination. The third patient with 2 days of fever was in the group with inactive or mildly active disease and did not have any other systemic or laboratory manifestations of disease.
Clinical items measured on a continuous scale that most strongly correlated with PGA were total number of joints with active arthritis (r = 0.79), and parent’s global assessment (r = 0.53) and PtGA (r = 0.51) of disease activity. For items measured on a dichotomous scale, presence of sJIA rash, lymphadenopathy, hepatomegaly, splenomegaly, and tenosynovitis were all found to have high specificity, but low sensitivity for active disease (Table 1).
Laboratory items most strongly correlated with PGA were ESR (r = 0.62) and CRP (r = 0.61). Based on the ROC generated, an ESR of 23 mm/h and CRP of 5.5 mg/l were found to be optimal cutoff values to differentiate mildly active from moderately or severely active disease (Table 2).
Final survey of experts
The overall response rate from international experts surveyed by the Delphi method was 154/187 (82%), with representation from all organizations (Table 3).
The items selected by expert votes as most important for inclusion in a disease activity measurement tool were, in descending order: number of joints with active arthritis, presence of sJIA rash (by physician assessment or parent history), number of days with fever in the preceding 2 weeks (by parental history), PtGA and parent’s global assessment of disease activity, ESR, CRP, and hemoglobin level (Table 4).
Synthesis of a proposed core set of items
Using the items with the highest number of expert votes and a minimum Pearson correlation of 0.5 with our criterion standard, we propose a possible core set of items for measuring disease activity in sJIA. For continuous variables, we include cutoff points that suggest moderately or severely active disease (rather than mildly active or inactive disease) based on the ROC generated in our validation study. (There is no cutoff point provided for PGA of disease activity because that was used as our standard of comparison.)
Clinical measures:PGA of disease activity (10 cm VAS)
No. joints with active arthritis (best cutoff ≥ 2 joints)
Presence of sJIA rash by physician assessment or parental history in the preceding 2 weeks (best cutoff ≥ 1 day)
No. days with fever attributed to sJIA in the preceding 2 weeks (best cutoff ≥ 2 days)
PtGA of disease activity (10 cm VAS, best cutoff > 0.4 cm)
Parent’s global assessment of disease activity (10 cm VAS, best cutoff > 0.9 cm)
Laboratory factors are CRP (best cutoff > 5.5 mg/l), ESR (best cutoff > 23 mm/h), and hemoglobin (best cutoff < 119 g/l).
A comparison of the above clinical and laboratory variables between the group with mildly active or inactive disease versus that with moderately or severely active disease is shown in Table 5.
DISCUSSION
To date, there is no validated core set of disease response measures that has been developed for sJIA. In this final phase of a multistep design, we conducted a validation study of previously identified disease activity items in a sample of patients with sJIA and conducted a followup survey of international experts to synthesize a core set of items for measuring disease activity in sJIA. A particular strength of this process was the high response rate (82%) by international experts to the survey.
Very few studies have discussed measurement of disease activity in sJIA. One study evaluated the feasibility and utility of data extraction from electronic medical records to establish the relationships between PGA of disease activity and various measures by JIA category16. The authors reported that for patients with sJIA, the items most strongly correlated with PGA were pain score, joint count, and PtGA. However, this study16 did not include measurement of inflammatory markers or the systemic features unique to sJIA.
Some studies have found molecular markers such as ST2 (the receptor for interleukin 33) and MRP8/14 protein complex (Toll-like receptor 4 agonist) to be highly correlated with disease activity17,18. Levels of S100A12, a marker of granulocyte activation, are also elevated in active sJIA19. Higher levels of interleukin 18 have also been implicated in more severe disease, specifically carrying an increased risk for developing macrophage activation syndrome20. While these studies carry great potential for monitoring disease activity with molecular measures, it remains essential that such markers be used in combination with other clinical and laboratory measures.
There are several potential limitations that must be considered when interpreting our results. First, patients were recruited consecutively during followup appointments rather than at initial presentation, disease flare, or hospitalization, when certain manifestations may be more significant markers of higher disease activity. This could explain some discrepancy in items that were important to experts, but not significant in our statistical analysis. For example, “presence of sJIA rash” received 74 expert votes and had a high specificity of 0.97, but a low sensitivity of 0.12, suggesting that it may be an important and specific marker at disease onset or flare, but not as sensitive at other times in the disease course. However, the goal of our study was to propose a tool to measure disease activity over the course of disease, most of which is managed in an outpatient setting, which renders recruitment during followup visits appropriate and more accurate. Further, the experts were made aware of the recruitment setting, which was presumably factored into their voting choices. This set of items may need to be further refined for use in clinical trials, which tend to enroll patients with higher disease activity, at times of disease onset or flare. The low sensitivity for rash and other systemic features such as fever, organomegaly, and lymphadenopathy may also be attributed to the low frequency with which these features were observed in our ambulatory subjects.
Second, although we used PGA as our criterion standard, there is no true gold standard established for measuring disease activity because this remains ambiguous and subjective5. This may explain the discrepancy in cutoff values between PGA that we chose as a criterion, and PtGA and parent global assessments. PtGA and parent assessments likely reflect individual idiosyncrasies; we minimized physician interrater variability in our study by appointing 1 physician at each study site who evaluated and scored all patients from that site.
Third, a PGA score of 1.5 was chosen subjectively as the cutoff point between the 2 groups. We felt it was necessary to provide a cutoff value so that the data provided to the expert voters represented 2 clear groups of patients. Based on preliminary discussions at the time of data collection, it was apparent that there was a reluctance among physicians to give a score of zero even for patients who they felt had completely inactive disease. Therefore, we chose 1.5 as a cutoff that was clinically sensible and also divided the patients into 2 roughly equal groups.
Fourth, our questionnaires inquired about the 2 weeks preceding the recruitment clinic visit. For certain systemic manifestations such as fever and rash, this may be too lengthy a time frame to interpret in conjunction with the level of disease activity observed at the particular clinic visit. As it turns out, the prevalence of these manifestations was low, and would have been even lower if a shorter time interval had been used.
Last, our study was conducted at 3 Canadian sites and may not necessarily be generalizable to other contexts. However, we included experts from recognized national and international specialty groups in our survey to obtain global input.
The development of tools to measure disease activity is essential to the care of patients with chronic diseases and to clinical research. Tools that have been developed for JIA thus far are largely based on joint disease, and do not address the systemic manifestations that render sJIA unique from and generally more severe than other JIA categories. To our knowledge, this multistep study is the first to propose a core set of items (6 clinical and 3 laboratory) for measuring disease activity in sJIA. This is a valuable indicator of disease activity in sJIA that will enhance clinical research in this field. Subsequent research efforts should be directed toward generalizability and validation of this core set in the international context, including evaluating its performance in the recent clinical trials examining disease activity in sJIA.
ONLINE SUPPLEMENT
Supplementary material accompanies the online version of this article.
Footnotes
Earlier phase of this study supported by Pfizer Inc. through an unrestricted research grant.
- Accepted for publication May 10, 2017.