Abstract
Objective. The current Juvenile Idiopathic Arthritis (JIA) Core Set used in randomized controlled trials (RCT) and longitudinal observational studies (LOS) was developed without the input of patients/parents. At the Outcome Measures in Rheumatology (OMERACT) 2016, a special interest group voted to reconsider the core set, incorporating broader input. We describe subsequent work culminating in an OMERACT 2018 plenary and consensus voting.
Methods. Candidate domains were identified through literature review, qualitative surveys, and online discussion boards (ODB) held with patients with JIA and parents in Australia, Italy, and the United States. A Delphi process with parents, patients, healthcare providers, researchers, and regulators served to edit the domain list and prioritize candidate domains. After the presentation of results, OMERACT workshop participants voted, with consensus set at > 70%.
Results. Participants in ODB were 53 patients with JIA (ages 15–24 yrs) and 55 parents. Three rounds of Delphi considering 27 domains were completed by 190 (response rate 85%), 201 (84%), and 182 (77%) people, respectively, from 50 countries. There was discordance noted between domains prioritized by patients/parents compared to others. OMERACT conference voting approved domains for JIA RCT and LOS with 83% endorsement. Mandatory domains are pain, joint inflammatory signs, activity limitation/physical function, patient’s perception of disease activity (overall well-being), and adverse events. Mandatory in specific circumstances: inflammation/other features relevant to specific JIA categories.
Conclusion. Following the OMERACT methodology, we developed an updated JIA Core Domain Set. Next steps are to identify and systematically evaluate best outcome measures for these domains.
The current Juvenile Idiopathic Arthritis (JIA) Core Set to assess efficacy of medications in randomized controlled trials (RCT) and longitudinal observational studies (LOS) was published in 1997 and developed without the input of patients or caregivers1. Researchers have since recognized the importance of garnering the patient/parent perspective to better understand the effect of medical conditions and treatments on health and quality of life.
The Outcome Measures in Rheumatology (OMERACT) is an international, independent entity of health professionals, methodologists, and patient research partners devoted to outcome measures in rheumatology. OMERACT recommends 4 core measurement areas (life impact, pathophysiologic manifestations, resource use, and adverse events including death) to include in RCT and LOS2. OMERACT depicts domains for study inclusion as a 3-layered “onion”: (1) inner circle: core set of domains mandatory for all RCT and LOS, (2) middle circle: important domains with optional inclusion, and (3) outer circle: “research agenda”3. The onion concept was recently updated to include within the inner core set domains that are mandatory for specific circumstances.
Since 2015, the OMERACT JIA Core Set Working Group has obtained global stakeholder input from patients with JIA, caregivers, healthcare providers (HCP), and researchers to update the JIA Core Domain Set. The current JIA Core Set includes 6 variables: physician’s global assessment (PGA) of disease activity, parent’s/patient’s global assessment of overall well-being, physical functional ability, active arthritis joint count, restricted motion joint count, acute-phase reactant and — for systemic JIA — fever in past week. At the OMERACT 2016 meeting, a special interest group (SIG) voted to reevaluate this core set4. This paper reports subsequent work, culminating in an OMERACT 2018 plenary session, and consensus on a new domain framework for JIA outcome measure prioritization for RCT and LOS.
MATERIALS AND METHODS, RESULTS
The OMERACT JIA Core Set Working Group expanded its international membership after the 2016 SIG meeting, adding representatives from Germany, the Netherlands, Serbia, and the United Kingdom to those from the United States, Canada, Australia, and Italy. The working group conducted monthly teleconferences providing input and reviewing work products. Co-chairs (AC, EMM, JEM), 3 patient representatives (BH, JH, JEM), an OMERACT fellow (AA), and OMERACT Executive Committee mentors (BS, COB, VS) directed activities. The Delphi survey was administered by researchers with relevant methodologic expertise (BF, HC).
A variety of inputs led to a candidate domain list for consideration in the updated core domain set. Initial activities prior to the SIG meeting (literature review, qualitative surveys of over 600 patients/parents) have been described4.
Patients’ and parents’ involvement
Attendees at the 2016 SIG recommended conducting additional qualitative studies to garner a global perspective before the Delphi survey. Hence, the online discussion boards (ODB) were replicated in Australia and Italy and compared to results from the United States4. In Australia, ethics approval was granted by the Royal Children’s Hospital, Melbourne (#37335A). In Italy, the study was considered non-human subjects research by the Institute for Research and Health Care Istituto Giannini Gaslini ethics board. Two sets of paired ODB were conducted in Australia and Italy following an established protocol4. In brief, private, 3-day moderated ODB were conducted with patients with JIA (teenagers and young adult groups) and separately with parents. The objective was to understand in the patients’ and parents’ words the experiences of living with JIA and of its treatment, and the effect on all aspects of health (physical, mental, social). We elicited information on the domains of health affected by JIA disease activity in search of candidate domains to measure in LOS. Typed discussion transcripts were analyzed and coded into domains. The United States and Italy teams used the software NVivo115.
Fifty-three patients with JIA across 2 groups, ages 15–18 years and 19–24 years, and 55 parents participated in the ODB (Table 1). Initial qualitative comparisons of Italian and American transcripts elicited similar themes, such as the psychosocial impact of JIA and limitations in daily activities. Relatively more fear of relapses and greater burden of medications, though noted in both samples, were more expressed by Italian participants; impact on children’s activities and family life was emphasized relatively more in the American groups. Patient groups from all 3 locations overlapped regarding identification of domains that varied depending on whether JIA was active (Table 1).
Online discussion board participants and main features when JIA is inactive by country.
Delphi study
Subsequently, a Delphi process was conducted within the JIA stakeholder community to assess agreement on a set of the most important core outcome domains to evaluate effects of JIA treatments in RCT and LOS. The Delphi process was approved by the ethics board (REB) at the Hospital for Sick Children, Toronto, Ontario, Canada (#1000059994). It began by categorizing the multiple health outcome domains of interest identified by extensive qualitative analysis (surveys, ODB), literature review, and workgroup input; respondents could add potential missing items. Next steps included shortening domain list to the most important categories to measure in all clinical trials.
With Delphi-type methodology electronic-based surveys were sent to invited stakeholders with target goal of 150 respondents to obtain maximum diversity in demographics, clinical characteristics including category of JIA6, and professional experience, and an 80% prespecified target completion rate. Eligible participants were patients with JIA aged ≥ 15 years, parents or primary caretakers of patients with JIA, HCP with adult or pediatric rheumatology experience, pediatric rheumatology researchers, regulators from medication agencies, and pharmaceutical company clinical trial investigators. Stakeholders in the categories of patients, parents/caregiver, representative or leader of parents/caregiver association were referred by the European Network for Children with Arthritis (n = 16) and the Arthritis Foundation (AF), a US-based advocacy group. The AF distributed the survey to a JIA-targeted mailing list of adults with a history of JIA and parents of children with JIA who expressed interest in participating in research studies (n = 178). Thus, the study team could not track e-mail addresses and completion rates from AF registrants. HCP, researchers, and clinical trials investigators were represented by the OMERACT JIA Core Set Working Group members (n = 36), the Pediatric Rheumatology Collaborative Study Group (PRCSG; n = 51), and the Pediatric Rheumatology International Trials Organization (PRINTO; n = 46, non-overlapping). Regulatory agencies were represented by the US Food and Drug Administration (FDA) and the European Medicines Agency (EMA; n = 5).
An iterative survey process was completed online using REDCap (https://projectredcap.org)7 electronic data recording hosted at The Hospital for Sick Children, Toronto, Ontario, Canada, to build group consensus. Items receiving > 70% average votes were advanced for consideration in the core set. All potential participants received e-mail invitations detailing study background, instructions, and study link with a 2-week deadline. The same participants were targeted for each round with 3 individual reminders. Respondent anonymity was preserved. Completion of a survey was considered as implicit consent to participate in that round. Demographic information on respondents was collected. No patient-related/personal health information was disclosed. The Delphi survey was written in lay language using feedback from the OMERACT JIA Working Group including patient research partners. Three rounds of Delphi were completed by 190 (response rate 85%), 201 (84%), and 182 (77%) stakeholders, respectively, from 50 countries. Parents represented children with JIA across categories6. Table 2 reports sample demographics.
Delphi survey respondent characteristics. Values are n unless otherwise specified.
Round 1: The initial survey asked for respondents to review 27 items (Appendix 1) presented in random order, generated from earlier phases of the project, and select their top 10 items to be assessed in clinical trials. Participants could write in any categories considered missing in the item list provided.
After the first Delphi round, 2 items achieved > 70% average votes: pain and joint inflammatory signs (swelling/redness/warmth). One item, “medication side effects,” had only 60% of votes, but was retained owing to OMERACT requirement. Given the option to suggest additional domains, 18 new write-in items were suggested (Appendix 2). The working group noted overlap of some write-in items with existing domains. Domains were relabeled, short domain definitions were developed, examples of domain content were provided for subsequent rounds (Table 3), and the number of domains was reduced to 22. The concept of global assessment, initially omitted because of the perception of representing measures rather than domains, was included because of historical precedent in the existing core set as a domain of “subjective assessment.” Manifestations of specific JIA categories6 were proposed (e.g., fever, psoriasis), raising the question of how to account for pathognomonic features of individual JIA categories (that personally affect subsets of patient/parent voters). Discordance in priorities was noted across stakeholder groups.
Domain definitions used for Delphi Survey voting reference.
Round 2: Participants were shown the results of round 1, including items reaching consensus as core set, and provided a revised list of items determined by round 1 shown with stakeholder group responses juxtaposed side by side. This incorporated write-in items, but not items that did not achieve either ≥ 30% average votes or ≥ 40% votes by any category of respondents. The revised list was framed in keeping with the language of the International Classification of Functioning, Disability and Health (ICF)8 with definitions provided (Table 3). Respondents were instructed to endorse their top 5 (unranked) of the 22 listed domains. This round showed similar discordance between stakeholder group priorities. Both groups rated activity limitation high (52% average vote), but otherwise priorities differed. Patients/parents prioritized effect on emotional function, joint damage, participation restriction, and coping with illness and fatigue, while other stakeholders prioritized eye inflammation, inflammation outside joints/eyes, laboratory, PGA, patient’s global perception, stiffness, and imaging.
Round 3: The final round was conducted as a ranking exercise to order domains as per the OMERACT Onion. Participants were shown round 2 results, presented in 2 groups “patient/parents” and “others,” and items reaching consensus as core set (> 70% votes on average, or as required by OMERACT; side effects/adverse effects). Instructions were to place each of the 22 domains into 1 of 3 categories: (1) core domain (inner circle): mandatory to measure in all JIA clinical trials (select up to 7); (2) important but optional domain (middle circle): inclusion depends on particular clinical trial (no limit on number); or (3) other domains (outer circle): more research or information is needed (no limit on number). The following rules were developed to provide guidance on which layer of the OMERACT Onion a domain ought to be placed based on the Delphi round 3 results. It was recommended that a domain be placed in the inner circle if the average of inner circle votes for both stakeholder groups was ≥ 70%. It was recommended that a domain be placed in the middle circle if the sum of the inner circle and middle circle votes was > 70% for both stakeholder groups (and the average of inner circle votes alone was not > 70%). It was also recommended that a domain be placed in the middle circle if the sum of the inner circle and middle circle votes was > 70% for only 1 stakeholder group, but the average inner circle and middle circle votes of both stakeholder groups together was also > 70%. It was recommended that a domain be placed in the outer circle [research agenda] if the sum of inner circle and middle circle votes was > 70% for only 1 or none of the stakeholder groups, and for both stakeholder groups together the average was < 70%. Results of 3 rounds of Delphi voting added physical function to the core set along with pain and joint inflammatory signs (Table 4). Discordance in stakeholder priorities persisted, consistent with findings of round 2. Patients/parents did not highly endorse global assessments, whereas other stakeholders did. Inconsistent with the proposed guidance on domain placement, patient perception of disease/overall well-being was included in the inner circle based on voting at the OMERACT Workshop, as described below.
Results of Delphi round 3 and domain placement in OMERACT Onion Framework after consensus voting.
OMERACT 2018 Workshop
A summary of qualitative studies and Delphi results was presented to the entire body of OMERACT delegates, along with a proposed core set. Breakout groups subsequently discussed and reviewed the proposed core set and survey data. Discussion summaries were shared with the larger group, after which a vote for the core set was conducted. At a separate plenary session, an additional vote on the full proposed OMERACT Onion was taken. Guidance was shared on how to consider placement of domains within the Onion based on Delphi round 3 voting.
The total number of delegates voting at OMERACT 2018 meeting was 170 (106 clinicians/researchers, 17 patient research partners, 11 pharmaceutical representatives, 33 fellows, and 3 regulatory authorities). Based on the Delphi process, the workgroup recommended inclusion of 4 domains: pain, joint inflammatory signs, activity limitation/physical function, and side effects/adverse events. After discussion, an initial vote was held on the proposed core domain set. Delegates achieved consensus on the 4 recommended domains (85% patients, 92% all other stakeholders). Subsequently, delegates voted one by one on other domains highly rated (but for which the average of inner circle votes for both stakeholder groups was < 70%). As a result of the voting, a fifth domain (“patient perception of overall well-being”) was also voted into the core domain set by the OMERACT delegates (86% patients, 74% other voters). In contrast, the corollary global assessment domain, physician’s perception of disease activity, was not voted for inclusion in the core domain set.
Final voting approved the domains as placed in the initial OMERACT Onion (Figure 1) with 83% endorsement. In response to discussions at OMERACT 2018, the OMERACT Onion framework was modified by the OMERACT Executive Committee to effectively add another layer to the inner circle to allow specification of certain domains as being mandatory in specific circumstances. This modification is well suited to planning RCT/LOS of JIA given that there is a diversity of JIA categories and need for a flexible framework to appropriately assess inflammatory features characteristic of disease activity/flare according to particular JIA categories. Adjustments were therefore made to the onion by the JIA Workgroup and this updated version is recommended for use (Figure 2).
Initial OMERACT domain framework for juvenile idiopathic arthritis clinical trials and longitudinal observational studies. Note this figure does not show domains considered mandatory in specific circumstances and is not recommended for use (see Figure 2).
OMERACT domain framework for juvenile idiopathic arthritis (JIA) studies including mandatory domains in specific circumstances. Recommended for use. *Illustrative examples (incomplete list): oligoarthritis: eye inflammation/uveitis; systemic JIA: fever, systemic rash, macrophage activation syndrome features; psoriatic arthritis: psoriasis; enthesitis-related arthritis: enthesitis, etc., to be determined.
DISCUSSION
An international workgroup has developed an updated JIA Core Domain Set following the OMERACT methodology with qualitative input from hundreds of patients/parents in international surveys, in-depth ODB replicated cross-culturally, and workshops. We prioritized the lists of candidate domains involving patients/parents and broad stakeholders through a Delphi process and consensus voting of OMERACT delegates. A notable change is the recommendation to add pain to the core set to be measured for all JIA RCT and LOS. While a patient’s/parent’s global assessment (overall well-being) was retained from the 1997 core set, Physician’s global assessment (PGA) is no longer mandatory, and laboratory tests are also considered important but optional. Of the new core set, 3 of 5 components are based on patient self-ratings — pain, physical function, and overall well-being — along with joint inflammatory signs and tracking of adverse events.
The inclusion of patients and parents in generating the list of candidate domains and participating in the Delphi voting highlighted important differences between the reported experience of living with JIA and other stakeholders’ (i.e., HCP’s, researchers’, regulators’) evaluation of disease activity. Patients/parents valued inclusion of a domain for joint damage over PGA of disease activity or the patient overall subjective perception. This preference may reflect lack of familiarity with global assessments or perhaps a greater concern about longterm consequences of JIA than a momentary evaluation perhaps based on incomplete information. In contrast, other stakeholders voted 83% to put the PGA in the core set. On a practical level, one may anticipate the PGA may continue to be measured in clinical trials despite not being voted into the core set because of familiarity among HCP/researchers, brevity, ubiquitous inclusion in past studies, and interest to compare new studies to prior outcomes attained in historical studies.
Qualitative data on patient experience and inclusion of patients/parents throughout the selection process resulted in additional patient/parent-reported domains being designated important to consider for inclusion in JIA RCT and LOS, including participation restriction, fatigue, and effect on emotional function, mood, or cognition.
Heterogeneity of JIA categories6 requires consideration of customizing the core set to identify relevant inflammatory features (e.g., fever, rash, back pain, enthesitis, uveitis) in specific populations. Although these extraarticular manifestations of inflammation were grouped into 1 domain for manageable voting, this landed in the “important but optional” circle. Having experience with all categories of JIA, HCP prioritized the domain, but the voting patient/parents did not, perhaps because they were not personally familiar with each manifestation, which affected only a small proportion of the voting patient/parent constituents. The new OMERACT Onion framework (Figure 2) allows domains to be mandatory under specific circumstances, according to study population, and thus a rationale for inclusion into the core set.
To raise awareness of study findings, results were presented at the 2018 American College of Rheumatology Annual Scientific Meeting. Results were also shared with the AF to aid in preparation for a patient-facing drug development meeting jointly held with the FDA and CARRA in August 2018. The domains identified in this project were included in a large-scale consumer survey shared at the meeting. EMA was represented on the working group (RV), and we will additionally engage with the FDA in the next phase of the project. PRCSG and PRINTO leadership are integral to the working group as we consider translation to use in RCT/LOS.
Limitations to the study included that, while we aimed for 80% response for the Delphi, we were unable to verify this for the AF-referred respondents. Because of the Internet format of the ODB and Delphi, we did not obtain the self-reported perspectives of those younger than age 15 years, though parents of younger children participated, and many youths had JIA onset at an early age. Future research on the perspective of younger patients may be required to ensure all relevant domains and experiences were considered.
Following the OMERACT methodology we developed an updated JIA Core Domain Set with broad international stakeholder input. The new core domain set has increased emphasis on patient/parent-reported domains, and our findings shed light on important aspects of living with JIA, which may inform clinical practice in addition to medication evaluation in RCT. Next steps will be to identify and systematically evaluate the best outcome measures for this purpose.
Acknowledgment
We acknowledge the Juvenile Idiopathic Arthritis (JIA) Core Set Working Group members for their contributions to the work reported herein; in addition to authors listed: M. Riebschleger for contributions to the literature review. We thank the OMERACT Executive for guidance and assistance with this project, in addition to mentors Beverly Shea and C.O. Bingham on the project, Shawna Grosskleg, Peter Tugwell, Dorcas Beaton, and Lara Maxwell. S. Thornhill designed the online discussion boards (ODB) research and facilitated the English-language ODB in the United States and Australia, and oversaw the Italian ODB facilitation by Ombretta Ferraro. ODB in Italy was coordinated by the Associazione per le Malattie Reumatiche Infantili and supported by AlfaSigma SpA, which had no direct access to patients’ information or ODB results. We are grateful to Nico Wulfraat and the European Network for Children with Arthritis, D.J. Lovell and the Pediatric Rheumatology Collaborative Study Group, and N. Ruperto and Pediatric Rheumatology International Trials Organization for facilitating surveys of their membership for the Delphi. We appreciate Laura Marrow and M. Suzanne Schrandt of the Arthritis Foundation for participation in the Delphi, along with the Research Committee of PARTNERS (Patients, Advocates and Rheumatology Teams Network for Research and Service), a group aimed at engaging patients in the process of conducting research that matters to them. It is supported by the Patient-Centered Outcomes Research Institute. Our deep gratitude goes to all patients and parents who responded to the surveys and participated in the ODB to bring the voice and lived experiences of patients and families into this project. We are thankful for the clinicians, researchers, regulators, and patients who participated in the surveys, and for contributing their experience and expertise in discussions at the JIA Outcomes Research Group at the 2018 Childhood Arthritis and Rheumatology Research Alliance scientific meeting and OMERACT 2018.
APPENDIX 1
List of 27 items included in Round 1 of Delphi survey.
APPENDIX 2
List of “write-in” items after Round 1 of Delphi survey.
Footnotes
OMERACT (Outcome Measures in Rheumatology) is an international health outcomes measurement group, which has been funded by unrestricted support from more than 12 pharmaceutical and clinical research organizations in the last 2 years. Drs. Bingham and Strand are members of the executive committee for OMERACT, but receive no financial remuneration for their efforts. OMERACT funded 2 patient research partners (BH, JH) to participate at OMERACT 2018. The online discussion boards (ODB) in Australia were supported with funding of Arthritis Australia. The ODB in Italy were supported by AlfaSigma SpA. The ODB in the United States were funded in part by the Arthritis Foundation and the Patient-Centered Outcomes Research Institute (PPRN-1306-04601). Conduct of the Delphi survey was funded by the Ho Family Research Chair in Autoimmune Diseases, awarded to Dr. Feldman.
- Accepted for publication January 24, 2019.
