Abstract
Objective. To update the 1997 OMERACT-OARSI (Outcome Measures in Rheumatology-Osteoarthritis Research Society International) core domain set for clinical trials in hip and/or knee osteoarthritis (OA).
Methods. An initial review of the COMET database of core outcome sets (COS) was undertaken to identify all domains reported in previous COS including individuals with hip and/or knee OA. These were presented during 5 patient and health professionals/researcher meetings in 3 continents (Europe, Australasia, North America). A 3-round international Delphi survey was then undertaken among patients, healthcare professionals, researchers, and industry representatives to gain consensus on key domains to be included in a core domain set for hip and/or knee OA. Findings were presented and discussed in small groups at OMERACT 2018, where consensus was obtained in the final plenary.
Results. Four previous COS were identified. Using these, and the patient and health professionals/researcher meetings, 50 potential domains formed the Delphi survey. There were 426 individuals from 25 different countries who contributed to the Delphi exercise. OMERACT 2018 delegates (n = 129) voted on candidate domains. Six domains gained agreement as mandatory to be measured and reported in all hip and/or knee OA clinical trials: pain, physical function, quality of life, and patient’s global assessment of the target joint, in addition to the mandated core domain of adverse events including mortality. Joint structure was agreed as mandatory in specific circumstances, i.e., depending on the intervention.
Conclusion. The updated core domain set for hip and/or knee OA has been agreed upon. Work will commence to determine which outcome measurement instrument should be recommended to cover each core domain.
Osteoarthritis (OA) is one of the most common musculoskeletal diseases, with an estimated prevalence of 12% to 22% worldwide1. It is the leading cause of disability among older adults, with an estimated lifetime risk of knee OA about 40% in men and 47% in women2. The most common symptoms associated with OA are pain, stiffness, and fatigue, associated with disability and loss of physical activity and functional independence1,2.
Clinical trials seek to determine whether treatments are safe and beneficial for patients by comparing their relative effects on outcomes chosen to identify benefit or harm3. The results can then be used to make decisions on whether a treatment under investigation should be recommended. It is, therefore, essential that outcomes reported in trials are those needed by decision makers, and reflect meaningful measures for patients, clinicians, and others4,5.
The Outcome Measures in Rheumatology (OMERACT) group was established in 1992 with the aim of bringing together people interested in the development, reporting, and application of core outcome sets (COS). A COS is an agreed set of outcomes (domains) that clinical trialists should measure and report in all clinical trials of a specific condition6,7. A COS also includes recommendations on what outcome measurement instrument should be used to measure these core domains6,7. Thus, a COS consists of “domains” and “instruments.”
There are 4 core areas that should be covered in an OMERACT core domain set with at least 1 domain in each of these areas: death, life impact, pathophysiological manifestations, and resource use (strongly recommended; if resource use will not be included, there needs to be an adequate and agreed-upon justification for its exclusion)6. All COS should also consider factors which are not the primary object of research, but that may influence the results or the interpretation of the results6. These are known as “contextual factors”6. An “instrument” is the outcome measurement instrument recommended to measure that specific domain, e.g., questionnaires to assess quality of life, scales to assess cost, instruments to measure body function, and tests and imaging to assess biomarkers. The key principles for selecting core domains and corresponding instruments are international consultation between patients, health professionals, researchers, and industry followed by consensus6,7,8. Through this, any consensus achieved by an OMERACT working group is perceived as being informed through opinions of key participants, and to have a worldwide perspective.
In 1997, OMERACT in conjunction with the Osteoarthritis Research Society International (OARSI) developed a COS hip and knee OA9, consisting of 4 core domains to be measured and reported in all hip and knee OA clinical trials: pain, physical function, patient’s global assessment (PtGA), and for studies with a followup period of a year or longer, joint imaging (such as radiography). Over the past 20 years, there have been developments in how the OMERACT COS are developed, with greater emphasis on patient involvement6,10,11. Further, there have been developments in how domains are identified through the recent adoption of the OMERACT Filter 2.06. These guidelines were not established when Bellamy, et al9 developed their COS for hip/knee OA in 1997.
Given developments in methodology, the OMERACT group agreed that the previous hip and knee OA COS should be reviewed, and that became the purpose of this work. The project was divided into 3 phases: review of current COS for patients with hip and knee OA (phase 1); Delphi exercise to establish worldwide perspectives on what are potential domains of interest (phase 2); and the OMERACT 2018 meeting to establish consensus and the update core domain set (phase 3).
This paper reports these phases and presents the OMERACT-OARSI core domain set to measure in clinical trials for people with hip and/or knee OA.
MATERIALS AND METHODS, AND RESULTS
Research ethics approval was gained from the University of East Anglia’s (UK) Faculty of Medicine and Health Sciences Research Ethics Committee on September 14, 2017 (Ref: 2016/2017-104). Patient consent was obtained as part of this ethical approval.
Phase 1
All COS that included the views of people with hip or knee OA were reviewed from the COMET (Core Outcome Measures in Effectiveness Trials) database, a repository of published and ongoing COS projects12. From 218 COS in musculoskeletal diseases, 4 COS where identified that included the views of people with hip or knee OA8,13,14,15.
Five patient and health professional/researcher meetings were held to consider the list of candidate domains, based on the results of the review of the COMET COS, prior to the Delphi project. These were conducted across 3 countries [Canada (Toronto), Australia (Sydney), and the United Kingdom (Leeds and Norwich)] involving 35 people with hip and/or knee OA, 34 healthcare professionals, and 1 nonclinical researcher. The role of these groups was to determine whether any candidate domains were missing, whether some domains were repetitious and required merging, or whether the Delphi Round 1 survey wording was ambiguous. Amendments were made in accordance with these recommendations before launching the Delphi exercise.
Phase 2
Participants and sample size
The study flow is illustrated in Figure 1. The target populations were people with hip and knee OA and professionals working in areas of relevance to OA, such as nurses, occupational therapists, orthopedic surgeons, physiotherapists, rheumatologists, and researchers; and people working in the pharmaceutical or device industry (e.g., knee braces and orthoses).
Delphi study flow diagram.
There is no consensus on the optimal sample size for a Delphi study16. Therefore, recruitment was based on time-scale. Round 1 was opened for 6 weeks (December 19, 2017, to January 27, 2018) using a broad sampling strategy to gain as large a sample as was feasible within the study time frames.
Distribution and approach
The Delphi survey was distributed through a number of streams to ensure broad coverage to the target population. These included distributing the survey to members of the OARSI, the Arthritis Research UK (ARUK) Osteoarthritis Clinical Study Group, recipients of the ARUK e-bulletin, members of the Spanish Society of Rheumatology, the Italian Rheumatology Society (SIR), the European League Against Rheumatism, People With Arthritis/Rheumatism (PARE), patient representatives through the Arthritis Foundation’s e-mail list, the Australian “myjointpain” group, and delegates to the Australian OA Summit. There were no restrictions on who from these groups could contribute. In addition, a social media campaign was designed through Twitter to gain further international participation of patient, clinical, research, and industry representatives.
A window of 6 weeks was allotted to recruit all potential respondents for Round 1 of the Delphi exercise. A reminder was sent after 3 weeks. After the 6-week recruitment campaign, the hyperlink for Round 1 was closed. Round 2 was undertaken from February 5, 2018, to February 26, 2018, while Round 3 was completed from March 5, 2018, to March 25, 2018.
Process
The Delphi survey was administered through the online software DelphiManager17. The DelphiManager program was presented in English and Italian for the PARE and the SIR.
Participants were asked to judge the importance of 50 potential core domains, generated from phase 1, by answering the question “how important are the following items to be assessed in trials with people with hip and knee OA?” As adopted previously18, responses were measured where 1–3 represented “not that important,” 4–6 “important,” and 7–9 “critically important.” There was also an “unable to score” option. We provided an open question where participants could indicate if there were any further domains that should be assessed but was not in the predefined list. Where such a response was reported, this was added to Round 2. Participants were also asked whether certain domains should be merged because of perceived overlap, i.e., pain intensity (overall) versus pain intensity (at rest) or pain intensity (with activity).
In agreement with MacLennan, et al’s19 approach, domains were excluded in Round 2 if they were rated as “not that important” (≤ 3 points) by ≥ 15% of 1 or more groups or included if they were rated as “important” (≥ 4 points) by ≤ 70% of 1 or more groups (i.e., patients, healthcare professionals, researchers, industry). If there was agreement from at least 70% of each group for a merger of domains, this was performed and included in Round 2 domains.
The Round 2 and Round 3 surveys followed the same format, asking the same questions as Round 1, adopting the same scoring system and approach to domain reduction and merger. Round 2 and 3 participants were provided with the mean responses for each domain from the previous round, presented by group.
Data analysis
The analysis determined which domains were considered most important to be assessed in future trials of people with hip and knee OA. For this, descriptive statistics and frequency distributions were used to collectively assess all completed Delphi surveys for each of the 3 Delphi rounds. The data were presented as frequency distributions and mean values with SD where appropriate. Data were analyzed by 2 groups to inform the OMERACT-OARSI core domain set: “people with OA” versus “other stake-holder groups.” Data analyses were performed using the Statistical Package for the Social Sciences version 25.0 (SPSS Inc).
Formation of the core domain set
The individual item responses provided from the Delphi survey were reviewed and categorized by members of the working group under overarching domains. This respected the recommendations made in Filter 2.17 and OMERACT20. Based on these domains, the rules for inclusion of domains were:
Mandatory (Core) Domains: domains considered “critical” by over 70% of both groups (patients AND others);
Important but Optional Domains: domains considered “critical” by over 70% of 1 group (either patients OR others) but not both;
Research Agenda: domains that need further research.
Adverse events including mortality/survival were included per default as a core domain as per Filter 2.17.
In response to discussions at OMERACT 2018, the OMERACT onion was adjusted and approved. The OMERACT onion is a schema that illustrates all 3 parts of the core domain set [mandatory (core domains); important but optional domains; research agenda] and identified contextual factors6. This adjustment adds another layer to the inner circle of the OMERACT onion structure to allow specification of certain domains as mandatory in specific circumstances.
Delphi results
The characteristics of those who participated in each round of the Delphi survey are presented in Table 1. In total, 343 participants completed Round 1 of the Delphi survey, with 177 (52%), and 119 (35%) completing Rounds 2 and 3 respectively (Figure 1). Table 1 illustrates that a cross-section of respondents was represented across the 4 groups, from different continents, representing different clinical presentations or health professionals/research backgrounds.
Demographic characteristics of Delphi participants.
Table 2 gives the results of the Round 3 Delphi exercise presented by domains by “people with OA” versus “other stakeholders” groups. This table shows those domains and items that reached the a priori threshold for the core domains and those eligible as “important but optional” and “research agenda” domains. These results are summarized in Figure 2.
Endorsed OMERACT-OARSI core domain set for trials of people with hip and knee osteoarthritis. OMERACT: Outcome Measures in Rheumatology; OARSI: Osteoarthritis Research Society International.
Formatted Delphi Round 3 results to illustrate the core areas, domains, and items for the Round 3 Delphi results.
Phase 3
The methods and results of phases 1 and 2 were presented to delegates on Thursday, May 17, 2018, at the OMERACT 2018 plenary meeting in Terrigal, Australia. This meeting included clinicians, patients and patient representatives, researchers, industry representatives, and methodologists. After being presented with this background, delegates were allocated to 8 groups where they were asked to consider for 60 min the composition of the OMERACT core domain set based on the Delphi Round 3 results as presented in Table 2. Each of the 8 groups provided feedback after which 102 delegates voted on the mandatory and important but optional domains. There was 100% agreement that pain and physical function, and over 90% agreement that quality of life and PtGA of target joint, should be included as core domains. However, the groups made the following recommendations: moving joint structure into a separate category of “mandatory in specific circumstances” because there was concern that this would not be relevant for all types of OA trial interventions (i.e., non-structure–modifiable interventions). The variability in Delphi score between patient and other votes for a number of domains classified as important but optional (i.e., cognitive function and fatigue) was highlighted by the groups (Table 2), and the terminology used to describe activity and participation and direct costs.
Following this, the working group members revised the preliminary OMERACT core domain set from the initial vote. A new rule was introduced to account for the wide variability in scores between the “patient” and “other stakeholders” groups. Where there was a discrepancy of > 30% between the 2 groups, and where either group presented with > 85% agreement that the domain was “critical” to measure, then that item would not be eligible for inclusion as an important but optional domain.
The revised core domain set (Figure 2) was presented on Friday, May 18, 2018, to the OMERACT 2018 plenary delegates for a final vote. This included 129 voting delegates. Because the included core domain passed the 70% threshold, the votes counted from the previous day’s voting were brought forward. Therefore, voting was cast on the composition of the “important but optional” and “research agenda” domains. In trials investigating structure-modifying interventions, joint structure should be assessed. The results of the vote on the core domain set are presented in Table 3. There was agreement by over the 70% threshold required by OMERACT to endorse the core domain set.
Summary of the voting scores for the core domain set from OMERACT 2018.
DISCUSSION
Our paper reports the agreed core domain set, developed using the OMERACT process, with international collaboration across a broad spectrum of people involved in the care of patients with hip and/or knee OA. This update has overcome previous limitations from the 1997 COS9, most notably through greater patient representation, internationalization of premeeting views through an international Delphi, and structuring the findings in accordance to the OMERACT Filter 2.17.
While the domains of pain, physical function, and PtGA remain core domains, quality of life has been introduced through this updated core domain set. It is likely that further work through OMERACT will be needed to define domains encompassed within the broader concept of quality of life. The project findings also include a number of new domains that are recommended (but not core) for clinical trials and that were not included in the 1997 core domain set9. These include cognitive function, fatigue, sleep, effect on family/caregivers, and psychosocial effect. This difference may correspond to the wider contribution of views compared to Bellamy, et al’s9 COS, particularly the patient perspective. It represents a change in domain selection toward a more diverse, biopsychosocial evaluation of clinical outcomes.
This is the first OMERACT core domain set to include a contextual factor. The inclusion of adherence was considered important given the results of the Delphi survey in which both patient and non-patient groups reported this as critical to include in trials with people for hip and knee OA. The working group considered this a contextual factor as opposed to a domain because it is important to understand how adherent a study participant is to an intervention, but it is, in most cases, not necessarily an outcome in itself (unless the trial is designed specifically to assess adherence). Through this means, adherence may be considered useful in the process evaluation of an interventional trial. The working group will consider how to expand on this list of contextual factors and determine the composition of this list. We hope the work of the OMERACT Contextual Factors Working Group will assist and guide the determination of what should be included in this list, to provide a consistent approach in identification and reporting.
This study had several limitations. First, as per OMERACT processes, the delegates at OMERACT 2018 had the final consensus vote on the core domain set composition. While this included 129 individuals, the percentage of patients in the OMERACT delegate group was smaller than the percentage of patients in the Delphi study. However, because delegates based their votes on the findings from the Delphi survey, this approach was considered appropriate because any voting was therefore underpinned by the views of a wider and more diverse cohort. Second, members of the working group were required to formulate domains from items reported in the Delphi. Participants in the Delphi survey were required to vote on items rather than domains to provide more detailed views on specific aspects of domains, e.g., “pain intensity” rather than just “pain.” However, this may be viewed as introducing subjectivity in domain formulation. To negate this, the working group consisted of clinicians, researchers, methodologists, and patients, to ensure that this process followed required OMERACT procedures and research or clinical perspective. Third, both phase 1 and phase 2 included representation largely from 3 continents, i.e., Europe, Australasia, and North America. There was limited representation from Africa and central Asia. While the social media strategy facilitated recruitment of some participants, most notably from Asia, the results from this core domain set may not necessarily represent global views. This is a recurrent limitation in COS development and one that requires further methodological consideration in future projects. Finally, while the Delphi survey gained a range of responses internationally and from a number of different participants (originally 343), the final Delphi round consisted of 119 participants, and therefore the Delphi reflected only the beliefs of those respondents rather than the original 343.
The goal of the next 24 months will be to commence work on assessing instrument selection for mandatory domains from this agreed core domain set. These will be reviewed in accordance with Filter 2.17 with the ultimate aim of developing a new core outcome measurement set. In combination with this, the working group will promote the dissemination of the core domain set and subsequent COS through presentation of work to patients, healthcare professionals, researchers, regulatory authorities, funders, and all individuals and groups involved in the care of people with OA.
Footnotes
This research was supported by the NIHR Oxford Biomedical Research Centre (T.O. Smith) and the NIHR Leeds Biomedical Research Centre (Prof. Conaghan). The views expressed are those of the authors and not necessarily those of the UK National Health Service, the NIHR, or the Department of Health.
- Accepted for publication December 3, 2018.
REFERENCES
In this issue
