Abstract
Objective. To reach consensus on the core domains to be included in a core domain set for clinical trials of shoulder disorders using the Outcome Measures in Rheumatology (OMERACT) Filter 2.1 Core Domain Set process.
Methods. At OMERACT 2018, the OMERACT Shoulder Working Group conducted a workshop that presented the OMERACT 2016 preliminary core domain set and its rationale based upon a systematic review of domains measured in shoulder trials and international Delphi sessions involving patients, clinicians, and researchers, as well as a new systematic review of qualitative studies on the experiences of people with shoulder disorders. After discussions in breakout groups, the OMERACT core domain set for clinical trials of shoulder disorders was presented for endorsement by OMERACT 2018 participants.
Results. The qualitative review (n = 8) identified all domains included in the preliminary core set. An additional domain, cognitive dysfunction, was also identified, but confidence that this represents a core domain was very low. The core domain set that was endorsed by the OMERACT participants, with 71% agreement, includes 4 “mandatory” trial domains: pain, function, patient global — shoulder, and adverse events including death; and 4 “important but optional” domains: participation (recreation/work), sleep, emotional well-being, and condition-specific pathophysiological manifestations. Cognitive dysfunction was voted out of the core domain set.
Conclusion. OMERACT 2018 delegates endorsed a core domain set for clinical trials of shoulder disorders. The next step includes identification of a core outcome measurement set that passes the OMERACT 2.1 Filter for measuring each domain.
Shoulder disorders, including rotator cuff disease (tendinopathy, impingement, subacromial bursitis, tears), adhesive capsulitis, instability, glenohumeral osteoarthritis, dislocation, proximal humeral or humeral head fractures, and unspecified shoulder pain, are highly prevalent disorders (7–26%)1, and are associated with significant morbidity, disability, and economic burden2,3,4. Despite increasing numbers of trials investigating the benefits and harms of treatments for these disorders, there is as yet no widely endorsed core domain set (for outcome domains) or core outcome measurement set (for instruments) that is advocated for clinical trials.
The Outcome Measures in Rheumatology (OMERACT) Shoulder Working Group was established in 2015 to develop these sets for clinical trials of shoulder disorders addressing all types of interventions5. Over the past 3 years, the working group has worked on several steps of this project in accordance with OMERACT methodology6,7. First, a systematic literature review identified 409 trials, and 32 outcome domains were assessed using 319 different measurement instruments8. Second, an international Delphi study was done that included 268 clinicians or researchers from 13 countries with experience in shoulder disorders and 67 patients with shoulder disorders9. This was followed by a pre-OMERACT meeting (including patient representatives) and a subsequent special interest group meeting at OMERACT 2016, in which a preliminary core domain set was presented and unanimously approved. In Table 19,10,11, the preliminary core domain set as well as the voting percentages of each domain from the Delphi study are summarized12.
Phases of the development of the Shoulder Core Domain Set and included domains.
This preliminary core domain set consisted of 4 domains in the “inner circle” of the OMERACT “onion” (indicating mandatory domains for all trials of shoulder disorders): pain, physical function/activity, global perceived effect, and adverse events including death; 3 domains in the “middle circle” (important but optional domains): emotional well-being, sleep, and participation (recreation and work); and a research agenda required to inform the final core domain set in the “outer circle.” The research agenda comprised clarifying the definition of physical function/activity, determining whether participation (recreation and work) should be in the inner circle, and determining whether to include pathophysiological manifestations in the core domain set and if it should be situated in the inner (mandatory) or middle (important but optional) circle. In the meantime, OMERACT nomenclature has been slightly updated as indicated later in the final core domain set.
Following the abovementioned initiatives, the working group proposed a workshop for OMERACT 2018. In preparation for it, the working group performed a systematic review of qualitative studies that had analyzed the lived experience of shoulder pain. The purpose of our review was to determine whether any potentially relevant domains were missing from the preliminary core domain set and to further inform the research agenda10. The present paper summarizes the results of this work presented at the OMERACT 2018 Shoulder Core Set Workshop, the domains that were presented for endorsement, the results of the plenary and breakout discussions, and the subsequent vote for endorsement of the core domain set for clinical trials of shoulder disorders.
MATERIALS AND METHODS
Review of qualitative studies
To ensure that all important patient outcome domains had been considered, we performed a systematic review of qualitative studies that had explored the experiences of people with a shoulder disorder.
The methods for our systematic review were prespecified (PROSPERO ID: CRD42017082628) and the full findings are presented elsewhere10. Briefly, we searched for eligible studies indexed in Ovid MEDLINE, Ovid Embase, CINAHL (EBSCO), SportDiscus (EBSCO), and Ovid PsycINFO to November 2017. Studies in which the authors used qualitative methods (e.g., focus groups, Delphi methods, nominal group techniques, participant observation, interviews) to analyze the experiences and perceptions of people living with a shoulder disorder were included.
The primary outcomes of interest for this review included the symptoms of people with shoulder disorders and the effect these symptoms have on their daily lives, and the outcome(s) of most importance to patients, as elicited by qualitative research methods. Two authors independently screened studies for inclusion, appraised their methodological quality (using the Critical Appraisal Skills Programme checklist for qualitative studies)13, coded text line-by-line to identify outcome domains (i.e., individual symptoms and perceived impacts on daily living) reported by participants, and assessed the confidence in each review finding (high, moderate, low, very low) using the GRADE-CERQual approach (Confidence in the Evidence from Reviews of Qualitative research)11.
Eight studies met the eligibility criteria and they included 133 participants (49 females and 84 males). Studies were conducted in the United Kingdom (4 studies), Canada (2 studies), Finland (1 study), or New Zealand (1 study). Participants had diagnoses of rotator cuff disease (3 studies), adhesive capsulitis (2 studies), proximal humeral fracture, shoulder instability, or unspecified shoulder pain (1 study each).
Seven domains were identified across the 8 qualitative studies: (1) pain; (2) physical function/activity limitations (difficulties performing activities of daily living such as dressing or bathing); (3) participation restriction (work disruption, limited recreation/leisure, and limited social interactions); (4) sleep disruption (difficulty falling, and subsequently staying, asleep); (5) cognitive dysfunction (poor concentration and memory); (6) emotional distress (frustration, anxiety, and depression); and (7) pathophysiological manifestations (problems related to muscle functions such as reduced range of motion and loss of muscle strength).
We mapped the outcome domains arising from our systematic review of qualitative studies against the domains included in the preliminary core domain set to determine whether any important domains were missing. Only 1 domain, which we termed “cognitive dysfunction” (referring to the reported experience that shoulder pain was so severe that it prevented the participant from being able to concentrate on anything else), had not been identified in our previous research8,9,12. Sleep deprivation due to shoulder pain was also reported to affect concentration and memory. However, we had very low confidence in these findings, because they were raised by only a few participants from only 2 studies, one of which had several methodological limitations. In contrast, we had greater confidence that the other identified outcome domains reflected the experiences of many people with shoulder disorders. For all the other domains, the confidence in the findings was moderate, except for pathophysiological findings, for which confidence was also low (Table 1).
The review findings were discussed in the monthly teleconferences of the core members of the working group, in which conveners, fellows, patient representatives, and representatives from the OMERACT technical advisory committee participated. Prior to OMERACT 2018, the final results were presented in a teleconference with all working group members. Members discussed potential modifications to the preliminary core domain set based on the findings of the review of qualitative studies. We decided to move condition-specific pathophysiological manifestations into the middle rather than inner circle and added to the outer circle (research agenda) the question, “Should ‘cognitive dysfunction’ be in the Onion?” No other modifications were made before the OMERACT 2018 meeting (Table 1).
OMERACT 2018 Shoulder Core Set Workshop and Plenary
At OMERACT 2018, the Shoulder Core Set Workshop presented the steps that had been taken toward arriving at the proposed core domain set for clinical trials of shoulder disorders and the details of the domains that had been included.
A total of 95 participants (11 patients, 84 other stakeholders — including clinicians, other health professionals, researchers, regulators) were split into 8 breakout sessions to facilitate in-depth discussions of the proposed core set. Each group had a facilitator, content expert, and reporter. All domains of the proposed core domain set were discussed by all breakout groups. After reconvening, the reporter for each breakout summarized their group discussions to all OMERACT 2018 participants and this was followed by general discussion. The Shoulder Working Group collected detailed feedback from each breakout group for further decision making regarding the naming and positioning of domains in the core domain set. In a final plenary session on the following day, a revised proposal of a core domain set was presented for final discussion and endorsement. To obtain endorsement, 70% of the votes in favor of the proposed core set was required. In case consensus was not reached, revisions were discussed, a reformulation proposed, and then the proposed core set was voted on again.
RESULTS
Breakout group discussions regarding each proposed domain and their definition
Pain: There was uniform agreement that pain, reflecting pain intensity, should be included as a mandatory domain within the inner circle.
Function: Function was also recognized as a mandatory domain. It was said that function should not reflect only the ability to fulfill basic needs, so the definition was adjusted to reflect normal activities of daily living (Table 2).
Definition of the domains included in the Core Domain Set for Shoulder Disorders.
Adverse events including death: This was also considered a mandatory domain reflecting OMERACT principles6.
Global perceived effect: This domain elicited the most discussion. There were some concerns raised about whether this domain was redundant in view of the overlap with the domains of pain and function. However, the results of the Delphi study were considered sufficient to justify its inclusion as a separate domain. Several participants raised the concern that as worded, this domain would include a change measure (for example, in relation to some treatment) while their preference was for a status measure (i.e., the patients’ perception of their current state). Others argued that having a change measure in the core domain set could have a desirable effect because it could act as an anchor for other measures when analyzing outcomes in relation to each other. Following discussion, consensus was reached to change the name of this domain to “patient global.” Consensus was also that this patient global should not be a measure of overall well-being, but a global rating of the shoulder, and therefore the name “patient global — shoulder” was chosen. Whether this should be measured using a status or change measure will be reconsidered in the next phase of identifying suitable measures for this outcome.
Participation: There were proposals to adjust the definition of participation to ensure that it covers both paid and unpaid types of work, and to align the definition of participation to the International Classification of Functioning definition14. Both changes were adopted.
Emotional well-being: This domain was identified as having some overlap with the domains of patient global and function. However, given the results from the Delphi and the qualitative review, no changes to the domain were proposed.
Condition-specific pathophysiological manifestations: There were a range of divergent opinions regarding this domain and where it fits within the OMERACT core domain set for shoulder disorders. Arguments for inclusion of this domain within the inner circle said that a move would be consistent with OMERACT rules for including at least 1 domain reflecting pathophysiologic manifestations in the inner circle. On the other hand, while pathophysiologic manifestations may be relevant for some trials — for example, determination of fracture healing in a trial investigating treatment for proximal humeral fractures — it may not be of value to have a pathophysiologic manifestation included as a core outcome in all trials. In addition, there were also arguments for including this domain in the outer circle to reflect that more research was needed before this could be resolved. For the purpose of voting, this domain was kept in the middle circle.
Sleep: Some discussion surrounded the idea that there are differing constructs that could be included within a sleep domain such as sleep quality and the interference/effect of sleep disturbance. However, there were no consistently recommended changes to the definition or position of sleep in the core domain set.
Cognitive dysfunction: This domain was considered to possibly overlap with other domains, such as sleep (i.e., lack thereof) or emotional well-being (e.g., depression can affect cognition). Some expressed it could be kept in the outer circle, while others recommended leaving it out of the core domain set altogether.
Results of the final vote
Seventy-one percent of OMERACT participants approved the inner core domains consisting of pain, function, patient global — shoulder, and adverse events including death (Table 3). When considered individually, pain was endorsed by 100% of participants, function by 99%, and patient global — shoulder by 80%. No vote was conducted for adverse events (including death) because this is a mandatory OMERACT domain.
Voting results from the OMERACT 2018 Shoulder Core Set Workshop. Values are n (%)§.
For the middle circle, 89% of participants endorsed inclusion of participation, 93% endorsed sleep, and 82% endorsed emotional well-being. Inclusion of condition-specific pathophysiological manifestations in the middle circle was endorsed by only 44% of participants in the first voting round. After discussion, a second vote recorded 35% preference for the inner circle, 35% for the middle circle, and 29% for the outer circle. Of note, the argument to move this domain to the inner circle occurred because of the OMERACT requirement for including at least 1 pathophysiological domain in the inner circle. Because pain can be considered a condition-specific pathophysiological manifestation and because 70% voted for either the inner or middle circle, we elected to keep condition-specific pathophysiological manifestations in the middle circle as originally preferred by the OMERACT Shoulder Working Group.
Keeping cognitive dysfunction in the outer circle for further research was endorsed initially by 64% of participants. After further discussion that reminded participants of the results of the qualitative review (which had identified poor concentration and memory as relevant to the experience of people with shoulder pain, but found very low confidence that this cognitive dysfunction was truly a main issue), a second vote was conducted and 64% of participants endorsed the removal of this domain from the core domain set altogether.
In response to discussions at OMERACT 2018, the OMERACT onion was adjusted and approved. This adjustment adds another layer to the inner circle of the OMERACT onion structure to allow specification of certain domains as mandatory in specific circumstances.
Currently, the OMERACT onion is the 3 circles consisting of “mandatory” domains (inner circle), including the domain adverse events including death; “important but optional” domains (i.e., dependent on the design of the study or research question asked; middle circle); and “research agenda” (domains of interest that need more research work but are under consideration; outer circle). The final core domain set for clinical trials of shoulder disorders is shown in Figure 1.
The OMERACT Onion: organization of domains for Working Group: Shoulder. OMERACT: Outcome Measures in Rheumatology.
DISCUSSION
We have outlined the final steps taken to reach OMERACT endorsement for a core domain set for clinical trials of shoulder disorders. It is likely that this core domain set will also be applicable for longitudinal observational studies. This core set development represents a major step forward in the harmonization of outcome measurement in this field of research. The core domain set consists of 4 “mandatory” domains within the “mandatory for all clinical trials” category: pain, function, patient global — shoulder, and adverse events including death; and 4 “important but optional” domains: participation (recreation/work), sleep, emotional well-being, and condition-specific pathophysiological manifestations. Currently there are no items in the new “mandatory in specific circumstances category” in the inner circle. There were no items remaining in the research agenda.
Potential limitations in the development of the shoulder disorders core domain set include the relatively low response rate in both Delphi rounds, particularly for some stakeholders. Further, Delphi participants were asked to judge the importance of each domain, but not to rank or prioritize a predetermined number of domains, which may have contributed to overrating the importance of some domains and the inclusion of some overlapping domains. The review of qualitative studies was conducted after the Delphi, which may have led to the inclusion of proposed domains (e.g., cognitive dysfunction) that may have been rated as unimportant during the Delphi exercise. On the other hand, strengths of the process include the high-quality methods that meet OMERACT standards, and the engagement of patients, clinicians, and other relevant stakeholders from the field of outcomes in rheumatology.
The next step will be to define a core outcome measurement set, which is a core set of instruments that should be used to measure each of the domains. The working group is already working toward determining which instruments pass the OMERACT 2.1 truth, discrimination, and feasibility filter. Results of this work will be presented at a future OMERACT conference to enable endorsement of specific measures for each domain to be included in the final core outcome measurement set and any priorities for future research.
After reaching widespread agreement on these core domain and outcome measurement sets through the OMERACT process, it will be important to implement them. While we work toward identifying which measurement instruments pass the OMERACT 2.1 truth, discrimination, and feasibility filter, researchers designing and conducting trials, observational studies, and systematic reviews should already consider inclusion of the core domain set as a minimum. We will disseminate the OMERACT-endorsed core domain set in workshops at relevant meetings as well as use other methods for reaching relevant stakeholder groups (e.g., patients, trialists, researchers, clinicians, regulators). We will also ensure a Web presence (e.g., OMERACT Website, links from other sites). In the future, we will also measure the success of our implementation strategies by monitoring whether the OMERACT-endorsed core domain and outcome measurement sets for clinical trials of shoulder disorders are being used in trials, as has previously been done in other areas15,16.
Acknowledgment
We gratefully acknowledge the participation and insights of OMERACT 2018 meeting participants at our workshop session.
Footnotes
RB is supported by an Australian National Health and Medical Research Council Senior Principal Research Fellowship. NEF is an NIHR Senior Investigator and was supported through an NIHR Research Professorship (NIHR-RP-011-015). TS and PGC are supported by the NIHR Biomedical Research Centre, Oxford and the NIHR Leeds Biomedical Research Centre, respectively. The views expressed in this article are those of the authors and not necessarily those of the NHS, the NIHR, or the UK Department of Health.
- Accepted for publication January 17, 2019.
