Abstract
Objective. The aim of this Outcome Measures in Rheumatology (OMERACT) Working Group was to determine the core set of outcome domains and subdomains for measuring the effectiveness of shared decision-making (SDM) interventions in rheumatology clinical trials.
Methods. Following the OMERACT Filter 2.0, and based on a previous literature review of SDM outcome domains and a nominal group process at OMERACT 2014, (1) an online Delphi survey was conducted to gather feedback on the draft core set and refine its domains and subdomains, and (2) a workshop was held at the OMERACT 2016 meeting to gain consensus on the draft core set.
Results. A total of 170 participants completed Round 1 of the Delphi survey, and 116 completed Round 2. Respondents came from 29 countries, with 49% being patients/caregivers. Results showed that 14 out of the 17 subdomains within the 7 domains exceeded the 70% criterion (endorsement ranged from 83% to 100% of respondents). At OMERACT 2016, only 8% of the 96 attendees were patients/caregivers. Despite initial votes of support in breakout groups, there was insufficient comfort about the conceptualization of these 7 domains and 17 subdomains for these to be endorsed at OMERACT 2016 (endorsement ranged from 17% to 68% of participants).
Conclusion. Differences between the Delphi survey and consensus meeting may be explained by the manner in which the outcomes were presented, variations in participant characteristics, and the context of voting. Further efforts are needed to address the limited understanding of SDM and its outcomes among OMERACT participants.
Clinical practice guidelines endorse shared decision making (SDM) for the management of osteoarthritis (OA), rheumatoid arthritis (RA), and psoriatic arthritis (PsA)1,2,3,4. The SDM process allows patients and healthcare professionals to jointly make a decision based on the best available evidence for treatment options while respecting each patient’s values and preferences5. The use of SDM interventions, such as patient decision aids, has been shown to reduce decisional conflict (i.e., patients feeling unsure about their best choice), increase knowledge of treatment options, clarify patients’ values, facilitate patient participation in decision making, and reduce overuse of interventions that are not beneficial for the majority6. Decision coaching and question prompts are other interventions to facilitate SDM7,8.
In rheumatology, the lack of consensus on how to measure the effectiveness of the SDM process and outcomes, as well as the lack of patient involvement in devising such a consensus, creates a barrier to further evaluation of SDM interventions. Systematic reviews of SDM interventions show that a wide range of outcome domains were assessed in trials6,7. The most common outcome domains concern the quality of the decision-making process, such as decisional conflict and patient participation in decision making, as well as the quality of the choice made, such as patients’ knowledge and informed value-based choice (i.e., match between features that matter most to the informed patient and the chosen option)6,7.
The International Patient Decision Aid Standards (IPDAS)9 has identified a set of 8 outcome domains for evaluating the effectiveness of patient decision aids: (1) recognize the decision to be made, (2) know the options, (3) their features, (4) understand that values that affect the decision, (5) clarify values, (6) discuss values with health providers, (7) participate in decision making in preferred ways, and (8) make an informed value-based choice. However, there is a need to gain consensus on the outcome domains from all perspectives in rheumatology, especially patients, to assess the effect of SDM interventions in an exhaustive and meaningful manner.
The aim of our original research was to determine the core set of outcome domains and subdomains for measuring the effectiveness of SDM interventions among adults with OA, RA, and PsA. Although SDM outcome domains may apply to various rheumatic conditions, our study focused on the most common debilitating conditions.
MATERIALS AND METHODS
The Outcome Measures in Rheumatology (OMERACT) SDM working group followed the OMERACT Filter 2.010. Previous steps taken by this group included (1) forming an OMERACT working group consisting of patients with rheumatic conditions, health professionals, and researchers, (2) conducting a literature review of domains of SDM and developing a draft core set, and (3) obtaining the opinions of OMERACT members on the draft core set at a special interest group (SIG) session at the OMERACT 12 (2014) meeting in Budapest, Hungary. These steps and their results are described in more detail in a previous publication11.
This work led to the development and refinement of a draft core set of 7 domains, with 12 subdomains, to be included in trials evaluating the effectiveness of SDM interventions in OA (Table 1). The current original study focused on the subsequent 2 phases of the project: (1) an electronic Delphi survey to gather feedback on the draft core set and refine it with input from a wide audience of experts in SDM and rheumatology; and (2) a workshop held at the OMERACT 2016 consensus meeting to gain consensus on the draft core set among attending rheumatology participants. The study was approved by the Research Ethics Board of the Children’s Hospital of Eastern Ontario (CHEO) Research Institute (CHEOREB#16/07X). Anonymous identifiers were used when analyzing the data.
Refined draft core set of outcome domains and subdomains*.
Electronic Delphi survey
A 2-round electronic Delphi survey12 created in RedCap and housed by the CHEO Research Institute was conducted to refine subdomains of the core set.
Eligible participants were patients with OA, RA, and PsA, caregivers of patients with OA, RA, and PsA, and clinicians and researchers involved in rheumatology practice or research, or in SDM research. Members of the OMERACT network were invited by e-mail to complete the online Delphi questionnaire by the chair of the OMERACT SDM working group (KTA). This invitation e-mail was initially sent and an additional e-mail was sent to OMERACT members a week later. Participants could also send the survey to other individuals whom they thought fit the criteria using a snowball method. Because of this, individuals from various consumer and research organizations participated in our survey (e.g., the Canadian Arthritis Patient Alliance, the Cochrane Consumer Group). Those who completed Round 1 were invited to participate in Round 2. Two weekly reminders were sent to participants who did not respond to Round 2. The invitation e-mail was first sent on April 12, 2016, for Round 1 and on April 28, 2016, for Round 2.
The Delphi questionnaire included information about the goals of the research project, the definition and importance of SDM, as well as the effectiveness of SDM interventions and concrete examples of these interventions (i.e., links to 2 rheumatology decision aids). The previous work conducted by our team was also described to explain how the draft core set was developed. The survey also listed the 7 outcome domains and 12 outcome subdomains of the draft core set (Table 1). The subdomains included those used in published trials of SDM interventions6,7, included in the IPDAS9, or suggested at the SIG at the OMERACT 2014 meeting11. The Delphi survey was written in lay language and then modified using feedback from the OMERACT SDM working group, including a panel of patient research partners (PRP; i.e., patients involved in OMERACT). During Round 1, participants were asked to rate the importance of each subdomain on a 9-point Likert scale using the Grading of Recommendations Assessment, Development, and Evaluation process and the RAND (Research and Development) appropriateness rating system13,14. Ratings of 7 to 9 indicated critically important subdomains, ratings of 4 to 6 indicated important but noncritical subdomains, and ratings of 1 to 3 indicated subdomains of low importance. Participants could add additional subdomains. All items were carried through to Round 2 together with descriptive statistics on participants’ Round 1 ratings (i.e., number of raters, mean score, and percentage of respondents who rated each subdomain as critically important). In Round 2, participants reviewed the scores and rated the original core set, and the additional subdomains identified in Round 1. Criteria for including subdomains in the core set were defined a priori. Subdomains were to be included in the potential core set if at least 70% of participants found them critically important (rankings of 7 to 9) and if fewer than 15% of participants found them not important (rankings of 1 to 3)15. ANOVA was used to compare the mean score of each subdomain for patients/caregivers versus clinicians/researchers, and for respondents who had some versus no experience in SDM. Paired Student t tests were conducted to compare the mean score of each subdomain between rounds.
OMERACT consensus meeting
A workshop with breakout groups was held at the OMERACT 2016 meeting in Whistler, British Columbia, Canada, and followed 3 steps.
Step 1: Presentation of background information. Meeting attendees were presented with a description of SDM and an example of a patient decision aid. Commonly assessed domains and subdomains of SDM derived from the literature and the results of the Delphi survey were provided. To help attendees determine outcome domains most important to them, 2 clinical vignettes, 1 with a high level of SDM behaviors and 1 with a low level11, were performed by a clinician and PRP (found at www.dropbox.com/sh/rfystn08dx6c042/AABnUMvPqBn-9dTJlkOF3jCsa?dl=0). Attendees were asked to complete a checklist of the draft core set to indicate whether the outcome domains of SDM were present in each vignette.
Step 2: Breakout group discussion. Attendees were then assigned to 8 breakout groups to ensure representation of all involved (e.g., patients, clinicians). Each one of these groups was led by a facilitator and notes were taken by a rapporteur. In these groups, participants decided whether each domain should be in the inner core set (essential to evaluate in all SDM trials), middle core set (relevant, but not essential to assess in every SDM trial), or outer core set (should be part of a research agenda for future consideration). These domains were displayed on a flip chart, and participants were given 7 stickers to attribute to the various domains (1 per domain). They also discussed reasons for including/excluding domains from the core set.
Step 3: Plenary voting. Rapporteurs from each breakout group presented a summary of their group’s discussion and vote at a final plenary session. Participants then voted using electronic voting clickers to decide whether each domain should be in the inner core set, middle core set, outer core set, or whether they had insufficient information to vote. Each outcome domain was included in a category of the core set if at least 70% of participants endorsed it. A final vote was held to decide whether outcomes associated with SDM interventions were within the scope of OMERACT.
RESULTS
Participants’ characteristics
A total of 170 participants completed Delphi survey Round 1, and 116 completed Round 2 (Table 2). Participants came from 29 countries in North America, Europe, Australia/New Zealand, Asia, and Africa. There was an equal representation of clinicians/researchers and patients/caregivers in both rounds (52% vs 48% in Round 1, and 53% vs 47% in Round 2). The majority of patients were diagnosed with RA (73% in Round 1 and 69% in Round 2) followed by OA and PsA. Participants with no SDM experience accounted for 40% in Round 1, those with limited experience (i.e., had used or developed 1 SDM intervention) accounted for 35%, and those experienced in SDM (i.e., had used or developed 2 or more SDM interventions) accounted for 25%. Participants’ characteristics were similar in Round 2.
Demographic and disease-related characteristics of Delphi participants. Values are n (%).
Ratings of outcome domains by Delphi participants
Delphi results for Round 1 showed that all 12 subdomains, representing all 7 domains, should be included in the potential core set (Table 3). Participants also suggested 5 new subdomains that were added in Round 2. The most important subdomains were knowledge of options, and values clarification and discussion. Compared to clinicians/researchers, patients/caregivers rated the following subdomains as more important: patients’ understanding that their values influence their decision (p = 0.018), values clarification (p = 0.006) and discussion (p = 0.037), patients’ involvement in decision making (p = 0.029), patients’ ability to weigh good and bad features of options (p = 0.021), informed value-based choice (p = 0.035), decisional conflict (p = 0.001), and adherence to the chosen option (p = 0.005). All 12 subdomains reached the 70% threshold for patients/caregivers, while all except decisional conflict reached this threshold for clinicians/researchers.
Voting results for the suggested subdomains from the Delphi survey. Values are n (%)*.
For Delphi Round 2, results showed that 14 out of the 17 subdomains, representing all 7 domains, met the 70% criteria for inclusion in the potential core set (Table 3). The 3 subdomains failing to meet the 70% threshold were decisional conflict, decisional regret, and trust in healthcare providers. The most important subdomains were knowledge of options and values clarification and discussion. According to patients/caregivers, all 17 subdomains reached the 70% threshold, while the same 14 subdomains that were important in the overall analysis were important to clinicians/researchers. Compared to clinicians/researchers, patients/caregivers had higher importance ratings for clarification of values (p = 0.003) and involvement in decision making (p = 0.036). Ratings did not vary with respondents’ experience in SDM.
OMERACT consensus meeting
A total of 189 individuals attended the OMERACT meeting and 21 of them had previously answered the Delphi survey.
Breakout groups. Initial voting was carried out in the 8 breakout groups to help decide which items needed the most discussion. The majority of domains were supported. Domains judged most important concerned the understanding of information and values clarification (each receiving 88% of votes; Table 4). However, as the small group discussions progressed, it became clear that many participants had limited prior experience with SDM. Many participants found it difficult to distinguish between the SDM process and outcomes.
Plenary session. A total of 96 attendees (51% of attendees at the OMERACT meeting), including 8 PRP, 83 clinicians/researchers, and 5 policymakers, voted at the plenary session held immediately after the breakout group discussions. According to this plenary vote, none of the domains reached the 70% threshold (Table 5). Participants felt that the most important domains to include in the inner core were the effect of the decision (68%), understanding information (67%), and clarifying and discussing patients’ values (62%). A further vote was taken to assess whether the audience supported this initiative going forward and 84% indicated their support.
Voting results of the OMERACT 2016 breakout groups. Values are n (%).
Voting results of the OMERACT 2016 meeting plenary session. Values are n (%).
DISCUSSION
Our current study aimed to determine the core set of outcome domains and subdomains for measuring the effectiveness of SDM interventions in rheumatology clinical trials. Results from the Delphi survey, with a high proportion of patients/caregivers (48%) and individuals with experience with SDM (60%), showed that 14 of 17 subdomains, representing all 8 IPDAS domains, were considered critically important. However, at the OMERACT 2016 workshop and breakout groups, only 8% were patients, and despite their strong support for the core set, their views were outvoted by the others, who were predominantly rheumatology clinicians/researchers.
To our knowledge, our study is one of the first to seek consensus on SDM outcome domains with a considerable amount of both patient and clinician/researcher input. With nearly equally sized groups of patients/caregivers and clinicians/researchers in the Delphi survey, the weight of both sets of opinions were relatively equivalent. Although 3 subdomains did not meet the 70% threshold in the overall Delphi results, they did among patients. This is consistent with the IPDAS Delphi survey, which found higher endorsement of outcome domains among patients versus researchers9. These contrasting levels of endorsement pose the question of whether studies should ensure a balance in number of patients/caregivers and others, and if not, whether results should be weighted to allow for opinions of both sets to be equally taken into account.
While all 7 domains included subdomains considered important in the Delphi survey, none reached the 70% threshold at the final plenary vote. These contrasting levels of endorsement may be explained by differences in participant characteristics and a different voting context in each setting. The lower proportion of patients at the consensus meeting compared to the Delphi survey (8% at the meeting vs 47% in Delphi Round 2) may have led to lower levels of outcome domain endorsement, because patients tend to rate SDM outcome domains higher than others do. Additionally, because the Delphi survey was sent by e-mail to potential participants, those taking the time to complete it may have been more motivated because of particular SDM interest or expertise, leading to stronger endorsement compared to attendees at a meeting not specifically dedicated to SDM. Further, because the second reminder for the Delphi survey was a few days before the OMERACT meeting, it is possible that only a subset of more motivated individuals found the time to answer the second round, thus potentially leading to stronger endorsement of the subdomains of the core set. Any effects from relatively lower SDM familiarity among consensus meeting participants may also have been heightened by the influences of group pressure that are more likely to arise in a face-to-face setting than an online survey16. However, as the final vote at the consensus meeting demonstrated, there is interest in continuing this work within OMERACT.
Breakout group discussions and plenary session voting results revealed some challenges in the comprehension of SDM concepts used, possibly because of the complexity of domains that included various subdomains. Attendees also mentioned confusion because some domains assessed the SDM process rather than its outcomes, and seemed to prefer outcomes, as shown by the high endorsement for the effect of the decision at the plenary session. These findings underscore the need to revise and clarify terms and definitions used by our working group to promote better understanding of SDM process and outcome domains. Using the language suggested by Sepucha, et al17 about the quality of the decision-making process and the quality of the choice made may help to increase comprehension of SDM outcome domains.
The results of the plenary session vote could be partly due to the participants’ lack of experience with SDM, above-mentioned process/outcome confusion, and the preference for outcomes of SDM rather than its process. The relatively low percentage of OMERACT attendees who voted at the plenary session (51%) may further reflect a lack of interest or experience with SDM within OMERACT or simply participant fatigue experienced on the last day of the meeting. It is also possible that voting and nonvoting participants at the plenary session may have had different opinions on the core set, but we do not have the data to verify this hypothesis.
This experience shows the importance of involving everyone to the same extent and to ensure clarity and a shared understanding of the core set of outcome domains. Many OMERACT attendees seemed to lack sufficient confidence in understanding to vote in favor of the core set. Further efforts should aim at providing additional education about SDM, dispelling its myths18, promoting understanding of its process and outcomes using clear terms and definitions, and ensuring that patients’ voices are heard throughout the consensus process. The next steps include the development of a white paper to justify and describe this work, interviews with opinion leaders within OMERACT and the SDM community to increase the clarity and relevance of the core set among OMERACT, as well as further consensus building efforts (Delphi survey and consensus meeting) after ensuring a thorough understanding of opinion leaders within OMERACT through educational activities (e.g., Webinars).
Meaningfully enhancing communication with patients as part of this initiative will facilitate the development of consensus on a core set to assess the effectiveness and safety of SDM interventions, as well as contextual factors that may affect their outcomes (see OMERACT Master Checklist on OMERACT’s Website). Patients’ high endorsement of outcome domains suggests a need to further develop this core set despite the lack of clinician/researcher endorsement. As an OMERACT PRP phrased it: “Patients must be involved in their care and treatment decisions not just to ensure patient-centered care, but also so that patients understand and take responsibility for these decisions. SDM is the wave of the future; we can’t run away from it, we have to tackle it together.”
Acknowledgment
The authors thank participants of the Outcome Measures in Rheumatology 13 (2016) meeting who participated in the working group on shared decision making. We also thank members of the working group: Sarah Collins, Thomas Chong, Pamela Richards, Ailsa Bosworth, Pamela Montie, Francis Guillemin, Jennifer Petkovic, Viviane Grandpierre, Melissa Mannion, Cécile Gaujoux-Viala, Anne Stiggelbout, Ayano Kelly, and Nick Bansback. Finally, thank the Canadian Arthritis Patient Alliance for disseminating our Delphi survey.
Footnotes
K. Toupin-April is funded by the Children’s Hospital of Eastern Ontario Research Institute and The Arthritis Society. J. Barton’s research reported in this publication was supported by the US National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), part of the National Institutes of Health (NIH), under award number K23-AR-064372. L. Fraenkel’s research reported in this publication was supported by the NIAMS, part of the NIH, under award number AR060231-01. R. Christensen’s research at the Parker Institute is supported by grants from The Oak Foundation. L. March’s research is supported by the Northern Sydney Local Health District. J. Kaufman and S. Hill receive funding from the Research Council of Norway (COMMVAC 2 project grant number 220873). S. Hill is also supported by a Funding and Service Agreement, Department of Health and Human Services, Victoria, and La Trobe University.
- Accepted for publication June 3, 2017.
