Abstract
Objective. The Contextual Factors Working Group aims to provide guidance on addressing contextual factors in rheumatology trials within OMERACT.
Methods. During the Special Interest Group session at OMERACT 2018, preliminary results were presented from a case scenario survey and semistructured interviews, including contextual factors mentioned in these. A group-based exercise sought to identify and rank important generic contextual factors.
Results. A total of 79 candidate factors were listed. Across the 3 groups, gender/sex, comorbidities, and the healthcare system were ranked as most important.
Conclusion. The identified important contextual factor domains may be considered a provisional list pending further research.
The term contextual factor is used in different settings without an agreed-upon definition. Contextual factors may be related to prognostic and/or predictive factors1,2,3, and may explain heterogeneity in treatment effects. In 2012, the concept of contextual factors was introduced for the development of core outcome sets (COS) within rheumatology. Within the Outcome Measures in Rheumatology (OMERACT) organization, the COS [including “what” (domains) and “how” (instruments) to measure] are developed according to OMERACT methodology4,5,6. COS developers need to consider whether any contextual factors need to be measured to fully understand the findings of trials. Great heterogeneity exists within OMERACT regarding identification and characterization of contextual factors; thus the Contextual Factors Working Group (CFWG) was formed to develop guidance. In the OMERACT filter, a contextual factor is defined as a “variable that is not an outcome of the study, but needs to be recognized (and measured) to understand the study results. This includes potential confounders and effect modifiers”4,5. This definition is conceptual and needs operationalization for proper consideration of contextual factors.
In our paper, we outline a list of provisionally important generic contextual factors, our 6-step research plan, and feedback from our Special Interest Group (SIG) session at OMERACT 2018.
MATERIALS AND METHODS
Based on our SIG session at OMERACT 20167, we developed a 6-step research plan (Figure 1). We clarified that overall objectives include providing an operationalized definition and guidance on how to address contextual factors in rheumatology trials when developing COS. Further, we aim to provide a generic set of important contextual factors (i.e., important across rheumatic diseases) that should always be considered in rheumatology trials.
Substudies and goals of the research plan for the Contextual Factors Working Group. OMERACT: Outcome Measures in Rheumatology; CF: contextual factors; RCT: randomized controlled trials; PRP: patient research partners.
Prior to OMERACT 2018, we conducted a case scenario survey (known as study 1a) and initiated expert interviews and patient research partner (PRP) group interviews (known as study 1b). For the case scenario survey, we e-mailed a survey to the chairs of OMERACT working groups asking them to provide up to 3 case scenarios relevant to their field of research (preferably with references to the evidence) in which contextual factors are strongly suspected to have affected the results of a trial.
For the interviews, we invited clinicians and researchers considered experts with recognized knowledge of contextual factors. Each expert participated in an individual semistructured interview based on an interview guide intended to obtain the definition, terminology, identification, and use of contextual factors, including reflections on the current OMERACT definition. PRP associated with the working group were invited for group interviews involving the same topics as the expert interviews. Study protocols are available online (at www.parkerinst.dk).
For OMERACT 2018, we organized a 1.5-h SIG session open to all OMERACT delegates. Prereading materials described the proposed research plan. During the session we presented preliminary results from the case scenario survey and the interviews, including a list of potential contextual factors. The factors were listed “as mentioned” and minimal merging (e.g., merging “disease duration” and “duration of disease”, but not “adherence to treatment” and “medication adherence”) was done to minimize influence from our own interpretations. Based on the list, a group-based exercise was conducted to identify and rank important generic contextual factors. The participants were divided into 2 × 2 groups; for each group, a facilitator and rapporteur had been identified. The list was split into 2 to make the exercise manageable within the allocated time; Group A-I and A-II received the first half, whereas group B-I and B-II received the second half. The groups were instructed to identify the factors from their list that they considered important across different rheumatic diseases (i.e., generic). Next, the 3 to 7 most important generic contextual factors were selected and ranked (factors not already on their list could be added if necessary).
Across the groups, an overall ranking of contextual factors according to perceived importance was based on mean reciprocal ranks (MRR)8 calculated for each factor as MRR = 1/N × ∑ 1/Ri, where N = 2 and Ri is the ranking by group i. Larger MRR means lower numerical overall rank, and hence greater importance. A plenary discussion invited participants to discuss and provide proposals for the research agenda.
In Denmark, no ethical approval is required for surveys and interviews. Approvals from the Danish Data Protection Agency have been obtained for the case-scenario survey (study 1a) and the interviews (study 1b). Consent to publish the survey results was obtained from the survey participants.
RESULTS
For the case scenario survey, we contacted the chairs of the 45 OMERACT working groups and received answers from 11, resulting in 19 case scenarios (Supplementary Table 1, available with the online version of this article). Sixteen of these referred to a publication, including 3 metaanalyses of randomized controlled trials (RCT), 6 RCT, 6 observational studies, and 1 narrative review. The case scenarios included 37 different contextual factors.
Preliminary data from the interviews provided 34 contextual factors from experts and 18 from PRP. Among these, PROGRESS-PLUS (acronym for place of residence, race/ethnicity/culture/language, occupation, gender/sex, religion, education, socioeconomic status, social capital, age, disability, and sexual orientation)9,10, i.e., factors considered relevant for health equity, were proposed as important. As anticipated, across the interviews, different opinions exist on which types of factors should be considered as contextual factors. Overall, the types of factors suggested can roughly be summarized into 5 categories: person characteristics (e.g., age, sex), disease-related (e.g., duration, severity), environmental factors (e.g., country, healthcare system), measurement-related (e.g., language proficiency when using questionnaires, characteristics of leisure activities when measuring function in these), and study design characteristics (e.g., year of conduct).
A total of 79 unique contextual factors were listed (Supplementary Table 2, available with the online version of this article) based on the case scenarios and interviews, with a small overlap among sources. At the SIG session, the group-based exercise included 3 groups (group A-I and A-II erroneously merged for convenience). Thirty-three people participated in the group-based exercise, with between 8 and 13 in each group. The result of the exercise is shown in Table 1 (and Supplementary Table 3), and the overall ranking of important generic contextual factors is shown in Figure 2.
Overall ranking of important contextual factors based on results from the group-based exercise at the OMERACT 2018 Contextual Factor Session. *Contextual factors are ordered according to overall rank, and those shown in bold are the 7 with the highest overall rank and may be considered a set of provisional important contextual factors. **The percentages illustrating importance of the factors was calculated by multiplying the mean reciprocal rank (MRR) by 100, where MRR = 1/N × ∑ 1/Ri. For example, “Sex” was ranked 1 and 4 by group B-I and B-II, respectively, meaning that the mean reciprocal rank is 1/2 × (1/1 + 1/4) = 0.625, corresponding to 62.5% on the scale. In the case of group A-I and A-II, these were considered separate groups obtaining the same results, despite actually merging their efforts. ***The overall ranking is based on percentages illustrating importance for the contextual factor. The larger MRR, the lower the overall rank, and the more important the factor is perceived. In case of contextual factors with the same percentages, they share 2 ranks in the overall ranking. For example, Comorbidities and Healthcare system both have a percentage of 50%, meaning they share the overall rank 2 and 3. ****Gender and sex were merged based on the assumption that these in practice will be very similar, and because clear distinctions were not made during the Special Interest Group session.
Results from the group-based exercise at the OMERACT 2018 Contextual Factor Session.
In the plenary discussion, it was suggested that “putative important generic contextual factors” should initially be factors that people intuitively think are important across diseases when interpreting trial results. These factors need to be confirmed as important in the trial setting and need additional evidence for (strong) effect modification. Some factors from our list, e.g., sex, race, and age, are already routinely used to stratify analyses of therapeutic trials submitted to the US Food and Drug Administration (FDA)/European Medicines Agency (EMA), and the FDA database, Drug Trials Snapshots, could contribute evidence. It was recommended that the working group focus on a few generic contextual factors and find evidence for their importance, investigate feasibility, and provide guidance.
The working group was tasked to make the concept of contextual factors easier to understand by elaborating on why contextual factors are important and provide good examples of the extent to which they can influence RCT findings. In addition to generic factors, disease-specific contextual factors were to be considered, and identified based on the knowledge of the disease. Finally, the list of candidate generic contextual factors will be refined further, prior to using it in study 2 (Figure 1), by removing factors that may be synonymous in concept and in practice. Further, use and implementation were to be considered because some factors may be too broad or difficult to measure (e.g., genetic factors, which might not be feasible).
DISCUSSION
Since OMERACT 2016, the working group has developed a 6-step research plan, completed a case scenario survey (study 1a), and is halfway through study 1b (Figure 1). At OMERACT 2018, the group-based exercise provided important input for the next study (study 2), i.e., a consensus process to produce a prioritized list of potentially important generic contextual factors. Prior to initiating study 2, the list will be refined based on the input provided by SIG participants. Further, the 3 to 7 highest-ranked generic contextual factors, i.e., gender/sex, comorbidities, the healthcare system, psychological well-being, adherence to treatment, age, and previous exposure to drugs, can be used as a draft of consensus-based, provisionally endorsed generic contextual factor domains for rheumatology trials until further results are available. However, different ranks might have been obtained if group A-I and A-II had kept their efforts separate as planned, because this was an assumption of the MRR calculations. In addition, trialists can further consider the factors within PROGRESS-PLUS9,10, factors recognized by the FDA and EMA, and factors specific to the outcome or disease studied (e.g., presence of autoantibodies). For metaanalyses, study design characteristics (e.g., year of conduct) may also be considered.
The next steps in the research plan will include a consensus process based on topics from the interviews (last part of study 1b), followed by study 2 that seeks consensus on a short, prioritized list of potentially important generic contextual factors. For these factors, evidence from trials and cohorts will be sought (studies 3 and 4). This is anticipated to inform a process for developing a generic set of important contextual factor domains that should always be considered in rheumatology trials. Subsequently, the working group will proceed with identification and selection of instruments to measure these domains.
ONLINE SUPPLEMENT
Supplementary material accompanies the online version of this article.
Acknowledgment
We thank all participants in our session at OMERACT 2018, as well as the chairs who responded to our survey (study 1a) and the experts and PRP who participated in the interviews (study 1b). A special thanks to the PRP for their positive engagement and input.
Footnotes
JAS is a member of the executive of OMERACT, an organization that develops outcome measures in rheumatology and receives arms-length funding from 36 companies. AB received research grants to her department from Celgene and AbbVie. The views expressed in this article are personal views and may not be understood or quoted as being made on behalf of or reflecting the position of the European Medicines Agency or one of its committees or working parties.
- Accepted for publication November 30, 2018.
