Abstract
Objective. The importance of contextual factors (CF) for appropriate patient-specific care is widely acknowledged. However, evidence in clinical trials on how CF influence outcomes remains sparse. The 2014 Outcome Measures in Rheumatology (OMERACT) Handbook introduced the role of CF in outcome assessment and defined them as “potential confounders and/or effect modifiers of outcomes in randomized controlled trials.” Subsequently, the CF Methods Group (CFMG) was formed to develop guidance on how to address CF in clinical trials.
Methods. First, the CFMG conducted an e-mail survey of OMERACT working groups (WG) to analyze how they had addressed CF in outcome measurement so far. The results facilitated an informed discussion at the OMERACT 2016 CFMG Special Interest Group (SIG) session, with the aim of gaining preliminary consensus regarding an operational definition of CF and to make a first selection of potentially relevant CF.
Results. The survey revealed that the WG had mostly used the OMERACT Handbook and/or the International Classification of Functioning, Disability and Health (ICF) definition. However, significant heterogeneity was found in the methods used to identify, refine, and categorize CF candidates. The SIG participants agreed on using the ICF as a framework along with the OMERACT Handbook definition. A list with 28 variables was collected including person-related factors and physical and social environments. Recommendations from the SIG guided the CFMG to formulate 3 preliminary projects on how to identify and analyze CF.
Conclusion. New methods are urgently needed to assist researchers to identify and characterize CF that significantly influence the interpretation of results in clinical trials. The CFMG defined first steps to develop further guidance.
The importance of contextual factors (CF) for appropriate, patient-specific care, especially in chronic conditions such as rheumatic diseases, is widely acknowledged1,2,3,4. CF may include sociodemographics, person-related factors, and physical and social environments5. However, despite logical arguments and clinical experience, evidence in clinical trials on how CF influence outcomes remains sparse6. Most researchers agree that CF, such as age, sex, and duration of disease, should be identified in rheumatic randomized controlled trials (RCT) to check whether an unequal distribution of CF, despite randomization, could confound the outcome. However, little is known concerning the influence of person-related factors or physical or social environment.
In addition, CF such as phenotypical subgroups (e.g., differences in disease subgroups, previous pharmacological management, or personal or environmental characteristics) can distort the net benefit (or harm), and thus have potential to act as “effect modifiers”7. Figure 1 illustrates a hypothetical RCT example where patients were randomized to either active intervention or placebo. The trial illustrates that these interventions are equally effective. However, reanalyzing the dataset and stratifying the analysis according to a potential CF revealed a divergent efficacy pattern in favor of the active intervention compared with the placebo in the CF-positive subgroup. For those who design trials, CF acting as effect modifiers can provide a quantitative perspective elucidating a difference in effect (i.e., net benefit) between subgroups. This has important implications for clinical practice and policymaking, such as calling for more individualized treatment strategies8.
A hypothetical randomized controlled trial example in which 1000 patients were randomized (499 vs 501) to either active therapy or placebo and showing that both interventions are equally effective (RR SubTotal = 1.00). However, reanalyzing the dataset, stratifying the analysis according to a potential contextual factor, reveals that active intervention is more effective than placebo in the CF-positive category compared with the CF-negative (RRCF+ = 2.68 and RRCF− = 0.44, respectively). RR: risk ratio; CF: contextual factor; M-H: Mantel-Haenszel test.
Acknowledging the need to integrate CF into the outcome measurement in rheumatic RCT, in 2012 the concept of CF was introduced for the first time in the Outcome Measures in Rheumatology (OMERACT) process in a preliminary version of the OMERACT Handbook. CF were defined as “variables that are not outcomes of studies, but need to be recognized (and measured) to understand the study results. This includes potential confounders and effect modifiers”9. Several OMERACT working groups (WG; Worker Productivity10, Hand-Osteoarthritis11, Vasculitis12, RA-Flare13, and Health Literacy WG14), in consideration of input from patient research partners (PRP), started to include CF in their research. However, the research presented in OMERACT 2014 revealed great heterogeneity in understanding, approaching, and identifying CF. To address this confusion, the CF Methods Group (CFMG) was formed, representing “an entirely new work stream to address newly identified challenges”15. The mission of this group is to provide guidance to the OMERACT community and other researchers on the fundamental steps that should be implemented to identify CF that are essential for interpreting results in the setting of an RCT in rheumatology. The group consists of clinicians, statisticians, researchers, and PRP from the OMERACT WG already involved in CF research. The first objectives of the CFMG were:
To agree on the operational definition of CF (that can be applied to core sets or specific outcomes) among all stakeholders.
To inform the CFMG research agenda on how:
To identify methods for the selection of relevant CF and for the statistical testing of its effect; and
To understand whether the agreed definition can be applied to all settings (core sets, specific outcomes).
In its 2016 report, the CFMG highlighted the need to clarify the concept of “CF” in light of outcome measure development according to the OMERACT process. Based on the OMERACT CF definition and the International Classification of Functioning, Disability and Health (ICF) framework16, an operational definition of CF was agreed on and a research agenda was formulated.
MATERIALS AND METHODS
In spring 2015, the CFMG analyzed the conceptualization of CF and previous research by OMERACT WG engaged in CF research in an e-mail survey. Ten questions were formulated by the CFMG members addressing the CF definitions used and the approaches to identify potentially important CF as well as strategies applied to measure and analyze the effect of CF (Table 1). The results were tabulated and the content summarized.
Result and summary of the survey on work done previously on CF within OMERACT WG.
At the OMERACT 2016 CFMG Special Interest Group (SIG) session, a preliminary consensus on a potential operational definition of CF was established based on an informed discussion. A preliminary list of candidate CF to be considered when interpreting an outcome in rheumatology clinical trials was collected in a group exercise and on individual written forms (post-SIG questionnaire). Then the CFMG requested recommendations to further develop the research agenda from the SIG participants.
RESULTS
Survey of OMERACT WG
Response to the survey was received from 8/10 OMERACT WG: Ankylosing Spondylitis, Equity, Hand-OA, Health Literacy, RA-Flare, Shared Decision-Making, Vasculitis, and Worker Productivity. The survey results are presented in Table 1 and Supplementary Table 1 (available with the online version of this article).
Five of the 8 groups used the OMERACT Handbook 2.0 definition9, of which 3 groups also used the CF definition of the ICF, i.e., environmental and personal factors16. The Health Literacy group defined CF specifically as “a factor/variable that may modify the level or importance of the patient-reported outcome (PRO) measured.”
Depending on the specific research focus, multiple methods were used to identify, refine, and categorize CF candidate categories including literature search, ICF or ICF core sets17, expert discussions, patient interview and focus groups, and PRP and SIG participant discussions. This variety emphasized the great heterogeneity in approaching and identifying CF across OMERACT WG.
In their research, some WG identified potential confounders or covariates specific to their research topic, e.g., “patient’s ability to accurately complete a PRO,” identified by the Equity group. As another example, self-management was identified initially as a domain to be measured by the RA-Flare WG, but when scoring was analyzed, variability of answers to questions designed to assess self-management resulted in determining that self-management is probably an effect modifier itself. The Assessment of Spondyloarthritis international Society Health Index identified 9 items of potentially relevant CF for testing in their new instrument, while others proposed factors used to identify phenotypical subgroups (Hand OA)18.
Participation of PRP
PRP initially focused on the influence of CF on transferability of study results to daily life. However, in discussions, PRP agreed to focus on CF influence on the interpretation of outcomes in RCT (not clinical care or daily life).
OMERACT SIG 2016
Forty-eight participants attended the CFMG SIG session, including 35 healthcare professionals, 6 fellows, 5 PRP, and 2 industry representatives.
After presenting the survey results to the participants, 28 variables were collected verbally and displayed, stimulating active discussion on the operational use of the OMERACT CF definition, the ICF framework, methods to identify CF, and approaches to select core CF. SIG participants acknowledged that research is complicated by the large number of CF. Further, depending on the setting, the study design, or research question, CF could be seen as potential confounders, effect modifiers, (co)-outcomes, or even as interventions. These findings were confirmed by 39 participants who provided written input to a post-SIG questionnaire (Data Supplement 1, available with the online version of this article), of which 11 listed the variables as potential core CF (Table 2A, Table 2B, and Table 2C).
Personal factors named by CF-SIG participants.
Health condition/symptom factors named by CF-SIG participants.
Environmental factors named by CF-SIG participants.
Moreover, the CFMG SIG participants agreed that the OMERACT definition (focusing on effect modification in most settings) should be used as the main operational definition with the ICF as the conceptual framework. They confirmed the relevance of the CFMG as an OMERACT methods group to provide guidance to other groups in identifying, measuring, and characterizing important/core CF.
Further, the CFMG SIG participants made the following recommendations for the research agenda:
The CFMG should closely collaborate with other WG because these groups may develop measures for CF.
Statistical methods are needed to prove the effect of CF on effect modification. As a first step, identifying existing datasets that can be used for secondary analysis should be considered.
Based on these recommendations, the CFMG formulated 3 main projects as first steps toward providing guidance to identify and characterize CF that significantly influence the interpretation of results in clinical trials:
Delphi exercises (including experts and patients) to identify CF of importance within rheumatology with suspected effect modification.
Literature reviews to find evidence whether these CF are affecting the effect sizes in either RCT (using stratification or posthoc analyses) or in metaanalyses19.
Investigation of how a CF should be (validly) measured.
DISCUSSION
In the context of outcome measurement in rheumatologic clinical trials, the OMERACT Handbook definition of CF, focusing on effect modification and using the ICF as a conceptual framework, was found to be pertinent. It is important to note that this definition depicts CF that are relevant to interpreting outcomes of clinical trials and may not cover the needs of clinical practice settings20,21,22.
Despite the consensus on a CF definition, the characterization of core CF remains a challenge, partially because the influence of most CF tends to vary according to the context23. Many CF have been identified as potentially relevant in interpreting outcomes of RCT, although only a few might fulfill the definition of effect modification24.
As healthcare evolves toward person-centered medicine, CF might be key to optimizing treatment allocation. However, to even have the opportunity to prove the effect of a distinct CF, studies providing strong arguments for including that specific CF in RCT are needed first19,25,26,27, and will provide a next step toward understanding the effect of CF on outcomes in clinical trials.
ONLINE SUPPLEMENT
Supplementary material accompanies the online version of this article.
Acknowledgment
We thank all PRP, previous OMERACT fellows, and the participants of the CFMG Special Interest Group at OMERACT 2016, the pre-OMERACT meetings at the European League Against Rheumatism meeting 2014/15, and the multiple teleconferences for their valuable input. We also thank all OMERACT working groups who generously shared their work on CF with us, including also the Gout and Osteoarthritis-Flare group. Specifically, we thank Drs. Maarten de Witt, Peter Merkel, Will Taylor, Désirée M. van der Heijde, Sarah Legget, and Roxanne Cooksey for sharing their thoughts and providing valuable input to this project in many discussions and meetings.
Footnotes
The Musculoskeletal Statistics Unit at the Parker Institute (SMN and RC) is supported by grants from the Oak Foundation.
- Accepted for publication March 27, 2017.
