Abstract
Objective. Outcomes important to patients are those that are relevant to their well-being, including quality of life, morbid endpoints, and death. These outcomes often occur over the longterm and can be identified in prospective longitudinal observational studies (PLOS). There are no standards for which outcome domains should be considered. Our overarching goal is to identify critical longterm outcome domains for patients with rheumatic diseases, and to develop a conceptual framework to measure and classify them within the scope of OMERACT Filter 2.0.
Methods. The steps of this initiative primarily concern rheumatoid arthritis (RA) and include (1) performing a systematic review of RA patient registries and cohorts to identify previously collected and reported outcome domains and measurement instruments; (2) developing a conceptual framework and taxonomy for identification and classification of outcome domains; (3) conducting focus groups to identify domains considered critical by patients with RA; and (4) surveying patients, providers, and researchers to identify critical outcomes that can be evaluated through the OMERACT filter.
Results. In our initial evaluation of databases and registries across countries, we found both commonalities and differences, with no clear standardization. At the initial group meeting, participants agreed that additional work is needed to identify which critical outcomes should be collected in PLOS, and suggested several: death, independence, and participation, among others. An operational strategy for the next 2 years was proposed.
Conclusion. Participants endorsed the need for an initiative to identify and evaluate critical outcome domains and measurement instruments for data collection in PLOS.
The ultimate goal of medicine is to improve health in ways that matter to patients. For medical research to be “patient-centered,” it must include outcome domains that are important to the well-being of patients, such as health-related quality of life, morbid endpoints (some of which may be rare, such as adverse events from therapy), or death. These outcomes often need longer-term evaluation because they are not completely identified in shorter-term randomized controlled trials (RCT). This is especially true for chronic conditions such as rheumatoid arthritis (RA).
The Outcome Measures in Rheumatology (OMERACT) initiative has successfully proposed and implemented strategies for improving the reporting of outcomes in different conditions, including RA1,2,3,4. The initiative has primarily focused on RCT, with some work on prospective longitudinal observational studies (PLOS). The more recent OMERACT filter 2.0 framework has been proposed for both5. Further, there has been an initiative for observational studies in rheumatology, but not specifically in RA6.
The past decade has seen exponential growth in PLOS. These studies complement RCT given that (1) RCT may not be ethical/feasible for longer durations (placebo limited to 12 weeks in RA), (2) RCT are not powered to detect rare important effects, and (3) PLOS provide “real-world evidence,” including populations excluded from RCT7. When RCT evidence is lacking for an important longterm outcome, it may be generated by PLOS8. It is then crucial that PLOS be held to high methodological standards, as RCT are9.
There have been previous efforts to standardize which outcomes should be systematically ascertained in registries10,11. Bias is likely to occur if selected populations are more likely to be included, or if only selected outcomes are reported, e.g., benefits but no harms. Further, there is no clear understanding of the value that patients may place on the specific outcomes or events measured in both PLOS and RCT12. Finally, clear methodologies for outcome collection and reporting have not been proposed, but are essential to compare (and potentially combine) outcome measures across studies9.
While extensive work has been performed in selecting a core set of outcomes for RCT in RA, many domains that may be critical to patients are not part of the core set; these include a number of patient-centered outcomes that relate to functioning and independence (e.g., productivity, social participation), and also longer term effects that can occur years after the trial has ended (e.g., survival, cardiovascular events).
We established a Special Interest Group (SIG) to identify, classify, and evaluate longterm critical outcome domains relevant to patients that can be collected in PLOS. We expect to set critical outcome measures that when included and reported in PLOS will contribute to the body of evidence necessary for informed health decision making8.
MATERIALS AND METHODS
The steps of this initiative include (1) performing a systematic review of RA patient registries and cohorts to identify reported outcomes and methods of collection; (2) evaluating and refining a conceptual framework and taxonomy that can be assessed using Filter 2.0 based on a systematic review of previous methods used in observational studies and registries13; (3) conducting patient focus groups to identify longterm critical outcome domains that are considered important by patients with RA (we consider longterm outcomes as those that occur after 10 yrs of disease duration. However, given that it is also important to include expectations of patients with shorter disease duration, we will invite patients with 5 or more yrs); and (4) surveying patients, healthcare providers, and researchers in the field of RA to identify a set of critical outcome domains and instruments for PLOS that can be fully evaluated through the OMERACT filter later.
These preliminary steps did not involve human subject research and did not require ethics board approval.
RESULTS
Systematic review of RA patient registries and cohorts
Within the scope of rheumatic diseases, several efforts have been launched to establish patient cohorts and registries. Many RA registries have been established with the aim of providing epidemiologic data on disease activity over time, treatments, prevalence and incidence of comorbidities, and longterm outcomes14.
There is overlap in the definitions of cohort and registry studies. A cohort study includes patients with defined characteristics, followed over time, with the purpose of answering a priori–defined clinical questions. A patient registry also follows a cohort, usually population- or community-based, aiming to include all patients with a specific feature (e.g., disease or received therapy) within a defined geographical location or defined healthcare systems. Often, registries collect data without defined hypotheses. Study questions are proposed as data are being collected over time15. Many registries include patients with RA irrespective of their treatment, while others collect specific therapies data, most often biologic agents. A comparison of selected European and US RA registries reported similarities, but also variations in data collection, including differences in variables ascertained (e.g., radiographs) and methods (e.g., linkage to other registries, administrative data sources)16. Clear specification of a ranking of outcomes according to their importance for patients is not typically included.
Definition of a “critical outcome.”
The Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group has suggested that direct evidence is more important in determining whether an outcome is critical for decision making, i.e., if outcomes important to patients are represented by surrogate measures when evidence summary is needed, there would be less confidence in the estimate17.
To judge how important any outcome is, GRADE suggests listing both desirable (e.g., ability to work) and undesirable outcomes (e.g., irreversible joint damage) and categorizing their relative importance by rating them numerically on a 1 to 9 scale (i.e., 7–9 = critical, 4–6 = important, and 1–3 = of limited importance; Figure 1)17. This approach helps to maintain balance while focusing on those outcomes that patients consider more significant. It also helps to resolve or clarify disagreements, recognizing that the reported importance of outcomes is likely to vary within and across cultures, between patients and others involved, and among individual patients.
Hierarchy of outcomes according to their importance to patients. An indicative example of the potential ranking of outcomes from the perspective of patients with rheumatoid arthritis. CRP: C-reactive protein.
The Outcome Measures Framework Model (OMFM)
The OMFM is a non–disease–specific conceptual framework for customary outcome measures to be used in longitudinal studies and registries18. We have mapped the OMERACT framework core areas and domains to the OMFM domains (i.e., characteristics, treatments, outcomes), categories, and subcategories to identify similarities and differences between the frameworks (Table 1)18. Whereas both frameworks focus on outcomes, the OMFM offers more detailed definitions for the “Characteristics” and “Treatment” domains. While these domains would be categorized primarily as Patient/Intervention/Control elements (PICO model) and contextual factors in the OMERACT framework, “Characteristics” and “Treatments” in the OMFM could also serve as outcomes in themselves, e.g., to identify a change in treatment. Precise definitions of the terminology to describe the core areas and domains in OMERACT and the categories and subcategories for OMFM are sometimes lacking and not always self-explanatory. Guidance to optimize the timing of assessments is relatively lacking from both frameworks. Overall, the comparability between the OMFM and OMERACT frameworks indicate that both broad non-disease–specific and therapeutic area–specific frameworks address similar concepts.
Comparing the OMERACT and OMFM frameworks. If specific OMERACT domains or OMFM subcategories are not identified, all are assumed to be relevant.
Initial OMERACT meeting
The overarching goal of the SIG is to establish a hierarchy of validated critical outcome domains that are important to patients and should be evaluated in PLOS. Attendees included patient advocates, clinical researchers, and industry representatives. Three major issues were discussed:
Scope: Cohort studies can have different objectives and methodologies. Differences between cohort studies, registries, and administrative databases were discussed, specifically regarding the objectives of each method (Figure 2)19. The general agreement was to limit the scope of our activities to large prospective cohorts or registries, under the term “prospective longitudinal observational studies.”
Feasibility: Concerns were expressed about possible burden in data collection, and potential difficulties in changing current processes used by registries in various countries. It was emphasized that the objectives of the SIG should incorporate efforts to be parsimonious, focusing primarily on domains considered to be critical.
Consideration of critical outcomes: Several patient partners attending the SIG session were engaged in the discussion of which outcome domains might be targeted for initial consideration. Participation, including work and social and leisure activities, were considered crucial, even more significant than survival. The importance of individual and contextual factors was recognized as relevant in determining whether an outcome domain may be more important to some individuals than others (e.g., death in younger vs older people). Participants agreed on the methods proposed to continue this work (qualitative data and surveys of people involved).
A variety of longitudinal observational studies. A registry is based on the population in mind while a cohort is based on the question to be answered. Claims databases are defined as data available and organized that can be used to answer many questions. Some longitudinal observational studies do not collect information related to 1 question, and they collect as much data as possible.
DISCUSSION AND RESEARCH AGENDA
All SIG participants agreed that critical outcome domains in PLOS may be different from those collected in RCT. The need for an initiative to identify, evaluate, and propose critical outcome domains and measures for data collection in PLOS was clearly expressed. Data will be gathered following the steps agreed upon by the SIG to identify a preliminary set of critical outcome domains and measures that can be further evaluated and ratified at the OMERACT meeting.
- Accepted for publication March 22, 2017.
