Abstract
Objective. The Outcome Measures in Rheumatology (OMERACT) Filter 2.0 framework was developed in 2014 to aid core outcome set development by describing the full universe of “measurable aspects of health conditions” from which core domains can be selected. This paper provides elaborations and updated concepts (OMERACT Filter 2.1).
Methods. At OMERACT 2018, we discussed challenges in the framework application caused by unclear or ambiguous wording and terms and incompletely developed concepts.
Results. The updated OMERACT Filter 2.1 framework makes benefits and harms explicit, clarifies concepts, and improves naming of various terms.
Conclusion. We expect that the Filter 2.1 framework will improve the process of core set development.
Since 1992, the Outcome Measures in Rheumatology (OMERACT) initiative has successfully improved outcome measurement for many rheumatologic conditions. It has done so by developing widely endorsed core outcome measurement sets that include a minimum set of outcome measures to be reported in all randomized controlled trials (RCT) in a given health condition. As OMERACT grew, its framework and process to develop core domain sets and core outcome measurement sets needed clarification. This was provided by the OMERACT Filter 2.0 framework and process published in 20141. Briefly, to improve content validity of core sets, Filter 2.0 intended to describe the full universe of “measurable aspects of health conditions” by introducing 4 core areas in which all domains of measurement would fall: death, life impact, resource use, and pathophysiological manifestations (Figure 1)1. Core set developers were required to choose at least 1 domain in each area, except for the area of resource use, that was considered optional. Since its launch, Filter 2.0 has been successfully applied to the development of several core sets both within and outside OMERACT2,3,4,5,6,7,8. However, there are challenges in application caused by ambiguous wording and terms and incompletely specified concepts. In this paper, the OMERACT Executive proposes a further elaboration and update of the framework. Thus it is not a product of consensus at the conference, but its use will continue to be evaluated by the users, i.e., the OMERACT community.
The OMERACT Filter 2.0 framework as published in 20141. OMERACT: Outcome Measures in Rheumatology; ICF: International Classification of Functioning, Disability and Health. From Boers M, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. Licensed under CC BY-NC-ND 4.0, and available from https://www.jclinepi.com/article/S0895-4356(13)00488-5/fulltext
Challenges in the original framework
In the original framework (Figure 1), adverse events and context “dangled” beneath the areas and domains with examples. During the first development of the framework, we realized that these concepts were essential, but struggled to identify the optimal way to include them. In addition, many users raised concerns with the core areas of death (as being distinct from an adverse event) and pathophysiological manifestations. Death is particularly a problem for core sets in health conditions that do not necessarily affect survival; selection of domains for pathophysiological manifestations are a problem when the health condition is nonspecific or no clear pathophysiology has been recognized, e.g., fibromyalgia or nonspecific low back pain. Finally, proper placement of domains within the areas of life impact or pathophysiological manifestations can be difficult or arbitrary, with some domains appearing to fit into both because of the lack of clear criteria for placement.
Improving the Terminology and Ordering of the Core Areas
To enable improvement, we must first return to the primary purpose of the framework. A core domain set specifies the domains that should always be measured in a trial of an intervention targeted at a health condition, regardless of whether the primary study question includes these domains1. The framework is designed to help the development of core sets that meet minimum requirements of content validity. Each core area in the framework has a specific function, and together they contain the whole “universe” of domains (concepts) that one could conceivably measure to assess the effects of an intervention. For each core set, OMERACT helps to ensure content validity by mandating the choice of at least 1 domain in each area (except for societal/resource use, which is optional). We have reordered the areas to better align them with existing biopsychosocial and biomedical models9,10. The framework is intended to complement these models by its focus on the choice of domains to optimize breadth and content of a core set. A key clarification discussed in more detail below is that beneficial and harmful effects of interventions are measured in the same domain space; for example, methotrexate (MTX) can increase participation through improvements in rheumatoid arthritis (RA) activity but can also decrease participation because of severe intermittent nausea (Figure 2).
Updated OMERACT Filter 2.1 framework. All measurable aspects of health conditions (diseases) are identified in the core areas. These contain all the domains in which effects of an intervention can be measured. Most core set developers focus on identifying core domains for intended benefits, i.e., improvements on disease-related manifestations and impacts. However, conceptually the areas can also be used to define core harm domains where required, i.e., the unintended effects, for example, worsening of comorbidity and harm. Core domain sets are defined for a specific context and should contain at least 1 domain from each of the core areas (societal/resource use is optional). OMERACT: Outcome Measures in Rheumatology.
Pathophysiology: manifestations/abnormalities
A health condition manifests itself by abnormal physiology (pathophysiology) as symptoms, signs, biomarker signals, or events. These include, for example, fever, fatigue, oral ulcers, rash, range of motion limitation, anemia, hematuria, erosive damage, etc. All of these can be seen as manifestations of the underlying health condition (disease, morbidity) and can be distinguished from their impact (see below). A better term for the concept is pathophysiology, and the core area term can be shortened to manifestations. In the International Classification of Function (ICF) framework, such manifestations are identified under “body structures” and “body functions”10. To fully assess an intervention, it is essential that its effects on the underlying pathophysiology be identified; hence this is a core area. In many conditions, the pathophysiology may not be fully elucidated, so the selection of domain(s) for this area must be a “best effort.” Further, the core domain set will need to be regularly updated to track developments in understanding. In some health conditions, pathophysiology is left undefined “by definition” (e.g., nonspecific low back pain) or it is heterogeneous (e.g., “shoulder disorders”). In such cases, the development group has 2 options: (1) they can decide to place domain(s) in this area chosen to reflect the main manifestation(s), for example, “pain,” “renal function;” or (2) they can leave the core set incomplete, to be further specified per trial. In the latter case, the core set can state that trialists must choose a domain that aligns with the manifestations that are the target of intervention in that RCT. In both cases, the area of pathophysiological manifestations will measure at least 1 domain.
Impact: life impact, death/lifespan, and societal/resource use
The description of outcome as “how a patient feels, functions or survives” has been ascribed to Temple11, and is frequently quoted in documents of the US Food and Drug Administration12. It certainly concurs with Fries, et al’s seminal work on the Health Assessment Questionnaire: “A patient certainly desires to be free of pain, functioning normally, experiencing minimal treatment toxicity, and financially solvent”13. In the framework, this is entitled “impact” and encompasses both the impact on the life lived (feeling and functioning) and on lifespan (survival, mortality). For patients, usually it is not only the manifestation but its impact that is relevant and should be measured. Life impact is identified by concepts such as well-being, health perception, and utility, and by the ICF categories of activities and participation. It also largely overlaps with the concept or global domain of health-related quality of life (HRQOL), and ability to live and function independently. Although such a global domain can be useful, OMERACT encourages developers to try to choose more specific domains that better align with the lived experience of the condition for which the core set is being developed. For example, fatigue and sleep loss have a major impact on quality of life in RA, as does social isolation in psoriasis, and weight gain in osteoarthritis; if these are deemed core, it may make more sense to propose them as core domains rather than to identify them under a generic “quality of life” domain.
To date, a symptom or event has frequently been taken as a proxy for its impact, but this is usually suboptimal. For example, core set developers could consider the quality and intensity of pain as the manifestation, but choose not to include impact of pain as a separate domain (e.g., “pain interference”). Pain intensity would then be seen to represent both pathophysiology and life impact. Using events as proxy for impact is more problematic. For example, “bone fracture” is a pathophysiologic manifestation, but its impact can be very small or very large, and span many different concepts, e.g., inability to perform key activities of daily living that differ depending on the location of the fracture or the way it has healed. Choosing domains that directly address the impact(s) is preferable and recommended in this framework.
Death/lifespan as a target of intervention is pertinent in potentially fatal health conditions such as vasculitis, systemic sclerosis, and systemic lupus erythematosus. In most other rheumatologic conditions, in the context of a clinical trial, death is only a rare adverse event; in core sets for such conditions, this core area will simply be filled by the domain “count of deaths.” The term “lifespan” was added to identify the duration of living rather than only focusing on the discrete event of death.
The impact of health conditions on individuals also translates to impact on society: this is expressed as healthcare use leading to so-called direct costs, and productivity losses leading to indirect costs. Although clearly important, we maintain OMERACT’s original choice to regard this area as optional, meaning that developers are not mandated to choose a domain from this area in their core set.
Benefit and Harm: Intended and Unintended Effects of Intervention
To study the effects of an intervention, our focus is on measuring changes (improvement or deterioration) or counting events (good or bad); however, apart from the intervention, such changes can be due to the natural course of the disease/health condition, or to unrelated comorbidities. Only in a properly designed clinical trial can causal inferences be made with confidence, and only at the group level: randomization creates groups with similar prognostic characteristics, and differences above random error (in mean change or the occurrence of events) can then be ascribed to the interventions administered14.
Interventions are targeted to reduce, eliminate, or prevent the manifestations of the health condition or their impact. When the intervention has beneficial effects, we say it is successful. However, interventions can also cause harm by inducing new manifestations or worsening existing ones. The original framework paper already stated: “Benefit and harm can be regarded as opposite directions on one ‘impact scale,’ measurable in one of the core areas.” This is further clarified in the current elaboration (Figure 2 and Figure 3): the framework areas allow for domains that can measure benefit, harm, or both, resulting in a “net effect.” For example, an intervention can increase work/family/leisure participation because of decreased pain and stiffness (benefit), but at the same time decrease participation because of increased fatigue (harm). Or in the case of glucocorticoids and osteoporosis, these agents damage bone, but in RA they also counteract the harm of inflammation on bone.
Example domains within the updated OMERACT Filter 2.1 framework. Mandatory adverse event reporting is not within the scope of a core domain set, except for the area of Death/Lifespan in settings where death is a rare event. Societal and resources use are outside of the shaded boxes as the same indicator, for example, cost can represent both intended effects and harms. OMERACT: Outcome Measures in Rheumatology.
Example domains
In Figure 3, the concepts of benefit and harm are again distinguished, and example domains are included with each. Note that most current core sets focus on benefit, and the framework does not require core set developers to include separate benefit and harm domains. For manifestations, we stress that symptoms and signs of the health condition per se belong here, as targets of benefit; their impact (if deemed core) should be separately measured and placed under life impact. For example, in polymyalgia rheumatica, manifestations included only systemic inflammation15, but in psoriatic arthritis (PsA) it also included separate domains for musculoskeletal and skin disease activity16. For life impact, both included the core domains of pain and physical function, to which polymyalgia added stiffness, and PsA patient’s global assessment (PtGA), fatigue, and HRQOL. Adverse event reporting is mandatory in trials, and the requirements for reporting are best seen as separate from the development of a core set. Nevertheless, all such events can be conceptually placed in core areas, and where necessary, specific core domains of harm can be specified. For example, developers could decide to include “infection” as a core domain.
For impact, historically the effort has been targeted at defining the core domains necessary to document benefit. For harm, it quickly became clear that adverse events were not adequately integrated into the model. At OMERACT 2018, pivotal discussions were held at a preconference meeting17. One key finding spurred the current elaboration of the framework: patients explained that they usually did not experience adverse effects or harm from treatment as a series of discrete events that could be identified according to regulatory guidelines18. Instead, they described harm as an intermittent or continuous discomfort “in the background,” often not serious enough to warrant priority in a consultation with their physician, but nevertheless significantly affecting well-being and participation, as well as potentially the choice of therapy17. For example, a patient described her days following MTX ingestion as “a feeling of nausea, fatigue, being generally unwell that hinders undertaking activities.” Currently, this potential core domain of harm is not identified at all, or only indirectly through PtGA of health. We have now added it under the label “unintended impact of treatment on functioning/participation.” We expect this domain to become core in many future core sets. It has already been picked up in PsA as “treatment burden” and placed in the “research agenda” ring of the OMERACT onion16,19. In addition, and especially pertinent for patients with a chronic condition, the concept “burden of treatment” has been introduced to describe a patient’s “workload” in terms of time and effort to adhere to treatment regimens (including taking drugs), visiting health professionals, and other health-related activities20,21. We now add the burden on personal finances that can also constitute real difficulties for patients, including their ability to continue a course of treatment.
Regarding the area of death/lifespan, OMERACT acknowledges that for many health conditions, death is very rare in clinical trials. However, OMERACT core outcome sets should follow clinical trial reporting guidelines that mandate reporting the number of deaths as adverse events. Our current thinking is shown by the distinction between “survival” as a domain to show benefit of interventions for conditions that are potentially fatal, and “mortality” for the other conditions, where death is likely only relevant as an adverse event in the clinical trial setting. The term lifespan was chosen to describe the duration of living rather than focusing on the discrete event of death.
For societal impact, the domain of costs is placed in between benefit and harm. Traditionally, costs are seen as “harm” to be traded against the benefits of the intervention. However, cost could also be the target of an intervention, and decreases seen as benefit.
Domain selection process
The process to select domains is described in the OMERACT Handbook19, and updated in the OMERACT 2018 conference proceedings22. Briefly, core set developers must name at least 1 domain in each of the areas: manifestations, life impact, and death/lifespan. Guidance has been developed for placement of domains in an area. If properly supported by arguments and agreed to by the OMERACT community, domains can be shared between areas (as described above) and choices for domains can be made on the benefit and on the harm side (as in the case of mortality for nonfatal conditions).
Setting: Personal and Environmental Context
In the original framework paper, we stated: “Core set developers need to specify the setting of the core set, and consider if any contextual factors need to be documented in every trial.” Setting (or scope) includes the health condition, target population for the intervention, type of intervention, and so on. Contextual factors can be defined as those that are not the primary object of research but that may influence the results or the interpretation of the results. These include potential confounders and effect modifiers (most of which should be eliminated by randomization), as well as factors that define the generalizability of the study findings1. In the current elaboration, the label has been made more specific by following the ICF terminology and adding “personal and environmental,” but otherwise things are unchanged. An OMERACT special interest group is analyzing the process of selecting important and core contextual factors for a core set23. Discussions are ongoing, but we posit that consideration of context is important in at least 3 phases: (1) domain selection; (2) instrument selection (for correct interpretation of measurements); and (3) consideration of other factors that should be included as effect modifiers/confounders in the analysis and interpretation of the trial.
Table 1 outlines the changes incorporated into the revised framework as described above.
What has changed in the revised OMERACT Filter 2.1 framework?
DISCUSSION
The OMERACT Filter 2.0 framework and process has been widely quoted and implemented both within and outside OMERACT. We hope the current elaboration, “OMERACT Filter 2.1,” clarifies and helps to solve problems encountered by users.
Footnotes
DEB, COB III, PGC, MAD, MPD, LG, LM, LS, JAS, VS, GAW, and PT are members of the executive of OMERACT, an organization that develops outcome measures in rheumatology and receives arms-length funding from 12 companies. PGC is funded in part by the NIHR Leeds Biomedical Research Centre. The views expressed in this article are those of the authors and not necessarily those of the UK National Health Service, the NIHR, or the Department of Health. JAS is supported by the resources and use of the facilities at the VA Medical Center at Birmingham, Alabama, USA. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the US government. LM is a principal investigator on the Australian Rheumatology Association Database, which has received arms-length funding from Pfizer Australia, AbbVie Australia, Eli Lilly Australia, and Janssen Australia.
- Accepted for publication December 6, 2018.
