Updating the OMERACT Filter at OMERACT 11

Why Update the OMERACT Filter?

The Outcome Measures in Rheumatology (OMERACT) community strives to develop core outcome sets for rheumatologic conditions¹. Such core sets specify, for each condition, the minimum set of outcomes (and associated measurement instruments) necessary to provide the best estimate of benefits of an intervention. After adoption of a core set, OMERACT recommends that all studies of a health intervention in a particular condition report the results of these outcomes, regardless of the primary study question and the intended primary outcome measure.

The original OMERACT filter² describes the procedure of consensus building regarding core outcome sets, and the filter’s components are summarized in 3 words: truth, discrimination, and feasibility. Truth addresses the notion that a core set measures what is intended and is unbiased and relevant. Discrimination refers to issues of reliability and sensitivity to change. Feasibility characterizes an essential element in the selection of measures, one that may be decisive in determining a measure’s success.

Three developments have coalesced to bring about a review of the theoretical basis and practical application of the OMERACT filter. First, OMERACT members have applied the filter in a wider range of conditions within rheumatology³. When discussions centered on the common conditions of rheumatoid arthritis and osteoarthritis, as they did in the early years, there was a shared understanding about the nature and effect of these conditions. This allowed an implicit common understanding of the domains required for a core set to emerge; hence, discussions often focused on the instruments or lack of instruments that could “pass” (i.e., meet the requirements of) the OMERACT filter. Over the last decade an increasingly wide range of conditions (such as myositis, gout, fibromyalgia, and polymyalgia rheumatica) have been brought forward for core set development, and these have not carried the same level of inherent familiarity.

Second, a wider range of people have become involved, illustrated by the encouragement of young investigators through the “Fellow” program and particularly by the embracing of patient participation as full partners in the OMERACT process⁴. Third, OMERACT members have collaborated with an emerging broader movement to identify core outcome sets in medicine as a whole [the Core Outcome Measures in Effectiveness Trials (COMET) group⁵].

Through these developments, and particularly through the detailed examination of the ways to incorporate the patient perspective in developing and validating patient-reported outcomes (PRO)⁶, it has become clear that it would be beneficial to expand the OMERACT filter from its current abbreviated form to provide a more explicit way of identifying the relevant core outcomes that might be universal to all randomized controlled trials within rheumatology and may offer a procedure more widely applicable within medicine in general. Therefore, in preparation for and at OMERACT 11 a substantial body of work was undertaken to achieve this aim, working toward a new version, termed OMERACT Filter 2.0.

Outline of the Issues and Approaches

The reports presented in this issue of The Journal of Rheumatology provide details of our work, including a preconference literature review of ways others have approached the problem of outcomes assessment in health and disease⁷. Based on this and on a priori discussions, debates, and surveys among researchers with an interest in outcomes assessment (in rheumatology and outside) a preliminary proposal was drawn up⁸ and widely circulated among conference participants. This raised first the question of whether there are a few universally applicable areas of assessment and suggested that there are; these were outlined and an explanation was offered. This proposal was tested in the first Filter 2.0 workshop⁹. In this and the following workshops, enquiry was facilitated by presentation of case studies of work already being undertaken in OMERACT working groups. These cases were then considered and compared, both theoretically and in practical application, to the questions posed in the workshops by small group discussions, plenary feedback, and by further debate. In the second workshop¹⁰, the detailed application of the truth section of the filter to instrument design and development was examined, concentrating on the ways in which face, content, and construct validity can be established, once again by direct comparison of the proposed wording with practical experience. The third workshop¹¹ dealt with discrimination and feasibility as applied to measurement instruments. Small discussion groups addressed questions about the strength of evidence required to demonstrate these qualities and reported back to a plenary discussion, where the components of a preliminary checklist of required evidence were pieced together.

Workshops 4 and 5 compared the proposed Filter 2.0 requirements to what has been achieved in developing and validating PRO¹², and the current validity of imaging and biomarker techniques¹³. PRO development was linked to work done by other agencies (such as the US Food and Drug Administration) and recent publications and was found to reflect the current practice of working groups in these areas. This is a noticeable progression from OMERACT 10, where further help and guidance on PRO development was requested¹⁴. The position with imaging and biomarkers was less satisfactory, because many new measurement instruments have been disseminated in daily practice before being rigorously evaluated and have already been used in clinical trials evaluating therapeutic interventions. Reviewing the current state of several techniques applied in several disease areas, it was felt that it should be clearly stated whether the measurement relates to current disease activity or resultant tissue damage. Further, a clearer statement of the required statistical validation steps for each technique should be augmented by a technical appraisal of instrument-specific issues such as how a chemical measurement is undertaken or how quality control is applied to the implementation of imaging techniques.

Bringing the Discussions Together

In the spaces between the workshops and all the other meetings at the OMERACT 11 Filter 2.0 sessions, reporters and session leaders met to compare notes, iron out misunderstandings, amend ambiguous or unacceptable wording, and balance the criticisms of the Filter 2.0 proposals against the benefits they were perceived to have. By the time of the final plenary session of OMERACT 11, leaders from each of the 5 Filter 2.0 sessions were able to not only present¹⁵ the arguments that arose, but also to offer modifications and amendments for further consideration. At each step there were votes indicating that the large majority of participants were satisfied that the different issues had been adequately addressed. In a final presentation, the new Filter 2.0 incorporating these changes was approved.

Implications

The Filter 2.0 proposals could not have been put to a more rigorous test within the OMERACT meeting. Workshop sessions were designed to challenge the proposals and closely examine the underlying assumptions (workshops 1, 2, and 3) or practical implications (workshops 4 and 5). Every workshop had direct case study input from the experiences of working groups developing their core sets or core set methodology, a total of 29 case studies being presented and then scrutinized by breakout group discussions. Noticeable changes were introduced or wordings clarified, illustrating that this strategy captured and dealt with issues of contention. Further, as well as approving the new content, participants also voted to implement the OMERACT Filter 2.0 immediately for all OMERACT activities, and to schedule an evaluation and (if necessary) revision in the next 6–8 years. (The designation 2.0 implies that small revisions may well follow.)

Translating these decisions into an OMERACT Filter 2.0 handbook to include a checklist of actions required to pass the filter is now under way, and will be a powerful aid to those working toward OMERACT endorsement for core sets in the conditions and settings of interest to them. We believe this framework will also prove to be a more generic guide, offering an approach to core outcome set development in many areas of healthcare¹⁶ [this standalone article, intended for a general (non-rheumatology) audience and published in the Journal of Clinical Epidemiology, summarizes the development of Filter 2.0, as described in detail in this part of the OMERACT proceedings].