Abstract
This overview draws out the main conclusions from the 4 workshops focused on incorporating the patient perspective into outcome assessment at the 10th Outcome Measures in Rheumatology (OMERACT 10) conference. They raised methodological issues about the choice of outcome domains to include in clinical trials, the development or choice of instruments to measure these domains, and the way these instruments might capture the impact of a disease and its treatment. The need to develop a more rigorous conceptual model of quantifying the way conditions affect health, and the need to ensure patients are directly involved in the decisions about domains and instruments, emerged clearly. The OMERACT participants voted to develop guidelines for domain and instrument selection, and conceptual and experimental work will be brought forward to revise and upgrade the OMERACT Filter.
At the Outcome Measures in Rheumatology (OMERACT) 10 meeting, 4 workshops focused on incorporating the patient perspective into outcome assessment. Although this was more program time than at any previous OMERACT meeting, 3 of the workshops were designed to tackle fundamental issues and were principally about methods. The fourth workshop (the Virtual Campus1) offered an opportunity to reflect on other work that had been progressing between OMERACT meetings. The 3 main workshops incorporated contributions from many working groups, using reports of the activities and experiences gained in specific disease areas as a way of exploring the underlying methodological challenges and ensuring the relevance of the methodological conclusions. The workshops were arranged in the following methods sequence: (1) How to choose domains of interest in outcome assessment; (2) How to choose or develop instruments to make reliable and valid measurements within those domains; and (3) How to calibrate the changes measured for use in personal and health service decision-making. All were conducted within the framework of the OMERACT Filter2. Here we offer an overview of the main issues to arise and draw out 3 emerging ideas that require further thought and evaluation. One of these ideas explains why one of the authors wore a blue triangle on his head when addressing an OMERACT 10 plenary session (Figure 1).
John Kirwan addressing an OMERACT 10 plenary session.
Choosing Domains of Interest
What constitutes a “domain” is poorly defined. In this workshop3 the domain selection process of 3 OMERACT groups working on patient-reported outcomes (PRO) were examined in detail and used as a tool for thinking about the issues. There was agreement that a gold standard for domain selection would include 3 important aspects: following a framework; including the clinically relevant benefits and harms; and remaining true to the clinical question. The framework might be the International Classification of Functioning, Disability and Health (ICF)4, but many participants felt that this itself was incomplete. One possible reason may be that, in its current form, the ICF concentrates on the mechanistic disease process (see Figure 2, devised by J. Kirwan and A. Boonen during OMERACT 10). Quality of life includes aspects of an individual’s psychological status (such as their ability to self-manage their condition) and a person’s personal (environmental) resources that are available to help them manage their condition (such as their financial situation or the availability of supportive relatives), and these are less well captured. Including harms as well as benefits reflects the point made by Dworkin in his plenary address in the Virtual Campus workshop: clinical importance and therapeutic decisions about treatments are a balance between the two1. Finding which domains are relevant to the clinical question may require insight from both clinicians and patients. For example, the need to prevent the development and progression of erosions in rheumatoid arthritis is an outcome that would not be obvious to patients, whose experience is focused around the symptoms of inflammation.
Quality of life represented by 3 beams shining through: disease pathology and symptoms; psychological factors; and environmental challenges and support. At present, the International Classification of Functioning, Disability and Health (ICF) covers psychology and environment incompletely. ADL: activities of daily living.
Choosing or Developing Valid Instruments
“Choosing or developing instruments” looked closely at the notion of how an instrument (a questionnaire or a scale) to measure a PRO is developed and designed in the first place may have profound effects on its performance and appropriateness for outcome assessment in clinical trials5,6. The experience of many OMERACT groups, reported as illustrations within this broad discussion, showed that directly involving patients at the earliest stages of defining a domain and the effects that illness may have in that area will result in greater validity. The broader “life impact” of illness and its treatment needs to be considered7. Further, there was an emerging recognition that for complex interventions, where changes outside the immediate effects of a condition may allow adaptation or improved coping within that condition, some broader framework of outcome assessment may be needed8,9.
Taking the Domains and the Instruments sessions together, the notion of a different approach to looking at outcomes from interventions was mooted, and has subsequently been published10. This proposes an “impact triad” (incorporating severity, importance, and self-management; Figure 3) as a method of enhancing the measurement of the personal life impact of rheumatic diseases. It is represented by a triangle — as worn by the author in the picture. Further, these aspects of severity, importance, and self-management (or coping) may be broadly equated with symptoms, environment, and psychology, as shown in Figure 2. To measure only the impact of treatment designed to control disease severity might fail to capture important changes in self-management or the supportive environment, which may in fact be responsible for the improvement, or may hide some of the treatment benefits.
The impact triad incorporating severity, importance, and self-management as 3 aspects of the overall impact of a condition on a patient. These aspects may be broadly equated with symptoms, psychology, and environment in Figure 2.
Calibrating the Changes Measured
How to describe to patients what benefits they might expect from a given intervention is difficult to do if only summary data on group mean changes are available. The way an instrument responds to change will determine if it is possible to offer patients a more clear understanding of what might happen to them personally if they take the treatment. This issue was explored in the workshop on response criteria and the importance of change11. When interpreting scores at the individual level, 4 main concepts need to be taken into account: improvement; status of well-being; onset of action; and sustainability. As the full report of that workshop proclaims — information from clinical trials on how many patients showed a response, what the level of response was, how quickly they improved, and how many patients are doing well over time, would be extremely useful — for patients and physicians alike11.
Plenary Summary
A summary of the plenary feedback from all 4 workshops, integrated and developed by further informal discussions between the workshop reporters, was presented to the final OMERACT plenary session for further consideration. It was evident that the main methodological issues were relevant to all those working on PRO, and there was a wish to see greater commitment to appropriate application of these methods. At the end of this final plenary session, participants were asked to vote on the proposal that: For OMERACT 11 we should collate (and provide evidence for) required principles and procedures for choosing domains, developing instruments, and specifying response criteria for PRO. This was approved by 92% of participants and will be taken forward as part of the development of version 2.0 of the OMERACT Filter.
Footnotes
-
P. Tugwell is supported in part by the Canadian Institutes of Health Research.
