Abstract
Objective. Failure to report harmful outcomes in clinical research can introduce bias favoring a potentially harmful intervention. While core outcome sets (COS) are available for benefits in randomized controlled trials in many rheumatic conditions, less attention has been paid to safety in such COS. The Outcome Measures in Rheumatology (OMERACT) Filter 2.0 emphasizes the importance of measuring harms. The Safety Working Group was reestablished at the OMERACT 2016 with the objective to develop a COS for assessing safety components in trials across rheumatologic conditions.
Methods. The safety issue has previously been discussed at OMERACT, but without a consistent approach to ensure harms were included in COS. Our methods include (1) identifying harmful outcomes in trials of interventions studied in patients with rheumatic diseases by a systematic literature review, (2) identifying components of safety that should be measured in such trials by use of a patient-driven approach including qualitative data collection and statistical organization of data, and (3) developing a COS through consensus processes including everyone involved.
Results. Members of OMERACT including patients, clinicians, researchers, methodologists, and industry representatives reached consensus on the need to continue the efforts on developing a COS for safety in rheumatology trials. There was a general agreement about the need to identify safety-related outcomes that are meaningful to patients, framed in terms that patients consider relevant so that they will be able to make informed decisions.
Conclusion. The OMERACT Safety Working Group will advance the work previously done within OMERACT using a new patient-driven approach.
This Safety Working Group within the Outcome Measures in Rheumatology (OMERACT)1,2,3,4,5,6 was reestablished at OMERACT 2016 to develop a core outcome set (COS) for assessing safety components in rheumatology trials using an evidence-based, consensus-driven standard definition of safety in patients with rheumatic diseases.
Because every healthcare intervention carries some risk of harm, clinical decision making needs to be supported by a systematic assessment of the balance of harm compared with the apparent benefit. For example, a systematic review that considers only the favorable outcomes of an intervention, without also assessing the adverse effects, can mislead by introducing a bias favoring the intervention7. It is critical that all patient-important outcomes are measured and subsequently reported, either directly in a journal article or as an elaborately detailed supplementary file.
Currently there is a large and disturbing amount of literature indicating a general failure in the quality of reporting health research8. According to Moher, et al, many publications lack clarity, transparency, and completeness in how the authors actually carried out their research8. To counterbalance that, internationally recognized reporting guidelines now exist for a diversity of research areas with different study designs; the Enhancing the QUAlity and Transparency Of health Research (www.equator-network.org) is an international initiative that seeks to improve the reliability and value of biomedical research literature by promoting the transparent and accurate reporting of studies9.
Members of the Consolidated Standards of Reporting Trials group have highlighted that the reporting of harms in randomized controlled trials (RCT) has received less attention than reporting of benefit, and the data available are often inadequate10. As a consequence, many trials that are published do not add value in clinical decision making because of fundamental flaws in reporting. Both scientific evidence and ethical necessity call for action to improve the quality of reporting of harms10.
There is strong international advocacy to shift the research model through developing COS across all disease areas that always include the assessment of both benefits and harms. Initiatives such as OMERACT11,12 and the Core Outcome Measures in Effectiveness Trials13 have demonstrated that COS improve the reporting of trial outcomes. The Cochrane Musculoskeletal systematic reviews routinely include core sets that state both benefits and harms in summary of findings tables14.
There is currently a focus on COS for assessing benefit in rheumatology trials; however, none have been developed to address safety components. The aim of the Safety Working Group was to address this need. Specific objectives were to (1) identify harmful outcomes in trials of diverse interventions evaluated in patients with rheumatic diseases, (2) identify components of safety that should be measured in trials in rheumatic diseases, and (3) develop a COS for safety in RCT in rheumatology. The COS should allow results of trials to be compared and combined and thus will contribute usable information for clinical decision making15.
The main contribution of our paper was the consensus obtained among leading (experienced) members of OMERACT, including patients, clinicians, researchers, methodologists, and industry representatives, that there was a need to continue the efforts to develop a COS for safety in rheumatology trials. This will advance the work previously done and published in previous publications within OMERACT1,2,3,4,5,6 using a new patient-driven approach.
Assembly of Working Group and Work Plan
The Safety Working Group follows the OMERACT Master Checklist for developing core outcome measurement sets as described in The OMERACT Handbook16. Adherence to the checklist items at this early stage of the process is described.
Forming an OMERACT working group and review of domain and instruments previously used
Following the approval of the OMERACT Filter 2.012, it was decided that OMERACT needed to establish consensus on the safety domains and instruments included in OMERACT COS. At OMERACT 2016, the Safety Working Group (previously called the Drug Safety Working Group, but now the mandate has been broadened to all types of interventions), as part of its research agenda, determined as its goal to develop a COS for assessing safety components in rheumatology — derived primarily from patients with rheumatic diseases12.
At OMERACT 3 (in 1996), the Toxicity Working Group was formed with the purpose of developing an adverse event assessment tool for the use in rheumatology clinical trials to improve consistency in reporting1. Through a literature review, existing tools were identified. One of these was the World Health Organization Common Toxicity Criteria (CTC) on which it was decided to build the Rheumatology CTC (RCTC) at OMERACT 5 in 2000. Following discussions at OMERACT 6 in 2002, it was decided to develop 2 tools, building on the RCTC1, the Stanford Toxicity Index17, the symptom list from the complete Health Assessment Questionnaire18, the Patient Self-Report Adverse Event Instrument, and the Investigator Report Adverse Event instrument19. These instruments were presented at OMERACT 7 in 2004, where the discussion included the advantages of electronic instrument versions and a “more patient-friendly system than the medical body systems approach”3. A second version of the RCTC was published following discussion and revision at OMERACT 8 in 20062.
At OMERACT 9 in 2008, the OMERACT Executive brought together clinical trialists, pharmacoepidemiologists, clinicians, clinical epidemiologists, statistical experts, and regulatory representatives to discuss different approaches to define risk and perhaps improved ways to express it4,5. One idea was the development of a single metric to assess both benefit and risk, a challenge that led to the development of a simple instrument to assess both benefit and harm in clinical trials6. The instrument proved to be feasible, but should be further developed in the context of the OMERACT initiative, including more elaborate work on what constructs were being measured. Thus, an obvious starting point would be to create a clear definition of the construct of interest (safety) embedded in a conceptual model (according to the OMERACT filter 2.0).
Groups involved and their contacts identified
The development of the working group will involve multiple key groups to ensure that a COS is suitable and well accepted in future research. Patients, clinical researchers, health professionals, methodologists, policymakers, and industry representatives will be included, and they will represent at least 3 continents (Europe, North America, and Australia).
Implementation of Delphi and/or focus groups
Consultations will be conducted to ensure content and face validity of the domains potentially included in the COS. The method applied will be concept mapping20,21, a structured group conceptualization process combining the qualitative approach of focus group/survey processes and statistical analyses to support the structuring of data, as described by Kane and Trochim22. Anticipated sample size for patient focus groups is 20, and for surveys including others, 200. One significant advantage of this concept mapping approach is that participants handle the first step of the data analysis, i.e., the organization of themes. This might provide a solution to the issue of developing a “more patient-friendly system than the medical body systems approach” that was called for at OMERACT 73.
The input will contribute to the pool of safety components identified through review of the literature, and allow an organization of data building on the perspectives of the patients. The consensus process will be done “the OMERACT way,” including Delphi surveys, discussions, and plenary sessions at OMERACT meetings23. It is an iterative process with the goal to develop a COS that everyone involved can accept.
Core Domain Set Selection
Definition of context: setting (scope)
Based on current and previous OMERACT discussions, the scope of the core set is safety components in rheumatology clinical trials. Specification of the scope is expected following discussions in the working group based on the PICO structure, defining the patients/population, intervention, comparator/control, and outcome.
Footnotes
As part of the supplement series OMERACT 13, this report was reviewed internally and approved by the Guest Editors for integrity, accuracy, and consistency with scientific and ethical standards.
