Abstract
Objective. The Outcome Measures in Rheumatology (OMERACT) Safety Working Group objective was to identify harm domains from existing outcome measurements in rheumatology.
Methods. Systematically searching the MEDLINE database on January 24, 2017, we identified full-text articles that could be used for harm outcomes in rheumatology. Domains/items from the identified instruments were described and the content synthesized to provide a preliminary framework for harm outcomes.
Results. From 435 possible references, 24 were read in full text and 9 were included: 7 measurement instruments were identified. Investigation of domains/items revealed considerable heterogeneity in the grouping and approach.
Conclusion. The ideal way to assess harm aspects from the patients’ perspective has not yet been ascertained.
Harms provide important context for healthcare practitioners about the benefit-risk ratio of interventions1. To improve transparency and credibility in the published results from randomized trials, the reporting of harms associated with an intervention needs to be explicit regarding what is patient-important, which may be different from that reported by clinicians submitting adverse event reports2. The Consolidated Standards of Reporting Trials (CONSORT) group has provided recommendations on the appropriate reporting of harms in randomized controlled trials (RCT)3. However, systematic reviews conclude that adherence to these CONSORT harm recommendations is suboptimal in RCT for (non)pharmacological treatment of rheumatoid arthritis and hip or knee osteoarthritis4,5 as reported in leading medical journals. According to Hadi, et al5, more than half (56%) of the RCT reported ≤ 50% of the recommended CONSORT harm items. While some CONSORT harm items might be more important to consider reporting than others, there is a need to improve harms reporting in RCT to allow transparent and balanced assessment of the benefit-risk ratio in clinical decision making5.
Following the concerns about inadequate reporting of harm outcomes in randomized trials3 and systematic reviews1,6, the Outcome Measures in Rheumatology (OMERACT) Safety Working Group is advancing the work to identify additional harm aspects for assessment in rheumatology trials7,8. To inform this work, we performed a scoping review of harm aspects, assessed in existing measurement instruments, using an approach suggested by Macefield, et al9 and McNair, et al10. The objective was to identify harm domains from the patient perspective by examining currently available outcome measurement instruments.
MATERIALS AND METHODS
The protocol was registered with the International Prospective Register of Systematic Reviews (PROSPERO, registration number CRD42017055861). A scoping review aims to map the existing literature in a field of interest in terms of the volume, design, and characteristics of the primary research, which is feasible when the topic has not yet been extensively reviewed or is of a complex or heterogeneous design11. The purpose of a scoping review is to sum up the best available research on a specific question12.
An electronic search was performed on January 24, 2017, using Medline through PubMed to identify studies describing or evaluating measurement instruments including harm outcomes that could be used in rheumatology trials. The search strategy included terms for harms, rheumatic disease, and outcome measures. No filters were activated (e.g., no article type, availability, publication date, language restrictions). Additional references were identified through reference lists of included studies and by consulting experts within rheumatology (i.e., snowballing). One review author (LK) screened the titles and abstracts of the identified publications. A second reviewer (RC) screened a random sample of abstracts to check accuracy of inclusion. Publications were eligible if they described or evaluated instruments including harm outcomes (either domains or measurements) that could be used in rheumatology trials. Full text was obtained for all titles that appeared to be eligible or where there was any uncertainty. Two reviewers (LK and RC) screened the full texts and excluded publications not in English, and publications reporting results from trials, i.e., studies with the purpose of evaluating the effects of a treatment. Reasons for exclusion of publications were documented. Every step of the selection process was documented by a flowchart. Reference manager 12 (Thomson Reuters) was used to manage references.
Verbatim names for the harm aspects as termed by the instrument developers were extracted and all patient-reported outcome measures (PROM; scales, subscales, and single items) were collated in a list. Using a standardized form, 1 reviewer (LK) extracted data from each included study. Another reviewer (RC) verified the data. Extracted data, if available, included first author, study publication year, aim of the study, name and abbreviation of outcome measurement instrument, reported harm aspects (i.e., scales/domains and items), definition of harm aspects, and target population. All PROM items assessing adverse effects were systematically categorized into conceptual health domains according to the issue they addressed. As suggested by Macefield, et al9 and applied by McNair, et al10, we summarize PROM and categorize their PRO content to inform the development of a minimum “safety core” outcome set to be measured in all rheumatology trials. Individual items from all questionnaires were extracted and formed into a longlist before categorization into health domains by 2 researchers (LK and RC).
Following this, 8 of the authors (LK, MB, DD, VSS, NG, LM, PT, RC) were encouraged to categorize all items “in any way they found meaningful,” and subsequently to name the categories as they rationalized based on experience (further details are available from the corresponding author upon request). Using concept mapping software, the average categorization was estimated through multidimensional scaling analysis, as an expression of consensus of the distribution of items13.
RESULTS
As illustrated in Figure 1, of 435 unique references identified, 24 were read in full text, and of these, 9 were included14,15,16,17,18,19,20,21,22. One reference was excluded because of “other language than English.” An overview of the 9 included studies is presented in Table 1. From these, 8 unique instruments were identified. Two instruments [the Stanford Toxicity Index (STI) and the Rheumatology Common Toxicity Criteria (RCTC)] were the subject of 2 studies each, the newest study describing a revision or update of the original instrument. There were 7 individual measurement instruments and 1 methodological proposal referred as the OMERACT 3 × 319: (1) STI14, (2) revised rSTI15, (3) RCTC 2.018, (4) The Patient Self-Report Adverse Event Instrument and the Investigator Report Adverse Event instrument17, (5) The BioSecure questionnaire20, (6) Safety of Estrogens in Lupus Erythematosus National Assessment-Systemic Lupus Erythematosus Disease Activity flare index21, (7) Glucocorticoid Toxicity Index22.
PRISMA flow diagram of studies considered for the scoping review. PRISMA: Preferred Reporting Items for Systematic Reviews.
Study characteristics, listed by year of publication.
Five of the 7 individual instruments aimed to assess “toxicity,” 1 of these instruments specifically in relation to treatment with corticosteroids. The content, indicated by subscales of the instruments, varied despite the common construct of “toxicity.” The other instruments aimed to assess different harm aspects: event importance, benefit and harm, self-care safety skills, and flare.
The structure of the instruments varied: 1 was a PROM20 and the others were investigator/clinician-reported. Altogether there were 205 unique items, or 223 when taking into account the response options [e.g., the item “What was (were) the side effects?” was accompanied by 37 response options]. Different types of information were retrieved by the items, as shown in Table 214,15,16,18,19,20,21. Most (125) items or response options each represented a symptom, sign, or diagnosis which could be an adverse effect (further details are available from the corresponding author).
Type of information retrieved by the instruments, with number of items for each type, an example of an item, and the instrument from which the example was taken.
DISCUSSION
Based on a scoping review, we identified instruments to assess harm aspects in rheumatology trials. “Harm aspects” is a very broad and complex construct, and this review illustrates that there are many potential approaches to address it. Harm aspects reported with existing instruments included toxicity, event importance, benefit and harm, self-care safety skill, and flare. These could be categorized as patient reports, clinician/researcher reports, laboratory results, qualitative descriptions of patients’ experiences, and data from medical records, and only 4 instruments provided a patient perspective. Feasibility around this review made us perform the systematic search including only 1 electronic bibliographic database (Medline), as well as the manual search in reference lists and contact with key opinion leaders in rheumatology. Thus, a potential limitation to the present manuscript is that we did not include additional electronic databases.
The current “clinical trial practice” for reporting adverse events is based on the implicit assumption that an accurate portrait of patients’ subjective experiences can be provided by clinicians’ documentation alone. Our findings derived from the existing instruments developed for rheumatology14,15,16,17,18,20,21,22 at least seem to support the grouping that was previously suggested by the US National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) initiative23. Our work supports the idea that there are 3 broad categories of “harms” available from the current Medical Dictionary for Regulatory Activities (MedDRA) framework: (1) laboratory-based events, (2) observable/measurable events, and (3) symptomatic adverse events. Yet, the clinician/trialist reports of symptomatic adverse events as recorded on case report forms lack reliability. There is a risk that clinicians underreport the incidence and severity of symptoms compared to patients’ direct reports, especially for subjective symptoms, in part because the clinician cannot observe these symptoms. If a PROM was available, it could enable patients to directly report their own symptomatic adverse events, providing important evidence of patients’ adverse experiences with an intervention to contribute to shared decision making.
From our scoping review, we hope to raise awareness about the need for a novel explicit harm reporting paradigm in rheumatology research, with a focus on patient self-report with the potential to enable reporting of safety rather than harms. One important issue is how best to collect data on harm and/or safety outcomes, and whether available measurement instruments are suitable for the purpose. Harm aspects can be defined and targeted in many ways, reflecting the complexity of the construct. It is clear from our review that the ideal way to assess harm aspects has not yet been achieved. In addition, the language used to cover the various “domains” is difficult to comprehend for a lay audience (including patients). The OMERACT Safety Working Group will continue to investigate harm aspects, with a specific focus on patients’ perspectives on safety.
Footnotes
The Parker Institute, Bispebjerg and Frederiksberg Hospital is supported by a core grant from the Oak Foundation (OCAY-13-309).
- Accepted for publication April 12, 2019.
