Abstract
Objective. The Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA) and Outcome Measures in Rheumatology (OMERACT) psoriatic arthritis (PsA) working group is developing a Core Outcome Measurement Set for PsA clinical trials [randomized controlled trials (RCT) and longitudinal observational studies (LOS)] using the OMERACT Filter 2.1 instrument selection algorithm. Our objective was to assess the Psoriatic Arthritis Impact of Disease questionnaire (PsAID12) for the measurement of the core domain PsA-specific health-related quality of life (HRQOL).
Methods. PsAID12 measurement property evidence gathered in a systematic literature review, and additional analyses conducted in LOS, were used to inform a consensus process. Analyses that had not been published were independently reviewed by the OMERACT technical advisory group. Data and process were presented, discussed in breakout groups, and voted on at the OMERACT conference (Terrigal, Australia, May 2018).
Results. PsAID12 fulfilled the green (good to go) OMERACT standards for domain match, feasibility, reliability, and construct/longitudinal construct validity. Discrimination and thresholds of meaning were amber (caution but good enough to go forward). The overall working group recommendation was amber/provisional endorsement of PsAID12 for measuring PsA-specific HRQOL in RCT and LOS. Of 96 participants who voted at the PsA OMERACT workshop, 87.5% (84) voted “yes” to endorse this recommendation; 14 of the 96 were patient research partners (PRP) and 93% of them (13) voted “yes”; 82 participants were not PRP and 87% of them (71) voted “yes.”
Conclusion. At OMERACT 2018, PsAID12 was the first patient-reported outcome measure provisionally endorsed as a core outcome measure for disease-specific HRQOL in PsA clinical trials. PsAID12 discrimination and improvement thresholds will be studied in future RCT.
The Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA) and the Outcome Measures in Rheumatology (OMERACT) are collaborating to develop a Core Outcome Measurement Set for psoriatic arthritis (PsA). Aiming to standardize outcome measures for PsA randomized controlled trials (RCT) and longitudinal observational studies (LOS), the project is based on the GRAPPA-OMERACT updated PsA core domain set1,2 and the new OMERACT guidance on instrument selection3,4.
The OMERACT Filter 2.1 instrument selection process3,4 requires a solid definition of the target domain as documented in the core domain set. It consists of 2 major steps: (1) confirmation of content validity (domain match) and feasibility for the domain and population of interest; and (2) appraisal of construct validity and discrimination (reliability, responsiveness, thresholds of meaning). The evidence is appraised at each step and scored green (good to go), amber (caution but good enough to go forward), or red (stop, do not continue)3. For any candidate outcome measure, the OMERACT selection process can be completed only if content validity and feasibility have initially been demonstrated3,4.
Health-related quality of life (HRQOL) is a PsA core domain1, but there are few outcome measures with dedicated content validity studies to support PsA-specific HRQOL domain match5. This paper reports the first patient-reported outcome (PRO), the Psoriatic Arthritis Impact of Disease (PsAID12)6,7, appraised through the OMERACT Filter 2.1 — from core domain match with PsA-specific HRQOL to provisional endorsement as core outcome measure for PsA clinical trials (RCT and LOS).
MATERIALS AND METHODS
The GRAPPA-OMERACT project is known by the acronym “COMPACT” (Core Outcome Measures for Psoriatic Arthritis Clinical Trials)8. Figure 1 illustrates the COMPACT study design. It was applied to PsAID12, the first PRO evaluated through the OMERACT Filter 2.1. Through steps of evidence review and consensus from many people guided by Filter 2.1, a conclusion can be achieved on suitability of a candidate as core outcome measure for the desired core domain. To arrive at a complete PsA Core Outcome Measurement Set, each PsA core domain must have at least 1 corresponding outcome measure that successfully met all requirements of the OMERACT Filter 2.1 instrument selection criteria.
The OMERACT Filter 2.1 Instrument selection algorithm is represented at the left (OMERACT handbook). The COMPACT workflow through OMERACT Filter 2.1 is represented at the right. The 4 signaling questions must be addressed in OMERACT Filter 2.1 instrument selection (items 1–4) in the left part of the figure. Following the OMERACT process, participants provide input initially into domain match and feasibility, and subsequently into the approach and results of evaluation of additional measurement properties, as well as the final voting for inclusion as core outcome measure(s). For each domain and candidate outcome measure, the OFISA Filter 2.1 is applied until a core outcome measure has been selected. The process is repeated for each core domain. Once each core domain has at least 1 corresponding core outcome measurement instrument, the PsA Core Outcome Measurement Set is complete. OMERACT: Outcome Measures in Rheumatology; COMPACT: Core Outcome Measures in Psoriatic Arthritis; OFISA: OMERACT Filter Instrument Selection Algorithm; GRAPPA: Group for Research and Assessment of Psoriasis and Psoriatic Arthritis; PsA: psoriatic arthritis; COSMIN: consensus-based standards for the assessment of health measurement instruments; LOS: longitudinal observational study; RCT: randomized controlled trial.
The working group5 completed a systematic literature review (SLR) of the evidence for measurement properties of all PRO used in PsA. PsAID12 had evidence for adequate validity and reliability for the core domain, PsA-specific HRQOL. At the time of the SLR, PsAID9, the 9-item version of PsAID12, did not have enough evidence to be considered on its own. More importantly, patient research partners (PRP) had indicated that PsAID12 met the first step of truth/domain match better than PsAID9. The SLR was updated with a focus on PsAID12 to support the ensuing consensus process9.
The PsAID12 outcome measure
The PsAID12 questionnaire6,7 was developed with PRP included in domain generation, scoring, and item formulation5. PsAID12 is available free of charge for any application in several languages. PsAID12 is composed of 12 numeric rating scales (NRS; range 0–11): (1) pain, (2) fatigue, (3) skin problems, (4) work and/or leisure activities, (5) functional capacity, (6) discomfort, (7) sleep disturbance, (8) coping, (9) anxiety, fear, and uncertainty, (10) embarrassment and/or shame, (11) social participation, and (12) depression. The total score is the sum of the weighted NRS scores. The NRS weights represent patient rankings of importance for each of the 12 items. The weighted raw score is divided by 20 to yield a final PsAID12 score (range 0–10). Higher scores represent worse life impact. The individual NRS of the PsAID12 have been evaluated in the Bath, UK, cohort10.
Domain match and feasibility
Participants were involved in 3 stages. First, GRAPPA PRP participated in a face-to-face premeeting (GRAPPA meeting July 2017). The PsA-specific HRQOL domain description was presented, followed by completion by the PRP of the OMERACT truth and feasibility questionnaires for PsAID123. Second, during the GRAPPA 2017 annual meeting COMPACT workshop, one breakout group was dedicated to PsAID12. Facilitators provided the HRQOL domain description, the PsAID12 questionnaire, and scoring method; participants then anonymously completed each item of the OMERACT domain match and feasibility questionnaires11. Third, the working group voted on domain match and feasibility.
Construct validity and discrimination
SLR PsAID12 articles were screened and selected3, and the evidence for measurement properties was extracted9 and placed in the Summary of Measurement Properties (SOMP) table (Figure 2)6,10,11,12,13,14,15,16 to record location, quality, and results of these studies. Two reviewers checked whether the study had used sufficient methods to avoid a risk of bias in its results. The COSMIN-OMERACT Good Methods Checklist was used and consensus was sought between 2 reviewers (among AMO, PH, RH, WT). Evidence from studies using good methods was extracted and compared to measurement standards and results tracked on the SOMP. Evidence from new studies was used to fill gaps and its quality was independently appraised by the OMERACT’s technical advisory group10,17. Final rating of synthesis of evidence for each measurement property was done according to OMERACT guidance, looking for consistent, good performance from studies identified as having good methods for a green rating. Amber was assigned when a noncritical limitation in the evidence was found. Amber ratings could go forward if accompanied by a research plan. These ratings were recorded on the SOMP and an overall rating for the instrument given.
OMERACT summary of evidence for measurement properties of PsAID12. Color designates quality of evidence: green = good methods used, use this evidence; amber = some cautions but we will use this evidence. In the rating row, color designates overall evidence-based instrument rating for the core instrument set: green = at least 2 pieces of evidence with good methods and consistent findings of adequate or better performance; amber = a noncritical limitation in the evidence was found. The plus signs mean adequate or better performance of the instrument according to that study (for each measurement property studied). OMERACT: Outcome Measures in Rheumatology; PsAID: Psoriatic Arthritis Impact of Disease; TAG: Technical Advisory Group; SOMP: Summary of Measurement Properties.
Results including the SOMP were sent out to the OMERACT community in prereading materials. At OMERACT 2018, the evidence was presented in a plenary session and discussed in 8 breakout groups. After breakout group results were reported, a plenary vote was held with all OMERACT attendees.
RESULTS
SLR results and additional analyses are reported separately9. Briefly, 7 studies met inclusion criteria and were used as evidence: domain match (n = 3), feasibility (n = 2), construct validity (n = 5), test-retest reliability (n = 2), longitudinal construct validity (n = 2), and thresholds of meaning (n = 2).
Domain match and feasibility
Twelve GRAPPA PRP participated in the face-to-face meeting: 9 voted green for PsAID12 domain match with the core domain PsA-specific HRQOL, and 3 voted amber. There was a unanimous green vote for PsAID12 feasibility. At the GRAPPA 2017 annual meeting, 24 people voted majority amber for PsAID12 domain match, and majority green for feasibility11. Twenty-two working group members voted: 17 (82%) green for domain match and 20 (91%) green for feasibility; the rest of the working group votes were amber.
Construct validity and discrimination
Of 5 studies assessing construct validity, 2 had good and 3 had amber methods. PsAID12 construct validity was adequate or better based on the studies’ hypotheses. We conducted our own analysis for construct validity in an international longitudinal dataset17. Hypotheses were confirmed for correlations with similar/dissimilar constructs, and for known groups where we expected better HRQOL/PsAID12 scores in PsA subgroups with low disease activity versus high disease activity. This analysis constituted our sixth piece of evidence for construct validity with quality recorded as green.
There were 2 good quality studies for test-retest reliability with consistent adequate or better ICC in stable PsA patients (ICC 0.91–0.95). For longitudinal construct validity, there were 2 good quality studies that replicated their hypotheses regarding longitudinal magnitude of change in patients who changed treatment compared with patients who were stable over time; the 2 studies had consistent results of good longitudinal construct validity.
For discrimination in RCT, because no RCT had included PsAID12 at the time of our current work, we conducted a separate analysis in an LOS, considered bronze-level evidence based on OMERACT guidance3. In the LOS, a subset of patients had improved after medication change and could be compared to a subset of patients in the same study that had been stable with no change in therapy10. We prespecified that PsAID12 effect size would be large in the improved group and small in the stable group. The hypothesis was met and the level of evidence for discrimination in RCT recorded as amber. For green level of evidence, the analysis should have taken place in an RCT that was not available at that time, and be consistent across at least 2 studies.
Summary of PsAID12 measurement properties
The PsAID12 SOMP is presented in Figure 2. The combined rating of all the evidence and measurement properties was supportive of an amber recommendation, or provisional endorsement as core instrument along with a roadmap/research agenda to complete final endorsement. The working group is committed to derive evidence on discrimination in RCT once these datasets become available, and for thresholds of meaning in an ongoing longitudinal study.
OMERACT workshop, endorsement, and research agenda
The OMERACT PsA workshop participants, including PRP (2 per group; 2 groups only had 1) provided feedback in 8 breakout groups (Appendix 1). Suggestions were made as follows: to evaluate double-barreled NRS; to evaluate coping, and embarrassment/shame NRS for their sensitivity to change; to evaluate whether all PsAID12 NRS change in the same direction with an intervention; and importantly, to define clinically meaningful changes at the person level.
At the OMERACT PsA workshop, the working group formulated a recommendation for amber/provisional endorsement of PsAID12 to measure PsA-specific HRQOL in RCT and LOS. Of 96 participants who voted at the OMERACT PsA workshop, 87.5% (84) voted “yes,” endorsing this recommendation; 14 were PRP and 93% of them (13) voted “yes”; 82 participants were not PRP and 87% of them (71) voted “yes.” The working group set a research agenda to fully endorse PsAID12, which includes implementation in RCT to examine discrimination, and validation of PsAID12 thresholds for score interpretation.
Core Outcome Measurement Set uptake and implementation strategies were discussed and the following were recommended for PsAID12: dissemination of PsAID12 provisional endorsement as core outcome measure for PsA RCT and LOS; implementation in RCT; and SLR of clinical trial outcomes to assess whether core outcome measures are being used.
DISCUSSION
The PsAID12 is the first PRO provisionally endorsed as core outcome measure for PsA RCT and LOS, based on the OMERACT Filter 2.1 criteria. PsAID12 had a solid foundation of evidence for its measurement properties, derived from 7 studies conducted with good methods. In addition, truth/domain match received strong endorsement from the beginning from both PRP and the working group; feasibility was highly endorsed. There was evidence for consistent good measurement properties across studies, and no evidence for poor measurement properties, allowing us to conclude that it performed well as a measure of PsA-specific HRQOL in cross-sectional as well as longitudinal studies.
Our results support the OMERACT approach to sequential core set development: a disease-specific core domain set, followed by the Core Outcome Measurement Set. The rigorous process of instrument appraisal helps identify high-quality instruments to identify the core domains for patients with PsA; and combines identifying existing evidence with guiding the working group to create new evidence to fill any gaps. The involvement of multiple people with different perspectives in core set development is key to dissemination and implementation. Standardization of PsA measurement will create the evidence necessary for core domain inclusion in treatment recommendations.
HRQOL has been routinely measured in PsA RCT using the generic health survey Medical Outcomes Study Short Form-36 (SF-36)18, although its content validity for PsA has not been studied5. Content validity evidence would be necessary to pass the first critical step of the OMERACT filter3,4. Psoriatic Arthritis Quality of Life (PsAQoL) is a PsA-specific measure, but the GRAPPA-OMERACT literature review and appraisal of measurement properties showed that PsAID12 content validity was superior to PsAQoL for PsA-specific HRQOL5. Feasibility is also limiting for copyrighted measures (SF-36, PsAQoL), especially for LOS.
PsAID12 received overwhelming support at OMERACT 2018 and was provisionally endorsed as a core instrument for PsA clinical trials. The provisional designation has been chosen until PsAID12 discrimination in RCT and clinically meaningful thresholds for score interpretation can be appraised (PsAID12 research agenda). Additional candidate outcome measures are entering the OMERACT process of domain match and appraisal. The result will be a complete PsA Core Outcome Measurement Set in which each core domain will have at least 1 recommended core instrument. In the interim, clinical trial designers can use the current findings to inform their outcome measure choice for PsA-specific HRQOL in clinical trials.
Acknowledgment
Study collaborators: C.A. Lindsay, PharmD, Patient Research Partner employed by Amgen Inc., Thousand Oaks, California, USA; D. Veale, MD, Adjunct Professor, School of Medicine, St. Vincent’s University Hospital, University College Dublin, Dublin, Ireland.
APPENDIX 1.
Feedback from OMERACT participants on the PsAID12.
Footnotes
AMO is a Jerome L. Greene Foundation Scholar and is supported in part by a research grant from the US National Institute of Arthritis and Musculoskeletal and Skin Diseases of the US National Institutes of Health under award number P30-AR070254 (Core B), a Rheumatology Research Foundation Scientist Development award, and a Staurulakis Family Discovery award.
- Accepted for publication November 6, 2018.
REFERENCES
In this issue
