Measures of Response in Clinical Trials of Systemic Sclerosis: The Combined Response Index for Systemic Sclerosis (CRISS) and Outcome Measures in Pulmonary Arterial Hypertension Related to Systemic Sclerosis (EPOSS)

EXPERT PANEL ON OUTCOME MEASURES IN PAH RELATED TO SYSTEMIC SCLEROSIS

Progress to date

The EPOSS group first published an article defining the need for improved outcome measures PAH-SSc¹⁵. Using a Delphi process, a series of domains and tools were developed that might be used as alternative endpoints to the use of RHC to define PAH-SSc, as repeated RHC are difficult to accomplish in clinical trials. The results of the EPOSS Delphi process have been published¹⁶: 69 PAH-SSc experts, including rheumatologists, cardiologists, and pulmonologists rated a list of domains and measurement tools in an Internet-based Delphi consensus study. A statistical cluster analysis was used to define specific domains and tools to be included in this preliminary measurement tool. A meeting of the EPOSS Steering Committee was convened to remove redundant items and inconsistencies, omit elements not feasible in clinical trials, and make other judgments on a clinical basis.

The following domains and tools were judged as most appropriate for randomized, controlled trials in PAH-SSc: lung vascular/pulmonary arterial pressure and cardiac function both measured by RHC and echocardiogram, 6-minute walk test, oxygen saturation exercise; severity of dyspnea on a visual analog scale; discontinuation of treatment as measured by serious adverse events; quality of life/activities of daily living measured by the SF-36 and Health Assessment Questionnaire Disability Index; global state assessed by physician measures and survival (Table 1).

An additional analysis examined differences in responses between pulmonologists and cardiologists compared to rheumatologists (Huscher D, et al. Unpublished data). In general, it was found that rheumatologists favored patient-derived questionnaires more frequently than cardiologists/pulmonologists while the latter favored specific pulmonary related measures of response; however, combined results encompassing both groups’ assessments are required for full description and measurement of PAH-SSc.

In preparation for a prospective study and/or for use of existing data to validate the domains and tools outlined above, an in-depth structured literature review was undertaken for echocardiography¹⁶ and 6-minute walk test¹⁷. The structured reviews show which parts of the selected outcome measures have already been sufficiently validated in PAH-SSc based on the OMERACT filter to require no more research, and more importantly, define those measures that need further validation before their use to assess PAH-SSc.

Plans

For the EPOSS effort we are presently:

1. Completing the systematic literature review on core set outcomes defined by the Delphi study.

2. Planning to validate domains and tools as chosen through the Delphi exercise by: (a) Obtaining data from completed clinical trials that include as many chosen domains/tools as possible to corroborate and winnow the tools to be used in future clinical trials; (b) Assessing, based on results of the systematic literature review, remaining aspects of the core set measures identified as not yet fully validated, in further validation studies. Fortunately, a large number of randomized controlled trials in PAH have recently been performed. A subanalysis on PAH-SSc patients from the placebo arms of these studies will be performed and we are attempting to get these data.

3. Developing a patient Delphi exercise: From the patients’ perspective, changes in functional ability may be more distressing and debilitating than changes in objective measures of disease often used by physicians. However, there is insufficient information regarding which domains may be most important to patients, because many outcome measures used in PAH-SSc report either generic HRQOL, or single domains (e.g., physical function), or are not measured (e.g., fatigue). (a) We will evaluate patient assessment of disease and symptom burden in PAH-SSc using patient-centered qualitative methods. This adds an important component to the core set defined by physicians outlined above. We will conduct focus groups and interviews of PAH-SSc patients in order to determine patients’ perceptions of disease activity, disability, and HRQOL and to identify areas most distressful and concerning to patients.

4. Developing physician derived and objective measures of response separately from the patient derived measures: When completed, the 2 approaches to PAH-SSc will be examined in combination and separately to ascertain which approach is more sensitive to change in clinical trials. Each probably describes somewhat different aspects of PAH-SSc; thus a combined measure could improve discriminating ability. On the other hand, combining them in all circumstances may not be appropriate if combination does not enhance discriminating ability, or if one or the other better discriminates response.

COMBINED RESPONSE INDEX FOR SYSTEMIC SCLEROSIS

Progress to date

The first step in developing a CRISS was to conduct a structured Delphi exercise with the Scleroderma Clinical Trial Consortium membership to develop a provisional core set of items for clinical trials¹⁸. In this exercise, 50 investigators provided 212 unique items for 11 domains in a one-year clinical trial of SSc patients. In this trial 92% of the original investigators responded in the second round and rated 177 items. Nominal group technique examined these data for consistency, redundancy, feasibility and validity, judging 31 of the items from the 11 domains to be appropriate for inclusion in a one-year multicenter clinical trial. Thereafter, in a modified third round, investigators ranked 30 of 31 items as acceptable for inclusion in the course. The final domains chosen are listed in Table 2. An additional 14 items were included in a research agenda. (Table 3).

Recruiting for the one-year prospective cohort study has begun, including obtaining data in all domains and by all tools suggested from the CRISS Delphi exercise.

Plans

Preliminary efforts to develop a CRISS have included pooling of data, comprising 635 patients with dcSSc, from previously completed clinical trials of dcSSc to test some individual aspects of a possible combined index. Unfortunately, the 635-patient dataset lacks a number of the core set items from the above Delphi exercise, making that dataset unsuitable to completely test the proposed CRISS. Therefore, we are assessing all proposed core set items in a 1-year longitudinal observational study (see above) and will then employ prospective, data-driven, consensus building techniques to develop and quantitatively evaluate candidate definitions for a CRISS that will capture different organ involvements and patient-reported outcomes.

For the CRISS effort, we are presently:

1. Performing a prospective longitudinal observational clinical study to define a reliable, valid and responsive set of dcSSc measures based on the above Delphi exercise: (a) We are collecting data specified in the above Delphi-exercise defined core set in a 1-year observational study in 200 patients with early dcSSc (defined as duration ≤ 5 yrs) at 4 academic scleroderma centers. To date about 150 patients have been recruited into the cohort. (b) We will assess the reliability, validity, and responsiveness of the core set items from the real patient data collected in the 1-year observational study. (c) We will revise and define a final core set based on results obtained from 1(b). (d) The same considerations as for EPOSS will be true for the patient-derived aspects of the CRISS, as the potential tools in the CRISS include a number of patient-derived outcomes (e.g., SF-36, HAQ-DI, pain, global evaluations, and dyspnea). These will be considered in the same manner as for EPOSS, although for the present no separate evaluation by patients is planned. Nevertheless, the multiple patient-derived measures can be examined as a separate group within the CRISS, if separating those aspects from their underlying instruments is statistically and clinically valid.

2. Employing prospective, data-driven, consensus building techniques to develop and quantitatively evaluate candidate definitions for a SSc-CRI for dcSSc. (a) We will create paper-patient profiles from data gathered as part of 1(c), above, from diffuse SSc patients over a 1-year period. (b) Using consensus formation techniques (nominal group technique) we will have key opinion leaders in the field rate paper-patient profiles to estimate validity characteristics of each candidate definition for a CRISS. We are planning a sample size of 200 patients to be able to select patient profiles plus anticipated dropouts. (c) We will use ratings of key opinion leaders as a gold standard to estimate validity characteristics, such as sensitivity and specificity, and compare each using area under the curve and receiver operating characteristics analyses. (d) We will select a definition of CRISS for use in clinical trials in dcSSc that has high statistical discriminatory power and is most credible (highest face validity).

In SSc, the multiple organ system involvements can be either reversible or irreversible and the measures to separate these 2 aspects of SSc are not available. Rather the process of choosing the tools within the domains will be utilized to choose tools that are most sensitive to change. This, by implication, assumes those tools reflect reversible aspects of SSc, although this is clearly an empiric rather than physiologic or pathologically based decision.

By grouping patient derived measures, it may also be possible to find a patient oriented portion of the CRISS that can be used separately or within the complete CRISS. However, if patient derived measures prove to be an integral and required portion of the CRISS when considering sensitivity to change and clinical relevance (as seen in ACR criteria for rheumatoid arthritis and juvenile idiopathic arthritis), such a determination would be specifically addressed when the final version of the CRISS has been completed.

CONCLUSION

An overall combined measure of response in systemic sclerosis (CRISS) and a combined measure of response for PAH-SSc (EPOSS) are being developed through the OMERACT process. Significant progress has been made using the Delphi process and literature review for both indices, and prospective data are being collected.