Negative trial reporting, especially in rare diseases, provides essential information. The design and analysis of a “negative” trial, in diseases for which few potential treatments and no cure exist, warrants particular scrutiny to determine whether the trial was truly negative, or rather a “failed” trial owing to other design, financial, planning, or recruitment impediments1. A trial that is discontinued early warrants the label negative if an unbiased, independent data safety monitoring board identified either a harm signal (excessive intolerance or drug-related adverse events) or a clear worsening of the disease state in comparison to placebo.
In an article by Hsu, et al, in this issue of The Journal, the authors describe a 52-week randomized double-blind placebo-controlled multicenter phase II study of pomalidomide2. The sponsors have conceded to the reviewers that the study was discontinued early because of poor recruitment. Yet the article’s discussion section persists in stating that discontinuation by the sponsor was due to lack of drug efficacy. To be clear, there exists no statistical application capable of calculating a reliable result in any of the study variables between the 4 treatment and the 7 placebo cases completing a study that was powered for a sample of 88. Thus, the study results carry similar statistical weight to the initial anecdotal case reports of perceived efficacy of pomalidomide by systemic sclerosis (SSc) specialists — cases that likely inspired moving the drug to trial.
SSc disease behavior remains complex and challenging to the most seasoned SSc clinician researchers who are well-acclimated to a cautious approach to data interpretation in this “rare” and exceptionally complex disease. In a journal less cognizant of the humbling landscape in SSc clinical trial design, this study would have slipped by as a negative study, generating an inaccurate interpretation of the conclusions in the less discriminant reader, who could have been an insurance provider or a grant reviewer presiding over a decision to grant compassionate treatment or to fund a new study in this drug class. Though declined as negative, the study is rightly afforded publication as an occasion to pause and collectively consider the complexity of design, preparation, and planning for a trial in SSc.
When patients with life-threatening progressive conditions such as certain malignancies or SSc enroll in clinical trials, patient life-years are bartered, in good faith, against the likelihood that a trial will at least provide information beneficial to future patients and researchers even if it does not have self-benefit. Patients bravely provide consent against the possibility of irreversible disease progression and death, potentially sacrificing otherwise currently acceptable treatment strategies, as well as forgoing participation in other (perhaps better designed) available trials.
Further, each clinical trial ties up the most premium research resource: patients. When multiple trials recruit concurrently, it potentially impedes enrollment across the SSc research community, thereby protracting study time frames, increasing costs, and financially dissolving trials unprepared for this. Such a scenario is equally applicable in trials in other rare diseases. Therefore, exquisite care and clarity in study design, preparation, planning, and implementation are required to safeguard investment of patient life-years and the provision of valuable insight into treatment efficacy and safety, and perhaps even disease behavior and subtyping. In this trial, the demands of continuing a trial with poor chances of efficiently producing meaningful results could not be met. Rather than embroiling patients and finances further, an appropriate decision was made to halt recruitment.
SSc research is fraught with idiosyncrasies related to disease and outcome measures. For example, we are learning that, depending on the targeted manifestation, SSc may demand longer study duration, to allow for improved demonstration of treatment response, than for example in idiopathic pulmonary arterial hypertension, with SSc lung studies extending treatment by 6 to 12 months3.
The single most enigmatic challenge to clinical trial design in progressive, potentially obliterative multisystem autoimmune diseases such as SSc, vasculitis, and sarcoidosis is deciphering irreversible, untreatable damage from active, progressive, treatment-responsive disease. Prolonged disease duration, especially in SSc–interstitial lung disease (ILD), is associated with a higher likelihood that less treatment-responsive damage exists and improvement is increasingly less discernible in a clinical trial. Trials are optimally designed to select for the likelihood of a subject to respond if exposed to a treatment that is effective — this is called “cohort enrichment.” The stretching of disease duration out to 7 years4 seems safer relegated to trials testing safety rather than efficacy5. Though SSc-ILD can continue to progress after 5 years, the greatest volume loss occurs in the first 2 years of onset, followed by much smaller increments thereafter6,7. Therefore, limiting disease duration until a more discriminating disease activity detection algorithm (which may depend upon disease subtype or target organ) can be identified is a vital consideration to trial criteria. Comparative disease duration is also essential descriptive information between trial arms.
In this trial, the authors speculate that cohort enrichment strategies were overly restrictive and resulted in poor enrollment. However, the study criteria do not differ greatly from other concurrent SSc-ILD trials successfully meeting recruitment5,8, except for variance regarding permissiveness of concurrent therapies. The reinforcement criteria for a baseline forced vital capacity (FVC) 70–80%, requiring demonstration of a ≥ 5% drop within a prior 2-year period, is a good enhancement, albeit still liberal in a stratum in which progressive disease is expected to decline by large increments4,5. Importantly, the vagueness and lack of standardization surrounding inclusion of pulmonary hypertension (PH) is curious, a vital distinction to preserving cohort enrichment. Here, the simple application of FVC:DLCO ratio9,10,11, designed to aid in distinction of ILD versus PH predominance, might have enhanced cohort enrichment.
The modified Rodnan skin score (mRSS), as a primary outcome measure, is often used in clinical trials, yet only ever validated in early diffuse disease (about initial 24 mos). Its limitations and enrichment potential over time have been described12. While the mRSS and ILD correlate well and make sense as co-primary endpoints, the University of California at Los Angeles Scleroderma Clinical Trial Consortium Gastrointestinal Tract (SCTC-GIT), a lengthy questionnaire addressing gastrointestinal symptomatology, lacks clarity in this trial as a primary endpoint choice. Here, the SCTC-GIT displaces well-validated relevant patient-reported instruments in an ILD trial, such as those measuring respiratory symptoms as well as the Scleroderma Health Assessment Questionnaire, a widely used, strongly validated, user-friendly outcome measure incorporating several additional validated scales that assess central SSc manifestations potentially responsive to treatment, including a validated lung symptom scale. These measures were relegated to secondary and “exploratory” assessments — and in the absence of robustly validated health-related quality of life (HRQoL) instruments such as the patient’s global assessment of disease activity, the Medical Outcomes Study Short Form-36, or EQ-5D. HRQoL is a parallel competitor to survival, as perceived by those living with pulmonary fibrosis13,14.
Concurrent SSc and SSc-ILD trials appear to have met recruitment targets5,8,15. There are 56 enrollment sites listed on clinicaltrials.gov for this trial16. Did site investigators lose confidence in the drug, study design, or financial feasibility? Or perhaps concurrent trials were competing at single sites with other trials that were more appealing or convincing. Was there hesitation to enroll sick patients in a placebo-controlled study without permissive background or rescue therapy or the reassurance of frequent FVC assessments to identify and rescue declining health sooner?
Since the demonstration of delayed ILD progression with cyclophosphamide in the SLS I trial17, controversy persists surrounding the ethics and science of placebo-only versus allowance of background therapy with placebo. This study seems likely to have accommodated “niche” patients — those intolerant of, failing, or unable to attain other available therapies or trial enrollment; compounded by investigator recruiting tendencies/fears expressing the weight of community sentiment regarding placebo, thus impairing recruitment numbers.
Assuring accountability, accuracy, and clarity in treatment-data reporting is crucial in a disease of scarce therapeutic options. This drug class with promising preclinical and clinical treatment properties has not been appropriately evaluated in this disease, and to discard its future use and research would be, at this time, a disservice.
We are heavily reliant upon partnership with industry to persevere in addressing treatment gaps in SSc. This partnership needs to be fostered with transparency and a strengthening of shared knowledge and patient-centered priorities. The reporting of this study’s results is somewhat confusing, requiring significant effort to establish actuality between the data tables and the narrative, with initial manuscript submissions mistaking the actual conclusion. As trial numbers in SSc continue to escalate, to assure the best use of resources and trial outcomes, the SSc research community needs to contemplate a pre- and post-study endorsement mechanism sanctioned by a global knowledgeable equanimous community, such as the SCTC.
This could involve a trial design preview for crucial preparatory guidance before a study is initiated, whereby the nuts and bolts of study design could be reviewed as well as alerting potential trialists to design enhancements and possible community sensibilities such as background/rescue therapy that could impair recruitment, and importantly, a poststudy review of results and interpretation. The poststudy endorsement process might include a template-based application/examination of a composite containing the study protocol, an analytic results questionnaire, a summarized interpretation of the data (free of narrative distraction), and a succinct addendum to qualify any potential irregularities or impediments in protocol or data acquisition. It is true that a risk of delayed reporting exists with such a system, but the benefits to research, use of resources, and patient care are a worthy exchange. Such an endorsement body could tandemly protect and promote much-needed creative examination of enrichment and design enhancements.
Lastly, in clinical trial design, the input of an expert is often overlooked and inherently important. This expert is the patient. Patient advisers/research partners intimately understand the logistics of living with SSc and can provide great insight on feasibility, compliance, and concerns about clinical trial design.