To the Editor:
Systemic sclerosis (SSc) is a connective tissue disorder characterized by a disturbance in fibroblast function culminating in the telltale skin thickening and fibrosis of visceral organs including the lungs. Since the histopathogenesis of SSc is characterized by immune dysfunction and inflammation, there is good scientific rationale for using immunosuppression in this disease.
However, SSc is uncommon, with an estimated prevalence ranging from 7 to 489/million and incidence from 0.6 to 122/million/year1. This rarity has undoubtedly contributed to the paucity of randomized controlled trials examining the benefit of immunosuppression in this disease. The Scleroderma Lung Study is probably the largest trial to date, with 158 patients recruited over 3.5 years from 13 centers2.
Moreover, the natural history of SSc is highly variable. The course of skin involvement differs between subsets, with limited disease having a more slowly progressive course over years and decades, while skin thickening progresses gradually in the first 3–5 years of diffuse disease and may even improve in later disease3. Lung involvement is common but there is tremendous uncertainty concerning the rate of decline in lung function in SSc. Although the annual decline in forced vital capacity (FVC) in patients with early, diffuse disease had been estimated to be approximately 9% in the Scleroderma Lung Study, in fact the FVC of patients in the placebo arm fell by only 2.6% during the 1-year study2.
Thus, at present, there are few trials of immunosuppressants in SSc and these trials have shown only modest benefits of immunosuppression. In a 1-year randomized placebo controlled trial of 71 patients with early (< 3 years of disease) diffuse SSc, the trend toward better modified Rodnan skin scores (range 0–51) in the methotrexate (MTX; score 21.4) compared to the placebo (score 26.3) groups did not reach statistical significance (p = 0.17)4. In the Scleroderma Lung Study (N = 158)2, patients treated with oral cyclophosphamide for 1 year had a smaller decline than those receiving placebo in FVC [adjusted difference of 2.5% (95% CI 0.3, 4.8%, p = 0.03) in FVC favoring cyclophosphamide]2. However, these trials were limited by short-term followup.
Yet despite the paucity of evidence supporting the use of immunosuppressants in SSc, MTX for skin involvement and cyclophosphamide for lung involvement have recently been recommended by the European Scleroderma Trials and Research Group (EUSTAR level A recommendation)5. In addition, these drugs continue to be widely used. In a survey of North American rheumatologists conducted in 2003, 62%–85% reported using MTX for skin disease, and 83%–99% cyclophosphamide for lung disease6. There is thus a large gap between evidence and practice concerning the use of immunosuppression in SSc.
Moreover, it has been well documented, including in Canadian patients with SSc7, that patients who participate in studies are often highly selected and may not be representative of “real” patients with scleroderma seen in everyday clinical practice. In addition, drug trials rarely assess drugs head-to-head or in sequence, and are inevitably short and cannot adequately address issues of longterm benefits and toxicity. Thus, longterm benefits and risks of individual immunosuppressant drugs as well as drugs used sequentially remain largely unknown, and considerable uncertainty persists about their use in SSc.
Given the importance of determining the benefits of immunosuppression in SSc and the difficulties of conducting trials in this disease, alternative study designs need to be considered. Analysis of longterm observational data with marginal structural models and inverse probability-of-treatment weighting to address issues of causal inference and potential confounding represents an excellent approach for answering questions involving dynamic treatment regimes. Indeed, the problem with observational studies is that the lack of randomization does not exclude the possibility that variables causally associated with the exposure (e.g., MTX and disease subset) would also be related to outcome (e.g., decreased lung function). Similarly, assignment to treatment by immunosuppressants at time 1 could depend on the response to previous treatment (or nontreatment) at baseline. Marginal structural models and inverse probability weighting allow one to create “pseudo-randomized” treatment groups by reweighting observations in a way that balances the potential confounders between the treatment groups. This results in estimates that are unbiased for the causal effect of the exposure on the outcome.
This analytic approach is being increasingly used. Recently in this journal, Herrick, et al used inverse probability weighting to estimate the relative effectiveness of different treatment approaches on skin disease in SSc8. They included 147 patients in 5 different treatment groups — intravenous cyclophosphamide followed by mycophenolate mofetil (MMF), antithymocyte globulin followed by MMF, MMF alone, no disease-modifying treatment, and other immunosuppressant treatment. They found that overall skin scores decreased from 24 at baseline to 16 at 3 years, but that there were no significant differences between groups in the rate of change of skin scores over time, even when inverse probability-of-treatment weights were applied.
In order to determine the potential of inverse probability-of-treatment weighting to estimate causal effects of immunosuppressants on lung outcomes, we performed a pilot study based on the Canadian Scleroderma Research Group (CSRG) registry data. The CSRG follows a large cohort of patients with SSc at 12 centers across Canada. All patients in the registry are assessed yearly by standardized clinical examinations, self-reported questionnaires, and laboratory investigations. As of January 2010, there were 1034 patients in the CSRG registry. Of these, 17.7% were taking an immunosuppressant (cyclophosphamide, MMF, azathioprine, MTX, or D-penicillamine) at the time of their baseline visit. Patients treated with immunosuppressants were younger, less likely to be women, and had shorter disease duration (Table 1). They were more likely to have diffuse SSc, interstitial lung disease, and more severe disease.
We included patients who had at least 2 yearly visits between September 2004 and January 2010 and who had complete data, including pulmonary function tests (N = 420). We classified patients as exposed if they were reported to be currently taking cyclophosphamide, MMF, or azathioprine at the first visit. The outcome of interest was FVC. In order to adjust for the potential confounding due to nonrandomization, we used a basic inverse probability-of-treatment weighted analysis. In a naive cross-sectional analysis with no weighting, FVC decreased by −1.1% (95% CI −10.4, 8.2) in the exposed compared to the unexposed patients, demonstrating the potential effect of confounding by indication and lack of randomization between the 2 groups. However, after adjusting for baseline FVC and introducing inverse probability-of-treatment weights, we estimated an increase of +2.5% in the FVC in exposed compared to unexposed patients (95% CI −1.0, 6.1). Our pilot estimate using the inverse probability-of-treatment weights is in strikingly close agreement with the results of the Scleroderma Lung Study [2.5% (95% CI 0.3, 4.8%, p = 0.03)]2 and a metaanalysis of cyclophosphamide for SSc-ILD [2.83% (95% CI 0.35, 5.31)]9.
We can conclude 3 important things from this relatively basic analysis. First, although these are observational registry data, one can, with the correct analysis, obtain reliable estimates of the effect of exposure on outcome. This approach thus offers a useful alternative to trials that are difficult and costly to conduct, especially in a rare, slowly evolving disease such as SSc. Second, given the natural history of SSc, the choice of an appropriate primary outcome for interventional studies remains highly challenging. While skin involvement can improve spontaneously and thereby confound the results of a study examining the effect of immunosuppression on skin disease in SSc, our data suggest that lung function may be a better outcome measure for studies of immunosuppression in SSc. Finally, it is critical to perform longterm data collection on large samples to identify true effects of dynamic treatment regimens used in SSc. Indeed, a clinically meaningful difference in FVC has been defined to be 10%10. Thus, given the estimates from our data, the benefits of immunosuppression on lung function in SSc may take 3 years or more to become detectable. Moreover, this is complicated by the possibility of a differential treatment effect in certain patient subpopulations. Indeed, assuming that only one-fifth of patients progress by an FVC of 12%, a drug that prevents all progression will appear to have an average FVC effect of only 2.5% and may be mistakenly viewed as having only a trivial effect. Randomized clinical trials on drug effectiveness may be underpowered to estimate differential treatment effects in patient subgroups, further emphasizing the need to use longitudinal, observational data in large patient samples.
Longitudinal patient registries are expensive to maintain, sensitive to patient dropout, and take a long followup time to generate useful data. Nevertheless, the findings that are generated, especially in the setting of a standardized, prospective, individual patient data collection protocol using validated outcome measures on a representative sample of the population (such as that of the CSRG registry), are of substantially superior quality to other study designs (e.g., retrospective or cross-sectional studies) and provide the best evidence to answer a wide range of questions that are simply not amenable to be studied in the context of randomized clinical trials.
We believe that the pilot data presented here provide justification to pursue further longterm observational studies of drug effectiveness in SSc. These studies using robust observational data analysis will allow us to fill important knowledge gaps in the treatment of SSc.
Footnotes
-
CSRG Investigators: M. Baron, Montreal, Quebec; J. Pope, London, Ontario; J. Markland, Saskatoon, Saskatchewan; D. Robinson, Winnipeg, Manitoba; N. Jones, Edmonton, Alberta; N. Khalidi, Hamilton, Ontario; P. Docherty, Moncton, New Brunswick; E. Kaminska, Hamilton, Ontario; A. Masetto, Sherbrooke, Quebec; D. Smith, Ottawa, Ontario; E. Sutton, Halifax, Nova Scotia; J-P. Mathieu, Montreal, Quebec; M. Hudson, Montreal, Quebec; S. Ligier, Montreal, Quebec; T. Grodzicky, Montreal, Quebec; S. Mittoo, Winnipeg, Manitoba; M. Fritzler, Advanced Diagnostics Laboratory, Calgary, Alberta.