Abstract
Objective. To develop and validate, as a measure of overall health status, a Frailty Index (FI) for patients (n = 1372) in the Canadian Scleroderma Research Group (CSRG) Registry.
Methods. Forty-four items were selected from the CSRG database as health deficits and recoded using FI criteria. To test construct validity, we compared measurement properties of the CSRG-FI to other FI, and related it to measures of damage, age, and time since diagnosis. To test criterion validity, we compared the baseline FI to that at last recorded visit and to mortality.
Results. The mean CSRG-FI was 0.33 with a sub-maximal limit of 0.67. In patients with diffuse disease, the mean was 0.38(SD 0.14); in patients with limited disease, the mean was 0.31(SD 0.13). The CSRG-FI was weakly (but significantly) correlated with the Rodnan Skin Score (r = 0.28 in people with diffuse disease; 0.18 with limited) and moderately with the Physician Assessment of Damage (r = 0.51 for both limited and diffuse). The risk of death increased with higher FI scores and with higher physician ratings of damage. The area under the receiver operating characteristic curve for the baseline FI in relation to death was 0.75, higher than for other measures (range: 0.57–0.67).
Conclusion. The FI quantifies overall health status in people with scleroderma and predicts mortality. Whether the FI might help with decisions about who might best be served by more aggressive treatment, such as bone marrow transplantation, needs to be evaluated.
As the population ages, so too does the prevalence of scleroderma1. This is both because autoimmune disorders become more common with age, and because advances in treatment mean that more people with scleroderma are living longer. Ageing itself reflects the accumulation of subcellular deficits, which arise in a diverse set of interactions between stochastic, damage-driven mechanisms and genetically specified pathways that influence longevity; such deficits commonly scale up to become manifest clinically (or detectable by laboratory methods)2. Although people with scleroderma might be expected to accumulate more deficits than do others of the same age, not everyone at the same age with scleroderma has the same level of need. This partly reflects disease activity and degree of damage, and partly reflects the variable health status of people of the same age. This increased vulnerability to adverse outcomes of people of the same age is a wider phenomenon, broadly known as frailty3.
The need to identify people at greatest risk means that frailty is a topic of growing interest across a wide range of medical literature4,5. Even so, its precise operational definition has yet to be settled. Two approaches are recognized6. One views frailty as a specific syndrome typically associated with wasting, and consisting in weight loss, impaired strength, reduced activities, slow gait speed, and exhaustion7. Another sees frailty as arising in relation to the accumulation of a wide variety of health deficits: people with few deficits are relatively fit; those with many things wrong are frail. Inasmuch as deficits can accumulate across the adult lifespan, frailty in this light is seen as a problem not just of the aged, but of the ageing process itself8.
Systemic sclerosis (SSc) is an example of an illness that affects many organ systems, with problems that can accumulate across the adult lifespan in some people, and much more rapidly in others9,10,11,12,13,14,15. Many people live decades with SSc14, but mortality risk is high15; our central hypothesis is that such variable health trajectories can be seen as another face of variable vulnerability in relation to deficit accumulation. If so, evaluating frailty as deficit accumulation might assist in understanding the variable health status of adults with SSc, so we evaluated the properties of a frailty index (FI) in the Canadian Scleroderma Research Group (CSRG) database. Our goals were to: (1) assess the feasibility of employing a standard procedure to calculate an FI from the CSRG database, and (2) assess the validity of the CSRG-FI.
MATERIALS AND METHODS
Design and setting
This is a secondary analysis of longitudinal data collected by the CSRG, a national research collaboration based in Montréal, Québec. Patients in the CSRG Registry are recruited from 15 centers across Canada and must have a diagnosis of either limited or diffuse scleroderma as determined by the recruiting rheumatologist, be > 18 years of age at the time of recruitment and fluent in either English or French. Patients are seen yearly. The baseline visit was between 2004 and 2012. Details of the registry have been published elsewhere12,13,14,16,17. As of June 26, 2012, the dataset consisted of 1372 patients.
Assessing the feasibility of constructing an FI in the CSRG database
An FI can be developed according to a standard procedure for selecting items and calculating the resulting scores18. The procedure for this is as follows: first, potential health deficits are identified. Here, the CSRG database contained 211 items available from a self-report questionnaire and a clinical evaluation. Items were selected if they met standard criteria to be a health deficit. A single health deficit can be any symptom, sign, functional impairment, or laboratory abnormality that meets the criteria of being: associated with age; associated with an adverse outcome, present in at least 1%, but not more than 80% of the sample; and present in 95% of participants (i.e., containing < 5% missing data)18,19. Next, any set of health deficits included in an FI must cover several organ systems. In addition, it is worth noting that even though an FI can include comorbidities, it must also contain more items than only comorbid illnesses, and so, for example, typically includes information about function and mobility18,19. Finally, the individual health deficits are combined to produce an FI score. For individual patients, their FI score is the proportion, between 0 and 1, of the number of health deficits present in them in relation to the number of health deficits considered. Here, for example, with 44 items from the CSRG qualifying as health deficits, a person in whom 22 were recorded as present would have an FI of 22/44 = 0.50. The FI developed from the CSRG database (the CSRG-FI; Appendix 2) was created from disease specific variables, as well as more general health measures from function, mood, fatigue, and sleep scales. The CSRG database contains the PHQ-9 depression scale20, the FACIT fatigue scale21, and the 8-item version of the PROMIS sleep scale22. We added laboratory and measurement data on target organ damage (Appendix 2). Each health deficit was calculated using established cut points, assigned a score between 0 and 1. In most cases, variables were binary, with a score of 0 representing no deficit, and 1 representing the deficit fully expressed. For trichotomous variables, a score of 0.5 was assigned to midpoint values.
Validation strategy
We followed a tripartite approach to validation, considering each of content, construct, and criterion validity23. Content validity is inherent in the use of items from known scleroderma scales. Their selection and combination followed a well established procedure18, satisfying the requirement that items included in a measure make sense on their face. To test construct validity, we first compared the properties of the CSRG-FI with those known to obtain in other FI. For example, we were interested in knowing the relationship with age. Note that in population-based studies from developed countries, the slope of the relationship to age is about 3% per year on a log scale; by contrast, in clinical and institutional samples, this is attenuated, usually to about 1% per year24. Further, a consistent feature of the FI approach is the presence of a submaximal limit: 99% of people have an FI value < 0.725. For comparison with other measures used in scleroderma studies, we correlated the CSRG-FI with the Rodnan Skin Score26 and the Physician Global Assessment of Damage27,28 (“On a 10-point scale, where 0 = no damage and 10 = most damage, how much damage do you think that the patient has from his/her scleroderma?”). We used linear regression to evaluate the relationship between the CSRG and age and time since diagnosis. Recognizing that, on average, people with diffuse scleroderma should have more deficits than those with limited scleroderma, we also compared CSRG-FI scores by scleroderma type. To evaluate the statistical significance of differences arising in relation to our objectives, differences in proportions were tested using chi-square, and in means using t test or 2-way ANOVA. The most rigorous form of validation is criterion validity. There are 2 accepted types of criterion validity: comparison with a reference standard and predictive validity23. The lacking of a reference standard (often also known as a “gold standard”) was one of the factors that motivated the creation of the CSRG Registry12. To evaluate the predictive validity of the CSRG-FI, we focused on predicting 2 outcomes, using the baseline CSRG-FI values: future health status, and mortality risk. Predicting future health status was done by evaluating the association between the baseline CSRG-FI and the final recorded visit CSRG-FI. To evaluate mortality prediction, we calculated the areas under the Receiver Operating Characteristic (ROC) curves for the baseline CSRG-FI and compared these with the Physician Global Assessment and Rodnan Skin Severity Scores. Likewise, to look at change in the mortality hazard rates associated with increments in the various test scores we first tested the assumption of proportionality with a log-minus-log-plus plot and then used Cox proportional hazards model. In these analyses, for each measure, and for covariates, we used baseline visit data.
Data analysis
Data were analyzed using MATLAB R2010b, version 7.11, and PASW Statistics, version 18.0. Ethics data collection under the CSRG was approved at McGill University (Montreal, Quebec) and at all participating sites. All patients provided written, informed consent. Permission for these secondary analyses was granted by the Research Ethics Committee of the Capital District Health Authority.
RESULTS
Cohort at baseline and feasibility of constructing the FI
Most study participants in each of the limited and diffuse scleroderma types were white women. Except for blood pressure, patients with diffuse disease tended to have worse health measures at baseline than did patients with limited cutaneous disease (Table 1). Of the 211 potential health deficits, most (N = 167) were excluded due to missing data. Of the remaining 44 items with < 5% missing data, each met all criteria for being a health deficit. As a group, they covered several organ systems. As detailed in Appendix 2, they included items on comorbidity, impairments, health attitude, affect, and function. The mean CSRG-FI for the group was 0.33 (SD 0.14). A baseline CSRG-FI could be calculated for 1260 of 1372 participants. Of the total, 293 dropped out or were lost to followup before study end. Compared to those on whom data are complete [mean age 55.9 years (12.0); 86.5% women; CSRG-FI = 0.33 (0.13)], people who dropped out were older [mean 59.5 years (13.4)], more often women (92.1%), and less frail [mean FI-CSRG 0.29 (0.13)].
Clinical and demographic characteristics of patients at the time of their baseline visit, by scleroderma type.
Validation of the CSRG-FI
In the construct validity analysis, as expected, older people had higher CSRG-FI values than did younger people, and those that had been admitted to the hospital in the last year had higher CSRG-FI than those who had not. On average, older people had more health deficits than younger people, so that the mean value of the CSRG-FI increased with age. Even so, reflecting that some young people with scleroderma can be very ill, this relationship was nonlinear, and the fit of the data was not improved by log transformation. Overall, the 99% limit was approximately 0.67, but the maximum recorded value was higher in diffuse (CSRG-FI = 0.70) than in limited scleroderma (CSRG-FI = 0.62). Similarly, patients with diffuse cutaneous disease had a higher mean CSRG-FI than did patients with limited scleroderma, even though they were 4 years younger on average (Table 1). The FI was slightly, but not significantly, higher in people with tender joints (0.37; SD 0.13) than in those without tender joints (0.31; SD 0.14). Similar results were seen in relation to swollen joints (0.37; SD 0.14 vs 0.32 vs 0.14); note that neither of these were items in the CSRG-FI. The correlation between the Rodnan Skin Score and the CSRG-FI was 0.28 in people with diffuse scleroderma and 0.18 in those with limited disease. The correlation between the physician assessed damage score and the CSRG-FI was the same (0.51) in people with diffuse and in those with limited scleroderma. For criterion validation, the risk of death increased with increasing values of the baseline CSRG-FI scores and with the physician assessed damage scores (Figure 1). In the ROC curve analyses (using baseline visit data), the area under the curve (AUC) for the FI was 0.75 in those with diffuse disease. By contrast, the AUC for the physician assessed damage score was 0.67. The AUC for limited disease were not significantly different: 0.70 versus 0.71, respectively. Each was significantly higher than the Rodnan Skin Score (e.g., AUC = 0.57 for diffuse) and age (AUC = 0.56). In the proportional hazards analysis, controlled for age and sex, each of the CSRG-FI (HR = 1.68, 1.50–1.87, p < 0.001), the physician assessed damage (HR = 1.35, 1.27–1.43, p < 0.001), and the skin score (HR = 1.03, 1.02–1.05, p < 0.001) predicted mortality when considered individually, but when combined, only the FI and physician assessed damage (and age) remained significant (Table 2). In relation to the final recorded visit scores, the mean CSRG-FI of the people with diffuse disease who died was 0.50 (SD 0.13), compared with 0.34 (SD 0.13) in those with diffuse disease who lived (p < 0.01). Likewise, in people with limited disease, the mean CSRG-FI score was 0.41 (SD 0.13) in those who died versus 0.29 (SD 0.12) in those who survived (p < 0.01). Physician assessed damage scores for people with diffuse disease who died were 6.4 (SD 2.3), compared with 4.4 (SD 2.1) in those with diffuse disease who lived (p < 0.01). Similarly, in people with limited disease, the physician assessed damage score in decedents was 5.2 (SD 2.7) compared with 2.9 (SD 1.9) in survivors (p = 0.02).
Risk of death increased with worsening disease as measured by the frailty index and physician global assessment of damage.
Cox proportional Hazards model for patients at the time of their baseline visit.
DISCUSSION
Using data from the multicenter Canadian Scleroderma Research Group database, we were able to create an FI for people with scleroderma. The resulting CSRG-FI showed good construct and criterion validity. The CSRG-FI was correlated with physician assessments of disease severity, but was more highly associated with the risk of death. The mean CSRG-FI scores were also higher in patients with diffuse scleroderma than in patients with limited scleroderma. Most important, regardless of age, disease type, or time since diagnosis, the risk of death was higher with higher FI scores.
Our data must be interpreted with caution. With 1372 participants, the numbers become small for some clinically useful levels of disaggregation. Even so, this is the largest database for individuals with scleroderma in Canada, and an ongoing national effort that enrolls about 175 patients annually and contributes information about a range of important topics12,13,16,17. Similarly, as a sample of people referred to subspecialty clinics, the database is not representative of the population, but this too is a necessary condition, given that the low prevalence of scleroderma (4 to 489 cases per million individuals)1,9 makes representative studies impractical. Similarly, although longitudinal, this study incorporates people enrolled at varying points in their disease course, resulting in considerable variability in baseline health status. Modeling longitudinal effects in such a circumstance is a particular challenge, especially when exposures have a nonlinear distribution and nonlinear impact on outcomes1. With the data to hand, we used a typical compromise of considering only baseline scores in the ROC and proportional hazards analysis; initial longitudinal analysis showed that the rate at which the FI might change may be important, but this will require a larger dataset and nonlinear models for further evaluation. Patients were only seen once a year, which is a limitation for individuals with accelerated disease, although for most of the study participants this was not an issue. The mortality rate was low (12%) making estimates associated with death more challenging. In this regard a common and troubling clinical problem is how to identify people with diffuse scleroderma who are at a high risk of death (such that early aggressive immunotherapy or experimental therapies might be considered). Here, of 142 people diagnosed with diffuse systemic sclerosis and an estimated disease duration less than 3 years, 25 (17%) died. Considering their baseline CSRG-FI scores, only modest differences between the 2 groups were observed: people who died within 3 years had a mean CSRG-FI equal to 0.44 (SD 0.13) compared with those who survived, in whom the mean score equalled 0.36 (SD 0.13). This might reflect that not all disease activity conveys equal risk. Considering trajectories, the mean CSRG-FI in those who died (last visit mean = 0.47; SD 0.13) was higher than at baseline, whereas in those who survived it was in fact slightly lower than their baseline score (next visit CSRG-FI score = 0.35; SD 0.14).
A proinflammatory state is often seen in many patients with frailty, especially increased IL-629,30, which has also been implicated in systemic sclerosis31,32. Although an FI has been used to evaluate frailty in immunosenescence29 and vaccination33 studies, this is the first study to develop an FI for scleroderma patients. The correlation of the CSRG-FI with the 17-item modified Rodnan Skin Score is consistent with other reports of higher skin scores being associated with poor prognosis34,35,36. Our study showed that although the modified Rodnan Skin Score predicted mortality in an age and sex adjusted model, the CSRG-FI is a better predictor. Other studies have shown that decreasing skin scores are associated with a lower risk of death35. The low correlation between the modified Rodnan Skin Score and CSRG-FI is most likely related to mechanisms of disease: those who survive will eventually see skin involvement decrease. Patients with limited disease generally show late skin thickening, which stays constant at a relatively low modified Rodnan Skin Score. Patients with diffuse scleroderma who survive their first 10 years with the disease will most likely die with their disease, not from it37. Other scales have been created to measure scleroderma severity, although as noted none has yet enjoyed widespread clinical use1,12. The CSRG-FI covers 9 organ systems, as well as mood and fatigue. As a more holistic measure of health, and not just a measure of disease severity, the CSRG-FI may aid in characterizing older scleroderma samples, as older adults in particular have other grounds on which to accumulate important health deficits. In this regard, it is worth considering that any FI works because it allows for the impact of small cumulative effects, which on their own might not meet a threshold for being statistically significant, but which might still provide information of value38. Our data also contribute to a broader understanding of frailty and its impact on healthcare. Although most studies have focused on older adults, it is clear that frailty can be detected across the lifespan39. Here, as in another recent study40, we expand that understanding to demonstrate its use in a disease that affects not just older adults, even though we note that older adults are a growing constituency of rheumatology clinics41. Even so, whereas the other measures were scored at the patient interview, the FI-CSRG is presented here for the first time. Its clinical utility will require prospective evaluation in a setting in which the scores could be made available to treating physicians at the time that they were seeing patients. We found that the FI was higher (0.33) than would be expected in a general population, where the median value is 0.0339. Even so, the 99% maximum limit value (0.67) did not exceed the 99% limit reported in a large number of other settings24,38,42,43,44. This is an important finding worth confirming in a clinical cohort, i.e., that there appears to be a biological limit to the number of health deficits that an individual can accumulate. The outcomes of people close to the limit will be of particular interest as the cohort expands, and as longer followup periods are recorded.
That small effects can aggregate to produce large ones is well established in domains such as information theory and signal processing. It has been adapted in a variety of healthcare settings19,40,42,43,44,45. In evaluating cumulative small effects, the FI can offer important insights. For example, an FI made up only of items not known to be associated with dementia proved to be a powerful risk factor for dementia, even trumping known traditional risk factors46. This has led to the suggestion that the epidemiology of late life cognitive disorders might need to be reevaluated47. Just as diseases present differently in people who have many illnesses, so too might risk factors operate in ways that need reconsideration (a point that might have special relevance in multisystem diseases like scleroderma, especially as survival improves). For now, using an FI appears justified by integrating useful, available information that is otherwise gathered in severity scoring systems. Whether that proposition holds more generally, or extends to a reevaluation of risk factors, will require additional cross-validation, both in the growing CSRG database, and by other groups. If it holds, then it would be possible to consider whether an FI might help with decisions about who might best be served by more aggressive treatment, such as bone marrow transplantation.
APPENDIX 1
List of study collaborators: Investigators of the Canadian Scleroderma Research Group: J. Pope, London, Ontario; M. Baron, Montreal, Quebec; J. Markland, Saskatoon, Saskatchewan; D. Robinson, Winnipeg, Manitoba; N. Jones, Edmonton, Alberta; N. Khalidi, Hamilton, Ontario; P. Docherty, Moncton, New Brunswick; E. Kaminska, Hamilton, Ontario; A. Masetto, Sherbrooke, Quebec; E. Sutton, Halifax, Nova Scotia; J-P. Mathieu, Montreal, Quebec; M. Hudson, Montreal, Quebec; S. Ligier, Montreal, Quebec; T. Grodzicky, Montreal, Quebec; S. LeClercq, Calgary, Alberta; C. Thorne, Newmarket, Ontario; G. Gyger, Montreal, Quebec; D. Smith, Ottawa, Ontario; M. Fritzler, Advanced Diagnostics Laboratory, Calgary, Alberta.
APPENDIX 2.
Canadian Scleroderma Research Group Frailty Index to measure health status in patients with systemic sclerosis.
Footnotes
-
For a list of CSRG members see Appendix 1.
-
The Canadian Scleroderma Research Group (CSRG) is funded by the Canadian Institutes of Health Research (CIHR; grant #FRN 83518), the Scleroderma Society of Canada and its provincial Chapters, Scleroderma Society of Ontario, Sclérodermie Québec, Cure Scleroderma Foundation, INOVA Diagnostics Inc. (San Diego, CA), Dr. Fooke Laboratorien GmbH (Neuss, Germany), Euroimmun (Lubeck, Germany), Mikrogen GmbH (Neuried, Germany), Fonds de la recherche en santé du Québec (FRSQ), the Canadian Arthritis Network (CAN), and the Lady Davis Institute of Medical Research of the Jewish General Hospital, Montreal, QC. The CSRG has also received educational grants from Pfizer and Actelion pharmaceuticals. Funds for secondary analysis came from the Fountain Innovation Fund of the Queen Elizabeth II Health Sciences Foundation and by an operating grant to KR from the Canadian Institutes of Health Research MOP-209888.
- Accepted for publication December 13, 2013.
