Abstract
Objective. To develop and validate a modified version of the Disease Activity Score with 28 joint count (mDAS28), for use in epidemiological research, when acute-phase reactant values are unavailable.
Methods. In a cross-sectional development cohort (5729 patients), statistically significant predictors of the logarithm of erythrocyte sedimentation rate (lnESR) were identified. After computation of the mDAS28, a cross-sectional validation cohort (5578 patients) was used to evaluate internal, criterion, and construct validities. The ability of the mDAS28 to discriminate between disease states was also assessed. A second validation cohort (longitudinal, 336 pairs of patient visits) was used to assess sensitivity to change.
Results. Significant predictors of lnESR included tender and swollen joints with 28 counts, patient’s and physician’s assessments of global health, and patient’s assessment of pain (visual analog scale 0–100 mm) and a physical function (modified Health Assessment Questionnaire 0–3; mHAQ). Satisfactory internal validity (α = 0.72) and strong criterion validity compared to the DAS28, the Simplified Disease Activity Index (SDAI), and the Clinical Disease Activity Index (CDAI) (r = 0.87–0.96) were found. Predictive validity was demonstrated by good correlation with the mHAQ (r = 0.58). The mDAS28 showed substantial agreement with the DAS28, SDAI, and CDAI in discriminating between disease states (κ = 0.70–0.77) and moderate to substantial agreement between response levels (κ = 0.52–0.73). Both mDAS28 and DAS28 measures classified patients similarly in remission compared to the SDAI and CDAI. The mDAS28 was superior in detecting change (standardized response mean = 0.58) followed by the DAS28, CDAI, and SDAI.
Conclusion. The mDAS28 is a valid and sensitive tool to assess disease activity in epidemiological research, as an alternative to the DAS28, when acute-phase reactant values are unavailable.
- DISEASE ACTIVITY SCORE
- DISEASE ACTIVITY SCORE 28 JOINT COUNT
- AMERICAN COLLEGE OF RHEUMATOLOGY
- DISEASE ACTIVITY MEASURES
- EUROPEAN LEAGUE AGAINST RHEUMATISM
- CLINICAL DISEASE ACTIVITY INDEX
- SIMPLIFIED DISEASE ACTIVITY INDEX
The Disease Activity Score with 28 joint count (DAS28)1 is one of the most widely used and validated composite measures of disease activity in rheumatology. It is a modified version of the original Disease Activity Score (DAS) developed by van der Heijde, et al2,3 in 1990. It is regarded by many as the “gold standard” measure in rheumatoid arthritis (RA) and is required by several regulatory bodies when determining patient eligibility for biologic treatments. Among other disease activity instruments, the DAS28, as part of the European League Against Rheumatism (EULAR) response criteria4, has been a reliable measure of treatment efficacy in clinical trials, along with the American College of Rheumatology (ACR) improvement criteria5, and its use has been recommended by EULAR in the clinical management of RA6. In addition, DAS28 has been used as a benchmark for validation of several new composite indices7,8,9,10.
The DAS28 uses a mathematical formula to combine values of 4 of the 7 ACR/EULAR core set measures of disease activity, tender joint count (TJC) based on 28 counts, swollen joint count (SJC) based on 28 counts, patient global health, and an acute-phase reactant, the erythrocyte sedimentation rate (ESR), to produce a continuous score. Several other composite measures have been developed: the Simplified Disease Activity Index (SDAI)11 and the Clinical Disease Activity Index (CDAI)12. Both are computed by a simple summation of a subset of the core measures: TJC, SJC, patient global assessment of disease activity [PGA; visual analog scale (VAS) 0–10 cm], physician global assessment of disease activity (EGA; VAS 0–10 cm), and an acute-phase reactant, C-reactive protein (CRP). The SDAI contains CRP, but the CDAI does not. The ACR-N13, the hybrid measure of ACR improvement criteria, assesses the change in disease activity rather than the current disease activity. Composite measures that include only patient-reported outcomes, such as the patient activity score14 and the RAPID315, have been found to discriminate response in clinical trials.
However, all disease activity measures have some limitations. In settings where laboratory values such as ESR may be unavailable, such as in health services or epidemiological research, the utility of the DAS28 has been limited16. In clinical trials, ESR values are available as mandated by study protocol, allowing computation of the DAS28. But in some practice settings the ESR laboratory test is not ordered routinely, impeding calculation of the DAS28, and in epidemiological research, this causes the omission of patients with missing ESR values from further analysis16. It has been suggested that acute-phase reactants add little to composite disease activity measures12. In addition, ESR has been found to be normal in up to 40% of RA patients with active disease, suggesting that its value as a measure of disease activity may be limited16. Limitations in the DAS28, SDAI, and CDAI suggested in previous research7 include the lack of a patient functional status measure such as the Health Assessment Questionnaire (HAQ), the best predictor of severe outcomes in RA17,18. The ACR-N or ACR improvement criteria measure change of disease activity over time and do not allow assessment of disease activity at one clinical visit.
Based on these limitations and to facilitate calculation of the DAS28 in epidemiological research, the aim of our study was to modify the DAS28 by replacing the acute-phase reactant, resulting in a modified DAS28 (mDAS28), and to assess its comparability with the DAS28 and its validity according to the Outcomes Measures in Rheumatoid Arthritis Clinical Trials (OMERACT) recommendations19,20.
MATERIALS AND METHODS
Subjects were selected from a large North American registry, the Consortium of Rheumatology Researchers of North America (CORRONA)21. The methods of this registry have been described22. Patients eligible for the study had all measures needed to calculate the DAS28. Patients who did not have all these components were excluded from further analysis. Demographic and disease activity measures of excluded and included patients were compared in a separate analysis to determine whether any bias was entered due to sample selection.
The study utilized 3 samples, cross-sectional “development” and “validation” datasets and a longitudinal “validation” dataset (Table 1).
Demographic and clinical characteristics in cross-sectional and longitudinal cohorts. Values are mean (SD) unless otherwise indicated.
The first 2 datasets were from a cohort of 11,307 patients with RA. A cross-section of this cohort was obtained with information from the patient’s most recent visit. This cross-sectional cohort was randomly and evenly divided into “development” and “validation” datasets. The cross-sectional development dataset (n = 5729) was used to build a prediction model to identify statistically significant predictors of the logarithm of ESR (lnESR), and the cross-sectional validation dataset (n = 5578) was used to subsequently validate the mDAS28.
The third dataset, a longitudinal “validation” dataset, was from a cohort of 703 patients with RA who had 2 paired visits. The first visit involved initiation of a disease modifying antirheumatic drug (DMARD); the second visit occurred at least 3 months after the first. This longitudinal cohort was then divided randomly and evenly into “development” (n = 336 pairs) and “validation” datasets (n = 367 pairs). The longitudinal validation dataset was used to evaluate the mDAS28 as a measure of response.
Disease activity measures needed to compute the DAS28 were collected by a rheumatologist. Measures included the modified HAQ (mHAQ) score23, a measure of functional status, the patient visual analog pain score (PAIN), physician global assessment of disease activity (EGA), and duration of morning stiffness. DAS28 values were calculated according to its formula1:
Statistical analysis
To develop the mDAS28, statistically significant predictors of the lnESR were identified in the cross-sectional development cohort, using univariate linear regression analysis. Candidate predictors of lnESR included TJC, SJC, PAIN, EGA, PGA, mHAQ, and duration of morning stiffness. Candidate variables significant at an alpha level of 0.10 were included in a multivariate model. Forward and backward stepwise regression was used to identify the most significant independent variables using a p value of 0.10 as the removal criterion. Multicollinearity between the significant independent variables, specified in the multivariable model, was determined using the variance inflation factor (VIF)26. Variables found to be collinear (VIF > 10) were dropped from the model. Goodness of fit of the model in predicting the dependent variable, lnESR, was assessed by R2 statistic, a measure of the proportion of the variation explained by the regression27. The reliability and validity of the mDAS28 as a measure of disease activity and response was then evaluated in the cross-sectional and longitudinal validation datasets. Internal validity, the extent that items in a score measure the same outcome, was assessed using Cronbach’s alpha28. Criterion validity, the extent that a measure correlates with a “gold standard,” was examined by correlating mDAS28 scores with DAS28, SDAI, and CDAI. Both the simplified composite indices SDAI and CDAI were used as comparators to better assess criterion validity, since one contains an acute-phase reactant (SDAI) and the other does not. Predictive validity, the ability of a measure to predict future outcome of the disease, was examined by correlating the mDAS28 scores with the mHAQ. Both validities were assessed by Spearman rank correlation coefficients29. The amount of agreement between mDAS28 and the other disease activity indices to discriminate between different disease states of individual patients (remission, low, moderate, high) and between good, moderate, or none levels of response based on the EULAR response criteria was examined using weighted kappa statistics30. EULAR response criteria were calculated according to its algorithm (Figure 1). Since new cutpoints were not derived for the mDAS28, modified EULAR (mEULAR) response criteria were calculated according to the EULAR response criteria, using mDAS28 scores and DAS28 cutpoints.
Algorithm to calculate the EULAR Response Criteria using published absolute and change cutpoints. *DAS28 absolute cutpoints. †DAS28 change cutpoints.
SDAI and CDAI response criteria were derived in the same manner as the EULAR response criteria, with the exception that the absolute cutpoints as defined24 and the change cutpoints as defined25 for both measures were used instead of DAS28 cutpoints.
The sensitivity to change or responsiveness of the mDAS28, the ability of a measure to detect important changes over time after a treatment has been initiated, was evaluated by calculating the effect size (ES)30 and standardized response mean (SRM)31. ES was calculated by taking the mean differences of the disease activity scores between the baseline and second study visits (mean change scores) and dividing by the standard deviation of the baseline scores. SRM was calculated by taking the mean change scores and dividing the result by the standard deviation of the change scores. The values of the ES were small with a range of 0.2–0.5, moderate if 0.5–0.8, or large if > 0.830. SRM were interpreted similarly31. Statistical analysis was carried out using Stata version 10.0 (Stata Corp., College Station, TX, USA)32.
RESULTS
A total of 11,307 patients were eligible for the cross-sectional cohort, and 703 pairs of patients with initiation of a DMARD and at least 3 months until the first followup visit were eligible for the longitudinal cohort. Demographics and clinical characteristics for the cross-sectional development and validation dataset were generally similar (Table 1). Both cross-sectional datasets exhibited mild to moderate disease levels but, in the longitudinal validation dataset, disease activity measures had higher values. This would be expected since the patients in the longitudinal dataset were initiators of DMARD. A sensitivity analysis was performed between patients utilized in the study and the subset of patients who did not have sufficient disease data to calculate the DAS28, and their demographic and clinical characteristics were found to be comparable (data not shown).
The modified DAS28 (mDAS28)
In the unadjusted univariate analysis, all 7 candidate predictors (TJC, SJC, PGA, EGA, PAIN, mHAQ, and morning stiffness) were found to significantly predict lnESR. Forward and backward stepwise regression analysis resulted in the same multivariable model with the following significant predictors: TJC, SJC, mHAQ, EGA, and PAIN (Table 2).
Results of forward and backwards stepwise linear regressions.
The PGA and duration of morning stiffness were significant in the unadjusted model but became insignificant when entered into the multivariable model. Upon examination of the functional associations between lnESR and several of the candidate predictors, transformations of TJC and SJC were performed to better fit the assumption of linearity. The multivariable model was refit after transforming the TJC and SJC to their logarithmic forms. A separate model was fit using TJC and SJC in the forms of log of (TJC + 1) and (SJC + 1) due to values of 0. No differences in the amount of variance were explained by these models when compared to the original model. In another series of models, TJC and SJC were both transformed to their square roots and refit in the multivariable model. Again, no difference was found with the amount of variance explained by this model compared to the original model. It was decided to use the model containing the square roots of TJC and SJC since the transformed forms could be combined with the squared forms of TJC and SJC that were already present in the DAS28 formula. Every possible interaction between the variables was also explored, but no significant interactions were found. The final model consisted of the 5 significant predictors of lnESR: TJC, SJC, mHAQ, PAIN, and EGA. The regression equation for the lnESR was as follows:
Imputation of the fitted regression equation of the lnESR into the DAS28 formula in place of the observed lnESR resulted in the following modified version of the DAS28:
Validation of the mDAS28
Measure of disease activity. Distributional properties
The distributions of mDAS28 scores and the DAS28 scores differed, the DAS28 being normally distributed and the mDAS28 exhibiting right-skewness (Figure 2). SDAI and CDAI values were also right-skewed, as reported in other studies33,34.
Distribution properties of composite disease activity indices in the cross-sectional validation cohort (n = 5578).
The means (SD) of the DAS28 and mDAS28 were 3.42 (1.54) and 3.41 (1.38), respectively, in the cross-sectional development dataset and were similar in the 2 validation datasets (Table 1). Upon examination, it was found that both the mDAS28 and the DAS28 were almost identical in detecting remission and low disease activity. Using the DAS28, 894 (16%) patients had scores ≤ 3.2 and > 2.6 (low disease) and 1871 (34%) had scores ≤ 2.6 (remission). When the mDAS28 was used, 915 (17%) patients had scores ≤ 3.2 and > 2.6 and 1939 (35%) had scores ≤ 2.6 (Table 3).
Proportion of patients classified in disease levels using composite indices in the cross-sectional validation cohort (n = 5578). Values are number (%).
When the CDAI was used to classify patients, a larger proportion of patients was classified into low disease and fewer into remission compared to the DAS28 and mDAS28. The proportion of patients classified into remission and low disease activity by the SDAI was similar to that of the CDAI (Table 3).
Internal consistency
The mDAS28 had satisfactory internal consistency (α = 0.71), while the DAS28, SDAI, and CDAI were not as valid internally (α = 0.39, 0.61, 0.60, respectively).
Criterion validity
On a group level, the mDAS28 was strongly correlated with the DAS28, SDAI, and CDAI (r = 0.87, 0.91, 0.96, respectively). All correlations were significant (p < 0.001).
Predictive validity
The mDAS28 was significantly correlated with the mHAQ (r = 0.58, p < 0.001). DAS28, CDAI, and SDAI were also significantly correlated with the mHAQ but not as strongly (r = 0.51, 0.51, 0.51, respectively, p < 0.001). Stronger correlation between mDAS28 and mHAQ would be expected given that the mHAQ is a component of the mDAS28.
Ability to discriminate
To determine the ability of the mDAS28 to classify individual patients by disease level, weighted kappa coefficients were used, and indicated strong agreement between mDAS28 and DAS28 (κ = 0.70). Kappa values > 0.60 indicate a substantial relationship35. There was strong agreement between mDAS28 and CDAI (κ = 0.77) and between mDAS28 and SDAI (κ = 0.71). Similar results were found between DAS28 and CDAI and DAS28 and SDAI (κ = 0.62, 0.63, respectively).
Measure of response to treatment. Ability to discriminate
Substantial agreement between the EULAR and the mEULAR was found in classifying individual patients (κ = 0.74). However, only moderate agreement was found when mEULAR was compared to CDAI response criteria (κ = 0.52) and SDAI response criteria (κ = 0.52). Moderate agreements were also found when the EULAR response criteria were compared with the CDAI (κ = 0.46) and SDAI response criteria (κ = 0.47).
Sensitivity to change
Mean changes in scores of the mDAS28 and DAS28 from the baseline initiation visit to the followup visit were similar (Table 4). The mDAS28 was the most sensitive measure to detect change over time compared to DAS28, CDAI, and SDAI. mDAS28 had moderate ES (0.50) and SRM values (0.58) while DAS28 and CDAI both had moderate SRM values (0.57, 0.52) but small ES values (0.47, 0.45). The SDAI was the weakest measure to detect change, with ES of 0.37 and SRM of 0.45.
Sensitivity to change assessed by effect size (ES) and standardized response mean (SRM). Values are mean (SD) unless otherwise indicated.
DISCUSSION
Our study demonstrates that a modified version of the DAS28 calculated without the ESR, the mDAS28, performs as well as the DAS28 as a measure of both disease activity and response, and could be used as an alternative to the DAS28 in epidemiological research when ESR values are unavailable.
Measures such as DAS28 have been used successfully in clinical trials where the goal was to measure the efficacy of therapies by comparing groups of patients. In our study, the mDAS28 was strongly correlated with DAS28, and also with the SDAI and CDAI on a group level. In addition, compared to the other disease activity indices, mDAS28 had the strongest association with the mHAQ (r = 0.58). This would be expected given that the mHAQ is a component of the mDAS28. Makinen, et al7 noted when developing the Mean Overall Index for Rheumatoid Arthritis (MOI-RA) that one of the limitations of the DAS28 was that it did not contain the HAQ36, considered the best predictor of outcomes in RA18,37,38. In studies comparing the HAQ with the mHAQ, both measures were found to be strongly correlated39 and sensitive to change of treatment40,41,42. Since a measure should have face validity, the addition of the mHAQ as part of the mDAS28 strengthens the overall credibility of the measure.
The mean baseline values of the mDAS28 were almost identical to the mean baseline values of the DAS28 in all 3 cohorts — for cross-sectional development 3.41 (1.38) versus 3.42 (1.57), respectively; for cross-sectional validation 3.40 (1.37) versus 3.41 (1.54); and for longitudinal validation 4.21 (1.41) versus 4.19 (1.59). Classifying proportions of individual patients into the disease states of remission, low, moderate, and high disease activity, the mDAS28 again performed almost identically to the DAS28. The DAS28 classified 16% and 35% of patients into remission and low disease activity, respectively, whereas mDAS28 classified 16% and 34% into remission and low disease activity.
Since measurement tools need to assess individual patients, we examined the agreement of the measures when classifying individual patients according to disease levels, and the mDAS28 agreed strongly with the CDAI, SDAI, and DAS28 (κ = 0.70–0.77), despite the absence of the ESR as a component. The mDAS28 was also compared to the DAS28 for classifying patients according to their level of response using the EULAR response criteria. Strong agreement was found between the mEULAR and EULAR criteria (κ = 0.74). However, only moderate agreement was found when the mEULAR criteria were compared with the CDAI and SDAI response criteria. The EULAR criteria were also moderately in agreement with the CDAI and SDAI response criteria.
The mean changes in scores of the mDAS28 and DAS28 from baseline initiation to the followup visit were similar [Δ = 0.698 (1.20) vs Δ = 0.732 (1.28), respectively]. In addition, the mDAS28 demonstrated similar sensitivity to detect disease activity changes after initiation of a DMARD compared to the DAS28. These results suggest that the mDAS28 is a valid measure of disease activity and could be used to measure disease activity when DAS28 cannot be calculated.
Our effort to replace the logarithm of ESR by identifying significant predictors provides insight into the complexity of disease activity measurement. Although the mDAS28 performs similarly to the DAS28, the complexity of its formula limits its use, especially in a clinical setting, compared to simplified composite measures — the CDAI, SDAI, and MOI-RA. Thus, it may be more suitable for epidemiological research using data from registries rather than in daily routine patient monitoring. This would prevent the exclusion of patients with missing ESR values from epidemiological research studies.
A limitation of our study is the use of only one observational dataset to develop and validate the measure. Additional validations of the mDAS28 should be performed in other populations, such as a clinic trial dataset. Another potential criticism of the study could be that the patients had low to moderate disease activity. Again, additional investigations of the mDAS28 using populations with greater ranges of disease levels including high disease activity should be undertaken.
Our intent to modify the DAS28 by substituting other measures for the ESR was not to diminish the importance of the ESR as a measure of RA disease activity. In effect, we suggest physicians continue to order laboratory measures regularly in the clinic, as the ESR is an important measure of disease activity and longterm outcomes43. Modification of the DAS28 was done to allow a measure comparable to it to be devised for research settings where laboratory values such as the ESR are not available and the DAS28 cannot be calculated.
In this observational study, we developed a modified version of the DAS28 without the ESR value, and then demonstrated that the mDAS28 is comparable to the DAS28 for measuring RA disease activity and response. The mDAS28 was also found to be a valid outcome measure as it fulfilled most of the criteria recommended by the OMERACT initiative. The mDAS28 can be calculated when ESR values are unavailable, preventing patients being excluded in epidemiological research using disease registries. Further testing of the mDAS28 in other patient populations is recommended.
- Accepted for publication May 20, 2010.
