Abstract
Objective. To estimate the cutoff points for the 28-joint Disease Activity Score (DAS28) calculated using C-reactive protein (CRP) measurements from patients with early arthritis.
Methods. We analyzed data from 568 visits of 207 patients enrolled in our prospective longitudinal register of early arthritis. Six rheumatologists evaluated the degree of disease activity at each visit on the basis of the available clinical data, and the final degree of disease activity was established by consensus. DAS28 values were calculated for each visit using CRP or erythrocyte sedimentation rate (ESR). Through a ROC analysis, cutoff points for both indices, as well as for minimal disease activity (MDA), were selected on the basis of the best tradeoff values between sensitivity and specificity.
Results. The cutoff values to classify disease activity with the DAS28-CRP were 2.3, 3.8, and 4.9, considering remission at < 2.3, low disease activity 2.3–3.8, moderate disease activity 3.8–4.9, and high disease activity > 4.9. The cutoff value for MDA when calculated with CRP was 2.6. The area under the ROC curves was always greater for DAS28-CRP than for DAS28-ESR, reaching statistical significance for low/moderate activity and for the MDA.
Conclusion. Our study confirms that the cutoff points for DAS28-CRP are lower than those described for DAS28-ESR, suggesting that DAS28-CRP may be more accurate to assess disease activity in our population.
- DISEASE ACTIVITY SCORE 28
- RHEUMATOID ARTHRITIS
- OUTCOME
- UNDIFFERENTIATED ARTHRITIS
- DISEASE ACTIVITY
- REMISSION
The management of rheumatoid arthritis (RA) has improved greatly in the last 2 decades, and the availability of new drugs, particularly the biological agents, has helped achieve better control of this disorder. However, the development of other clinical tools such as the Disease Activity Score (DAS)1 may also help to improve the control of RA2. Indeed, close followup of treatments in accord with predetermined DAS values may be decisive to improving radiographic outcome, physical function, and quality of life, compared with traditional management strategies3.
While the DAS is a useful tool in clinical trials and observational studies, applying it in daily clinical practice is complicated due to the time required to perform joint counts. As such, there is no clear evidence regarding its efficacy in daily practice. Accordingly, a simplified version of the DAS has been described that uses 28-joint counts and is more feasible to implement, without losing significant information regarding the original score4. Due to concerns regarding the specificity of the erythrocyte sedimentation rate (ESR) as an acute-phase reactant, particularly as it may be influenced by unrelated factors such as age or gender, a C-reactive protein (CRP)-based version of the DAS28 was recently proposed5. The use of CRP levels has different advantages: laboratory tests are faster, CRP is more sensitive to short-term changes in disease activity6, and its measurement can be standardized in a central laboratory for multicenter clinical trials. The main concern is that DAS28 based on CRP (DAS28-CRP) has not been formally validated, and the accepted cutoff points established for DAS28 using the ESR (DAS28-ESR) do not seem to perform well for DAS28-CRP7. Indeed, we recently showed that both indexes are not fully equivalent since the DAS28-ESR tends to provide higher values than DAS28-CRP, especially in women and patients with long disease evolution8.
Therefore, we were interested in generating specific cutoff points for DAS28-CRP based on patient evaluations. We set out to define the cutoff points for DAS28-CRP in a population of patients from our early arthritis register.
MATERIALS AND METHODS
We analyzed data from a prospective longitudinal observational study based on a register that includes all patients attending the early arthritis clinic (EAC) at our center. Our catchment area covers a population of 500,000 inhabitants, of whom more than 90% are covered by public health insurance. In addition, all primary care physicians in the area are aware of the EAC and how to refer patients to it.
To be referred to the clinic, patients must have had 2 or more swollen joints for at least 4 weeks and symptoms for less than a year. Patients diagnosed with gouty arthritis, septic or viral arthritis, osteoarthritis, spondyloarthropathies, or connective tissue diseases during the followup period were excluded from the study. Thus, we included only data from patients that fulfilled American College of Rheumatology (ACR) criteria for the diagnosis of RA9 or patients with chronic undifferentiated arthritis. The protocol for the register included 4 visits during a followup period of 2 years and it was reviewed and approved by the local research ethics committee. Prior to entry into the register, all patients signed a written informed consent form.
Our study focused on the visits that took place between September 2001 and June 2006. At each visit, the following data were collected according to an established protocol and entered into an electronic database: clinical and demographic information; disease duration at the beginning of followup; data on treatment with disease modifying antirheumatic drugs (DMARD) and steroids; 28-joint count of tender (TJC) and swollen joints (SJC); global disease activity on a 100 mm visual analog scale assessed both by the patient (GDAP) and the physician (GDAPh); and the Spanish version of the Health Assessment Questionnaire (HAQ)10 and basic laboratory tests including the evaluation of the ESR (Westergren method) and CRP (nephelometry). The DAS28 indices, both with ESR and CRP, were calculated as described (http://www.das-score.nl):
Disease activity assessment
Six rheumatologists were involved in assessing disease activity at each visit: 4 senior rheumatologists and 2 fellows from our department (3 of whom were female); the median experience in assessing RA patients was 13.2 years (range 2–24 yrs). The degree of disease activity was classified as remission, low activity, moderate activity, or high activity at each visit, based on TJC, SJC, GDAP, HAQ, ESR, and CRP. Because assessment was retrospective, the physicians could not physically examine the patients. In addition, they were blind to the GDAPh obtained at each visit. It is noteworthy that there was total agreement between these rheumatologists for 476 visits, while in the remaining 92 visits, the disease activity status was established by consensus of the evaluators. As such, the patients were considered to be in remission at 104 visits, while low, moderate, or high disease activities were considered in 289, 126, and 49 visits, respectively. We analyzed variables that influence disagreement at some visits using logistic regression.
Statistical analysis
We analyzed interobserver agreement of disease activity evaluation using the kappa command of Stata 9.2® for Windows (StataCorp LP, College Station, TX, USA). Two logistic regression models were applied to assess independent variables related to the level of disagreement. The first model included level of disease activity assessed by the rheumatologists and the second model excluded this variable.
A receiver operating characteristic (ROC) analysis was performed on the data using the “roctab” command of Stata. Each cutoff point was selected on the basis of the best tradeoff values between sensitivity and specificity. ROC curves were also obtained with the roctab command of Stata, using the “graph” option. To estimate whether differences in the area under the curve (AUC) between indices were statistically significant, we used the Stata “roccomp” command that provides a test for the equality of the AUC using an algorithm described by DeLong, et al12. Statistical significance was accepted if the p value was less than 0.05.
The same procedure was followed to obtain the cutoff value to consider MDA for both DAS28 indices.
RESULTS
Patient characteristics
A total of 568 visits by 207 patients enrolled in the register were analyzed in this study, 76.4% of whom were female. The mean age at the onset of the disease was 51 ± 16 years and the median disease duration at the first visit was 6 months (interquartile range 3.6–9). A more detailed description of this population has been published8.
Agreement on evaluation of disease activity
The kappa index of the information recorded by the 6 rheumatologists indicated that the best agreement between physicians was observed for the state of remission (κ = 0.65), followed by that of high activity (κ = 0.52). However, there was only moderate concordance in the intermediate degrees of disease activity (κ = 0.30–0.41).
These values reflect the variability between physicians in their perception of disease activity, as evident in Table 1 and as confirmed with a multivariate logistic regression model where disagreement was clearly more striking for the moderate level of disease activity (Table 2, model 1). Moreover, TJC and HAQ contributed to the disagreement irrespective of the moderate level of disease activity (Table 2, model 2). Intriguingly, the characteristics (gender, age, years of experience) of the evaluators did not contribute to the disagreement in the perception of disease activity, suggesting that highly individual variability exists.
Estimation of the cutoff points for DAS28-CRP
The best threshold values of the DAS28-CRP to stratify the patients in our population according to the state of their disease (in remission, low, moderate, or high disease activity) were 2.3 [sensitivity (Se) 87%; specificity (Sp) 96%], 3.8 (Se 78%; Sp 88%), and 4.9 (Se 84%; Sp 83%). These differed from the cutoff points obtained for DAS28-ESR of 2.7, 4.3, and 5.5, respectively. We analyzed the ROC curves for DAS28-ESR and DAS28-CRP in the patients considered in remission/low activity, low/moderate activity, and moderate/high activity to determine which score best evaluates disease activity. The areas under the ROC curves were always greater for DAS28-CRP than for DAS28-ESR at each level of disease activity, although statistically these differences were only significant for the low/moderate activity group (Figure 1).
In addition, we estimated a set of cutoff points for DAS28-CRP according to the classification of disease activity that was based on the conventional DAS28-ESR cutoff points, as described7. Although this strategy does not allow us to estimate which index is more accurate, we obtained the following cutoff values: 2.35 for remission/low activity (Se 90%; Sp 92%), 2.95 for low/moderate activity (Se 91.5%; Sp 90.5%), and 4.35 for moderate/high activity (Se 93%; Sp 96%).
Estimation of the MDA cutoff point for DAS28-CRP and DAS28-ESR
A more practical approach for daily clinical practice could be to use the MDA rather than the 4 levels of disease activity. The best MDA threshold estimated for DAS28-ESR was 2.8 (Se 86%; Sp 83%), and for DAS28-CRP 2.6 (Se 85%; Sp 89%). The area under the ROC curve was again significantly higher for DAS28-CRP versus DAS28-ESR (Figure 2), suggesting that, to assess MDA, the DAS28-CRP cutoff point is more accurate.
DISCUSSION
The DAS28-CRP was developed on the basis that CRP is a more reliable acute-phase reactant than ESR6; therefore CRP should be more useful to evaluate disease activity in patients with RA. When the formula to calculate DAS28-CRP was first described, it appeared to correlate well with DAS28-ESR (http://www.das-score.nl), although it has only recently been validated with respect to functional disability and radiographic progression in patients with RA13. Despite this validation, the authors suggested that it might be necessary to derive a new set of cutoff points for DAS28-CRP to increase the agreement between DAS28-CRP and DAS28-ESR13.
We show that cutoff points estimated specifically for DAS28-CRP in an early arthritis population are lower than those used classically for DAS28-ESR. This is the case when we calculated the DAS28-CRP cutoff points using the classic gold standard proposed for DAS28-ESR4, as well as when we estimated new sets of cutoff points for both these indices using our evaluation of disease activity as the gold standard. This latter strategy was adopted mainly for 2 reasons: (1) to be able to compare “the accuracy of DAS28-ESR versus DAS28-CRP”; and (2) because the classic cutoff points calculated for DAS28-ESR are derived, through a mathematical transformation, from those estimated for the original DAS.
The original DAS was developed on the basis of prospective data in patients with recent-onset RA untreated at baseline1. Active disease was defined as a need to start or to modify DMARD therapy. Not initiating or modifying DMARD therapy over a 1-year period, or discontinuing DMARD due to disease remission, defined minimally active disease. By contrast, our gold standard to define levels of disease activity was the physicians’ assessment of the clinical data recorded in the database. Despite these differences, our estimated cutoff points for DAS28-ESR are quite similar to those calculated for DAS28-ESR from the original DAS. Further, they are even closer to those described by Aletaha, et al14, who also used expert assessment of patient files.
The second finding from our study is that DAS28-CRP seems more accurate than DAS28-ESR to determine RA activity in our population. This finding might be biased by our confidence in CRP versus ESR. However, we also replicated this observation through independent and validated criteria for MDA, as reported11. This outcome measurement was proposed because assessing patients with low and moderate activity is very difficult, and there is currently no precise definition of RA remission15. Therefore, Quinn, et al proposed that current RA treatment should aim to achieve MDA16. Interestingly, our best MDA thresholds for both DAS28-ESR and DAS28-CRP were very close to their respective cutoff values for remission, suggesting that MDA and clinical remission are very similar concepts.
A possible limitation of our study is that we considered patients fulfilling ACR criteria for RA as well as those with undifferentiated arthritis. Although this might introduce bias, we demonstrated previously that both DAS28-ESR and DAS28-CRP behave similarly in both these subgroups of patient8. Alternatively, our study might be biased by the fact that all the evaluators belonged to the same department. However, there was some disagreement between physicians, especially in the moderate level of disease activity. Hence, individual variation in the perception of intermediate levels of disease activity is likely to exist.
Our work supports previous studies suggesting the cutoff points for DAS28-CRP and DAS28-ESR are distinct. In addition, our data suggest that DAS28-CRP is more accurate. On the other hand, we consider that further studies are necessary to confirm whether these cutoff points for DAS28-CRP are heterogeneous in different populations. Lastly, if DAS28-CRP is confirmed to be more accurate than DAS28-ESR in other populations, specific DAS28-CRP EULAR response criteria should be described with ad-hoc cutoff values and magnitudes of improvement.
Acknowledgments
We thank Loreto Carmona, MD, PhD, for the critical reading of the manuscript; and Jose M. Álvaro Gracia, MD; Inmaculada Carvajal, MD; Rosario García-Vicuña, MD, PhD; Alicia Humbría, MD, PhD; Juan P. López Bote, MD, PhD; Pedro Sabando, MD, PhD; Eva Tomero, MD; and Esther Vicente, MD, PhD, for their continuous support with the early arthritis register.
Footnotes
-
Supported by the Instituto de Salud Carlos III (grant numbers FIS 05/2044 and 05/2041 to Dr. García-Vadillo). Dr. González-Álvaro’s work is funded in part by a grant from the Instituto de Salud Carlos III (Intensificación de la Labor de Investigación) to promote research in the Spanish National Health Service. Supported in part by RETICS Program, RD08/0075 (RIER) from Instituto de Salud Carlos III (ISCIII).
- Accepted for publication February 23, 2010.