Abstract
Objective. We analyzed whether a patient self-report remission criterion, such as that according to the Rheumatoid Arthritis Disease Activity Index-5 (RADAI-5), meets the criteria of the 2011 proposed American College of Rheumatology/European League Against Rheumatism (ACR/EULAR) definition of remission.
Methods. The 2 approaches of the ACR/EULAR proposal [Boolean- and Simplified Disease Activity Index (SDAI)-based] as well as the RADAI-5 were used to assess whether patients with RA are in remission. Sensitivity, specificity, positive and negative predictive values (PPV, NPV), and kappa analyses were performed to illustrate the relationship among the different approaches defining remission at a group level.
Results. In total, 705 patients' assessments were included. Eighty-nine patients were classified as being in remission according to the Boolean-based and 169 according to the SDAI-based definition of the ACR/EULAR proposals, and 154 according to the RADAI-5. Sixty-eight assessments were classified as being in remission according to all 3 definitions. In the case of RADAI-5 remission, sensitivity was 78%, specificity 86%, PPV 45%, and NPV 96%, indicating remission according to the Boolean-based definition; and 60%, 92%, 66%, and 90%, respectively, indicating remission according to the SDAI-based definition. In the case of remission according to the SDAI-based ACR/EULAR definition, sensitivity was 52%, specificity 100%, PPV 98%, and NPV 87%, also indicating remission according to the Boolean definition; while according to the Boolean definition the values were 98%, 87%, 52%, and 100%, respectively. Kappa statistics showed fair to good agreement for all 3 definitions.
Conclusion. Nearly twice as many assessments were classified as being in remission using the SDAI-based or the RADAI-5 definitions when compared to the Boolean-based definition. Remission according to the RADAI-5 also was highly specific for both ACR/EULAR criteria. Sensitivity for the RADAI-5 criterion was even better for the Boolean-based definition than that for the SDAI-based definition.
The American College of Rheumatology (ACR) and the European League Against Rheumatism (EULAR) recently provisionally defined remission in rheumatoid arthritis (RA) for clinical trials1. Two propositions were put forward: the Boolean-based definition with ≤ 1 tender joint (TJ), ≤ 1 swollen joint (SJ), patient's global assessment (PATGA) on a 10-cm visual analog scale (VAS; range 0.0–10.0) ≤ 1.0, and C-reactive protein (CRP) ≤ 1.0 mg/dl; and the index-based definition using a Simplified Disease Activity Index (SDAI) ≤ 3.3. Although 2 proposals were stated, the original report does not suggest that these 2 definitions would identify identical patients.
A patient self-report RA assessment tool, the Rheumatoid Arthritis Disease Activity Index, comprising 5 questions (RADAI-5) to be answered using an 11-part numerical scale, was introduced in 20082. This questionnaire is a simplified and shortened revision of the original RADAI3 and shows significant correlation with commonly used composite indices such as the Disease Activity Score, comprising a 28-joint count (DAS28)4, the SDAI5, and the Clinical Disease Activity Index (CDAI)6, although it is completed exclusively by the patient and does not require a joint count2. These correlations were confirmed recently in a French population, providing evidence that the RADAI-5 constitutes a suitable tool for detecting flares7.
The RADAI-5 activity categories are as follows: 0.0 up to 1.4 for a remission-like state, 1.6 to 3.0 for low disease activity, 3.2 to 5.4 for moderate disease activity, and 5.6 to 10.0 for high disease activity8. DAS28 and CDAI levels, as well as tender and swollen joint counts, physician's global assessment, and erythrocyte sedimentation rate (ESR), proved to be significantly different within the different RADAI-5 categories8.
We assessed whether the RADAI-5 remission criterion meets the proposed 2011 ACR/EULAR remission definitions.
MATERIALS AND METHODS
Patients
Seven hundred five outpatients with RA, all meeting the 1987 American Rheumatism Association (ARA) classification criteria9, were enrolled in our study with their informed consent. Assessment of their last visit was carried out for this study. Five hundred thirty-five patients (75.9%) were female, 170 male, the mean age was 62.7 years (± 13.4 SD; minimum 17 to maximum 89 yrs), and 54.4% were rheumatoid factor (RF)-positive. The mean disease duration was 97.3 months (± 98.0; range 1–749 mo), 89% of the patients were taking various disease-modifying antirheumatic drugs including biologic agents, and 31% were taking additional glucocorticoid therapy (2.5–12.5 mg prednisolone equivalent per day). All patients were treated with nonsteroidal antiinflammatory drugs, at least on demand. Patient demographic data are presented in Table 1.
Patients' characteristics. Data are mean ± SD (minimum-maximum), unless indicated otherwise.
Assessments
Assessments included a 28-joint count for TJ and SJ, evaluation of PATGA (determined by the medical assessor using a 100-mm VAS), first-hour ESR, CRP in mg/dl, and the physician's global assessment (MDGA) to calculate the SDAI as described5, and also to identify patients in remission according to the 2011 proposed ACR/EULAR remission definition. The DAS28 was calculated using an electronic device to classify patients' disease activity according to this commonly used RA disease activity measure4. Patient's pain on a 100-mm VAS (VASpain) was recorded for each visit by a healthcare professional to obtain another patient-relevant measure and one of the core set measures10 to compare patient characteristics in the different remission groups. A subgroup of patients was asked to complete the Health Assessment Questionnaire Disability Index (HAQ-DI)11,12 to obtain a measure of function and an outcome measure of RA including work disability and mortality13. All patients additionally had to complete the RADAI-5 during the waiting time for each visit. A short introduction on how to tackle the RADAI-5 was given by a nurse, if requested by the patient, without influencing the completion of the form.
Comparison of disease activity indices
The Spearman correlation coefficient for pairwise comparisons of the RADAI-5 and DAS28 as well as SDAI was calculated. A value of 1 indicates perfect agreement, a value of 0 indicates that agreement is not better than chance. Kappa values > 0.60 are commonly regarded as indicating substantial agreement14.
Comparison of patients' core set measures and HAQ-DI with the different groups defined by Boolean, SDAI, and RADAI-5-based remission
Patients' core set measures (TJ, SJ, PATGA, MDGA, ESR, CRP, and VASpain) as well as HAQ-DI scores were compared with the different remission groups, calculating the mean and SD. An unpaired t test was performed to identify significant differences between these measures in the different groups. A value for p < 0.05 was defined as significant.
Calculation of the relation of RADAI-5 remission criterion with the ACR/EULAR proposed definition of remission and of both remission definitions of the ACR/EULAR remission proposal
All assessments fulfilling the 2011 ACR/EULAR remission proposal according to the Boolean-based definition were respectively marked and tabulated. The same procedure was done to identify patients according to the SDAI-based definition. To identify the assessments fulfilling RADAI-5 remission criterion, all assessments ≤ 1.4 according to this index were marked and tabulated with their respective SDAI and Boolean criteria. Sensitivity, specificity, and positive and negative predictive values (PPV, NPV) fulfilling the 2011 ACR/EULAR remission proposals (based on the Boolean and SDAI definitions) were calculated for the RADAI-5 remission criterion. Cohen's kappa value was calculated to illustrate the relationship and estimate their agreement. Guidelines characterize values > 0.75 as excellent, 0.40–0.75 as fair to good, and < 0.40 as poor15. The same procedure was followed to elucidate the association with both ACR/EULAR approaches.
RESULTS
Patients' characteristics according to disease activity
The mean DAS28 was 3.31 (± 1.37 SD; range 0.38–8.09), the mean SDAI 10.4 (± 10.2; range 0.0–65.9), and the mean RADAI-5 3.4 (± 2.2; range 0.0–10.0), indicating moderate disease activity according to the DAS28 and to the RADAI-5 and low to moderate disease activity according to the SDAI. All 3 indices correlated significantly with each other and showed substantial agreement (Spearman's rho between 0.655 and 0.882, all p < 0.01; Table 2).
Patients' characteristics according to disease activity and correlation of the single instruments for disease activity measurement.
In total, 89 assessments fulfilled remission criteria according to the Boolean-based definition, 169 assessments according to the SDAI-based definition, and 154 assessments according to RADAI-5-defined remission (Figure 1). Of these assessments, 68 fulfilled all 3 definitions of remission. Just 1 assessment fulfilled only the Boolean-based definition, whereas 51 fulfilled only RADAI-5 definition and 48 only the index-based definition (Figure 1).
Number of assessments in remission according to the 3 criteria (Boolean-based, SDAI-based, and RADAI-5) and overlap among them. SDAI: Simplified Disease Activity Index; RADAI-5: Rheumatoid Arthritis Disease Activity Index comprising 5 questions.
Relation of core set measures and HAQ-DI in the different remission groups
No significant difference was found comparing TJ, SJ, MDGA, ESR, and CRP measures in the 2 remission groups according to the newly proposed ACR/EULAR definition (Table 3a, 3b).
Characteristics of patients according to the different definitions of remission. Data are mean ± SD (minimum-maximum).
Disparity of the core set measures and the Health Assessment Questionnaire-Disability Index (HAQ-DI)† in the different remission groups (unpaired t test).
Patients in RADAI-5 remission showed significant differences from both ACR/EULAR definitions in almost all core set measures, except for ESR in the case of the Boolean-based definition, and PATGA and VASpain in the SDAI-based definition (Table 3a, 3b).
HAQ-DI was collected in 61% of the patients in Boolean-based, in 65% of the patients in SDAI-based, and in 67% of the patients in RADAI-5-based remission. In order to confirm reliability despite this incomplete dataset, we also calculated the proportion of patients with HAQ-DI score in the particular overlapping group of those in remission due to all 3 remission criteria sets among all patients, with a result of 60%. No significant difference was found comparing the HAQ-DI in the 3 distinct remission groups (Table 3a, 3b).
Sensitivity, specificity, PPV, and NPV of each definition to predict another definition
The RADAI-5 remission criteria showed a sensitivity of 78% (95% CI 68%-88%), specificity of 86% (95% CI 83%-89%), PPV 45% (95% CI 37%-53%), and NPV 96% (95% CI 95%-98%) equally fulfilling remission criteria according to the Boolean-based definition of the 2011 ACR/EULAR remission proposal. Using the SDAI-based definition as a reference, the RADAI-5 remission criterion shows a sensitivity of 60% (95% CI 53%-68%), specificity of 92% (95% CI 89%-94%), PPV 66% (95% CI 58%-74%), and NPV 90% (95% CI 87%-92%) to predict remission according to this definition (Table 4a, 4b).
Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of disease activity indices according to the 2011 ACR/EULAR proposed remission criteria. Reference is the Boolean-based proposal.
Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of disease activity indices according to the 2011 ACR/EULAR proposed remission criteria. Reference is the SDAI-based proposal (SDAI ≤ 3.3).
Comparing both approaches to define remission according to the 2011 ACR/EULAR criteria, an SDAI ≤ 3.3 has a sensitivity of 98% (95% CI 92%-100%), specificity of 87% (95% CI 84%-89%), PPV 52% (95% CI 44%-59%), and NPV 100% (95% CI 99%-100%) to the Boolean approach, whereas the Boolean criteria have a sensitivity of 52% (95% CI 44%-59%), specificity of 100% (95% CI 99%-100%), PPV 98% (95% CI 92%-100%), and NPV 87% (95% CI 84%-89%) to the SDAI-based definition (Table 4a, 4b).
For all the 3 different remission criteria sets, kappa statistics show fair to good agreement (κ = 0.52 for RADAI-5- and SDAI-based definition, κ = 0.49 for RADAI-5- and Boolean-based definition, and κ = 0.61 between both ACR/EULAR remission criteria (all p < 0.001).
DISCUSSION
The SDAI-based definition and the RADAI-5 criterion for remission have the same specificity and nearly the same PPV and NPV to also indicate remission according to the Boolean criteria. The SDAI-based definition is more sensitive than the RADAI-5 criterion in this respect. When the SDAI-based definition is taken as a point of reference, the Boolean criteria have a higher PPV than the RADAI-5, also fulfilling the SDAI-based proposal, although the RADAI-5 criterion, with a specificity of 92% and NPV of 90%, is also remarkable, indicating that an assessment not fulfilling RADAI-5 remission is also not in remission according to that criterion. The RADAI-5 is more sensitive than the Boolean criterion in this respect.
Regarding the 2 references for comparison with the RADAI-5, our findings are in accord with other investigations16,17: there was an overlap of 87 assessments. The SDAI-based definition results in 82 more remission assessments than the Boolean-based definition. This indicates that the Boolean definition is included in the index-based approach, which is also expressed by nearly 100% sensitivity and NPV of 100% when the Boolean-based proposal is taken as a point of reference.
PATGA and VASpain, 2 patient-dependent measures that are assessed by the physician in our department, were significantly different among the patients in these 2 groups. This was previously demonstrated for PATGA18. VASpain is not included in the 2 definitions, but composite indices such as the SDAI are highly influenced by the patient's pain assessment19,20,21,22,23, and it has been shown that remission according to the Boolean approach is difficult to achieve when comorbidities associated with pain are present24. The results presented in Table 3a and 3b show no disparities in the core set measures TJ, SJ, MDGA, ESR, and CRP of the 2 groups in remission as defined by the ACR/EULAR proposal. This comes as no surprise, as these definitions use the same core set measures as listed except for ESR. Patients assessed as being in remission according to the RADAI-5 criterion shows different results in almost the entire core set of measures compared to patients fulfilling remission requirements according to the ACR/EULAR proposal, as discussed below.
To our knowledge this is the first attempt to compare remission on the basis of the recently proposed definition by an ACR/EULAR expert committee with a patient-reported disease activity assessment tool omitting a formal joint count. Conveniently, patient-reported assessment is important in defining remission as it focuses primarily on patients' answers to questions about general health and well-being without formal joint counts or blood tests. Why have we chosen the RADAI-5 and not the perhaps more popular Routine Assessment of Patient Index Data-3 (RAPID3)25 as the patient-report disease activity measurement tool? The RADAI-5 was developed in our department (at about the same time as RAPID3) from the original RADAI as a tool that was simpler to calculate than its parent instrument, and it performs as well as the RAPID326, but in our view it is even simpler to calculate8. The RADAI-5 is a 5-item questionnaire covering the arthritis activity over the last 6 months, the current arthritis activity with respect to joint tenderness and swelling, the current severity of arthritis pain, a description of the patient's general health, and the previous morning's stiffness2. The patient her/himself ticks answers to the questionnaire on an 11-point numerical scale without being influenced by the investigator, medical doctor, or nurse. The ACR/EULAR criteria comprise 5 items out of 7 core set measures10 used to elaborate the ACR/EULAR remission proposal, but only 2 are directly influenced by the patient: the PATGA and joint tenderness.
The overall result of our investigation is encouraging: mainly that the specificity of the RADAI-5 remission criterion for both proposed remission criteria is remarkable, indicating that a patient not being in remission according to the RADAI-5 also cannot be classified as being in remission according to the newly proposed ACR/EULAR criteria. At a group level, the RADAI-5 remission definition can also be compared with the SDAI approach because it comprises as many assessments as rated by the SDAI — although it only partly includes the same assessments (60.4%) — and has nearly the same PPV and NPV as the SDAI-defined remission to also indicate being in remission according to the Boolean approach. This is not surprising because a prerequisite for generating the RADAI-5 remission criterion is that patients should also be in remission according to the CDAI6. The CDAI can be seen as comparable to or even more stringent than the SDAI27.
Because the 2 approaches to assess remission, the RADAI-5 and the ACR/EULAR remission criteria, are different, all remission criteria have to be judged differently. The Boolean approach with 89 assessments appears to be the most stringent in the definition of this target. These assessments are included in roughly 80% of their entirety by the other 2 approaches. Eighty-five more assessments according to RADAI-5, compared with 82 more for the SDAI-based definition (34 were equal), were classified as being in remission status (Figure 1). Comparing both definitions according to the ACR/EULAR proposal, the same core set measures are used in both approaches, with the significant difference that the Boolean criteria only allow each core set measure to a limited extent, whereas the SDAI-based definition is characterized by the limitation of the sum of the same measures. However, the PPV of the SDAI-based definition at 52% is comparable to that of the RADAI-5 criterion predicting the Boolean proposal.
What does remission mean? Different percentages of patients in remission can be found in the literature depending on its definition28. In our investigation, 12.6% of the patients' assessments satisfied the Boolean-based definition, which is in agreement with the 8.6% of patients in ARA remission receiving usual care in 24 countries29; 21.8% of the assessments were in remission according to the RADAI-5 definition; and 24.0% to the SDAI-based definition. Remission defined by the ARA is very restrictive and hard to achieve but with the definition of 5 out of 6 measures (TJ, SJ, ESR, joint pain, morning stiffness, and fatigue), the patient shows no reliable clinical signs of illness29. This definition showed 72% sensitivity for clinical remission and 100% specificity compared with active disease. One notable point is that there is a substantial difference in the recently proposed ACR/EULAR remission criteria: according to the ARA criteria the condition of remission should be stable for 2 months; the newly proposed criteria require stability at only 1 point in time, whereas the RADAI-5 remission criterion integrates a 6-month period of disease inactivity by the first question2.
Generally, remission of a chronic or incurable disease is defined as “the state of absence of disease activity in patients known to have incurable chronic illness”30. Accepting some minor disease activity, as in the newly proposed ACR/EULAR Boolean-based definition, with 1 SJ, 1 TJ, and CRP of 1.0, does not comply with this definition. The possible 3 SJ or 3 TJ or CRP up to 3.3 mg/dl in SDAI remission also does not satisfy this definition. According to RADAI-5, patients in a remission-like state had a mean of 0.48 TJ (minimum 0 to maximum 7), a mean of 0.92 SJ (0–9), a mean ESR of 17.5 (1–81), a mean PATGA of 9.4 mm (0–56), and a mean MDGA of 3.72 mm (0–40), thus also not always fulfilling this common definition. Calculating the RADAI-5 thresholds of patients of a daily routine care unit, one prerequisite for classification as being in a remission-like state was that these patients had to express a satisfaction level of 1 (excellent) with their disease activity status8. In a chronic and progressive disease, we must differentiate between patients' and physicians' perspectives on what is controlled by therapy, but not cured31,32. It would be interesting to investigate patients with early arthritis defined by the recently proposed diagnostic criteria for RA33, using the RADAI-5, because remission in our patients has to be seen within the context that most of them have longstanding disease with a mean duration > 8 years, some of them even fulfilling Steinbrocker functional stage 3 or 434, and never being in remission as defined by RADAI-5. An MDGA from 0 up to 40 mm in the RADAI-5 remission group in our investigation also indicates that sometimes the physician was not in agreement with the patient's assessment19,31, but generally a mean MDGA of 3.7 mm indicates good agreement between the physician's judgment and the patient's opinion about being in remission. MDGA is influenced by SJ count, whereas PATGA is influenced most extensively by pain, which is not included in the Boolean-based or the SDAI-based definitions32.
Although HAQ-DI scores were collected in just 61%–67% of respondents in the respective remission groups, this is supported by the same proportion in the overlap group (60%), and it is notable that the HAQ-DI as a major longterm outcome measure did not differ statistically in the 3 remission groups. In calculations of the mean, patients in remission by the Boolean-based approach showed the lowest scores but patients with the SDAI approach showed the highest scores, although not statistically different (p = 0.09). However, since these were not evaluated in all patients, these results must be interpreted with caution.
Treating to target results in a better outcome for the patient35. An overarching principle of this approach is that the treatment of RA must be based on a shared decision between patient and rheumatologist36. In asking patients about their disease activity, the RADAI-5 is an appropriate means to determine the patient's perspective of the disease process and it could have potential to document the patient's judgment of the disease activity. Using the RADAI-5 as well as CDAI or SDAI in our department makes the discussion about treatment decisions easier and involves the patient in this process. It was therefore of particular interest to look at the feasibility of this score for the newly proposed ACR/EULAR remission criteria.
The Boolean proposal was more stringent than the SDAI-based ACR/EULAR remission criteria as well as the RADAI-5 criteria. The RADAI-5 had a weaker PPV than the SDAI-based approach, but both remission criteria had a high NPV when the Boolean-based proposal was used as the point of reference. When the SDAI-based approach was taken as the point of reference, the Boolean approach had nearly 100% PPV and nearly 90% NPV, while the RADAI-5 had a weaker PPV but the same NPV, indicating remission. The RADAI-5 remission criterion had a high specificity according to both ACR/EULAR criteria for remission. It was even more sensitive for the SDAI-based remission criteria than the Boolean approach. Kappa statistics showed fair to good agreement for both the ACR/EULAR remission definitions. It is important to know the feasibility of a patient self-report disease activity assessment tool in order to define remission in RA based on the new ACR/EULAR guidelines. The RADAI-5 allows better participation in the care of patients with RA, especially by nonspecialists such as primary care physicians, without requiring training in formal joint counts. In this regard it might also be useful for the assessment when a patient not satisfying the RADAI-5 remission criterion also does not satisfy the ACR/EULAR remission criteria. These results could support the use of the RADAI-5 as a patient self-report disease activity measure in clinical trials and particularly in daily clinical routine.
ACKNOWLEDGMENT
We thank DGKS Gerlinde Ramharter, DGKS Brigitte Binder, DGKS Elisabeth Hagmann, and Monika Weiskirchner for distributing the RADAI-5 questionnaire to patients and instructing patients in how to complete it without influencing their answers. We also thank the team of Cambridge-Editing for editing the manuscript.
- Accepted for publication November 27, 2012.
