The past decade has seen extensive advances in knowledge regarding the epidemiology, pathophysiology and treatment of rheumatoid arthritis (RA). Early aggressive treatment of RA with disease-modifying antirheumatic drugs (DMARDs) and biological agents: (a) decreases the development of radiographic erosions;1 2 (b) decreases disease-related disability;3^–5 and (c) increases the rate of disease remission.6 7 The opportunity to impact the natural history of RA is most pronounced in the first months after symptom onset.

The current American College of Rheumatology (ACR, previously American Rheumatism Association) criteria were developed initially in 1957 by a subcommittee of five expert rheumatologists with a goal of (a) appropriately classifying patients with RA, and (b) providing a homogeneous population for inclusion into clinical studies and trials.8 9 The 1957 criteria classified subjects into four groups: (a) possible RA; (b) probable RA; (c) definite RA; and (d) in 1958, a fourth category, classical RA was added.9 In 1987 the criteria were revised, with simplification of the description of “definite RA” to those individuals with four of seven criteria (table 1). These revised criteria were developed through the examination of subjects with long-standing RA (mean duration of symptoms of 7.7 years)10 and are the current standard for the selection of subjects for clinical research studies and trials.

Over the past 10 years, more sensitive and specific laboratory tests for the identification of RA have been developed, specifically, antibodies to citrullinated proteins. These autoantibodies, directed against citrullinated residues of proteins, including keratin and filaggrin, are present in the blood of the majority of subjects with RA with established disease and their presence is associated with more severe, erosive disease.11^–20 Utilising this new knowledge, a better test for the detection of antibodies to cyclic citrullinated peptide (CCP) was developed.18 In early inflammatory arthritis, the specificity of anti-CCP antibody for RA has ranged from 90 to 97%, compared with rheumatoid factor (RF), which ranges from 80 to 90%.17 21^–23 While the sensitivity of anti-CCP antibodies in early RA is comparable with that of RF, the detection of either RF or anti-CCP has significantly improved sensitivity for the diagnosis of RA compared with either laboratory test alone.22

The 1987 ACR criteria have a sensitivity of 91–94% and specificity of 89% for the classification of established RA.24 They include characteristics that are rare in new onset RA, such as radiographic erosive changes and rheumatoid nodules. Studies have shown that the 1987 ACR criteria are suboptimal when identifying subjects with early RA (employing a range of arthritis symptom duration from 4 weeks to 2 years), with sensitivity ranging from 40 to 90% and specificity from 50 to 90%.24^–28 With mounting evidence of the importance of early diagnosis and aggressive treatment, there is a need for criteria with the ability to appropriately diagnose and classify subjects with early and established RA.29

We hypothesised that the inclusion of anti-CCP antibody results into the existing 1987 ACR Criteria for the Classification of RA would increase their sensitivity, and that further adjustments to the existing criteria would improve, particularly upon the identification of individuals with early RA. We evaluated the performance characteristics of newly revised sets of classification criteria for RA.

MATERIALS AND METHODS

Subject identification

We identified all subjects evaluated by staff rheumatologists at the Brigham and Women’s Hospital Arthritis Center between 1 January and 30 June 2004 who had serum anti-CCP and RF measured on the same day. Subjects diagnosed with juvenile RA or juvenile idiopathic arthritis with age of onset at 16 or younger were excluded.

Laboratory measurements

Anti-CCP and RF measurements were performed in the hospital clinical immunology laboratory. Second generation anti-CCP titres were measured using an enzyme-linked immunosorbent assay from the Axis-Shield Corporation (Dundee, Scotland, UK). RF titres were determined by nephelometry.22 The cut-off for positivity was 5 U/ml for anti-CCP and 10 IU/ml for RF.

Data collection

All data available in the Brigham and Women’s electronic medical record, including office notes, referral letters, laboratory results and diagnostic reports, were reviewed. The records were independently reviewed by four rheumatology reviewers (KB, KC, KL, PS) in 2006 and 2007. The date of the RF and anti-CCP request was defined as the baseline date, and the duration of arthritis symptoms up to the baseline date were recorded. Each subject’s medical record at the baseline visit was also evaluated for the presence of components of the 1987 Revised Criteria for the Classification of RA10 (table 1). The first four were noted as positive if they had been present for at least 6 weeks at the time of the initial visit when anti-CCP and RF tests were ordered. We also collected additional information including:

1. results of serum anti-CCP test (second-generation anti-CCP titre >5 U/ml = positive);

2. date of onset of arthritis symptoms;

3. arthritis symptom duration (months);

4. the rheumatologist’s diagnosis at the baseline visit;

5. the rheumatologist’s diagnosis at the first follow-up visit;

6. the rheumatologist’s diagnosis at the most recent follow-up visit.

Diagnosis was classified as definite RA, possible/probable RA, or not RA based on the wording used by the treating rheumatologist at the most recent follow-up visit. All patients clearly diagnosed with RA were included as such. All patients that were clearly felt not to have RA (usually given an alternate diagnosis) by the rheumatologist were labelled as “not RA”. All patients thought to have possible or probable RA by the treating rheumatologist (terminology included “likely inflammatory arthritis”, “possible mild RA”, “new onset polyarthritis”), were assigned a definition of possible/probable RA. To standardise and verify our interpretations, an initial group of 79 subject records were independently reviewed by three rheumatologists. Disagreements were resolved by consensus among all reviewers.

Lastly, in 2007 we identified the most-recent follow-up visit for each subject in our electronic medical records and noted the rheumatologist’s diagnosis at that time. Each of the above criteria was considered positive if documented as present at any time since symptom onset to the date that RF and anti-CCP were ordered. Criteria were negative if documented as such at any time since symptom onset or no documentation of a positive finding through the date that RF and anti-CCP were ordered. If a criterion was not documented, it was considered missing. The presence or absence of erosions on hand and foot radiographs were determined from documentation either by the rheumatologist from studies at other institutions and/or radiographs performed and interpreted by the radiology department at Brigham and Women’s Hospital. We included radiographic joint erosions, but not “bony decalcification localised in or most marked adjacent to the involved joints” as per the 1987 ACR criteria,10 as the latter is more subject to interpretation and we did not have access to original radiographs in all cases.

Our goals were to investigate the effects of adding anti-CCP to the existing ACR criteria, and of replacing existing criteria to maximise sensitivity and specificity for the classification of both early and long-standing RA. As nodules and erosions are late RA manifestations, we investigated the effect of removing them from the criteria. Thus, we tested the following three revisions of the 1987 ACR criteria: (a) addition of anti-CCP testing (1987 ACR criteria+anti-CCP); (b) replacement of rheumatoid nodules with anti-CCP as a criterion (CCP 7 criteria); (c) replacement of rheumatoid nodules and erosions as criteria with anti-CCP (CCP 6 criteria) (table 1). We compared the sensitivity and specificity of the 1987 ACR criteria with our three sets of modified criteria and compared these with the rheumatologists’ diagnosis at the most recent follow-up visit. Partners’ Institutional Review Board approved of all aspects of this study.

Statistical analysis

The sensitivities and specificities of the sets of criteria for the classification of RA were calculated using 2×2 tables using the rheumatologist’s diagnosis at most recent follow-up as the standard. For each of the three sets of revised criteria, we chose the cut-off that maximised sensitivity and specificity on ROC curves. The following formulas were used for the calculations: Sensitivity = (number of subjects classified as RA by criteria)/(number of subjects diagnosed with RA by the rheumatologist at the most recent follow-up); Specificity = (number of subjects classified as not having RA by criteria)/(number of subjects diagnosed as not RA by the rheumatologist at most recent follow-up).

The data were also stratified by the duration of arthritis symptoms at baseline of 6 months or less (short duration), or more than 6 months (long duration). In our primary analyses, we included subjects who had been diagnosed at the most recent visit by their rheumatologists as having either definite RA, or not RA and we excluded those subjects who were labelled as possible or probable RA. In the sensitivity analyses of our new criteria sets, we investigated the effects of including these subjects either with the individuals with definite RA or with those without RA. We analysed the data using the SAS software package version 9.1 (SAS Institute, Cary, North Carolina, USA).

RESULTS

A total of 391 subjects were identified and their medical records were reviewed. Ninety-nine subjects were excluded, 20 with juvenile RA and 79 with no follow-up visit. Thus, 292 subjects were included in this analysis. The mean age of the subjects was 54 years; 82% were women, and the mean duration of symptoms was 4.1 years. Seventeen per cent of subjects were RF positive and 14% were anti-CCP positive at initial testing (table 2).

When stratified by arthritis symptom duration of 6 months or less or more than 6 months, findings of radiographic erosions and rheumatoid nodules differed greatly between the two groups. All documented findings of rheumatoid nodules were among subjects with arthritis symptoms of greater than 6 months, although seldom were nodules specifically noted. Similarly, 91% of documented radiographic erosions were found in that group. Patients with arthritis symptoms of 6 months or less were also slightly older (table 2).

Forty-two subjects (14%) had definite RA according to their rheumatologist at the baseline visit. At the next visit, a mean of 2.8 months later, 30 additional individuals were diagnosed with RA. Eighteen of these subjects had an initial diagnosis of possible/probable RA and 12 were thought not to have RA. Of the 18 subjects initially diagnosed as possible/probable RA and subsequently diagnosed as RA at the next visit, 33% were anti-CCP positive and 44% were RF positive. In comparison, of the 30 subjects who were initially diagnosed as possible/probable RA and subsequently diagnosed as not RA in the next follow-up visit, none were anti-CCP positive and 7% were RF positive (table 3). Of the 12 subjects who were initially diagnosed as not having RA and subsequently diagnosed with RA at the next visit, 42% were anti-CCP positive and 33% were RF positive. In comparison, of the 127 subjects who maintained a diagnosis of not RA, 6% were anti-CCP positive and 8% were RF positive (table 3).

There were three subjects initially diagnosed as RA who had their diagnoses at next follow-up changed. Two of these subjects were thought not to have RA in follow-up, and one was diagnosed as possible/probable RA. These subjects were all negative for anti-CCP and RF (table 3).

Compared with the rheumatologists’ diagnosis at the most recent follow-up, a mean of 9 months after the baseline date when RF and anti-CCP were checked, the 1987 ACR criteria had a sensitivity of 51% and specificity of 91% for the classification of RA for all subjects. Adding anti-CCP to the existing criteria resulted in an improvement in sensitivity from 51 to 55% with no change in specificity (table 4). Our CCP 6 and 7 criteria, removing nodules and then removing nodules and erosions, however, had increased sensitivity compared with the ACR criteria for all subjects (77% and 74% compared with 51%), as well as for the classification of subjects with both early and longer standing RA symptoms (63% for each compared with 25% for those with early symptoms, and 81% or 77% compared with 58% for the ACR criteria among those with a longer duration of symptoms). For all of our newly revised sets of criteria, there was a decrement in the specificity to 70–80%, compared with the ACR criteria.

In sensitivity analyses, we added the subjects who were diagnosed with possible/probable RA by their rheumatologists to those defined as not RA and then to those defined as definite RA (tables 5 and 6, web-only tables). The sensitivities observed remained similar regardless of the definitions employed.

DISCUSSION

In this study, including anti-CCP in the classification criteria for RA improved upon the sensitivity of the 1987 ACR criteria without affecting the specificity in all subjects and in subjects with arthritis symptoms of ⩽6 months. This is consistent with previous studies that have reported that detection of either RF or anti-CCP can improve sensitivity of the diagnosis of RA compared with either test alone.22 Additionally, we have found that removing rheumatoid nodules alone (CCP 7 criteria) or both rheumatoid nodules and erosions (CCP 6 criteria), while substituting anti-CCP, adds further sensitivity, with loss of specificity. For subjects with arthritis symptoms of 6 months or less, our CCP 6 criteria significantly improved upon the sensitivity for classifying early RA subjects compared with the 1987 ACR criteria from 25% to 63% with a decrement in specificity from 86 to 72%. For subjects with arthritis symptoms suggestive of RA lasting >6 months, the CCP 6 and 7 criteria systems we investigated also performed better than the ACR criteria with an increase in sensitivity and a decrease in specificity. In sensitivity analyses (tables 5 and 6, web-only tables), the sensitivity and specificity of our newly revised criteria sets remained high.

Several recent editorials have called for the addition of anti-CCP into a newly revised classification criteria for RA.30 31 Aletaha and colleagues have highlighted the need for criteria that “are useful for the state of the disease at which physicians most frequently need the help of criteria, namely, very early in the course.32” The actual definition and terminology of new onset RA has not yet been established. The Norfolk Arthritis Register has examined individuals with “inflammatory polyarthritis,” defined as swelling in two or more joints for 4 weeks or longer, or less than 2 years.33 Studies published from the Leiden Early Arthritis Center studied patients with “undifferentiated arthritis.” Duration of arthritis symptoms were between 3 weeks and 2 years.34 With these studies in mind, we chose arthritis symptoms of 6 months or less for classifying early RA.

In our study population, the widely used 1987 criteria performed poorly compared with previous studies with a sensitivity of 51% in all subjects and as low as 25% in subjects with early RA. This lower sensitivity likely reflects a higher proportion of subjects in our study population with early or undifferentiated arthritis, for whom the ACR criteria in past studies have been shown to be less sensitive.24^–28 Using physician diagnosis at the latest visit (an average of 9 months since the initial visit) as a point of comparison, may also explain the lower sensitivity and specificity of the 1987 ACR criteria compared with previous studies. This conferred the advantage of time to the rheumatologist who had more information for accurate diagnosis. We used this diagnosis to assess the performance of our revised sets of criteria. However, the rheumatologist had arrived at this diagnosis in part based on the current ACR criteria and anti-CCP results. Visser and colleagues attempted to avert the circularity of using either the physician diagnosis or the ACR criteria by employing arthritis outcome at 2 years as a gold standard. The authors noted that this outcome is also not free from the influence of the classification criteria, in so much as they inform rheumatologists’ decisions to start DMARDs.28 DMARDs in turn affect erosions and arthritis outcomes. Until a definitive diagnostic test for the diagnosis of RA is developed, RA research will continue to require tolerance of a degree of circularity.

The impact of the decreased specificity in the CCP 6 criteria from 91% in the ACR criteria to 81% for all subjects and from 86 to 72% in subjects with arthritis symptoms of ⩽6 months could result in misclassification of patients with a short duration of arthritis symptoms who do not in fact have RA. The implications of adopting new criteria for the classification of RA and early RA for clinical studies are far-reaching and must be addressed by the ACR, and panels of experts and trialists. Potentially, this misclassification could bias results of clinical trials toward the null. On the other hand, slightly lower specificity and increased sensitivity could increase subject inclusion and generalisability of trial results.

This was a retrospective medical record review and our data are limited by what is recorded by the physician at each visit. Rheumatoid nodules were infrequently documented. The low prevalence of nodules in RA with current treatments may contribute to low priority on the part of rheumatologists to assess and document nodules.

This study was conducted in a large tertiary care centre where pre-test probability for disease may be higher. The tertiary care centre setting may also explain the large variation in the length of follow-up (0.1–35 months). Our patients are not part of an early RA cohort. Rather, they presented with a variety of arthritis symptoms and were thought to potentially have or be developing RA. A subset of our subjects were seen for second opinions and thus were followed for a shorter period of time and then returned to their outside rheumatologist for further care. Others were lost to follow-up or were thought not to have a rheumatic illness and returned to their primary care physician for further management. Given this real-life situation, we cannot exclude that anti-CCP antibodies, rheumatoid factor, or other symptoms later developed in those judged not to have rheumatic disease by the treating rheumatologist, and not followed in our centre long-term. However, there was no significant difference in length of follow-up in those who were anti-CCP positive and those who were negative at the initial visit (8.1 vs 9.5 months, p = 0.43). There was also no significant difference between length of follow-up between subjects diagnosed with definite RA, possible/probable RA and not RA (p = 0.26).

Many advances have occurred in the management of RA over the past 20 years. Biological agents can slow the progression of the disease and induce remission. The discovery and development of the anti-CCP assay has allowed for improvements in diagnoses. Anti-CCP antibodies may also be involved in the pathogenesis of the disease.35 36 As we move toward a new paradigm of early aggressive treatment of RA, it is important to re-evaluate our mechanisms for identifying subjects with RA for inclusion into clinical studies. It is time to revise the 1987 ACR Classification Criteria for RA to account for changes in technology and knowledge, and to develop more sensitive criteria for the classification of early RA. Further research and validation of our newly proposed criteria is needed.