Does a Joint Count Calibration Exercise Make a Difference? Implications for Clinical Trials and Training

To the Editor:

Formal joint counts are an integral part of disease assessment in rheumatology. They form the basis of disease responder indices including Disease Activity Score (DAS), American College of Rheumatology responder criteria, Clinical Disease Activity Index, and Simplified Disease Activity Index. Wide variability among examiners may therefore have a significant effect on outcomes in clinical trials¹. In an attempt to reduce examiner variability, the European League Against Rheumatism developed standardized joint assessment criteria for the presence or absence of joint swelling and/or tenderness². Formal training may improve the degree of variability between examiners.

A joint count calibration exercise was organized as part of the New Zealand Treat-to-Target initiative. Twenty-eight tender and swollen joint counts as described by Fuchs, et al³ were undertaken on 5 separate patients with rheumatoid arthritis (RA) by examiners working in groups of 4 or 5. Joint counts on the first 2 patients were done by individual examiners without comparing their findings with those of their colleagues. After examining the second patient, a 10-min calibration exercise was undertaken. This consisted of the groups of examiners reexamining their first 2 patients, observing each other’s technique, and discussing their findings. Repeated examination occurred until there was consensus between the examiners on the swollen and tender joints for the individual patient. The remaining 3 patients were then examined with joint counts being kept confidential from the other examiners. DAS28-C-reactive protein⁴ was calculated based on the joint counts from each examiner.

A total of 23 individuals participated: 12 (52.2%) rheumatologists, 5 (21.7%) trainees, 5 (21.7%) other health professionals, and 1 (4.3%) patient with RA. The mean number of years of experience (excluding the patient) was 13.7 (range 1–41).

The mean tender joint count (TJC), swollen joint count (SJC), and DAS28 for each patient from all rounds are shown in Table 1. There was wide variability among individual examiners as evidenced by a large standard deviation (SD; Figure 1). There was a significant reduction in the variability of SJC (p = 0.002) and DAS28 (p = 0.038) as evidenced by a decline in the SD between examiners after the calibration exercise (rounds 3–5) as compared to before the calibration exercise (rounds 1–2). There was a trend toward reduction in variability of TJC after the calibration (p = 0.063; Figure 1). Despite the reduction in variability, there was still significant variability between examiners in the final round.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1.

After calibration, variability between examiners declined in the mean tender joint count (TJC), swollen joint count (SJC), and the 28-joint Disease Activity Score (DAS28).

Tender and swollen joint counts are regarded as the most effective ways of assessing disease activity by rheumatologists⁵. Current strategies for treatment of RA as well as funding of biological therapies are guided by validated disease activity assessments such as DAS28, which heavily weight TJC and SJC. As demonstrated in this and other studies, there is marked variation in joint counts between examiners. While calibration exercises may reduce interexaminer variability, they do not eliminate it.

Variation in joint scores in clinical trials protocols is minimized by undertaking calibration exercises at clinical trial investigator meetings and by using the same trained observer at each visit^6,7. However, in day-to-day clinical practice, variation in examination technique has the potential to produce observer-dependent discrepancies in the provision of access to biologic therapies. Resources such as that recently produced by Starz, et al outlining the mechanics of joint examination may also assist in reducing variability⁸. Our training and calibration exercise was brief and informal, yet it was followed by a reduction in the SD. Joint count calibration exercises that teach a standardized examination technique could potentially minimize this variation. There is a need for further studies to validate specific joint examination protocols and to determine the value of formal training. A new video demonstrating the standard techniques for examining the joints in the DAS28 has been developed (Internet: http://youtu.be/RnvsbD6NKoc).