Abstract
Objective. To test the intrareader and interreader reliability of assessment of joint space narrowing (JSN) in rheumatoid arthritis (RA) wrist and metacarpophalangeal (MCP) joints on magnetic resonance imaging (MRI) and computed tomography (CT) using the newly proposed OMERACT-RAMRIS JSN scoring method, and to compare JSN assessment on MRI, CT, and radiography.
Methods. After calibration of readers, MRI and CT images of the wrist and second to fifth MCP joints from 14 patients with RA and 1 healthy control were assessed twice for JSN by 3 readers, blinded to clinical and imaging data. Radiographs were scored by the Sharp/van der Heijde method. Intraclass correlation coefficients (ICC) and smallest detectable differences (SDD) were calculated, and the performance of various simplified scores was investigated.
Results. Both MRI and CT showed high intrareader (ICC ≥ 0.95) and interreader (ICC ≥ 0.94) reliability for total (wrist + MCP) assessment of JSN. Agreement was generally lower for MCP joints than for wrist joints, particularly for CT. Intrareader SDD for MCP/wrist/MCP + wrist were 1.2/6.1/6.4 JSN units for MRI, while 2.7/8.3/9.9 JSN units for CT. JSN on MRI and CT correlated moderately well with corresponding radiographic JSN scores (MCP 2–5: 0.49 and 0.56; wrist areas assessed by Sharp/van der Heijde: 0.80 and 0.95), and high ICC between scores on MRI and CT were demonstrated (MCP: 0.94; wrist: 0.92; MCP + wrist: 0.92).
Conclusion. The OMERACT-RAMRIS MRI JSN scoring system showed high intrareader and interreader reliability, and high correlation with CT scores of JSN. The suggested JSN score may, after further validation in longitudinal studies, become a useful tool in RA clinical trials.
- MAGNETIC RESONANCE IMAGING
- COMPUTED TOMOGRAPHY
- RADIOGRAPHY
- RHEUMATOID ARTHRITIS
- JOINT SPACE NARROWING
- RELIABILITY
The Outcome Measures in Rheumatology (OMERACT) Rheumatoid Arthritis (RA) Magnetic Resonance Imaging (MRI) Scoring System (RAMRIS) evaluates bone erosions, synovitis, and bone edema1,2, and is increasingly being used as an outcome measure in RA clinical trials3,4,5. As evaluation of cartilage thickness on MRI was impaired by lower quality images because of lower signal and resolution, the evaluation of joint space narrowing (JSN) was omitted in the early phase of developing the OMERACT RAMRIS. However, because of more recent technical advances of MRI, images with higher signal and resolution are now available, allowing a more detailed visualization of cartilage6,7. Because cartilage damage is an essential aspect of structural joint damage in RA, a reliable MRI assessment system of JSN would be an important improvement in the RAMRIS, for potential use as an outcome measure in RA clinical trials. Because future clinical trials will very often involve an active comparator rather than placebo, highly sensitive methods for differentiating treatment arms with respect to structural joint damage, including cartilage loss, are needed. One such method could be assessment of cartilage loss by MRI. Recently, an MRI JSN scoring system was developed by the OMERACT MRI in Inflammatory Arthritis Task Force8. Computed tomography (CT) has been used in several RA studies as a reference method for bone erosions9,10,11,12, and owing to its precise tomographic visualization of bone surfaces, CT is also well suited as a reference method for assessment of JSN.
The aim of the present initiative was to further validate the newly proposed assessment method of JSN in RA, by investigating the intrareader and interreader reliability and by comparison with CT and conventional radiography using an established and validated scoring method for joint space narrowing12.
MATERIALS AND METHODS
In a multireader exercise, undertaken by the OMERACT MRI in Inflammatory Arthritis Task Force, MRI, CT, and conventional radiographs of 14 patients with RA and 1 healthy control subject (see Table 1 for patient characteristics) were assessed for JSN in the wrist and second to fifth metacarpophalangeal (MCP) joints by 3 readers.
Patient characteristics. All data are median (range) unless otherwise indicated.
Image Acquisition
MRI of the wrist and second to fifth MCP joints was performed on a Philips Panorama 0.6T unit (Philips Medical Systems). High-resolution T1-weighted 3-dimensional fast-field-echo sequences were obtained separately of wrist and second to fifth MCP joints [repetition time 20 ms, echo-time 8 ms, acquired slice thickness 0.8 mm (reconstructed at 0.4 mm), field of view 100 mm, matrix 216*216, averages 1]. Multidetector CT images of the wrist and second to fifth MCP joints were obtained on a Philips Mx8000IDT unit (Philips Medical Systems). Images were obtained with an in-plane resolution of 0.4 mm*0.4 mm, slice thickness 0.4 mm, pitch 0.4 mm, slice spacing 0.4 mm, overlap 50%, 90 kV, and 100 mAs.
Radiography was performed on a Philips Digital Diagnost unit (Philips Medical Systems; resolution 0.143 mm). Posterior-anterior of both hands and feet were obtained. All imaging procedures were done at the Department of Radiology, Copenhagen University Hospital at Herlev, Denmark, as part of a cohort study13.
Scoring of Joint Space Narrowing
As proposed by Østergaard, et al8, JSN on MRI is defined as a reduced joint space width compared to normal, as assessed in a slice perpendicular to the joint surface. JSN is scored at 17 sites in the wrist and at the second to fifth MCP joints on a 0–4 scale [0: No narrowing; 1: Focal or mild (< 33%); 2: Moderate (34%–66%); 3: Moderate to severe (67%–99%); 4: Ankylosis) leading to a total score ranging from 0–848. The same definition of JSN was applied on CT for the purpose of the study.
The performance of various simplified scores was investigated. As in the previous exercise, we tested intrareader and interreader ICC of separate MCP and wrist scorings and scorings of the joint space locations assessed by the Sharp/van der Heijde (SvdH) and the Sharp/Genant radiographic methods and by 2 further suggestions for simplified scores, assessing 14 and 7 joint spaces in the wrist, respectively. Detailed description of the joint spaces assessed by the different approaches has been reported8.
Radiographs of hands and feet were scored for joint space narrowing according to the SvdH method12.
For comparison between MRI, CT, and radiography we chose to use the scores from the radiologist (IE), in accordance with the procedure in the previous OMERACT MRI JSN exercise8.
Multireader Exercise
An exercise was held in September 2011 in Copenhagen and included 3 readers of MRI and CT images [1 musculoskeletal radiologist (IE) and 2 rheumatologists (PC and MØ), experienced in MRI assessment of RA joints]. Radiographs were assessed by a single reader, a rheumatologist (AB) experienced in assessment on radiographs according to the SvdH method.
Phase A (calibration)
To calibrate readers, MR and CT images of unilateral wrist and MCP joints from 4 patients with RA and 1 healthy control were assessed in common, the day before the exercise.
Phase B (multireader exercise)
In this exercise, 14 persons with RA and 1 healthy control without radiographic evidence of structural joint damage was included. MR and CT images of unilateral wrist and second to fifth MCP joints were assessed separately by all readers, blinded to person identity and clinical and other imaging data. Images were read twice with different patient identification numbers and with a 1-day interval.
Statistics
Descriptive statistics, intraclass correlation coefficients (ICC; mixed effects, absolute agreement definition; average measure for interreader agreement, single measure for intrareader and comparisons between MRI and radiography and CT) and intrareader smallest detectable differences (SDD) were calculated, using SPSS Statistics, version 16.0.
RESULTS
Patient characteristics are presented in Table 1.
All patients had evidence of structural joint damage on radiographs, with a median (range) total SvdH score of 75 (3–106), joint space narrowing score of 40 (1–70), and erosion score of 27 (2–44).
Intraobserver and interobserver ICC of MRI and CT readings with various scoring approaches, with respect to number of joints and joint areas, as well as ICC between JSN scores on MRI, CT, and radiography, are presented in Table 2.
Magnetic resonance imaging and computed tomography joint space narrowing scores in patients with rheumatoid arthritis and healthy controls. Intrareader and interreader agreement.
Intraobserver SDD and the intraobserver SDD as a fraction of the observed maximum score are presented in Table 3.
Intraobserver smallest detectable difference of MRI and CT JSN scoring.
In Figures 1 and 2, examples of joint space assessments on MRI, CT, and radiography are presented.
Computed tomography (CT; A), magnetic resonance imaging (MRI; B), and radiography (C) of the wrist of a patient with rheumatoid arthritis. A coronal CT slice, the corresponding coronal T1-weighted slice, and the conventional radiograph in posterior–anterior projection are shown (note that not all joint spaces are visible in this slice). CMC: carpometacarpal; Tm: trapezium; Td: trapezoid; C: capitate; H: hamate; S: scaphoid; L: lunate; Tr: triquetrum; R: radius. Joint space narrowing scores on CT/MRI were CMC1 = 3/3; CMC2 = 3/1; CMC3 = 2/0; CMC4 = 2/0; CMC5 = 1/0; Tm-Td = 3/1; Td-C = 3/1; C-H = 0/1; Tm-S = 2/3; Td-S = 0/1; C-S = 1/1; C-L = 2/0; H-T = 0/0; S-L = 0/0; L-T = 1/1; R-S = 1/1; R-L = 1/1. Sharp/van der Heijde joint space narrowing scores were CMC3 = 2; CMC4 = 2; CMC5 = 0; Tm/Td-S = 2; C-S = 3; R-S = 2.
Computed tomography (CT; A), magnetic resonance imaging (MRI; B), and radiography (C) of the metacarpophalangeal (MCP 2–4) joints (left to right) of a patient with rheumatoid arthritis. A coronal CT slice, the corresponding coronal T1-weighted slice and the conventional radiograph in posterior-anterior projection are shown. Joint space narrowing scores on CT/MRI/radiography were MCP2 = 1/2/3, MCP3 = 2/3/3, MCP4 = 0/0/0.
DISCUSSION
The results presented here build on experiences from a former OMERACT exercise, where initial results on the OMERACT-RAMRIS scoring method of JSN on MRI were presented8, and is a further step for developing and validating a MRI scoring system for JSN in RA wrist and MCP joints. This is the first study comparing JSN assessment on MRI, CT, and radiography.
CT is generally accepted as a reference method for assessment of bone, and is an ideal imaging modality to visualize bone surfaces, and therefore the cortical margins of joints. Further, in contrast to conventional radiography, CT is a tomographic imaging technique allowing assessment of joint space without projectional superimposition and distortion. Consequently, CT is the best possible imaging external reference for the MRI JSN score.
Close correlation between JSN assessment on MRI and CT was found, and a statistically significant, albeit lower, correlation with radiographic scores was detected. Further, high to very high intrareader and interreader reliability of JSN assessment on both MRI and CT was found, especially at the wrist joints and irrespective of the number of bone surfaces assessed. Scoring of MRI JSN at a reduced number of sites has also showed high reproducibility, and good correlation with corresponding radiographic scores of JSN in other studies6,7. As in the previous exercise, substantial reader variation in absolute scores was observed, highlighting a need for further reader calibration and potentially the development of an atlas, which could serve as a reference image set. Also in agreement with the previous exercise, lower ICC for assessment of MCP joints were found. However, it should be noted that the scores of JSN were significantly higher in the wrist compared to MCP (median scores 5 and 0, respectively), which may have influenced our results in the direction toward the lower agreements in the MCP joints. The higher mobility between the metacarpal and phalangeal bone compared to the tightly bound wrist bones may also contribute to the lower reliability of scoring the MCP joints. In recent studies by Peterfy, et al a specially designed acrylic hand frame was used to ensure reproducible positioning of the hand. This possibly contributed to the high correlation to radiography7,14. Also in future MRI studies of JSN, fixed positioning of the joints must be the goal.
A major advantage of the current study compared to our previous study, and to other studies investigating JSN in RA6,7, was that an additional comparison with CT was added. A strength of using CT is its tomographic visualization of bone with a high resolution and great contrast to the surrounding soft tissues. Both MRI and CT allow visualization of joint surfaces that are inaccessible to radiographic assessment because of projectional superimposition, and consequently omitted from accepted radiographic scoring methods. In this exercise, MRI JSN assessment showed reproducibility similar to what was observed with CT. Further, although CT does not directly visualize cartilage, very high correlation with CT JSN scores supports the construct validity, if not the criterion validity (if CT can be accepted as a gold standard for joint space width), of MRI JSN. The total RAMRIS JSN score (incorporating unilateral wrist and second to fifth MCP joints) appears to have the highest correlation to the overall total SvdH, and based on these results a reduced JSN assessment score is not supported.
Radiography is a projectional 2-dimensional representation of 3-D anatomy, and some areas in the SvdH score are not assessed because they have been found not to contribute to the responsiveness of the score. This in turn is probably mainly due to projectional superimposition on radiographs that hinders appropriate assessment of all joint areas. This hindrance at least partly explains the moderate correlation between the radiographic and MRI JSN scores. In contrast, radiography is favored for its ability to visualize many different joint regions during the same examination. However, the moderate correlation of radiographs of both hands, wrists, and forefeet with MRI scores of unilateral wrist and second to fifth MCP joints demonstrates the construct validity of the MRI measure, and also that the information provided by MRI is adding to the information provided by radiography. Although direct assessment of cartilage loss with MRI in randomized controlled clinical trials of RA have been reported14,15,16, the sensitivity to change and the possible added clinical value of MRI assessment of JSN remain to be elucidated.
A limitation to our study is the cross-sectional design, because testing the MRI JSN score in a longitudinal setting would be beneficial to describe the responsiveness and discriminatory validity of the score. A future step in the validation of the scoring system will be to determine discriminant validity by testing sensitivity to change in longitudinal studies. Another limitation is that only 1 healthy control individual was included. JSN on MRI has been observed in healthy controls6,8, and in the current study JSN in the wrist was also scored at a few locations in the control person, by 2 of 3 assessors. It is expected that JSN narrowing can be seen frequently in nonrheumatoid controls, particularly with increasing age and in certain joint areas (e.g., CMC1) more commonly affected by degenerative joint disease. We did not test the feasibility of the score by measuring the time spent for assessment of JSN. However, it should be noted that when assessing MRI for JSN, important information on erosions, synovitis, and bone marrow edema are obtained in parallel, because the images used are T1-weighted images, which are also used for erosion assessment. Thus, the sequences needed for assessment of JSN did not require additional scanning time.
A theoretical limitation of JSN measurement, regardless of whether it is measured with MRI, radiography, or CT, is that it is only an indirect measure of cartilage loss. While radiography and CT cannot visualize cartilage directly, MRI is capable of delineating cartilage, and may thus offer additional accuracy14.
The OMERACT-RAMRIS MRI JSN scoring system showed high intrareader and interreader reliability, and the construct validity of the score was supported by high correlation with CT scores of JSN. Further studies on the discriminatory validity are needed, but the suggested MRI JSN score seems to be a potentially important outcome measure of cartilage damage in RA and may, after further validation in longitudinal studies, become a useful tool in RA clinical trials.
