Microscopic measurement of inflammation in synovial tissue is employed globally by centres working in the field of arthritis research.1 Adequate and comparable synovial tissue can be safely obtained using blind-needle biopsy or rheumatological arthroscopy.2^–4 In the acquired samples, various parameters may be examined, including cell populations, vascularity, cytokines and adhesion molecules. In rheumatoid arthritis (RA), many of these have been found to relate to disease activity, severity, outcome, and to exhibit a change after treatment with corticosteroids, disease-modifying antirheumatic drugs (DMARDs) and biological therapy.5^–15

Several analysis techniques have been employed to measure these parameters. Semiquantitative analysis is a relatively quick method and therefore may facilitate examining large quantities of tissue.7 Quantitative analysis is time-consuming but more sensitive than semiquantitative scoring to change in individual patients.16 It has been shown in previous studies that these methods can reflect overall joint inflammation when applied to relatively limited amounts of synovial tissue, even though inflammation may differ widely between individual sites in a single joint.17^–19 Computerised digital image analysis has been applied more recently in this area and has been shown to correlate well with conventional methods of measurement.20^–22

This multi-centre study was undertaken to standardise and validate the methods mentioned previously by evaluating inter-observer agreement between multiple examiners in the measurement of selected parameters of inflammation in RA synovial tissue by manual quantitative, semiquantitative and computerised image analysis.

METHODS

Preparation of synovial tissue

Samples of synovial tissue were obtained at knee arthroscopy from 12 patients with active RA. In each patient, multiple synovial biopsies (at least 6) were taken throughout the knee joint under local anaesthesia using a 2.7-mm arthroscope and 2.8-mm universal biopsy forceps (Karl Storz, Tuttlingen, Germany). Biopsy tissue was prepared for analysis as previously described.7 In brief, the tissue samples were snap-frozen together en bloc in Tissue-Tek OCT (Miles Diagnostic Division, Elkhart, IN) by immersion in methylbutane (−70°C). Frozen blocks were stored in liquid nitrogen until sectioned for staining. Five-micron sections were cut in a cryostat and mounted on glass slides. The slides were fixed in acetone at room temperature for 10 min and stored at −70°C until the immunohistochemical analysis was performed.

After thawing for 20 min at room temperature, serial sections were stained with 2 monoclonal antibodies: anti-CD3 (Leu-4, Becton-Dickinson, San Jose, CA), which stains T lymphocytes, and anti-CD68 (EBM11, Dako), which stains tissue macrophages. A standard three-stage immunoperoxidase method was used and was followed by counterstaining with haematoxylin.7

Microscopic analysis of synovial tissue

Paired serial sections of synovial tissue from the 12 patients were selected for analysis. One section from each patient was examined for the presence of CD3+ and CD68+ cells. Twenty-four tissue sections (12 slides) were coded and circulated to each participating centre for measurement of synovial sublining layer CD3+ and CD68+ cell infiltration. Rheumatology units in six cities took part in the study: Amsterdam, Dublin, Leeds, Philadelphia, Stockholm and Sydney. A single examiner from each centre performed the analysis (note: in Amsterdam, SQ was done by 2 independent assessors). Each examiner was blind to the results obtained at other centres. No formal prior communication or meeting initially took place between centres to agree on or train together in the use of the scoring systems employed.

Manual scoring techniques

Sections were examined under 400× magnification using a graticule, and all suitable high-powered fields (HPF) in each section were examined. The number of positively stained cells in every HPF was recorded, and quantitative scores for mean CD3+ and CD68+ cells per HPF were derived for each slide. A semiquantitative analysis of sub-lining layer infiltration was also performed for each section using a 5-point scale, as previously described (0 = minimal infiltration, 4 = infiltration by numerous positively stained cells).7 This scale is calibrated separately for each cellular marker, as the 2 cell types under scrutiny occur with different relative frequencies in RA synovium.

After an initial interim analysis, this exercise was repeated after standardisation of scoring methods. Specifically, it was emphasised that examiners should ignore synovium without intact lining layer, or in which fibrous sub-synovial connective tissue predominated. Reference photomicrographs representing semiquantitative grades 0–4 for CD3+ and CD68+ infiltration were also circulated to each examiner prior to the second round of scoring. Due to tissue degradation over time, 2 centres (Amsterdam and Philadelphia) were unable to assess the tissue sections on a second occasion.

Computerised digital image analysis

Of the 6 centres participating in the study, 3 (Amsterdam, Dublin and Stockholm) subjected the tissue samples to computerised digital image analysis. The techniques employed in Amsterdam and Stockholm were as previously described.21 23 The methods used in Dublin are described here in brief. For each tissue section, a single low-powered-field image was acquired on an Olympus Bx51 microscope, captured using a video camera (Sony, Tokyo, Japan), and digitised using a 16-bit colour video digitiser card. The resultant colour images were in a 1392×1040 pixel RGB format with 24-bit resolution. For each acquisition session, the microscope, camera and computer were calibrated according to a standardised procedure. The images obtained were stored as tagged image file format (TIFF) graphics files. Image acquisition, modification and analysis were performed using AnalySIS software (Soft Imaging Systems, Denver, CO) and a personal computer with Intel Pentium III 800-MHz processor and Windows™ 2000 Professional Version 5.0.2195 environment. For each of CD3 and CD68, separate threshold RGB values were specified for total tissue (blue or brown) and for positive stain (brown). These threshold values remained constant throughout the analysis of sections stained with each antibody. For each digital image, regions other than the intact sub-lining layer were manually erased, and an automated macro routine calculated the percentage of sub-lining tissue occupied by positively stained cells.

Statistical analysis

Correlation statistics (Spearman’s Rho) were calculated between pairs of centres for each method, and between paired scoring methods for each centre. Single-measure intraclass correlation coefficients (ICCs) were calculated to evaluate overall inter-centre agreement, using a two-way mixed effects model. All statistical calculations were performed using SPSS software (version 10.0). Values for p of less than 0.05 were taken as significant.

RESULTS

Agreement between reader pairs

Inter-centre correlation statistics for each of the 3 methods of measurement evaluated in the study are presented in tables 1–3. A range of inter-observer r-values was observed for quantitative (r = 0.66–0.97), semiquantitative (r = 0.64–0.97) and computerised digital image analysis (0.49–0.92). For the 2 manual methods of measurement, almost all inter-centre r-values were highly statistically significant (p<0.01) for both CD3+ and CD68+ infiltration. All possible reader pairs demonstrated agreement that was significant at levels of p<0.05. For quantitative analysis (table 1), the mean inter-observer r values before and after the standardisation exercise were, respectively, 0.87, 0.89 for CD3+ and 0.84, 0.85 for CD68+ infiltration. Corresponding mean inter-centre r values of 0.84, 0.86 and 0.86, 0.81 were observed for semiquantitative scoring (table 2). In general, computerised digital image analysis of both tissue markers agreed well between the 3 participating centres (table 3). All r-values for inter-centre CD3+ scores exceeded 0.90 and were highly significant (p<0.01). The correlation coefficient for digital image analysis of sub-lining CD68 infiltration failed to reach statistical significance for one reader pair only (Amsterdam, Stockholm), whereas the 2 remaining pairs of centres derived scores that agreed well, with r values >0.70 and high levels of statistical significance.

Inter-observer agreement across all centres

Intraclass correlation coefficients for manual quantitative, semiquantitative and digital image analysis of CD3+ and CD68+ infiltration were derived from the scores generated by all participating centres and are presented in table 4. Highly significant (p<0.0001) inter-observer ICCs were observed irrespective of the method used or the tissue marker being measured. While agreement using manual quantitative analysis improved modestly following explicit standardisation of scoring techniques, from 0.61 to 0.73 for CD3+ and from 0.67 to 0.73 for CD68+, ICCs for semiquantitative analysis changed in a variable manner in the second round of scoring, from 0.80 to 0.83 for CD3+ and from 0.87 to 0.78 for CD68+ infiltration.

Agreement between computerised and manual methods

At each centre employing digital image analysis, computerised measurement of synovial sub-lining CD3+ and CD68+ infiltration agreed well with both the quantitative (fig 1A) and semiquantiative manual techniques (fig 1B). Correlation coefficients between digital and manual methods ranging as high as r = 0.98 were observed. All r values were statistically significant (p<0.05), and the majority (10 of 12) were highly significant (p<0.01).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1 Agreement between 3 methods of computerised digital image analysis and (A) quantitative, (B) semiquantitative manual techniques for measurement of synovial sub-lining CD3+ and CD68+ infiltration. Ams, Amsterdam method; Dub, Dublin method; Sto, Stockholm method. *p<0.05; **p<0.01.

DISCUSSION

This study is the first to systematically evaluate the concept of inter-observer agreement for microscopic measurement of inflammation in synovial tissue beween globally distributed research centres. The current literature describes single centre studies that have never addressed comparison between sites. Interobserver agreement for measurement of synovial parameters other than sublining layer CD3+ and CD68+ infiltration was not evaluated. The 12 paired tissue sections used in this study were selected to include a wide range of synovial inflammation. This variation was reflected in the immunohistochemical scores obtained, with some slides consistently showing little, and others extensive, cellular infiltration. Each participating centre had extensive experience in this area, and had participated together in over 10 years of concerted collaborative work towards developing and validating these methodologies. Excellent, highly significant inter-observer agreement was demonstrated for both quantitative and semiquantitative manual analysis of both cellular markers quantified. An additional explicit inter-reader standardisation exercise was of some value for manual quantitative but not semiquantitative analysis. The changes in inter-reader agreement that were observed following this step were modest. It seems probable, therefore, that understanding of the scoring methods had been satisfactorily consistent across centres from the study’s outset.

The evaluation of the performance characteristics of 3 methods of computerised digital image analysis of synovial tissue represented a further novel aspect of the study. At each centre, computerised image analysis of the 12 tissue samples circulated generated results that correlated strongly with those obtained using manual measurement. In addition, the overall general level of inter-method agreement for the 3 digital systems was significant and satisfactory. The Amsterdam and Stockholm methods did not correlate significantly for the measurement of sub-lining CD68+ infiltration only. The former centre has demonstrated a clear relationship between sub-lining CD68+ cell infiltration and disease activity across a large number of studies employing both effective and ineffective therapies in RA, while a recent study from Stockholm did not replicate this finding.14 23 There exist significant systematic differences between algorithms and methodology applied in the various image analysis systems used at the 3 centres, which may explain these discrepant findings. For example, the Amsterdam system utilises an algorithm that seeks to identify and count individual CD68+ cells as units, whereas the Dublin and Stockholm methods quantify CD68+ infiltration as the percentage of tissue occupied by positive stain. Further collaborative studies aimed at resolving these methodological issues are presently under way.

Microscopic measurement of inflammation in synovial tissue may be useful in identifying sensitive biomarkers of response to novel therapies in RA and other types of arthritis.24 This study is the first to demonstrate clear agreement between several geographically remote centres employing manual and digital methods to measure T cell and macrophage infiltration in synovial tissue. The present study further supports the development of these methods as outcome measures for use in clinical trials in rheumatology.