Abstract
Objective. To test the learning curve of rheumatologists with different experience in videocapillaroscopy (VCP) attending an intensive training program focused on interpretation of the main capillary nailfold abnormalities, the scleroderma (systemic sclerosis, SSc) pattern, and the normal pattern, and to determine their interreader agreement with an experienced investigator.
Methods. Five investigators (1 senior, 1 junior, and 3 beginners) participated in the exercise. The study was composed of 2 steps. First, an independent investigator selected representative VCP images of normal patterns and capillary abnormalities. The second step included the training program, which ran 4 hours per day for 7 days. The senior rheumatologist taught investigators to recognize and interpret the normal pattern, the capillary abnormalities, and the different types of SSc pattern. These abnormalities were considered: homogeneously enlarged capillaries, giant capillaries, irregularly enlarged capillaries, microhemorrhages, neoangiogenesis, avascular areas, and capillary density.
Results. A total of 300 VCP images were read from all the investigators. Both κ values and overall agreement percentages of qualitative and quantitative assessments showed progressive improvement from poor to excellent from the beginning to the end of the exercise. The sensitivity and specificity of the participants in the assessment of SSc pattern at the last lecture session were high.
Conclusion. Our pilot study suggests that after an intensive 1-week training program, novice investigators with little or no experience in VCP are able to interpret the main capillary abnormalities and SSc pattern and to achieve good interreader agreement rates.
- VIDEOCAPILLAROSCOPY
- LEARNING CURVE
- INTERREADER AGREEMENT
- SCLERODERMA PATTERN
Nailfold videocapillaroscopy (VCP) has raised considerable interest among rheumatologists because of its utility in both clinical practice and research. Thanks to technical improvements, VCP has become an established method to assess the microcirculation status in patients with Raynaud’s phenomenon (RP) and connective tissue diseases, especially scleroderma (systemic sclerosis, SSc) pattern1,2,3. Moreover, VCP has demonstrated a key role in both diagnosis and disease monitoring1,2,3,4. VCP has remarkable advantages such as easy and quick assessment, noninvasiveness, affordability, repeatability, and high patient acceptability5,6,7. Like other imaging methods, expertise in this field requires knowledge, skill, and practical clinical experience.
The teaching and training of rheumatologists in the use of VCP was neglected for many years. Full-immersion capillaroscopy courses (held in Italy in 2004, 2006, 2008, and 2010) have only recently been provided by international groups such as the European League Against Rheumatism (EULAR).
It has been demonstrated that technical skill in VCP can be attained quickly7; however, correct recognition and/or interpretation of both the “normal” VCP pattern and the pathological findings typical of the SSc pattern require more time and a more precise standardization8.
To date there is little evidence of the reliability of different findings in VCP8,9 and no data about the time needed to develop skills in interpreting VCP examinations are available. Thus, our study was aimed at testing the learning curve of 4 rheumatologists with little or no experience in VCP who attended an intensive training program focused on the interpretation of the main single capillary nailfold abnormalities, SSc pattern, and normal VCP pattern, and at determining the interreader agreement with an experienced investigator.
MATERIALS AND METHODS
Patients
The images used for the exercise were obtained from 30 patients (all women) with SSc, diagnosed according to the international criteria10, 15 patients with clinically isolated RP (11 women, 4 men), and 25 healthy subjects (15 women, 10 men). Excluded were current smokers, patients < 18 years of age, and patients with finger amputations, poor nailfold capillary visibility, concomitant onychopathy, diabetes or hypertension. All patients were attending the outpatient and inpatient clinics of the Rheumatology Department of the Università Politecnica delle Marche, Ancona, Italy. The mean ± SD age was 55.1 ± 14.2 years and mean disease duration 7.1 ± 5 years.
Investigators
Five investigators with varying experience in VCP participated in the exercise. The senior (RDA) was a rheumatologist with > 10 years of VCP experience and a faculty member of the EULAR capillaroscopy courses. Her lectures were considered the “gold standard” for all the other investigators. The junior (MT) was a rheumatologist with < 2 years of experience with VCP and who had not been practicing during the previous year. The beginners (AB, MDC, and MD) were fellows in rheumatology who reported knowledge about the utility of VCP but had no previous experience with it.
Study design
The study was composed of 2 main steps. In the first step, an independent investigator (CB; acting as the neutral investigator) with wide experience in VCP assessment, who did not participate in the interreader exercise, selected and stored 300 representative VCP images (150 images involving SSc, 75 images of patients with primary RP, and 75 images of healthy volunteers). All participants were blind for all images. These images were chosen for the interreader exercise between the senior, junior, and beginner investigators. The criteria used by the neutral investigator for selection of VCP images included a good quality of images and a clear depiction of the VCP findings.
All the VCP images used for the exercise were obtained using a video-capillaroscope equipped with a 200× optical probe. The images were collected, coded, and stored using Videocap 3.0 software (DS Medica, Milan, Italy). To minimize reader bias, all the images remained anonymous.
The following single abnormalities were considered for the exercise: homogeneously enlarged capillaries, giant capillaries, irregularly enlarged capillaries, microhemorrhages, and neoangiogenesis. Avascular areas and capillary density were also considered8,9,11,12. The following international definitions for the capillary abnormalities were adopted: microhemorrhages, homogeneously enlarged capillaries (width ≥ 30 and ≤ 50 micron, according to recently proposed measurements), giant capillaries (a homogeneous enlarged diameter of both afferent and efferent limb > 50 micron), and irregularly enlarged capillaries (an irregular enlarged branch diameter of > 50 micron). The respective measurements were performed including the diameter of single limbs. Neoangiogenesis was defined as the presence of extremely tortuous, bushy, branching, ramified and coiled capillaries, 4 or more capillaries within a single dermal papilla, very elongated loops, and thin and branching interconnected capillaries originating from a single loop. An avascular area was defined as the lack of > 2 consecutive capillaries. Capillary density was calculated as the number of capillaries in the end row per each image (1.57 mm), and capillaries appearing as normal (hairpin and/or crossed) were counted8,9,11,12.
The combination in number, distribution, and arrangement of the abnormalities allowed the investigators to characterize the different types of SSc pattern, according to the main accepted classification systems13,14. The classifications were “slow” (giant loops with no capillary loss) or “active” (definite capillary loss with disorganization of the capillary bed, neoangiogenesis, and giant capillaries), according to a system devised by Maricq (classification A)13. Classification B was “early” (few giant capillaries/microhemorrhages, no capillary loss), “active” (frequent giant capillaries/microhemorrhages, mild architectural disorganization, absence of neoangiogenesis), and “late” (irregular enlargements, few giant capillaries/microhemorrhages, severe loss of capillaries, neoangiogenesis, disorganization of the capillary bed), on the basis of a new classification proposed by Cutolo, et al14.
The second step included the training program. It was run for 4 hours/day for 7 days. On the first day, the senior rheumatologist used a slide presentation to provide the junior and beginners the basic knowledge necessary to recognize and interpret the normal pattern (with its varying forms), the main single capillary nailfold abnormalities, and the different types of SSc pattern. The second day, the investigators interpreted the VCP images under the direct supervision of the senior rheumatologist, who helped to interpret the normal pattern, the distinct capillary abnormalities, and the SSc pattern. Here, the investigators had access to a core set of 50 VCP images collected from a database, different from those selected for the exercise, displaying examples of normal patterns, capillary abnormalities, and SSc pattern (Figures 1, 2, and 3). On the third day (a lecture session), the ability of the investigators to both recognize and interpret the VCP findings was compared with that of the senior rheumatologist. For this aim, the set of 300 VCP images selected in the first step were used and read independently by the 5 investigators (1 senior, 1 junior, and 3 beginners), who were blinded to the other investigators. The investigators were asked to make a quantitative evaluation of some single capillary abnormalities (i.e., the number of homogeneously enlarged capillaries, giant capillaries, irregularly enlarged capillaries, microhemorrhages, and neoangiogenesis). A qualitative assessment was adopted for the avascular areas (presence/absence), normal pattern (presence/absence), and capillary density (normal/reduced).
On the fourth day, 30 images from those used for the exercise (including images of patients with SSc or RP and healthy subjects) were selected blindly to the investigators and at random by the neutral investigator. The senior rheumatologist showed those images to the junior and beginner investigators, who gave their interpretation or VCP diagnosis. The senior rheumatologist then corrected and commented on the investigators’ answers to improve the agreement and resolve residual doubts.
On the fifth day (second lecture session) the set of VCP images selected in the first step (not including the 30 images used in the fourth day) were read again, independently, by all the investigators. At the end of this session the neutral investigator reselected 20 images from the database used in the exercise, this time choosing only images depicting different types of SSc pattern. The senior rheumatologist resolved further questions or problems, focusing specifically on the interpretation of SSc pattern.
On the sixth day (third lecture session) all investigators again read the remaining 250 images to test their ability to recognize and interpret the VCP findings (normal pattern, capillary abnormalities, and SSc pattern).
The seventh day consisted of a final independent lecture about all 250 images.
Our study was conducted according to the Declaration of Helsinki and local regulations. The institutional ethics committee (Comitato Etico dell’Azienda Sanitaria Unica Regionale di Ancona) approved the study and informed consent was obtained from all patients and healthy subjects.
Statistical analysis
All statistical analyses were performed using MedCalc® version 10.0 (MedCalc Software, Mariakerke, Belgium) for Windows XP (Microsoft Corp., Redmond, WA, USA). Descriptive results were expressed as mean and SD. For the interreader agreement, the junior and beginner results were compared with those of the senior as the gold standard. We adopted an unweighted κ and overall agreement (percentage of exact agreement) for qualitative assessment of avascular areas (presence/absence), normal pattern (presence/absence), and capillary density (normal/reduced); whereas a weighted κ was used for a quantitative assessment of both the single capillary abnormalities (number of homogeneously enlarged capillaries, giant capillaries, irregularly enlarged capillaries, microhemorrhages, neoangiogenesis) and the type of the SSc pattern (for classification A: slow = 1, active = 2; classification B: early = 1, active = 2, late = 3). A κ value of 0–0.20 was considered poor, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 good, and 0.81–1.00 excellent15. Sensitivity (percentage of investigator-tagged SSc pattern images among the senior-tagged SSc pattern images) and specificity (percentage of investigator-tagged non-SSc pattern images among the senior-tagged non-SSc pattern images) were calculated at the last evaluation.
RESULTS
A total of 300 VCP images were read by all the investigators in the first round, while 270 were read in the second round and 250 each in rounds 3 and 4.
Table 1 shows the κ values and overall agreement percentages among all the investigators for both the single capillary abnormalities and the SSc pattern at different points in the exercise. Figure 4 shows the learning curve of all the participants by the κ coefficients. Global κ values in the first round, Day 3, according to classification A, were 0.468 for the junior and 0.394, 0.529, and 0.514 for the first, second, and third beginners, respectively. Global κ values for classification B were 0.349 for the junior and 0.391, 0.398, and 0.397 for the first, second, and third beginners. In the second round (Day 5), the global κ values for classification A were 0.641 for the junior and 0.517, 0.726, and 0.720 for the first, second, and third beginners. Global κ values for classification B in the second round were 0.565 for junior and 0.410, 0.691, and 0.485 for the first, second, and third beginners.
In the third round (Day 6), the global κ values for classification A were 0.728 for the junior and 0.812, 0.816, and 0.780 for the beginners; and for classification B, 0.686 for the junior and 0.726, 0.712, and 0.686 for the first, second, and third beginners.
In the fourth round (Day 7), the global κ values for classification A were 0.812 for the junior and 0.842, 0.844, and 0.794 for the first, second, and third beginners. Global κ values for classification B were 0.786 for the junior and 0.812, 0.743, and 0.832 for the first, second, and third beginners. Comparisons between senior and beginners’ lectures in the recognition of SSc pattern according to Maricq’s13 and Cutolo’s14 classification including κ values, percentage of overall agreement, sensitivity and specificity are shown in Tables 2 and 3.
The most frequent questions formulated by the participants during the exercise were (1) How can I distinguish between homogeneously enlarged and giant capillary; (2) How can I distinguish between homogeneous and irregularly enlarged capillary; (3) Where should I measure the diameter of a capillary; (4) What type of single abnormality is acceptable within a “normal pattern”; (5) How can I distinguish the number of real capillaries in a picture characterized by poor visibility (i.e., for edema or skin thickness); (6) When can I consider that some abnormalities (i.e., microhemorrhages or neoangiogenesis) are related to manual work, manicure, onychophagia, or local trauma; and (7) Is the presence of a single giant capillary sufficient to classify an SSc pattern as early?
DISCUSSION
For the first time, our study provides evidence of a learning curve for inexperienced readers in interpreting nailfold capillary abnormalities, SSc pattern, and normal patterns by VCP. The readers achieved good interreader agreement after a 1-week intensive training course.
The requirement to demonstrate competence in interpreting VCP abnormalities is now part of the core curriculum for the rheumatologist in Europe7. Currently, an adequate competence in confirming or excluding the presence of an SSc-type microangiopathy, especially in patients with clinically isolated RP, represents the real target of VCP analysis training. This is remarkable because the SSc pattern has recently been included as part of the preliminary criteria for the very early diagnosis of SSc16.
To our knowledge, at present no studies aimed at developing and testing VCP training programs for rheumatologists have been proposed, so there is still no international consensus on the best VCP educational program. Considering that a large number of rheumatologists are incorporating VCP into their clinical practice as a valid imaging method for both diagnostic and research purposes, standardized dedicated VCP programs have become an urgent issue.
Previous studies concerning reliability of VCP assessment were performed among experienced investigators, achieving good to excellent interobserver and intraobserver agreement8,9. Our study demonstrates that investigators without previous experience in VCP may achieve good interreader agreement rates for the interpretation of the main pathological capillary abnormalities after an intensive 1-week program.
From a further analysis of our results, the following additional considerations can be formulated. First, at the end of the exercise, all the beginners had improved the agreement level for both the single abnormalities and the classification of SSc pattern compared to the first session. In some cases the range of κ values changed from poor to good or excellent. Second, both SSc pattern classifications (A and B) showed high κ values at the end of the exercise, but from a practical point of view, the B classification may be preferred because it is currently the most widely accepted, describes comprehensively the heterogeneity of the SSc pattern, and has been developed using a new generation of VCP machines. Third, the capillary abnormalities with the lowest level of agreement among all the investigators were homogeneously enlarged capillaries, irregularly enlarged capillaries, and neoangiogenesis. This more difficult recognition could be due to the lack of clear standardization in measurement for homogeneous and irregularly enlarged capillaries, and because neoangiogenesis can also assume different shapes, appearing as bizarre or odd or indistinguishable capillaries8. Fortunately, although important in the global evaluation, these abnormalities are not integral to the major abnormalities for the definition of SSc pattern. This could explain the good agreement, sensitivity, and specificity in the global evaluation of SSc pattern among all the investigators despite low κ values for the single abnormalities. Fourth, the results of the junior investigator are very interesting and should be discussed separately. In contrast to the other investigators, the junior had some experience with VCP, but she had not been practicing for some years. Unlike the fellows, the junior showed a decrease over time in her accuracy in detecting some major capillary abnormalities such as microhemorrhages and avascular areas. It should be noted that beginner investigators had spent time to follow the VCP training, while the junior investigator often had to interrupt it for her clinic obligations. She gradually became less enthusiastic about the training. This suggests that to obtain proper training in VCP, a rheumatologist must have constancy, and have enough time to acquire the skills. Finally, it is interesting that the investigators were not able to discern the normal pattern (finding it difficult to reach agreement in its recognition). This aspect is noteworthy because a rheumatologist should be able to distinguish the normal VCP pattern, to reassure the patient. Although this unexpected phenomenon represents a drawback of our proposed educational program, it leads us to consider that an ideal training program in VCP should have different steps: a first one dedicated exclusively to the recognition of the normal VCP pattern, and a subsequent step including identification of the pathological nailfold findings. Our participants (with the exception of the junior) had no previous experience in VCP, which can justify the low κ reached in the normal pattern. A relevant problem is that the theoretical concept of normal appearance (represented by a typical disposition of the capillary loops as a “comb structure”) differs frequently from that seen in clinical practice, where several factors (i.e., cutaneous transparency, manicure, onychophagia, microcumulative trauma disorders) could affect wide interindividual and intraindividual (different patterns in different fingers of the same hand) variability of the normal pattern17.
Our data may be interpreted with the following limitations. First, junior and beginner investigators have been trained by the same senior, whose interpretations served as the imaging gold standard, which could mean less objectivity. Second, our exercise was focused exclusively on the reading of the images and not on the attainment of the technical skills needed for an intraobserver and interobserver examination. However, our previous experience indicated that the technical knowledge and skills required to carry out VCP can be attained by an inexperienced operator in a very short time (about 5 nonconsecutive hours) by means of a self-teaching program under expert supervision6. Learning how to perform capillaroscopy should constitute one of the first steps in a comprehensive training, but deep knowledge of the wide range of normal findings, as well as the pathological characteristics of the SSc pattern, demands a different type of education and teaching.
Our preliminary results indicate that after an intensive 1-week training program, novice investigators without experience in VCP are able to interpret the main capillary abnormalities and SSc pattern. Nevertheless, additional similar investigations involving a larger number of participants from more centers may be useful to more strongly support these observations. In particular, the involvement of different expert trainers could improve the standardization of the teaching program.
- Accepted for publication January 4, 2012.
REFERENCES
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.