Abstract
Objective Nailfold capillaroscopy (NFC) is increasingly used in the early identification of systemic sclerosis (SSc)-related disorders. A consensus “Fast Track algorithm” was developed by the European Alliance of Associations for Rheumatology to aid differentiation of scleroderma from nonscleroderma pattern on NFC. Our objective was to evaluate the online training of NFC using the Fast Track algorithm in the assessment of scleroderma vs nonscleroderma NFC pattern.
Methods Participants attended the NFC online training workshop and were taught the Fast Track algorithm. Following the training, participants independently evaluated 45 NFC images in the same session, and then 2 to 4 weeks later, through the online platform. Participants had to differentiate between scleroderma vs nonscleroderma pattern, and additionally nonscleroderma pattern (normal) vs nonscleroderma pattern (nonspecific). The inter- and intrarater Cohen 
agreement was calculated.
Results Ninety-eight participants took part in the baseline evaluation, and 61 in the reevaluation session. For identification of scleroderma vs nonscleroderma pattern, the mean (95% CI) inter- and intrarater 
were 0.86 (0.83-0.88) and 0.83 (0.79-0.87), respectively. The overall inter- and intrarater 
in the identification of scleroderma, nonscleroderma (normal), and nonscleroderma (nonspecific) patterns were 0.71 (0.69-0.74) and 0.71 (0.67-0.75), respectively. For nonscleroderma (normal) vs nonscleroderma (nonspecific) pattern, the inter- and intrarater 
were 0.59 (0.55-0.63) and 0.59 (0.54-0.65), respectively.
Conclusion In this first study evaluating NFC online training using the Fast Track algorithm, we showed very good inter- and intrarater agreement for the identification of scleroderma and nonscleroderma NFC pattern, supporting the feasibility of online NFC standardized training workshops.
Nailfold capillaroscopy (NFC) is increasingly used in the early identification of systemic sclerosis (SSc)-related disorders and is useful in the evaluation of patients to differentiate primary from secondary Raynaud phenomenon (RP). NFC changes characteristic of SSc and its related disorders consist of enlargement of capillary loops (enlarged and giant capillaries), dropout of capillaries (avascularity), and changes in the capillary morphology and architecture.1-3
Abnormal nailfold capillaries (scleroderma pattern) on NFC have been incorporated into the 2013 American College of Rheumatology (ACR)/European Alliance of Associations for Rheumatology (EULAR) classification criteria, as well as in criteria to facilitate a very early diagnosis of systemic sclerosis.4,5
Patients with RP and presence of both abnormal NFC and positive SSc-specific autoantibodies at baseline were 60 times more likely to experience progression to SSc.6 NFC is also useful in the evaluation and exclusion of the scleroderma spectrum of diseases in patients without RP who present with other undifferentiated features or have isolated SSc-associated antibodies.7 Apart from its diagnostic role, NFC has also shown encouraging results in predicting disease progression including the development of digital ulcers and monitoring of treatment effects.8-10 Establishing standardization and reliability of NFC analysis is therefore important for its application in daily clinical practice and its potential use as an outcome measure in clinical trials.
Data on standardization and inter- and intrarater reliability in NFC assessments, however, are limited and confined to a few centers. Previous studies have shown that an optimized simple capillaroscopic definition of normal and abnormal capillary morphology is reliable.3,11,12 Capillaroscopic characteristics including density, dimensions, morphology, and hemorrhages have been evaluated in a standardized way by the EULAR Study Group on Microcirculation in Rheumatic Diseases (SG MC/RD).13 In 2019, a multicenter consensus “Fast Track algorithm” was developed by the EULAR SG MC/RD to facilitate training of capillaroscopists to classify NFC images as scleroderma or nonscleroderma pattern.14
The aim of this study was to evaluate the online training of NFC using the Fast Track algorithm by assessing the inter- and intrarater reliability in identifying scleroderma pattern from nonscleroderma pattern. We secondarily assessed the overall reliability in the identification of NFC images, which included the nonscleroderma pattern (normal) vs nonscleroderma pattern (nonspecific).
METHODS
Conduct of the NFC online training workshops. The NFC training workshops were conducted through an online platform over 3 different workshops by the same SSc expert (AHLL), who had previously attended the 3-day EULAR workshop on NFC and has > 10 years of experience in NFC.
Participants with varying levels of experience in capillaroscopy attended the online NFC training workshops. Participants were asked to report their level of experience and were classified into novice (no experience), moderately experienced (< 5 yrs of NFC experience), or experienced (≥ 5 years of NFC experience).
The workshop comprised a training phase and an evaluation phase. In the first part of the workshop (training), all participants attended a 45-minute talk on NFC by the SSc expert (AHLL), wherein they were taught the Fast Track algorithm to identify and classify NFC image patterns using a training set of images. Existing nailfold videocapillaroscopy images of ×200 magnification captured from patients attending the NFC clinic at the Singapore General Hospital Department of Rheumatology & Immunology were selected and categorized into scleroderma, nonscleroderma (normal), and nonscleroderma (nonspecific) patterns for the workshop by AHLL and SAN, the latter of whom has 5 years of experience in NFC and previously attended the SSc congress capillaroscopy course. The Fast Track algorithm was developed by the EULAR SG MC/RD and consists of 3 simple rules: (1) the presence of ≥ 7 capillaries/mm (capillary density) AND the absence of giant capillaries (capillary dimension) allows the rater to call the capillaroscopic image a “nonscleroderma pattern (category 1)”; (2) the presence of giant capillaries or the presence of an extremely lowered capillary density (≤ 3 capillaries/mm) in combination with abnormal shapes (“late” scleroderma pattern) allows the rater to call the capillaroscopic image a “scleroderma pattern (category 2)”; and (3) if the image does not meet rule number 1 or 2, then the image is automatically classified as a nonscleroderma pattern (category 1).14 In addition, within the nonscleroderma pattern, the participants were taught to further subcategorize the images into those that were absolutely normal or those that were of a nonspecific pattern. The nonspecific pattern was defined by the presence of 4 to 6 capillaries/mm with absence of giant capillaries plus presence of any one of these abnormalities: hemorrhages, dilated capillaries (20-50 µm), or capillaries with abnormal morphology. Otherwise, the image was considered as normal pattern. Capillaries with a hairpin shape, crossing (once or twice) shape, or tortuous shape were defined as being normal, on the condition that the tip of the capillary was convex. All other shapes were defined as being abnormal.14
In the second part of the workshop (evaluation), each participant independently rated 45 NFC images in the same session. Participants were provided with the Fast Track algorithm and figure showing the nonspecific nonscleroderma pattern to refer to during the evaluation. Participants were allowed 30 seconds to rate each NFC image and submitted their answers in real-time using an electronic form. The rating of the expert (AHLL) was considered the gold standard.
Within 2 to 4 weeks of the initial rating exercise, participants re-evaluated the same set of images reshuffled without repeated training for the intrarater exercise, under supervised conditions through the online platform. Participants were also provided with the Fast Track algorithm and figure showing the nonspecific nonscleroderma pattern to refer to during the reevaluation.
We assessed the inter- and intrarater reliability for (1) identifying scleroderma from nonscleroderma NFC pattern; (2) identifying scleroderma, nonscleroderma (normal), and nonscleroderma (nonspecific) patterns; and (3) identifying nonscleroderma (normal) and nonscleroderma (nonspecific) pattern.
Statistical analysis. We computed the inter- and intrarater agreement using Cohen 
statistics. Cohen 
interpretation for degrees of agreement were as follows: ≤ 0.20 poor; 0.21 to 0.40 fair; 0.41 to 0.60 moderate; 0.61 to 0.80 good; and 0.81 to 1.00 very good.15 Mean and 95% CI of Cohen 
were reported for the entire cohort and by participant’s level of experience.
Statistical analysis was performed using SAS software version 9.4 for Windows (SAS Institute). This study was approved by the Singhealth Centralised Institutional Review Board (CIRB 2020/2628), with a waiver of consent obtained as this study fulfilled institutional exemption criteria.
RESULTS
Participants. Participants of the workshops came from 8 different Asian countries (Singapore, Malaysia, Indonesia, Hong Kong, Philippines, Taiwan, Myanmar, Brunei) and included rheumatologists, rheumatology trainees, pediatric rheumatologists, internists, and nurses. Of the 98 participants, 54 were novices, 38 were moderately experienced and 1 was experienced. For purposes of subgroup analysis, as there was only 1 participant considered to be experienced, participants were reclassified into 2 groups: novice (n = 54) or experienced (n = 39). Five participants did not specify their level of experience and were excluded for subgroup analysis. Sixty-one participants attended the reevaluation session.
Inter- and intrarater reliability in scleroderma vs nonscleroderma pattern. The mean (95% CI) interrater 
of the 98 participants for the identification of scleroderma vs nonscleroderma pattern was 0.86 (0.83-0.88), reflecting very good agreement (Table). Subgroup analysis according to the level of experience of the participant demonstrated a mean 
of 0.85 (0.82-0.89) and 0.87 (0.83-0.91) for novices and experienced raters, respectively.
Inter- and intrarater reliability for NFC virtual workshops.
In the reevaluation session, the mean inter- and intrarater 
of the 61 participants was 0.89 (0.85-0.93) and 0.83 (0.79-0.87), respectively, reflecting very good agreement. Subgroup analysis according to the level of experience of the participants, demonstrated a mean inter- and intrarater 
of 0.89 (0.84-0.94) and 0.82 (0.77-0.87), respectively, for novices, and 0.91 (0.86-0.96) and 0.87 (0.82-0.91), respectively, for experienced raters.
Overall reliability in scleroderma, nonscleroderma (normal), and nonscleroderma (nonspecific) patterns. For overall reliability in the identification of scleroderma, nonscleroderma (normal), and nonscleroderma (nonspecific) patterns, there was good agreement with mean 
of 0.71 (0.69-0.74). Subgroup analysis according to the participants’ level of experience showed mean interrater 
of 0.70 (0.66-0.73) for novices and 0.74 (0.69-0.78) for experienced raters. In the reevaluation session, the mean inter- and intrarater 
was 0.74 (0.70-0.78) and 0.71 (0.67-0.75) respectively, reflecting good agreement. Subgroup analysis according to the participants’ level of experience showed mean inter- and intrarater 
of 0.72 (0.67-0.77) and 0.69 (0.64-0.74), respectively, for novices, and 0.79 (0.74-0.84) and 0.75 (0.70-0.80), respectively, for experienced raters.
Reliability in normal vs nonspecific pattern. There was moderate agreement for the identification of normal vs nonspecific NFC pattern with mean 
of 0.59 (0.55-0.63). For subgroup analysis, agreement was higher in experienced raters with mean 
of 0.63 (0.56-0.70), reflecting good agreement, compared to novices with mean 
of 0.57 (0.51-0.62), reflecting moderate agreement. In the reevaluation session, the mean interrater 
was 0.62 (0.56-0.68), reflecting good agreement, and the intrarater 
was 0.59 (0.54-0.65), reflecting moderate agreement. For subgroup analysis, agreement was higher in experienced raters, with a mean interrater 
of 0.69 (0.62-0.77) and intrarater 
of 0.64 (0.56-0.71), reflecting good agreement, compared to novices with mean interrater 
of 0.57 (0.49-0.64) and intrarater 
of 0.56 (0.48-0.64), reflecting moderate agreement.
DISCUSSION
To our knowledge, this is the first study that examined the online training of NFC teaching, incorporating the Fast Track algorithm developed by the EULAR SG MC/RD. We demonstrated that after an initial structured training, there was very good inter- and intrarater agreement among the participants for the identification of scleroderma vs nonscleroderma NFC pattern. We also found that there was good agreement for the overall identification of scleroderma, nonscleroderma (normal), and nonscleroderma (nonspecific) NFC patterns. Agreement for the identification of normal vs nonspecific NFC pattern was moderate in the baseline evaluation session, with good interrater and moderate intrarater agreement seen in the reevaluation session. Agreement was generally higher in experienced raters compared to novices.
In the study conducted by the EULAR SG MC/RD using the Fast Track algorithm, excellent interrater reliability was demonstrated among the course attendees at the 8th EULAR course on capillaroscopy in Rheumatic Diseases in Genoa in 2018 (mean Cohen 
= 0.96) and corroborated with external validation at the 8th European Scleroderma Trials and Research Group (EUSTAR) course on SSc in Nijmegen in 2019 (mean Cohen 
= 0.94).14 Similarly, we demonstrated very good interrater agreement among our participants for the identification of scleroderma vs nonscleroderma NFC pattern. Compared to the EULAR SG MC/RD study, our study had a lower proportion of participants with prior experience in NFC (40% of participants in our study compared to 48% in the EULAR SG MC/RD study), which may explain the slightly lower interrater agreement. Further, due to the nature of the online training of our NFC workshops, some participants encountered technical difficulties with the online answer response system and were unable to rate some NFC images in real-time online within the 30-second limit (based on anonymous post course feedback) and this may also partly contribute to the lower interrater agreement. Once participants became familiar with the online answer response system, they generally performed better, with higher interrater agreement seen in the reevaluation session.
For the overall identification of scleroderma, nonscleroderma (normal), and nonscleroderma (nonspecific) NFC patterns, we showed good inter- and intrarater agreement. By contrast, moderate inter- and intrarater agreement in the overall rating of NFC images was demonstrated in a study by Rodriguez-Reyna et al, with higher inter- and intrarater agreement reported in experienced compared to inexperienced readers (moderate vs fair interrater agreement and good vs fair intrarater agreement for experienced and inexperienced readers, respectively).16 The lower inter- and intrarater agreement reported by Rodriguez-Reyna et al may be explained by participants having to identify NFC image patterns in more categories (normal, nonspecific, early, active, or late scleroderma pattern). Similar to our study, Rodriguez-Reyna et al also reported moderate interrater agreement in the identification of normal vs nonspecific pattern.16 In classifying the early, active, and late scleroderma patterns, agreement for identifying active and late patterns was higher than for the early pattern.16 In the study by Rodriguez-Reyna et al, scleroderma pattern on NFC was based on the definitions by Cutolo et al, and normal NFC patterns were based on definitions by Ingegnoli et al.2,16,17
Other studies by the EULAR SG MC/RD evaluating NFC image definition of normal or abnormal capillary morphology have shown moderate to excellent interrater agreement. In the study by Smith et al, there was moderate reliability of simple capillaroscopic definitions for describing morphology of capillaries as normal or abnormal by attendees with varying levels of expertise, at the 6th EULAR capillaroscopy course in Genoa in 2014.11 After optimization of this definition for capillary morphology, excellent reliability was obtained at the 7th EULAR capillaroscopy course in Genoa in 2016.12
We demonstrated very good agreement among our participants for the identification of scleroderma vs nonscleroderma pattern after initial training, lending support that the Fast Track algorithm is a swiftly trainable and reliable algorithm that may be used as a teaching tool for trainees with any level of experience, to differentiate a capillaroscopic image as a scleroderma pattern or nonscleroderma pattern. In our study, we have proposed an additional step to further differentiate NFC images with nonscleroderma pattern to either a normal or nonspecific pattern. We reported moderate agreement in the baseline evaluation session and good agreement in the reevaluation session for the identification of normal vs nonspecific NFC pattern, with higher agreement seen in experienced raters (good agreement) compared to novices (moderate agreement). Identifying an NFC image as normal or nonspecific can usually be more challenging as it will require participants to reliably identify capillary abnormalities including capillary density, capillary dimensions, capillary morphology, and hemorrhages. It is important to recognize nonspecific NFC images as these could represent very early capillary changes in the scleroderma spectrum of diseases or be indicative of other underlying connective tissue disorders such as systemic lupus erythematosus that do not have unique capillary patterns.13
As NFC is increasingly incorporated into routine clinical practice, there is great emphasis on the need for standardized NFC training and the unmet need for online NFC training. In a survey of SSc specialists in the United States, 74% of specialists reported informal teaching or self-education on NFC.18 Seventy-six percent wanted more formal training, and 71% would attend online training; only 33% indicated they would attend face-to-face NFC training courses.18 The encouraging results from our online NFC training workshop support and demonstrate the feasibility of online NFC training.
Nonetheless, there are some limitations to online NFC training, including (1) lack of training on NFC image acquisition; (2) single NFC image interpretation that may differ from interpretation of the complete set of NFC images for each patient, although SSc pattern recognition on NFC is more relevant for diagnosis; and (3) the use of different-sized screens on mobile devices or computer screens that would affect visualization of NFC images and potentially the learning experience.
In summary, this is the first study that examined the online training of NFC incorporating the “Fast Track algorithm.” We demonstrated very good inter- and intrarater agreement for the identification of scleroderma and nonscleroderma NFC patterns, and good overall reliability in distinguishing scleroderma, nonscleroderma (normal) and nonscleroderma (non-specific) NFC patterns. Online training workshops are therefore a feasible mode of delivering a structured and standardized NFC workshop to enable reliable interpretation of NFC.
ACKNOWLEDGMENT
This work was supported by SingHealth Duke-NUS Medicine Academic Clinical Program – Reverie Rheumatology Research Fund. We would also like to thank the attendees of the NFC online training workshop for their participation in the study.
Footnotes
AHLL was supported by the National Medical Research Council Clinical Scientist Award (grant no. MOH-CSAINV19may-0005).
The authors declare no conflicts of interest relevant to this study.
- Accepted for publication October 21, 2022.
- Copyright © 2023 by the Journal of Rheumatology
