Abstract
Objective. To define the ultrasonographic characteristics of calcium pyrophosphate crystal (CPP) deposits in joints and periarticular tissues and to evaluate the intra- and interobserver reliability of expert ultrasonographers in the assessment of CPP deposition disease (CPPD) according to the new definitions.
Methods. After a systematic literature review, a Delphi survey was circulated among a group of expert ultrasonographers, who were members of the CPPD Ultrasound (US) Outcome Measures in Rheumatology (OMERACT) subtask force, to obtain definitions of the US characteristics of CPPD at the level of fibrocartilage (FC), hyaline cartilage (HC), tendon, and synovial fluid (SF). Subsequently, the reliability of US in assessing CPPD at knee and wrist levels according to the agreed definitions was tested in static images and in patients with CPPD. Cohen’s κ was used for statistical analysis.
Results. HC and FC of the knee yielded the highest interobserver κ values among all the structures examined, in both the Web-based (0.73 for HC and 0.58 for FC) and patient-based exercises (0.55 for the HC and 0.64 for the FC). Kappa values for the other structures were lower, ranging from 0.28 in tendons to 0.50 in SF in the static exercise and from 0.09 (proximal patellar tendon) to 0.27 (triangular FC of the wrist) in the patient-based exercise.
Conclusion. The new OMERACT definitions for the US identification of CPPD proved to be reliable at the level of the HC and FC of the knee. Further studies are needed to better define the US characteristics of CPPD and optimize the scanning technique in other anatomical sites.
Calcium pyrophosphate deposition disease (CPPD) is one of the most common arthropathies of the elderly. Prevalence rates range from 4% to over 50%1,2,3,4, depending on the age of the patient and on the diagnostic method. According to the European League Against Rheumatism (EULAR) recommendations for the diagnosis of CPPD5, synovial fluid (SF) analysis is the gold standard for diagnostic purposes, ultrasonography (US) is a promising tool that needs additional studies for demonstrating its involvement in the disease assessment, and conventional radiography is burdened by lower sensitivity compared with SF analysis.
More recently, studies demonstrated the usefulness of US in identifying CPP crystal deposits in the hyaline cartilage (HC) and fibrocartilage (FC) at different joint sites4,6,7,8,9. However, a recent systematic literature review emphasized that even if the diagnostic accuracy of US in CPPD is relatively high in all studies, the definitions of the US characteristics of CPP crystal deposits are significantly different10. This makes it difficult to compare US results among different studies and makes multicenter studies difficult to perform.
Our aim in this study was to define the ultrasonographic characteristics of CPP deposits in joints and periarticular tissues and to evaluate the intraobserver and interobserver reliability of expert ultrasonographers in the assessment of CPPD according to the new definitions.
MATERIALS AND METHODS
Study design and setting
The Outcome Measures in Rheumatology (OMERACT) US CPPD task force was created and held the first meeting during the American College of Rheumatology congress in 2014. Here, the preliminary results of a systematic literature review and metaanalysis on the use of US in CPPD10 were presented and the necessity of a validation process in the field was discussed. Following the OMERACT methodology11, a Delphi survey on the definition and characteristics of US-detected CPP crystal deposits was circulated. Subsequently, a Web- and a patient-based exercise were performed with the aim of testing the reliability of US in the detection of CPP deposits at different joint and periarticular sites.
Reporting of the results in our manuscript followed previously published guidelines12. The study was reported to the local ethics committee and no further approval has been deemed necessary. All patients gave an informed consent before participation in the workshop.
First step: Delphi survey for defining the US aspect of CPP crystal depositions
Eighteen rheumatologists from 10 countries (1 from Denmark, 1 from France, 1 from Germany, 6 from Italy, 1 from the Netherlands, 2 from Mexico, 2 from Romania, 1 from Serbia, 2 from Spain, and 1 from Switzerland), who were experts in US and microcrystalline arthritides and members of the OMERACT US CPPD task force, participated in the study.
A preliminary survey was circulated to present the results of the systematic literature review10 to all participants and to collect their comments and suggestions on the items to be included in the Delphi survey. Different sets of definitions were included for FC, HC, tendon, and SF to better describe the features of deposits at different anatomical structure levels. For each anatomical site, the following items were defined: the shape, the echogenicity, the localization, and the behavior of the deposits at dynamic scanning.
In the first round, the Delphi survey consisted of 30 statements and the participants rated their level of agreement for each according to a Likert scale (1 = strongly disagree to 5 = strongly agree) and gave their comments. Based on the results and comments obtained, the survey was modified and proposed again to the participants until agreement was reached for at least 1 item at each category. Group agreement was considered achieved with a total cumulative agreement of 75% or more (a score of 4 or 5 in the Likert scale). Statements that did not reach this cutoff were eliminated from the following rounds while statements that achieved agreement were proposed again for voting only in the case of the presence of new statements that were formulated according to the panel’s suggestions. If no statement achieved 75% of agreement, those that reached 60% or more plus new statements were proposed again for voting to avoid missing values in the definitions. The Delphi was considered complete when agreement was achieved for all categories of the survey and the panel proposed no further modifications.
Second step: Web-based exercise
A pool of 152 US images of the anatomical sites under examination (FC, HC, tendon, SF) were collected in an equally distributed manner by 5 participants who spontaneously offered to contribute. The sample was estimated to be the minimum size to accurately estimate κ values significantly greater than 0.4, setting α at 0.05 and β at 0.10. The FC images included both menisci of the knees and triangular fibrocartilage of the wrist; HC images were collected from the femoral condyles; tendons included patellar tendon, quadriceps tendon, and Achilles tendon; and SF images were mainly collected from the suprapatellar and lateral recesses of the knee.
Each participant rated the images according to a dichotomous score (presence/absence) by applying the definitions approved in the Delphi survey. The definitions were available above every image to avoid misinterpretations.
Two weeks after the first assessment, all participants rated the same images again to assess the intraobserver reliability.
The whole Delphi process and the Web-based agreement exercise were carried out on a Web-based platform (RedCap). Only the facilitator and the epidemiologists of the study had access to the online data and were responsible for the upload and preparation of the Delphi rounds and the Web-based exercise.
Third step: Patient-based exercise
The patient-based exercise was held in Siena, Italy, in December 2015. Nine identical US machines were used (MyLabSeven, Esaote) equipped with a 3–13 MHz linear probe. In all machines, the same settings were used. The settings were created to better enhance calcific depositions and were tested and approved by the experts before the workshop. Each sonographer was allowed to modify only the basic functions (depth, gain, time gain control, frequency) to obtain the best possible image for CPP identification according to the patient’s physical characteristics.
Nine patients [4 with a diagnosis of CPPD and 5 with osteoarthritis (OA) according to SF examinations performed within 6 months before the workshop] were invited to participate. Fifteen ultrasonographers out of the 18 included in the panel participated in the exercise. Each sonographer examined the right knee and the right wrist of each subject and rated the presence/absence of CPP deposits in the HC, meniscal FC, patellar and quadriceps tendons, SF (if present) of the knee and triangular FC, and SF (if present) of the wrist. Three independent rheumatologists, experts in US and members of the local organizing committee, assisted the ultrasonographers during the procedure by collecting the data sheets and organizing and timing the shifts.
The US examination was performed according to a standardized sequence, using techniques already described in the literature for the identification of CPP deposits13,14,15. The posterior portion of the HC was overlooked to prevent patients’ discomfort from having to roll over several times; they were generally old and had functional limitation from the disease. The knee tendons were examined with the joint in complete extension, in semiflexion, and maximal flexion by transverse and longitudinal scans. SF was examined in all anterior joint recesses with the quadriceps tendon under contraction. At the wrist joint, SF was examined at the dorsal recesses with both longitudinal and transverse scanning. The triangular FC of the wrist was examined by sliding the probe over the structure, without lifting it, from the dorsal to the palmar aspect in longitudinal scanning and from proximal to distal for the transverse scanning. In all cases, dynamic scanning could be used if considered necessary (for example, flexion-extension of the knee or medial-lateral motion of the wrist).
Each sonographer had 10 min to assess the requested sites. After time expiration, the sonographer moved to the next station until every sonographer examined all patients. Power Doppler (PD) examination was not necessary for CPPD identification, but PD examination was allowed upon sonographers’ judgment to better identify anatomical landmarks (vessels) or avoid pitfalls/artifacts (posterior enhancement of vessels that could mimic CPPD). Each sonographer rated the images according to a dichotomous score (presence/absence) by applying the definitions approved in the Delphi survey. The definitions were printed and provided to each sonographer before the exercise to avoid misinterpretations.
The procedure was repeated twice with the same patients the same day (morning and afternoon) to assess the intraobserver reliability.
Statistical analysis
Intra- and interobserver reliability were calculated using the κ coefficient. Intraobserver reliability was assessed by Cohen’s κ. Interobserver reliability was studied by calculating the mean κ on all pairs (i.e., Light’s κ)16. Kappa coefficients were interpreted according to Landis and Koch17. Kappa values of 0–0.20 were considered poor, 0.20–0.40 fair, 0.40–0.60 moderate, 0.60–0.80 good, and 0.80–1.00 excellent. The percentage of observed agreement (i.e., percentage of observations that obtained the same score) and prevalence of the observed lesions were also calculated.
Analyses were performed using R Statistical Software (Foundation for Statistical Computing).
RESULTS
Delphi survey
All participants responded to all rounds of the Delphi survey (100% response rate). At the preliminary round, the definitions extrapolated from the systematic literature review were elaborated and presented to the panel divided by anatomical site and US characteristics as described above (Supplementary Table 1 is available with the online version of this article).
After the collection of the panel’s comments, the first Delphi round included 94 statements for voting. At the first round, 16 statements reached agreement and 24 statements were modified and/or added according to the comments received by the panelists, and were proposed again for voting in the second Delphi round. At the second round, 7 more statements reached agreement for a total of 23 statements, covering all aspects of US characteristics of all anatomical sites, except for the echogenicity of CPP deposition in the SF. At the third round, only 2 modified definitions regarding this aspect were proposed and finally agreement was also achieved on it. A summary of the results and the course of the Delphi survey can be seen in Supplementary Table 2, available with the online version of this article. The final definitions of the OMERACT US task force for the US aspect and characteristics of CPPD are represented in Figure 1.
Outcome Measures in Rheumatology definitions.
Web-based interobserver and intraobserver reliability exercises
All participants successfully completed both rounds of the Web-based exercise. Interreader variability, including both rounds, ranged from 0.28 for tendons (fair agreement) to 0.73 achieved for HC (good agreement). Intraobserver reliability was higher in all sites varying from a minimum value of 0.64 for SF (good agreement) to a maximum value of 0.85 for HC (very good agreement).
Patient-based interobserver and intraobserver reliability
The patient-based exercise was successfully completed in 2 rounds of about 3 and a half h each, 1 in the morning and 1 in the afternoon of the same day. Interobserver reliability, including both rounds, ranged from 0.09 (poor agreement) for the patellar tendon to 0.74 (good agreement) for the medial meniscus. Intraobserver reliability was higher in all sites and varied from 0.28 for the quadriceps tendon (fair) to 0.78 for the medial meniscus (good). Detailed results of all rounds of the patient-based exercise are presented in Table 1 (interobserver) and Table 2 (intraobserver).
Interobserver results of the Web-based and patient-based exercise. Strength of agreement: < 0.20 poor, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, 0.81–1.00 excellent.
Intraobserver results of the Web-based and patient-based exercise. Strength of agreement: < 0.20 poor, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, 0.81–1.00 excellent.
DISCUSSION
US has been increasingly used during the last decade for the diagnosis of patients with CPPD, as highlighted by a recent systematic literature review10. Further, the utility of US in assessing patients with suspected CPPD has been recognized by the EULAR recommendations, which acknowledged US as a promising tool for the diagnosis of the disease5. US is a feasible, safe, and available tool used by the rheumatologist in bedside settings. However, agreed-upon definitions for identifying CPPD by US are lacking10.
The OMERACT US group acknowledged those gaps regarding the use of US in CPPD, and in 2014 the OMERACT US subtask force on CPPD was created to proceed with the standardization of the technique in that disease. According to the OMERACT procedures, the first step was to create the definitions for the US identification of CPPD through a Delphi exercise. Division of the definitions into 4 sections (shape, form, localization, and behavior in dynamic scanning) could allow the inclusion of all previously published definitions, even if incomplete. Then, the panel’s expertise could fulfill the missing statements to achieve homogeneous and complete definitions for each anatomical site.
Before final approval of the definitions that took place during the Siena workshop, some questions related to Delphi results were raised and discussed. The major concern was whether those definitions could be applied to all joints or only to the knee joint because most of the papers that were used for retrieving the first set of statements were dealing with the knee joint7,13,14,15,18. The panel was not able to answer this question and agreed that the patient-based exercise could come up with some elements to address this aspect. Another point of discussion was the final definition of tendon deposition of CPP crystals because not all experts agreed to include the word “multiple” in the shape category. After discussing this issue, the panel conveyed that multiple depositions could increase the specificity of the finding because “single spots” could be indicative of many different kinds of pathology, and the final definition was thus approved entirely.
The Web-based reliability exercise demonstrated good results for the deposits at the level of the HC, SF, and FC while tendon assessment needs further evaluations. The good results globally obtained in all fields demonstrate consistency in the application of the new definitions, even though the interpretation of the US findings may be different among the ultrasonographers. For the patient-based exercise, 9 volunteers participated. Four of them had crystal-proven CPPD and 5 had OA, but with no examination that could rule out the presence of CPP crystals in the joints. The observers were blinded to any clinical or laboratory findings. The mean prevalence of the findings indicating CPPD in the joints of the volunteers was adequate and allowed the drawing of safe conclusions on the results, which were not biased by a lack or abundance of US findings. During the patient-based exercise, κ values changed considerably compared with the κ values obtained at the Web-based exercise.
The results of the Web-based and the patient-based exercises raise some considerations. First, as stated by the different results obtained at the FC level of knee and wrist, the new definitions may be applied only at the knee level, while they need to be retested in other FC sites. This discrepancy may be due to either local technical difficulties at the triangular fibrocartilage of the wrist or to the presence of local abnormalities that could mimic the presence of CPP deposits and create misinterpretations. This aspect needs further investigation.
Second, tendons and SF yielded the lowest values of interobserver agreement both in static and in the patient-based exercise. This may be related to the difficult applicability of the definitions and the need for a standardized scanning technique. Further, regarding tendons, in this workshop we decided not to include the Achilles tendon and the plantar fascia in our assessment, despite the high specificity of CPPD at these tendons10. This choice was made to reduce the discomfort of the patient during the examination, considering that to properly assess the Achilles tendon and the plantar fascia, the patient must roll over on the examination bed. We preferred to prevent this discomfort and also to save the time that the patient would use to take the correct position. Perhaps assessing the Achilles tendon would provide higher κ values, being a more familiar site for CPPD evaluation, but it would not reflect the reliability of assessing other tendons, less frequently involved by CPPD. Further studies are needed to better identify the features that indicate the presence of CPP deposition in those 2 structures.
To our knowledge, this is the first attempt by an international panel to create US CPPD diagnostic criteria following the rigorous OMERACT methodology. These new OMERACT definitions yielded good results in terms of reliability in the assessment of the knee HC and FC. Results at the level of other anatomical sites proved not to be satisfactory. However, the knee joint, and especially the medial meniscus, appears to be the most frequently involved site in CPPD6, and thus of interest in terms of US reliability. Taking this into account, the OMERACT US definitions for identification of CPP crystal deposits can be used at the knee level (HC and FC assessments) and may be a good support for disease diagnosis. The use of agreed and reliable definitions is of great interest, particularly in a multicenter setting. Further studies are needed to better define the US appearance of CPP deposits and the most appropriate scanning technique in anatomical sites other than the knee, and then to test their diagnostic accuracy for CPPD identification.
ONLINE SUPPLEMENT
Supplementary material accompanies the online version of this article.
Acknowledgment
The authors appreciate the unconditional support of Esaote for the patient-based exercise and the Italian Society for Rheumatology for making available the RedCap platform for this workshop.
APPENDIX 1
List of study collaborators. OMERACT US in CPPD Group: Anthony Reginato, Mario Enrique Diaz Cortes, Tomas Cazenave, Daryl MacCarter, Florentin Verzu, Mohamed Atia Mortada, Teodore Serban, Iulia Satulu, Gael Monterde, Frederich Glandjbakhch, Marco A. Cimmino, Bruno Frediani, and Luca M. Sconfienza.
- Accepted for publication January 13, 2017.
