Article Text

Extended report
Measurement of damage in systemic vasculitis: a comparison of the Vasculitis Damage Index with the Combined Damage Assessment Index
  1. Ravi Suppiah1,
  2. Oliver Flossman2,
  3. Chetan Mukhtyar3,
  4. Federico Alberici4,
  5. Bo Baslund5,
  6. Denise Brown1,
  7. Nadeem Hasan1,
  8. Julia Holle6,
  9. Zdenka Hruskova7,
  10. David Jayne2,
  11. Andrew Judge8,
  12. Mark A Little9,
  13. Peter A Merkel10,
  14. Alessandra Palmisano4,
  15. Philip Seo11,
  16. Coen Stegeman12,
  17. Vladimir Tesar7,
  18. Augusto Vaglio4,
  19. Kerstin Westman13,
  20. Raashid Luqmani1,8
  1. 1Department of Rheumatology, Nuffield Orthopaedic Centre, Oxford, UK
  2. 2Department of Nephrology, Addenbrooke's Hospital, Cambridge, UK
  3. 3Department of Rheumatology, Norfolk and Norwich University Hospital, Norwich, UK
  4. 4Department of Nephrology, University of Parma, Parma, Italy
  5. 5University Hospital of Copenhagen, Copenhagen, Denmark
  6. 6Vasculitis Centre/Department of Rheumatology, University Hospital of Schleswig–Holstein, Luebeck, Germany
  7. 7First Faculty of Medicine and General Teaching Hospital, Prague, Czech Republic
  8. 8NIHR Musculoskeletal BRU, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
  9. 9Centre for Nephrology, UCL Royal Free Campus, London, UK
  10. 10Boston University Vasculitis Center, Boston, Massachusetts, USA
  11. 11The John Hopkins Vasculitis Center, Baltimore, Maryland, USA
  12. 12University Medical Center Groningen, Groningen, The Netherlands
  13. 13University Hospital MAS, Malmö, Sweden
  1. Correspondence to Dr Ravi Suppiah, Nuffield Orthopaedic Centre, Windmill Road, Headington, Oxford OX3 7LD, UK; ravi.suppiah{at}gmail.com

Abstract

Objectives To compare the Vasculitis Damage Index (VDI) with the Combined Damage Assessment Index (CDA) as measures of damage from vasculitis.

Methods A total of 283 patients with vasculitis from 11 European centres were evaluated in a cross-sectional study using the VDI and CDA.

Results Wegener's granulomatosis (58.4%) and microscopic polyangiitis (11.0%) were the most common diagnoses. Agreement between VDI and CDA scores (Spearman's correlation) was 0.90 (95% CI 0.87 to 0.92). There was good correlation between individual comparably evaluated organ systems (Spearman's correlation 0.70–0.94). Interobserver reliability (assessed by intraclass correlation coefficient (ICC)) was 0.94 (95% CI 0.89 to 0.98) for VDI and 0.78 (95% CI 0.63 to 0.93) for CDA. Intraobserver reliability was 0.92 (95% CI 0.83 to 1.00) for VDI and 0.87 (95% CI 0.75 to 1.00) for CDA. A total of 13 items were not used in the VDI compared to 23 in the CDA. Observers agreed that the CDA covered the full spectrum of damage attributable to vasculitis but was more time consuming and thus possibly less feasible for clinical and research purposes.

Conclusions The VDI and CDA capture reliable data on damage among patients with vasculitis. The CDA captures more detail but is more complex and less practical than the VDI. Further evolution of damage assessment in vasculitis is likely to include key elements from both instruments.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

The prognosis for a patient with systemic vasculitis has improved with treatment.1,,6 However, the long-term outlook is characterised by morbidity from recurrent flares, low-grade grumbling disease and/or accumulation of damage from previous disease activity or treatment.6,,9 Systematic recording and quantification of damage allows recording of the natural history of the disease, provides distinction from disease activity and can be used as an outcome measure for clinical trials.10

The Vasculitis Damage Index (VDI) is a validated11 method for measuring damage sustained from vasculitis or its treatment. It was developed by consensus by a group of vasculitis experts and is widely used in clinical trials.4 12,,14 However, the VDI may not adequately capture all damage caused by small and medium vessel vasculitis or treatment.10 A group of international experts in vasculitis from Europe and the USA constructed a new tool to measure damage called the Combined Damage Assessment Index (CDA). It is based on the VDI,10 and includes additional items of damage that were recorded in the Wegener's Granulomatosis Etanercept Trial (WGET) but not captured by individual items on the VDI.9 10

The VDI comprises 64 items grouped into 11 categories. The CDA has 135 individual items in 17 categories, and includes some bilaterality for items involving the eyes and ears; 8 items assign gradation. The VDI and the CDA measure damage that has occurred since the onset of vasculitis; pre existing comorbidity is not counted.

The Outcome Measures in Rheumatology (OMERACT) filter consists of the following criteria. (1) Truth: does it measure what it intends to measure? (2) Discrimination: does it discriminate from situations of interest? (3) Feasibility: can the measure be easily applied given the constraints of time, money and interpretability?15

The objective of this study was to: (1) compare the performance of the CDA to the VDI in a cross-sectional study of patients with vasculitis, (2) begin to evaluate the CDA with respect to the OMERACT filter and (3) review the use of individual items in VDI and CDA.

Methods

Consecutive patients (inpatient and outpatients) with new or existing diagnoses of vasculitis were recruited from 11 European centres. Local medical ethics requirements were met by each participating site. Participants gave their written informed consent before participating in the study.

Basic demographics, type of vasculitis, duration of disease, C reactive protein (CRP) and anti-neutrophil cytoplasmic antibody (ANCA) results were obtained on each patient. Patients were assessed for disease activity using the Birmingham Vasculitis Activity Score version 3 (BVAS v3)16 and disease damage using the VDI and CDA by an observer at each site (total of 11 observers). All forms were completed in English. For the purpose of this study any damage scored had to be present following the onset of vasculitis and be present for at least 3 months. The total VDI score and the total CDA score are each represented by the cumulative number of items that are recorded, respectively. The VDI and CDA scores can stay the same or worsen over time, but cannot improve. Each item in CDA or VDI contributes 1 point to the total score.17

Convergent validity measures the extent to which assessments that are theoretically related to each other are actually related. In this case VDI and CDA should be closely correlated. Convergent validity was assessed by comparing overall VDI and CDA scores as well as individual organ scores. To evaluate discrimination we assessed the relationship between the damage assessment tools with the BVAS v3, CRP and ANCA result. In addition, interobserver and intraobserver reliability was investigated. A total of 28 (9.9%) patients were scored by 2 different observers at the same time point and 14 (4.9%) by the same observer at 2 different time points within 3 months of each other. This was the total number achieved during the study and not specifically chosen, but our expectation was that using trained observers would demonstrate good agreement based on previous experience with the VDI.11

In addition to real patients, a VDI and CDA were completed on up to 20 different paper cases by an independent group of specialist doctors, fellows and research nurses with an interest in vasculitis. The paper cases were used to assess feasibility only. The paper cases were designed on real cases seen by RL and CM but modified in order to encompass the range of items recorded in the VDI and CDA. The observers were provided with written instructions on how to complete the assessment. All observers who completed a CDA and a VDI on patients or paper cases were invited to complete a feasibility questionnaire for each of the damage assessment tools. The feasibility questionnaire was a series of 10 statements or questions that the respondents had to rate or answer on a 4-point Likert scale.

We identified unused items in VDI from the current study and combined data published on damage assessment in the WGET trial9 and from unpublished 5-year follow-up results from the European Vasculitis Study Group (EUVAS) cohorts4 13 18 19 to provide a large sample of patients to determine the potential redundancy of VDI items.

Statistical analysis

Stata V.10, (StataCorp, College Station Texas, USA) was used for analysis. The distributions of the BVAS v3, VDI and CDA scores were not normally distributed so we used a non-parametric approach based on ranks to measure their correlation. Spearman's rank correlation coefficient was calculated by independently ranking the VDI and CDA scores, then calculating the Pearson correlation between the ranks rather than the original measurements. We used the intraclass correlation coefficient (ICC) to calculate interobserver and intraobserver reliability for overall VDI and CDA scores. This method estimates the average correlation between all possible orderings of pairs and was calculated using a one-way analysis of variance. To assess interobserver reliability between observers for each of the categories in the VDA and CDA a linear-weighted κ statistic was calculated, in which observed and expected proportions of agreement are modified to include partial agreements by assigning a weight between 0 (complete disagreement) and 1 (complete agreement) to each category. The 17 subcategories of the CDA were collapsed into the same 11 categories of the VDI for this analysis.

Results

A total of 283 patients (51% women, 49% men) with vasculitis were evaluated. Disease duration ranged from 0 to 480 months. A summary of the range of diagnosis, VDI and CDA scores and disease duration is shown in table 1. Wegener's granulomatosis (58.4%) and microscopic polyangiitis (11.0%) were the most common diagnoses. The remaining patients were a mixture of other primary and secondary vasculitis. The scores ranged from 0 to 12 for the VDI and 0 to 26 for the CDA, with the largest range seen in patients with Wegener's granulomatosis with renal involvement. Table 2 shows organ system involvement as recorded by each of the damage tools. Of the 192 patients with a disease duration of at least 12 months, 170 (89%) had some damage recorded on the VDI compared to 176 (92%) on the CDA (as determined by a score >0 on each tool, respectively).

Table 1

Diagnoses, disease duration, disease damage and disease activity scores in patients with vasculitis

Table 2

Frequency of organ system damage as determined by VDI and CDA and the correlation between the total score for each organ system between the two disease damage tools in patients with vasculitis

Convergent validity

Measurements taken in an individual patient on the same date for VDI and CDA scores were paired together. In instances where more than one paired observation was available in a single patient (ie, patients assessed twice to calculate interobserver or intraobserver reliability), one of the paired observations was randomly chosen. For the total VDI and CDA scores there was a high positive correlation (ρ=0.90, p<0.001); a graphical representation of this is shown in figure 1. There was a high positive correlation between the organ system scores, except for ‘skin/mucous membrane’, where there was a moderate correlation (ρ=0.47, p<0.001). When the two skin-related items found in CDA but not VDI, ‘easy bruising’ (15.8% of patients) and ‘cutaneous scarring’ (9.0%) in the CDA, were removed from the analysis, the correlation was 0.70 (p<0.001). A complete list of the correlations between the organ systems between the VDI and CDA is provided in table 2.

Figure 1

Scatterplot showing ranked Vasculitis Damage Index (VDI) versus Combined Damage Assessment Index (CDA) scores. Patients with the same score (ties) were assigned the average rank. For example, 66 patients had a CDA score of 0. Therefore, patients 1 to 66 were assigned the rank of (1+66)/2=33.5. The next 23 patients had a CDA score of 1 (in order 67 to 89), so the rank was (67+89)/2=78, and so forth.

Discrimination

The correlation (Spearman's ρ) with BVAS 2003 was −0.17 (95% CI −0.28 to −0.05) and −0.19 (95% CI −0.30 to −0.07); CRP −0.09 (95% CI −0.21 to 0.04) and −0.12 (95% CI −0.24 to 0.01); ANCA −0.26 (95% CI −0.45 to −0.06) and −0.32 (95% CI −0.49 to −0.12), for VDI and CDA, respectively. This shows that there was no correlation between the two measures of disease damage with measures of disease activity or items considered unrelated to disease damage.

Reliability

The interobserver reliability using the ICC was 0.94 (95% CI 0.89 to 0.98) for the VDI and 0.78 (95% CI 0.63 to 0.93) for the CDA. The interobserver reliability was better for patients with short compared with long disease duration: ICC was 0.99 (95% CI 0.97 to 1.00) and 0.87 (95% CI 0.74 to 0.99) for disease duration ≤3 years versus 0.90 (95% CI 0.79 to 1.0) and 0.67 (95% CI 0.33 to 1.0) for disease duration >3 years on the VDI and CDA, respectively. Observations for intraobserver reliability were restricted to patients who were reassessed by the same observer within 3 months (14 patients for the VDI, 15 patients the CDA). The intraobserver reliability was 0.92 (95% CI 0.83 to 1.00) for the VDI and 0.87 (95% CI 0.75 to 1.00) for the CDA. There were not enough patients to determine intraobserver reliability stratified by disease duration. The κ statistics for the individual systems for interobserver and intraobserver reliability (table 3) demonstrated fair to good agreement, although CIs were wide due to small numbers (only 3/28 patients had any items recorded in the musculoskeletal system). No individual musculoskeletal item could account for the wide CIs.

Table 3

Interobserver and intraobserver reliability of measurement of damage in vasculitis for each organ system

The use of individual items

Table 4 shows the 10 most commonly used items for each of the damage assessment tools. The items mainly comprised upper respiratory tract, renal, auditory features and peripheral neuropathy (in keeping with Wegener's granulomatosis being the most common diagnosis). Items frequently used in the CDA, but not captured by the VDI were easy skin bruising (15.8%), weight gain >10 lbs/4.4 kg (14.8%) and cutaneous scarring (9.0%). Due to the increase number of options for recording damage on the CDA this has resulted in discrepancy in scoring items on the CDA compared to the VDI. For example, the proportion of patients with glomerular filtration rate <50% is different between the two assessment tools, primarily because there are other options on the CDA for recording renal impairment.

Table 4

The 10 most commonly used individual items of damage in vasculitis

A total of 13 items of damage were not used in the VDI; 11 additional items were used less than 1% of the time. In comparison, the CDA had 23 items of damage, 4 gradations of severity and 2 items attributing causality that were not used. There were an additional 45 items that were used less than 1% of the time. Table 5 shows a list of the least used items in both damage tools.

Table 5

Least used items in the VDI and CDA in patients with vasculitis

Redundant items on the VDI

Combining our study population with the WGET trial and patients with 5 years of follow-up in the EUVAS cohorts represent a total of 804 patients. The following seven items of damage were not used in the VDI in this combined population: second episode of fresh loss of pulses in one limb, chronic peritonitis, major psychosis, oesophageal stricture/upper gastrointestinal surgery, pericarditis ≥3 months/pericardectomy, subsequent major tissue loss and subsequent myocardial infarction.

Feasibility

In all, 12 observers completed the feasibility questionnaire (including 7/11 observers who scored the VDI and CDA in real patients and 5 who completed paper cases only). The five observers who completed the paper cases only were new users to both tools. Completion time was 5–10 min (range: <5–10 min) for VDI compared to 10–15 min (<5–20 min) for CDA. Experienced users completed both assessments in <5 min. In all, 10 observers (83%) reported that the VDI and CDA were useful to record the natural history of vasculitis. All observers stated that the CDA covered the full spectrum of damage attributable to vasculitis compared to 7/12 (58%) for the VDI. In all, 8 (67%) observers said that the VDI was a practical tool for clinical use compared to 5/12 (42%) for the CDA; however, only 7/12 (58%) and 3/12 (25%), respectively, would use it in clinical practice. Nine (75%) observers found the VDI easy to complete compared to five (42%) for the CDA. All observers stated that the VDI was a useful tool to measure outcomes in clinical trials whereas two disagreed with this statement for the CDA. Overall preference for the tools was mixed; 8/12 (67%) favoured the VDI. The CDA was preferred by some experienced observers, especially by those individuals who could complete both tools in a similar timeframe.

Discussion

Damage assessment represents the permanent cumulative burden of disease morbidity from vasculitis or its treatment. It records the disease course, identifies the manifestations that do not warrant further immunosuppressive treatment and serves as an outcome measure in clinical trials.17 Both tools evaluated in this study serve this function well but have contrasting benefits and drawbacks.

The level of damage detected is consistent with previous reports; 89% of patients with at least 12 months of disease duration had ≥1 item of damage captured by VDI and 92% by CDA. This compares to 89% in the WGET trial.9 A Norwegian study of Wegener's granulomatosis showed 100% of patients having damage by the end of follow-up (mean 4.7 years)20; and a UK series with systemic vasculitis demonstrated 96% with a VDI score of ≥1 by the end of follow-up (mean 6.1 years).21 The median disease duration of 39 months in this study may have been too short to detect some items of damage such as malignancy which was recorded in only 1.4% of patients. However, the relationship between vasculitis and malignancy is complex.8 22,,25

The advantages of the VDI are that it is simple to complete, has very good reliability and is a widely accepted outcome measure in clinical trials,4 9 13 with proven prognostic value. A score ≥1 at diagnosis predicts increased mortality and future organ damage.20 The VDI was preferred by the majority of observers in this study, mainly due to its relative simplicity, especially by less experienced users, which is of key importance if it is to be used infrequently in clinical practice. However, the main application of the VDI is in clinical trials, where it functions as a generic damage assessment tool for all types of vasculitis, thereby enabling widespread use, which facilitates familiarity, accuracy and completion speed.

The CDA is intended for use in clinical trials of ANCA associated vasculitis. The CDA is more comprehensive than the VDI, and may be more sensitive in detecting damage. In addition, the ranges of scores are larger and may be better at detecting change, although this was not tested in the current study. The CDA takes longer to complete than the VDI in less experienced observers, but the difference was minimal among experienced investigators. In a clinical trial setting where more investment in training is available and there is less time pressure, the increased level of data capture by the CDA may be more desirable. There is disagreement among experts as to whether or not we should move towards disease-specific assessment tools in vasculitis clinical trials (ie, whether specific forms should be used for specific types of vasculitis, or if VDI could apply to types of vasculitis).10 26 The benefit of increased sensitivity of a disease-specific tool such as the CDA has to be balanced against more limited application (ie, confined to use only in ANCA vasculitis). In addition, if multiple tools are developed for different forms of vasculitis, it reduces the ability for comparison between broadly similar conditions. Ultimately it may be useful to discuss a damage form that has a generic component and a specific component.

Gradations of severity and weighting of items are not adequately captured by existing damage tools. Intuitively, some forms of damage or gradations of severity may have more impact on a patient's quality of life or prognosis than others. The future weights applied to individual items on the CDA or VDI should improve the correlation between mortality and quality of life.17 Efforts are underway to address this.27 In addition, there are redundant items in both tools; the seven unused items on the VDI (from WGET and EUVAS studies) could be omitted from any future damage tools that are specific for ANCA associated vasculitis in order to simplify the forms. Even if these items are removed from the main form, they will be retained in the glossary under ‘other items’ so these less common items can be recorded and contribute to the index. However, unused items such as cardiomyopathy or loss of pulses may be important for some diseases (eg, Takayasu's disease) therefore should be retained in generic damage assessment tools.

There are limitations in this study. Study observers were already familiar with the VDI from previous clinical or trial experience whereas for most investigators, this study was the first time they used the CDA. This may explain the lower interobserver and intraobserver reliability of the CDA. Further training and more experience with the CDA could improve its reliability and acceptability. The current study is cross-sectional, and therefore cannot demonstrate changes to the CDA over time. Grading severity of individual items and allowing resolution of items may influence its correlation with quality of life indices and mortality. The classification of patients with less well defined forms of disease is difficult and there may be overlap between the categories listed in table 1. This is the real-life setting and therefore inclusion of these heterogeneous patients allows for the generalisability of our results.

In summary, this is the first study to test the CDA as a measure of damage in vasculitis. We have started evaluating the CDA with respect to the OMERACT filter, but more experience, especially in a longitudinal setting is required. The VDI remains the standard for damage assessment in vasculitis, and this study further validates its use. If there is move toward disease-specific damage assessment, then future revisions including a weighting system are likely to serve as outcome measures for trials in ANCA associated vasculitis.

Acknowledgments

We thank Drs M Chan, A Miller, J Robson, A Soni and R Waller, for completing paper cases and answering the feasibility questionnaire. RS was supported by the Rose Hellaby Medical Scholarship, New Zealand. CM received support from the European League Against Rheumatism via a project grant held by RL. AJ was supported by NIHR BRU Musculoskeletal Research Group, University of Oxford. DRWJ was supported by the Cambridge Biomedical Research Centre. MAL was supported by a HEFCE new blood Senior Lecturer grant.

References

Supplementary materials

Footnotes

  • Funding This study was substantially funded by a project grant from the European League Against Rheumatism (EULAR) with additional support from the US National Institutes of Health (grants U54 RR019497, U54AR057319 and U01 AR1874).

  • Competing interests None.

  • Ethics approval Local ethics requirements were met by each of the 11 sites.

  • Provenance and peer review Not commissioned; externally peer reviewed.