Abstract
There has been a marked increase in the past 15 years in the number and quality of clinical trials in the idiopathic inflammatory vasculitides, especially the small-vessel vasculitides known as antineutrophil cytoplasmic autoantibody (ANCA)-associated vasculitis [AAV; granulomatosis, with polyangiitis (Wegener’s)]. These trials have been conducted by multicenter, international groups in Europe and the United States with financial support provided by government agencies and biopharmaceutical companies. This increased clinical trial activity in vasculitis has been accompanied by the development and validation of new outcome measures — a challenging process for these complex, multiorgan system diseases. The international OMERACT Vasculitis Working Group has developed and implemented an iterative research agenda that has utilized accumulated experience and datasets from several multicenter clinical trials and large cohort studies. This work has led to the development, evaluation, validation, and endorsement, through the OMERACT consensus and validation processes, of a “core set” of outcome measurements for use in clinical trials of AAV. The core set includes domains of disease activity, damage assessment, patient-reported outcomes, and mortality; there is at least one validated outcome measurement instrument available for each domain. This report reviews the domains of illness in AAV included in the OMERACT core set, describes the instruments validated to measure these domains, and presents the approved core set.
The past decade has seen a marked increase in the number and quality of clinical trials in the idiopathic inflammatory vasculitides, especially the small-vessel vasculitides known as antineutrophil cytoplasmic autoantibody (ANCA)-associated vasculitis [AAV; granulomatosis, with polyangiitis (Wegener’s)]1,2,3,4,5,6,7. These trials have been conducted by groups in Europe and the United States; sponsorship of these activities has come from the US National Institutes of Health, US Food and Drug Administration, the European League Against Rheumatism, and industry partners. There is now significant interest among biopharmaceutical companies in conducting trials in vasculitis. This increased clinical trial activity in vasculitis has been accompanied by development and validation of new outcome measures — a challenging process for these complex, multiorgan system diseases.
The international OMERACT Vasculitis Working Group has developed and implemented an iterative research agenda through activities related to the OMERACT 7, 8, and 9 meetings8,9,10. The research has utilized accumulated experience and datasets from several multicenter clinical trials and large cohort studies. This work has led to the development, evaluation, validation, and endorsement, through the OMERACT consensus and validation processes11, of a “core set” of outcome measurements for use in clinical trials of AAV.
Core sets of outcome measures include the domains of the illness under study considered crucial to assess in clinical trials. The OMERACT process will endorse core sets only when the following requirements are met: (1) the set includes the key domains of illness; (2) there is at least one validated assessment tool available for each domain; and (3) there is international consensus for adoption and application of the core set. Assessment tools are considered valid only if they have been demonstrated to fulfill the 3 levels of the OMERACT filter: truth, discrimination, and feasibility12.
This report reviews the domains of illness in AAV included in the OMERACT core set, describes the instruments validated to measure these domains, and presents the approved core set.
CORE SET DOMAIN: DISEASE ACTIVITY ASSESSMENT
Determining disease activity in vasculitis is complex because there is no single biomarker to evaluate the heterogeneous, multisystem nature of vasculitis. Disease activity encompasses remission (complete absence of disease activity), response (quantifiable improvement in disease activity), and relapse (increase in disease activity from a previous low or absent state), which are used as outcomes in clinical trials1,2,3,4,5. Remission, durable remission, and relapse are used as primary outcomes in many studies evaluating induction and maintenance therapy; thus, assessing disease activity is crucial to conducting clinical trials in vasculitis.
Currently Used Instruments to Evaluate Disease Activity in Vasculitis
Quantifying disease activity, measuring extent of involvement, and providing prognosis are all features of disease activity assessments. The Birmingham Vasculitis Activity Score (BVAS) and its revisions are by far the most widely accepted measures of disease activity and are able to fulfill all these functions. Alternative options for disease assessment exist but are not used for major clinical trials in vasculitis13,14,15. BVAS was developed by consensus expert opinion and clinical practice and comprises 59 individual items, in 9 organ systems16. Most items are based on history and clinical examination, but some require additional investigations. Positive findings are recorded only if attributed to active vasculitis.
The BVAS for Wegener’s granulomatosis (BVAS/WG) is a modification of BVAS specifically designed for use in studies of granulomatosis with polyangiitis (Wegener’s). BVAS/WG was been validated for use and was the primary outcome measurement tool in 2 major multicenter trials of AAV4,7,17.
The most recent revision of BVAS is BVASv.3; changes were made by consensus expert opinion and included item reduction and simplification of the form. The weighting of items was unchanged. BVASv.3 was initially validated in 313 patients with either primary or secondary forms of vasculitis (excluding giant cell arteritis)18.
Summary of Validation of Instrument(s) for Domain as per Omeract Filter
Truth
BVAS was validated in 213 patients with vasculitis16 and subsequently modified (BVASv.2) and revalidated in multiple longitudinal studies. BVAS and BVASv.2 have been used in several clinical trials1,2,3,5,19.
BVAS/WG was found to be a valid, disease-specific activity index for granulomatosis with polyangiitis (Wegener’s). Tested in simulation exercises and in actual patients, BVAS/WG correlated well with the Physician’s Global Assessment, was sensitive to change, and had good inter- and intraobserver reliability. BVAS/WG has been used in the 2 largest clinical trials conducted to date for AAV [granulomatosis with polyangiitis (Wegener’s)]4,7.
BVASv.3 was found to correlate highly with BVASv.2 (ρ = 0.94, 95% CI 0.92–0.96), treatment decisions (ρ = 0.66, 95% CI 0.59–0.72), physician global assessment (ρ = 0.91, 95% CI 0.89–0.93), C-reactive protein (CRP; ρ = 0.43, 95% CI 0.31–0.54), and an older activity measure (Vasculitis Activity Index; ρ = 0.88, 95% CI 0.86–0.91) to show convergent validity, i.e., to confirm that the BVAS.v3 does in fact measure disease activity12,14.
Expert observers agreed that the current BVASv.3 and previous BVAS versions made biological sense (construct validity)9,12,13.
Discrimination
The reliability of various versions of BVAS has been consistently shown to be high12,14. Importantly, BVAS, BVAS/WG, and BVASv.3 have each been demonstrated to be sensitive to change in activity compared to external measures and differentiate between different disease states (e.g., remission and active phases of illness)12,14.
Feasibility
Completing any of the BVAS instruments takes under 3 minutes, and only modest training (with paper cases) is needed for investigators. These measures have now been used in thousands of subjects, with high investigator acceptance.
Current Status of Instruments in This Domain (Disease Activity)
Both BVAS/WG and BVASv.3 are validated measures recommended for use in clinical trials of AAV. Both tools have been incorporated into large, international multicenter trials. The correlation between the 2 instruments is quite high14.
CORE SET DOMAIN: DISEASE DAMAGE ASSESSMENT
Introduction to Domain
Chronic diseases can be described in terms of 2 separate components: disease activity and disease damage. Disease activity represents the reversible aspects of disease while disease damage denotes irreversible aspects of disease. The concept of damage is important to the clinical care of patients with vasculitis because it encourages the clinician to identify which disease manifestations do not merit immunosuppression, thereby helping avoid overtreatment, an important cause of both morbidity (including damage) and mortality. The concept of damage is also important to clinical trials, because it represents the overall, longterm burden of disease borne by the patient as the result of vasculitis. Damage is expected to correlate with other outcomes of interest, such as cumulative disease activity, mortality, and quality of life.
Currently Used Instruments to Evaluate Disease Damage in Vasculitis
The Vasculitis Damage Index (VDI) was developed to standardize the assessment of damage in patients with systemic vasculitis20. The VDI consists of 64 items of damage selected by expert consensus as representative of the forms of damage developed by subjects with systemic vasculitis. Given the diverse expression of damage that can affect patients with systemic vasculitis, perhaps the most valuable contribution of the VDI is that it has provided investigators with a method to ensure that items of damage are uniformly collected across centers.
Endstage renal disease is a particularly notable subset of the domain of disease damage. Renal disease is one of the most common and most serious manifestations of AAV. Among patients with primary systemic vasculitis, renal function is a strong predictor of mortality. Therefore, endstage renal disease (or stage 5 chronic kidney disease, defined by the US National Kidney Foundation as a glomerular filtration rate < 15) or dialysis is another important “hard” endpoint for clinical trials.
Summary of Validation of Instrument(s) for Domain as per OMERACT Filter
Truth
The original validation of the VDI was published in 1997 based on data from patients with various forms of vasculitis. Item definition and selection was by consensus of international expert opinion (face validity). VDI correlated with the Systemic Necrotizing Vasculitis (SNV) Damage Index (r = 0.67, p < 0.001), which is an older measure of damage for vasculitis, and the Systemic Lupus International Collaborating Clinics/American College of Rheumatology Damage Index (SDI) (r = 0.63, p < 0.001; i.e., showed convergent validity)20. If death is used as the ultimate endpoint (gold standard) of damage then the VDI has criterion validity: a score ≥ 1 on the VDI at initiation of immunosuppression predicts higher mortality (HR 6.1, 95% CI 1.7–22.1) in WG17. A score ≥ 5 at 6 months in a cohort of various vasculitides showed a much higher risk of mortality by 2 years (HR 12.3, 95% CI 4.2–36.9)20.
Studies have shown only a weak correlation between VDI and disease activity. This is appropriate since damage has a different construct. The VDI score has been demonstrated to correlate with disease activity (measured by the BVAS/WG) that occurred during the previous year (r = 0.20, p = 0.02) and VDI scores correlated inversely with the physical component score of the SF-36 (r = −0.31, p < 0.0001)18. However, the VDI did not correlate with concurrent measures of disease activity such as the BVAS and the CRP21.
As noted above, there are issues with content validity in the VDI as it may not detect all forms of damage incurred in AAV. This provides impetus to further improve the measurement of damage.
Discrimination
Reliability when assessed during the original validation between 1 expert and 1 novice assessor had complete interobserver agreement in 64% and within 1 point in 78%20. Recent studies have shown excellent reliability, with intraclass correlation coefficients for inter- and intraobserver reliability of 0.94 (95% CI 0.89–0.98) and 0.92 (95% CI 0.83–1.00), respectively21.
Sensitivity to change over time (i.e., increase in VDI score over time) has been demonstrated in several studies including the original validation cohort20, but also in several clinical trial cohorts from the EU and USA1,3,4,5.
VDI scores after one year of disease correlate with both serious adverse events and number of flares during the previous year, thus validating the concept of damage as a measure of the cumulative burden of disease and that VDI can discriminate between diseases states18. It has also been demonstrated that damage (as measured by the VDI) predicts both mortality and the accumulation of future damage. For example, when applied to a cohort of 59 patients with WG, the median VDI among nonsurvivors [7, interquartile range (IQR) 5–8] was significantly higher than the median VDI among survivors (4, IQR 2–5, p = 0.003 for the comparison)20.
The VDI is better able to discriminate between different levels of damage among patients with vasculitis than other instruments, including the SDI and the SNV Damage Index. When all 3 instruments were applied to the same patient population, the VDI reliably detected more forms of damage20.
Feasibility
The VDI has been implemented widely in clinical trials of AAV, and is rapid and straightforward to complete. Training to use the VDI is brief and the instrument takes only 1–2 minutes to complete; scoring is rapid.
Current Status of Instruments in This Domain (Disease Damage)
Assessment of damage using the VDI has become standard in clinical trials of systemic vasculitis, and the VDI is validated for use in clinical trials.
Application of the VDI has raised the possibility that it may not be adequate to measure all forms of damage, particularly those specific to the AAV. Therefore, through consensus of investigators in the USA and EU, the AAV Index of Damage (AVID) was created as a first step towards a new damage-assessment instrument for these diseases. AVID comprises 12 organ-based categories of damage selected to reflect the types of damage incurred by patients with WG and microscopic polyangiitis; items were selected by expert opinion and supplemented by data from clinical trials. AVID was shown to detect more forms of damage among patients with AAV than the VDI22. However, because efforts to develop AVID overlapped with European-based efforts to revise the VDI, the OMERACT group resolved to unite efforts towards the development of a Combined Damage Assessment Index (CDA), which would be particularly suitable to AAV, but also potentially applicable to other small- and medium-vessel vasculitides. Under development, the CDA is not yet ready for routine use. The current draft version, when compared to the VDI in a cross-sectional study, was more sensitive at detecting damage but was more complex and time-consuming to complete21. The next-generation damage assessment tool needs to retain the simplicity of the VDI while expanding the scope and precision of damage assessment and aiming for a more scalable score.
Thus, while VDI remains the most validated and widely used damage assessment tool for vasculitis, efforts continue to develop the next-generation of damage assessment tools for vasculitis, with a focus on vasculitis disease-specific instruments.
CORE SET DOMAIN: MORTALITY
Introduction to Domain
When first described, the systemic form of granulomatosis with polyangiitis (Wegener’s) was a uniformly fatal disease, with mortality approaching 90% in the first year after diagnosis. The advent of modern immunosuppressive regimens, including cytotoxic agents, has largely transformed AAV into chronic conditions, characterized by cycles of relapse and remission. Despite these advances, recent clinical trials of the primary systemic vasculitides continue to experience substantial mortality, ranging from 2% to 9%, depending on the patient population recruited. It is important to note that patients recruited to clinical trials continue to experience increased mortality that persists well beyond the end of the trial. For example, in a longterm followup study of participants in the Wegener’s Granulomatosis Etanercept Trial, the median age of death was 57.2 years, 19 years less than the average life expectancy in the US23. A recent study using the UK General Practice Research Database showed that the risk of death was 9-fold higher in the first year after a diagnosis of granulomatosis with polyangiitis (Wegener’s) compared with an age, sex, and location-matched control group24. Analysis of longterm followup of 4 European trials in AAV demonstrated a 2.6-times higher rate of death compared with a matched control population25.
Current Status of Mortality as an Outcome Measure in Vasculitis
Mortality is a validated outcome for use in clinical trials of AAV
The primary outcome for the current 500-person, international Plasma Exchange in Vasculitis clinical trial is death or endstage renal disease (ClinicalTrials.gov identifier no. NCT00987389).
CORE SET DOMAIN: PATIENT-REPORTED OUTCOMES: HEALTH-RELATED QUALITY OF LIFE
Introduction to Domain
Physician-reported disease-specific measures are the most widely used instruments for determining efficacy of treatment in AAV. However, physicians and patients rate differently the importance of individual disease manifestations of AAV26,27. Currently used outcome measures in AAV that are based only on physician assessments might overlook potential benefits of a treatment that are important to patients. Incorporation of health-related quality of life (HRQOL) measured by a generic instrument such as the Medical Outcomes Study Short-Form 36 (SF-36) therefore will likely add to the content validity of the core set of outcome measures for AAV. This is also likely to improve the discriminatory power properties of the core set; a composite measure identifying different, but somewhat overlapping, domains of disease is likely to differentiate better between active treatment and placebo than its individual components28. Incorporating patients’ concerns and values into trial outcomes is both appropriate and increasingly mandated by regulatory bodies. The OMERACT community encouraged the OMERACT Vasculitis Working Group to gather data on HRQOL assessment in vasculitis, and these new data have helped inform the process of approving the core set.
Currently Used Instruments to Evaluate Patient-Reported Outcomes—HRQOL in Vasculitis
The SF-36 is a widely used measure of HRQOL, one aspect of quality of life that is affected by health status29,30. Inclusion of the SF-36 has become standard practice for almost all clinical trials and observational studies for most rheumatic diseases, including vasculitis21,23. Until recently, the performance of the SF-36 had not been formally evaluated as an outcome measure in AAV. Both clinical practice and trial data consistently demonstrate that HRQOL is impaired among patients with AAV24,25,31,32,33,34.
Summary of Validation of Instrument(s) for Domain as per OMERACT Filter
Truth
During the OMERACT 10 meeting, additional data were presented demonstrating that the SF-36 fulfills criteria for face and content validity as measures of HRQOL35. In AAV, data demonstrating its association with another patient-reported outcome (PRO), the patient-reported global assessment of burden of disease, are supportive of SF-36 having construct validity as an outcome measure in AAV. The SF-36 measures can also discriminate between active disease and sustained remission in AAV.
Extensive experience with SF-36 in other diseases has consistently demonstrated the instrument’s construct validity and criterion validity.
Discrimination
Data suggest the SF-36 is reliable, and scores discriminate between disease states of importance in AAV. Extensive experience with SF-36 in other diseases has consistently demonstrated the instrument’s ability to measure change in clinical trials and differentiate among various disease states.
Feasibility
The feasibility of including the SF-36 in clinical trials has been demonstrated repeatedly for multiple rheumatic and other diseases. The form is easy to complete. The lack of a substantial number of missing forms or missing data elements in recently completed clinical trials of AAV speaks to the feasibility for trial use and acceptance by trial subjects. The instrument has some well known limitations including a complicated scoring system and a less than straightforward interpretation of the various subdomains versus the component summary scores.
Current Status of Instruments in This Domain (HRQOL:PRO)
Inclusion of the SF-36 has become standard practice for all clinical trials in vasculitis and is validated for this use. Its addition to the disease-specific instruments in AAV could improve statistical properties of the core set measures and identify domains of disease important to patients. Future research includes (1) validation of its properties in other cohorts of patients with small-vessel vasculitis; (2) determination of its sensitivity to change in longitudinal studies; (3) determination of its additive contribution beyond other core set items; (4) its ability to differentiate between truly effective treatment and placebo; and (5) its predictability for hard outcomes in AAV such as disability and mortality.
Development of a Patient-reported, Disease-specific Assessment Tool
Beyond the data on “generic” HRQOL, little is known or published about patients’ perspectives of their burden of disease in systemic vasculitis. While the SF-36 is now included in nearly all clinical trials of vasculitis (see above), and is the main PRO instrument for use in vasculitis, no vasculitis-specific PRO tool has been validated. Because patients’ subjective experiences are important, particularly in making treatment decisions, such patient-reported burden of disease should be included in the outcome measurement process. The OMERACT Vasculitis Working Group has been advancing the development of a vasculitis-specific PRO tool. A draft of an instrument was developed for patients to identify and rank vasculitis-related burdens26. Patients were asked to rank 40 vasculitis-related items and to list the 5 most important aspects of the disease in open-text spaces. Data were gathered from 265 patients at 3 meetings (one each in the USA, UK, and Germany). Among the many findings were that fatigue, reduced energy level, pain, and musculoskeletal symptoms were considered the most important disease burdens in vasculitis; manifestations associated with severe organ damage were rated lower. Patients’ perspectives of the effects of vasculitis differed substantially from previous physicians’ ratings.
The preliminary questionnaire captures face, content, and construct validity and therefore seems to measure what it intends to measure26. Discrimination between patients in remission and those in active disease was also demonstrated26. The first version of the instrument is easy to use, was accepted by the patients with almost no missing data, and appears feasible in terms of interpretability, completion time and cost26.
The initial work on a vasculitis-specific PRO tool has been promising. However, much remains to be done to arrive at a final version of a vasculitis-specific PRO instrument and to validate the instrument per the OMERACT filter. Plans for this project include conducting focus groups with patients to help ensure more complete data elements and/or new subdomains are included, with subsequent form revision and validation.
OMERACT CORE SET OF OUTCOME MEASURES FOR AAV
After review and discussion of the available data on domains of illness and associated validated outcome assessment tools for AAV (as summarized above), the OMERACT community endorsed (through the OMERACT voting process) the proposed core set for use in clinical trials (Figure 1, Table 1). This endorsement indicates that the core set includes the domains of illness felt necessary for study in any clinical trial of AAV, and that at least one assessment instrument exists for each domain, and that each such instrument has been validated per the OMERACT filter12. The development and endorsement of the OMERACT core set for AAV represents a decade of cooperative work by international investigators, and is an extremely important step in the field of vasculitis research that reflects the growing sophistication and success in conducting clinical trials in AAV.
RESEARCH AGENDA FOR THE OMERACT VASCULITIS WORKING GROUP
The development and endorsement of this core set does not mean the end of development and progress in outcome measurement in ANCA-associated vasculitis. To the contrary, the core set is the new foundation upon which much future work will be based. Next steps in outcome development for AAV include (1) development of validated response criteria for the instruments included in the core set; (2) development of validated weighting systems for items within the disease activity and damage tools; (3) consideration and testing of new instruments for each of the domains within the core set; and (4) exploration of novel methods of disease assessment.
Acknowledgment
The authors thank the many members of the OMERACT community, especially the OMERACT Executive Committee, for their longstanding support and guidance of our work; the patients with vasculitis who have provided key insight into the research agenda; and the many international colleagues who participated in the research that provides the basis for this work.
Footnotes
-
The OMERACT Vasculitis Working Group is supported by the Vasculitis Clinical Research Consortium through The National Institutes of Health/National Institute of Arthritis and Musculoskeletal and Skin Diseases (U54AR057319, U01 AR51874, and P60 AR047785) and the National Center for Research Resources/NIH: U54 RR01949703. The Vasculitis Clinical Research Consortium is part of the NIH Rare Diseases Clinical Research Network (www.RareDiseasesNetwork.org/vcrc).