Abstract
Objective. Patients with rheumatoid arthritis (RA) have a high risk of premature cardiovascular disease (CVD). We developed CVD quality indicators (QI) for screening and use in rheumatology clinics.
Methods. A systematic review was conducted of the literature on CVD risk reduction in RA and the general population. Based on the best practices identified from this review, a draft set of 12 candidate QI were presented to a Canadian panel of rheumatologists and cardiologists (n = 6) from 3 academic centers to achieve consensus on the QI specifications. The resulting 11 QI were then evaluated by an online modified-Delphi panel of multidisciplinary health professionals and patients (n = 43) to determine their relevance, validity, and feasibility in 3 rounds of online voting and threaded discussion using a modified RAND/University of California, Los Angeles Appropriateness Methodology.
Results. Response rates for the online panel were 86%. All 11 QI were rated as highly relevant, valid, and feasible (median rating ≥ 7 on a 1–9 scale), with no significant disagreement. The final QI set addresses the following themes: communication to primary care about increased CV risk in RA; CV risk assessment; defining smoking status and providing cessation counseling; screening and addressing hypertension, dyslipidemia, and diabetes; exercise recommendations; body mass index screening and lifestyle counseling; minimizing corticosteroid use; and communicating to patients at high risk of CVD about the risks/benefits of nonsteroidal antiinflammatory drugs.
Conclusion. Eleven QI for CVD care in patients with RA have been developed and are rated as highly relevant, valid, and feasible by an international multidisciplinary panel.
Patients with rheumatoid arthritis (RA) have about a 50% higher mortality rate if they have a cardiovascular event compared to the general population1. Although chronic inflammation probably contributes to premature atherosclerosis and endothelial dysfunction2, improvements in treatment for RA have not consistently translated into reduced cardiovascular disease (CVD) events3,4. Some authors have recently proposed there may be a trend toward a “widening mortality gap” in RA due to CVD, because survival rates from CV events have improved in the general population but have remained stagnant in RA populations4,5.
Despite general recognition that patients with RA are at increased risk of CVD events, there is ample evidence that even well-established CVD risk factors such as hypertension (HTN) and dyslipidemia are not identified and managed consistently in RA populations6,7,8,9,10,11,12,13,14,15. One strategy to reduce CVD risk in RA is therefore to better identify and manage modifiable risk factors such as smoking, HTN, obesity and dyslipidemia.
Quality indicators (QI) are statements about optimum process or outcomes of care that can be further developed into performance measures. Performance measures have a specified numerator, denominator, and exclusion criteria. They can be used for measuring the percent adherence to the process or the proportion of patients achieving the desired outcome and are critical for quality improvement initiatives16.
QI are often based on guidelines from national or international professional societies but have a greater specificity than the recommendations from which they are derived16,17. They can represent either an optimum or minimum standard of care depending on the intended use of the measure (quality improvement versus accountability, e.g., in the form of pay for performance or use for accreditation). Although a variety of methodologies exist for developing QI17, a key feature of indicator development is the use of high-grade evidence combined with expert opinion.
Current QI for RA focus primarily on the treatment of joint inflammation and the monitoring of drug side effects18,19,20. To date, they have not completely addressed the most important cause of mortality in this population: CVD. The objective of our study was to develop a set of QI to address CV risk factor management in patients with RA for the purposes of quality improvement and research. The intended audience for the proposed indicators is healthcare professionals caring for patients with RA, primarily rheumatologists, and other rheumatology providers (e.g., rheumatology nurses, clinical nurse specialists, or nurse practitioners). In addition, the indicators are highly relevant to cardiologists, internists, and primary care practitioners responsible for the medical management of patients with RA.
MATERIALS AND METHODS
The approach used to develop the CVD QI is shown in Figure 1. The development of the CVD QI for RA involved 3 phases: (1) a systematic review to define the existing best practices for CVD preventive care in RA and drafting of the candidate indicators, (2) an in-person expert panel meeting with 6 experts to achieve consensus regarding the scope, wording, and specifications of the candidate measures, and (3) an international online modified-Delphi panel (n = 43) using a novel RAND-developed platform to finalize the indicators and rate the relevance, validity, and feasibility to RA care, while also obtaining feedback about whether the measures would be used in clinical practice.
Methods for developing the quality indicator set for cardiovascular care in rheumatoid arthritis.
Phase 1
This phase consisted of a systematic review of existing guidelines and indicators and development of background reports. Quality measures are often developed from existing guideline recommendations17. Therefore in Phase 1, a systematic review of existing guidelines and quality indicators in both the general population and RA literature was conducted and relevant manuscripts from 5 years (2008–2013) were included. The detailed methods and results of the systematic review are published elsewhere21. The results of the systematic review were used to identify best practices for CV care in patients with RA and to inform the development of candidate QI. The QI were worded in a standard format (If-Then-Because) to identify the clinical situation of interest (If), the recommendation (Then), and the evidence and rationale for the QI (Because). A clear numerator, denominator, and exclusion criteria were also identified for each QI to ensure accurate measurement of the indicator upon application in a routine clinical practice setting22.
A report describing each candidate QI and its specifications as well as the supporting guidelines and reported level of evidence were generated for subsequent stages.
Phase 2
An expert consensus meeting was held to finalize the scope, wording, and specifications of candidate indicators. The candidate indicators and associated reports were presented to a select group of cardiologists and rheumatologists from 3 academic centers in Canada (NA, GBJM, DL, SK, JAA-Z, JME). These individuals represented a convenience sample of clinicians and researchers with an expertise in CVD in RA and/or quality improvement. The specifications and wording of the indicators were refined in an in-person meeting and consensus was achieved through an iterative process. These experts were not part of the online modified-Delphi panel in Phase 3.
Phase 3
The candidate QI were presented to an international panel using an innovative, iterative, online, previously used platform called ExpertLens23,24. The platform was used to conduct a modified RAND/University of California at Los Angeles (UCLA) appropriateness method to finalize the QI. ExpertLens has been used to elicit expert opinion on a range of healthcare topics, including the identification of definitional features of Continuous Quality Improvement25 and of aspirational research goals for preventing suicide26. Nonetheless, this is the first time, to our knowledge, that the platform has been used for QI development.
ExpertLens combines a number of approaches including the Delphi, Nominal Group, and Crowdsourcing techniques23. With it, a larger number of panelists can be included than in typical RAND/UCLA appropriateness panels, enabling diverse geographical representation, and the findings have been shown to be reproducible among different groups24. The online panel is therefore more cost-efficient and time-efficient than typical large international consensus meetings. Additional benefits of the system are the use of unique identifiers, which can avoid dominance of the group by a small number of vocal individuals23.
Online panel composition and recruitment
A diverse group of 43 experts from North America and Europe were invited to participate including cardiologists, rheumatologists, and primary care physicians from both academic and nonacademic practices. Pharmacists, nurses, clinician scientists, and patients were also represented. Although many participants were recruited based on their prior publications in the area of CVD in RA, effort was also made to include clinicians in community practices and other types of participants, such as patients. These individuals were identified through a variety of means including national societies (e.g., Canadian Rheumatology Association, Allied Health Professions Association), patient advocacy groups (e.g., Arthritis Patient Advisory Board of the Arthritis Research Center of Canada) as well as “snowball recruitment” (chain-referral sampling). Participants were recruited by an e-mail invitation and agreed to participate prior to the panel start date. Participants did not receive financial incentives. The University of Calgary Conjoint Health Research Ethics Board (ethics identification REB13-0210) approved the project and the RAND Human Subjects’ Protection Committee determined this study to be exempt from review.
Panel rating of relevance, validity, feasibility, and likelihood of use of the QI
Between November 4 and December 3, 2013, participants took part in a 3-round ExpertLens process. In Round 1, participants rated QI on 4 criteria (see Table 1 for description of criteria). In Round 2, they reviewed the automatically generated distribution of the group’s responses to each question that included (1) a bar chart showing the frequency of each response category for the group, (2) a group median, (3) interquartile range, and (4) the participant’s own response to that question (see Supplementary Figure 1, available online at jrheum.org). Participants also were encouraged to participate in an online, asynchronous, anonymous discussion moderated by one of the authors to ensure they remained on-topic and constructive. Finally, in Round 3, participants were requested to revise their Round 1 responses based on group feedback and discussion and share their study experiences by answering a brief series of satisfaction questions. Each round was open for 7 to 14 days, depending on participation rates. Periodic reminders to participate were sent by e-mail to maximize participation.
Description of criteria used to select quality indicators.
Participants were asked to rate candidate QI on the following 4 rating criteria during Round 1 and again in Round 3. The first 2 criteria have been used previously to assess validity and feasibility27,28; the third and fourth criteria were formulated during Phase 2 with the expert panelists to assess relevance and likelihood of use. All criteria were Likert-type 1–9 scales with labeled endpoints. Detailed definitions of each criterion are shown in Table 1.
Analysis of panelist responses
To be included in the final set, indicators had to be rated as highly valid and feasible (median validity and feasibility scores ≥ 7), with no disagreement among participants. Disagreement was calculated using a formula that examines the distribution of the ratings according to the RAND/UCLA Appropriateness Method handbook29. Disagreement exists when the interpercentile range (IPR; difference between the 30th and 70th percentiles) is larger than the IPR adjusted for symmetry (IPRAS), which was calculated using the following formula:
Derivation of the formula is shown in the RAND/UCLA Appropriateness Method handbook29.
RESULTS
The results of the systematic review of existing guidelines and QI are presented elsewhere21. Briefly, recommendations and indicators were abstracted from both RA and general population guidelines that were relevant to primary CVD prevention (e.g., CVD risk assessment, lipid, diabetes and HTN screening, exercise, smoking and lifestyle counseling).
Based on the systematic review results, a set of 12 candidate indicators was drafted and presented to the Phase 2 small expert working group of 2 cardiologists and 4 rheumatologists at a meeting (Phase 2, Figure 1). The 12 drafted indicators encompassed the following topics: communication of the importance of CVD care in RA to the primary care provider, CVD risk assessment, smoking status and counseling for cessation, screening for HTN, communicating to a primary care provider about an elevated blood pressure, blood pressure control, measurement of a fasting lipid profile, dietary counseling, exercise counseling, corticosteroid tapering to lowest dose, and avoidance of nonsteroidal antiinflammatory drugs in patients at high risk of CVD events.
The expert panel in Phase 2 made recommendations as to the wording of the indicators and the specifications. Major expert panel recommendations included the following:
Outcome measures (or interim outcome measures) such as blood pressure targets should not be included in the QI set because such measures were not considered in the rheumatologist’s scope of practice
Maintaining a low disease activity state or remission is important in reducing CVD risk in patients with RA; however, general treatment of RA and measurement of disease activity is not within the scope of these QI because of the focus on QI that especially address CVD
All physicians who care for a patient with RA share responsibility for CVD risk management. Many of the QI were designed to measure communication between the rheumatologists and other care providers, to encourage a cohesive approach to monitoring and caring for CVD risk and to recognize that it is not necessarily the primary responsibility of the rheumatologists
Frequency of conducting a formal CVD risk assessment should follow minimum intervals, which were proposed in accordance with guideline recommendations. The timing of other measures where there was no guideline for the frequency of measurement was based on the panel’s consensus
Standalone QI that recommended only measuring a risk factor without a specific action (e.g., measuring a lipid profile or ascertaining smoking status) were discouraged; it was decided that this may lead to clinical inertia (e.g., measuring more lipid profiles but not calculating CVD risk or appropriately treating if indicated)
A QI on screening for diabetes should be included
Body mass index should be measured instead of using the QI on dietary counseling, which was not within the purview of a rheumatologist.
A revised set of 11 QI was selected for presentation to the ExpertLens panelists.
ExpertLens panel participant characteristics and participation rates
Forty-three individuals were invited to participate in the online ExpertLens panel, of whom 37 participated (86.0%). The self-reported characteristics of the ExpertLens participants are shown in Table 2. Twenty-eight completed all 3 rounds (65%). There were 4 participants who completed Round 1, but not Round 3, and 5 who completed Round 3, but not Round 1. During Round 2 (online discussion board), there were 24 discussion threads and 113 discussion comments. Because this demographic information was asked during Round 1, a maximum of 32 respondents answered these questions. In a sensitivity analysis, we analyzed the responses from all participants who completed Round 3 and also those who completed both Round 1 and 3; there were no significant differences (in Table 3 we report the results from individuals who participated in both rounds).
Self-reported ExpertLens participant characteristics (Round 1*).
Final validity, feasibility, and relevance ratings from the ExpertLens panel on the 11 cardiovascular quality indicators (QI)
CVD QI
After Round 2, a few minor changes to the QI were made based on feedback from earlier rounds and presented to the panel in Round 3. For example, for QI 4 (Screening for HTN), the specifications of the measure originally suggested at a minimum measuring the blood pressure once per year; however, panelists expressed during Round 2 discussions that this was not frequent enough (especially since many rheumatologic medications affect blood pressure) and the measure was modified to better reflect guideline recommendations, which suggest more frequent screening.
In Round 3, all 11 CVD QI were rated as highly relevant, valid, and feasible by the panelists without significant disagreement (Table 3). Participants also agreed that they were likely to advocate for the QI to be used in local quality improvement initiatives.
The final indicator statements are shown in Table 4, and the full specifications, including descriptions of the numerator, denominator, and relevant exclusions, are shown in Supplementary Table 1, available online at jrheum.org.
Final set of 11 cardiovascular (CV) disease quality indicators for patients with rheumatoid arthritis (RA).
DISCUSSION
The proposed set of 11 CVD QI for RA has been agreed upon as highly relevant, valid, and feasible for measurement and quality improvement initiatives in RA by a large international, multidisciplinary panel of healthcare professionals and patients. The recommended QI comprehensively cover many aspects of CVD preventive care in RA and are evidence-based and aligned with current high-quality guidelines, based on our systematic review21. Where RA guidelines were lacking in evidence to support these measures, it was decided that at a minimum, general population guidelines should be followed and these were used to support and help define the QI.
The 11 QI define processes important to CVD preventive care in RA. Risk-adjusted outcome or interim outcome measures (e.g., lipid or blood pressure targets) were not felt to be within the scope of this project because it was determined that screening for CVD risk factors and primary prevention should be a shared responsibility and not solely the responsibility of the rheumatologist. Importantly, in the United States, the National Quality Strategy recognizes care coordination as an important gap in quality measurement30. The set of CVD QI are unique because they emphasize communication between providers in caring for patients with RA, which we hope will improve coordination of comorbidity care for patients with RA.
Our work represents the first time ExpertLens has been used for QI development and shows that the online platform had a number of advantages. Typical RAND/UCLA appropriateness panels used for QI development frequently have a limited number of participants (often 9), which limits the diversity among participants and may prevent inclusion of informed patients because included participants are often experts29. By using the online platform, we were able to get broader representation from a diverse group of international participants. Unfortunately, not all recruited participants ended up providing their expert opinion. This may have been for a variety of reasons. Some participants may not have been available during panel times or decided not to participate. Others may not have received ExpertLens invitation e-mails, which could have been re-routed to their spam folders. Nonetheless, our overall participation rate of 86% was excellent and compared favorably with participation rates in other ExpertLens24 panels and online Delphi panels with fewer rounds31.
An advantage of the online platform was the ability to anonymously obtain responses and discussion threads on topics from participants, which avoided dominance of the discussion by a subset of participants. Some participants commented, however, that they would have liked more time to discuss the QI or would have benefited from a conference call to review certain aspects of the measures. However, by Round 3, general consensus was achieved, and it is unknown whether further discussion in person or online would have altered the measures significantly.
To mitigate the potential disadvantages alluded to above of holding a QI development panel entirely online, we first had a small meeting of experts to review the QI specifications and wording. Based on the recommendations of this group, QI that only measured clinician documentation of a risk factor, e.g., smoking status or lipid profile, were discouraged because it was expected that they would be unlikely to lead to quality improvement if clinicians were not prompted to “do something” when a risk factor was identified. Therefore, some of the proposed measures included more than 1 measurement concept (e.g., documenting smoking status AND recommending smoking cessation). This potentially made voting challenging for some of the online participants who may have agreed with 1 part of the QI but not another and consequently may have led to lower ratings for some of the presented QI. Because the measures are for quality improvement and not accountability, it is reasonable that a measurement concept be coupled with an action concept to avoid clinical inertia, while recognizing that it increases the complexity of executing and practically assessing the measure.
An additional strength of this work is the diversity of individuals who participated from around the world. Nonetheless, it should be noted that recruitment of individuals from community practice (especially rural practice), as well as certain physician types (e.g., general internists and primary care practitioners) was challenging because these individuals were harder to identify and recruit. Consequently, these groups were underrepresented. As shown in the literature32, it is possible that panels with a different composition could have voted differently on the proposed indicators. In this case, it is possible that feasibility ratings for some of the indicators may have been lower if the panel composition had included more rheumatologists in community practice. We encourage pilot testing of the indicators and selection of the most appropriate measures depending on the clinical setting.
We plan to further validate the QI in rheumatology practice. This further work will involve evaluation of the feasibility of measuring the indicators in different practice settings, measurement of interrater reliability, assessing whether a gap in care exists, and determining how best to implement improvements22,33.
Patients with RA have a significantly higher rate of death due to CVD than the general population. Ensuring high-quality CVD preventive care for patients with RA is one method of potentially mitigating this risk and is in keeping with current RA and general population guidelines. In this study, we proposed a comprehensive set of 11 CVD QI for patients with RA for the purposes of quality improvement and research. Our work represents the first time, to our knowledge, that ExpertLens has been used for QI development and shows that the online platform had a number of advantages and is a useful tool for QI development.
ONLINE SUPPLEMENT
Supplementary data for this article are available online at jrheum.org.
APPENDIX
Appendix
Members of the Quality Indicator International Panel: Todd J. Anderson, MD, FRCPC, Professor of Medicine, Head, Department of Cardiac Sciences, Alberta Health Services and the University of Calgary, Director, Libin Cardiovascular Institute; Sandeep Aggarwal, MD, FRCPC, Associate Clinical Professor, University of Calgary; Pooneh Akhavan, MD, MSc, Division of Rheumatology, Mount Sinai Hospital, Toronto; Christie Bartels, MD, MS, Assistant Professor Department of Medicine, Rheumatology Division, University of Wisconsin School of Medicine and Public Health; Carolyn Bell, MBBS, London, MRCP Rheumatologist for Worcestershire Royal Hospital, Worcester, UK; Gilles Boire, MD, MSc, Professor, Department of Medicine, Rheumatology Division, University of Sherbrooke; Ailsa Bosworth, Chief Executive, National Rheumatoid Arthritis Society; Linda Brown, RN, Alberta Health Services; Alexandra Charlton, BScPharm, ACPR Pharmacy Clinical Practice Leader, and Pharmacist, Division of Rheumatology, Alberta Health Services; Shirley Chow, MD, FRCPC, Assistant Professor of Medicine, Division of Rheumatology, Department of Medicine, University of Toronto; Cynthia Crowson, MS, Statistician III, Division of Biomedical Statistics and Informatics Associate Professor of Medicine (Rheumatology) and Assistant Professor of Biostatistics, Mayo Clinic; Milan Gupta, MD, FRCPC, Associate Professor of Medicine, McMaster University, Hamilton, and Medical Director, Canadian Cardiovascular Research Network, Brampton, Ontario, Canada; John G. Hanly, MD, FRCPC, Professor of Medicine and Pathology, Dalhousie University and attending staff rheumatologist, Capital Health, Halifax, Nova Scotia, Canada; Alison M. Hoens, MSc, BScPT, Physical Therapy Knowledge Broker, University of British Columbia Department of Physical Therapy and Member, Arthritis Patient Advisory Board of the Arthritis Research Center of Canada; Wes Jackson, MD, CCFP, FCFP, Assistant Professor, Department of Medicine, University of Calgary; George D. Kitas, MD, PhD, FRCP, Director of Research and Development, Dudley Group, UK National Health Service Foundation Trust and Professor of Clinical Rheumatology, Arthritis Research UK Epidemiology Unit, University of Manchester, UK; Tabitha N. Kung, MD, FRCPC, MPH, Clinical Associate, Division of Rheumatology, Department of Medicine, University of Toronto; Theresa Lupton, RN, CCRP, Nurse Clinician Rheumatology, Alberta Health Services; Paul MacMullan, MB, BCh, BAO, MRCPI, Clinical Assistant Professor, Division of Rheumatology, Department of Medicine, University of Calgary; Sherif Moustafa, MBBCh, Mayo Clinic Arizona; Michael T. Nurmohamed, MD, PhD, Associate Professor, departments of Rheumatology and Internal Medicine, VU University Medical Center, Amsterdam, the Netherlands; Sara Partington, MD, Assistant Professor of Clinical Medicine, University of Pennsylvania; Athanase D. Protogerou, MD, Hypertension Unit and Cardiovascular Research Laboratory, 1st Department of Propaedeutic Internal Medicine, Laiko Hospital, Medical School, National and Kapodistrian University of Athens, Greece; Anne-Grete Semb, MD, PhD, Preventive Cardio-Rheuma clinic, Department of Rheumatology, Diakonhjemmet Hospital, Oslo, Norway; Amanda J. Steiman, MD, MSc, FRCPC, Lupus Clinical Research Fellow, Division of Rheumatology, Toronto Western Hospital; Lisa Gale Suter, MD, Assistant Professor of Medicine, Section of Rheumatology, Yale School of Medicine; Deborah Symmons, MD, FFPH, FRCP, Professor of Rheumatology and Musculoskeletal Epidemiology, Arthritis Research UK Centre for Epidemiology; Jinoos Yazdany, MD, MPH, Associate Professor of Medicine, Division of Rheumatology, Department of Medicine, University of California, San Francisco.
Footnotes
Funded by a grant from the Canadian Institutes for Health Research (CIHR; funding ref. # 129628). Dr. Barber is a PhD candidate and her work is supported by a Health Research Clinical Fellowship from Alberta Innovates Health Solutions 2011–15 as well as a Rheumatology Postgraduate Fellowship funded by UCB Canada, The Canadian Rheumatology Association, and The Arthritis Society from 2011–13, and a Vanier Canada Graduate Scholarship 2014–15. Dr. Marshall is a CIHR Canada Research Chair in Health Services and Systems Research and the Arthur J.E. Child Chair in Rheumatology Outcomes Research.
- Accepted for publication March 27, 2015.
