Skip to main content

Main menu

  • Home
  • Content
    • First Release
    • Current
    • Archives
    • Collections
    • Audiovisual Rheum
    • 50th Volume Reprints
  • Resources
    • Guide for Authors
    • Submit Manuscript
    • Payment
    • Reviewers
    • Advertisers
    • Classified Ads
    • Reprints and Translations
    • Permissions
    • Meetings
    • FAQ
    • Policies
  • Subscribers
    • Subscription Information
    • Purchase Subscription
    • Your Account
    • Terms and Conditions
  • About Us
    • About Us
    • Editorial Board
    • Letter from the Editor
    • Duncan A. Gordon Award
    • Privacy/GDPR Policy
    • Accessibility
  • Contact Us
  • JRheum Supplements
  • Services

User menu

  • My Cart
  • Log In

Search

  • Advanced search
The Journal of Rheumatology
  • JRheum Supplements
  • Services
  • My Cart
  • Log In
The Journal of Rheumatology

Advanced Search

  • Home
  • Content
    • First Release
    • Current
    • Archives
    • Collections
    • Audiovisual Rheum
    • 50th Volume Reprints
  • Resources
    • Guide for Authors
    • Submit Manuscript
    • Payment
    • Reviewers
    • Advertisers
    • Classified Ads
    • Reprints and Translations
    • Permissions
    • Meetings
    • FAQ
    • Policies
  • Subscribers
    • Subscription Information
    • Purchase Subscription
    • Your Account
    • Terms and Conditions
  • About Us
    • About Us
    • Editorial Board
    • Letter from the Editor
    • Duncan A. Gordon Award
    • Privacy/GDPR Policy
    • Accessibility
  • Contact Us
  • Follow Jrheum on BlueSky
  • Follow jrheum on Twitter
  • Visit jrheum on Facebook
  • Follow jrheum on LinkedIn
  • Follow jrheum on YouTube
  • Follow jrheum on Instagram
  • Follow jrheum on RSS
Research ArticleArticle

Development of Cardiovascular Quality Indicators for Rheumatoid Arthritis: Results from an International Expert Panel Using a Novel Online Process

Claire E.H. Barber, Deborah A. Marshall, Nanette Alvarez, G.B. John Mancini, Diane Lacaille, Stephanie Keeling, J. Antonio Aviña-Zubieta, Dmitry Khodyakov, Cheryl Barnabe, Peter Faris, Alexa Smith, Raheem Noormohamed, Glen Hazlewood, Liam O. Martin, John M. Esdaile and the Quality Indicator International Panel
The Journal of Rheumatology September 2015, 42 (9) 1548-1555; DOI: https://doi.org/10.3899/jrheum.141603
Claire E.H. Barber
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: cehbarbe@ucalgary.ca
Deborah A. Marshall
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nanette Alvarez
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
G.B. John Mancini
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Diane Lacaille
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Stephanie Keeling
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
J. Antonio Aviña-Zubieta
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Dmitry Khodyakov
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Cheryl Barnabe
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peter Faris
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Alexa Smith
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Raheem Noormohamed
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Glen Hazlewood
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Liam O. Martin
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John M. Esdaile
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Data
  • Supplemental
  • Info & Metrics
  • References
  • PDF
PreviousNext
Loading

Abstract

Objective. Patients with rheumatoid arthritis (RA) have a high risk of premature cardiovascular disease (CVD). We developed CVD quality indicators (QI) for screening and use in rheumatology clinics.

Methods. A systematic review was conducted of the literature on CVD risk reduction in RA and the general population. Based on the best practices identified from this review, a draft set of 12 candidate QI were presented to a Canadian panel of rheumatologists and cardiologists (n = 6) from 3 academic centers to achieve consensus on the QI specifications. The resulting 11 QI were then evaluated by an online modified-Delphi panel of multidisciplinary health professionals and patients (n = 43) to determine their relevance, validity, and feasibility in 3 rounds of online voting and threaded discussion using a modified RAND/University of California, Los Angeles Appropriateness Methodology.

Results. Response rates for the online panel were 86%. All 11 QI were rated as highly relevant, valid, and feasible (median rating ≥ 7 on a 1–9 scale), with no significant disagreement. The final QI set addresses the following themes: communication to primary care about increased CV risk in RA; CV risk assessment; defining smoking status and providing cessation counseling; screening and addressing hypertension, dyslipidemia, and diabetes; exercise recommendations; body mass index screening and lifestyle counseling; minimizing corticosteroid use; and communicating to patients at high risk of CVD about the risks/benefits of nonsteroidal antiinflammatory drugs.

Conclusion. Eleven QI for CVD care in patients with RA have been developed and are rated as highly relevant, valid, and feasible by an international multidisciplinary panel.

Key Indexing Terms:
  • RHEUMATOID ARTHRITIS
  • CARDIOVASCULAR DISEASES
  • HEALTHCARE QUALITY INDICATORS
  • PRIMARY PREVENTION

Patients with rheumatoid arthritis (RA) have about a 50% higher mortality rate if they have a cardiovascular event compared to the general population1. Although chronic inflammation probably contributes to premature atherosclerosis and endothelial dysfunction2, improvements in treatment for RA have not consistently translated into reduced cardiovascular disease (CVD) events3,4. Some authors have recently proposed there may be a trend toward a “widening mortality gap” in RA due to CVD, because survival rates from CV events have improved in the general population but have remained stagnant in RA populations4,5.

Despite general recognition that patients with RA are at increased risk of CVD events, there is ample evidence that even well-established CVD risk factors such as hypertension (HTN) and dyslipidemia are not identified and managed consistently in RA populations6,7,8,9,10,11,12,13,14,15. One strategy to reduce CVD risk in RA is therefore to better identify and manage modifiable risk factors such as smoking, HTN, obesity and dyslipidemia.

Quality indicators (QI) are statements about optimum process or outcomes of care that can be further developed into performance measures. Performance measures have a specified numerator, denominator, and exclusion criteria. They can be used for measuring the percent adherence to the process or the proportion of patients achieving the desired outcome and are critical for quality improvement initiatives16.

QI are often based on guidelines from national or international professional societies but have a greater specificity than the recommendations from which they are derived16,17. They can represent either an optimum or minimum standard of care depending on the intended use of the measure (quality improvement versus accountability, e.g., in the form of pay for performance or use for accreditation). Although a variety of methodologies exist for developing QI17, a key feature of indicator development is the use of high-grade evidence combined with expert opinion.

Current QI for RA focus primarily on the treatment of joint inflammation and the monitoring of drug side effects18,19,20. To date, they have not completely addressed the most important cause of mortality in this population: CVD. The objective of our study was to develop a set of QI to address CV risk factor management in patients with RA for the purposes of quality improvement and research. The intended audience for the proposed indicators is healthcare professionals caring for patients with RA, primarily rheumatologists, and other rheumatology providers (e.g., rheumatology nurses, clinical nurse specialists, or nurse practitioners). In addition, the indicators are highly relevant to cardiologists, internists, and primary care practitioners responsible for the medical management of patients with RA.

MATERIALS AND METHODS

The approach used to develop the CVD QI is shown in Figure 1. The development of the CVD QI for RA involved 3 phases: (1) a systematic review to define the existing best practices for CVD preventive care in RA and drafting of the candidate indicators, (2) an in-person expert panel meeting with 6 experts to achieve consensus regarding the scope, wording, and specifications of the candidate measures, and (3) an international online modified-Delphi panel (n = 43) using a novel RAND-developed platform to finalize the indicators and rate the relevance, validity, and feasibility to RA care, while also obtaining feedback about whether the measures would be used in clinical practice.

Figure 1.
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1.

Methods for developing the quality indicator set for cardiovascular care in rheumatoid arthritis.

Phase 1

This phase consisted of a systematic review of existing guidelines and indicators and development of background reports. Quality measures are often developed from existing guideline recommendations17. Therefore in Phase 1, a systematic review of existing guidelines and quality indicators in both the general population and RA literature was conducted and relevant manuscripts from 5 years (2008–2013) were included. The detailed methods and results of the systematic review are published elsewhere21. The results of the systematic review were used to identify best practices for CV care in patients with RA and to inform the development of candidate QI. The QI were worded in a standard format (If-Then-Because) to identify the clinical situation of interest (If), the recommendation (Then), and the evidence and rationale for the QI (Because). A clear numerator, denominator, and exclusion criteria were also identified for each QI to ensure accurate measurement of the indicator upon application in a routine clinical practice setting22.

A report describing each candidate QI and its specifications as well as the supporting guidelines and reported level of evidence were generated for subsequent stages.

Phase 2

An expert consensus meeting was held to finalize the scope, wording, and specifications of candidate indicators. The candidate indicators and associated reports were presented to a select group of cardiologists and rheumatologists from 3 academic centers in Canada (NA, GBJM, DL, SK, JAA-Z, JME). These individuals represented a convenience sample of clinicians and researchers with an expertise in CVD in RA and/or quality improvement. The specifications and wording of the indicators were refined in an in-person meeting and consensus was achieved through an iterative process. These experts were not part of the online modified-Delphi panel in Phase 3.

Phase 3

The candidate QI were presented to an international panel using an innovative, iterative, online, previously used platform called ExpertLens23,24. The platform was used to conduct a modified RAND/University of California at Los Angeles (UCLA) appropriateness method to finalize the QI. ExpertLens has been used to elicit expert opinion on a range of healthcare topics, including the identification of definitional features of Continuous Quality Improvement25 and of aspirational research goals for preventing suicide26. Nonetheless, this is the first time, to our knowledge, that the platform has been used for QI development.

ExpertLens combines a number of approaches including the Delphi, Nominal Group, and Crowdsourcing techniques23. With it, a larger number of panelists can be included than in typical RAND/UCLA appropriateness panels, enabling diverse geographical representation, and the findings have been shown to be reproducible among different groups24. The online panel is therefore more cost-efficient and time-efficient than typical large international consensus meetings. Additional benefits of the system are the use of unique identifiers, which can avoid dominance of the group by a small number of vocal individuals23.

Online panel composition and recruitment

A diverse group of 43 experts from North America and Europe were invited to participate including cardiologists, rheumatologists, and primary care physicians from both academic and nonacademic practices. Pharmacists, nurses, clinician scientists, and patients were also represented. Although many participants were recruited based on their prior publications in the area of CVD in RA, effort was also made to include clinicians in community practices and other types of participants, such as patients. These individuals were identified through a variety of means including national societies (e.g., Canadian Rheumatology Association, Allied Health Professions Association), patient advocacy groups (e.g., Arthritis Patient Advisory Board of the Arthritis Research Center of Canada) as well as “snowball recruitment” (chain-referral sampling). Participants were recruited by an e-mail invitation and agreed to participate prior to the panel start date. Participants did not receive financial incentives. The University of Calgary Conjoint Health Research Ethics Board (ethics identification REB13-0210) approved the project and the RAND Human Subjects’ Protection Committee determined this study to be exempt from review.

Panel rating of relevance, validity, feasibility, and likelihood of use of the QI

Between November 4 and December 3, 2013, participants took part in a 3-round ExpertLens process. In Round 1, participants rated QI on 4 criteria (see Table 1 for description of criteria). In Round 2, they reviewed the automatically generated distribution of the group’s responses to each question that included (1) a bar chart showing the frequency of each response category for the group, (2) a group median, (3) interquartile range, and (4) the participant’s own response to that question (see Supplementary Figure 1, available online at jrheum.org). Participants also were encouraged to participate in an online, asynchronous, anonymous discussion moderated by one of the authors to ensure they remained on-topic and constructive. Finally, in Round 3, participants were requested to revise their Round 1 responses based on group feedback and discussion and share their study experiences by answering a brief series of satisfaction questions. Each round was open for 7 to 14 days, depending on participation rates. Periodic reminders to participate were sent by e-mail to maximize participation.

View this table:
  • View inline
  • View popup
Table 1.

Description of criteria used to select quality indicators.

Participants were asked to rate candidate QI on the following 4 rating criteria during Round 1 and again in Round 3. The first 2 criteria have been used previously to assess validity and feasibility27,28; the third and fourth criteria were formulated during Phase 2 with the expert panelists to assess relevance and likelihood of use. All criteria were Likert-type 1–9 scales with labeled endpoints. Detailed definitions of each criterion are shown in Table 1.

Analysis of panelist responses

To be included in the final set, indicators had to be rated as highly valid and feasible (median validity and feasibility scores ≥ 7), with no disagreement among participants. Disagreement was calculated using a formula that examines the distribution of the ratings according to the RAND/UCLA Appropriateness Method handbook29. Disagreement exists when the interpercentile range (IPR; difference between the 30th and 70th percentiles) is larger than the IPR adjusted for symmetry (IPRAS), which was calculated using the following formula: Embedded Image

Derivation of the formula is shown in the RAND/UCLA Appropriateness Method handbook29.

RESULTS

The results of the systematic review of existing guidelines and QI are presented elsewhere21. Briefly, recommendations and indicators were abstracted from both RA and general population guidelines that were relevant to primary CVD prevention (e.g., CVD risk assessment, lipid, diabetes and HTN screening, exercise, smoking and lifestyle counseling).

Based on the systematic review results, a set of 12 candidate indicators was drafted and presented to the Phase 2 small expert working group of 2 cardiologists and 4 rheumatologists at a meeting (Phase 2, Figure 1). The 12 drafted indicators encompassed the following topics: communication of the importance of CVD care in RA to the primary care provider, CVD risk assessment, smoking status and counseling for cessation, screening for HTN, communicating to a primary care provider about an elevated blood pressure, blood pressure control, measurement of a fasting lipid profile, dietary counseling, exercise counseling, corticosteroid tapering to lowest dose, and avoidance of nonsteroidal antiinflammatory drugs in patients at high risk of CVD events.

The expert panel in Phase 2 made recommendations as to the wording of the indicators and the specifications. Major expert panel recommendations included the following:

  • Outcome measures (or interim outcome measures) such as blood pressure targets should not be included in the QI set because such measures were not considered in the rheumatologist’s scope of practice

  • Maintaining a low disease activity state or remission is important in reducing CVD risk in patients with RA; however, general treatment of RA and measurement of disease activity is not within the scope of these QI because of the focus on QI that especially address CVD

  • All physicians who care for a patient with RA share responsibility for CVD risk management. Many of the QI were designed to measure communication between the rheumatologists and other care providers, to encourage a cohesive approach to monitoring and caring for CVD risk and to recognize that it is not necessarily the primary responsibility of the rheumatologists

  • Frequency of conducting a formal CVD risk assessment should follow minimum intervals, which were proposed in accordance with guideline recommendations. The timing of other measures where there was no guideline for the frequency of measurement was based on the panel’s consensus

  • Standalone QI that recommended only measuring a risk factor without a specific action (e.g., measuring a lipid profile or ascertaining smoking status) were discouraged; it was decided that this may lead to clinical inertia (e.g., measuring more lipid profiles but not calculating CVD risk or appropriately treating if indicated)

  • A QI on screening for diabetes should be included

  • Body mass index should be measured instead of using the QI on dietary counseling, which was not within the purview of a rheumatologist.

A revised set of 11 QI was selected for presentation to the ExpertLens panelists.

ExpertLens panel participant characteristics and participation rates

Forty-three individuals were invited to participate in the online ExpertLens panel, of whom 37 participated (86.0%). The self-reported characteristics of the ExpertLens participants are shown in Table 2. Twenty-eight completed all 3 rounds (65%). There were 4 participants who completed Round 1, but not Round 3, and 5 who completed Round 3, but not Round 1. During Round 2 (online discussion board), there were 24 discussion threads and 113 discussion comments. Because this demographic information was asked during Round 1, a maximum of 32 respondents answered these questions. In a sensitivity analysis, we analyzed the responses from all participants who completed Round 3 and also those who completed both Round 1 and 3; there were no significant differences (in Table 3 we report the results from individuals who participated in both rounds).

View this table:
  • View inline
  • View popup
Table 2.

Self-reported ExpertLens participant characteristics (Round 1*).

View this table:
  • View inline
  • View popup
Table 3.

Final validity, feasibility, and relevance ratings from the ExpertLens panel on the 11 cardiovascular quality indicators (QI)

CVD QI

After Round 2, a few minor changes to the QI were made based on feedback from earlier rounds and presented to the panel in Round 3. For example, for QI 4 (Screening for HTN), the specifications of the measure originally suggested at a minimum measuring the blood pressure once per year; however, panelists expressed during Round 2 discussions that this was not frequent enough (especially since many rheumatologic medications affect blood pressure) and the measure was modified to better reflect guideline recommendations, which suggest more frequent screening.

In Round 3, all 11 CVD QI were rated as highly relevant, valid, and feasible by the panelists without significant disagreement (Table 3). Participants also agreed that they were likely to advocate for the QI to be used in local quality improvement initiatives.

The final indicator statements are shown in Table 4, and the full specifications, including descriptions of the numerator, denominator, and relevant exclusions, are shown in Supplementary Table 1, available online at jrheum.org.

View this table:
  • View inline
  • View popup
Table 4.

Final set of 11 cardiovascular (CV) disease quality indicators for patients with rheumatoid arthritis (RA).

DISCUSSION

The proposed set of 11 CVD QI for RA has been agreed upon as highly relevant, valid, and feasible for measurement and quality improvement initiatives in RA by a large international, multidisciplinary panel of healthcare professionals and patients. The recommended QI comprehensively cover many aspects of CVD preventive care in RA and are evidence-based and aligned with current high-quality guidelines, based on our systematic review21. Where RA guidelines were lacking in evidence to support these measures, it was decided that at a minimum, general population guidelines should be followed and these were used to support and help define the QI.

The 11 QI define processes important to CVD preventive care in RA. Risk-adjusted outcome or interim outcome measures (e.g., lipid or blood pressure targets) were not felt to be within the scope of this project because it was determined that screening for CVD risk factors and primary prevention should be a shared responsibility and not solely the responsibility of the rheumatologist. Importantly, in the United States, the National Quality Strategy recognizes care coordination as an important gap in quality measurement30. The set of CVD QI are unique because they emphasize communication between providers in caring for patients with RA, which we hope will improve coordination of comorbidity care for patients with RA.

Our work represents the first time ExpertLens has been used for QI development and shows that the online platform had a number of advantages. Typical RAND/UCLA appropriateness panels used for QI development frequently have a limited number of participants (often 9), which limits the diversity among participants and may prevent inclusion of informed patients because included participants are often experts29. By using the online platform, we were able to get broader representation from a diverse group of international participants. Unfortunately, not all recruited participants ended up providing their expert opinion. This may have been for a variety of reasons. Some participants may not have been available during panel times or decided not to participate. Others may not have received ExpertLens invitation e-mails, which could have been re-routed to their spam folders. Nonetheless, our overall participation rate of 86% was excellent and compared favorably with participation rates in other ExpertLens24 panels and online Delphi panels with fewer rounds31.

An advantage of the online platform was the ability to anonymously obtain responses and discussion threads on topics from participants, which avoided dominance of the discussion by a subset of participants. Some participants commented, however, that they would have liked more time to discuss the QI or would have benefited from a conference call to review certain aspects of the measures. However, by Round 3, general consensus was achieved, and it is unknown whether further discussion in person or online would have altered the measures significantly.

To mitigate the potential disadvantages alluded to above of holding a QI development panel entirely online, we first had a small meeting of experts to review the QI specifications and wording. Based on the recommendations of this group, QI that only measured clinician documentation of a risk factor, e.g., smoking status or lipid profile, were discouraged because it was expected that they would be unlikely to lead to quality improvement if clinicians were not prompted to “do something” when a risk factor was identified. Therefore, some of the proposed measures included more than 1 measurement concept (e.g., documenting smoking status AND recommending smoking cessation). This potentially made voting challenging for some of the online participants who may have agreed with 1 part of the QI but not another and consequently may have led to lower ratings for some of the presented QI. Because the measures are for quality improvement and not accountability, it is reasonable that a measurement concept be coupled with an action concept to avoid clinical inertia, while recognizing that it increases the complexity of executing and practically assessing the measure.

An additional strength of this work is the diversity of individuals who participated from around the world. Nonetheless, it should be noted that recruitment of individuals from community practice (especially rural practice), as well as certain physician types (e.g., general internists and primary care practitioners) was challenging because these individuals were harder to identify and recruit. Consequently, these groups were underrepresented. As shown in the literature32, it is possible that panels with a different composition could have voted differently on the proposed indicators. In this case, it is possible that feasibility ratings for some of the indicators may have been lower if the panel composition had included more rheumatologists in community practice. We encourage pilot testing of the indicators and selection of the most appropriate measures depending on the clinical setting.

We plan to further validate the QI in rheumatology practice. This further work will involve evaluation of the feasibility of measuring the indicators in different practice settings, measurement of interrater reliability, assessing whether a gap in care exists, and determining how best to implement improvements22,33.

Patients with RA have a significantly higher rate of death due to CVD than the general population. Ensuring high-quality CVD preventive care for patients with RA is one method of potentially mitigating this risk and is in keeping with current RA and general population guidelines. In this study, we proposed a comprehensive set of 11 CVD QI for patients with RA for the purposes of quality improvement and research. Our work represents the first time, to our knowledge, that ExpertLens has been used for QI development and shows that the online platform had a number of advantages and is a useful tool for QI development.

ONLINE SUPPLEMENT

Supplementary data for this article are available online at jrheum.org.

APPENDIX

Appendix

Members of the Quality Indicator International Panel: Todd J. Anderson, MD, FRCPC, Professor of Medicine, Head, Department of Cardiac Sciences, Alberta Health Services and the University of Calgary, Director, Libin Cardiovascular Institute; Sandeep Aggarwal, MD, FRCPC, Associate Clinical Professor, University of Calgary; Pooneh Akhavan, MD, MSc, Division of Rheumatology, Mount Sinai Hospital, Toronto; Christie Bartels, MD, MS, Assistant Professor Department of Medicine, Rheumatology Division, University of Wisconsin School of Medicine and Public Health; Carolyn Bell, MBBS, London, MRCP Rheumatologist for Worcestershire Royal Hospital, Worcester, UK; Gilles Boire, MD, MSc, Professor, Department of Medicine, Rheumatology Division, University of Sherbrooke; Ailsa Bosworth, Chief Executive, National Rheumatoid Arthritis Society; Linda Brown, RN, Alberta Health Services; Alexandra Charlton, BScPharm, ACPR Pharmacy Clinical Practice Leader, and Pharmacist, Division of Rheumatology, Alberta Health Services; Shirley Chow, MD, FRCPC, Assistant Professor of Medicine, Division of Rheumatology, Department of Medicine, University of Toronto; Cynthia Crowson, MS, Statistician III, Division of Biomedical Statistics and Informatics Associate Professor of Medicine (Rheumatology) and Assistant Professor of Biostatistics, Mayo Clinic; Milan Gupta, MD, FRCPC, Associate Professor of Medicine, McMaster University, Hamilton, and Medical Director, Canadian Cardiovascular Research Network, Brampton, Ontario, Canada; John G. Hanly, MD, FRCPC, Professor of Medicine and Pathology, Dalhousie University and attending staff rheumatologist, Capital Health, Halifax, Nova Scotia, Canada; Alison M. Hoens, MSc, BScPT, Physical Therapy Knowledge Broker, University of British Columbia Department of Physical Therapy and Member, Arthritis Patient Advisory Board of the Arthritis Research Center of Canada; Wes Jackson, MD, CCFP, FCFP, Assistant Professor, Department of Medicine, University of Calgary; George D. Kitas, MD, PhD, FRCP, Director of Research and Development, Dudley Group, UK National Health Service Foundation Trust and Professor of Clinical Rheumatology, Arthritis Research UK Epidemiology Unit, University of Manchester, UK; Tabitha N. Kung, MD, FRCPC, MPH, Clinical Associate, Division of Rheumatology, Department of Medicine, University of Toronto; Theresa Lupton, RN, CCRP, Nurse Clinician Rheumatology, Alberta Health Services; Paul MacMullan, MB, BCh, BAO, MRCPI, Clinical Assistant Professor, Division of Rheumatology, Department of Medicine, University of Calgary; Sherif Moustafa, MBBCh, Mayo Clinic Arizona; Michael T. Nurmohamed, MD, PhD, Associate Professor, departments of Rheumatology and Internal Medicine, VU University Medical Center, Amsterdam, the Netherlands; Sara Partington, MD, Assistant Professor of Clinical Medicine, University of Pennsylvania; Athanase D. Protogerou, MD, Hypertension Unit and Cardiovascular Research Laboratory, 1st Department of Propaedeutic Internal Medicine, Laiko Hospital, Medical School, National and Kapodistrian University of Athens, Greece; Anne-Grete Semb, MD, PhD, Preventive Cardio-Rheuma clinic, Department of Rheumatology, Diakonhjemmet Hospital, Oslo, Norway; Amanda J. Steiman, MD, MSc, FRCPC, Lupus Clinical Research Fellow, Division of Rheumatology, Toronto Western Hospital; Lisa Gale Suter, MD, Assistant Professor of Medicine, Section of Rheumatology, Yale School of Medicine; Deborah Symmons, MD, FFPH, FRCP, Professor of Rheumatology and Musculoskeletal Epidemiology, Arthritis Research UK Centre for Epidemiology; Jinoos Yazdany, MD, MPH, Associate Professor of Medicine, Division of Rheumatology, Department of Medicine, University of California, San Francisco.

Footnotes

  • Funded by a grant from the Canadian Institutes for Health Research (CIHR; funding ref. # 129628). Dr. Barber is a PhD candidate and her work is supported by a Health Research Clinical Fellowship from Alberta Innovates Health Solutions 2011–15 as well as a Rheumatology Postgraduate Fellowship funded by UCB Canada, The Canadian Rheumatology Association, and The Arthritis Society from 2011–13, and a Vanier Canada Graduate Scholarship 2014–15. Dr. Marshall is a CIHR Canada Research Chair in Health Services and Systems Research and the Arthur J.E. Child Chair in Rheumatology Outcomes Research.

  • Accepted for publication March 27, 2015.

REFERENCES

  1. 1.↵
    1. Aviña-Zubieta JA,
    2. Choi HK,
    3. Sadatsafavi M,
    4. Etminan M,
    5. Esdaile JM,
    6. Lacaille D
    . Risk of cardiovascular mortality in patients with rheumatoid arthritis: a meta-analysis of observational studies. Arthritis Rheum 2008;59:1690–7.
    OpenUrlCrossRefPubMed
  2. 2.↵
    1. Full LE,
    2. Monaco C
    . Targeting inflammation as a therapeutic strategy in accelerated atherosclerosis in rheumatoid arthritis. Cardiovasc Ther 2011;29:231–42.
    OpenUrlCrossRefPubMed
  3. 3.↵
    1. Bergstrom U,
    2. Jacobsson LT,
    3. Turesson C
    . Cardiovascular morbidity and mortality remain similar in two cohorts of patients with long-standing rheumatoid arthritis seen in 1978 and 1995 in Malmo, Sweden. Rheumatology 2009;48:1600–5.
    OpenUrlAbstract/FREE Full Text
  4. 4.↵
    1. Radovits BJ,
    2. Fransen J,
    3. Al Shamma S,
    4. Eijsbouts AM,
    5. van Riel PL,
    6. Laan RF
    . Excess mortality emerges after 10 years in an inception cohort of early rheumatoid arthritis. Arthritis Care Res 2010;62:362–70.
    OpenUrlCrossRef
  5. 5.↵
    1. Gonzalez A,
    2. Maradit Kremers H,
    3. Crowson CS,
    4. Nicola PJ,
    5. Davis JM 3rd,
    6. Therneau TM,
    7. et al.
    The widening mortality gap between rheumatoid arthritis patients and the general population. Arthritis Rheum 2007;56:3583–7.
    OpenUrlCrossRefPubMed
  6. 6.↵
    1. Desai SS,
    2. Myles JD,
    3. Kaplan MJ
    . Suboptimal cardiovascular risk factor identification and management in patients with rheumatoid arthritis: a cohort analysis. Arthritis Res Ther 2012;14:R270.
    OpenUrlCrossRefPubMed
  7. 7.↵
    1. Dougados M,
    2. Soubrier M,
    3. Antunez A,
    4. Balint P,
    5. Balsa A,
    6. Buch MH,
    7. et al.
    Prevalence of comorbidities in rheumatoid arthritis and evaluation of their monitoring: results of an international, cross-sectional study (COMORA). Ann Rheum Dis 2014;73:62–8.
    OpenUrlAbstract/FREE Full Text
  8. 8.↵
    1. Gossec L,
    2. Salejan F,
    3. Nataf H,
    4. Nguyen M,
    5. Gaud-Listrat V,
    6. Hudry C,
    7. et al.
    Challenges of cardiovascular risk assessment in the routine rheumatology outpatient setting: an observational study of 110 rheumatoid arthritis patients. Arthritis Care Res 2013;65:712–7.
    OpenUrlCrossRef
  9. 9.↵
    1. Keeling SO,
    2. Teo M,
    3. Fung D
    . Lack of cardiovascular risk assessment in inflammatory arthritis and systemic lupus erythematosus patients at a tertiary care center. Clin Rheumatol 2011;30:1311–7.
    OpenUrlCrossRefPubMed
  10. 10.↵
    1. Toms TE,
    2. Panoulas VF,
    3. Douglas KM,
    4. Griffiths H,
    5. Sattar N,
    6. Smith JP,
    7. et al.
    Statin use in rheumatoid arthritis in relation to actual cardiovascular risk: evidence for substantial undertreatment of lipid-associated cardiovascular risk? Ann Rheum Dis 2010;69:683–8.
    OpenUrlAbstract/FREE Full Text
  11. 11.↵
    1. Scott IC,
    2. Ibrahim F,
    3. Johnson D,
    4. Scott DL,
    5. Kingsley GH
    . Current limitations in the management of cardiovascular risk in rheumatoid arthritis. Clin Exp Rheumatol 2012;30:228–32.
    OpenUrlPubMed
  12. 12.↵
    1. Bartels CM,
    2. Kind AJ,
    3. Everett C,
    4. Mell M,
    5. McBride P,
    6. Smith M
    . Low frequency of primary lipid screening among medicare patients with rheumatoid arthritis. Arthritis Rheum 2011;63:1221–30.
    OpenUrlCrossRefPubMed
  13. 13.↵
    1. Bartels CM,
    2. Johnson H,
    3. Voelker K,
    4. Thorpe C,
    5. McBride P,
    6. Jacobs EA,
    7. et al.
    Impact of rheumatoid arthritis on receiving a diagnosis of hypertension among patients with regular primary care. Arthritis Care Res 2014;66:1281–8.
    OpenUrlCrossRef
  14. 14.↵
    1. Protogerou AD,
    2. Panagiotakos DB,
    3. Zampeli E,
    4. Argyris AA,
    5. Arida K,
    6. Konstantonis GD,
    7. et al.
    Arterial hypertension assessed “out-of-office” in a contemporary cohort of rheumatoid arthritis patients free of cardiovascular disease is characterized by high prevalence, low awareness, poor control and increased vascular damage-associated “white coat” phenomenon. Arthritis Res Ther 2013;15:R142.
    OpenUrlCrossRefPubMed
  15. 15.↵
    1. Panoulas VF,
    2. Douglas KM,
    3. Milionis HJ,
    4. Stavropoulos-Kalinglou A,
    5. Nightingale P,
    6. Kita MD,
    7. et al.
    Prevalence and associations of hypertension and its control in patients with rheumatoid arthritis. Rheumatology 2007;46:1477–82.
    OpenUrlAbstract/FREE Full Text
  16. 16.↵
    1. McGlynn EA,
    2. Asch SM
    . Developing a clinical performance measure. Am J Prev Med 1998;14:14–21.
    OpenUrlCrossRefPubMed
  17. 17.↵
    1. Kotter T,
    2. Blozik E,
    3. Scherer M
    . Methods for the guideline-based development of quality indicators—a systematic review. Implement Sci 2012;7:21.
    OpenUrlCrossRefPubMed
  18. 18.↵
    1. MacLean CH,
    2. Saag KG,
    3. Solomon DH,
    4. Morton SC,
    5. Sampsel S,
    6. Klippel JH
    . Measuring quality in arthritis care: methods for developing the Arthritis Foundation’s quality indicator set. Arthritis Rheum 2004;51:193–202.
    OpenUrlCrossRefPubMed
  19. 19.↵
    1. Khanna D,
    2. Arnold EL,
    3. Pencharz JN,
    4. Grossman JM,
    5. Traina SB,
    6. Lal A,
    7. et al.
    Measuring process of arthritis care: the Arthritis Foundation’s quality indicator set for rheumatoid arthritis. Semin Arthritis Rheum 2006;35:211–37.
    OpenUrlCrossRefPubMed
  20. 20.↵
    1. Strombeck B,
    2. Petersson IF,
    3. Vliet Vlieland TP;
    4. EUMUSC.net WP6 group
    . Health care quality indicators on the management of rheumatoid arthritis and osteoarthritis: a literature review. Rheumatology 2013;52:382–90.
    OpenUrlAbstract/FREE Full Text
  21. 21.↵
    1. Barber CE,
    2. Smith A,
    3. Esdaile JM,
    4. Barnabe C,
    5. Martin LO,
    6. Faris P,
    7. et al.
    Best practices for cardiovascular disease prevention in rheumatoid arthritis: a systematic review of guideline recommendations and quality indicators. Arthritis Care Res 2015;67:169–79.
    OpenUrlCrossRef
  22. 22.↵
    1. Campbell SM,
    2. Braspenning J,
    3. Hutchinson A,
    4. Marshall MN
    . Research methods used in developing and applying quality indicators in primary care. BMJ 2003;326:816–9.
    OpenUrlFREE Full Text
  23. 23.↵
    1. Dalal S,
    2. Khodyakov D,
    3. Srinivasan R,
    4. Straus S,
    5. Adams J
    . ExpertLens: A system for eliciting opinions from a large pool of non-collocated experts with diverse knowledge. Technol Forecast Soc 2011;78:1426–44.
    OpenUrlCrossRef
  24. 24.↵
    1. Khodyakov D,
    2. Hempel S,
    3. Rubenstein L,
    4. Shekelle P,
    5. Foy R,
    6. Salem-Schatz S,
    7. et al.
    Conducting online expert panels: a feasibility and experimental replicability study. BMC Med Res Methodol 2011;11:174.
    OpenUrlCrossRefPubMed
  25. 25.↵
    1. Rubenstein L,
    2. Khodyakov D,
    3. Hempel S,
    4. Danz M,
    5. Salem-Schatz S,
    6. Foy R,
    7. et al.
    How can we recognize continuous quality improvement? Int J Qual Health Care 2014;26:6–15.
    OpenUrlAbstract/FREE Full Text
  26. 26.↵
    1. Claassen CA,
    2. Pearson JL,
    3. Khodyakov D,
    4. Satow PM,
    5. Gebbia R,
    6. Berman AL,
    7. et al.
    Reducing the burden of suicide in the U.S.: the aspirational research goals of the National Action Alliance for Suicide Prevention Research Prioritization Task Force. Am J Prev Med 2014;47:309–14.
    OpenUrlCrossRefPubMed
  27. 27.↵
    1. Yazdany J,
    2. Panopalis P,
    3. Gillis JZ,
    4. Schmajuk G,
    5. MacLean CH,
    6. Wofsy D,
    7. et al.
    A quality indicator set for systemic lupus erythematosus. Arthritis Rheum 2009;61:370–7.
    OpenUrlCrossRefPubMed
  28. 28.↵
    1. Mangione-Smith R,
    2. DeCristofaro AH,
    3. Setodji CM,
    4. Keesey J,
    5. Klein DJ,
    6. Adams JL,
    7. et al.
    The quality of ambulatory care delivered to children in the United States. N Engl J Med 2007;357:1515–23.
    OpenUrlCrossRefPubMed
  29. 29.↵
    1. Fitch K,
    2. Bernstein SJ,
    3. Aguilar MD,
    4. Burnand B,
    5. LaCalle JR,
    6. Lazaro P,
    7. et al.
    The RAND/UCLA appropriateness method user’s manual. Santa Monica, CA: RAND Corporation; 2001.
  30. 30.↵
    1. US Department of Health and Human Services (HHS) Public Reporting of Quality and Efficiency Measures Workgroup
    . Prioritizing the public in HHS public reporting: using and improving information on quality, cost, and coverage to support health care improvement. 2013. [Internet. Accessed May 29, 2015.] Available from: www.ahrq.gov/workingforquality/reports/hhsreporting.pdf
  31. 31.↵
    1. Elwyn G,
    2. O’Connor A,
    3. Stacey D,
    4. Volk R,
    5. Edwards A,
    6. Coulter A,
    7. et al.
    Developing a quality criteria framework for patient decision aids: online international Delphi consensus process. BMJ 2006;333:417.
    OpenUrlAbstract/FREE Full Text
  32. 32.↵
    1. Coulter I,
    2. Adams A,
    3. Shekelle P
    . Impact of varying panel membership on ratings of appropriateness in consensus panels: a comparison of a multi- and single disciplinary panel. Health Serv Res 1995;30:577–91.
    OpenUrlPubMed
  33. 33.↵
    1. Campbell SM,
    2. Kontopantelis E,
    3. Hannon K,
    4. Burke M,
    5. Barber A,
    6. Lester HE
    . Framework and indicator testing protocol for developing and piloting quality indicators for the UK quality and outcomes framework. BMC Fam Pract 2011;12:85.
    OpenUrlCrossRefPubMed
PreviousNext
Back to top

In this issue

The Journal of Rheumatology
Vol. 42, Issue 9
1 Sep 2015
  • Table of Contents
  • Table of Contents (PDF)
  • Index by Author
  • Editorial Board (PDF)
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word about The Journal of Rheumatology.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Development of Cardiovascular Quality Indicators for Rheumatoid Arthritis: Results from an International Expert Panel Using a Novel Online Process
(Your Name) has forwarded a page to you from The Journal of Rheumatology
(Your Name) thought you would like to see this page from the The Journal of Rheumatology web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Development of Cardiovascular Quality Indicators for Rheumatoid Arthritis: Results from an International Expert Panel Using a Novel Online Process
Claire E.H. Barber, Deborah A. Marshall, Nanette Alvarez, G.B. John Mancini, Diane Lacaille, Stephanie Keeling, J. Antonio Aviña-Zubieta, Dmitry Khodyakov, Cheryl Barnabe, Peter Faris, Alexa Smith, Raheem Noormohamed, Glen Hazlewood, Liam O. Martin, John M. Esdaile, the Quality Indicator International Panel
The Journal of Rheumatology Sep 2015, 42 (9) 1548-1555; DOI: 10.3899/jrheum.141603

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero

 Request Permissions

Share
Development of Cardiovascular Quality Indicators for Rheumatoid Arthritis: Results from an International Expert Panel Using a Novel Online Process
Claire E.H. Barber, Deborah A. Marshall, Nanette Alvarez, G.B. John Mancini, Diane Lacaille, Stephanie Keeling, J. Antonio Aviña-Zubieta, Dmitry Khodyakov, Cheryl Barnabe, Peter Faris, Alexa Smith, Raheem Noormohamed, Glen Hazlewood, Liam O. Martin, John M. Esdaile, the Quality Indicator International Panel
The Journal of Rheumatology Sep 2015, 42 (9) 1548-1555; DOI: 10.3899/jrheum.141603
del.icio.us logo Twitter logo Facebook logo  logo Mendeley logo
  • Tweet Widget
  •  logo
Bookmark this article

Jump to section

  • Article
    • Abstract
    • MATERIALS AND METHODS
    • RESULTS
    • DISCUSSION
    • ONLINE SUPPLEMENT
    • Appendix
    • Footnotes
    • REFERENCES
  • Figures & Data
  • Supplemental
  • Info & Metrics
  • References
  • PDF

Keywords

RHEUMATOID ARTHRITIS
CARDIOVASCULAR DISEASES
HEALTHCARE QUALITY INDICATORS
PRIMARY PREVENTION

Related Articles

Cited By...

More in this TOC Section

  • Comparative Analysis of Damage Accrual in Lupus Nephritis Stratified by Biological Sex
  • Long-Term Effectiveness and Safety of Denosumab for Osteoporosis in Patients With Rheumatic Diseases
  • Clinically Inactive Disease and Remission in Patients With Juvenile Idiopathic Arthritis Receiving Tofacitinib: Post Hoc Analysis of a Phase III Trial
Show more Article

Similar Articles

Keywords

  • rheumatoid arthritis
  • CARDIOVASCULAR DISEASES
  • HEALTHCARE QUALITY INDICATORS
  • primary prevention

Content

  • First Release
  • Current
  • Archives
  • Collections
  • Audiovisual Rheum
  • COVID-19 and Rheumatology

Resources

  • Guide for Authors
  • Submit Manuscript
  • Author Payment
  • Reviewers
  • Advertisers
  • Classified Ads
  • Reprints and Translations
  • Permissions
  • Meetings
  • FAQ
  • Policies

Subscribers

  • Subscription Information
  • Purchase Subscription
  • Your Account
  • Terms and Conditions

More

  • About Us
  • Contact Us
  • My Alerts
  • My Folders
  • Privacy/GDPR Policy
  • RSS Feeds
The Journal of Rheumatology
The content of this site is intended for health care professionals.
Copyright © 2025 by The Journal of Rheumatology Publishing Co. Ltd.
Print ISSN: 0315-162X; Online ISSN: 1499-2752
Powered by HighWire