Abstract
Objective To develop and evaluate interventions to improve quality of care in 4 priority areas in an urban safety net adult rheumatology clinic serving a racially/ethnically and socioeconomically diverse patient population.
Methods The Institute for Healthcare Improvement’s Model for Improvement was used to redesign clinical processes to achieve prespecified benchmarks in the following areas from 2015 to 2017: 13-valent pneumococcal conjugate vaccine (PCV13) administration among immunocompromised patients; disease activity monitoring with the Clinical Disease Activity Index (CDAI) for patients with rheumatoid arthritis; latent tuberculosis infection (LTBI) screening for new biologic users with RA; and reproductive health counseling among women receiving potentially teratogenic medications. We measured performance for each using standardized metrics, defined as the proportion of eligible patients receiving recommended care.
Results There were 1205 patients seen in the clinic between 2015 and 2017. Regarding demographics, 71% were women, 88% identified as racial/ethnic minorities, and 45% were eligible for at least 1 of the quality measures. Shewart charts for the PCV13 and CDAI measures showed evidence of improved healthcare delivery over time. Benchmarks were achieved for the CDAI and LTBI measures with 93% and 91% performance, respectively. Performance for the PCV13 and reproductive health counseling measures was 78% and 46%, respectively, but did not meet prespecified improvement targets.
Conclusion Through an interprofessional approach, we were able to achieve durable improvements in key rheumatology quality measures largely by enhancing workflow, engaging nonphysician providers, and managing practice variation.
Despite recent trends to improve the quality of healthcare in the United States, receipt of basic healthcare services among patients with rheumatic conditions remains suboptimal. Examples include subpar vaccination of immunocompromised patients1,2, variable use of treat-to-target approaches in rheumatoid arthritis (RA)3,4, and underuse of osteoporosis screening and treatment5. Compounding these deficits are healthcare disparities that place vulnerable populations at greater risk for poor health outcomes6. Racial and ethnic minorities experience more severe disease in multiple autoimmune conditions, including systemic lupus erythematosus (SLE) and ankylosing spondylitis7,8,9, exacerbating socioeconomic barriers to health among these populations.
The American College of Rheumatology (ACR) and the National Quality Forum (NQF) have identified key quality measures to assist rheumatology practices in measuring and improving the quality of care across diverse clinical settings10, with the potential to reduce disparities and improve healthcare outcomes regardless of racial, ethnic, or socioeconomic background11. Research on the implementation of quality measures among vulnerable populations with rheumatic conditions in the US is limited, and only a few studies have focused attention on the opportunities and challenges afforded by safety net clinical settings12,13,14.
In our study, we used the Institute for Healthcare Improvement’s (IHI) Model for Improvement to facilitate process improvement and system redesign across several domains of healthcare in a safety net rheumatology clinic that serves a racially/ethnically and socioeconomically diverse patient population. Four areas were chosen by rheumatologists and clinic staff as high-priority clinical processes, including 13-valent pneumococcal conjugate vaccine (PCV13) administration among immunocompromised patients; regular disease activity monitoring with the Clinical Disease Activity Index (CDAI) for patients with RA; latent tuberculosis infection (LTBI) screening for new biologic users with RA; and reproductive health counseling for women receiving potentially teratogenic medications. We evaluated the success of our interventions and sought to identify generalizable strategies for implementing quality improvement (QI) in safety net settings.
MATERIALS AND METHODS
Study setting and interventions
This study took place in an academic rheumatology clinic at the Zuckerberg San Francisco General Hospital, a safety net hospital in San Francisco affiliated with the University of California, San Francisco (UCSF). An incentive for the project was a pay-for-performance initiative for MediCal clinics (California’s Medicaid program). The Performance Improvement Program (PIP) allowed individual clinics to select the most relevant clinic-specific quality measures with reasonable benchmarks over a specified time interval. Achievement of these targets resulted in small financial incentives, used by the rheumatology division to support faculty and staff salaries or other needs. Faculty rheumatologists and clinic staff were involved in the selection process of the 4 quality measures shown in Table 1, chosen for their feasibility, validity, and relevance to the clinic population10.
Description of quality measures in the safety net quality improvement program, including prespecified performance targets.
Interventions were planned and executed using the IHI’s Model for Improvement with Plan, Do, Study, Act (PDSA) methodology, in which small-scale cycles of change are implemented in a consecutive fashion to improve care15,16. All providers working in the clinic were included in the intervention: attending physicians, rheumatology fellows, nurses, and medical assistants. UCSF medical students had a particularly active role in planning, executing, and evaluating QI interventions as part of their medical school curriculum. Patients were also involved in the early planning phases of these QI interventions, especially in providing feedback on workflow and developing educational materials. Data from the electronic health record (EHR) were extracted on a quarterly basis to evaluate performance on the quality measures. This investigation was considered exempt from institutional review board approval because it qualified as a QI project.
PCV13 quality measure
Vaccination with PCV13 is recommended by the Advisory Committee on Immunization Practices for all adults older than 18 years with immunocompromising conditions17. Prior to our study, the clinic lacked a standardized protocol to ensure administration of this vaccine.
We measured the proportion of patients age ≥ 18 years taking immunosuppressive medications with documented PCV13 vaccination from February 2015 to March 2017. Immunosuppressive medications included biologic agents [abatacept (ABA), adalimumab (ADA), anakinra, certolizumab pegol (CZP), etanercept (ETN), golimumab (GOL), infliximab (IFX), rituximab (RTX), tocilizumab (TCZ)] and nonbiologic medications [tofacitinib, azathioprine, cyclophosphamide (CYC), cyclosporine, gold, leflunomide (LEF), methotrexate (MTX), minocycline, penicillamine, sulfasalazine]. Our target for this measure was 80% of eligible clinic patients by March 2017.
Several PDSA cycles were designed to improve vaccination rates. First, in February 2015, a multidisciplinary conference was held to educate providers and clinic staff on the role of PCV13 vaccination. Second, in April 2015, a coordinated effort led by ancillary staff helped identify patients in need of vaccination on a weekly basis. Medical assistants also identified eligible clinic patients from the EHR using chart review prior to each clinic session, and this information was used to flag all unvaccinated patients. Of note, the decision to order the vaccine was left to physicians (this was intentional given that some patients decline or have contraindications to vaccination).
RA disease activity monitoring (CDAI) quality measure
To promote a treat-to-target approach in RA18, regular disease activity monitoring with a validated tool has been endorsed by the ACR and NQF as a critical quality metric19. Before our study, there were 3 separate processes related to disease activity measurement. Providers received an RA-specific paper note with most disease activity data elements (e.g., a homunculus for joint counts). Patients were separately given a paper document to record their global assessments. Last, providers could document disease activity in the EHR, although there was no way to document this information in a structured EHR field (i.e., it could be included in the history, physical examination, or assessment sections of the clinic note). We chose the CDAI given its ease of use and inclusion of data elements that can be obtained within a single visit18.
We measured the proportion of patients with RA age ≥ 18 years with at least 1 CDAI score between February and December 2016. Our target was 75% of eligible patients by December 2016.
Several PDSA cycles were used. First, in February 2016, the EHR was reconfigured with assistance from information technology (IT) staff to allow for identification of a numerical CDAI score in a structured template. Medical assistants were trained to merge this electronic CDAI template to EHR notes prior to all encounters with patients with RA. In April 2016, additional one-on-one provider training on the CDAI template was provided. A final PDSA cycle followed in June 2016, when physicians began receiving their individual performance rates on a quarterly basis during preclinic conferences.
LTBI screening quality measure
Screening for LTBI was endorsed by the ACR and NQF as a critical quality measure for patients with RA newly started on biologic disease-modifying antirheumatic drugs (bDMARD)19. Prior to our study, there was no standardized way of tracking tuberculosis (TB) screening history at the rheumatology clinic. Worse, the hospital TB clinic, which archives the LTBI treatment history of many San Francisco residents, was not fully integrated with the hospital-wide EHR.
We measured the proportion of patients with RA age ≥ 18 years who had documented TB screening or history of prior LTBI therapy prior to initiating new bDMARD between 2015 and 2017. DMARD included ABA, ADA, anakinra, CZP, ETN, GOL, IFX, and TCZ. Our target was 90% of eligible patients by the end of 2017.
PDSA cycles were implemented between January and April 2015. First, a patient safety checklist was introduced to standardize workflow for LTBI screening (Figure 1). This paper document included patient information, intended biologic therapy, assessment of TB status, and history of LTBI treatment. TB status could be confirmed with a purified protein derivative or interferon-γ release assay result within 12 months of biologic initiation. If patients had a positive screen or a history of LTBI, they were referred to the hospital’s TB clinic for evaluation. Second, a structured template for TB history was developed in the EHR with assistance from IT staff, which was used to track adherence to the quality measure.
Patient safety checklist for LTBI screening quality measure. Patient safety checklist developed for LTBI screening prior to initiation of biologic DMARD. Hepatitis B screening was also incorporated into the form. Documents were to be completed by nurses and then scanned into the EHR. CXR: chest X-ray; DMARD: disease-modifying antirheumatic drug; EHR: electronic health record; PPD: purified protein derivative; LTBI: latent tuberculosis infection.
Reproductive health counseling measure
Many women with rheumatic diseases receive potentially teratogenic medications during their reproductive years. Attention to this important aspect of clinical care has been proposed as a quality measure in SLE20 and other rheumatologic conditions, given the suboptimal receipt found in prior studies21,22. There was no standardized way of documenting contraception counseling in the clinic prior to our study.
We measured the proportion of women age 18–45 years who had received standardized contraception counseling at least once between March and December 2016 and were taking medications with either high teratogenic potential or with unknown or potential pregnancy risks. Medications included MTX, mycophenolate mofetil, LEF, CYC, minocycline, ADA, CZP, ETN, GOL, IFX, ABA, RTX, anakinra, TCZ, and tofacitinib. Our target was 50% of eligible patients by the end of December 2016.
PDSA cycles were used over the course of March 2016. First, IT staff added a teratogen counseling template to the EHR which provided a simple yes/no/not applicable option to track counseling events and offered a standardized template for providers to fill regarding the personalized content of their counseling; this template was used to track adherence to the quality measure. Second, handouts in English, Spanish, and Chinese were developed with patient feedback to improve education regarding family planning. Third, a paper consent form requiring physician and patient signatures was created to reinforce discussions regarding the teratogenicity of medications (Figure 2A and 2B). Physicians were trained in the use of the educational materials, paper consent forms, and EHR templates. A clinic nurse generated weekly lists of eligible patients, which medical assistants then used to merge the teratogen counseling template into the appropriate clinic note; the paper documents described above were also attached to the physical charts of eligible patients.
Patient education and counseling forms for reproductive health counseling quality measure. (A) Educational handouts provided to women of reproductive age receiving potentially teratogenic medications. These were made available in English, Spanish, and Chinese. (B) Counseling form to be signed by the physician and patient to document discussions regarding medication toxicity and reproductive health. Document was to be scanned into the electronic health record after completion.
Data analysis
For the analysis, we included eligible patients with at least 2 rheumatology clinic visits. The outcome was overall performance on the quality measures, defined as the proportion of eligible patients receiving recommended care by the end of the respective measurement period (Table 1). Baseline performance rates prior to study onset were available only for the PCV13 and LTBI screening measures; baseline documentation and therefore performance data for the CDAI and reproductive health counseling measures were inconsistent and not included in our study. There were enough discrete data points for the PCV13 and CDAI quality measures to construct Shewart charts to analyze performance over time. Given subgroups of varying size, p-charts were constructed, which depict quality measure performance over time in relation to the average of plotted points (also known as the center line), and the expected range of variation in a stable healthcare process (bounded by upper and lower control limits)23. Monthly and biweekly time intervals were chosen for the PCV13 and CDAI measures, respectively, in part to maximize the ability of charts to detect significant changes in healthcare delivery24. Raw data were extracted from the EHR and analysis was conducted using SAS version 9.4 (SAS Institute Inc.) and the QI Macros application version 2017.11 for Excel.
RESULTS
Clinic population
During the study period from 2015 to 2017, 1205 patients with at least 2 clinic visits were seen in the clinic; 547 patients (45%) were eligible for at least 1 of the 4 quality measures (Table 2). The mean age was 56 years (± SD 14), and 856 patients (71%) were female. The majority of patients identified as a racial or ethnic minority, with 150 patients (12%) identifying as white. Almost half (47%) reported a language preference for their encounters other than English.
Sociodemographic characteristics of the rheumatology clinic population with at least 2 clinic visits, 2015–2017.
PCV13 measure
There was a total of 505 patients seen in the rheumatology clinic who were eligible for PCV13 vaccination with a mean of 159 eligible patients each month. At the beginning of the measurement period in February 2015, only 21 eligible patients (15%) taking immunosuppressive medications had documented vaccination with PCV13. This rose to 74% by 12 months after implementation. By the end of the measurement period, 392 patients (78%) received PCV13 vaccination, which did not meet our target of 80%. Figure 3 depicts a Shewart chart of quality measure performance over time. Presence of greater than 8 data points above the center line shows evidence of improvement in performance over time25.
Shewart chart depicting performance of the PCV13 quality measure over time. Chart depicts the proportion of patients vaccinated with PCV13 by month of intervention. Red vertical lines correspond to PDSA cycle 1 in February 2015 (multidisciplinary conference educating clinic staff on vaccination) and PDSA cycle 2 in April 2015 (identification of eligible patients). The blue horizontal line indicates the center line, while the black lines above and below the center line indicate the upper and lower control limits, respectively. PCV13: 13-valent pneumococcal conjugate vaccine; PDSA: “Plan, Do, Study, Act” methodology.
PDSA cycles for this measure focused on identifying the relevant patient population and enhancing knowledge and awareness among physicians and other clinic staff of indications for PCV13 vaccination. We observed that success in this quality measure was largely due to engagement of many members of the interprofessional care team with diverse clinical roles. Clinic nurses reviewed lists of unvaccinated patients on a weekly basis to flag patients in need of pneumococcal vaccination. Medical assistants also integrated review of vaccination history into their routine clinical duties. Physicians responded to these notices by increasing the number of orders for vaccines. These changes to clinic workflow reinforced the education that was given to clinic staff and persist to this day.
CDAI measure
There were 295 eligible patients with RA from February to December 2016. They contributed 1003 clinic visits with a mean of 76 eligible patients per month. In the first 2 weeks of the intervention in February 2016, 7 patients (19%) had documented CDAI scores in the EHR. Performance improved to 74% by the third month following the first PDSA cycle. The number of RA patients with at least 1 CDAI documented by the end of the measurement period was 273 (93%), exceeding our target goal. The Shewart chart in Figure 4 shows the presence of a data point above the upper control limit and at least 8 data points above the center line, both evidence of significant improvement in care delivery25.
Shewart chart depicting performance on the CDAI quality measure over time. Chart depicts the proportion of patients with CDAI completion in biweekly intervals. Red vertical lines correspond to PDSA cycle 1 in February 2016 (introduction of EHR CDAI template); PDSA cycle 2 in April 2016 (one-on-one provider training); and PDSA cycle 3 in June 2016 (physician feedback). The blue horizontal line indicates the center line, while the black lines above and below the center line indicate the upper and lower control limits, respectively. CDAI: Clinical Disease Activity Index; EHR: electronic health record; PDSA: “Plan, Do, Study, Act” methodology.
We observed sustained success in this quality measure, in large part because of efforts to incorporate disease activity monitoring into routine clinical practice. The EHR structured template enhanced the ability to record this information in a standardized way and to track performance. Medical assistants also prepared EHR charts prior to RA encounters by merging the electronic CDAI template to provider notes. Peer reporting and one-on-one physician education on use of the CDAI template further complemented interventions to improve performance. These interventions are still in use at the present time.
LTBI screening measure
There were 77 patients started on biologic therapies during the study period from 2015 to 2017. Prior to the intervention, only 23 (56%) of the patients initiating biologic therapies had documented TB screening results. By the end of the measurement period, 70 patients (91%) had been screened for LTBI or had a documented history of prior LTBI, which exceeded our target goal.
In the first series of PDSA cycles, we created a patient safety checklist that required physician and nurse endorsement to proceed with biologic initiation (Figure 1). Importantly, the checklist prompted providers to test patients for or investigate history of LTBI. We discovered that many patients flagged as eligible for TB testing had already been treated for LTBI by the TB clinic in the past; however, these data were not easily accessible to rheumatology clinic providers. Standardized documentation in the EHR remains challenging given the TB clinic’s separate health record system, which is incompatible with that of the outpatient clinics; currently, this aspect of the quality measure is not consistently pursued. Clinic workflow does continue to use the patient safety checklist, which is scanned into the EHR, as part of the biologic initiation process.
Reproductive health counseling measure
There were 57 women of reproductive age eligible for this quality measure between March and December 2016. By the end of the first 4 months of the intervention, 18 patients (78%) had documented counseling. By the end of the measurement period in December 2016, 26 patients (46%) had been counseled at least once in the prior year, thus not meeting our 50% target.
Early success in this quality measure was not sustained through the study period. Weekly lists of eligible women were generated by nurses prior to clinic sessions, and medical assistants included the appropriate EHR template and printed contraception counseling materials with eligible patients’ charts — practices that continue to this day. There are several possible reasons for underperformance in this measure, including burdensome electronic and paper documentation and suboptimal patient and physician education.
DISCUSSION
In our study, we report on our multifaceted QI program in a safety net rheumatology clinic serving a racially, ethnically, and socioeconomically diverse patient population. Using the IHI’s Model for Improvement, we were able to significantly improve processes of care on all quality measures examined while achieving sustained improvements in 3 of the 4 areas we addressed.
Introducing practice-specific EHR templates was crucial to our QI interventions, and we found that they were most successful when they enhanced existing structures for clinical care. For instance, prior to implementation of the CDAI quality measure, a printed RA-specific clinic note was already in circulation to help providers collect information for disease activity assessment. The creation of a simple-to-use EHR template complemented this practice by offering a reliable means of gathering CDAI elements and tracking response to therapy over time, replacing the prior practice of reviewing handwritten notes. The CDAI template in the EHR was a simple structured field which documented only numerical information, and could likely be implemented in many clinical settings26. This exemplifies the importance of understanding healthcare context to ensure that EHR-based interventions are successful27,28.
QI also helps manage day-to-day practice variation in busy clinical settings. Two ways to do this are improving the categorization of patients into subgroups in need of certain services29,30 and engaging nonphysician clinic team members27,29,31,32. For instance, the PCV13 quality measure was bolstered by the active identification of eligible patients by 2 team members: first by nurses who generated lists of unvaccinated patients from the EHR, and second by medical assistants who reviewed vaccination history independently. Although seemingly redundant, these efforts to actively categorize patients across the entire team helped engage clinic staff under unified goals.
A strength of our study was the involvement of various staff members in the clinic. Providers were crucial to the selection of appropriate quality measures. We engaged patients in providing feedback on clinic workflows and development of educational materials in the early stages of QI planning, especially for the reproductive health counseling intervention. Medical students were intimately involved in the development and implementation of QI measures as part of their medical school curriculum. The success and durability of most interventions described above was in large part due to this team-based approach. To this day, the clinic uses the workflows established by these quality interventions, despite significant fluctuations in workforce (in particular among trainees).
We observed that QI can exert powerful positive downstream effects even if primary aims are not achieved33. For instance, in the LTBI screening quality measure, success in creating an electronic template for TB history was limited by an inability to access public health–level data. Even though a structured TB field in the EHR was not as successful as hoped, our study inspired the creation of a patient safety document for bDMARD initiation, a document that is still in use today, reflecting the utility of checklists for high-risk interventions34. Thoughtful analysis of QI data can also shed light on care processes when prespecified benchmarks are not achieved. In the PCV13 measure, for instance, performance at the end of the measurement period was just shy of our target of 80%, but construction of a Shewart chart showed significant improvement in measured performance over time.
We faced challenges in achieving our target in the reproductive health measure despite robust initial performance. Counseling discussions may not have been prioritized in busy clinic visits, and patients may not have felt empowered to raise the issue of reproductive health during visits. Suboptimal provider and patient education may have played a role. Alternatively, under-performance may simply reflect failure to use the EHR template rather than true low performance, although this was not formally measured.
Inconsistent documentation of baseline quality measure performance in the CDAI and reproductive health counseling measures may have limited our interpretation of data for these measures. The Shewart charts for the PCV13 and CDAI quality measures would also have benefited from preintervention data collection to document a stable healthcare process24. In addition, balancing measures were not formally assessed to ensure our interventions were efficient and not associated with unintended consequences. Patients were not systemically surveyed to examine how these measures affected their experiences of care. Last, other safety net clinics may not have access to certain resources we used, including the PIP incentive program; studies have shown an association between improvements in care and receipt of financial rewards35.
The Model for Improvement effectively improved performance on quality measures in prioritized clinical areas in a safety net rheumatology clinic. Through a multidisciplinary approach focusing on systems redesign, we achieved durable improvements in key clinical practices by enhancing workflow, engaging nonphysician providers, and managing practice variation. QI can achieve sustainable improvements in healthcare among diverse patients with rheumatologic conditions.
Acknowledgment
We thank the patients and staff of Zuckerberg San Francisco General Hospital.
Footnotes
This work was supported by the Zuckerberg San Francisco General Hospital Foundation, AHRQ R01 HS024412 and AHRQ R18 HS025638. Dr. Yazdany is also supported by the Robert L. Kroc Chair in Rheumatic and Connective Tissue Diseases, and Drs. Yazdany, Margaretten, and Goglin are supported by the Russell/Engleman Medical Research Center for Arthritis. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.
- Accepted for publication February 7, 2020.
