Abstract
The commitment to improve care processes and patient outcomes is a professional mandate for clinicians and is also seen as an operational priority for institutions. Quality improvement now figures in the accreditation of training programs, specialty examinations, and hospital scorecards. Rheumatologists have traditionally focused primarily on quality problems such as guideline adherence; however, improvement goals should also include other aspects of care that are helpful to patients and are professionally rewarding for practitioners. This review makes use of improvement projects in outlining tangible tools rheumatologists can use to resolve quality concerns in their practices.
Improving processes of care and patient outcomes is a mandate for all individuals and institutions in medicine1,2. Quality improvement (QI) is now involved in the accreditation of residency programs3,4, maintenance of certification5,6, hospital scorecards, and strategic plans7.
Widespread attention to healthcare quality began with the release of To Err is Human8. This Institute of Medicine (IOM) report identified the interest of the public, clinicians, and decision makers, with its focus on morbidity and mortality from medical error. Capitalizing on this interest, the IOM then released Crossing the Quality Chasm9, which addressed quality problems more broadly. The second report characterized quality in terms of 6 constituent domains: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity (Table 1)9,10,11,12,13,14,15,16,17,18.
Rheumatologists traditionally focus on quality problems related to effectiveness (e.g., guideline adherence) so it may seem daunting to add QI targets from the other domains listed in Table 1; however, doing so will not only help patients, but also improve the care experience for clinicians (e.g., improving clinic flow). Some domains also reinforce each other for a rewarding effect: engaging patients in shared decision making regarding treatment decisions (patient-centeredness) may improve medication adherence (effectiveness). Often guideline adherence does not fall solely in the quality domain of effectiveness, but may also lead to improvement in other areas, such as safety.
Quality indicators to measure and improve performance have proliferated in both adult19,20,21 and pediatric rheumatology22,23. The American College of Rheumatology has introduced a quality-reporting system for health practitioners called the Rheumatology Informatics System for Effectiveness (RISE)24,25. RISE interfaces with electronic health records to produce performance reports that rheumatologists can use to measure quality and facilitate compliance with new payment models and evolving certification requirements.
But quality indicators and performance reports do not in themselves lead to improvement. We need both methods for understanding the basis for quality problems and pragmatic tools for addressing them. Our paper illustrates how rheumatologists can use these tools for rapid-cycle change to address a variety of quality problems in their practice.
The Model for Improvement
Though not the only approach to QI, the Model for Improvement has broad appeal because of its simplicity and practicality. It combines 3 questions with the Plan-Do-Study-Act (PDSA) cycle to guide specific improvement activities, as seen in Figure 126,27. We used 2 example projects to illustrate the Model for Improvement and 3 other common QI tools: Pareto analysis, the fishbone diagram, and run charts (a glossary of these and other key terms appears in the Supplementary Data, available with the online version of this article).
Case 1 Example: Increasing Access to Rheumatology Care
Question 1: What are we trying to accomplish?
A common quality problem is excessive wait times for new appointments. The Model for Improvement stipulates a measurable accomplishment in a specific time frame. “We plan to improve access” specifies nothing concrete. A good aim statement should be “SMART”: Specific (What are we trying to improve?), Measurable (By how much will we improve it?), Applicable, Realistic, and Timely (By when will this occur?). For instance, “We will reduce the median wait time for new appointments by 50% over the next 12 months.”
Sometimes we need to revise the aim statement as the project unfolds. Once we understand the full range of contributing causes, we recognize that we have no control over some important aspects of the problem. For example, perhaps as the access project unfolds it becomes clear that wait times for patients are not as important as the delays in seeing urgent referrals. A prudent solution would be to revise the aim to: “By this time next year, we will consistently see 90% of urgent referrals within 4 weeks.”
The idea is to start with a concrete aim, including a specific measure of success, the amount one plans to improve that measure, and the timeline to do so, but also to recognize that the specifics may evolve as the target problem is better understood.
Question 2: How will I know if a change leads to improvement?
Improvement projects typically use 3 types of measures: outcome, process, and balancing measures28. Outcomes reflect what happens to patients — the rate at which they received guideline-concordant care (effectiveness), or as in our example, wait times for urgent referrals. Process measures, by contrast, characterize the degree to which an improvement is playing out as intended. Are clinicians using the new order set or referral form? Are they reading the performance report cards? Think of process measures such as compliance in a clinical trial. Evaluating efficacy requires knowing whether patients took the study drug. Similarly, to determine whether our improvement idea works, we need to know that it was implemented successfully — that clinicians or patients “took the pill.” Finally, balancing measures assess the emergence of undesirable consequences. A project aiming to reduce length of hospital stay might monitor 30-day readmissions because we would not want to reduce length of stay by prematurely discharging patients who must then return to the hospital.
For our example, the proportion of urgent referrals seen within 4 weeks constitutes the outcome. As a balancing measure, we would track the proportion of nonurgent referrals waiting more than 3 months to ensure this did not increase as an unintended consequence of our efforts to see urgent patients within 4 weeks. Process measures depend on what “change idea” or intervention we think might result in improvement. A hypothetical change idea could be to implement a new referral form to facilitate identifying urgent referrals. An early process measure might track the proportion of referrals using the new form. Once usage reaches a reasonable level, we may need to audit the accuracy of the information related to assessing urgency.
References to multiple process and balancing measures may raise concerns over measurement burden. Importantly, frequent small sample sizes (e.g., 10 patients) suffice for monitoring these key steps in the change processes29. For instance, in the early stages of our project, we want to know whether clinicians are using a new referral form 90% of the time. Auditing 10 consecutive referrals finds that the new form is used in only 5 of them. The 95% CI for this proportion of 50% ranges from 31% to 86%. A CI this wide would serve little purpose in clinical research. No one wants to say that a treatment works somewhere between 31% and 86% of the time, but for our improvement project, this small sample size suffices to show we have not yet achieved 90% usage, so that further refinements to the referral form (or further efforts to promote its use) are required29.
Question 3: What changes might result in improvement?
Identifying candidate improvements (“change ideas”) that will bring about the project’s aim constitutes the most difficult step. Some ideas can be discarded based on common sense. For example, working harder to see more patients represents a quick fix, but not a sustainable one. Hiring more rheumatologists would help, but might not be feasible in the desired timeline. Moreover, after some initial reductions in wait times, the problem may reappear as the new clinicians’ panel sizes grow.
In general, rather than brainstorming solutions, start with understanding why the problem exists. Even experienced clinicians should not underestimate the myriad of contributing factors to a given quality problem. Some may indeed be immediately apparent, but identifying others requires a systematic approach to diagnosis. Two commonly encountered tools for characterizing quality problems include the Ishikawa (or “fishbone”) diagram, which will be described later, and the Pareto analysis. We do not discuss process mapping, which is involved in many efficiency projects30.
A Pareto analysis helps focus on causes of the target problem with greatest potential effect. For instance, a restaurant owner notes that complaints about service quality, price, and noise account for 80% of all negative online reviews. The owner reasonably decides to target those 3 categories, not wasting efforts on the multiple other complaints that account for only a small percentage of negative reviews. Returning to our example, we realize that inappropriate referrals substantially contribute to the problematic wait time (1 report judged that 40% of referrals as inappropriate31). We conduct a Pareto analysis to identify the most common types of inappropriate referrals.
We screen referrals that were received but not scheduled, as well as patients seen just once with no followup, to identify patients whom we believe did not require a rheumatologist. Three categories account for 85% of inappropriate referrals — referrals better directed to another specialty, borderline antinuclear antibody results with no relevant symptoms, and patients with fibromyalgia already assessed by another rheumatologist and receiving appropriate treatment yet seeking another opinion. We could develop standard letters to send back to referring doctors for each of these referral types. Even if each required a different approach, the Pareto analysis (often accompanied by a simple bar chart showing the cumulative percentage accounted for by each category) efficiently highlights the minority of issues that cause most problems.
Step 4: PDSA cycles
After asking the 3 crucial questions, we can start testing changes. PDSA cycles (Figure 1) amount to inductive learning–iterative cycles of change, measurement, and reflection on the consequences of the change.
For our first change idea, we modify the referral form to better identify the reason for referral and ensure all necessary tests were provided. After several weeks in a row where at least 90% of new referrals used the new form, we audit a small sample for accuracy and realize that sometimes the reason cited was not entirely correct. While discussing this with the clinic secretary, it also becomes apparent that the new referral form has generated frequent requests back to referring doctors for more information. Rapid cycle improvement means testing change ideas quickly, sometimes refining them in successive cycles, but other times abandoning them. Changing the referral form may not work for all referrals and the assistant may become overwhelmed. Do not rush to a solution.
Sometimes in the process of analyzing 1 change idea, a new one emerges. The clinic has hired several advanced practice and extended role practice (AP/ERP) professionals. These include physiotherapists, occupational therapists, registered nurses, and physician assistants who have received advanced training, skills, and experience in the care of people with arthritis32,33. Through multiple iterative modifications to the process for triage, patient assessment, test ordering, and facilitating system navigation of referred patients34,35, we incorporate the AP/ERP into a different model of care that drops the median wait times. Different evaluations have shown improved wait times for patients with inflammatory arthritis36, including 1 with a dropped median reduction from 39 days to 22 days37.
Subsequent PDSA might modify the scheduling process, but how do we track our improvements and monitor for any unintended consequences? Run charts provide effective tools for displaying the data over time, with formal rules for detecting statistically significant changes38. These rules share the same conventional p = 0.05 threshold for rejecting null hypotheses that guide clinical research. For instance, under normal circumstances, wait times should fluctuate randomly, with a 50-50 chance that the median number of days to clinic appointment in a given month is above or below the median. Thus, if 6 successive data points fall below the median (from before the change), it resembles tossing a coin on heads 6× in a row, an event that has a probability of about 1%. When this occurs, as it clearly does in this case (Figure 2), we can infer that a statistically significant change has occurred. One busy academic rheumatology office lowered its third available appointment (a common wait time metric) from 60 days to < 2 days by creating space39. Cancellations fell and patient satisfaction measures improved significantly. Financial performance improved as well.
This project successfully decreased median wait times for urgent patients through multiple interventions: a change in scheduling, incorporating an AP/ERP, and using a triage form (though in this case, this was the least robust change). Successful projects usually involve multifaceted interventions, consistent with the idea that several factors contribute to most QI problems, as highlighted by the fishbone diagram in the next example.
Case 2 Example: Increasing Vaccinations in Immunosuppressed Patients
Given the high risk of influenza and its complications in patients with autoimmune rheumatic diseases40,41, guidelines recommend that all patients with rheumatoid arthritis receive the flu shot42. However, an audit of rheumatology patients revealed a vaccination rate of only 50%43. In a practice audit, one of us (SC) observed a similarly low vaccination rate of 40%. Patients were variably asked about vaccination status, and immunization history was not routinely documented. While the previous example involved a pastiche of projects from the literature, one of us (SC) undertook this second example project with several colleagues. We aimed to increase the proportion of patients with autoimmune rheumatic disease who received the flu shot to at least 80% in 1 year.
Like many undertaking improvement projects, we made the mistake of jumping to a solution44 by assuming that simply reminding patients to get the flu shot would solve the problem. We created a reminder to prompt clinicians to tell their patients about vaccination. An audit a few months into the project revealed no increase in uptake. In QI, it is better to understand the problem and complete small tests of change, rather than just deciding on a change and plowing ahead.
A fishbone diagram provides a framework for considering the various possible variables influencing a problem, with categories for patient factors (e.g., beliefs and expectations), provider factors (time, knowledge, attitudes), equipment problems (e.g., usability and maintenance), and organizational characteristics, including staffing and infrastructure, but also culture. Figure 3 shows our fishbone diagram for barriers to obtaining the flu shot. These barriers were identified from the literature, local data collection, and reflection by an interdisciplinary group allowing for more varied perspectives. We asked patients about their views because they often provide crucial information.
Our first PDSA cycle focused on standardizing the process for documenting vaccination status in immunosuppressed patients. Obtaining an up-to-date immunization status by sending a template letter to family doctors helped to document this. Next, we surveyed patients to identify reasons for not receiving immunization. The most common reasons included not knowing about the need for vaccination, forgetfulness (for patients and their providers), and the inconvenience of an extra appointment just to receive the flu shot. It is important to drill down to understand why each of these causes exist. Identifying the reasons informs creating a change strategy. To address these challenges, we needed an intervention that not only increased awareness among patients and physicians, but also improved access to the flu shot.
Our next PDSA cycle created a patient pamphlet highlighting the risks of the flu and benefits of immunization in immunosuppressed patients. These different colored pamphlets placed in examination rooms reminded physicians to ask about vaccination and included a walking map to the hospital pharmacy where patients could not only pick up their medications, but also receive the flu shot.
We made some progress in terms of informing patients and physicians about the need for flu shots. However, providers often did not distribute pamphlets. The flyer was too small to read, and it was easy to overlook immunosuppressed patients during busy clinics. For the next PDSA cycle, the pamphlets were made more patient-friendly with larger graphics and fonts. We also created neon stickers to be placed on the disease-modifying antirheumatic drug prescriptions to remind pharmacists to administer the flu shot when they renewed medications.
Following these refinements, 60%–90% of our patients received the pamphlets or sticker (process measure), and 65% of immunosuppressed individuals obtained the flu shot (project outcome measure), up from 40% prior to the project. Although we did not meet our goal of > 80% at the end of the season, we did achieve a relative improvement of 60%. Similar to clinical research, modest improvements are common and one should not become discouraged. Additional interventions may move the needle further since QI almost always requires multifaceted changes. If an intervention succeeds in achieving change, it is vital to plan to sustain and spread improvements. Ensuring that the improved vaccination rates are maintained and shared with other similar organizations can be achieved through different methods such as having a supportive management structure, building a culture of improvement, and deeply engaging staff45.
Final Thoughts
PDSA cycles provide a practical approach for learning and informing action, but learning is rarely linear because each cycle can reveal other issues that need to be addressed to achieve the improvement goal46. For larger scale projects, success typically requires investment in participant engagement, management support, alignment of financial incentives, and additional methodological expertise47. For many smaller projects in a clinic, however, success can come with application of the Model for Improvement and basic QI tools such as the fishbone diagram, Pareto analysis, and run charts as illustrated in the examples presented. Online tutorials provide further guidance for using tools48. A recent review walks readers through an easy-to-follow PDSA project, highlighting the easily understood, practical design of each step, yet the powerful effect the methodology produced as a whole49.
Like all physicians in contemporary practice, rheumatologists must address quality problems. Doing so falls within our professional responsibility because external bodies increasingly hold us accountable to performance expectations, a trend that will only accelerate. Beyond these incentives, in our experience and those of many colleagues and trainees learning QI methods in recent years, successful improvement projects deliver an intrinsic satisfaction associated with solving problems that frustrate efforts to deliver high-quality care. These projects also bring a special type of professional satisfaction through improving the care of many patients. Further, just as we enjoy exchanging clinical experiences with colleagues, sharing successful redesigns also brings a particular professional pleasure.
ONLINE SUPPLEMENT
Supplementary material accompanies the online version of this article.
- Accepted for publication April 5, 2017.