Abstract
Objective Systemic sclerosis (SSc) is a multisystemic autoimmune disease with high morbidity and healthcare costs. Inconsistent quality of care delivery, including inadequate screening and monitoring, necessitates improvement. This study aimed to enhance the uptake of validated quality indicators (QIs) for SSc.
Methods An interrupted time series study was conducted at 4 scleroderma clinics across 2 hospitals using the Model for Improvement methodology, employing Plan-Do-Study-Act (PDSA) cycles. A retrospective chart review assessed baseline frequencies of selected QIs. The primary aim was to increase rates of 7 baseline and 5 follow-up QIs to 80%. Root-cause analysis identified barriers to QI uptake, leading to interventions including provider education, equipment procurement, and care standardization with reminder systems. Real-time data tracking was facilitated through run charts.
Results The average completion rate for baseline QIs increased from 48% to 83% over 8 months, with sustained improvements post-PDSA cycle 3. Monitoring and treatment QI completion improved from 40% to 77%. Process measures saw increases in completion rates: baseline spirometry and diffusing lung capacity for carbon monoxide rates improved from 63.5% to 92%, documented counseling to perform weekly blood pressure self-measurement increased from 19% to 86.6%, referrals to hand range-of-motion exercise programs rose from 54% to 92%, baseline creatine kinase measurement rates increased from 52% to 88%, and oxygen saturation documentation rose from 31% to 65%. Stakeholders reported high satisfaction (median rating of 4), with minimal additional time per patient (median 2.5 minutes).
Conclusion This QI study significantly improved SSc care through low-cost, applicable interventions, setting a precedent for future work on long-term sustainability and broader application in chronic disease management.
Systemic sclerosis (SSc) is a complex, multisystemic rheumatic autoimmune disease associated with substantial morbidity and mortality.1 The disease affects several organ systems, placing patients at high risk of respiratory, cardiac, musculoskeletal, and renal manifestations. These manifestations increase the disability experienced by patients.2-4 Despite the effect of SSc on individuals and healthcare systems,5 notable disparities and inconsistencies in care delivery for patients with SSc have been identified.6,7 These include inadequate baseline screening rates for internal organ involvement, low rates of specialty consultations, and low rates of referrals to physical therapy services. Moreover, significant variability in diagnostic testing and treatments between tertiary SSc centers has been observed, highlighting the need for standardization in investigations and treatment of SSc.6,7
To address these challenges, a comprehensive set of validated quality indicators (QIs) for SSc has been developed. These QIs cover 8 disease domains systematically classified by body systems. They include baseline screening QIs tailored for patients newly diagnosed with SSc, alongside follow-up monitoring and treatment QIs designed for patients with established SSc.8 Despite the pivotal development of these SSc-specific QIs, research into their uptake among patients with SSc remains scant.
Therefore, the objectives of this study were to evaluate the uptake of baseline screening, follow-up monitoring, and treatment QIs at academic scleroderma clinics, and to implement quality improvement interventions to improve their uptake.
METHODS
Study design and setting. An interrupted time series study was conducted at 4 academic scleroderma clinics across 2 hospitals in Toronto, Ontario, Canada. The study used the Model for Improvement methodology, which employs Plan-Do-Study-Act (PDSA) cycles.9
Baseline assessment. We included all patients (48 charts) with a new diagnosis of SSc and patients with early SSc (diagnosed within the past 5 years) over the 4-month period in a retrospective baseline analysis. The sample size of 48 charts was deemed adequate to identify care gaps based on guidelines for quality improvement studies, which suggest that smaller, manageable samples can provide actionable insights while allowing for iterative testing and improvement.9,10
Seven baseline QIs included transthoracic echocardiography within 12 months of diagnosis, spirometry and diffusing lung capacity for carbon monoxide (DLCO) within 12 months of diagnosis, computed tomography (CT) scan of the chest if the forced vital capacity or DLCO was < 80% of the expected range, creatine kinase (CK) level within 12 months of diagnosis, creatinine level within 6 months of diagnosis, documentation of counseling to perform weekly blood pressure (BP) self-measurement for screening of scleroderma renal crisis, and measurement of oxygen saturation during the initial clinic visit.8
Five follow-up and treatment QIs included annual spirometry and DLCO, measurement of oxygen saturation at every clinic visit, annual weight measurement, BP measurement at every clinic visit, and referral for a range-of-motion (ROM) exercise program within 6 months for patients who present with decreased hand ROM or function.8
The proportion of patients with completed QIs was ascertained at monthly intervals and graphed on run charts. Completion of the QI was defined as completion of the test (eg, transthoracic echocardiography, pulmonary function tests, CK level) or documentation of the task in a chart (eg, counseling for weekly BP self-measurements, referral to physical therapy for hand motion exercise program).
Selection of target quality indicators. QIs with suboptimal uptake (defined as < 80%) were selected as the targets of quality improvement interventions for this study.
Aim statement. The primary aim of this study was to increase the percentage of patients who had completed baseline QIs to 80% and the percentage of patients who completed follow-up and treatment QIs to 80%. The target of 80% was chosen to keep the aim realistic, acknowledging that some QIs may not be completed despite the rheumatologist’s best efforts.11
Root-cause analysis. An Ishikawa diagram was created after interviewing relevant stakeholders to understand the root causes for suboptimal uptake in baseline QIs and follow-up and treatment QIs (Figure 1). The interviews took place after the baseline assessment and prior to the implementation of the first intervention.
Ishikawa cause-and-effect diagram showing the root causes of suboptimal utilization rates of baseline, monitoring, and treatment QIs in SSc. COVID-19: coronavirus disease 2019; SSc: systemic sclerosis; QI: quality indicator.
Family of measures. The family of measures framework was used to comprehensively evaluate the effect of quality improvement interventions on the care of patients with SSc. This framework included outcome, process, and balancing measures to provide a holistic assessment of the changes implemented. Outcome measures focus on the direct effect on patient care, process measures track the steps taken to achieve these outcomes, and balancing measures ensure that improvements in 1 area do not lead to negative consequences in another.
Outcome measures
• The rate of completion of suboptimal baseline QIs among newly diagnosed patients with SSc (spirometry and DLCO within 12 months of diagnosis, CK level within 12 months of diagnosis, documentation of counseling to perform weekly BP self-measurement, and measurement of oxygen saturation during the initial clinic visit).
• The rate of completion of suboptimal monitoring and treatment QIs among patients with early SSc (annual spirometry and DLCO, measurement of oxygen saturation at every clinic visit, and referral for a ROM exercise program within 6 months for patients who present with decreased hand ROM or function).
Process measures
• Rate of spirometry and DLCO completed within 12 months of diagnosis among newly diagnosed patients with SSc, and annually for patients with early SSc.
• Rate of documented weekly BP self-measurement counseling among newly diagnosed patients with SSc.
• Rate of referral to a ROM hand exercise program among patients with early SSc and reduced hand ROM.
• Rate of baseline CK measurements completed within 12 months of diagnosis among newly diagnosed patients with SSc.
• Rate of oxygen saturation documentation during in-person visits among newly diagnosed patients with SSc and those with early SSc.
Balancing measures. The balancing measures were stakeholder satisfaction and physician-perceived increases in time spent with each patient due to the implemented interventions. Stakeholder satisfaction was evaluated using a 5-point Likert scale, where stakeholders were asked to rate the effectiveness of each intervention in enhancing the uptake of QIs. A score of 5 indicated the highest level of satisfaction, whereas 0 represented the least satisfaction. Additionally, physicians were requested to provide an estimate of the additional time spent with each patient as a result of the implemented interventions.
PDSA cycles. Three change ideas were refined through iterative PDSA cycles to address the key causes of suboptimal QIs.
• PDSA cycle 1. Education and awareness (month 5). Lead physicians were briefed on suboptimal rates, and rotating learners received an introductory email outlining the expectations of care for patients with SSc (the importance of documenting oxygen saturation, ordering annual pulmonary function tests [PFTs], assessing baseline CK levels, and counseling patients on weekly BP self-measurement), including all suboptimal QIs.
• PDSA cycle 2. Procurement of appropriate devices (month 6). Each clinic room was equipped with finger and forehead pulse oximeters to address equipment unavailability and SSc-related physical barriers.
• PDSA cycle 3. Improvement of care standardization and reminder systems (month 8). A detailed standardized consultation letter template was developed for physician use, featuring specific fields for tracking suboptimal QIs and acting as a reminder system. Additionally, a prefilled baseline laboratory form/electronic bundle was introduced, which included CK levels, prefilled requests for annual spirometry and DLCO tests, and prefilled referrals for hand ROM exercise programs.
Data collection and analysis. Monthly chart reviews were conducted to determine the completion of suboptimal QIs. All patients with newly diagnosed or early SSc were included. Three investigators (AA, SA, and HJ) performed audits, with disagreements resolved by consensus. Descriptive statistics were used to summarize the data. Outcome and process measures were plotted monthly on run charts to analyze data over time and determine statistically significant special-cause variation. Special-cause variation refers to variation that arises from specific, identifiable sources that are not part of the inherent variability of the process.10,12,13
RESULTS
Baseline assessment. Forty-eight patient charts were reviewed during the 4-month baseline period. The analysis revealed that only 48% of patients with a new diagnosis of SSc had completed baseline QIs, and 40% of patients with early SSc had completed monitoring and treatment QIs. The rate of spirometry and DLCO completed within 12 months of diagnosis among newly diagnosed patients with SSc, and annually for patients with early SSc, was 63.5%. The rate of documented weekly BP self-measurement counseling among newly diagnosed patients with SSc was 19%. The rate of referrals to a hand ROM exercise program among patients with early SSc and reduced hand ROM was 54%. The rate of baseline CK measurement completed within 12 months of diagnosis among newly diagnosed patients was 52%. The rate of oxygen saturation documentation during in-person visits among newly diagnosed patients with SSc and those with early disease was 31%. Examination of run charts indicated no special-cause variation in these outcome measures before implementing quality improvement interventions (Figures 2-5; Supplementary Figures S1-S3, available with the online version of this article).
The proportion of completed baseline suboptimal QIs among newly diagnosed patients with SSc. SSc: systemic sclerosis; QI: quality indicator.
The proportion of completed monitoring and treatment suboptimal QIs among patients with early SSc. SSc: systemic sclerosis; QI: quality indicator.
The proportion of spirometry and DLCO completed within 12 months of diagnosis among newly diagnosed patients with SSc, and annually for patients with early disease. DLCO: diffusing lung capacity for carbon monoxide; SSc: systemic sclerosis.
The proportion of documented weekly BP self-measurement counseling among newly diagnosed patients with SSc. BP: blood pressure; SSc: systemic sclerosis.
Outcome measures. Following the implementation of quality improvement interventions through PDSA cycles, a significant improvement was observed in the rates of completed baseline QIs. The rate increased to 75% after the first PDSA cycle (month 1). Subsequent PDSA cycles led to further improvements, with completion rates of 83% after PDSA cycle 3 (month 5), 94% in month 6, and 100% in months 7 and 8. There was a special-cause variation following the third PDSA cycle, evidenced by an upward trend after month 5, suggesting that the upward trend was not due to random variation. (Figure 2)
The completion rate of monitoring and treatment QIs showed a progressive increase post intervention, reaching 62% after the first PDSA cycle (month 1). The rate continued to improve, reaching 72% following PDSA cycle 3 (month 4), 91% in month 5, and 96% in month 6. There was a special-cause variation after the third PDSA cycle due to an upward shift (Figure 3).
Process measures. Following the interventions, the baseline spirometry and DLCO rates for new patients and the rates observed during the annual monitoring for patients with early disease increased from 63.5% to 92% over an 8-month period. There is evidence of special-cause variation following the second PDSA cycle at month 3 due to a shift (Figure 4).
The documented weekly BP self-measurement counseling rate among newly diagnosed patients with SSc increased from 19% to 86.6%. There is evidence of special-cause variation following the third PDSA cycle due to a trend (Figure 5).
Following the interventions, the rate of referrals to a ROM hand exercise program among patients with early SSc and reduced hand ROM or function increased from 54% to 92%. Baseline CK measurement and oxygen saturation documentation rates during in-person visits rose from 52% to 88% and from 31% to 65%, respectively. However, there was no evidence of special-cause variation (Supplementary Figures S1-S3, available with the online version of this article).
Balancing measures. All stakeholders indicated overall satisfaction with the interventions, with a median rating of 4 out of 5 (range 3-5). The median physician-perceived time spent with each patient due to the interventions was 2.5 minutes per patient, ranging from 2 to 3 minutes.
DISCUSSION
This quality improvement study underscores the critical role of QIs in the screening, monitoring, and management of SSc. These QIs are paramount as they serve as benchmarks for best practices in SSc care, enabling early detection of organ involvement and timely intervention, which are vital in mitigating the morbidity and mortality associated with this condition. However, having a list of SSc-specific QIs is not enough. Addressing the deficits in QI uptake was therefore not only a measure of enhancing care quality but also a crucial step in improving patient outcomes.
In the context of health care, quality improvement refers to systematic efforts to enhance patient care and outcomes through the continuous assessment and refinement of care processes. Quality measures are specific, evidence-based criteria used to evaluate the extent to which healthcare services meet established quality standards.14 Clinical practice guidelines, such as the 2023 American College of Rheumatology/American College of Chest Physicians guidelines for the screening, monitoring, and treatment of interstitial lung disease (ILD) in people with systemic autoimmune rheumatic diseases, recommend screening with PFTs and high-resolution CT (HRCT) scans for those at increased risk, as well as monitoring with PFTs and HRCT to assess ILD progression in addition to referrals to lung and stem cell transplantation in a specific risk group.15,16 These guidelines aim to reduce practice variation across regions and to educate providers on best practices. Despite recommendations, a gap often exists between guidelines and their implementation in clinical practice. The present study demonstrates effective strategies to bridge this gap, enhancing the delivery of high-quality SSc care by ensuring adherence to recommended guidelines through tailored interventions and systematic approaches.
The tailored interventions developed to address the suboptimal baseline, follow-up, and treatment QIs have led to the achievement of our primary aims. Particularly, the third intervention (PDSA cycle 3), which focused on care standardization by using note templates, laboratory forms, referrals, and implementing reminder systems, was pivotal in accomplishing sustained improvements. Whereas PDSA cycles 1 and 2 allowed for the foundational work of educating healthcare providers and addressing equipment unavailability, the systematizing of care in PDSA cycle 3 catalyzed enduring change. This streamlined the care process, reducing reliance on individual recall and vigilance, which are often susceptible to the pressures of a high-volume clinical setting. Standardizing care processes is a more effective, system-based intervention than education, which is often a necessary first step but is rarely sufficient.17,18
The balancing measures indicated a minimal increase in physician-perceived time spent per patient, suggesting that the interventions were not overly burdensome to clinic operations. The high satisfaction ratings provided by stakeholders reinforce the interventions’ feasibility and effectiveness, further supporting the case for their sustained implementation.
The main strengths of this study reside in the simplicity, cost-effectiveness, and applicability of the interventions. The deployment of low-cost solutions like standardized note templates and prefilled laboratory forms provided a blueprint for quality care that can be replicated and scaled across similar healthcare settings. Additionally, using run charts for real-time analysis allowed our team to swiftly identify trends and respond with appropriate interventions. Feedback on performance can additionally reinforce positive behavioral changes.
This study may be affected by some limitations. One potential limitation is that the awareness of being observed and measured—the Hawthorne effect—may have altered provider behavior independent of the interventions themselves. Second, the relatively short duration of the study limits our ability to claim that these changes will be sustained in the long term. A longer observation period post implementation is necessary to verify the lasting effect of these interventions.
However, this study has several strengths. Although many QI studies are conducted at a single clinic or by a single provider, this is a multiclinic study across 2 hospitals and includes several providers, learners, and clinic administrative staff. This study demonstrated significant improvements in QI use despite the variability across clinics and providers. Second, this study evaluated deficiencies in the uptake of SSc quality measures,14 and to our knowledge, it is the first quality improvement study to focus on improving care for patients with SSc.
In conclusion, our study demonstrated that quality improvement methods result in high-quality SSc care. This study serves as a valuable model for implementing quality improvement in SSc and highlights the transformative potential of simple, cost-effective interventions that can significantly enhance healthcare delivery. It also lays the groundwork for future initiatives aimed at long-term sustainability and provides a replicable framework for other institutions to emulate. Moving forward, it will be crucial to monitor the enduring effects of these interventions and refine them in response to longitudinal outcomes and evolving healthcare landscapes.
Footnotes
CONTRIBUTIONS
AA: conceptualization, data curation, investigation, formal analysis, methodology, validation, visualization, writing original draft, and writing (review and editing); SA: conceptualization, methodology, and writing (review and editing); HJ: investigation and writing (review and editing); ZA and MS: methodology, data curation, and writing (review and editing); DM: methodology, data curation, project administration, and writing (review and editing); SRJ: conceptualization, data curation, investigation, formal analysis, methodology, supervision, validation, visualization, writing original draft, and writing (review and editing).
FUNDING
The authors declare no funding or support for this work.
COMPETING INTERESTS
The authors declare no conflicts of interest relevant to this article.
ETHICS AND PATIENT CONSENT
Ethical approval for this project was obtained from the University Health Network’s Quality Improvement Review Committee (no. 23-0605). The Research Ethics Board at Mount Sinai Hospital exempted the project from ethics approval as it did not constitute research.
- Accepted for publication January 27, 2025.
- Copyright © 2025 by the Journal of Rheumatology
REFERENCES
SUPPLEMENTARY DATA
Supplementary material accompanies the online version of this article.
