Abstract
Objective. To test the feasibility of reporting on 4 national performance measures for patients with rheumatoid arthritis (RA) in 5 different models of care.
Methods. The following performance measures were evaluated in 5 models of care: waiting time (WT) to rheumatologist consultation, percentage of patients seen in yearly followup (FU), percentage taking disease-modifying antirheumatic drugs (DMARD), and time to starting DMARD. All models aimed to improve early access and care for patients with RA.
Results. A number of feasibility issues were encountered in performance measure evaluation because of differences in site data collection and/or the duration of the model of care. For example, while 4/5 programs maintained clinical or research databases, chart reviews were still required to report on WT. Median WT for care in 2015 varied by site between 21 and 75 days. Yearly FU rates could only be calculated in 2 sites (combined owing to small numbers) and varied between 83% and 100%. Percentage of patients taking a DMARD and time to DMARD could be calculated in 3 models, and rates of DMARD use were between 90% and 100%, with median time to DMARD of 0 days in each.
Conclusion. Our review has shown that even in models of care designed to improve access to care and early treatment, data to document improvements are often lacking. Where data were available for measuring, deficits in WT performance were noted for some centers. Our results highlight a need to improve reporting processes to drive quality improvement.
Early and targeted treatment strategies in rheumatoid arthritis (RA) have been shown to dramatically improve patient outcomes, reducing disease activity and improving functional status1,2. This model of early and targeted treatment is concordant with national3 and international4,5 RA guidelines. The achievement of these targets relies heavily on appropriate and effective models of care6, seen as the structuring of health services in a center, region, province, or nation, to ensure appropriate service provision and to prioritize identification of new cases of inflammatory arthritis, especially in the setting of limited provider resources7. While access to care is a key facet of many models of care and is the focus of the current work, the Arthritis Alliance of Canada’s (AAC) Pan-Canadian Approach to Inflammatory Arthritis Models of Care8 highlights that model of care should encompass the continuum of care from early diagnosis, treatment, through to ongoing and shared care.
A vital part of the AAC’s approach was the development of an evaluation framework for models of care9, to report on the success of a model of care. The AAC framework includes a set of 6 system-level performance measures that were rigorously developed to measure model of care performance with a focus on access to care and early treatment10.
Across Canada, many rheumatology centers have developed programs to improve access to care8,11,12,13,14. The objective of our study was to evaluate the feasibility of reporting on 4 of the performance measures in RA [time to rheumatologist consultation, percentage of patients seen yearly by rheumatologists, percentage of patients taking disease-modifying antirheumatic drugs (DMARD) therapy, and time to DMARD start] using clinic-level and/or triage data from 5 clinical sites. Two measures were not evaluated: percentage of patients with inflammatory arthritis seen (because that relies on administrative data), and rheumatologists per capita, because that is explored elsewhere7.
MATERIALS AND METHODS
Performance measures were assessed in 5 clinical sites, using data between 2013 and 2016 (depending on the start date of the site model and the availability of data). The data used were exclusively from triage or cohort databases from each site, supplemented with minimal chart reviews directed at confirming diagnosis and/or one or more dates required for performance measure calculation. Detailed information about patient demographics or care was not collected, because the primary objective of the study was to assess the feasibility of measure reporting.
Location of included sites
Each site had improved access to care and treatment for patients with early inflammatory arthritis as central model objectives. Sites included in the study represent a convenience sample and were selected based on willing local champions to test the measures. All site leads had participated in the measure development process. Table 1 describes each site: The Rheumatic Health Unit (RHU) Central Triage Model of Care at the St. Clare’s Mercy Hospital in St. John’s, Newfoundland; The Arthritis Program (TAP) at the Southlake Regional Hospital in Newmarket, Ontario; The Early Arthritis Clinic (EAC) at the William Osler Health System in Brampton, Ontario; the Siksika Arthritis Screening Project (SASP) on the Siksika Reserve in Alberta15; and the Urban Aboriginal Arthritis Care Program (UAACP) at the Elbow River Healing Lodge in Calgary, Alberta16. Data from the 2 indigenous sites were pooled for analysis owing to small numbers. The sites were selected as test sites for evaluating the performance measures because they represented different types of models of care, provided representation of practices in different provinces, and had willing champions who had been part of the original development of the measures. Our study is part of a larger study evaluating the performance measures using different data sources, including administrative and electronic medical record data.
Descriptions of the 5 models of care included in the study.
Assessment of the feasibility of performance measure reporting
Meetings were held with the leads of each site to discuss the data available to report on each performance measure as well as resources available for continuous reporting. At a minimum it was decided that chart reviews would be used, if necessary, to determine the waiting time measure. If other measures were not feasible to report without extensive chart reviews, then they would not be reported, and this would be noted as part of feasibility testing and rectified in future data collection if desired by the site.
Patient inclusion and exclusion criteria
Sequential RA cases over the study period were included at each site, using the treating rheumatologist’s RA diagnosis for inclusion. However, for the TAP site, only patients enrolled in a national early arthritis cohort (the Canadian early Arthritis CoHort, CATCH17) were included. At other sites, limited chart reviews were conducted to confirm RA diagnosis, with the exception of the 2 indigenous arthritis care sites, because these data were available. Cases seen in the emergency room, self-referrals, and transfers of care were excluded from measurement of waiting times for care.
Measure calculation
Calculation of the performance measures has previously been described10,18. In brief, for the waiting time measure, median and 90th percentile waiting times in days (d) for rheumatologist care was calculated by measuring the time between referral receipt and the first visit to the rheumatologist for patients with a confirmed diagnosis of RA (confirmed before, at, or after that first visit). The percentage of cases meeting a benchmark of < 4 weeks was also calculated.
The percentage of patients seen in yearly followup by rheumatology was reported by calendar year. Each patient’s followup time began at their initial visit and continued until the last available followup, or the patient was lost to followup, whichever came first. The proportion of patients meeting this measure was calculated using the denominator (e.g., expected visits: 1 for each calendar year), and whether there was at least 1 observed visit in the calendar year as the numerator. If patients had less than 1 year of followup, they were excluded from the analysis for the measure because performance could not be determined. Similarly, if there was not a complete year of available followup for the final year of measurement, patients were excluded from the denominator of the final year. The number of gaps in care of > 12 and > 14 months between 2 consecutive visits was also calculated and reported for the duration of total followup for all patients in the cohorts.
The percentage of patients who received a traditional DMARD, biologic DMARD, or small-molecule inhibitor at least once during the measurement year was calculated between the baseline date (enrollment date) and end of followup date on a yearly basis for each patient. The denominator included all patients still in the cohort at the end of the measurement year and the numerator included all patients with at least 1 traditional DMARD, biologic DMARD, or small-molecule inhibitor prescription during the year. Patients were excluded from the denominator in years in which they had a documented contraindication to DMARD use (e.g., pregnancy or concomitant malignancy) or a documented patient refusal of treatment.
The time to DMARD therapy was calculated by measuring the time between the physician-reported date of RA diagnosis and the date of first DMARD start. The median and 90th percentiles were reported. Additionally, the percentage of patients treated within the benchmark of 14 days was also reported. For this measure, patients with a missing diagnosis date were excluded.
Ethics
Ethical approval for the study was obtained at each individual site. In particular, the indigenous cohorts adhere to research ethics with indigenous populations and respect OCAP principles (Ownership, Control, Access, and Possession). OCAP is a registered trademark of the First Nations Information Governance Centre19. The overall study was approved by the University of Calgary Research Ethics Board (REB15-2271).
RESULTS
Feasibility of reporting on each performance measure
There were a number of feasibility issues encountered in the evaluation of the performance measures as a result of variations in data collected by site (Table 2). Four of the sites had databases (2 triage databases and 2 cohort databases); however, in all 5 sites chart reviews were necessary to gather data for the waiting time measure (3/5 missing diagnosis and 3/5 missing referral dates). Because data were limited owing to the recent start of the care model in 2 sites (EAC and RHU data collection for sites commenced in 2014), the evaluating percentage seen in yearly followup was determined not to be feasible to report. At a third site, TAP, longitudinal chart reviews over the entire study duration were determined to be too cumbersome, leaving only the SASP and UAACP models of care reporting on this measure. Only 3 models of care (EAC, SASP, UAACP) had readily available treatment data that could be used to report on the percentage of patients taking a DMARD and time to DMARD start performance measures.
Feasibility of measuring 4 performance measures in 5 models of care.
Waiting times of patients for rheumatology consultation
Waiting times for rheumatology consultation for patients subsequently diagnosed with RA were calculated and varied by site (Table 3). To illustrate, in 2015 the median waiting time varied between sites from 21 days to 75 days and the 90th percentile was between 57 and 137 days. There were limited data on the longitudinal trends of waiting times at model sites because of their recent inception; however, data were available from TAP between 2013 and 2015. This showed a trend to improvement in waiting times from a median of 37 days to 21 days and the percentage meeting the 4-week benchmark increased from 29% to 64%. The TAP site was also the only one with a median wait time below the benchmark of 28 days (by 2015). It should be noted that as described in Table 1, 36% of SASP and UAACP patients self-referred to the programs and this was an exclusion criterion for performance reporting (indeed, in 2011 self-referral was the only mode of referral and this year was entirely excluded from reporting). Also, owing to small numbers of new patients with RA per year (cell sizes < 5 for 3 of the 4 potential reporting years), only the overall waiting times for 2012 to 2015 are reported (as opposed to yearly waiting times; Table 3).
Waiting times for rheumatology consultation in 4 models of care.
Percentage of patients with RA seen in yearly followup
As described above, the only sites with available data on yearly followup rates were the SASP and UAACP models of care. In the 2 sites combined, there were 66 prevalent RA cases and 50 with followup data over a mean measurement period of 32.6 ± 19.5 months. A total of 5 patients had followup shorter than 1 year and were also excluded from measurement, leaving 45 prevalent RA cases with followup data. Yearly followup rates in the cohorts were excellent: between 83 and 100%. This is also reflected in the analysis of the number of gaps in care (reported on all 50 prevalent cases, Table 4). Over half of all patients (56%) had no gaps of > 12 months between visits and 64% had no gaps of > 14 months. There were 18 patients (36%) with 1 gap of > 12 months and 15 (30%) with 1 gap of > 14 months; < 5 patients had 2 or more gaps in care > 12 months.
Percentage of prevalent RA cases (n = 451) seen in yearly followup in 2 indigenous models of care.
Percentage of patients taking a DMARD and time to DMARD start
The percentage of patients taking a DMARD and time to DMARD start were readily calculated at 3 sites: SASP, UAACP (Table 5), and EAC (Table 6). In the SASP and UAACP sites there were 66 prevalent RA cases; however, 2 cases met exclusion criteria owing to concomitant contraindications for DMARD use during certain measurement years and an additional 5 had a documented decline of a DMARD prescription and are excluded from the measurement years in which they declined treatment. Overall rates of DMARD treatment were high (range 90–100% between 2011 and 2016; Table 5). In the SASP and UAACP sites, time to DMARD therapy for patients with new-onset RA was calculated on 30 patients with incident RA. Yearly reporting of this performance measure was not possible because of small sample sizes (< 5) in 2 of the 5 years of available data. Between 2011 and 2015 the median time between RA diagnosis and DMARD prescription was 0 days (indicating treatment prescription occurred at the time of diagnosis); however, the 90th percentile was 59.5 days. Overall, 87% of patients (26/30) were prescribed DMARD within 2 weeks of diagnosis.
Percentage of rheumatoid arthritis patients taking DMARD in the SASP and UAACP cohorts1.
Percentage of RA patients taking DMARD and time to DMARD start in the EAC.
In the EAC clinic, data were available since the start of the clinic in June 2014 until the end of 2015 (Table 6). During both years, the median days between diagnosis and DMARD prescription was 0, as was the 90th percentile. The majority (92% and 94%) of patients were treated within the 2-week benchmark in 2014 and 2015, respectively, and after excluding patients with documented refusals, 100% of eligible patients were treated with DMARD.
DISCUSSION
In our study, we provide the assessment of feasibility of measuring system-level performance measures, and report on adherence to 4 of the performance measures in 5 sites. As we have illustrated, even in models designed to improve access to care for early inflammatory arthritis, all the data required to measure and report on the success of these programs are not routinely collected. Variations in reporting of critical data elements, such as the availability of final diagnosis, referral and visit date information, and treatment information, highlight challenges in measure reporting.
Where data were available, our study also highlights significant disparities in waiting times across sites, with many not meeting benchmarks. Reasons for delays in rheumatologist consultation may be complex and not necessarily within the control of the rheumatologist’s office. In particular, long waiting times for patients with RA were observed in Newfoundland. The St. John’s RHU Central Triage Model is new (started in 2011 as a pilot project, and permanent funding for an interprofessional model of care was achieved in 2014) and it is possible that a decline in waiting times for care may happen over time. It is also possible that given the low number of rheumatologists per capita in the province7, additional resources are needed to meet the demand for timely rheumatology services.
Interestingly, some of the sites had a first point of contact with an allied health professional (AHP; RHU, EAC, and many TAP patients) and in some cases data were available on the wait time to this first point of contact (although this was not the focus of our study). For example, at the EAC site in 2015, 79 patients saw the AHP prior to a visit with a rheumatologist (99% of patients), and median time to the AHP was 32 days, with a 90th percentile of 100 days (compared with a median of 45 days and a 90th percentile of 83 days for rheumatologist consultation). It is unclear, because data collection was started after this model of care was implemented, whether first contact with an AHP has decreased overall waiting times for patients with inflammatory arthritis. The EAC model of care team estimates that twice as many patients were screened to find eligible patients with suspected RA for the clinic, which likely shortened overall waiting times for these patients. Similar models of AHP screening have been implemented in 2 of the other sites we evaluated (RHU and TAP) and a similar phenomenon of shorter waiting times for AHP review was also observed at TAP (data not available for RHU). Further study of the effect of having an AHP as a first contact, and ongoing monitoring of the effect on waiting times at these sites, are recommended but were not the objectives of the current project.
Waiting times in the 2 sites serving indigenous populations in Alberta were challenging to report on as a component of the model of care was self-referral (which was an exclusion criterion for reporting). This should not be taken as an indication of “timely” diagnosis because it is still possible that patients may have had prolonged symptoms and/or multiple healthcare barriers20 before seeking care, leading to a delay in diagnosis and/or treatment. Indeed, evidence from Alberta21,22, other provinces in Canada23, and around the world24 suggests a high burden of arthritis among indigenous populations as well as disparities in patient outcomes22,25. Thus, waiting times for consultation will need to be examined in closer detail.
While we were hoping that all performance measures could be reported at each site, this was a challenge owing to the available data collected by the models of care and/or the length of time the sites had been operational. Currently there is no funding for any of the models of care to routinely collect and report on these important metrics. Available data on the DMARD measures collected from 2 cohorts suggest high levels of treatment with DMARD and low waiting times to DMARD starts, with a majority of patients treated at the time of diagnosis by a rheumatologist. This finding is in keeping with other studies18. However, the lack of delay between rheumatologist diagnosis and starting DMARD should not be interpreted as timely treatment with DMARD, because waiting times between symptom onset and DMARD start may be longer and suboptimal, as previously found in other studies26.
Percentage of patients seen in yearly followup could only be evaluated in the SASP and UAACP cohorts with results combined because of low numbers. Followup rates were excellent and comparable with rates of annual followup recently evaluated in the CATCH cohort18. One potential issue that may not be adequately addressed by the performance measure is attrition from patient factors unrelated to quality of care (e.g., moving away or withdrawing consent). In our analysis, we excluded a small number of patients who dropped out within the first year of cohort entry or had less than 1 year of followup by study end, because it was possible they had moved away or withdrawn consent for followup. For this reason, removal may have inflated performance on the measures, during the first year of expected followup.
While this is the first study, to our knowledge, looking at the feasibility of reporting on recently developed system-level performance measures using triage or clinic data available from multiple sites, there are a number of limitations to this work. As noted, performance measures could not be reported for all sites because data were lacking. In the future, we recommend that models of care directed at improving access to care and early treatment routinely receive funding to collect basic data including the date referrals were received, date of visits (including to rheumatologist and/or AHP), date of DMARD start, and final diagnosis. While the majority of the data we included was on consecutive patients, it should also be highlighted that the TAP site included patients already participating in an early arthritis cohort study (this was how we were able to identify RA cases, because no other data source was readily available). The site does try to include all early RA cases in CATCH; however, it is unclear whether those patients recruited could have differed in their waiting times or in performance of other measures from patients not included, or whether the study protocol could have influenced care. The numbers of sites included in this study was also small (n = 5), with low numbers of patients at many sites, and results may not be representative of other centers or provincial performance on the measures. Lastly, because the focus of the study was to assess the feasibility of measurement, additional details about each cohort were not recorded, including potential predictors of poor adherence to the measures. For example, seronegative cases or those with very low disease activity may have longer waiting times than cases with very active disease. Examining predictors of performance on the measures as well as whether better performance improves longterm patient outcomes will be an important ongoing line of study.
This study has uncovered gaps in data collection that make it challenging to evaluate models of care promoting early access to care. While it is possible that these gaps are unique to the sites we selected, we suspect similar issues with data collection and reporting are likely present in other jurisdictions in Canada, because reporting on quality metrics is currently not mandated. We recommend instituting basic data collection and analysis to evaluate performance when new models of care are initiated, and this should be included in the funding model. Such data can be helpful in monitoring successes as well as potential unintended consequences of a model of care, and can also be useful when advocating for resources including additional healthcare staff.
Acknowledgment
The authors acknowledge the help of Diane Tin, BSc Phm, Pharmacist, CATCH Liaison, The Arthritis Program at Southlake Regional Health Centre, Newmarket, Ontario; Tyler White, CEO, Siksika Health Services, Siksika Nation, Alberta; Lana Potts, MD, CCFP, Medical Director, Elbow Healing Lodge, Aboriginal Health Program, Alberta Health Services, Alberta; and Grishma Patel, Summer research associate, William Osler Health System, Brampton, Ontario.
Footnotes
Funding provided by a grant from the Canadian Initiative for Outcomes in Rheumatology Care.
- Accepted for publication March 2, 2018.
