Abstract
Objective. The dissemination and adoption of clinical practice guidelines (CPG) has been suggested as one method for improving arthritis care delivery. This article provides a review and synthesis of studies evaluating the influence of educational programs designed to implement CPG for osteoarthritis (OA) and rheumatoid arthritis (RA) in primary care.
Methods. A systematic literature search was conducted to identify relevant educational interventions that reported behavioral outcomes that ensured actual knowledge utilization in primary care. A standardized approach was used to assess the quality of the individual studies and a modified version of the Philadelphia Panel methodology allowed for grading of studies based on strength of design, clinical relevance, and statistical significance.
Results. The search identified 485 articles; 7 studies were selected for review. In OA, peer facilitated workshops with nurse case-management support for patients decreased the number of referrals to orthopedics by 23%, and educational outreach by trained physicians improved prescribing of analgesics. Interprofessional peer facilitated workshops were successful in increasing referrals to rehabilitation services for people with OA and RA.
Conclusion. There was sparse literature on educational programs for the implementation of arthritis CPG in the primary care environment. Future studies are needed to evaluate which specific organizational, provider, patient, and system level factors influence the uptake of arthritis CPG in primary care.
Arthritis and related conditions affect over 4.5 million Canadians aged 15 years and older and in 2003 alone accounted for about 9 million physician visits1. Most treatment for people with arthritis occurs in primary care settings; however, many studies have documented the need for improved arthritis management in this environment2,3,4,5,6,7. The dissemination and adoption of clinical practice guidelines (CPG) has been suggested as one method for improving care delivery and patient outcomes. CPG can be used as a tool to address gaps in care as part of broader quality assurance initiatives8 and to reduce variations in practice9. Grimshaw, et al10 and Davis and Taylor-Vaisey9 have reviewed the general literature on CPG implementation in a variety of healthcare settings; however, neither review addresses arthritis specifically nor identifies the studies described in the present review. We review and synthesize studies evaluating the influence of educational programs to implement CPG for osteoarthritis (OA) and rheumatoid arthritis (RA) in primary care.
MATERIALS AND METHODS
In consultation with a library science professional, a systematic literature search was conducted using Cochrane, Embase, PubMed, and CINAHL databases to identify relevant educational interventions and factors that might influence utilization of arthritis CPG in primary care. Articles were included if they were in English, were published between 1994 (when the first arthritis CPG were published in the USA) and 2009, and were related to implementation of arthritis CPG in the primary care environment. US National Library of Medicine Medical Subject Headings included Arthritis or Rheumatic Disease and Guideline, Best Practice, Professional Education (including Continuing Medical Education), Professional Development, Disease Management or Evidence-based Practice. The reference lists of retrieved articles were also reviewed for relevant articles. Articles were selected for review if they were prospective evaluation studies that targeted primary healthcare providers working with adults with RA or OA and if they reported behavioral outcomes that ensured actual knowledge utilization in primary care.
A standardized approach was used to assess the quality of the individual studies, as based on methods recommended by Law, et al11,12. Guidelines and an accompanying form allowed each article to be evaluated based on the outcome measures chosen, potential study biases, intervention integrity, and appropriateness of analytical methods. For each study, we identified potentially important limitations. To guide our summary and interpretation of the findings, we used a modified version of the Philadelphia Panel methodology13. This framework allowed for grading of studies based on strength of design, clinical relevance, and statistical significance, as summarized in Table 1.
RESULTS
The search identified 485 articles relating to implementation of arthritis guidelines; 7 studies were selected for review based on inclusion and exclusion criteria described above. Figure 1 illustrates the inclusion/exclusion criteria and the success of the search strategy. The studies represented the following 3 educational strategies for dissemination of guidelines: (1) educational outreach, (2) peer-facilitated workshops, and (3) audit and feedback. Each study is described briefly below by type of intervention.
Educational outreach
Two randomized controlled trials (RCT)14,15 provided evidence that educational outreach improved physician prescribing behavior. Educational outreach can be defined as “a personal visit by a trained person to health professionals in their own setting”16. In a well designed study, Ray, et al evaluated the effect of a physician education program on reducing longterm exposure to nonsteroidal antiinflammatory drugs (NSAID) in elderly patients15. This program was based on American College of Rheumatology guidelines for hip and knee OA and included several strategies, namely educational outreach by physician educators, reminder systems, and nurse followup. There was a significant reduction in the number of patients taking NSAID (7%) in the intervention group relative to the control group at one-year followup. Seventy-three percent of the participating physicians received the full intervention. In this group, there was a 15% increase in the days of acetaminophen use relative to the control group and a 10% decrease in the number of patients taking NSAID, both findings in accordance with the guidelines to decrease NSAID use in the elderly. The study’s main limitation was high physician attrition (27%).
In the Evidence-Based OutReach (EBOR) study, Freemantle, et al14,17 evaluated the effectiveness of an educational outreach program delivered by trained pharmacists on physician prescribing practices for 4 conditions, including paracetamol as first-line treatment, followed by NSAID for nonspecific joint pain. There was a significant improvement in overall prescribing practices for the 4 conditions. However, for joint pain, there was a 3% decrease in the number of patients being managed according to the guidelines (p value not provided). The authors speculated that this could have been due to failure to monitor over-the-counter medication use or due to a lack of power to detect change in prescribing practices for nonspecific joint pain. Lack of diagnostic specificity, i.e., nonspecific joint pain, was also an issue in this study, resulting in an inability to generalize the findings to OA or RA. In a followup to this study, Nazareth, et al evaluated process outcomes that contributed to the primary outcome18. Participating physicians were surveyed and rated the visits highly, but only 63% reported application of the joint pain guideline in practice. Feedback from pharmacists indicated that general practitioners experienced difficulties in identifying appropriate patients with arthritis, and some were skeptical of the guidelines and lacked interest in changing their behaviors.
Peer-facilitated interactive workshops
Four studies evaluated the effect of peer-facilitated interactive workshops to improve physicians’ management of arthritis. Interactive workshops can be defined as any workshop where there is group interaction among participants19. Using a 3-arm clustered RCT, Rosemann, et al20 evaluated the effect of an educational intervention based on European League Against Rheumatism guidelines for the management of OA. One intervention arm consisted of peer-facilitated group meetings and a second intervention arm consisted of peer-facilitated group meetings plus nurse case-management. Behavioral outcomes were x-ray orders, referrals to orthopedics, and prescription of analgesics and antiinflammatory medications20. At 9 months postintervention, participants in both intervention arms of the trial decreased their x-ray orders and increased prescriptions for acetaminophen in keeping with the guidelines, relative to the control arm (p ≤ 0.05). The intervention arm, which included nurse case-management, had an additional increase in prescriptions for opioids (p ≤ 0.02) and a decrease in referrals to orthopedics (p = 0.04) in accord with the guidelines. Unfortunately, details were insufficient regarding the control group, training and reliability of chart extractors, blinding of assessors, and delivery of workshop content.
Glazier, et al evaluated the effects of the Getting a Grip on Arthritis educational program on the use of 10 arthritis best practices21, based on Ontario Program for Optimal Therapeutics guidelines for management of OA and RA22, and invited multidisciplinary providers from 5 Ontario community health centers (CHC) to participate. The program incorporated constructs of social cognitive theory and included an interactive 2-day interprofessional workshop that focused on arthritis best practices and skills enhancement (joint examination) delivered by trained local faculty. After the workshop, reinforcement activities were provided for participating organizations, their providers, and the community. One-year post-workshop, there was a significant increase in the number of referrals to rehabilitation services, compared to 2 control CHC, as well as improvements in both provider confidence and satisfaction and a reduction in their perceptions of barriers to arthritis care (p < 0.05). Further, patients from intervention CHC reported receiving more information on their type of arthritis, medications, disease management strategies, and community resources compared with the control group (p < 0.05). Major study limitations were the lack of randomization to intervention and control groups and potential lack of generalizability due to the unique characteristics of the CHC environment. In key-informant interviews one year after the workshop, providers indicated they had improved their assessment skills and their knowledge of arthritis and community resources, were more consistent in their pharmacological management of arthritis, improved their team function, and made more timely referrals to specialists.
In a 5-month RCT based on guidelines for prescribing NSAID or acetaminophen for OA, Rahme, et al evaluated the effect of the CURATA peer-facilitated workshops with and without a decision tree on physician prescribing behavior23. Eight towns in Quebec, Canada, were randomized into one of 4 intervention options for physicians. Group 1 (n = 84) received a peer and rheumatologist facilitated case-based accredited workshop and decision tree for OA (including nonpharmacological management). Group 2 (n = 29) received the workshop only. Group 3 (n = 54) received the decision tree only, and Group 4 (n = 82) received no intervention (control). Using intent to treat analysis, the authors reported a reduction in the number of arthritis prescriptions filled (assessed through a provincial administrative database) and improved prescribing practices in the 2 workshop groups compared to the control group [odds ratio (OR) 1.5 (95% CI 0.9, 2.3)]. The odds ratio for the “per protocol” analysis (excluding physicians who did not attend the workshop) was 1.9 (95% CI 0.9, 3.8). The largest improvement from baseline was associated with the peer-facilitated workshops (4%) compared to the control group (2%). Larger practices benefited less and recent graduates benefited more. Poor physician attendance due to bad weather was a problem in this study (20%).
In a 6-month multicenter RCT, Verstappen, et al evaluated the effects of the dissemination of the Dutch College of Primary Care Physicians guidelines for ordering of diagnostic tests for a number of chronic diseases24. In addition to guideline distribution, the intervention included small-group discussion and personal feedback reports on physician test ordering. Practices in 5 regions in The Netherlands were stratified by region and group size, then randomized to intervention or control groups. Overall, the mean number of tests per physician and the total number of inappropriate tests were significantly reduced in the intervention group (p = 0.01). For degenerative arthritis, in keeping with the guidelines, x-ray orders decreased 19% in the intervention group compared to 9% in the control group at 6 months postintervention; however, the difference was not statistically significant, possibly due to insufficient statistical power.
Audit and feedback interventions
Audit and feedback can be defined as any summary of clinical performance over a specified period of time that is given to healthcare providers in a written, electronic, or verbal format in order to improve performance25. Curtis, et al conducted a 7-month clustered RCT to determine the influence of chart audit, feedback, and educational materials on physicians’ use and monitoring of NSAID and cytoprotective agents26. Physicians were identified through randomly selected patients in an administrative pharmacy database and clustered by physician type (rheumatologists, internists, general physicians) in the analysis. There were no significant differences between intervention and control groups at followup, except that frequency of complete blood count testing increased 52% in the intervention group compared to 25% in the control group. Improved prescribing was more strongly related to physician type, with rheumatologists being more likely than family physicians to increase their rate of monitoring toxicity and patient factors (e.g., risk status, number of NSAID prescriptions, number of physician visits) than the intervention type (p < 0.05). A subset of the physicians (n = 50) was surveyed by fax to determine receipt of intervention materials; only 20 (40%) confirmed receipt of the materials. In this study the percentage of patients with a diagnosis of arthritis was unclear, and high physician attrition (16%) may have reduced the authors’ ability to detect group differences. A ceiling effect (some physicians following guidelines at baseline) might have also influenced the results.
Grading of interventions
As indicated above, the 7 studies were graded according to the Philadelphia Panel system (refer to Table 1). These results are presented in Table 2. The majority of outcomes were assigned a grade of C. The strongest evidence was associated with peer-facilitated workshops. There was Grade A evidence for workshops plus nurse case-management decreasing physician referrals to orthopedics in patients with OA (Rosemann20), as well as Grade B evidence for interprofessional workshops improving referral patterns for patients with OA and RA (Glazier21). In terms of educational outreach, there was Grade B evidence for statistically significant and clinically important improvements in physicians’ prescribing of acetaminophen for elderly patients with hip and knee OA, at least for those physicians who received the full intervention (Ray15). In the study by Curtis26, audit and feedback resulted in clinically important but not statistically significant improvements in the use and monitoring of cytoprotective agents (i.e., Grade C+ evidence).
DISCUSSION
This study highlights that there was sparse literature evaluating educational interventions for the implementation of CPG for arthritis in the primary care environment, particularly for RA and targeting nonphysicians. Based on the modified Philadelphia Panel grading system, there was modest improvement overall, with only 4 studies reporting changes of 15% or greater in behavioral outcomes. It is difficult to change health professional behaviors. For example, in a systematic review of the effectiveness and efficiency of guideline implementation strategies, Grimshaw, et al found only small to modest improvements in health professional outcomes and they suggest that a theoretical framework for professional and organizational behavior change might improve the choice of educational interventions10. In a systematic review of guideline dissemination strategies by Davies, et al27, the authors concluded that few strategies were explicitly theory based (6%) and that greater use of theory may lead to the design of better interventions. In this review, the study by Glazier21 was the only one to mention inclusion of a specific theoretical approach to behavior change.
Of the 7 studies reviewed, only one received an A grade in terms of study design and resulted in both statistical and clinically important behavioral outcomes. Peer facilitated workshops with nurse case-management support for patients decreased the number of referrals to orthopedics in OA by 23%20. One non-RCT (Getting a Grip on Arthritis) received a B grade for showing that interprofessional peer facilitated workshops were successful in increasing referrals to rehabilitation services for people with OA and RA21. These results support the findings of a review of the effects of continuing education meetings on professional practice by Forsetlund, et al19, which concluded that 6 of 10 studies on interactive workshops demonstrated small, significant, and potentially important effects on physician prescribing practices.
In addition, one well designed RCT provided fair (Grade B) evidence to support educational outreach by trained physicians for improving prescribing of analgesics for OA15. In a systematic review of the effects of strategies for the implementation of CPG on physician performance and healthcare outcomes, Davis concluded that educational out-reach was among the strongest methods for changing provider performance9. In a Cochrane review on educational outreach strategies, O’Brien concluded that educational outreach visits alone or in combination with other interventions have a small but potentially important effect on prescribing and small to modest improvements on other types of professional performance16.
Our review also suggested that audit and feedback might have promise for improving physician monitoring and use of NSAID and cytoprotective agents for OA26 (Grade C+ evidence). In other chronic diseases, there have been mixed results from audit and feedback interventions28,29,30,31,32,33,34,35. However, in their review article on CPG implementation strategies, Davis and Taylor-Vaisey concluded that audit and feedback was “moderately effective” in changing provider performance, particularly when the feedback was concurrent and delivered by peers or opinion leaders9. Often audit and feedback are a part of multifaceted interventions, making it difficult to determine the true effect of each component of the intervention. In a Cochrane Review on audit and feedback interventions, Jamtvedt, et al concluded that although results were varied, audit and feedback might be effective in improving practice, especially when baseline compliance with guidelines was low and when the intensity of the feedback was high25.
The results of our study point out several challenges in disseminating CPG in primary care. Several studies experienced difficulties in physician recruitment and retention15,23,26, a major challenge identified by others9,36,37. It has been suggested that more objective practice based assessments of the educational needs of physicians are needed in order to develop programs that are relevant to actual practice38. As well, many of the studies included multiple interventions, making it difficult to attribute outcomes to any one component of the program. Methodological issues such as lack of power, ceiling effects, and problems with program implementation may have been factors in the studies showing nonsignificant results. Failure to assess and report on the success of program implementation has been reported by others39.
Provider characteristics were also important. In the studies reviewed here, Rahme, et al concluded that more recent graduates may be more receptive to guideline implementation23. Curtis, et al found that improved prescribing was more related to physician type and patient characteristics, for example, higher patient risk status, than to the intervention itself26.
Rahme reported that providers in larger practices benefited less from the peer-facilitated workshop intervention23 and, in the Freemantle study14, the authors suggested that larger practices may make guideline implementation more complex. Practice size was measured differently in each study (number of prescriptions23, number of full-time equivalents14), making interpretation more difficult. Given the methodological issues involved in these studies and the different definitions used, the effect of practice size is still unclear.
Limitations in our review must be taken into account. Study selection was conducted by only one reviewer (SL), perhaps introducing bias. As well, the use of other methods for evaluating the quality of the literature, such as the Jadad score40, may have yielded different results.
In conclusion, literature on educational programs for the implementation of arthritis CPG in the primary care environment is sparse, particularly relating to RA and targeting nonphysicians. Educational outreach by trained physician educators may improve physician prescribing for OA, and peer-facilitated workshops with nurse case-management support may decrease referrals to orthopedics. In addition, interprofessional workshops facilitated by peers may improve referral patterns for both OA and RA. It will be important to evaluate what specific organizational, provider, patient, and system level factors influence the uptake of arthritis CPG in primary care. Future studies are needed to better understand the “active” component of the intervention and explore the importance of interprofessional learning in facilitating behavior change. It will also be important to address challenges in recruiting and retaining participants for followup. Future educational interventions might also benefit from the use of theory in their design.
- Accepted for publication March 25, 2010.