Design paper
A Randomized Trial of Evidence-Based Outreach (EBOR): Rationale and Design

https://doi.org/10.1016/S0197-2456(99)00023-9Get rights and content

Abstract

There is increasing interest in evaluating the methods used to implement the findings from medical research. This paper describes the Evidence-based OutReach (EBOR) trial, which is the first large randomized study in the United Kingdom that will evaluate the effectiveness and efficiency of educational outreach visits by trained pharmacists who are delivering messages derived from four evidence-based clinical practice guidelines. General practices form the unit of allocation and analysis. The study design addresses important factors that may influence the effectiveness of the intervention, such as the pharmacist who delivers the messages, the health authority in which practices are located, and the size of a practice. Control Clin Trials 1999;20:479–492

Introduction

Concern that findings from research do not automatically bring about changes in practice 1, 2 has led to considerable interest in the introduction and evaluation of methods promoting their application 3, 4. The use of experimental processes in the evaluation of the effectiveness and efficiency of implementation methods raises a number of fundamental methodologic issues and requires approaches different from those appropriate to clinical trials where individual patients are randomised 5, 6, 7. These issues include the selection of subjects, the choice of the unit of analysis, the choice of interventions and outcomes, and the implications of clustering and hierarchical structures. All are relevant to the choice of approach to the statistical and economic analysis.

The Evidence-based OutReach (EBOR) trial will provide guidance to policy-makers in the U.K. National Health Service (NHS) on the effectiveness and cost-effectiveness of the policy called “academic detailing” whose use has been increasing and by which trained, community-based pharmacists visit family practitioners in an attempt to influence prescribing. This paper describes the trial's rationale and design.

Section snippets

Background

In the United Kingdom, family practice is configured in contracting units of doctors (general practices) that provide services to the NHS on a mixed capitation and fee-for-service basis. Currently, fund-holding practices have cash-limited prescribing budgets and the planned health service reforms will extend to all practitioners [8]. Prescribing for NHS patients is limited to those products listed in the British National Formulary. Few mechanisms, however, may influence the choice of drug

Intervention

Unusually for implementation interventions, the principles and practice of educational outreach visits have been well described [11]. They involve seven key techniques:

  • 1.

    Investigating baseline knowledge and motivations for current activity

  • 2.

    Focusing programs on specific categories of physicians as well as on their opinion leaders

  • 3.

    Defining clear educational and behavioral objectives

  • 4.

    Establishing credibility through a respected organizational identity, referencing authoritative and unbiased sources of

Health authority recruitment

We recruited 12 health authorities, six located in the south of England (in the North Thames NHS Region) and six in the north (within the former Yorkshire Regional Health Authority). For practical reasons, we did not sample health authorities randomly from those in England, but rather chose a group of authorities that reflects the breadth of variation among authorities.

Authorities included mean (SE) 263 (28) general practitioners (range 105 to 442). Practices within authorities differed

Practices

We selected four practices at random from lists provided by each health authority. Selection was stratified, with two practices selected from below and two from above the median-sized practice. We undertook no further stratification because of confounding (i.e., larger practices are often fund-holding practices) and, more fundamentally, because of the small number of practices selected in each authority.

We needed to have at least three practitioners prescribing around the national average in

Educational intervention

We recruited 12 community pharmacists on a locum basis to provide outreach visits to practices within their localities. Each pharmacist received training in two of the guidelines. This training, which we describe in detail elsewhere, addressed both specific content and marketing skills, with extensive role-play and practice orientation. Each pharmacist was responsible for delivering the outreach intervention to practices within the authority in which he or she worked. The intervention consisted

Allocation of guidelines

In each health authority, the practices received outreach visits for two of the guidelines, with prescriptions for the other two conditions acting as controls. With four guidelines available, we could choose six unique combinations of two guidelines for active dissemination. We allocated each of these possible choices at random to one of the authorities in the north and one in the south. The control intervention was simply the publication and dissemination of all four guidelines.

This sort of

Timing of Evaluation

To avoid the potential of a Hawthorne effect (a change in prescribing behavior that resulted simply from involvement in a study), we did not estimate the treatment effect in practices until after the intervention was complete, following a period during which prescribers could incorporate messages they had received from the outreach workers in practice. Because the guidelines often indicated substantial changes in repeat prescribing, we left a period of 6 months, whenever possible, for practices

Secondary outcome measures

Although we shall derive primary outcome measures from patient notes, we shall determine secondary outcomes from non–patient-linked drug reimbursement data by practice, for example, the overall prescription of antidepressant drugs. This determination will enable an assessment of the extent to which focused changes in prescribing in specific areas (identified through the main outcomes) are generalized.

Unit of Analysis

Because the trial focuses on professional prescribing behavior, it would be inappropriate for the patient to be the unit of analysis, even if we used estimates of intracluster variability to “correct” for the reduction of power through cluster randomization [6]. If, within a practice, we could attribute prescribing to individual doctors, such an approach may have provided a more efficient analysis than aggregating to the level of practice, although such data are rarely available routinely.

Discussion and conclusions

The EBOR trial is a real-world randomized study that will provide internally and externally valid estimates of the effect of educational outreach visits by trained community pharmacists, intended to improve the quality of prescribing in English primary care. Its complex design is not reflected in the data from the individual health authorities taking part; selected practices simply receive two guidelines by outreach. The design does, however, have substantial advantages over a simple randomized

Acknowledgements

EBOR is funded by the R&D Division of the U.K. Department of Health. The views expressed in the paper are those of the authors and not necessarily those of the Department of Health. We are very grateful to the general practitioners who have agreed to participate in the educational initiative, the health authorities that have collaborated with us, and the pharmacists who have undertaken the outreach visits. Without the participation of these groups, the project would not be possible.

We are most

References (24)

  • S. Juul-Möller et al.

    Double blind trial of aspirin in primary prevention of myocardial infarction in patients with stable chronic angina pectoris

    Lancet

    (1992)
  • E.M. Antman et al.

    A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts

    JAMA

    (1992)
  • J. Lomas et al.

    Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians

    N Engl J Med

    (1989)
  • D.A. Davis et al.

    Changing physician performancea systematic review of the effect of continuing medical education strategies

    JAMA

    (1995)
  • D.A. Davis et al.

    Evidence for the effectiveness of CME. A review of 50 randomized controlled trials

    JAMA

    (1992)
  • N. Freemantle et al.

    Evidence into practice, experimentation and quasi experimentationare the methods up to the task?

    J Epidemiol Community Health

    (1998)
  • J. Wood et al.

    Choosing an appropriate unit of analysis in trials of interventions that attempt to influence practice

    J Health Serv Policy

    (1999)
  • J.M. Mason et al.

    Designing evaluations of interventions to change professional practice

    J Health Serv Policy

    (1999)
  • The New NHS

    (1997)
  • K. Bloor et al.

    Lessons from international experience in controlling pharmaceutical expenditure IIinfluencing doctors

    BMJ

    (1996)
  • Thomson MA, Oxman AD, Haynes RB, et al. Outreach visits to improve health care professional practice and health care...
  • S.B. Soumerai et al.

    Principles of educational outreach (“academic detailing”) to improve clinical decision-making

    JAMA

    (1990)
  • Cited by (27)

    View all citing articles on Scopus
    View full text