Original Article
A practice-based tool for engaging stakeholders in future research: a synthesis of current practices

https://doi.org/10.1016/j.jclinepi.2012.12.010Get rights and content

Abstract

Objective

A major goal of patient-centered outcomes and comparative effectiveness research is to increase the involvement of stakeholders throughout the research process to provide relevant and immediately actionable information. In this report, we review the current practices for engaging stakeholders in prioritizing research.

Study Design and Setting

To evaluate the range of approaches to stakeholder engagement, we reviewed the relevant literature and conducted semistructured interviews with (1) leading research organizations in the United States, Canada, and the United Kingdom; and (2) eight Evidence-based Practice Centers that engage stakeholders in comparative effectiveness research.

Results

We identified 56 articles related to stakeholder engagement in research prioritization. Studies and research organizations interviewed frequently used mixed methods approaches combining in-person venues with structured ranking or voting processes such as Delphi. EPCs similarly used group web/conference calls combined with Delphi ranking or voting. Research organizations reported difficulties engaging the public and policy makers, and EPCs reported challenges engaging federal stakeholders.

Conclusion

Explicit and consistent use of terminology about stakeholders was absent. In-person techniques were useful to generate ideas and clarify issues, and quantitative methods were important in the prioritization of research. Recommendations for effective stakeholder engagement and a reporting checklist were developed from the accumulation of findings.

Introduction

What is new?

  • Increasing emphasis is placed on engaging stakeholders in research.

  • Consistent and explicit terminology to describe stakeholders and engagement methods is essential.

  • In-person techniques are useful to explain topics and to clarify and develop a deeper understanding of stakeholders’ perspectives.

  • Quantitative methods such as voting, survey, and Delphi technique are useful for prioritization.

  • We developed a checklist of practices and reporting in stakeholder engagement to promote consistency and advance the field.

Clinicians, patients, policy makers, and other stakeholders regularly face medical decisions in complex areas that have little or no published scientific evidence [1]. To inform this process and ensure a patient-centered research agenda in the United States, national health policy and research organizations have initiated patient-centered outcomes research and comparative effectiveness research to inform and improve health care [2], [3]. Consumer involvement has been shown to positively affect research by increasing its relevance to patients, families, and communities [4]. However, the best methods to engage a wide range of stakeholders in prioritizing research are largely unknown. To support a new future research needs initiative [5] by the Agency for Healthcare Research and Quality (AHRQ), we undertook a project to describe stakeholder engagement methods used by a range of academic institutions, practice-based research networks, nonprofit organizations, and other organizations devoted to health care issues on local and national levels in the United States, Canada, and Europe.

AHRQ created the Evidence-based Practice Centers (EPCs) in 1997 to conduct research reviews for the Effective Health Care Program. Housed at major universities, medical centers, and research institutions in the United States and Canada, the EPCs synthesize existing scientific literature about important health care topics and promote evidence-based practice and decision making. EPCs apply rigorous systematic review methods to produce systematic evidence reviews and comparative effectiveness reviews on medications, devices, and other health care services, with the goal of helping patients, physicians, and policy makers make better decisions about treatments. In addition to synthesizing the evidence, the reviews identify the gaps in evidence that limited the ability to answer the systematic review questions. AHRQ has worked with a formal, broad-based stakeholder group at the agency level, and stakeholder engagement is welcome throughout the phases of the EPC systematic review process (Fig. 1). In 2010, AHRQ expanded its support for EPCs to work with various stakeholders to further develop and prioritize the future research needed by decision makers. To develop guidance for this effort, AHRQ commissioned the Oregon and the Vanderbilt EPCs to conduct a collaborative multidimensional project to outline best practices for engaging stakeholders in defining and prioritizing research needs.

Section snippets

Methods

This project consisted of three complementary activities that occurred between May and September 2010, as shown in Fig. 2. We searched the published literature for methods of stakeholder engagement to set research priorities (phase I), conducted interviews with leading research organizations in the United States, Canada, and the United Kingdom known to engage stakeholders in their work (Phase II), and interviewed EPCs that engaged stakeholders for their future research pilot studies in 2010

Phase I: Literature summary

After the exclusion of editorials, commentaries, and studies that did not provide data for stakeholder engagement, we identified 56 articles that provided information about methods for engaging stakeholders specifically for the purpose of prioritizing future research (Fig. 3). The literature lacked consistent terminology and definitions for stakeholders and lacked consistency in processes for stakeholder engagement. For the purposes of this report, stakeholders are individuals or organizations

Discussion

National attention has recently focused on engaging patients, clinicians, and policy makers to create a more patient-centered research agenda that is relevant to everyday practice and directly actionable. Stakeholder involvement may increase a researcher’s understanding of the decisional dilemmas facing the public, thus increasing the relevancy of the research. Several themes emerged from the literature and interviews with research organizations and EPCs:

  • Engage stakeholders early in the process;

Acknowledgments

The authors extend their deepest appreciation for the contributions of the key informants and Evidence-based Practice Center participants who agreed to be interviewed for the project. This project was funded under Contract Nos. 290-2007-10057-I (Oregon Evidence-based Practice Center) and 290-2007-10065-I (Vanderbilt Evidence-based Practice Center) from the Agency for Healthcare Research and Quality, US Department of Health and Human Services.

References (13)

  • IOM Roundtable on Evidence-based Medicine. Learning What Works Best: The Nation's Need for Evidence on Comparative...
  • Initial national priorities for comparative effectiveness research

    (2009)
  • Federal Coordinating Council for Comparative Effectiveness Research. Report to the President and the Congress....
  • K. Staley

    Exploring impact: public involvement in NHS, public health and social care research

    (2009)
  • S. Chang et al.

    Identifying research needs for improving health care

    Ann Intern Med

    (2012)
  • C. Jinks et al.

    The Keele community knee pain forum: action research to engage with stakeholders about the prevention of knee pain and disability

    BMC Musculoskelet Disord

    (2009)
There are more references available in the full text version of this article.

Cited by (72)

View all citing articles on Scopus

Disclaimer: The authors of this document are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the US Department of Health and Human Services.

View full text