Abstract
Objective. To describe the experience of the first OMERACT Emerging Leaders Program (ELP).
Methods. A Delphi process identified positive aspects, areas for improvement, and future directions. Core items were defined as essential if they received ≥ 70% ratings.
Results. Participants valued relatable/accessible mentors (100%), including an OMERACT Executive mentor (100%), and a support network of peers (90%). Key items for future development were funding support (100%) and developing knowledge about OMERACT processes (90%) and politics (80%).
Conclusion. The ELP has the potential to provide targeted training for early career researchers to develop relevant skills for future leadership roles within OMERACT.
Outcome Measures in Rheumatology (OMERACT) is an international network of patients, clinicians, researchers, methodologists, and industry representatives that aims to improve and standardize outcome measurement in musculoskeletal clinical trials through a data-driven consensus process1,2. At a biennial conference, delegates have the opportunity to review and debate evidence presented by the working groups (WG) in facilitated sessions. Views expressed in these breakout sessions are then discussed with the wider group at a plenary session, and final consensus is sought through interactive voting1. A key principle of OMERACT is that all delegates have an active role. To help new delegates navigate the process and effectively contribute, education programs, including the Newbies3, Fellows, and Patient Programs, have been developed4.
The Fellows program was developed to educate and mentor early career researchers in the methods of OMERACT. It involves an opening session introducing participants to OMERACT’s history, philosophy, concept, and process, followed by daily mentor sessions to ensure comprehension and solidify understanding4.
Until 2018, participants could only attend the OMERACT Fellow program once, becoming regular delegates at subsequent meetings4. This model presented a lost opportunity for OMERACT to further develop “returning fellows” with the skills, knowledge, experience, and networks needed to develop into OMERACT leadership roles such as a WG co-chair, or member of the Technical Advisory Group (TAG) or the Executive, and thus ensure sustainability. Following the feedback from OMERACT 2016, and to address this gap, the Emerging Leaders Program (ELP) was piloted at OMERACT 2018.
Each OMERACT emerging leader (EL) was assigned 3 fellows to mentor. Daily sessions provided the EL with skills training and mentoring. EL were an available resource for WG needing rapporteurs. This report describes participants’ experiences and suggestions for improving the ELP.
MATERIALS AND METHODS
Participants
Fourteen registered returning OMERACT fellows were invited to participate in the ELP by e-mail 2 weeks before the meeting. The e-mail described the ELP as “a new initiative to better support you and offer you opportunities to develop valuable networks and skills. As future leaders of OMERACT, we are keen to support you in developing more advanced skills and competencies in the OMERACT Process and Methods.” Of these, 9 took part in the ELP (described in Table 1). Of the 5 returning fellows choosing not to participate, 2 of them cited insufficient time within the OMERACT conference as a reason, while others did not provide a reason. Following the meeting, all participants and the ELP convener were invited to take part in a Delphi survey (n = 10) to evaluate the ELP. Demographic data were collected by e-mail.
Format of the OMERACT 2018 Emerging Leaders pilot program.
Delphi process
The Delphi process5 was used to identify (1) positive aspects of the ELP, (2) the areas for improvement of the ELP, and (3) ways to improve the ELP. At OMERACT 2018, the emerging leaders were asked to brainstorm possible responses (“items”) for each of these questions, which were recorded by an experienced facilitator (BR). Participants were invited to submit additional items to the lead author (CAF) by e-mail. The Delphi survey was conducted online using DelphiManager. Participants scored each item from 1 to 9 into categories of not important (1–3), important (4–6), and essential (7–9). Items receiving ≥ 70% consensus as important/essential (4–9) were taken forward into the next round. At the end of Round 1, participants were invited to provide additional items for inclusion in subsequent rounds. In Rounds 2 and 3, participants were shown their previous response, and the percentage distribution of other participants’ responses for each item.
Finally, the 9 EL were asked 3 yes/no questions:
Overall, do you think the ELP was successful?
Would you consider taking part in a similar program in the future?
Would you recommend the ELP to a colleague?
Ethics approval was not required. Tacit consent to publish these data was received because all participants have contributed as co-authors.
RESULTS
Participants
All 9 EL and 1 ELP convener (BR) participated in all 3 rounds of the Delphi survey (n = 10). The majority (9/10) were women with a mean age of 37 years (SD 3.60). Five were from the United Kingdom and 5 were from Australia. Four EL were rheumatologists, 3 were researchers, 1 was a consultant podiatrist, and 2 were biomedical engineers; the EL convener was a rheumatologist.
What were the positive aspects of the ELP?
Twenty positive aspects of the program were identified (Table 2). All items received > 70% consensus that they were important or essential to the success of the ELP in rounds 1 and 2, and ≥ 80% consensus in Round 3. Twelve items received ≥ 70% consensus that they were essential to the success of the program (Table 2). The top 5 according to mean score (range 0–9) were “conveners as relatable/accessible mentors” (mean 8.8, 100% consensus); “having a support network of peers” (mean 8.6, 90% consensus); “OMERACT Executive representative as a mentor” (mean 8.5, 100% consensus); “development and strengthening of networks with other EL” (mean 8.3, 100% consensus); and “having a purpose at OMERACT in addition to the standard program” (e.g., having an identity beyond being a delegate, having dedicated evening sessions; mean 8.2, 100% consensus).
Consensus scores on the Emerging Leaders Program (n = 10).
What are the areas for improvement for the ELP?
Six areas for improvement were identified (Table 2). Five items received ≥ 70% consensus that they were either important or essential to the success of the ELP in Round 1 and were taken forward to Round 2. Four items received > 70% consensus that they were either important or essential in Round 2 and were taken forward to Round 3. In Round 3, all 4 remaining items reached ≥ 80% consensus that they were either important or essential (Table 2). Ranked by mean score, these were “discovering extra responsibilities as EL after agreeing to participate” (mean 8.8, 80% consensus); “not being sure what to expect from the program” (mean 8.6, 100% consensus); “lack of planned sessions to spend time with mentees” (mean 8.5, 80% consensus); and “being given short notice when asked to participate in the ELP” (mean 8.3, 100% consensus). Consensus was reached (100%) that being given short notice to participate was an important item but not essential. However, no consensus could be reached on whether the remaining items were important or essential.
What solutions do you propose to improve the ELP?
Twenty-three solutions to improve the ELP were identified, including 12 added during Round 1 of the Delphi (Table 2). All items received > 70% consensus that they were either important or essential for future development of the program in rounds 1 and 2 and ≥ 90% consensus in Round 3. Thirteen items received ≥ 70% consensus that they were essential for future development of the ELP (Table 2). The top 5 according to mean score (range 0–9) were “funding support for EL” (mean 8.4, 100% consensus); “opportunity to gain higher level knowledge about OMERACT (technical) processes” (i.e., the methods required for the development and endorsement of core domain and outcome measurement sets; mean 7.8, 90% consensus); “opportunity to gain higher level knowledge about OMERACT politics” (the governance and operational structure, e.g., how to establish a special interest group, how to navigate the route to become a WG co-chair, or of the TAG or Executive; mean 7.7, 80% consensus); “opportunity to learn about WG structures and how they should be led/managed” (mean 7.6, 100% consensus); and “opportunity to learn facilitation skills” (mean 7.5, 100% consensus).
Nine EL (100%) reported that the ELP was successful, and that they would take part in a similar program in the future and would recommend it to a colleague.
DISCUSSION
Our report describes the first OMERACT ELP, and identifies positive aspects, areas for improvement, and potential solutions for developing the program. The pilot program was well received with all the participants reporting that they considered it successful, would take part in future similar programs, and would recommend it to colleagues.
Items receiving consensus as essential to the success of the program were predominantly related to mentorship and support, personal and professional development, and contributing to “collaboration and collegiality” of OMERACT. The latter has been previously reported as a key aspect delegates value about OMERACT6. However, to our knowledge, the importance of developing specific skills in younger OMERACT participants such as mentoring, delivering effective feedback, and facilitation has not previously been reported. Opportunities for practical skill development in a supportive environment likely contributed to the perceived value of this program7.
Other benefits included the positive experience associated with mentoring. This was the first year that fellows were allocated an EL mentor in addition to their OMERACT Executive mentor. This extra level of support provided further opportunities for mentoring skill development, peer support, and networking.
No areas for improvement were considered essential to the success of the ELP. This was the first time this program was implemented with limited planning time, therefore areas for improvement relating to more information and advance notice will be easily addressed for OMERACT 2020.
Funding support for EL to attend OMERACT received the highest mean score for improving the ELP. It is likely that this reflects the difficulty faced by early career researchers in accessing institutional funding needed to attend OMERACT meetings. Areas for future development felt to be essential focused on opportunities to learn more about the technical processes and politics of OMERACT, continuing to develop existing skills, and contributing to OMERACT in more senior roles. Ideas for skills development fell into 3 broad categories: (1) mentoring and feedback; (2) methodology, process, and politics; and (3) facilitation and leadership. As EL numbers increase in future years, and with limited time available at each meeting to deliver targeted face-to-face workshops, a proposed model for future ELP with 3 streams is shown in Figure 1. This model provides a structure that aligns with the skills, knowledge, and experience required to take on leadership roles in OMERACT.
Flow diagram for potential streamlining of the OMERACT Emerging Leaders Program. OMERACT: Outcome Measures in Rheumatology; EL: emerging leaders; ELP: Emerging Leaders Program; SIG: Special Interest Group.
This report is limited by the small number of participants from only 2 continents. However, all EL took part and 100% completion was achieved in all rounds of the Delphi survey. Important positive areas were identified and key suggestions for development of the ELP have been formulated.
The 2018 OMERACT ELP was positively received and provided a targeted, supportive training opportunity for early career researchers. Inclusion of an ELP program in future OMERACT meetings is recommended. For inclusivity, opening the ELP invitation to all returning OMERACT participants (including returning “newbies” and returning EL) should be considered. Encouraging cross-generational relationships with formal mentoring, providing skills-based learning opportunities, and sharing institutional knowledge may benefit OMERACT regarding succession planning, efficiency of process, and organizational culture.
Acknowledgment
The authors thank the OMERACT Executive for supporting the OMERACT Emerging Leaders initiative, and Professor Paula Williamson and Richard Crew at Core Outcome Measure in Effectiveness Trials (COMET) for complimentary use of the DelphiManager Web-based system and support in study setup (http://www.comet-initiative.org/delphimanager).
- Accepted for publication January 8, 2019.
