Skip to main content

Main menu

  • Home
  • Content
    • First Release
    • Current
    • Archives
    • Collections
    • Audiovisual Rheum
    • 50th Volume Reprints
  • Resources
    • Guide for Authors
    • Submit Manuscript
    • Payment
    • Reviewers
    • Advertisers
    • Classified Ads
    • Reprints and Translations
    • Permissions
    • Meetings
    • FAQ
    • Policies
  • Subscribers
    • Subscription Information
    • Purchase Subscription
    • Your Account
    • Terms and Conditions
  • About Us
    • About Us
    • Editorial Board
    • Letter from the Editor
    • Duncan A. Gordon Award
    • Privacy/GDPR Policy
    • Accessibility
  • Contact Us
  • JRheum Supplements
  • Services

User menu

  • My Cart
  • Log In

Search

  • Advanced search
The Journal of Rheumatology
  • JRheum Supplements
  • Services
  • My Cart
  • Log In
The Journal of Rheumatology

Advanced Search

  • Home
  • Content
    • First Release
    • Current
    • Archives
    • Collections
    • Audiovisual Rheum
    • 50th Volume Reprints
  • Resources
    • Guide for Authors
    • Submit Manuscript
    • Payment
    • Reviewers
    • Advertisers
    • Classified Ads
    • Reprints and Translations
    • Permissions
    • Meetings
    • FAQ
    • Policies
  • Subscribers
    • Subscription Information
    • Purchase Subscription
    • Your Account
    • Terms and Conditions
  • About Us
    • About Us
    • Editorial Board
    • Letter from the Editor
    • Duncan A. Gordon Award
    • Privacy/GDPR Policy
    • Accessibility
  • Contact Us
  • Follow Jrheum on BlueSky
  • Follow jrheum on Twitter
  • Visit jrheum on Facebook
  • Follow jrheum on LinkedIn
  • Follow jrheum on YouTube
  • Follow jrheum on Instagram
  • Follow jrheum on RSS
Research ArticleWorkshops and Special Sessions

The Third Biennial 2018 OMERACT First-time Participant Program: A Qualitative and Quantitative Study

Victor S. Sloan, Shawna Grosskleg, George Wells and Jasvinder A. Singh
The Journal of Rheumatology August 2019, 46 (8) 1036-1040; DOI: https://doi.org/10.3899/jrheum.181196
Victor S. Sloan
From the UCB Biosciences Inc., Raleigh, North Carolina; Rutgers-Robert Wood Johnson Medical School, New Brunswick, New Jersey; University of Alabama at Birmingham; Birmingham Veterans Affairs Medical Center, Birmingham, Alabama; Department of Orthopedics, Mayo Clinic College of Medicine, Rochester, Minnesota, USA; OMERACT; University of Ottawa; Ottawa Heart Institute, Ottawa, Ontario, Canada.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Shawna Grosskleg
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
George Wells
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jasvinder A. Singh
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Jasvinder A. Singh
  • For correspondence: Jasvinder.md@gmail.com
  • Article
  • Figures & Data
  • Info & Metrics
  • References
  • PDF
PreviousNext
Loading

Abstract

Objective. To assess the expanded/refined first-time participant training program.

Methods. We conducted a refined new participant program at OMERACT 2018 on days 1–4, in which first-time participants provided feedback with online surveys and a nominal group on Day 4.

Results. Twenty first-time participants attended the introductory session and 8–12 attended followup sessions. A high proportion valued the newbie session (100%), rating it overall (91%), content-wise (62%), for presentation quality (82%), and value for the money (82%) as outstanding or good. The nominal group technique identified opportunities for further improvement of breakouts/voting.

Conclusion. The expanded new participant training program is valued by attendees.

Key Indexing Terms:
  • OMERACT
  • FILTER
  • OUTCOME MEASURES
  • TRAINING SESSION
  • PARTICIPANT EXPERIENCE

The Outcome Measures in Rheumatology (OMERACT) develops validated outcome measures for clinical trials in rheumatic diseases. The success of OMERACT is due in large part to its adoption of rigorous methodology in the development of the core set for a disease of interest, and completing this process by achieving consensus1,2. The original OMERACT filter for a core measure consists of truth, discrimination, and feasibility (TDF)3. More recently, the OMERACT filter 2.0, i.e., the process for developing core domains for a condition or disease, was defined and described2, which evolved to filter 2.1 before the OMERACT 2018 meeting1.

Endorsement of a core domain or core measurement set for a condition occurs at OMERACT face-to-face meetings in 3 steps: (1) presentation of data supporting the validity of domain/measure by the working group (30 min); (2) discussion of evidence in detail in small group breakouts to achieve consensus (1.5 h); and (3) a report back from each breakout session, clarification of concerns raised during the breakout sessions by working groups, and an open discussion, followed by voting that results in endorsement or no endorsement of each domain/measure (30 min). Successful participation in the consensus process requires methodological knowledge/training about the actual OMERACT process (briefly described above).

Most first-time OMERACT participants are not primarily study methodologists, and given the new vocabulary, the OMERACT process, and the different types of sessions, the learning curve can be steep. At each face-to-face meeting, about 1 in 5 OMERACT attendees have no prior OMERACT experience. After the 2012 meeting, there were requests for more education on OMERACT for first-time participants prior to the next OMERACT meeting. The OMERACT executive agreed on the importance of such training, because all participants’ votes carry equal weight, so that better education would lead to more informed voting.

In 2014, the first formal OMERACT new participant training was implemented4. The results of this training showed an improvement in participants’ involvement and understanding of the meeting. This was repeated in 2016, with similar results, with the addition of more concrete suggestions for improvement of both the training and the OMERACT meeting process5. Our objective was to describe the process and results from the refined and expanded new participant session at OMERACT 2018, also referred to as the 2018 OMERACT newbie program. This program had a renewed focus on the new participant experience before, during, and after the meeting, in addition to methodology training more by discussion rather than didactic teaching.

MATERIALS AND METHODS

OMERACT first-time participant program

A first-time participant program was conducted at OMERACT 2018, consisting of a 1-h introductory session on the morning of Day 1, followed by a 1.5-hour question and answer session. There were 1-h evening sessions on days 1–4. Before the meeting, first-time participants were provided with premeeting reading materials and invitations to the sessions. The OMERACT premeeting reading materials provided by each working group contain a summary of the work completed in the 2 or more years prior to the meeting, including key evidence supporting the validity of core domains/measures. First-time participants were reminded of these meetings at plenary sessions and a schedule was provided at registration.

Introductory session format

Prior to any meetings, all first-time participants completed an anonymous survey assessing their familiarity with OMERACT, including the concepts of the OMERACT filters and the structure of the meeting. At the first session, there were presentations on OMERACT’s history and process, followed by open discussion and a question and answer session.

Daily evening session format

Discussions and debriefings in the 2018 OMERACT first-time participant program focused on the new participant experience before, during, and after the OMERACT meeting. Prior to each evening session, participants completed an anonymous online survey reassessing their comfort with the OMERACT structure and concepts. Two moderators staffed each 1-hour session (JAS, VSS). This was followed by a debriefing6,7 during each evening session, in which each participant was asked to list the “best” and “worst” experience of the day. The intent of these open-ended questions was to gauge the participants’ understanding of the process and to obtain their perspectives as new participants on what they felt went especially well (or poorly). These responses were recorded and analyzed by the session coordinator (SG). A post-meeting survey regarding the first-time participant experience itself was sent to all new participants. The session coordinator collected the responses.

During the last day of the meeting, we reviewed the most frequently cited concern(s) expressed by first-time participants from each day, which were confirmed by the group. Based on this, we used the nominal group technique (NGT) to ask 1 question to assess this concern: “What improvements do you think can make the breakouts and voting process at OMERACT more productive?” The NGT is a variant of traditional methods aimed at identifying the overall opinion of a group. The NGT is a structured process that facilitates development of an inclusive list of issues related to a specific question followed by feedback on the relative importance of these lists through rank-ordering6,7,8. The NGT approach promotes even more participation rates compared to focus groups, with equal weighting of input from all participants. An experienced NGT facilitator (JAS) first presented the questions, followed by participants noting their responses independently and reading each response to the facilitator, who recorded them verbatim. After all the responses had been presented by the participants in a round-robin fashion and listed by the facilitator (JAS), participants were asked to elaborate and discuss their responses. After discussion, none of the responses could be consolidated. Participants chose their top 5 responses and gave them individual rank scores from 1 to 3 (higher score indicating the top choice). Scores were aggregated for group rank-order of these questions, with higher scores representing the highest ranked questions.

First-time participant post-conference feedback

OMERACT first-time participants were asked about their experience in the OMERACT first-time participant session, including questions about the importance of a dedicated newbie session, the content and quality of the presentations and facilitation, the overall value of the OMERACT meeting, and whether they planned to attend OMERACT 2020 (to be held in the United States). There were free-text boxes for “What is the best thing about the first-time participant program?” and “What could be improved about the program?”

RESULTS

Participant characteristics and familiarity and comfort with the OMERACT process

There were 24 first-time participants. Characteristics of the new participants are shown in Appendix 1. Twenty first-time participants attended the introductory session and 8–12 attended each followup session on days 1–4. Eleven (46%) completed the post-meeting survey.

There was a high level of understanding of OMERACT at baseline. In 2018, by the end of Day 1, 100% said they felt extremely welcome at the OMERACT sessions, 88% felt that they could contribute during the breakout sessions, 86% understood the TDF (truth, discrimination, feasibility) OMERACT filter somewhat or extremely well, and 72% had some or extremely good understanding of the different types of OMERACT sessions. These proportions stayed stable or improved slightly through the entire duration of OMERACT. Compared to OMERACT Day 1 evening meetings in 2014 and 2016, a higher proportion reported understanding of the TDF filter in 2018 (12% vs 22% vs 43%, respectively).

First-time OMERACT participant experience

Free-text responses on the positive aspects of the newbie session included the fact that the session exists at all, that the session was structured as a small group with open discussion, the introduction to OMERACT concepts and processes and the ability to ask questions, and good/great facilitation of the session.

Areas for improvement included requests for video examples of the different types of OMERACT sessions, more premeeting materials on outcome measure development, a premeeting Webinar, more structure in the program, and improved facilitation.

Post-OMERACT 2018 survey

All responders felt that it was extremely important or important to have a dedicated newbie session. Almost all (91%) felt that the newbies sessions were either outstanding or good, and most (62%) felt that the content was either outstanding or good. Most (82%) felt that the quality of the presentations was outstanding or good (Table 1). While 9% felt that “way too much” time was devoted to the sessions, 64% felt that “a little too much” time was devoted, and 27% felt the time devoted was “just right,” suggesting that minor adjustments to timing should be considered. No participants rated any aspect of the sessions as bad or unacceptable. Despite the time concerns, 82% felt that the value for money of the newbie meeting was outstanding or good, and 18% thought it to be just right, and most (82%) plan to attend the OMERACT 2020 meeting.

View this table:
  • View inline
  • View popup
Table 1.

First-time participants’ post-OMERACT conference feedback survey regarding the 2018 OMERACT newbie session (n = 11).

Nominal group

At the end of Day 4, we conducted an NGT with 8 participants addressing 1 question. New participants identified and ranked their responses to the question as to how to make the OMERACT breakouts and voting process more productive (Table 2). The top 4 ranked themes that garnered 58% of the votes (28 of the 48) were the following:

  1. Improved facilitation during the breakouts (15 votes)

  2. The need for more discussion time in general meeting room after breakout sessions for comments from the team/each breakout to be addressed by the working group (5 votes)

  3. Tough structure for the voting process (4 votes)

  4. Encouraging participation in the breakout sessions (4 votes)

View this table:
  • View inline
  • View popup
Table 2.

2018 OMERACT First-time participant Nominal Group Technique (n = 8) addressing the question “What improvements do you think can make the breakouts and voting process at OMERACT more productive?”

DISCUSSION

Our report describes results of the third formal OMERACT first-time participant training program, results of the post-meeting survey, and innovative solutions to problematic aspects of the new-attendee experience as seen by those with limited OMERACT experience. At OMERACT 2018, baseline understanding and comfort levels of participants were high. A much higher proportion of first-time participants reported understanding the OMERACT TDF filter extremely well, compared to those in 2014 and 2016 (12% vs 22% vs 43%), which supports the expanded/refined program with a focus on the new participant experience before (pre-meeting readings), during, and after the meeting. The OMERACT 2018 newbie program included an updated shorter presentation of the OMERACT process, a detailed open-forum discussion on the OMERACT TDF filter and the OMERACT filter 2.0, a shift in focus to that of the first-time participants’ own experience, the identification of best and worst experiences of this group during the daily evening sessions, and an elaboration of innovative solutions to the areas for improvement. To continue to improve the experience of OMERACT first-time participants, and thereby encourage their continued involvement in OMERACT, their solutions will be provided to OMERACT’s Executive Committee for possible implementation.

The nominal group findings provide insight and present suggestions for future consideration and improvements. These findings should be interpreted with caution, because these opinions and observations come from a small subgroup of all people attending the OMERACT meeting — the first-time participants. Low response rate for the final survey (46%) limits the generalizability of survey findings.

While OMERACT first-time participants generally had positive opinions of the in-meeting sessions, they continue to feel that more premeeting preparation would have been of benefit. There was a specifically identified need for more preparation regarding the central OMERACT process of outcome measure development, as well as video examples of the different types of OMERACT sessions.

APPENDIX 1

View this table:
  • View inline
  • View popup
APPENDIX 1

Participant characteristics for the 2018 OMERACT first-time participant program, as compared to similar sessions at OMERACT 2014 and 2016.

Footnotes

  • VSS is an employee of UCB Biosciences, Inc. JAS has received consultant fees from Crealta/Horizon, Fidia, UBM LLC, Medscape, WebMD, the National Institutes of Health and the American College of Rheumatology. JAS is a member of the Veterans Affairs Rheumatology Field Advisory Committee. JAS is the editor and the director of the UAB Cochrane Musculoskeletal Group Satellite Center on Network Meta-analysis. JAS served as a member of the American College of Rheumatology’s (ACR) Annual Meeting Planning Committee (AMPC) and Quality of Care Committees, the Chair of the ACR Meet-the-Professor, Workshop and Study Group Subcommittee and the co-chair of the ACR Criteria and Response Criteria subcommittee. JAS is a member of the executive of OMERACT, an organization that develops outcome measures in rheumatology and receives arms-length funding from 36 companies.

  • Accepted for publication November 14, 2018.

REFERENCES

  1. 1.↵
    1. Boers M,
    2. Kirwan JR,
    3. Tugwell P,
    4. Beaton D,
    5. Bingham CO III,
    6. Conaghan PG,
    7. et al.
    The OMERACT Handbook. [Internet. Accessed May 17, 2017.] Available from: https://omeract.org/resources
  2. 2.↵
    1. Boers M,
    2. Kirwan JR,
    3. Wells G,
    4. Beaton D,
    5. Gossec L,
    6. d’Agostino MA,
    7. et al.
    Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol 2014;67:745–53.
    OpenUrlCrossRefPubMed
  3. 3.↵
    1. Boers M,
    2. Brooks P,
    3. Strand CV,
    4. Tugwell P
    . The OMERACT filter for outcome measures in rheumatology. J Rheumatol 1998;25:198–9.
    OpenUrlPubMed
  4. 4.↵
    1. Sloan VS,
    2. Grosskleg S,
    3. Pohl C,
    4. Wells GA,
    5. Singh JA
    . The OMERACT first-time participant (“newbie”) program: initial assessment and lessons learned. J Rheumatol 2015;42:1976–81.
    OpenUrlAbstract/FREE Full Text
  5. 5.↵
    1. Sloan VS,
    2. Grosskleg S,
    3. Pohl C,
    4. Wells GA,
    5. Singh JA
    . The OMERACT first-time participant program: fresh eye from the new guys. J Rheumatol 2017;44:1560–3.
    OpenUrlAbstract/FREE Full Text
  6. 6.↵
    1. Gallagher M,
    2. Hares T,
    3. Spencer J,
    4. Bradshaw C,
    5. Webb I
    . The nominal group technique: a research tool for general practice? Fam Prac 1993;10:76–81.
    OpenUrlCrossRefPubMed
  7. 7.↵
    1. Miller D,
    2. Shewchuk R,
    3. Elliot TR,
    4. Richards S
    . Nominal group technique: a process for identifying diabetes self-care issues among patients and caregivers. Diabetes Educ 2000;26:305–10, 12, 14.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Hegger MJ
    . A fresh approach to group goal setting. The three-way rotation nominal group technique. J Nurs Staff Dev 1986;2:13–7.
    OpenUrlPubMed
PreviousNext
Back to top

In this issue

The Journal of Rheumatology
Vol. 46, Issue 8
1 Aug 2019
  • Table of Contents
  • Table of Contents (PDF)
  • Index by Author
  • Editorial Board (PDF)
Print
Download PDF
Article Alerts
Sign In to Email Alerts with your Email Address
Email Article

Thank you for your interest in spreading the word about The Journal of Rheumatology.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
The Third Biennial 2018 OMERACT First-time Participant Program: A Qualitative and Quantitative Study
(Your Name) has forwarded a page to you from The Journal of Rheumatology
(Your Name) thought you would like to see this page from the The Journal of Rheumatology web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
The Third Biennial 2018 OMERACT First-time Participant Program: A Qualitative and Quantitative Study
Victor S. Sloan, Shawna Grosskleg, George Wells, Jasvinder A. Singh
The Journal of Rheumatology Aug 2019, 46 (8) 1036-1040; DOI: 10.3899/jrheum.181196

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero

 Request Permissions

Share
The Third Biennial 2018 OMERACT First-time Participant Program: A Qualitative and Quantitative Study
Victor S. Sloan, Shawna Grosskleg, George Wells, Jasvinder A. Singh
The Journal of Rheumatology Aug 2019, 46 (8) 1036-1040; DOI: 10.3899/jrheum.181196
del.icio.us logo Twitter logo Facebook logo  logo Mendeley logo
  • Tweet Widget
  •  logo
Bookmark this article

Jump to section

  • Article
    • Abstract
    • MATERIALS AND METHODS
    • RESULTS
    • DISCUSSION
    • APPENDIX 1
    • Footnotes
    • REFERENCES
  • Figures & Data
  • Info & Metrics
  • References
  • PDF

Keywords

OMERACT
FILTER
OUTCOME MEASURES
TRAINING SESSION
PARTICIPANT EXPERIENCE

Related Articles

Cited By...

More in this TOC Section

  • Instrument Selection Using the OMERACT Filter 2.1: The OMERACT Methodology
  • The OMERACT Core Domain Set for Clinical Trials of Shoulder Disorders
  • Endorsement of the 66/68 Joint Count for the Measurement of Musculoskeletal Disease Activity: OMERACT 2018 Psoriatic Arthritis Workshop Report
Show more Workshops and Special Sessions

Similar Articles

Keywords

  • OMERACT
  • FILTER
  • OUTCOME MEASURES
  • TRAINING SESSION
  • PARTICIPANT EXPERIENCE

Content

  • First Release
  • Current
  • Archives
  • Collections
  • Audiovisual Rheum
  • COVID-19 and Rheumatology

Resources

  • Guide for Authors
  • Submit Manuscript
  • Author Payment
  • Reviewers
  • Advertisers
  • Classified Ads
  • Reprints and Translations
  • Permissions
  • Meetings
  • FAQ
  • Policies

Subscribers

  • Subscription Information
  • Purchase Subscription
  • Your Account
  • Terms and Conditions

More

  • About Us
  • Contact Us
  • My Alerts
  • My Folders
  • Privacy/GDPR Policy
  • RSS Feeds
The Journal of Rheumatology
The content of this site is intended for health care professionals.
Copyright © 2025 by The Journal of Rheumatology Publishing Co. Ltd.
Print ISSN: 0315-162X; Online ISSN: 1499-2752
Powered by HighWire