Abstract
Objective. To implement a rheumatology department education retreat to systematically identify and address the key factors necessary to improve medical education in our division in preparation for developing a rheumatology academy.
Methods. The Hospital for Special Surgery organized a retreat for the Rheumatology Department aimed at (1) providing formal didactics and (2) assessing participants’ self-reported skills and interest in education with the goal of directing this information toward formalizing improvement. In a mixed-methods study design, faculty and fellows in the Division of Rheumatology were surveyed online pre- and post-retreat regarding various aspects of the current education program, their teaching abilities, interest and time spent in teaching, divisional resources allocated, and how education is valued.
Results. Enthusiasm for teaching was high before and rose further after the retreat. Confidence in abilities was higher than expected before but fell afterward. Many noted that the lack of specific feedback on teaching skills and useful metrics to assess performance prevented the achievement of educational excellence. Most responding felt lack of time, knowledge of how to teach well, and resources prevented them from making greater commitments to educational endeavors and participating fully and effectively in the department’s teaching activities.
Conclusion. While most rheumatology faculty members want to improve as teachers, they know neither where their educational strengths and weaknesses lie nor where or how to begin to change their teaching abilities. The key elements for an academy would thus be an educational environment that elevates the quality of teaching throughout the division and promotes teaching careers and education research, and raises the importance and quality of teaching to equivalence with clinical care and research.
Most academic rheumatologists are expected to teach, although often very little formal instruction in education is provided to give faculty the knowledge and tools to succeed in that important scholarly activity. Although it is assumed that most individual faculty members want to improve as teachers, they know neither where their educational strengths and weaknesses lie nor where or how to begin to change their teaching abilities. The lack of actionable, directed, and specific feedback and sensible and sensitive metrics to assess performance and improvement complicates attainment of educational excellence1. Further, teaching efforts are rarely appropriately compensated monetarily or adequately valued in promotion decisions2, resulting in high turnover and fast burnout of promising individuals. Those who wish to pursue high-quality and nationally recognized research in education have limited resources for funding and support. Complicating this is the finding that faculty whose values may be well aligned with institutional missions describe institutional behaviors that led them to conclude that excellence in academics was undervalued3. Education permeates every part of what we value as academic rheumatologists, including our responsibility to teach patients, medical students, residents, fellows, and the general public.
The faculty of the Hospital for Special Surgery (HSS) Rheumatology Division sees teaching as central to the mission of the division. Outstanding teaching is one of the defining characteristics of the division, with the Rheumatology Fellowship Program the educational activity with the highest influence. At the same time, multiple demands on the time of faculty and the practical reality that education, in contrast to patient care and basic science and clinical research, does not generate much revenue, limit the capacity and desire of some faculty to fully participate in a broad range of divisional teaching activities. In addition, the value of the teaching contributions of individual faculty members to the division’s mission has not been optimally measured, because those contributions are not as fully or reliably documented as those of research or patient care, and clear and unambiguous division-wide teaching priorities and metrics have not been delineated.
For this reason, we set out to define how to best strengthen the institution’s educational mission, envisioning an academy at the center to give structure, formality, and physicality to this endeavor, and using a planning retreat format to gain the insights, enthusiasm, and recognition of our faculty and fellows. The clear advantage of the retreat format was seen in its ability to bring together diverse persons with varying values for formal discussion and constructive input about the goals and objectives of the new academy. Participants would define the program development and identify the institutional politics and commitment, all key elements to succeeding in curricular reform4. The concept of the academy, where intellectual and financial resources are made available to academic clinician scholar educators and their research, is not new, having been successfully implemented at the University of California San Francisco (UCSF) and several other US medical schools5. A 2010 survey identified 36 out of 122 medical schools that had initiated academies since 20036. The new HSS Rheumatology Academy similarly aims to establish a stimulating and empowering academic educational environment that enhances the quality of teaching and promotes teaching careers and education research. Our study seeks to determine the key elements that would form the basis for the goals and activities of a newly created Academy of Medical Educators in our rheumatology department.
MATERIALS AND METHODS
Retreat format
In an attempt (1) to provide basic educational tools to the faculty, and (2) to understand what the education needs and priorities of the HSS Rheumatology Division were, we convened a day-long education retreat on a Saturday at the HSS in May 2011. The format, planned content, speakers, written materials, and activities for the day were planned and approved by the Rheumatology Education Council’s 6 members, all faculty with substantial interest and time devoted to teaching at all levels of learning. The 8-hour retreat day was divided into (1) formalized didactics given by invited educators in the areas of the history of medical education from Flexner7 onward, the UCSF Academy of Medical Educators, education feedback and the use of modern, Web-based technologies in medical education; and (2) breakout groups divided into 3 themes: (i) Educational Programs, (ii) Educator Development/Teaching Skills, and (iii) Institutional Values. Each group was assigned 1 facilitator and 1 recorder. About 20 min was spent brainstorming on each of the following: the present state, the desired future state, a list of “action items,” and time frames for the completion of each item and the faculty member(s) responsible for doing so. Following the sessions the recorder from each theme spent 15 min presenting a brief summary to attendees. The last third of the day was devoted to discussion of identifying the main elements that would form the basis of a newly created Academy of Medical Educators and revolutionize teaching in our department.
Survey design
Faculty survey
Surveys were developed to identify major issues regarding education at HSS and solicit the opinions of faculty about interest in teaching, time spent in education, and perception of educational resources available to them. The questions were agreed upon by the consensus of the Rheumatology Education Committee at HSS. All 6 faculty members generated, reviewed, and edited the wording of questions of interest to create a 30-item questionnaire to address satisfaction and consensus on the present and desired future states of education in the division (see Appendix A). Faculty were surveyed online pre- and post-retreat. Responses were assessed on a 5-point Likert scale, ranging from 1 = strongly disagree to 5 = strongly agree. The questionnaire was made available online and could be accessed only through passwords assigned at random. The responses were collated by an independent source with all identifiers removed to ensure anonymity.
Fellow survey
Fellows were asked to rate each one of the faculty in the division on overall teaching quality and mentoring ability and to rate the overall teaching quality in the department. In addition, trainees were asked to list the top 3 requirements to be a successful teacher, the 3 best teachers in the department, 3 things that could be improved, and whether all members of the faculty should be involved in teaching. When assessing “ability to teach,” trainees were asked to take into consideration appropriate and timely feedback, communication, professionalism, availability, and approachability (see Appendix B). These responses were also assessed on a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree.
RESULTS
Of the 41 faculty surveyed, 34 (83%) completed the survey pre- and 19 (46%) post-retreat. All 9 trainees (100%) responded to their questions and were surveyed only once. Of the 36 faculty respondents, the majority of those who had been at HSS < 10 years were women and those who had been at HSS > 20 years were men. Of those who had been working (i) < 10 years, 66.7% (6) were women and 33.3% (3) were men; (ii) between 10 and 20 years, 54.5% (6) were women and 45.4% (5) men; and (iii) > 20 years, 31.3% (5) were women and 68.8% (11) were men.
Teaching commitment
When asked the amount of time spent per week on activities related to trainee education on average, 33.3% stated that they spent 4 hours per week, and 22.2% spent 2 hours per week; 16.7% stated that they spent more than 1 day per week on teaching activities, although only 5.6% were spending more than 1 day a week in direct face-to-face teaching with trainees; 8.3% spent < 2 hours per week.
Confidence in teaching abilities
Pre-retreat survey
The majority of faculty answered that they agreed strongly or very strongly to the following questions: “I have the interest to be a great teacher” (percentage agreeing 91%; average rating 4.47), “I have the commitment to be a great teacher” (94%; 4.47), and “I understand what makes a great teacher (82%; 4.09). Most stated that they were already “satisfied with [my] performance as teacher” (71%; 3.74). More than half of the faculty had a high enough level of confidence in their abilities to respond in agreement that “I am an excellent teacher” (54%; 3.70), although not all felt comfortable that they had an “understanding of the latest pedagogical techniques” (36%; 3.33). The data showed that 42% agreed or agreed strongly that “I would like to be part of an Education Academy and move along a Clinician Educator track” pre-retreat (Table 1).
Post-retreat survey
Average ratings for “I have the interest to be a great teacher” rose from 4.47 pre- to 4.65 post-retreat and “I would like to teach more” rose from 3.35 pre- to 3.53 post-retreat. Average ratings for “I am satisfied with my performance as a teacher” fell from 3.74 pre- to 3.5 post-retreat, and “I am an excellent teacher” fell from 3.7 pre- to 3.5 post-retreat. Post-retreat interest in participation in the educational activities of an academy rose sharply (68%; 3.79) (Table 1).
Time and resources for teaching
Pre-retreat survey
Few agreed that they had “the time to be a great teacher” pre-retreat (39%; 3.15). Younger faculty in particular felt there was less time to teach than older members of the division (< 10 years, 2.5 vs > 20 years, 3.4). Similarly, few agreed that they had “the resources and tools to teach well” pre-retreat (32%; 3.00; Table 2).
Post-retreat survey
Those responding that they had the time to be a great teacher fell from 39% to 24%, 2.59, following the retreat. One participant commented, “Time limitations are the biggest deterrents to teaching.” Similarly, those answering positively that they had the necessary tools to teach decreased from 32% by more than half post-retreat (16%; 2.58; Table 2).
Institutional support for faculty teaching endeavors
Pre-retreat survey
Few agreed that “I have the incentives to be a great teacher” (33%; 2.97), with some discrepancy in response depending on age (< 10 years, 2.13 vs > 20 years, 3.73). However, a majority agreed that “if it did not negatively impact my income, I would like to teach more” (67%;3.88). This was especially true for those working < 10 years (4.88) versus those working > 20 years (3.40).
Definition of what constitutes teaching achievement
Pre-retreat survey
Although a large majority of faculty stated pre-retreat that they “Understand what makes a great teacher” (82%; 4.09), 4 out of 5 respondents agreed that there was no useful process in place to determine what they did and did not do well and whether they were great teachers. When asked if “the current process to evaluate teaching is useful,” the minority felt it was (21%; 2.76). Similarly, 28% of the faculty did not have a clear understanding of what the institutional expectations were when it came to teaching and mentoring (Table 3).
Post-retreat survey
Faculty responses to the question “I know where to get help to improve my teaching” improved only slightly post-retreat (53%; 3.37, compared with pre-retreat 41%; 3.26; Table 3). Several comments included “the notion of developing a formal curriculum in order to enhance teaching skills is an excellent idea” and that “developing metrics for educators and education” is important.
Identification of what constitutes excellence in teaching
Teaching quality
The statement receiving the highest rating was “Teaching is a critical component of the mission of the division.” Data showed that 100% of the faculty strongly agreed or agreed, for an average of 4.65. Fellow responses were similar, 4.5. However, in responding to “Teaching is of excellent quality,” there was a significant discrepancy between fellow and faculty answers, with fellows perceiving the teaching as being of higher quality (100%; 4.5) than the faculty (73%; 3.82; p = 0.004).
Good teachers
There was a discrepancy when the fellows and faculty were both asked to name the best teachers in the division. Many faculty members who were named as very good teachers by the fellows were not mentioned by the faculty.
Faculty:
There was high agreement among the faculty as to who the top 3 teachers in the department were; however, on average, those named tended to be more senior with more visibility and greater time spent in direct teaching activities of the department. When faculty respondents were asked, “How do you know they are good teachers?” 31/36 (86%) cited having personally learned from those individuals and directly experienced the faculty member’s interest in, excitement about, and commitment to education, as well as their “reputation as an excellent teacher.”
Fellows:
Fellows more often cited direct learning from and observation of those individuals as evidence of teaching excellence. Faculty members who were noted to be accessible, approachable, and practical were most often cited as examples of teaching excellence. Teachers rated most highly for their abilities as teachers received the highest ratings for their mentorship abilities as well. The correlation was high between the 2, with R = 0.983.
Academy concept
The majority of retreat participants (94.4%) agreed that establishing an academy would improve the quality of education and elevate the reputation of the department. Many (72.2%) thought that it would add an important academic track for clinicians. Data showed that 50% agreed with the statement “I want to become a Clinician Scholar Educator and the Academy will propel me in that direction.” Comments for the value of an academy included “the Academy can transform HSS into a world leader in education,” and noted the “collaborative spirit” and that people felt “stimulated about the impact.” Comments included enthusiasm that the academy would “prioritize teaching goals” for the department, and “provide appropriate recognition for the work that goes into teaching.” Others identified the potential positive changes to the curriculum, citing the academy’s ability to “support innovation” and “incorporate new technology” into the curriculum. Some noted that the academy would establish valuable faculty development goals by creating a much needed structure for faculty to document their contributions to education, and by helping to “develop metrics for educators” and “enhance faculty members’ skills as teachers.”
The recognition that resources may need to be greater to support educators was one goal for the academy to tackle, with several stating that the “department needs to provide support and academic recognition for teaching” and noting “the paradox of those who want to teach but are not adequately compensated to do so” continues to exist. The overall value was felt to be greatest “if most of our members could join even if their number of teaching hours wasn’t as high as some.” Overall, however, there was the sense of the positive influence of the retreat and the value of the goals of the academy, with comments including “I was truly stimulated about the impact of an Academy” and “I sense that everyone is excited by education and committed to making things better.”
DISCUSSION
Our education retreat helped to demonstrate clearly that the faculty of the HSS Rheumatology Division sees teaching as central to the mission of the division. This is not surprising because outstanding teaching is one of the defining characteristics of the division. However, our data show that while faculty enthusiasm for teaching can be high in an institution such as ours, there may be no consensus to define what constitutes excellence in teaching or what the expectations are. In addition, the lack of a formal feedback process and commitment to ongoing faculty education may well hinder this process. Recognizing the commitment of faculty to teaching, the achievement of those who pursue research and academic careers as Clinician Scholar Educators, and supporting these endeavors financially, could further enhance an institution’s excellence in the area of teaching. The academy is one such approach to providing a formal structure that seeks to organize these goals together under one roof.
Data gathered pre-retreat showed that the lack of a formal mechanism for obtaining accurate feedback about one’s teaching prowess may lead to overestimation of one’s skills. The majority of faculty indicated pre-retreat a significant ongoing commitment to education on a regular basis and that existing confidence in their individual teaching abilities overall was very high. The majority was highly satisfied with their teaching abilities and more than half the faculty felt that they were already “excellent” at teaching, although most had little or no formalized teaching education and minimal feedback to support this assertion. While after the retreat there was even higher enthusiasm and interest for teaching, this was accompanied by a slightly lower degree of confidence in performance, overall satisfaction with one’s performance as a teacher, and confidence in one’s degree of teaching excellence. Significant declines post-retreat most likely reflected a greater understanding on the part of the faculty about the actual teaching tools needed to succeed and proficiency required to be excellent. This suggests that new insights gained in the setting of the retreat led some, now armed with a new educational ideal, to question their abilities. It is important to note, as we compared pre- and post-responses, that the sample that responded to the post-retreat survey (n = 19) could be significantly different in terms of commitment to and interest in education than the sample that responded to the pre-retreat survey (n = 34). The 19 individuals who attended the retreat and took the time to respond to the post-retreat survey may represent a bias toward those who are more likely to have a positive view of their own skills and a higher degree of enthusiasm for education.
Without a formal mechanism in place, faculty members were evaluating their educational abilities in a vacuum, since most agreed that there was no useful process in place to help them understand what they do well and do poorly, and how to become great teachers. Similarly, they did not have a clear understanding of what the institutional expectations were when it came to teaching and mentoring even though the majority had been doing regular teaching on the clinic service and on the inpatient rheumatology consult service for 2 months every year. Not only did faculty members have no way of judging their teaching abilities but responses demonstrated that very few faculty members even knew where to get help to improve their teaching, emphasizing the lack of an existing structure for going about this formally.
Not surprisingly, there was a striking lack of understanding about the definition of and requirements for being promoted on an academic educator’s track. A larger number than expected, 42%, agreed or agreed strongly pre-retreat that they wanted to move along a Clinician Educator track. This was surprising, given that many of those faculty members responding as such had not taken on substantial divisional teaching commitments, gained experience in teaching methodology or feedback skills through coursework, been involved in curricular design or revision, or published any type of education research. Most were not on the academic educator’s track although some had already been promoted on other tracks. Responses may have been biased toward the positive due to the enthusiasm generated by the retreat and the implicit departmental support for the new track since this was receiving new attention. In all likelihood these responses were due in part to lack of a rigorous institutional definition and delineation of the various academic tracks, and to misunderstanding what a Clinician Educator actually does and what types of scholarship are required to be promoted.
Clearly, the current system was noted to be deficient in protecting faculty time and supporting them appropriately for time spent in teaching activities. Despite an enthusiasm for teaching, in the pre-retreat survey, the majority of faculty did not feel that they actually had time or incentives to be a good teacher. Younger faculty in particular felt there was less time to teach, perhaps because they already had taken on a greater burden of those responsibilities than older members of the division because of the lack of compensation attached to taking on these responsibilities. As in many institutions, although the expectation of a certain amount of time spent teaching was implicit, it was not explicitly defined, or more substantial commitments rewarded appropriately. The significant decline in respondents stating they would like to teach more post-retreat most likely reflected a greater understanding on the part of the faculty about the actual burden on time and resources needed to support teachers’ time after minimal exposure to these issues at the retreat. Regarding institutional support for faculty teaching endeavors, few agreed that incentives existed to be a great teacher, with some discrepancy again in response depending on age. If money were not factored into the equation, however, a majority agreed that they would want to teach more. This suggests that if an institution can structure greater resources to teachers, in the form of both educational programs and financial support, it is likely that the teaching quality would improve and the numbers of those willing and able to teach among the available faculty would increase.
The discrepancy between faculty and fellows as to what constitutes a good teacher was highlighted by the differences between the 2 groups when naming the best teachers in the division, suggesting an important difference in the vantage point and perspective of the learner. Those named by the faculty tended to be more senior, with greater cumulative knowledge, experience, and expertise. Fellows more often cited those faculty members who were accessible, approachable, and practical as examples of teaching excellence. This underscores the unique needs, expectations, and capabilities of learners at different stages in their careers, and highlights the needs to better assess and define those needs and create appropriate learning environments for each level of learner. The structure of the academy aims to do just that.
As a result of the retreat process, our survey was able to identify 4 main issues that we believe are essential for institutions to address to enhance academic education: (1) enthusiasm and interest for teaching is high, but confidence in performance is lower because of a lack of education resources and defined metrics; (2) lack of feedback about one’s abilities and lack of knowledge about where to get help to improve hamper the attainment of greater teaching achievement; (3) consensus about achievement of education scholarship and pathways that promote the Clinician Scholar Educator are lacking; and (4) the lack of protected time and incentives for teaching prevent the participation of willing and able faculty in important educational activities. This suggests that if an institution can establish a structured, ongoing plan to devote greater resources to teachers, in the form of both educational programs and financial support, their interest, ability, and time set aside to teach would increase.
To address some of these concerns, the new HSS Academy of Rheumatology Medical Educators will have 2 goals: (1) the support of career development for members of the division who commit their academic activities to the advancement of medical education, education research, and curriculum development; and (2) a general elevation of the teaching skills of all faculty members. Identification of strengths and weaknesses of the current educational programs at all levels of training needs to be accomplished to pave the way for instituting improvements. The academy will then serve as a foundation on which we can build a well-defined program of educational goals for each level of learner and teacher. A feedback process will enhance each teacher’s individual progress and will consist of defined metrics that will be used to measure attainment of specific skills. Reevaluation of career-specific promotion pathways, the institutional compensation infrastructure, and incentives to encourage excellence in teaching will be explored. The result will be a stimulating academic environment for educators akin to those already available to clinicians and investigators that enhances the quality of teaching, promotes teaching excellence, and elevates the status of medical educators in the division.
Acknowledgment
The authors are grateful to Michael Battistone, MD, Molly Cooke, MD, Marc L. Dickstein, MD, and Kenneth M. Ludmerer, MD, for helping to lay the groundwork for the creation of the HSS Academy and Peggy Crow, MD, for her leadership and vision.
APPENDIX A. Division of Rheumatology Faculty Teaching Survey
-
You have been in the Division of Rheumatology at HSS (check one):
-
Less than 10 years
-
Between 10 and 20 years
-
More than 20 years
-
-
You are:
-
Female
-
Male
-
-
How much time you spend on teaching activities on average per week (check one):
-
Less than 2 hrs per week
-
2 hrs per week
-
4 hrs per week
-
One day per week
-
More than one day per week
-
-
Which of the following teaching activities do you participate in yearly? (check all that apply):
-
Rheumatology Consult service
-
Rheumatology HSS clinic and in-service
-
HSS resident rotation lecturer
-
NYP internal medicine attending
-
Weill/Cornell first year course
-
Weill Cornell Physical diagnosis course
-
Weill/Cornell Mechanisms of Disease Course
Other (please specify):
-
-
Which of the following teaching activities do you want to continue doing? (check all that apply)
-
Rheumatology Consult service
-
Rheumatology HSS clinic and in-service
-
HSS resident rotation lecturer
-
NYP internal medicine attending
-
Weill/Cornell first year course
-
Weill Cornell Physical diagnosis course
-
Weill/Cornell Mechanisms of Disease Course
Other (please specify):
-
-
I feel I contribute to the teaching mission of the Division.
-
I have the:
-
Interest to be a great teacher
-
Commitment to be a great teacher
-
Understanding of what makes a great teacher
-
Understanding of the latest pedagogical techniques
-
The time to be a great teacher
-
The incentives to be a great teacher
-
-
I am very satisfied with my performance as a teacher.
-
I know what is expected of me as a teacher and mentor.
-
The Division is supportive of my teaching activities.
-
All my colleagues put in a fair and equitable amount of effort and work vis-a-vis teaching.
-
I am an excellent teacher.
-
The teaching provided by the Division is of excellent quality.
-
I would like to teach more.
-
The Division supports teaching at an appropriate level in relation to clinical care and research.
-
The current process to evaluate and provide me with feedback on my teaching is useful.
-
I have the all the tools and resources I need to teach well.
-
If it did not negatively impact my income, I would like to teach more.
-
Teaching is a critical component of the mission of the Division.
-
I think that every member of the Division should be involved in teaching.
-
I would like to become a part of an Education Academy and move along a Clinician-Educator track.
-
I know where to go to get help to improve my teaching.
-
I really like teaching.
-
The concept of an Education Academy appeals to me because? (check all that apply)
-
It will elevate the reputation of the Division.
-
It will add an important academic track.
-
It will improve the quality of education.
-
I want to become a clinician-scholar-educator and this will propel me in that direction.
Other:
-
-
Please list the top three requirements to be a successful teacher:
-
Please list the obstacles that are in the way to make you a better teacher:
-
Other than you, who are the 3 best teachers in the division?
-
How do you know they are good teachers?
-
Why are they good teachers?
-
-
What do you see as your strengths as a teacher?
-
What do you see as your weaknesses as a teacher?
-
What do you think will be helpful in order to improve on those areas in which you are weakest?
What else do you really want us to know?
APPENDIX B. Division of Rheumatology Fellows’ Teaching Survey
As you may know, the Division of Rheumatology will have a retreat to discuss a wide range of academic issues including teaching in the Fellowship Program. In preparation for that retreat, we would appreciate your input on the quality of teaching in the Program and are asking you to please complete the survey below.
When assessing “ability to teach” please take into consideration the following criteria: The Faculty member gives appropriate and timely feedback; communicates clearly; exhibits the highest standards of professionalism; explains clinical decisions thoughtfully; is available for questions, and to see patients; is approachable; and is prepared, arrives on time, and stays throughout each teaching experience.
Scale: 1: Poor; 2: Fair (lower than average); 3: Average; 4: Good (above average); 5 Excellent; n/o: not able to observe.
-
List the top three requirements to be a successful teacher:
-
List the three obstacles that are in the way to making your learning experience better:
-
Please list three improvements that you would like us to implement in the Program:
-
Overall, the teaching provided by the Division is of excellent quality. (Scale 1 to 5; strongly disagree to strongly agree)
-
In my opinion, teaching is a critical component of the mission of the Division. (Scale 1 to 5; strongly disagree to strongly agree)
-
In my opinion, every member of the Division should be involved in teaching. (Scale 1 to 5; strongly disagree to strongly agree)
What else do you really want us to know?
- Accepted for publication February 29, 2012.