Abstract
Objective. To evaluate the effectiveness of an online module in the development of medical students’ clinical hand examination skills.
Methods. We developed a Web-based module to teach examination of the hand to first-year medical students (n = 99) to address the core skills expected in undergraduate medical training in Canada. The module was compared to the standard recommended text and tutor-led teaching using a validated objective structured clinical examination (OSCE) and a written knowledge test.
Results. A total of 17 students completed the OSCE from the book-based learning group, 18 from the tutor-led group, and 26 from the online module group. The average total OSCE score was significantly higher for students in the online module group compared to the textbook group (73.2% and 60.5%, respectively; p = 0.003). There was no significant difference between students in the online module and tutor-led groups (73.2% and 69.0%, respectively; p = 0.31). The online module group had a significantly higher mean total knowledge score than the textbook group (8.4 and 5.7, respectively; p < 0.001; maximum score 10) and the tutor-led group (8.4 and 7.4, respectively; p = 0.04).
Conclusion. Our study provides evidence that a well designed Web-based module, supported by sound educational theory, is an effective tool in the teaching of musculoskeletal examination skills, and provides some advantages over tutor-led teaching in terms of knowledge retention.
- RHEUMATOLOGY
- MEDICAL EDUCATION
- COMPUTERS
- PHYSICAL EXAMINATION
The teaching of musculoskeletal examination is challenging for a number of reasons. Students find it overwhelming and difficult to develop an approach that is consistent, clinically relevant, and succinct, resulting in deficiencies in musculoskeletal examination skills at both under- and postgraduate level1,2. This difficulty may be compounded by the lack of a standardized approach presented by different specialists and the level of detail in textbooks and manuals. Additional challenges include financial constraints at institutions, reduced tutor availability, and increasing pressure on the medical curriculum3. Focused feedback from groups of medical students at this institution has confirmed these perceived deficiencies. For example, students consistently noted an incongruity among the content of the textbook, the undergraduate course curriculum, and the Medical Council of Canada (MCC) Objectives4. We have chosen a Web-based solution as a means of circumventing these problems with traditional educational media.
Computer assisted learning (CAL) is increasingly being used in medical education, but research justifying its use remains limited5. Many CAL packages have focused on knowledge acquisition6,7. CAL has also been seen to be effective in teaching some psychomotor skills; Xeroulis, et al showed that computer based video instruction is an effective instructional tool in teaching suture and knot-tying skills to medical students8. CAL can be used for teaching examination skills such as cardiac auscultation9, skin examination10, and mobility assessment11. In terms of musculoskeletal clinical skills, Vivekananda-Schmidt, et al3 showed that an educational CD teaching rheumatology examination skills had a positive measurable educational effect on acquisition of examination skills. However, the role of Web-based modules for teaching musculoskeletal clinical examination needs formal research before innovative teaching methods supported by educational theory are more widely accepted.
Our aim was to evaluate the role of an online module on clinical examination of the hands in teaching students to develop examination skills as compared to tutor-led clinical skills teaching and the use of a textbook on physical examination.
MATERIALS AND METHODS
We developed a Web-based module to teach examination of the hand in non-emergency rheumatological conditions to help students develop their approach. The Web-based module and tutor-led teaching sessions aimed to address the core skills that are expected of undergraduate medical training in Canada defined in the MCC Objectives4 and also the Regional Examination of the Musculoskeletal System (REMS) core set developed by Coady, et al12. A key factor in the design of the Web-based learning tool was consistency between the module, the curriculum, and the MCC objectives.
The module taught the sequence of the examination, with photographs demonstrating a range of possible conditions that could be seen on inspection, photographs showing clearly how to palpate and move the joints, discussion of pattern recognition in osteoarthritis and rheumatoid arthritis, and assessment of nerve injury. In the module, objectives were clearly defined and put in context by describing typical clinical scenarios. The learning sequence was made clear by the intuitive nature of the Web-based design. The student had the opportunity for reinforcing and repeating the learning, and needed only to move to the next step when ready. A checklist at the end of the module reinforced the key learning points and sequence.
The effectiveness of the module was assessed using a validated OSCE13 with the permission of the authors, and a brief written knowledge test (see Appendix).We ensured that all parts of the OSCE examination and knowledge test were clearly covered in the module, lecture, and the textbook. We were careful to ensure that the content of the module and tutor-led sessions was consistent, and to facilitate this we preceded the practical part of the tutor-led teaching by a brief PowerPoint introduction.
The learners were all first-year medical students at Queen’s University, Kingston, who had never been exposed to formal training in hand examination, but who had attended a lecture in the first term on the GALS (gait, arms, legs, and spine) musculoskeletal screen14 and were thus familiar with the concept of inspection, palpation, and movement of joints. Since we were targeting a specific group this was a sample of convenience. We did not formally ascertain if previous teaching of hand examination had occurred in training outside the medical program.
The objectives of the hand teaching were: (1) to develop a standard approach to examination of the hands and wrists in any patient presenting with rheumatic disease; (2) to recognize a range of clinical signs and understand the relevance of these in the examination of the hands and wrists; (3) to recognize and describe the classical patterns of joint involvement in rheumatoid arthritis and osteoarthritis; and (4) to be aware that other (i.e., nonrheumatological) diseases can result in clinical signs in the hands and to be able to describe some of these.
The students were randomized using a computer-based random number generator into 3 groups: Group 1 – Students attended a practical session led by a rheumatologist taking them through the examination of the hand in a patient with polyarthritis. The demonstration included discussion of possible findings at each stage with an initial PowerPoint presentation showing photographs identical to those described in the online module. During the practical component the sequence of the examination was emphasized, and reinforced by the students practicing on each other with the tutor watching and ensuring correct clinical method. The tutor was able to ask questions to the students to ensure understanding and reinforce learning points and to offer immediate and relevant feedback. Group 2 – Students were invited to learn about the examination of the hand using the online module. They did not have practical teaching. Access to the Web-based module was limited to that group by user-name and password to avoid contamination of the other 2 groups. Group 3 – Students were asked to read the recommended text on hand examination15. The book contained all the information necessary but not in the sequence of learning as presented in the module. The students were asked to learn how to undertake clinical examination of the hand in a patient with joint pain.
We did not limit the time for Groups 2 and 3 to read about the skills or use the Website, as we felt this should mirror the current practice of students and the aims of the module. Practice of the clinical examination techniques was encouraged within the groups. No additional relevant teaching or lectures were scheduled on their timetable before the assessment.
Seven days after the material was available we assessed the students with the hand OSCE. The examiners were blinded to the intervention. A brief written evaluation of the teaching method and a knowledge-based test took place immediately after the OSCE.A mean difference in OSCE scores of 10% was assumed to be of educational significance16. As part of the ethics submission we agreed to offer remedial training to students if it was clear that they had been at a disadvantage in terms of their group allocation.
The following 2 primary hypotheses were prespecified: (1) Null hypothesis: The average total OSCE score is the same between students randomized to the standard text method compared to the online module teaching method. Alternative hypothesis: the average is not the same in the 2 groups (2-sided). (2) Null hypothesis: The average total OSCE score is at least 10% higher with the tutor-led teaching technique than with the online module. Alternative hypothesis: The average total OSCE score is no more than 10% higher with the tutor-led teaching compared to the online module (1-sided).
Hypotheses 1 and 2 were tested separately by the independent t-test. Normality of the total OSCE scores was confirmed within group by box plots and the Kolmogorov-Smirnov test. The total OSCE score, which has a possible range from 0 to 28 points, was converted to a percentage by multiplying the raw score by 100/28.
The 5-point visual analog scale ratings of usefulness, ease, and understanding were compared between groups by a 2-sided exact Cochrane-Armitage trend test.
The average total knowledge score was compared between groups by a 2-sided independent t-test. Time spent, which was positively skewed, was compared between the book and the online module groups by a 2-sided Wilcoxon-Mann-Whitney test. All analysis was performed with SAS Version 9.1 (SAS Institute Inc., Cary, NC, USA).
RESULTS
A total of 99 students were invited to participate in the study and were randomized into 3 groups of 33. The study was entirely voluntary and many students were anxious about attending an OSCE. The statistical analysis is based on a total of 17 students who completed the OSCE from the book-based learning group, 18 from the formal teaching group, and 26 from the online module group. The age and sex distribution in the 3 groups is shown in Table 1.
Primary outcome (total OSCE score).
The distribution of the total OSCE score appears roughly Gaussian in each group (Figure 1), and in each group the Kolmogorov-Smirnov test did not reject the null hypothesis that the data came from a normal distribution (all 3 groups p > 0.15). Table 2 presents the means of the total OSCE scores with 95% confidence intervals for each group. The average total OSCE score was 12.7% (95% CI 4.7% to 20.7%; p = 0.003) higher for students randomized to the online module compared to those randomized to the book method, and 4.2% (95% CI –4.0% to 12.3%; p = 0.31) higher than those randomized to the tutor-led teaching technique. This nonsignificant advantage of the online module compared to the tutor-led teaching technique rules out the possibility that the online module is at least 10% worse than the tutor-led teaching technique at p = 0.0006.
Secondary outcomes.
The total knowledge score was also highest in the online training group and lowest in the book group (Table 3). The online module group had a significantly higher total knowledge score than the book group [mean difference 2.7 (95% CI 1.5 to 3.9); p < 0.001] and a slightly higher score than the formal training group [mean difference 1.0 (95% CI 0.05 to 1.9); p = 0.04].
Compared to reading a text, the students agreed more strongly that only the module was useful (p = 0.004), was easy to use (p = 0.013), and helped understand the sequence (p = 0.046). The students found the online module of similar usefulness to formal instruction (p = 1.0), and not significantly different for understanding the sequence (p = 0.22).
The median time spent was 60 min for the online module compared to only 30 min for the book (p = 0.033). Students felt the online module was more useful and easy to use, and allowed them to understand the sequence better than the text method, and they found it of similar usefulness to the tutor-led teaching technique.
DISCUSSION
Our study suggests that in terms of the total OSCE score, the online module is superior to reading the recommended text and comparable to practical clinical skills teaching led by a clinician, and for the knowledge test the module was superior to both other teaching methods. These results are reassuring for clinical skills course organizers.
Intuitively, one would expect a Web-based module to have advantages over a textbook. With Web-based modules the teacher still selects and organizes the instructional method, defining objectives as learning outcomes. Such instructional objectives are less clear in a book chapter, which may not define the terminal skill as clearly. Another disadvantage of many textbooks, including the one chosen by this medical school, is lack of clarity regarding the sequence of the hand examination, and a paucity of photographs clearly showing the methods of palpation. The organization of the book’s content was less intuitive, and the students were required to filter out what might be important. Both CAL programs and textbooks can be used at times that suit the learner. A Web-based module or video cannot aim to replace a tutor’s ability to demonstrate and reinforce correct technique, but if it is well designed, we have shown that students can learn the psychomotor skills and still perform at a high level. We do not anticipate that such resources would fully replace formal tutor-led clinical skill teaching, but the results of our study do show that this is a possible goal. Further studies would be important to investigate how students would respond to substitution of faculty-based skills teaching by CAL. More realistically, we would suggest that use of such Web-based resources prior to formal clinical skills classes might improve the efficiency of teaching by ensuring students arrived already prepared. In this way, CAL has the potential to decrease the amount of time needed by both students and faculty to master the acquisition of clinical skills17.
Web-based resources thus offer a method of teaching clinical examination skills, and potential savings in terms of resources are very significant, particularly in terms of physical space and tutor time. With the Virtual Rheumatology CD the cost of this intervention ($22,045) predominantly arose from the cost of producing the CDs3. Web-based resources are considerably less expensive; for example this module was prepared on Exe, a freely available authoring tool. CAL is also used for teaching other aspects of rheumatology; Wilson, et al developed an interactive Web-based tool teaching rheumatology through clinical case studies18. This tool was both effective and well received by medical students and thus had a clear role in effective use of medical school resources.
Novice learners benefit most when information is organized, provided in a logical and clinically relevant sequence, and readily available. When information is organized into specific and manageable portions, the cognitive load in working memory is reduced, allowing the material to be recalled more easily19. Providing a logical sequence that models clinical decision-making promotes schema development, longterm recall, and the ability to transfer these skills into clinical situations. For example, Owen and Plummer20 broke down the procedure of endotracheal intubation into meaningful and discrete elements for students and trainees. They were careful not to overwhelm them with details in the early stages, allowing instead for the students to progress through a sequence of subtasks towards mastery. After a single session, the majority (93%) reached the required standard to attempt intubation on a patient.
When learners can access material when it is convenient or most relevant, they are able to advance their learning at a point where they are most receptive to the information. This “just-in-time” learning21 can be facilitated by Web-based instruction that gives students flexibility and choice in terms of where and when they learn.
We found that the OSCE design was useful in informing the lesson planning and sequencing, and thus attention to the assessment tool can be used in an iterative manner in revising and developing the instructional sequence.
A potential confounder in this study is the Hawthorne effect, i.e., the unavoidable fact that the knowledge of the planned OSCE would have acted as a strong motivator for learning. Other confounders to this study include the recall bias inevitable in the textbook and Web-module groups, who could potentially revise up to the day before the OSCE, although we did allow students in the practical group to take notes (reflecting their normal behavior in a practical skills tutorial). Whether an OSCE is an effective way of measuring performance is beyond the scope of this article. We did not prospectively record the time students spent using the module and book or practicing within groups. We are also aware that a knowledge test that has not been formally validated has some limitations as an assessment tool.
Students have different learning styles and perhaps modules such as this will not appeal to all students. Media such as videos seem to “age” quickly; even if the content is sound, changing clothing fashions, for example, can appear to make the content seem out of date. This is less likely with a Web-based module, although it would still need to be maintained in order to relate clearly to the curriculum and to have relevant educational objectives.
Our study provides evidence that a well designed Web-based module, supported by sound educational theory, is an effective tool in the teaching of musculoskeletal examination skills. CAL may also provide some advantages over tutor-led teaching in terms of knowledge retention.
APPENDIX
(For question 2 we accepted any answer that could be found in the module, textbook, or lecture, or was considered reasonable.)
Knowledge Test (maximum score 10)
-
What is the correct sequence for examination? ( /1)
Inspect – Palpate – Active movement – Passive movement – Function
Inspect – Palpate – Passive movement – Active movement – Function
Inspect - Function - Active movement – Passive movement – Palpate
-
Name four clinical signs to look for in hand examination ( /4)
-
What condition is Phalen’s test used to evaluate ( /1)
-
Name two signs to look for at the elbow in a patient with an arthropathy in the hand ( /2)
-
Heberden’s nodes occur in what condition? ( /1)
-
Boggy swelling across the MCP joints of each hand suggests what condition? ( /1)
Footnotes
- Accepted for publication October 6, 2008.
REFERENCES
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13.
- 14.
- 15.
- 16.
- 17.
- 18.
- 19.
- 20.
- 21.