Abstract
Objective. Rheumatoid arthritis (RA) is a common debilitating autoimmune disease, with unmet need for knowledge among patients and the general population. YouTube is a popular, consumer-generated, video-sharing website, which can be a source of information on RA. We investigated the quality of information on RA on YouTube and analyzed audience interaction.
Methods. YouTube was searched using the term “Rheumatoid Arthritis,” for videos uploaded on RA. Two physicians independently classified videos as useful, misleading, or patient views, and rated them on a 5-point global quality scale (GQS; 1 = poor quality, 5 = excellent quality). Useful videos were rated for reliability and content, on a 5-point scale (higher scores represent more reliable and comprehensive videos). Source of videos was also noted. Audience interaction was assessed through video viewership.
Results. A total of 102 relevant videos were identified; 54.9% were classified as useful (GQS 2.9 ± 1.0) and 30.4% deemed misleading (GQS 1.3 ± 1.6). Mean reliability and content score of useful videos was 3.2 (± 1.0) and 2.5 (± 1.2), respectively. All videos uploaded by university channels and professional organizations provided useful information but formed only 12.7% of total videos, whereas 73.9% of medical advertisements and videos by for-profit organizations were misleading. There was no difference in the viewership/day (10.0 vs 21.5; p = nonsignificant) of useful and misleading information.
Conclusion. YouTube is a source of information on RA, of variable quality, with wide viewership and potential to influence patients’ knowledge and behavior. Physicians and professional organizations should be aware of and embrace this evolving technology to raise awareness about RA, and empower patients to discriminate useful from misleading information.
Rheumatoid arthritis (RA) is a chronic multisystem autoimmune disorder, characterized by systemic inflammation and progressive joint destruction associated with severe morbidity and increased mortality. Its prevalence is 0.5%–1% of adults in developed countries, with a significant majority affected in the productive years of life1. Studies have demonstrated a considerable need for information among patients with RA2,3. Patient empowerment has been associated with better satisfaction, compliance with treatment, and improved health outcomes in RA4.
The Internet has increasingly become an important source of healthcare information, with 60%–80% of Americans having used the Internet to find health information in 2008–2010, as compared to 25% in 20005. One of the key developments leading to this rapid popularity is the rise of social media that allow individuals to generate and share content6. YouTube is one such readily accessible, easy-to-use, video-sharing Website, which since its inception in 2005 has grown into the third most popular Website in the world, with > 2 million page views per day7. It enables different modes of information presentation, moving beyond the usual text-based information to enhance lay understanding of health, making YouTube a potentially lucrative source of disseminating and sharing information on healthcare. However, given the limited evaluation of the quality of this consumer-generated information, there is risk of spreading misleading information8. YouTube has been evaluated recently as a source of information on vaccination, prostate cancer, H1N1 pandemic, kidney stones, and cardiopulmonary resuscitation9,10,11,12,13.
With patients increasingly embracing these new technologies for obtaining health information and not necessarily disclosing it to their physicians14, we as physicians should be aware of the content and quality of this information, to be able to guide patients appropriately. We conducted this descriptive study to characterize the content and quality of information on RA on YouTube; and to analyze audience response and interaction with the videos, a unique opportunity offered by this mass media tool.
MATERIALS AND METHODS
YouTube (www.youtube.com) was searched using the keyword “rheumatoid arthritis” on April 8, 2011, for videos containing pertinent information on epidemiology, risk factors, symptoms, diagnosis, treatment, and other information regarding RA. The search yielded 3350 videos in total. The first 200 videos (first 10 pages), ranked by relevance (default option on YouTube, which uses a complex algorithm based on view count, upload date, rating, comments, bookmarks, age of user, etc.), were screened for information on RA. Studies on Internet search engines have shown > 90% of search engine users click on a result within the first 3 pages of search results15. Only unique videos in the English language were included; videos duplicated in part or whole were excluded; videos in multiple parts were counted as 1 video with the average view count used for analysis. All videos were viewed and analyzed for content by 2 independent physicians (AGS, PS); in the event of a discrepancy, a third reviewer (SS) arbitrated the disagreement.
All selected videos were classified based on the primary theme as useful, misleading, or as personal experience-based11,12,13, as follows: (1) Useful — if the video contained scientifically correct and accurate information about any aspect of the disease; (2) Misleading — if the video contained scientifically unproven or inaccurate information based on currently available scientific evidence (e.g., unsubstantiated claims about pathogenesis and treatment with unproven dietary, herbal, or alternative therapy, or negative portrayal of evidence-based treatment); and (3) Patient views — if the video describes a patient’s personal experience while having/being treated for RA.
All videos were also categorized according to source into 5 groups: independent users; government/news agencies (e.g., Global Medical News Network, National Institute of Health); university channels/professional organizations (e.g., Mayo Clinic, Arthritis Foundation); health information Websites (e.g., AnswersTV.com); or medical advertisements/for-profit companies (e.g., Immunogizerfatreducer.com, Dr. McDougall’s Health and Medical Center). Other attributes including length of video and time since upload were noted. Audience interaction with the video was assessed by video popularity (defined as views per day for a particular video, calculated as total views for video divided by number of days on YouTube) and video viewer “likability” (number of “likes” for a video).
All videos rated as useful were further analyzed for reliability and completeness of information, based on a 5-point scale. Reliability of information was scored from 1 to 5 (reliability score), based on 5 questions (adapted from the DISCERN tool for assessment of written health information), as shown in Table 116. Comprehensiveness of information was also scored from 1 to 5 (content score), based on different aspects of disease information covered in the video (epidemiology/risk factors, pathogenesis, clinical features, additional diagnostic tests, treatment; 1 point was given for each aspect covered on the video, to give the lowest possible score of 1, and a maximum score of 5). Misleading videos were analyzed regarding the key aspects in which they were misleading. Patient personal experiences were analyzed to assess if they were positive (either providing emotional support to the audience or providing useful information on RA and its treatment) or negative (portrayed evidence-based therapy negatively and appeared to promote alternative therapy with unproven scientific benefit). In addition, all videos were also rated using a global quality score (GQS), using a 5-point scale to rate the overall quality of the video, based on the quality of the information and how useful the reviewer thought the particular video would be to a patient (Table 2)17.
Assessment of reliability of useful videos on rheumatoid arthritis found on YouTube18.
Global quality scale (GQS) criteria used to score videos with information on rheumatoid arthritis on YouTube19.
Data entry and analysis were done using SPSS software (SPSS 19.0, IBM, Chicago, IL, USA). The degree of agreement between the 2 reviewers was assessed using the κ-coefficient. Descriptive statistics were calculated to describe the overall characteristics of the videos. Continuous variables were compared using independent sample t test and nominal variables were compared by chi-square test to assess differences between groups. Patient views were excluded from comparative analysis. A p value < 0.05 was considered significant.
RESULTS
Of the 200 videos screened, 102 unique relevant videos, with a total of 581,819 views, were identified for further analysis; 63 videos were deemed irrelevant (primary focus on osteoarthritis, juvenile rheumatoid arthritis, and other rheumatic diseases), 30 were duplicated in part or whole; 9 were split into 2 or 3 parts, and were counted as 4 unique videos. The mean video length was 5.9 (± 9.3) min and mean video popularity was 14.1 (± 33.8). The average duration on YouTube was 633 (± 419) days.
Fifty-six videos (54.9%) were classified as useful, 31 (30.4%) as misleading, and 15 (14.7%) as patient views. The κ-coefficient of agreement between users was 0.85. The classification of relevant videos along with their attributes is described in Table 3.
Characteristics of videos on YouTube on rheumatoid arthritis.
Useful videos
The mean reliability score of useful videos was 3.2 (± 1.0; κ-coefficient of agreement 0.63). Thirty-one videos (55.3%) were from reliable sources and 17 (30.3%) provided additional sources of information. The mean content score of useful videos was 2.5 (± 1.2; κ-coefficient of agreement 0.72), with 10 videos (17.9%) being comprehensive (score ≥ 4). Treatment of RA, including pharmacological and/or physical therapy, was discussed in 35 videos (62.5%). The quality of useful videos according to source of information is shown in Table 4.
Quality of useful videos (n = 56) on rheumatoid arthritis according to source of information.
Misleading videos
About one-third of videos were misleading. In 42% of these, the primary theme of misinformation was pathogenesis, with proposed mechanisms being “leaky gut”/food allergies/dysbiosis, heavy metal poisoning, and/or hormonal imbalances, and some refuted the autoimmune origin (for example, “how can God design the human body to turn against itself”). In 92% of videos, unscientific therapies were promoted, the majority promoting dietary alterations and/or herbal or naturopathic therapies, and some unique treatments including marijuana, acupuncture, chiropractic, and “energy healing,” as well as stem cell infusion. The common underlying feature of these videos was that information was presented as sensationalistic claims, invoked nature and spirituality, and was self-promoting and biased, from untrustworthy sources. In 19% of misleading videos, proven evidence-based treatments were discredited as “toxic” and even pathogenic. Misleading videos had a uniformly low GQS, and were felt to be of limited use for patients.
Patient views
Of the 15 videos classified as patient views, 13 (86.7%) were felt to be positive and 2 were viewed as negative. Both the video reviewers felt it was difficult to differentiate true patient views from patient testimonials in medical advertisements.
Sources of information
The leading sources of information were independent users (37 videos; 36.3%), medical advertisements/for-profit companies (23 videos; 22.5%), health information Websites (20 videos; 19.6%). Government/news agencies and university channels/professional organizations accounted for only 21.5% of videos, although they were consistently sources of useful information, and none of these videos had misleading information (Table 4). In comparison, 73.9% videos from medical advertisements/for-profit companies were misleading (p = 0.001). About 10% of videos from health information Websites were misleading.
Audience interaction with videos
There was no significant difference in the audience response to useful and to misleading information, with similar video popularity, likability, and number of comments on videos (useful vs misleading, 8.6 vs 7.2, respectively; p = 0.67; Table 3). Similarly, there was no significant difference in the popularity of videos with high GQS (4–5) and low GQS (1–2), with mean video popularity for high GQS vs low GQS of 16.1 vs 17.2, respectively (p = 0.93). The audience did not seek any specific source of information, with viewership distributed equally across all sources of information as follows: total viewership share and mean popularity (SD): independent users 30.4%, 16.9 (SD 43.6); university channels/professional organizations 13.4%, 7.4 (SD 10.2); government/news agencies 15.3%, 10.4 (SD 24.3); health information Websites 17%, 9.9 (SD 18.7); and medical advertisements/for-profit companies 24%, 18.5 (SD 38.4; p = 0.8).
DISCUSSION
Driven by a desire for a more active role in healthcare decision making, patients with chronic diseases are increasingly using the World Wide Web to learn more about their diseases and possible treatments5. YouTube is one such very popular, open-access, video-sharing Website hosting an increasing number of clips on diagnosing, treating, and preventing illness18,19.
Assessing the quality of information on RA on YouTube, we found that half the videos were useful for patients with RA. Nonprofit professional organizations and university channels were the best sources of information, but they formed only 12.7% of the total videos and 13.4% of proportion viewership. These findings are similar to quality of information on other diseases or procedures on YouTube. Only 48% of videos on YouTube portrayed immunization in a positive manner and had lower ratings and viewership than videos that depicted immunization negatively9. Only 58% of videos on kidney stones were deemed useful using criteria similar to ours, and most of these were uploaded from university channels/professional organizations, news agencies or independent users12. At the peak of the H1N1 epidemic, only 61% of videos had useful information about the disease11. In a study on Internet use by patients attending a rheumatology clinic, two-thirds sought online information about their disease. Reasons for going online were to gather general information (45%), to research treatment options including natural therapies (17%), to try to diagnose themselves (12%), and to identify people with similar experiences (9%)20. While information on RA on YouTube may not be of excellent quality, it can serve to increase awareness about the clinical features and treatment options, and augment the information provided by the physician.
We also found that about one-third of videos on RA were misleading, providing inaccurate and unfounded information. This was similar to information on other diseases analyzed on YouTube — 32% of videos on immunization, 23% on H1N1, and 18% on kidney stones were felt to be misleading9,11,12. Medical advertisements/for-profit companies were the leading sources of this biased information, with almost three-quarters of videos from these sources being misleading. Many used patient testimonials without clearly labeling their videos as advertisements, making it difficult for viewers to differentiate whether these opinions are truly original thoughts or whether they represent paid advertisements. Patients with rheumatic disorders are among the most frequent seekers of alternative therapy21. In reviewing published Websites as sources of information on RA, Suarez-Almazor, et al21b found that over half the Websites are maintained by for-profit companies, which advertise an alternative product claimed to be effective for many conditions. To date, no high-quality randomized controlled trial has demonstrated consistent benefit of any particular diet or herbal medicine and most apparently favorable trials have a moderate to high risk of bias22,23. This pitfall of open access, unregulated health information has been recognized in several other diseases and may potentially compromise health behaviors and health outcomes by delaying diagnosis and initiation of appropriate treatment, and result in unnecessary anxiety and at times inappropriate requests for clinical interventions8,20,24,25, or may in rare instances lead to preventable morbidity and mortality26.
While several studies have looked at the quality of healthcare information, most of these are unidimensional and have not sought the audience response and viewership. Our study was unique in that we were able to assess viewer interaction with this available information. It was both eye-opening and distressing to learn that the popularity as well as likability of the lowest physician-rated misleading videos was similar to the highest physician-rated useful videos, suggesting that it may be difficult for patients to judge the quality of information presented. Recent studies have shown that three-quarters of the people seeking health information on the Internet “never,” “hardly ever,” or only “sometimes” check the source of information5.
To improve usability of online healthcare information, physicians could emphasize that patients need to be selective when accessing medical information from the Web. Physicians could provide patients with basic guidelines for content evaluation, such as how to evaluate sponsorship and disclosures as well as factual information, based on the Medical Library Association’s User’s Guide to Finding and Evaluating Health Information on the Web27.
Our study was limited to analysis of only English-language videos on YouTube, in a single snapshot. YouTube content changes over time. Moreover, our study was limited to a direct YouTube search and did not account for YouTube videos viewed at other sites that embed or link videos; nor did we study videos on health information Websites other than YouTube. The videos were analyzed by physicians with evidence-based knowledge on RA, with inherent bias, in contrast to the general public, who are most likely going to view and learn from these videos. Seeking the public’s opinion on this subject would have been helpful.
The quality of information on RA on YouTube is variable. There is no difference in the viewership and popularity of useful and misleading videos. This should serve as an impetus to physicians and professional organizations to embrace this evolving technology to raise awareness about RA, and empower patients to recognize useful from misleading information.
Footnotes
-
A.G. Singh, MD; S. Singh, MD; P.P. Singh, MD.
- Accepted for publication December 7, 2011.
