Abstract
Background Patients often use the internet for information on their spinal surgeries. The goal of this study was to assess and compare the quality of lumbar fusion and arthroplasty videos on YouTube and to identify predictors of video quality.
Study Design Cross-sectional.
Methods YouTube was searched utilizing 3 search terms for both lumbar fusion and lumbar arthroplasty. Fifty videos from each search were categorized and analyzed. Videos were analyzed using 3 scoring systems: JAMA, informative, and clinical scores. The JAMA score rates online information based on 4 factors: authorship, attribution, disclosure, and currency. The informative score previously devised by Zhang et al was also applied to each video. Finally, 2 surgery-specific scores were created for lumbar fusion and lumbar arthroplasty based on peer-reviewed information. These were modeled on the informed consent procedure. Data analysis was conducted using the Jamovi 1.1.9.0.
Results Eighty-four unique lumbar fusion videos and 82 lumbar arthroplasty videos were analyzed. Educational videos were the most common in fusion (78%) and arthroplasty (47%) groups; however, arthroplasty videos were more likely to be commercial (17%, P = 0.01). Fusion videos were more viewed (P < 0.001); however, arthroplasty videos had higher positivity ratings (P < 0.01). Overall, quality was poor for videos in both categories. Mean JAMA scores were 1.57 and 1.70 for fusion and arthroplasty, respectively, and did not differ significantly (P = 0.32). Fusion videos had higher informative scores (1.57 vs 1.23, P = 0.02) and higher clinical scores (21.8% vs 15.9%, P = 0.06).
Conclusion Information on YouTube for lumbar fusion and arthroplasty is poor. However, information on fusion is better than arthroplasty. Metadata can be used to help patients pick higher quality videos.
Clinical Relevance This paper provides clinicians with an oversight of what their patients may accessing on the internet. Patients may have incorrect information regarding the surgical proceedure they are being offered. These misconceptions must be resovled in order to gain true informed consent from the patient and avoid damage to the surgeon-patient relationship.
Level of Evidence 3.
- YouTube
- quality
- lumbar
- fusion
- arthroplasty
- information
- internet
- web
- JAMA score
- consent
- online
- videos
- commercial
- educational
INTRODUCTION
Fifty-four percent of patients use the internet to access health-related information.1 This information is generally obtained via a search engine,2 of which Google is by far the most popular.3 The most common search suggestion is a link to YouTube.com,4 and YouTube itself is the second-highest trafficked website worldwide.3,5
Previous work by Baker et al has shown that 30% of patients attending spinal elective clinics have used the internet to research their condition,6 and Diaz et al found 60% of patients felt that the medical information from the internet was at least as good as information from their doctor, if not better.1 This raises concern as multiple studies have found the quality of information on the internet to be poor that can result in consult time spent correcting misinformation.7–11
For discogenic back pain, lumbar disc arthroplasty is proposed as a valid alternative to fusion in select patient groups with improved pain reduction and lower adjacent segment disease as promoted benefits.12,13
The aim of this study was to assess and compare the quality of videos on YouTube providing information on both lumbar disc arthroplasty and lumbar fusion. Given the lack of high-quality data in the literature for arthroplasty, we hypothesize that arthroplasty videos may be of comparatively lower quality.13,14 Second, by assessing different characteristics of the videos we also aim to find predictors of higher quality that could be recommended to improve search results for patients.
METHODS
Ethical board approval was not required for this study.
YouTube searches were made using Google Chrome (Windows 10 OS). Browser cookies were disabled to prevent search results from being tailored to any previous searches. All searches were performed on 20 April 2020. For each surgery, 3 search terms of varying complexity were used to simulate patients’ varying levels of medical knowledge and obtain a more representative sample of videos.15 The search terms used for arthroplasty were “low back disc replacement,” “lumbar disc replacement” and “lumbar disc arthroplasty.” The search terms used for fusion were “low back fusion,” “lumbar fusion,” and “lumbar arthrodesis.”15
For each search term, the top 50 videos were selected and reviewed. Duplicate videos were excluded. During the review process, a video was defined as irrelevant if its main purpose was not to describe any aspect of lumbar arthroplasty or fusion, and these videos were excluded.
Metadata was collected for each video including position in search results, length of the title, view count, subscriber count, duration in minutes, publication date, country of origin, and like/dislike counts. A positivity rating (percentage) was calculated from the like/dislike counts.
Videos were assigned to 1 of 4 categories: “educational,” “testimonial,” “commercial,” or “academic.” A video was defined as educational if its main purpose was to inform a patient about surgery through narrative description or surgeon interview, without testimonial elements. Testimonial videos had patients discussing their experience with the surgery. Commercial videos advertised a product or hospital, with this being the focus of the video rather than a secondary aspect. Academic videos were either presentations by experts in the field or video adjuncts to published academic papers.
It was also noted if a video contained a commercial element, even if the commercial nature of the video was more subtle. A commercial element was defined as any suggestion to access products or services from a named brand/ private hospital. For example, educational videos were often presented by a spinal surgeon, whose contact details were published at the end of the video.
The subcategory of surgery was also recorded. In the fusion videos, this included the approach for the procedure (eg, Anterior Lumbar Interbody Fusion (ALIF), Transformainal Lumbar Interbody Fusion (TLIF), etc); in the arthroplasty category, this included details of the implant (eg, ProDisc-L, M6-L, etc). A video was categorized as “non-specific” if it discussed fusion or arthroplasty generally, rather than a specific approach or implant.
Videos were scored using 3 systems: JAMA, informative, and clinical.
In 1997 Silberg et al proposed a set of minimum requirments to establish reliabity of information on the internet.16 This has subsequently been used as a measurement tool but numerous studies into the quality of online information.17–19 The 4 minimum requirements established were: authorship, attribution, disclosure, and currency. Each requirment was further defined by the authors16 Although Silberg et al did not formally name thier tool, it has often been referenced in subsequent literature as the “JAMA Score”, named after the journal in which it was published.17–19 A search on PubMed for "JAMA Score" AND ("online" OR "internet) yeilds 49 results in topics ragning from contraception to glaucoma to ankle fusion.17–19
An informative score analogous to that utilized by Zhang et al was employed.15 This score rates online information based on 6 factors: indication, outcome, complications, alternative treatment options, procedure description, and peer-reviewed literature. The calculation of this score is set out in Table 1.15
For this paper, we also devised procedure-specific “clinical” scores for each surgery. The model of informed consent was used as a basis for scoring, and peer-reviewed research was implemented in its establishment. This allowed scoring based on the accuracy of the information and allowed for scoring of specific important details. The score is outlined in Table 2.
Statistical Methods
Data were analyzed using Jamovi 1.1.9.0 (www.jamovi.com). Data are presented as mean (±SD) and rounded to 2 significant figures. The χ 2 test of independence was used to assess the significance of variations between categorical and nominal variables. The Mann-Whitney U test or the Student t test was used to compare the binomial categorical variables depending on the normality of the data. The Kruskal-Wallis χ 2 test (rank-sum) was used for analysis involving the clinical score because the data did not meet the assumption of normality required for parametric testing (Shapiro-Wilk, P < 0.01). Linear regression analysis was used to compare continuous variables.
RESULTS
The fusion searches yielded 84 unique videos, of which 54 (64%) were deemed relevant. The arthroplasty searches resulted in 82 unique videos, 47 (57%) of which were relevant. This difference was not statistically significant (P = 0.36). All videos were in the English language.
Figure 1 outlines geographic origin. The total number of videos produced outside the United States was too low to draw significant conclusions on regional differences.
Fusion videos were more frequently viewed (18 000 ± 350 000 vs 15 000 ± 31 000; P < 0.001) and had higher subscriber counts (150 000 ± 480 000 vs 9900 ± 27,000; P < 0.001) than arthroplasty. Arthroplasty videos had a higher positivity rating than fusion videos (99 ± 6% vs 94 ± 5%; P < 0.01).
Fusion videos had a higher informative score (1.6 ± 0.79 vs 1.2 ± 0.67; P = 0.02) and clinical score (22 ± 14% vs 16% vs 12%; P = 0.06) than arthroplasty videos. The difference in JAMA scores was not significant (P = 0.32). Details on the mean score and distribution for each score is outlined in Table 3.
The prevalence of each video category was significantly different between the 2 surgeries, with arthroplasty having a higher proportion of testimonial and commercial videos and fusion having a higher proportion of educational videos (P = 0.01, Figure 2).
When fusion videos were analyzed by category, commercial videos were found to have much lower clinical scores (3.8 ± 5.4%). This was followed in ascending order by academic (10 ± 4.4%), testimonial (18 ± 13%), and educational videos (24 ± 14%, P = 0.02, Figure 3). Academic videos had the highest JAMA score (2.8 ± 0.5, Figure 4). This was due to higher attribution, disclosure, and authorship. Commercial fusion videos had the lowest JAMA scores, with a mean score of <1. These differences were statistically significant (P = 0.004). The informative score did not differ significantly within fusion categories (P = 0.46, Figure 5).
Fusion videos with a commercial element had lower JAMA scores (1.4 ± 0.5 vs 1.8 ± 0.7; P = 0.06) and lower informative scores (1.5 ± 0.72 vs 1.6 ± 0.88; P = 0.06) but no difference in clinical score. (P = 0.72).
When arthroplasty videos were analyzed by category, commercial videos again had the lowest clinical scores (8.3 ± 11%); however as opposed to fusion, academic videos scored the highest in the clinical score (27 ± 11%, P = 0.04; Figure 3). Academic videos scored the highest in informative score (2.0 ± 2.0, P < 0.05; Figure 5). Academic videos also had a higher JAMA score (2.3 ± 1.2, P ≤ 0.001) scoring more highly for disclosure, attribution, and authorship. Testimonial and commercial videos had the lowest JAMA scores (1.2 ± 0.44 and 1.4 ± 0.52, respectively, P < 0.001; Figure 4). In arthroplasty, videos with a commercial element had lower JAMA scores (1.5 ± 0.5 vs 2.2 ± 0.58, P < 0.01).
Metadata was analyzed, searching for predictors of higher quality. Newer fusion videos had higher JAMA scores (P < 0.01) and fusion videos appearing sooner in search results had higher clinical scores (P < 0.01). Longer fusion and arthroplasty videos both had higher clinical scores (P < 0.01 and P < 0.001, respectively, Figures 6 and 7).
Otherwise, none of the metadata collected (video duration, title length, view count, positivity rating, subscribers, and position in search results) predicted video quality as defined by any of the 3 scores.
DISCUSSION
The aim of this study was to assess and compare the quality of information on YouTube for both lumbar fusion and arthroplasty. We found that the quality of information on YouTube regarding both procedures is poor, but the positivity rating was paradoxically high. This was confirmed by all 3 scoring systems that are in keeping with the wider literature assessing numerous other spinal conditions and procedures.7–11
The JAMA scores in both fusion and arthroplasty were similar to each other and to other spine-related topics on the internet. When assessing the wider literature, arthroplasty and fusion in our study scored higher than kyphosis and scoliosis information but lower than discectomy and failed spinal surgery.7,9–11
Zhang et al previously assessed the quality of lumbar fusion information on the internet as a whole, using top results from search engines.15 The clinical-based scoring system used in that study was comparable with our informative score. They found a mean score of 2.1 ± 1.4 in comparison with 1.57 ± 0.79 in our study.15 This may indicate that the quality of fusion information on YouTube is poorer than information on the internet generally. This is concerning given that YouTube is an exceedingly popular source of information.
Our clinical score mimics many other “topic specific” scores used in similar spine research. For example, the “scoliosis specific score”, “discectomy specific score”, and “failed back surgery syndrome–specific content score” have all been used to study the quality of information on the internet.7–11 These previous studies that have utilised a “surgery specicific score” have also found the quality of online information to be poor, similar to our findings. However, even in the context of these other studies, the quality of information on fusion and arthroplasty is particularly poor.7–11
Despite physician advice to the contrary, it is likely patients will continue to access YouTube as a source of medical information. According to YouTube.com, video search results are designed to “follow the audience.”34 This means the order of search results is based upon “how well the title, description, and video content match the viewer’s query” rather than how accurate the video content is.34
Patients who have obtained information from YouTube may continue to hold false beliefs regarding the surgery despite contradiction by their clinician. This poses a significant legal risk, as laid out by Todd et al, who presented a number of cases where the legal judgment was based on issues of consent without any criticism of other surgical procedures itself.35 This study highlights the poor quality of information on YouTube and therefore raises concern around its utility as part of the informed consent process. However, we acknowledge that patients will continue using the internet including YouTube for medical information, and we therefore give guidance on how to access higher quality information within YouTube in order to minimize adversely impacting the informed consent process.
For example, a clinician may recommend “educational” fusion videos, as these had the highest clinical scores, and the presence of a commercial element within these videos had no significant bearing on the score. Also, longer fusion videos and those appearing earlier in search results had higher clinical scores.
When searching for arthroplasty videos, the clinician may recommend academic videos as these had the highest clinical score. However, we recommend caution here as information may not be suitable for a lay audience. Longer arthroplasty videos also had better clinical scores. The supplemental data includes a link to an educational video that equaled the clinical score of the best academic video. We have also included the highest-scoring fusion video in the supplemental d.
Interestingly, testimonial videos had the second-highest clinical score in both the fusion and arthroplasty categories. They may also be more appealing to patients and pitched at a level that allows greater comprehension. Therefore, an argument could be made to recommend and direct patients toward testimonial videos for either procedure, to facilitate understanding or at the very least prompt further patient questions. Commercial videos scored poorly across the board, and clinicians should warn patients away from using these videos as a source of information.
It is important to note the weaknesses of this study. Scoring systems did not deduct points for incorrect information mixed in with correct “point-scoring” information. Furthermore, we gathered information on the top 50 videos in search results, but we feel it is unlikely that most patients would view results past the first page (20 results).
All videos were in English, and this may have been due to YouTube’s ability to access the searcher’s internet protocol address and tailor results based on geographical location. This means that our findings cannot necessarily be extrapolated to non-English speaking audiences. Finally, we did not assess language complexity in our study, so the ability for patients to understand the information provided was not assessed.
CONCLUSION
YouTube offers a poor source of information on both lumbar fusion and lumbar disc arthroplasty. Clinicians should be cautious about directing patients to YouTube as a potential educational source. Given that patients will continue to use the internet as an information source, clinicians may consider distributing written information recommending specific videos or using the findings of this study to guide patients toward higher quality videos.
Footnotes
Funding The author(s) received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests The authors report no conflicts of interest in this work.
Disclosures Joseph Baker reports grants or contracts from Medtronic, NuVasive, and Smith and Nephew; and payment or honoraria for lectures, presentations, speakers bureaus, manuscript writing, or educational events from Medtronic. Andrew Lee Muller has no disclosures.
- This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2022 ISASS. To see more or order reprints or permissions, see http://ijssurgery.com.