Abstract
Background Patient education is a key element of spinal surgery informed consent. Patients frequently access health information online, yet this information is unregulated and of variable quality. We aimed to assess the quality of information available on degenerative cervical myelopathy (DCM) websites with a focus on identifying high-quality information websites.
Methods We performed a Google search using keywords pertaining to DCM. The top 50 websites returned were classified based on their publication source, intended audience, and country of origin. The quality of these websites was assessed using both the DISCERN instrument and Journal of the American Medical Association (JAMA) benchmark criteria. We also utilized a novel Myelopathy Information Scoring Tool (MIST) to assess the comprehensiveness, accuracy, and detail of online DCM information.
Results The mean DISCERN score was 39.9 out of 80. Only one-quarter of these websites were rated “good” or “excellent” using DISCERN, and the remaining were rated “very poor,” “poor,” and “fair.” The mean JAMA benchmark score was 1.6 out of 4, with 23 out of 50 websites scoring 0. Evaluation using MIST found a mean score of 25.6 out of 50. Using 30 points as a satisfactory MIST cutoff, 72% of DCM websites were deemed critically deficient and unsatisfactory for comprehensive patient education. Both DISCERN and MIST indicated poorest information pertaining to surgical risks and complications as well as treatment outcomes. Websites such as Orthoinfo.aaos.org and Myelopathy.org provided reliable, trustworthy, and comprehensive patient education.
Conclusions Information available on almost three-quarters of DCM websites was of poor quality, with information regarding complications and treatment outcomes most deficient. Clinicians should be aware of quality sites where patients may be directed to augment patient education and surgical counseling.
Introduction
Searching for health information online is an increasingly common practice among patients. Patients were found to favor the Internet as their initial source of medical information,1 and almost two-thirds of adult patients seek medical information online.2,3 At the same time, health information on the Internet is unregulated and requires no verification from relevant health authorities, leading to various degrees of inaccuracies and biases.
Online health information plays a substantial role in determining outcomes of medical care, as it has been found to greatly impact the patient-physician relationship4 and patients satisfaction with the medical consultation.5
Degenerative cervical myelopathy (DCM) is the most common cause of spinal cord dysfunction, with prevalence as high as 2.3%.6 Despite its prevalence, diagnosis of DCM remains complex. Although some patients may present with classic symptoms of clumsiness or loss of balance, many exhibit a nonspecific clinical presentation, and its insidious onset is often dismissed as part of the normal aging process.7 The lack of awareness and low index of suspicion among physicians8 further contributed to diagnostic delays of up to multiple years.9 Radiographic evidence of the severity of cord compression is poorly correlated with clinical presentation and disease progression, adding to diagnostic uncertainty.10,11
We are uncertain of the overall quality of DCM information available on the Internet and whether this information is beneficial or detrimental to patient education. Being able to direct patients to high-quality websites may assist in patient education and counseling, particularly in the preoperative period when informed consent is paramount.12 Surgery is often recommended for DCM, and there are a variety of approaches available—whether an anterior, posterior, or combined approach is selected can greatly influence the risk profile and the postoperative kinematics.13 The outcomes of surgery can also vary substantially depending on the timing of surgery and preoperative deficit. It is well understood that patient recall of information is poor from clinical consultations, and being able to take time to educate themselves online may help overcome this.14
Therefore, we aimed to assess the quality of DCM information available on the Internet, with a particular view to identifying high-quality sites that may be recommended.
Methods
Institutional review and ethics approval were not required for this study.
Search engine is the most common method of seeking information online, and Google.com dominates the market with a market share of more than 90%.15 Thus, a Google search was performed by the primary author with the following keywords using the OR Boolean operator: “degenerative cervical myelopathy” OR “cervical spondylomyelopathy” OR “cervical myelopathy” OR “DCM” OR “CSM.” The search was performed incognito to block use of cookies and minimize the effect of personalized results as well as any influence based on location.The top 50 websites in the English language were included after applying the following exclusion criteria: websites unrelated to orthopedic spine surgery, DCM in nonhuman (veterinary) subjects, duplicate webpages from the same website, and websites requiring payment to access. The search was performed on 23 May 2023 and returned approximately 6,680,000 results in 0.42 seconds.
The 50 included websites were categorized by their publication source (academic, government, or professional board and private health care), intended audience (physician vs patient), and country of origin. Available standardized tools, such as the DISCERN instrument and Journal of the American Medical Association (JAMA) benchmark criteria, were used to assess these 50 websites. DISCERN was developed in 1996 by the University of Oxford and used to assess written consumer health information on treatment choices.16 The tool consists of 16 questions scoring 5 points each, assessing the reliability of the publication and the quality of treatment information. DISCERN scores were interpreted as follows: excellent, 63 to 75 points; good, 51 to 62 points; fair, 39 to 50 points; poor, 27 to 38 points; and very poor, 16 to 26 points. The JAMA benchmark criteria were developed by Silberg in 1997 and use a 4-point scoring system to assess authorship, attribution, currency, and declaration.17 Both the DISCERN instrument and JAMA benchmark criteria were well-established and validated health information assessment tools.18
The quality of information was assessed over the entire website and not limited to the single webpage directed to by the search engine. If the accessed webpage provided hyperlinks to another page within the same domain, these webpages were also included in the assessment. However, hyperlinks to another website or an external domain were not further assessed.
We introduced a novel tool called Myelopathy Information Scoring Tool (MIST), which consists of 10 questions (Figure 1). Each question evaluates a fundamental aspect of patient education that would be considered essential when gaining fully informed consent: (1) definition; (2) etiology; (3) clinical presentation; (4) investigations and imaging; (5) progression, natural history, and timing of intervention; (6) nonoperative treatment options; (7) operative treatment options; (8) postoperative recovery, (9) treatment goals and outcomes; and (10) risks and complications of surgery. Each question is scored from 0 to 5. A score of 0 is given for no mention of the aspect in question, score of 1 for mentioning a short answer with no elaboration, score of 2 for providing elaboration with major/critical deficiencies or biases, score of 3 for providing accurate information with minor/acceptable deficiencies, score of 4 for providing accurate and comprehensive information sufficient for patient education, and score of 5 for providing accurate, comprehensive, detailed, and current information sufficient for physician education. The minimum total score of MIST was 0, and the maximum score was 50.
Myelopathy Information Scoring Tool score sheet.
A score of 30 points and above was designated to be the threshold adequate for satisfactory patient education as each question is an important aspect of the condition and, therefore, should be reasonably comprehensive and accurate. A MIST score below 30 would indicate that some of these 10 key aspects either had minimal-to-no information or the information provided was inaccurate, biased, or critically deficient. All assessments using DISCERN, JAMA, and MIST were performed by the primary author.
Statistical Methods
Statistical analysis was performed on DISCERN, JAMA, and MIST scores of these 50 websites. Scores were compared between different publication sources, intended audience, and countries. We also assessed Pearson correlation between DISCERN, JAMA, and MIST scores. Significance threshold was set at P < 0.05. All statistical analyses were performed using Microsoft Excel and GraphPad Prism software.
Results
Table 1 describes the publication characteristics of all 50 websites evaluated. Ten were academic websites, 7 were government or professional board websites, and 33 were private health care websites. The majority of the websites originated from the United States (n = 30) followed by unspecified country (n = 10), United Kingdom (n = 4), Australia (n = 2), India (n = 2), Canada (n = 1), and Korea (n = 1). Forty websites were targeted at patients, while the remaining 10 were targeted at physicians.
Publication characteristics of websites (N = 50).
The mean DISCERN score for all 50 websites was 39.9 out of 80. Categorization of scores was as follows: excellent for 5 websites (10%), good for 7 websites (14%), fair for 13 websites (26%), poor for 13 websites (26%), and very poor for 12 websites (24%; Figure 2A). JAMA scores were generally poor, with a mean score of 1.6 out of 4. The score of 0 was given to 23 out of 50 websites, where none of authorship, attribution, disclosure, and currency were described (Figure 2B). The mean MIST score was 25.6 out of 50. Distribution of MIST scores was as follows: 5 websites (10%) scored 41 to 50, 9 websites (18%) scored 31 to 40, 20 websites (40%) scored 21 to 30, 13 websites (26%) scored 11 to 20, and 3 websites (6%) scored less than 10 (Figure 2C). Applying the designated cutoff score of 30 found 72% of websites to be unsatisfactory and critically inadequate for comprehensive patient education.
Distribution of (A) DISCERN scores, (B) Journal of the American Medical Association scores, and (C) Myelopathy Information Scoring Tool scores for 50 websites.
Comparison of scores between publication source, intended audience, and country of origin is shown in Table 2. Government and professional board websites performed better than private health care websites on all 3 scores, demonstrating statistical significance with DISCERN and JAMA, with MIST approaching statistical significance. Websites intended for physicians similarly scored higher than websites intended for patients across 3 scores, again with DISCERN and JAMA scores reaching statistical significance. Websites outside of the US performed better than websites from the US, with only the JAMA score being significant.
Comparison of DISCERN, JAMA, and MIST mean scores by publication source, intended audience, and country of origin.
Pearson correlation found poor correlation between MIST and DISCERN scores, reporting a correlation coefficient of 0.6868 (P < 0.0001). MIST and JAMA scores were also poorly correlated, with an r value of 0.2755 (P = .0264).
Observing the scores for individual questions of the DISCERN instrument, we found that the highest mean score (3.30 out of 5) was given to question 14, “Is it clear that there may be more than 1 possible treatment choice?” The second highest mean score of 3.28 was given to question 6: “Is it balanced and unbiased?” The lowest mean score (1.72 out of 5) was given to question 12 describing outcomes of no treatment, and the second lowest mean score of 1.86 was given to question 11 describing possible risks of each treatment (Figure 3A).
Mean individual component scores of (A) DISCERN and (B) Myelopathy Information Scoring Tool (MIST) for 50 websites.
Of the 10 questions posed by MIST, questions 1 and 3 pertaining to definition and clinical presentation scored the highest mean score of 3.66 and 3.64, respectively, out of 5. The lowest mean score of 1.6 was given to question 10 regarding risks and complications, followed by question 9 regarding treatment goals and outcomes, which scored 1.82 (Figure 3B). Private health care websites were particularly poor at presenting information around surgical risks and complications. Of the 33 private health care websites, 20 websites (60.6%) had no mention of surgical complications.
The 3 highest scoring websites using the DISCERN instrument were Orthoinfo.aaos.org, Myelopathy.org, and Orthobullets.com, with scores of 75, 71, and 70, respectively. Top-scoring websites using MIST were Myelopathy.org and Orthoinfo.aaos.org, tied at a maximum score of 50, followed by Orthobullets.com with a score of 45.
Discussion
Few studies have assessed the quality of DCM information available to the public.19,20 Patient awareness and education of the condition are generally lacking, and there is a paucity of trustworthy sources available outside of the clinical setting.21 One study found that information on DCM across various media largely pertained to health professionals, with only 15% targeted toward patients or layperson.19 Another study on DCM and social media (Twitter.com) reported that less than a third of tweets focused on spreading awareness and information.20 To the best of our knowledge, no published study to date has assessed the quality of information on DCM websites.
Assessment using DISCERN, JAMA, and MIST found almost three-quarters of DCM websites to be mediocre and inadequate for comprehensive patient education. Only 24% of DCM websites scored “good” or “excellent” on DISCERN, and 28% scored more than 30 points on MIST. Previous systematic reviews concluded that quality of online medical information was generally poor across various medical specialties,22 and orthopedics was no exception.23 Our findings advise spine surgeons to be cognizant that DCM patients are likely to present to their initial consultation with little prior information and thus require substantial education. Written information sheets and booklets can be published and disseminated in primary health settings and orthopedic clinics to help raise awareness and provide patient education. The importance of patient education and information booklets cannot be overstated as each forms 1 of the 3 main pillars of the “3-legged stool” model for spinal surgery informed consent.12
Higher-quality information being available on government and professional board websites as well as websites intended for physicians was within expectations. JAMA scores were higher for websites outside of the US due to a higher proportion of academic publication websites being classified as unspecified country. Poor correlation between MIST and the other two scores was similarly expected as evaluation using MIST differed greatly from both JAMA and DISCERN. JAMA benchmark criteria assess the credibility of the publication, such as authorship and declaration, but not the information content itself. The DISCERN instrument assesses information quality regarding treatment but not aspects such as etiology, prognosis, and investigation.
A recent study by Robillard et al found that no single optimal quality evaluation tool exists to assess health information;24 therefore, medical literature supported the use of novel scores for each specific medical condition, examples being the Scoliosis Specific Score25 and Anterior Cruciate Ligament/Anterolateral Ligament content-specific score.26 In this study, MIST was introduced to evaluate the quality of DCM information. MIST has a few distinct advantages. The health information content itself is evaluated, focusing on accuracy, neutrality, comprehensiveness, detail, and currency. Each question is scored with an objective and unambiguous 5-point system. The tool itself is concise, quick, and easy to use. Its generic template also fits into most surgical conditions, not limited to just myelopathy. We believe MIST may adopt a generic moniker of “Medical Information Scoring Tool” with its expanded use in the future.
We found surgical risks and complications among the poorest-scored components of DCM health information. Previous studies similarly reported surgical risk information to be often absent or lacking on online patient education materials,27,28 yet it is one of the most-desired pieces of information by patients prior to surgery.29 We hypothesize that websites detailing a comprehensive list of potential risks and complications may increase patient anxiety and undermine the confidence of the surgical practice, thus being frequently under-represented, especially on private health care websites. Another aspect of DCM information scored poorly was outcomes of surgery vs no treatment. Deficiency of this information may be due to DCM itself having variable and unpredictable progression,30,31 and the goal of decompressive surgery is to halt neurological deterioration, with varying degrees of recovery.32,33 These uncertainties, despite being well described in orthopedic spine literature, may be difficult to convey to patients.
Orthoinfo.aaos.org, Myelopathy.org, and Orthobullets.com consistently ranked in the top 3 in our assessment for best DCM information websites. Orthobullets.com is targeted at physicians, and information presented in bullet-point form is not ideal for patient education; thus, Myelopathy.org and Orthoinfo.aaos.org appear as our go-to recommendation for patients seeking quality DCM information. Myelopathy.org has the added benefit of patient advocacy and collaboration with DCM support groups, whereas Orthoinfo.aaos.org is largely informative with fewer patient-centered elements. We appreciate Myelopathy.org as a unique resource, itself being a charitable initiative that combines scientific research, physician education, and patient engagement in their effort to raise global myelopathy awareness. This multimodal approach offers hope for improved information provision within health care communities.
Our study had a few limitations. First, assessment was performed on DCM websites in the English language. Thus, our findings may not be generalizable to non-English websites. Second, our assessment was targeted at written online information and did not include video-based online information such as YouTube.com or social media websites, where patients were equally likely to go to seek information. Third, our study did not involve actual patients and was performed by investigators with medical qualifications. We are uncertain whether high quality, comprehensive information that is well-regarded by our investigators would be similarly well-received and appreciated by the average patient population. Last, all evaluations in this study were performed by a single author. However, other published studies assessing quality of patient education materials were similarly performed by a single author.34,35
Conclusion
Despite the large amount of information regarding DCM available online, much was of poor quality. Information surrounding surgical risks and complications as well as outcomes was especially deficient. Orthoinfo.aaos.org and Myelopathy.org provided some of the most reliable, accurate, and comprehensive DCM patient education materials, and clinicians may safely direct patients to these websites.
Footnotes
Funding The authors disclose no financial or material support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests Leo S.L. Chong and Mark Zhu have nothing to disclose. Joseph F. Baker discloses that he has received teaching honoraria from Medtronic and grants/contracts from NuVasive, Medtronic, and Smith and Nephew (all paid to the institution). All authors were fully involved in the study and preparation of the manuscript. All authors declare this study is original and they have full rights in the study materials.
Author Contributions Leo S. L. Chong – Conceptualization, methodology, investigation, data collection, writing: original draft, writing: review and editing. Mark Zhu – Methodology, data analysis, writing: original draft, writing: review and editing. Joseph F. Baker – Conceptualisation, methodology, writing: original draft, writing: review and editing, supervision.
- This manuscript is generously published free of charge by ISASS, the International Society for the Advancement of Spine Surgery. Copyright © 2023 ISASS. To see more or order reprints or permissions, see http://ijssurgery.com.
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.