Assessing the Readability of Online Information About Hip Arthroscopy

Socioeconomic Factors Including Patient Income, Education Level, and Health Insurance Influence Postoperative Secondary Surgery and Hospitalization Rates Following Hip Arthroscopy

PURPOSE

To evaluate a large cross-sectional sample of patients utilizing administrative database records and analyze the effects of income, insurance type, and education level on outcomes after hip arthroscopy, including 2-year revision surgery, conversion to total hip arthroplasty (THA), and 90-day hospitalizations.

METHODS

Current Procedural Terminology codes were used to query the PearlDiver Mariner database from October 2015 to January 2020 for patients undergoing hip arthroscopy with a minimum 2-year follow-up. Patients were categorized by mean family income in their zip code of residence (MFIR), health insurance type, and educational attainment in their zip code of residence (EAR). Two-year revision arthroscopy, conversion to THA, and 90-day hospital readmissions or emergency department (ED) visits were analyzed along socioeconomic strata.

RESULTS

Multivariate analysis of 33,326 patients revealed that patients with MFIR between $30,000 and $70,000 had lower odds of 2-year revision arthroscopy (odds ratio [OR], 0.63; P < .001), THA conversion (OR, 0.76; P = .050), and 90-day readmission (OR, 0.53; P = .007) compared to MFIR >$100,000. Compared to patients with commercial insurance, patients with Medicare had lower odds of revision arthroscopy (OR, 0.60; P = .035) and THA conversion (OR, 0.46, P < .001) but greater odds of 90-day readmission (OR, 1.74; P = .007). Patients with Medicaid had higher odds of 90-day ED visits (OR, 1.84; P < .001). Patients with low EAR had higher odds of revision arthroscopy (OR, 1.42; P = .005) and THA conversion (OR, 1.58; P = .002) compared to those with high EAR.

CONCLUSIONS

Following hip arthroscopy, patients residing in areas with lower mean family income were less likely to undergo reoperations and readmissions. Medicare patients showed lower reoperation but higher readmission odds, while Medicaid patients showed higher odds of ED visits. Additionally, higher educational attainment in the zip code of residence is protective against future reoperation.

LEVEL OF EVIDENCE

Level III, retrospective case series.

The Potential of ChatGPT for High-Quality Information in Patient Education for Sports Surgery

BACKGROUND AND OBJECTIVE

Artificial intelligence (AI) advancements continue to have a profound impact on modern society, driving significant innovation and development across various fields. We sought to appraise the reliability of the information offered by Chat Generative Pre-Trained Transformer (ChatGPT) regarding diseases commonly associated with sports surgery. We hypothesized that ChatGPT could offer high-quality information on sports-related diseases and be used in patient education.

METHODS

On September 11, 2023, specific sports surgery-related diseases were identified to ask ChatGPT-4 (personal communication, March 4, 2023). The informative texts provided by ChatGPT were recorded by a non-observer senior orthopedic surgeon for this study. Ten texts provided by ChatGPT related to sports surgery diseases were evaluated blindly by two observers. Observers assessed and scored these texts based on the sports surgery-specific scoring (SSSS) and DISCERN criteria. The precision of the disease-related information offered by ChatGPT was evaluated.

RESULTS

The calculated average DISCERN score of the texts in the study was 44.75 points and the average SSSS score was 13.3 points. In the interclass correlation coefficient analysis of the measurements made by the observers, the agreement was found to be excellent (0.989; p < 0.001).

CONCLUSION

ChatGPT has the potential to be used in patient education for sports surgery-related diseases. The potential to provide quality information in this regard seems to be an advantage.

Readability and Quality of Online Patient Education Materials Concerning Posterior Cruciate Ligament Reconstruction

Objective This study aimed to assess the quality of online patient educational materials regarding posterior cruciate ligament (PCL) reconstruction. Methods We performed a search of the top-50 results on Google® (terms: "posterior cruciate ligament reconstruction," "PCL reconstruction," "posterior cruciate ligament surgery," and "PCL surgery") and subsequently filtered to rule out duplicated/inaccessible websites or those containing only videos (67 websites included). Readability was assessed using six formulas: Flesch-Kincaid Reading Ease (FRE), Flesch-Kincaid Grade Level (FKG), Gunning Fog Score (GF), Simple Measure of Gobbledygook (SMOG) Index, Coleman-Liau Index (CLI), Automated Readability Index (ARI); quality was assessed using the JAMA benchmark criteria and recording the presence of the HONcode seal. Results The mean FRE was 49.3 (SD 11.2) and the mean FKG level was 8.09. These results were confirmed by the other readability formulae (average: GF 8.9; SMOG Index 7.3; CLI 14.7; ARI 6.5). A HONcode seal was available for 7.4 % of websites. The average JAMA score was 1.3. Conclusion The reading level of online patient materials concerning PCL reconstruction is too high for the average reader, requiring high comprehension skills. Practice implications Online medical information has been shown to influence patient healthcare decision processes. Patient-oriented educational materials should be clear and easy to understand.

A standardised method for improving patient education material readability for orthopaedic trauma patients

PURPOSE

While the National Institutes of Health and American Medical Association recommend patient education materials (PEMs) should be written at the sixth-grade reading level or below, many patient education materials related to traumatic orthopaedic injuries do not meet these recommendations. The purpose of this study is to create a standardised method for enhancing the readability of trauma-related orthopaedic PEMs by reducing the use of ≥ three syllable words and reducing the use of sentences >15 words in length. We hypothesise that applying this standardized method will significantly improve the objective readability of orthopaedic trauma PEMs.

METHODS

A patient education website was queried for PEMs relevant to traumatic orthopaedic injuries. Orthopaedic trauma PEMs included (N = 40) were unique, written in a prose format, and <3500 words. PEM statistics, including scores for seven independent readability formulae, were determined for each PEM before and after applying this standard method.

RESULTS

All PEMs had significantly different readability scores when comparing original and edited PEMs (p < 0.01). The mean Flesch Kincaid Grade Level of the original PEMs (10.0 ± 1.0) was significantly higher than that of edited PEMs (5.8 ± 1.1) (p < 0.01). None of the original PEMs met recommendations of a sixth-grade reading level compared with 31 (77.5%) of edited PEMs.

CONCLUSIONS

This standard method that reduces the use of ≥ three syllable words and <15 word sentences has been shown to significantly reduce the reading-grade level of PEMs for traumatic orthopaedic injuries. Improving the readability of PEMs may lead to enhanced health literacy and improved health outcomes.

Readability and Quality of Online Information on Sickle Cell Retinopathy for Patients

PURPOSE

This study aims to evaluate the readability and quality of Internet-based health information on sickle cell retinopathy.

DESIGN

Retrospective cross-sectional website analysis.

METHODS

To simulate a patient's online search, the terms "sickle cell retinopathy" and "sickle cell disease in the eye" were entered into the top 3 search engines (Google, Bing and Yahoo). The first 20 results of each search were retrieved and screened for analysis. The DISCERN questionnaire, the Journal of the American Medical Association (JAMA) standards, and the Health on the Net (HON) criteria were used to evaluate the quality of the information. The Flesch-Kincaid Grade Level (FKGL), the Flesch Reading Ease (FRES), and the Automated Readability Index (ARI) were used to assess the readability of each website.

RESULTS

Of 16 online sources, 12 (75%) scored moderately on the DISCERN tool. The mean DISCERN score was 40.91 (SD, 10.39; maximum possible, 80). None of the sites met all of the JAMA benchmarks, and only 3 (18.75%) of the websites had HONcode certification. All of the websites had scores above the target American Medical Association grade level of 6 on both the FKGL and ARI. The mean FRES was 57.76 (±4.61), below the recommended FRES of 80 to 90.

CONCLUSION

There is limited online information available on sickle cell retinopathy. Most included websites were fairly difficult to read and of substandard quality. The quality and readability of Internet-based, patient-focused information on sickle cell retinopathy needs to be improved.

Assessing the readability of online information about jones fracture

BACKGROUND

Hand in hand with technological advancements, treatment modalities continue to grow. With the turn of the century, the internet has become the number one source of information for almost every topic. Thus, many patients look toward the internet as their primary source of information to learn about their respective medical conditions. The American Medical Association and National Institute of Health strongly recommend that online medical information be written at the 6^th to 8^th-grade level to aid comprehension by patients of all literacy backgrounds.

AIM

To assess the readability of online information regarding Jones fracture. Our hypothesis is that the reading level of medical information published on websites far exceeds the recommended reading level of 6^th-8^th grade as proposed by the American Medical Associate and National Institute of Health. The result of this study can help us formulate improved recommendations for publishing more comprehensible material and, thus, eventually improve patient compliance and clinical outcomes.

METHODS

The exact phrase “Jones fracture” was queried on the three most common search engines, Google, Yahoo!, and Bing, on December 28, 2022. As of December 2022, Google held 84%, Bing held 9%, and Yahoo! held 2% of the worldwide search engine market share. Web pages uniform resource locator from the first three pages of search results were recorded from each search engine. These web pages were classified according to academic, physician-sponsored, governmental and non-government organizations (NGO), commercial, and unspecified as per formally defined categories. Websites associated with an educational institution or medical organization were classified as academic. Websites with products for sale, corporate sponsorship, or advertisements were classified as commercial. Governmental websites or NGOs comprised those that received government subsidies or grants. Webpages that were independently owned by physicians or physician groups were respectively classed as physician sponsored. The remainder of websites that did not fall under the above categories were classified as unspecified.

RESULTS

A total of 93 websites were analyzed for reading assessment. A whopping 44% of websites were commercial, followed by 22% of physician-sponsored websites. Third place belonged to non-government organization websites holding a 15% share. The academic website held a meager 9% portion, while unspecified sites were 3%. The table illustrates mean readability scores, along with average cumulative grade level. The average grade level was 10.95 ± 2.28 for all websites, with a range of 6.18 to 18.90. Since P values were more than 0.05, there was not a significant statistical difference between the first page results and the results of all pages. Thus, we can rationalize that readability scores are consistent throughout all pages of a website.

CONCLUSION

Hand in hand with technological advancements, treatment modalities continue to grow. With the turn of the century, the internet has become the number one source of information for almost every topic. Thus, many patients look towards the internet as the primary source of information to learn about their respective medical conditions. Our study demonstrates that current online medical information regarding Jones fracture is written at an extraordinarily high-grade level, with an average grade level of all websites at 10.95, nearly an 10^th-grade educational level. The American Medical Association and National Institute of Health strongly recommend that online medical information should be written at the 6^th to 8^th-grade level to aid comprehension by patients of all literacy backgrounds. On the contrary, most of the medical information evaluated was at an 10^th-grade level, which far exceeds recommendations by AMA and NIH. This is particularly relevant because readability scores are directly proportional to the level of comprehension attained by readers, thus directly impacting patient outcomes. In conclusion, we suggest and encourage that all online reading materials should be re-written at the 6^th to 8^th-grade level in a public service effort to increase compliance with treatment goals and raise awareness of preventive measures.

Online Patient Education Resources for Anterior Cruciate Ligament Reconstruction: An Assessment of the Accuracy and Reliability of Information on the Internet Over the Past Decade

PURPOSE

The purpose of this study is to evaluate the quality of patient education materials accessible through popular online search engines regarding anterior cruciate ligament (ACL) injuries and anterior cruciate ligament reconstruction (ACLR).  Methods: Two search terms ("ACL surgery" and "ACL reconstruction") were entered into three search engines (Google, Yahoo, and Bing). The quality of information was scored using a novel scoring system developed and overseen by sports medicine orthopedic clinical research fellows and fellowship-trained orthopedic surgeons. Website quality, credibility, and readability were further assessed by the DISCERN score, Journal of the American Medical Association (JAMA) benchmark criteria, and Flesch-Kincaid Reading Grade Level (FKRGL), respectively. The Health On the Net Code of Conduct (HONcode) certification was also utilized to assess the transparency of health information for each website.

RESULTS

We evaluated 39 websites. The average score for all websites was 11.2±5.6 out of 28 total points. Six out of the 39 websites (41%) were HONcode certified. The websites that contained HONcode certification had a higher average JAMA benchmark score (3.5±0.7) and DISCERN score (44.6±14.7) when compared to the websites without the certification, 2.2±1.2 and 37.6 ± 15.9 for JAMA and DISCERN, respectively. The mean JAMA benchmark score was 2.7±1.2 (67.5%) for all websites out of a possible four points. The average FKRGL for all 39 websites was 10.0±2.0 (range: 5.4-13).

CONCLUSION

The quality of patient education materials accessible on the internet regarding ACL injuries and ACLR can be misleading and directly impact the patient's decision-making process essential to the patient-physician relationship over the past decade.

CLINICAL RELEVANCE

The internet can be a helpful online resource, however, surgeon clarification and consultation with qualified healthcare professionals are strongly recommended prior to clinical decision-making regarding potential treatment options.

Readability of spine-related patient education materials: a standard method for improvement

PURPOSE

Orthopaedic patient education materials (PEMs) have repeatedly been shown to be well above the recommended reading level by the National Institute of Health and American Medical Association. The purpose of this study is to create a standardized method to improve the readability of PEMs describing spine-related conditions and injuries. It is hypothesized that reducing the usage of complex words (≥ 3 syllables) and reducing sentence length to < 15 words per sentence improves readability of PEMs as measured by all seven readability formulas used.

METHODS

OrthoInfo.org was queried for spine-related PEMs. The objective readability of PEMs was evaluated using seven unique readability formulas before and after applying a standardized method to improve readability while preserving critical content. This method involved reducing the use of > 3 syllable words and ensuring sentence length is < 15 words. Paired samples t-tests were conducted to assess relationships with the cut-off for statistical significance set at p < 0.05.

RESULTS

A total of 20 spine-related PEM articles were used in this study. When comparing original PEMs to edited PEMs, significant differences were seen among all seven readability scores and all six numerical descriptive statistics used. Per the Flesch Kincaid Grade level readability formula, one original PEM (5%) versus 15 edited PEMs (75%) met recommendations of a sixth-grade reading level.

CONCLUSION

The current study shows that using this standardized method significantly improves the readability of spine-related PEMs and significantly increased the likelihood that PEMs will meet recommendations for being at or below the sixth-grade reading level.

The Effect of Education Planned According to Health Literacy Level on Functionality, Problems Experienced, and Quality of Life in Patients Undergoing Total Knee Replacement: A Nonrandomized Comparison Group Intervention Study

Total knee arthroplasty procedures significantly improve quality of life for persons with debilitating pain. It is essential to plan patient education according to the level of health literacy. This nonrandomized comparison group intervention study examines the effect of education planned according to health literacy level on functionality, postoperative problems (pain and anxiety), and quality of life in patients who undergo total knee replacement. During the study, patients in the intervention group (n = 51) were provided with the planned patient education according to their health literacy level, whereas patients in the comparison group were given routine care (n = 51). Patients in the postoperative intervention group experienced fewer problems (p < .05). Education tailored to patients' health literacy levels had a positive effect on their functionality and quality of life (p < .05). Our findings suggest planning education according to patient health literacy levels may have a positive influence on functionality, postoperative problems, and quality of life among patients undergoing total knee replacement.

Evaluation of the quality of information online for arthroscopic Bankart repair

PURPOSE

The purpose of this study was to evaluate the quality and readability of information online for patients searching terms related to arthroscopic Bankart repair (ABR).

METHODS

Google, Yahoo and Bing were searched with terms related to ABR. The quality of information was assessed using the Journal of the American Medical Association (JAMA) Benchmark criteria, DISCERN Score, and the Flesch-Kincaid Reading Ease & Grade Level. The presence of the HONcode marker was noted. Additionally, we used a scoring system specific to content relating to the ABR (AB score), a 1-20 Likert scale. Websites were also categorised according to the source: academic institution, physician, allied healthcare, commercial, media or social media. Statistical analysis was performed using GraphPad Prism.

RESULTS

Ninety-six unique websites were evaluated, with the most common website category being physician websites (52) and academic institution websites (24). There were nine websites with the HONcode marker. The average JAMA Benchmark criteria score was 1.95 (1-4), the average DISCERN score was 48.8 (20-78), with an average Flesch-Kincaid reading ease of 50.9 (11-96) and grade level of 8 (1-18). The average AB score was 5.9 (0-18), and there was a strong correlation with a higher DISCERN score (r=0.57), but not JAMA score (r=0.18) or Flesch-Kincaid grade (r=0.16). Websites with the HONcode marker did not score higher in any criteria than those without it (p>0.05). The quality of information on physician websites was better than (statistically insignificant) non-physician websites; however, the readability of information in the former was poorer (statistically significant) than the latter.

CONCLUSION

There was wide variability in the quality and readability online of the information on ABR, and the AB scoring system was shown to correlate strongly with increased quality.

Systematic Assessment of the Quality and Comprehensibility of YouTube Content on Ulnar Collateral Ligament Injury and Management

Background

Ulnar collateral ligament (UCL) reconstruction has received a unique level of attention in the press and social media. There has also been an increasing use of the internet by patients to seek medical information. Concern exists regarding the quality and comprehensibility of online information when used for patient education.

Purpose

To evaluate the quality and comprehensibility of the most-viewed YouTube videos related to the diagnosis and management of UCL injuries. Based on our new evidence-based scoring rubrics, we hypothesized that the quality and comprehensibility of these videos would be poor.

Study Design

Cross-sectional study.

Methods

The YouTube platform was searched on September 7, 2021, with the terms "UCL injury," "ulnar collateral ligament injury," "UCL surgery," "ulnar collateral ligament surgery," and "Tommy John surgery," and the 50 most-viewed videos from each search were compiled, yielding 250 videos. After removal of duplicates and application of exclusion criteria, the 100 most-viewed videos remained. Basic attributes, including duration of video and number of views, were recorded. Each video was then analyzed by 2 independent reviewers and evaluated for 4 key parameters (quality of diagnostic content [QAR-D], quality of treatment content [QAR-T], presence of inaccurate information, and comprehensibility) and graded on a novel scale from 1 to 4 (4 being the most appropriate for patient education).

Results

The mean QAR-D was 4.83 ± 3.41 (fair quality), and the mean QAR-T was 2.76 ± 3.26 (poor quality). Physician-led educational videos had both the highest mean QAR-D (6.37) and the highest mean QAR-T (4.34). No correlation was observed between video quality and views/likes. A total of 12 videos included ≥1 inaccuracy. The mean comprehensibility score was 2.66 ± 1.12, with 39 videos falling below the acceptable comprehensibility threshold (score <3).

Conclusion

The overall quality of UCL injury-related YouTube content was low. In addition, the absence of correlation between video quality and views/likes suggests that patients are not preferentially utilizing the limited high-quality content that does exist on the YouTube platform. In addition, inaccurate videos were prevalent (12%), and almost half of all videos were deemed inappropriate for patient education in terms of comprehensibility, as defined by our comprehensibility parameter.

Omit needless words: Sentence length perception

Short sentences improve readability. Short sentences also promote social justice through accessibility and inclusiveness. Despite this, much remains unknown about sentence length perception-an important factor in producing readable writing. Accordingly, we conducted a psychophysical study using procedures from Signal Detection Theory to examine sentence length perception in naive adults. Participants viewed real-world full-page text samples and judged whether a bolded target sentence contained more or fewer than 17 words. The experiment yielded four findings. First, naïve adults perceived sentence length in real-world text samples quickly (median = 300-400 ms) and precisely (median = ~90% correct). Second, flipping real-world text samples upside-down generated no reaction-time cost and nearly no loss in the precision of sentence length perception. This differs from the large inversion effects that characterize other highly practiced, real-world perceptual tasks involving canonically oriented stimuli, most notably face perception and reading. Third, participants significantly underestimated the length of mirror-reversed sentences-but not upside-down, nor standard sentences. This finding parallels participants' familiarity with commonly occurring left-justified right-ragged text, and suggests a novel demonstration of left-lateralized anchoring in scene syntax. Fourth, error patterns demonstrated that participants achieved their high speed, high precision sentence-length judgments by heuristically counting text lines, not by explicitly counting words. This suggests practical advice for writing instructors to offer students. When copy editing, students can quickly and precisely identify their long sentences via a line-counting heuristic, e.g., "a 17-word sentence spans about 1.5 text lines". Students can subsequently improve a long sentence's readability and inclusiveness by omitting needless words.

Patient education information material assessment criteria: A scoping review

BACKGROUND

Patient education information material (PEIM) is an essential component of patient education programs in increasing patients' ability to cope with their diseases. Therefore, it is essential to consider the criteria that will be used to prepare and evaluate these resources.

OBJECTIVE

This paper aims to identify these criteria and recognize the tools or methods used to evaluate them.

METHODS

National and international databases and indexing banks, including PubMed, Scopus, Web of Science, ProQuest, the Cochrane Library, Magiran, SID and ISC, were searched for this review. Original or review articles, theses, short surveys, and conference papers published between January 1990 and June 2022 were included.

RESULTS

Overall, 4688 documents were retrieved, of which 298 documents met the inclusion criteria. The criteria were grouped into 24 overarching criteria. The most frequently used criteria were readability, quality, suitability, comprehensibility and understandability.

CONCLUSION

This review has provided empirical evidence to identify criteria, tools, techniques or methods for developing or evaluating a PEIM. The authors suggest that developing a comprehensive tool based on these findings is critical for evaluating the overall efficiency of PEIM using effective criteria.

Reducing the Use of Complex Words and Reducing Sentence Length to <15 Words Improves Readability of Patient Education Materials Regarding Sports Medicine Knee Injuries

Purpose

To develop a standardized method to improve readability of orthopaedic patient education materials (PEMs) without diluting their critical content by reducing the use of complex words (≥3 syllables) and shortening sentence length to ≤15 words.

Methods

OrthoInfo, a patient education website developed by the Academy of American Orthopedic Surgeons, was queried for PEMs relevant to the care of athletic injuries of the knee. Inclusion criteria were PEMs that were unique, pertained to topics of knee pathology in sports medicine, and written in a prose format. Exclusion criteria were information presented in video or slideshow format, or topics not pertaining to knee pathology in sports medicine. Readability of PEMs was evaluated using 7 unique readability formulas before and after applying a standardized method to improve readability while preserving critical content (reducing the use of ≥3 syllable words and ensuring sentence length is ≤15 words). Paired samples t-tests were conducted to assess the relationship between reading levels of the original PEMs and reading level of edited PEMs.

Results

Reading levels differed significantly between the 22 original PEMs and edited PEMs across all 7 readability formulas (P < .01). Mean Flesch Kincaid Grade Level of original PEMs (9.8 ± 1.4) was significantly increased compared to that of edited PEMs (6.4 ± 1.1) (P = 1.9 × 10^-13). 4.0% of original PEMs met National Institutes of Health recommendations of a sixth-grade reading level compared with 48.0% of modified PEMs.

Conclusions

A standardized method that reduces the use of ≥3 syllable words and ensures sentence length is ≤15 words significantly reduces the reading-grade level of PEMs for sports-related knee injuries. Orthopaedic organizations and institutions should apply this simple standardized method when creating PEMs to enhance health literacy.

Clinical Relevance

The readability of PEMs is important when communicating technical material to patients. While many studies have suggested strategies to improve the readability of PEMs, literature describing the benefit of these proposed changes is scarce. The information from this study details a simple standardized method to use when creating PEMs that may enhance health literacy and improve patient outcomes.

Can Patients Read, Understand, and Act on Online Resources for Anterior Cruciate Ligament Surgery?

Background:

Patients undergoing elective procedures often utilize online educational materials to familiarize themselves with the surgical procedure and expected postoperative recovery. While the Internet is easily accessible and ubiquitous today, the ability of patients to read, understand, and act on these materials is unknown.

Purpose:

To evaluate online resources about anterior cruciate ligament (ACL) surgery utilizing measures of readability, understandability, and actionability.

Study Design:

Cross-sectional study; Level of evidence, 4.

Methods:

Using the term “ACL surgery,” 2 independent searches were performed utilizing a public search engine (Google.com). Patient education materials were identified from the top 50 results. Audiovisual materials, news articles, materials intended for advertising or medical professionals, and materials unrelated to ACL surgery were excluded. Readability was quantified using the Flesch Reading Ease, Flesch-Kincaid Grade Level, Simple Measure of Gobbledygook, Coleman-Liau Index, Automated Readability Index, and Gunning Fog Index. The Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P) was utilized to assess the actionability and understandability of materials. For each online source, the relationship between its Google search rank (from first to last) and its readability, understandability, and actionability was calculated utilizing the Spearman rank correlation coefficient (ρ_S).

Results:

Overall, we identified 68 unique websites, of which 39 met inclusion criteria. The mean Flesch-Kincaid Grade Level was 10.08 ± 2.34, with no website scoring at or below the 6th-grade level. Mean understandability and actionability scores were 59.18 ± 10.86 (range, 33.64-79.17) and 34.41 ± 22.31 (range, 0.00-81.67), respectively. Only 5 (12.82%) and 1 (2.56%) resource scored above the 70% adequate PEMAT-P threshold mark for understandability and actionability, respectively. Readability (lowest P value = .103), understandability (ρ_S = –0.13; P = .441), and actionability (ρ_S = 0.28; P = .096) scores were not associated with Google rank.

Conclusion:

Patient education materials on ACL surgery scored poorly with respect to readability, understandability, and actionability. No online resource scored at the recommended reading level of the American Medical Association or National Institutes of Health. Only 5 resources scored above the proven threshold for understandability, and only 1 resource scored above it for actionability.

Automated Text-Messaging After Hip Arthroscopy: A Randomized-Controlled Trial of "Post-Op Buddy"

PURPOSE

To assess an automated text-messaging system for patients after hip arthroscopy and its impact at 90 days on the Hip Disability Osteoarthritis Outcome Score Physical Function Short form (HOOS-PS, HOOS-Pain), compliance with rehabilitation guidelines, and patient satisfaction.

METHODS

One hundred twenty-one participants (average age 29 ± 8.7 years, 52% female) undergoing hip arthroscopy at 2 academic institutions were prospectively enrolled and randomized to receive (1) standard perioperative communication or (2) additional automated mobile phone text messages. Inclusion criteria included ability to communicate in written English and access to a mobile phone with text-messaging capability. Patients undergoing revision surgery or simultaneous femoral or acetabular osteotomy were excluded. HOOS-PS and HOOS-Pain were collected preoperatively, and after surgery an automated mobile phone robot sent participants in the therapeutic arm intermittent text messages for 90 days. At 90 days all participants again completed HOOS-PS, HOOS-Pain, and additional survey questions on satisfaction with their experience (10-point scale), communication from the surgical team (10-point scale) and adherence to physical therapy exercises, weight-bearing guidelines, and brace use, The primary outcome assessed was a statistically significant change in HOOS-PS and HOOS-Pain; secondary outcomes included change in satisfaction, communication, and adherence to physical therapy exercises, weightbearing guidelines, or brace use. Wilcoxon rank sum was used to compare HOOS-PS and HOOS-Pain scores at 0 and 90 days. Demographic characteristics and survey variables were compared using Students t test for continuous variables and χ² or Fisher exact test for categorical variables as appropriate.

RESULTS

There were statistically significant and clinically relevant improvements in HOOS-PS and HOOS-Pain in both groups (P < .05). Subjective feedback was strongly positive, with 96% of text message participants reporting they would choose automated messages if it was offered to them again in the future.

CONCLUSIONS

Ninety days of automated text messaging after hip arthroscopy failed to show a significant difference in HOOS-PS (P = .09), HOOS-Pain (P = .13), patient-reported compliance with postoperative guidelines, or satisfaction with support and communication from the surgical team.

LEVEL OF EVIDENCE

I, randomized control trial (RCT).

Online Readability Analysis: Mohs Micrographic Surgery Postsurgical Wound Care

OBJECTIVE

Although various treatment modalities exist for skin cancer care, Mohs micrographic surgery (MMS) is one of the most effective and cosmetically sensitive surgical techniques. The readability of online informational materials specifically related to MMS postsurgical wound care is extremely relevant for patients given the high rates of skin cancer in the US and the number of patients utilizing the internet for information.

METHODS

Investigators conducted Google searches to evaluate the current state of readability on MMS postsurgical wound care using the specific keywords "healing after Mohs surgery" and "wound care Mohs surgery." The readabilities of individual websites were analyzed using the WebFX online software. Health-specific click-through rate was used to select the number of samples assessed.

RESULTS

Based on the 33 unique websites evaluated, the aggregate readability of online MMS postsurgical wound care materials was 11.3, corresponding to an 11th-grade reading level by US standards.

CONCLUSIONS

Although this level of readability is an improvement in online MMS postsurgical wound care information relative to prior literature, there is still considerable work to be done by the dermatologic community in improving the readability of online patient materials regarding MMS.

The readability of online patient education materials for slipped capital femoral epiphysis

Given the long-term complications of undiagnosed slipped capital femoral epiphysis (SCFE) and the importance of readable health information materials on positive, equitable health outcomes, the objective of this study was to determine if the online patient education materials regarding SCFE are written at or below accepted recommendations. The secondary objective was to determine whether the readability of these materials varied when stratified by the type of website. 'Slipped capital femoral epiphysis', 'SCFE', and 'slipped femoral head' were used as search queries in three common search engines. The readability of each website was evaluated using five established metrics, and the scores were compared by website type and by the complexity of the search query. In this study of 53 unique websites about SCFE, we demonstrated that only one of the web pages was written at the recommended sixth-grade level, and the mean reading level of the online material was above the 10th-grade level. Post hoc testing showed that only websites associated with pediatric academic institutions were written at a significantly lower grade level than general health websites [P < 0.05 for all, range (0.003, 0.04)]. The materials about SCFE that are available to patients and their families continue to be written at an inappropriate level. To increase accessibility and allow for equitable long-term health outcomes, physicians, universities, hospitals and medical societies must ensure that they produce readable education to increase patients' understanding of SCFE, its symptoms and available treatment options. Future studies evaluating progress regarding these metrics are warranted.

Assessing the readability of patient-targeted online information on musculoskeletal radiology procedures

OBJECTIVE

To assess the readability of patient-targeted online information on musculoskeletal radiology procedures.

METHODS

Eleven common musculoskeletal radiology procedures were queried in three online search engines (Google, Yahoo!, Bing). All unique patient-targeted websites were identified (n = 384) from the first three pages of search results. The reading grade level of each website was calculated using 6 separate validated metrics for readability assessment. Analysis of word and sentence complexity was also performed. Results were compared between academic vs. non-academic websites and between websites found on different pages of the search results. Statistics were performed using a t test.

RESULTS

The mean reading grade level across all procedures was 10th-14th grade. Webpages for nerve block were written at a higher reading grade level on non-academic websites (p = 0.025). There was no difference in reading grade levels between academic and non-academic sources for all other procedures. There was no difference in reading grade levels between websites found on the first page of search results compared with the second and third pages. Across all websites, 16-22% of the words used had 3+ syllables and 31-43% of the words used had 6+ characters (complex words); 13-24% of the sentences used had 22+ words (complex sentences).

CONCLUSION

Patient-targeted online information on musculoskeletal radiology procedures are written at the 10th-14th grade reading level, which is well beyond the AMA and NIH recommendation. Readability can be lowered by decreasing text complexity through limitation of high-syllable words and reduction in word and sentence length.

A Systematic Assessment of YouTube Content on Femoroacetabular Impingement: An Updated Review

Background:

Online video-sharing platforms such as YouTube have become popular sources of medical information for patients. However, concern exists regarding the quality of such non–peer reviewed content. In fact, a previous investigation found the majority of YouTube information related to femoroacetabular impingement (FAI) to be of poor quality.

Purpose:

To provide an updated assessment of the quality of FAI-related videos available on YouTube.

Study Design:

Cross-sectional study.

Methods:

The terms FAI, femoroacetabular impingement, and hip impingement were searched on YouTube, and exclusion criteria were applied to the first 100 results for each term. The diagnostic and treatment content of each video was graded and assigned a quality assessment rating based on a previously used rubric. Video characteristics (e.g. duration, views, “likes”) were compared using both quality assessment rating and video source.

Results:

A total of 142 videos were included in the final analysis. The most common video source was educational (48.6%), followed by physician-sponsored (30.3%). The majority of videos were graded as “somewhat useful” for both diagnostic and treatment content (59.4% and 61.6%, respectively); however, treatment content was rated “not useful” more often than diagnostic information (20.3% vs. 8.7%, respectively). Videos rated as “somewhat useful” received the most views per day on average, while educational videos were the most viewed by source (views and views per day). Educational videos had more views and likes on average than physician-sponsored videos (P < .05), but all other comparisons of video characteristics by source were not significant. Video duration was the only characteristic found to vary significantly by quality assessment rating (P < .001 for both diagnostic and treatment analyses), with higher-quality videos tending to be longer. Videos rated as “excellent” and “very useful” had mean durations >30 minutes but were viewed the least.

Conclusion:

The overall quality of FAI-related content on YouTube remains low. Clinicians should be familiar with medical information available to patients on the internet, as it can influence patients’ perspectives and shared decision-making processes. This review substantiates the need for more publicly available, high-quality video content regarding the diagnosis and treatment of FAI.

Assessing the Readability of Online Information About Achilles Tendon Ruptures

The American Medical Association (AMA) and National Institutes of Health (NIH) currently suggest that health care materials be written at a sixth-grade reading level. Our study investigates the readability of online information on Achilles rupture and reconstruction. Achilles tendon rupture, Achilles tendon repair, and Achilles tendon reconstruction were queried using advanced search functions of Google, Bing, and Yahoo!. Individual websites and text from the first 3 pages of results for each search engine were recorded and categorized as physician based, academic, commercial, government and nongovernmental organization, or unspecified. Individual readability scores were calculated via 6 different indices: Flesch-Kincaid grade level, Flesch Reading Ease, Gunning Fog, SMOG, Coleman-Liau index, and Automated Readability Index along with a readability classification score and average grade level. A total of 56 websites were assessed. Academic webpages composed the majority (51.8%), followed by physician-based sources (32.1%). The average overall grade level was 10.7 ± 2.54. Academic websites were written at the highest-grade level (11.5 ± 2.77), significantly higher than physician-based websites (P = .040), and only 2 were written at, or below, a sixth-grade reading level. Currently, online information on Achilles tendon rupture and reconstruction is written at an inappropriately high reading level compared with recommendations from the AMA and NIH.Level of Evidence: Level IV.

Will the Effect of New Federal Breast Density Legislation Be Diminished by Currently Available Online Patient Educational Materials?

RATIONALE AND OBJECTIVES

To evaluate readability of commonly accessed online patient educational materials (OPEM) on breast density in setting of recently passed federal breast density legislation.

MATERIALS AND METHODS

The term "breast density" was queried using an online search engine to identify the top 50 commonly accessed websites based on order of search results on December 15, 2018. Location, cookies, and user account information were disabled prior to our query. Only websites with OPEM, defined as any educational material on breast density targeted towards the general public, were evaluated in our study. Sponsored hits and research journal articles were excluded. Available patient-directed information from websites meeting inclusion criteria was then downloaded. Grade-level readability was then determined from formatted content using generalized estimating equations, with observations nested within readability metrics from each website. Results were compared to American Medical Association recommended readability parameters (sixth-grade reading level). All interval estimates were calculated for 95% confidence.

RESULTS

Fouty-one websites met inclusion criteria representing patient-directed OPEM on breast density. Average grade-level readability of health information on breast density in our study ranged from 8.5-16.5 years with an average grade reading level of 11.1 years across all websites. Of websites fitting into a specific category, academic websites had the highest average grade reading level (12.0), while nonprofit websites had the lowest average grade reading level (10.4). Nearly half (19/41) of all websites in our study had diagrams to aid in patient comprehension, while few websites (2/41; 4.8%) displayed videos in addition to written content. The website with the lowest average grade reading level was WebMD, which had an average reading level of 8.5. No individual website in our study met American Medical Association recommended parameters of a sixth-grade reading level CONCLUSION: Readability of currently available OPEM on breast density may be written at a level too difficult for the general public to comprehend, which may represent a barrier to educational goals of newly passed federal breast density legislation.

Evaluation of Hip Preservation-related Patient Education Materials From Leading Orthopaedic Academic Centers in the United States and Description of a Novel Video Assessment Tool

The readability, reliability, and quality of online hip preservation–related patient education materials from the top 20 orthopaedic academic centers in the United States were evaluated.

Online Resources for Rotator Cuff Repair: What are Patients Reading?

Purpose

The purpose of this study was to use a novel scoring system to evaluate the content and grade the quality of websites that patients may use to learn about rotator cuff repair.

Methods

Two search terms (“rotator cuff repair” and “rotator cuff surgery”) were entered into 3 Internet search engines (Google, Yahoo, and Bing). We scored the quality of information using a novel scoring system. Website quality was further assessed by the Journal of the American Medical Association (JAMA) benchmark criteria and Health on the Net Foundation (HON) code certification. The readability of the websites was evaluated with the Flesch-Kincaid score.

Results

We evaluated 47 websites. The average quality for all websites was 6.47 ± 5.21 (maximum 20 points). There was a large difference in scores between the top 5 websites and the remaining websites (16.30 vs 5.51, P < .001). There was no difference in scores when comparing the 3 different search engines (P = .85). The mean reading level was 10.17 ± 2.24. Reading level was not significantly correlated with quality (r_s = 0.14, P = .36). The average JAMA benchmark criteria score for all websites was 2.34 ± 1.11 (maximum 4 points). JAMA criteria score was not significantly correlated with quality (r_s = 0.02, P = .91). Sites without HONcode certification had higher quality scores (8.33 ± 4.80) than sites with HONcode certification (6.18 ± 4.66), but this difference was not statistically significant (P = .15).

Conclusion

The quality of patient-level information on rotator cuff repair on the Internet is both incomplete and written at a reading level higher than current recommendations. Information quality is not significantly correlated with reading level or JAMA criteria, and does not depend on the search term used or HONcode certification.

Clinical Relevance

Patients having rotator cuff repair may seek information on the Internet; the information may require surgeon clarification.

Readability of Online Patient Educational Materials Related to Breast Lesions Requiring Surgery

Purpose To evaluate readability of websites that are commonly accessed for information on breast lesions requiring surgery. Materials and Methods An internet search using three malignant and eight nonmalignant breast lesions that traditionally require lumpectomy or excisional biopsy as search terms was conducted to identify websites commonly accessed for patient information on breast lesions requiring surgery. Nine websites with information on breast diagnoses were identified based on search engine results for each breast lesion queried. Available patient-directed information was downloaded for each lesion from each website on May 15, 2018. Grade-level readability of downloaded content for each lesion was then determined by using generalized estimating equations, with observations nested within readability metrics from each website. Readability of associated terms breast biopsy, breast cancer, and breast surgery was also evaluated with the same method. Results were compared with American Medical Association (AMA) recommended readability parameters (sixth-grade reading level). All interval estimates were calculated for 95% confidence. Results Average grade level readability score of health information on breast lesions requiring surgery was 11.7, which exceeded the AMA parameters. Information on Wikipedia was written at the highest reading level (grade level readability score, 14.2), while information on the National Institutes of Health website ( http://cancer.gov ) was written at the lowest reading level (grade level readability score, 9.7). Educational materials on malignant breast lesions (grade level readability score, 12.3) were written at a higher reading level than were those on nonmalignant breast lesions (grade level readability score,11.4). Information on the terms breast biopsy (grade level readability score, 10.9), breast cancer (grade level readability score, 10.6), and breast surgery (grade level readability score, 11.1) were all written above a sixth-grade reading level. Conclusion Readability of current online resources on breast biopsy lesions traditionally requiring surgery may be too complex for the general public to comprehend, leading to misinformation and confusion. © RSNA, 2019 See also the editorial by Haygood in this issue.

Editorial Commentary: Smarter Than a Sixth Grader? Hip Arthroscopists, Check Your Training at the Door

Literacy is one of the best predictors of an individual's health, and providing effective, patient-centered communication to our less literate patients has become a challenge for the entire medical community. The internet has become patients' first-line source of medical information, and as caretakers of musculoskeletal medicine, it is orthopaedists' responsibility to provide high-quality, patient-centric resources that are simple and clear. When creating our own practice or institution's websites, we must check our training and technicality at the door to create resources that are easily readable and digestible.