Readability Assessment Of Allergy Online Patient Education Materials
Samir A. Ballestas, MD; Sara K. Wise, MD; Oswaldo A. Henriquez, MD.
Emory University Department of Otolaryngology-Head and Neck Surgery, Atlanta, GA.
Background
The national institute of health and the American medical association suggest that patient information across the internet should be in a 4th to 6th grade educational level for it to be understood proficiently by the average American. Of the 29 Rhinology fellow program webpages, 19 have health information for patients explaining what is allergy. Being allergic rhinitis one of the most common allergic diseases in the United Stated affecting between 20 and 25% of the population (1)
METHODS
All allergy related information was taken from the rhinology fellow programs webpages and pasted in a blank document, where non-related material such as images, videos, hyperlinks, contact information and others were deleted. Readability scores was evaluated using the www.readable.io online premium software and the scores taken into consideration were Flesch-Kincaid Grade Level (FKGL), Gunning-Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), Automated Readability Index (ARI) and Flesch Reading Ease Score (FRES). Fry Reading Graph (FRG) and Raymond Reading Graph (RRG) were also made. The grade results were correlated using IBM SPSS statistics software (version 25.0.0.0, IBM corp., Armonk, New York.)
RESULTS:
The results show that the information presented to the patients in these webpages is above their understanding level that is between 4th and 6th grade as suggested by The National Institutes of Health and the American Medical Association. We could see that the average scores for grade levels were 12th and 13th grade and college students for the Flesch Kincaid reading ease
Conclusions:
The average American doesn't have the access to accurate information in certified webpages that are at their level of understanding. Making it harder for people to have an accurate background of the condition they have, which could affect the way they see it and take actions to control it.
Key words: Allergy; Allergic Rhinitis; health literacy, Rhinology fellowship
Introduction
Allergy is the manifestation of an adverse immune response after the patient is in contact with harmless substances such as such as pollens, mold spores, animal dander, dust mites, foods, and stinging insects. (1) One of its manifestations, Allergic rhinitis, affects between 20 and 25% of United States population and is the inflammation of nasal mucosa caused by IgE-mediated reaction. (1)
Patients have at the reach of a click plenty of material related to any disease they encounter. Internet is today an essential medium for the diffusion of patient education information. (2)
At the other part, many of the people accessing the internet to receive knowledge about any kind of disease are not proficient enough to understand all the data found, being this a major limitation to the proper collection of educational material online (3)
Nowadays anybody can post any kind of articles across the internet related to any topic, without any kind of control, being this a major contribution to misinformation in general population looking for answers to their disease, which can cause real harms to patients. (4) Which makes us look for the easiest way to handle information, taking into consideration that lower functional reading level patients are frequently with a worse health condition than those with higher levels. (5)
The transcendence of Health organizations in this aspect of patient's life is to be a major source of adequate material, this is why we took into consideration for this study the webpages listed at the American Rhinology Association as rhinology fellow programs, being information relevant to allergy only found in 19 of the 29 programs.
Materials and Methods
Web pages of the list of Rhinology fellow programs from the American Rhinology Society were looked for and allergy related information was searched and copied into a blank document were non-relevant information such as References, images, hyperlinks, acknowledgements, author information was deleted. Information was copied from the following program's webpage: California Sinus Centers (6), Cleveland Clinic Foundation Rhinology Fellowship (7), Emory University Rhinology Fellowship (8), Jefferson Rhinology And Skull Base Fellowship(9), Johns Hopkins Sinus Center (10), Kaiser Permanente Orange County Rhinology Fellowship (11), Mass Eye & Ear Infirmary/Harvard (12), Medical University Of South Carolina (13), New York Presbyterian Hospital – Weill Cornell & Columbia (14), Texas Sinus Institute (15), University Of Arizona (16), University Of California Los Angeles (17), University Of Kansas Medical Center (18), University Of Miami Rhinology And Endoscopic Skull Base Surgery Fellowship Program / International Rhinology Clinical Fellowship Program (19), University Of North Carolina (20), University Of Pennsylvania (21), University Of Pittsburgh (22), University Of Utah (23), Vanderbilt University (24).
All the texts were analyzed as a whole, but they were also divided into subsections, which included general information, definition, causes, who is at risk, pathophysiology, signs and symptoms, diagnosis, treatment and prevention.
All the information was then copied and analyze in the online software of ''http:/readable.io/'' under a premium account, retrieving scores of the following readability tools: Flesch-Kincaid Grade Level (FKGL), Gunning-Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), Automated Readability Index (ARI) and Flesch Reading Ease Score (FRES).
The complete texts were also plotted in Fry readability graphs (FRG) and Raygor readability graphs (RRG).
FKGL = (11.8 × SY) + (0.39 × W) − 15.59 SY: Average number of syllables per word
W: average number of words per sentence
C: Average number of words with 3 or more syllables
L: Average number of letters per 100 words
CH: Average number of Characters per words
GFS = 0.4 × (W/S + {[C/W] × 100})
CLI CLI = (0.0588 × L) − (0.296 × S) − 15.8
SMOG = 1.0430 × C + 3.1291
ARI = (4.71 x CH) + (0.5 x W) – 21.43
FRES = 206.835 − ([84.6 × SY] − [1.015 × W])
FRG Average number of sentences and syllables per 100 words
RRG Average number of sentences and long (6 or more characters) words per 100 words
Table 1. Readability Assessment Formulas: Flesch-Kincaid Grade Level (FKGL), Gunning-Fog Index (GFI), Coleman-Liau Index (CLI), Simple Measure of Gobbledygook (SMOG), Automated Readability Index (ARI) Flesch Reading Ease Score (FRES), Fry readability graphs (FRG) and Raygor readability graphs. (RRG)
For the FKGL, GFI, CLI, SMOG and ARI, the result is given in the grade of school they should be in to be able to fully understand the text. In the case of the FRES, the lower the score, the better.
Flesch Reading Ease Score
Score School level
100 – 90 5th grade
90 – 80 6th grade
80 – 70 7th grade
70 – 60 8th and 9th grade
60 – 50 10th to 12th grade
50 – 30 College
30 – 0 College graduate
Table 2. Flesch Reading Ease Score
Results
19 webpages with their different subsections were analyzed, and after retreiving all the information, the results where then correlated using IBM SPSS statistics software (version 25.0.0.0, IBM corp., Armonk, New York.) the correlation was considered positive when the r value was positive. Being 1 the strongest correlation.
Flesch-Kincaid Grade Level Gunning-Fog Score Coleman-Liau Index Smog Index Automated Readability Index Grading Score Average Flesch-Kincaid Reading Ease
All the information 10.27 13.67 15.75 12.64 9.9 12.45 43.02
General information 12.2 15.64 17.24 13.82 11.9 14.16 32.14
Definition 10.8 13.75 15.47 13.02 10.22 12.65 41.18
Causes 6.79 10.03 15.56 9.54 7.42 9.86 59.62
Population at risk 10.6 13.81 13.58 12.98 9.65 12.12 46
Patophysiology 10.75 14.25 13.27 13.45 10.42 12.42 48.5
Signs and symptoms 7 8.87 13.84 9.67 7.01 9.27 61.6
Diagnosis 9.53 13.22 13.48 12.62 8.73 11.51 44.4
Treatment 9.69 13.68 15.9 12.4 9.7 12.27 45.98
Prevention 7.25 9.87 13.62 10.3 6.82 9.57 59.82
Table 3. Results given by the different categories
Table 4.
Table 5
Table 6
As it is known, the reading level of the patient information recommended by the INH should not be above 6th grade, but in our general analysis, the mean of readability grade levels (Flesch-Kincaid Grade level, Gunning-Fog score, Coleman-Liau index, Smog index and Automated readability index) was of 12.45, which means, the person reading the information should be at least in its 12th to 13th grade of scholarity, making this a college first year student. Also a Flesch Kincaid Reading Ease told us that patients should be in college in able to understand this information, with a general score of 45.14.
Given the results of each grading tool, none of the different categories had a final average result, the closest one to a 6th grade score was the category of causes using the Flesch-Kincaid Grade Level with a result of 6.79
The lowest grading scores in average were encountered at the categories of causes, signs and symptoms and prevention, but as a general rule, none of the averages of the results were lower than recommended.
All these general results were correlated one to another, being all of them statistically positive, which reinforced our confidence in the results, as can be seen in table 2.
Lastly, the Fry readability graphs (FRG) and Raygor readability graphs (RRG) results showed that none of the general information were suitable for the general population to read.
Discussion
The term Health literacy is defined by The National Assessment of Adult Literacy as ''the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions.' (25) Studies have shown the health literacy can be an important predictor of health outcomes. (26)
With the internet revolution, patients now have the access to all of the information of their actual diseases available in their computers or smartphones, which has made them become more active in the management of their own health and being able to educate themselves with online resources. (27)
But having all this information at their reach and not being able to adequately understand it is not useful. The National institute of Health provides a guide of "How to write easy to real health materials" (28) with several suggestions to provide patients the most adequate resource for their understanding.
Even though it is difficult to avoid the use of medical terminology, simpler and well-known words should be used. Recommendations given by Kasabwala el al. 2013, (29) encourage people in charge of producing this patient information to be very diligent when it comes to word choice, avoiding wordiness by replacing what seems to be long and complicated with words that are used commonly, have preference for 1-2 syllable words if possible, use concrete nouns, shorter sentences with varied style to captivate patients attention and keep him interested in what he is reading, break up text with bulleting or numbering and consider layout and supplemental material using colors, maintaining adequate white spaces and giving images and figures.
Resources like the Plain Language Action and Information Network (PLAIN) (30), which offers examples of word choices replacements, and the CDC's Health Literacy page (31) provide guidelines to adequately write the health information that should be given to the general public
Conclusion
Online patient education materials on allergy is written above the recommended 4th to 6th grade level material. Modifying the complexity in which these materials are written might be necessary to better engage our patient. Online patient education material provides a unique opportunity for patients to educate themselves and potentially improve the doctor-patient relationship. However, for it to be efficient the content has to be accurate and delivered in the proper degree of complexity.
Bibliography
1. Lalwani, Anil. CURRENT Diagnosis & Treatment in Otolaryngology – Head & Neck Surgery 3rd Edition. 1 Dec. 2016
2. Fox S: The social life of health information, 2011. Pew Research Center. May 12, 2011. (http://www.pewinternet. org/2011/05/12/the-social-life-of-health-information-2011/)
3. Agarwal N, Sarris C, Hansberry DR, Lin MJ, Barrese JC, Prestigiacomo CJ: Quality of patient education materials for rehabilitation after neurological surgery. NeuroRehabilita- tion 32:817-821, 2013
4. van der Marel S, Duijvestein M, Hardwick JC, et al. Quality of web-based information on inflammatory bowel diseases. Inflamm Bowel Dis 2009; 15:1891-1896.
5. Lee PP: Why literacy matters. Links between reading ability and health. Arch Ophthalmol 117:100-103, 1999
6. http://www.calsinus.com/conditions-allergies.html
7. http://my.clevelandclinic.org/health/articles/allergy-overview
8. https://www.emoryhealthcare.org/centers-programs/sinus-nasal-allergy-center/glossary.html#rhinitis
9. http://hospitals.jefferson.edu/diseases-and-conditions/allergy.html
10. http://www.hopkinsmedicine.org/healthlibrary/conditions/allergy_and_asthma/allergy_overview_85,P09504/
11. https://healthy.kaiserpermanente.org/health/care/!ut/p/a0/FchNDoMgEEDhs3iAyShRSrvzB6_Qwm5CJ0gCaJC016_dve-hxRfaTJ_gqYY9U7xsHOfK5UExcvGBT6D8BjrrlgifaNEehfzVJu_gyG38f1RqcJHRaD0tixQC1l620HW6BTXpG9z7YZhHtQqpNR4pqe_YND_A8Acs/
12. http://www.masseyeandear.org/specialties/otolaryngology/sinus-center/rhinitis
13. http://academicdepartments.musc.edu/medicine/divisions/pulmonary/patient_care/allergy.htm
14. http://www.nyp.org/library/85%7CP09504?l=en
15. https://med.uth.edu/orl/opal/nose-and-sinuses/allergic-rhinitis-hay-fever/
16. http://peds.arizona.edu/division/allergy-immunology-and-rheumatology
17. http://healthinfo.uclahealth.org/Search/90,P01663
18. http://www.kumed.com/medical-services/otolaryngology/allergy
19. http://uhealthsystem.com/health-library/allergy/allergy/allover
20. https://www.med.unc.edu/cmep/specialties/air
21. https://www.pennmedicine.org/for-patients-and-visitors/patient-information/conditions-treated-a-to-z/allergies
22. http://www.upmc.com/health-library/Pages/HealthwiseIndex.aspx?qid=hw33436#hw33438
23. http://healthcare.utah.edu/ent/specialties/allergy.php#tabs-accordion1
24. http://www.childrenshospital.vanderbilt.org/library/article.php?ContentTypeId=90&ContentId=P01663&Category=SearchTitle&SubtopicId=33153&lang=en§ion=33153&term=allergy
25. Badarudeen S, Sabharwal S. Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res. 2010;468:2572-2580.
26. Walsh TM, Volsko TA. Readability assessment of internetbased consumer health information. Respir Care. 2008;53: 1310-1315.
27. Patel PP, Hoppe IC, Ahuja NK, Ciminello FS. Analy- sis of comprehensibility of patient information regard- ing complex craniofacial conditions. J Craniofac Surg. 2011;22:1179-1182.
28. National Institutes of Health. How to Write Easy to Read Health Materials. http://www.nlm.nih.gov/ medlineplus/etr.html. Accessed September 4, 2012.
29. Kasabwala, K., et al. (2013). "Readability assessment of the American Rhinologic Society patient education materials." Int Forum Allergy Rhinol 3(4): 325-3
30. The Plain Language Action and Information Network. Word Suggestions. http://www.plainlanguage.gov/ howto/wordsuggestions/index.cfm. Accessed Septem- ber 4, 2012.
31. Strategic and Proactive Communication Branch, Di- vision of Communication Services, Office of the Asso- ciate Director for Communication, Centers for Disease Control and Prevention, U.S. Department of Health and Human Services. Simply Put: A Guide for Creating Easy-to-Understand Materials. April 2009. Atlanta, GA. http://www.cdc.gov/healthcommunication/ ToolsTemplates/Simply_Put_082010.pdf. Accessed September 4, 2012.