The Impact of Lexical Complexity on the Public’s Understanding of Governmental Communication Patrick J. Lyons, III · PAD Capstone Project, May 2018· Project Advisor: Dr. Atta Ceesay Abstract Complexity of language and specific jargon can make documents challenging to comprehend. Government has lagged in implementing these recommended guidelines. This research aims to determine if modified documents: a) are easier to read; b) are easier to comprehend; and c) require less assistance to understand than the original, unmodified documents Three focus groups were conducted and participants read modified and unmodified documents Data analysis included descriptive statistics, sign tests, effect sizes, paired sample t-tests, and qualitative explanations to determine outcomes. Results indicated that all documents were significantly more readable with moderate to large effect sizes found. Significant differences were not found, however, for improved comprehension and less required assistance to understand. Future steps should be taken to determine more feasible and parsimonious ways to improve readability and perhaps more importantly, comprehension. Data Collection Methods Focus groups read unmodified and modified documents from Social Security Administration Completed a survey asking for demographic information, impressions of readability on a likert scale, and responses to comprehension questions Groups qualitatively discussed their perceptions of the modified versus unmodified documents Sign Test Analyses of Employee Assistance   count p Focus Group 1 (SSI) 4 0.69 Focus Group 2 (RIB) 9 0.25 Focus Group 3 (MED) 8 0.36 Results Sign Test Analyses of Friend/Family Assistance Participant Demographics   count p Focus Group 1 (SSI) 5 0.19 Focus Group 2 (RIB) 8 0.14 Focus Group 3 (MED) 6 0.34 Variable Percentage of Focus Group 1 Percentage of Focus Group 2 Percentage of Focus Group 3 Education level Some high school High School Some College Associate’s degree Bachelor’s degree Post graduate degree Gender Male Female Age range Under 19 20-25 25-30 30-40 40-50 50-60 Race White Black Hispanic Asian Non-latino Native American Native Alaskan Mixed race Missing   0% 14% 43% 21% 7% 79% 36% 83% 17% 42% 58% 50% 8% 100% 25% 19% 6% 13% Purpose The Purpose of the study is to determine if recommended practices from the Plain Writing Act are implemented in unmodified documents from Social Security. Specifically, the study seek to answer: What does the public think about the complexity of the documents? Are modified documents significantly more readable than unmodified documents? Are participants significantly less likely to seek assistance from Social Security employees or family members/friends in order to understand the modified than the unmodified document? Does reading the modified document significantly improve participants’ comprehension? Sign Test Analyses of Readability   count p Cohen’s d Focus Group 1 (SSI) 8 .035 0.73 Focus Group 2 (RIB) 11 .0005 2.16 Focus Group 3 (MED) 12 .02 0.61 Review of Literature Average Percentage of Correct Responses for Comprehension The research has helped to pave the way for more accessible documents. Nonetheless, gaps in the literature are still apparent. One limitation is that many articles focus on the opinions of the researchers rather than the specific readability guidelines and formulas that have been set forth. Using a standardized way to improve readability and comprehension is necessary in order to reduce objectivity. Second, much of the research that has been done to modify documents has not considered whether or not the document still meets necessary legal standards. If this is not accomplished, the document may no longer be valid.   Unmodified Modified n M Focus Group 1 (SSI) 15 57% 60% Focus Group 2 (RIB) 14 66% Focus Group 3 (MED) 16 49% 51% Study Design Measured the complexity of Retirement, Medicare, and Supplemental Security Income documents using the Flesch-Kincaid tool (Flesch, 1948) Rewrote the documents using guidelines from the Plain Writing Act Had each document reviewed by a judge to determine if it met the same legal standards as the original Ran three focus groups (one for each document) to determine qualitative perceptions Assessed for readability using a likert scale Assessed for comprehension using 6-8 questions for each document Paired Samples t-test Analyses for comprehension Conclusions / Future Research Flesch-Kincaid tool is efficient and cost effective to determine readability Modifying documents is time consuming, and a cost/benefit analysis needs to be conducted The study found that readability was significantly improved with modified documents, but comprehension was not Limitations include small sample size, omission of cloze comprehension assessment, reading similar documents twice Future research should determine effective ways to improve readability and the impact this has on employee efficiency Should also consider high-stakes documents and stress and the influence that could have on comprehension in addition to readability   Unmodified Modified n M SD t df p Focus Group 1 (SSI) 13 0.56 0.05 0.59 0.06 0.88 12 0.40 Focus Group 2 (RIB) 0.03 0.63 0.96 0.36 Focus Group 3 (MED) 15 0.51 0.09 0.52 0.07 0.22 14 0.83 Sample Selection Convenience samples 2 focus groups at SUNY Buffalo State 1 focus group at Canisius High School