1. The Lexile Framework for Reading: Integrating Measurement and Instruction COPENHAGEN Sunday, August 14, 2011 A.Jackson Stenner Chairman & CEO, MetaMetrics Research Professor, UNC-Chapel Hill

2. The Lexile Framework is NOT: A reading program A test or method of assessment Educational software

3. What is The Lexile Framework for Reading? An educational tool – a scale - that links reading material and readers under a common metric – the Lexile. −Measures the complexity of reading material −Measures the reading level of individual readers

4. The Lexile Framework for Reading A student’s Lexile measure is her reading level. A book’s Lexile measure is its reading difficulty. an 1170L reader an 11 70L book A Lexile is a measure of both reading level and text complexity, on the same scale.

5. Why The Lexile Framework? LINK reader to text under a common measure FORECAST levels of comprehension TRACK growth over time

6. Where are Lexile Measures Found? All major norm-referenced tests are linked Over 30 million students received a Lexile measure last school year Over 450 publishers Nearly 115,000 book titles to search at www.lexile.comwww.lexile.com Over 400 million Lexiled articles

7. Where are Lexile Measures Found? Formative and Norm-Referenced Assessments

8. Where are Lexile Measures Found? Library Resources

9. The Lexile Scale How high does the Lexile scale go? How low does the Lexile scale go?

10. The Lexile Scale Lexile measures typically range from below 200L for beginning readers to above 1700L for advanced readers. Text below 200L represents beginning-reading material. A student’s Lexile measure of 0L or below is coded BR, signifying ‘Beginning Reading’.

11. Lexile Measures & Text Complexity “Quantitative dimensions of text complexity refer to those aspects of text complexity, such as word length or frequency, sentence length, and text cohesion, that are difficult if not impossible for a human reader to evaluate efficiently, especially in long texts, and are thus today typically measured by computer software.” -from the Common Core State Standards

12. Determining Text Complexity Sentence Length (syntactic complexity) Longer sentences are more complex and require more short-term memory to process.

13. Determining Text Complexity Vocabulary (semantic difficulty) Less familiar words impede reading fluency and affect comprehension.

14. Analyzing Text Readability Read and examine Camping is Fun. Easy or difficult to read? Why?

15. We went camping. On Monday, we put up the tent. The tent fell down! On Tuesday, we hiked. It rained and we got wet! On Wednesday, we fished. Uncle Pete caught a boot! On Thursday, we left food on the table. The raccoons ate it! On Friday, we cooked marshmallows. They fell in the fire! On Saturday, we heard a strange noise. We were too scared to sleep! On Sunday, we went home. "Did you like camping?" asked Uncle Pete. "Yes!" we said. "Camping is fun!" Camping is Fun (M. Garcia)

16. We went camping. On Monday, we put up the tent. The tent fell down! On Tuesday, we hiked. It rained and we got wet! On Wednesday, we fished. Uncle Pete caught a boot! On Thursday, we left food on the table. The raccoons ate it! On Friday, we cooked marshmallows. They fell in the fire! On Saturday, we heard a strange noise. We were too scared to sleep! On Sunday, we went home. "Did you like camping?" asked Uncle Pete. "Yes!" we said. "Camping is fun!" Camping is Fun (M. Garcia) 200L

17. Analyzing Text Readability Read and examine an excerpt of a speech from Lee Kuan Yew: The Separation of Singapore from Malaysia Difficult or easy to read? Why?

18. Speech from Lee Kuan Yew: Separation of Singapore from Malaysia "For me, it is a moment of anguish. All my life, my whole adult life, I believed in merger and unity of the two territories.... Now, I, Lee Kuan Yew, Prime Minister of Singapore, do hereby proclaim and declare on behalf of the people and the Government of Singapore that as from today, the ninth day of August in the year one thousand nine hundred and sixty-five, Singapore shall be forever a sovereign democratic and independent nation, founded upon the principles of liberty and justice and ever seeking the welfare and happiness of the people in a most and just equal society.“

19. Speech from Lee Kuan Yew: Separation of Singapore from Malaysia "For me, it is a moment of anguish. All my life, my whole adult life, I believed in merger and unity of the two territories.... Now, I, Lee Kuan Yew, Prime Minister of Singapore, do hereby proclaim and declare on behalf of the people and the Government of Singapore that as from today, the ninth day of August in the year one thousand nine hundred and sixty-five, Singapore shall be forever a sovereign democratic and independent nation, founded upon the principles of liberty and justice and ever seeking the welfare and happiness of the people in a most and just equal society.“ 1550L

20. Why 75% Comprehension? Research suggests that at 75%… A reader can have a successful reading experience without frustration or boredom A reader can achieve “functional comprehension” of the text A reader will be sufficiently challenged (by vocabulary and syntax) to improve 75% is the “right amount of challenge”

21. Managing Comprehension Readers may experience frustration when… −Text readability is 100L+ above their Lexile level. Readers may experience ease when… −Text readability is 50-100L below their Lexile level. Readers may experience growth when… −Text readability is within their Lexile range. General Reading Recommendation: Targeted text range of 100L below to 50L above the student’s Lexile level Note: This range may vary based on text type, reading context and purpose, reading strategies and support, and reader motivation.

22. Finding targeted text on ecosystems: Books: Lexile Measure: Inside Ecosystems and Biomes700L Exploring Ecosystems with Max Axiom, Super Scientist750L Earth’s Ecosystems IG810L Ecosystems 860L Populations and Ecosystems880L Biomes and Ecosystems970L The Eastern Forest1010L The Deserts of the Southwest1080L The Prairie1140L Endangered Species1160L

23. Time Reading and Reading Achievement from Anderson et al., 1988, Table 3, N = 155. Reading material included books, magazines, and newspapers. Fifth-Grade Students Percentile Rank Minutes of Text Reading per Day Estimated Number of Words Read per Year 98 90 70 50 20 10 67.3 33.3 16.9 9.2 2.4 1.0 4,733,000 2,357,000 1,168,000 601,000 134,000 51,000

25. College and Career Readiness Skills Reading Demand of Newspapers USA Today (1200L) Associated Press (1310L) Chicago Tribune (1310L) Wall Street Journal (1320L) Washington Post (1350L) NY Times (1380L) Reuters (1440L)

26. College and Career Readiness Skills Reading Demand of International Newspapers The Egyptian Gazette (Egypt) 1440L Oman Daily Observer (Oman) 1430L Financial Times (Great Britain) 1430L Straits Times (Singapore) 1410 L Gulf Times (Qatar) 1420L China Daily (China) 1400L France Daily (France) 1400L The Moscow Times (Russia) 1400L The Australian (Australia) 1390L German Times (Germany) 1390L Copenhagen Post (Denmark) 1390L Irish Times (Ireland) 1380L Santiago Times (Chile) 1380L Jerusalem Post (Israel) 1370L

27. FREE Lexile Tool s www.lexile.com Lexile Find a Book Lexile Analyzer

28. www.Lexile.com

29. Find-a-Book

30. Select reading interest:

31. Search results:

32. Create a reading list:

33. Edit reading list to personalize:

34. Search for books:

35. Search results:

36. Link to public library

38. Tools - Lexile Analyzer

39. The Lexile Analyzer

40. Copy the article

41. The Lexile Analyzer Paste the text into the Word document

42. The Lexile Analyzer Remove titles, pictures, graphs, illustrations, captions, sub-headings, and bylines

43. The Lexile Analyzer Be sure to remove hyper-links

44. The Lexile Analyzer Be sure to save as ‘Plain Text’

45. Oasis  A student-centered personalized learning platform –Does not require teacher direction  Differentiated activities  Embedded assessment in the background  Web-enabled with minimal bandwidth requirement  Built on the principles of deliberate practice…

46. Demo Accounts for Copenhagen  45 demo accounts have been created inside Oasis for the folks in Copenhagen.  All the usernames are: rasch# where the # is a number from 1 to 45 (eg., rasch1, rasch15, rasch42).  All the passwords are: jakob  Oasis can be accessed by browsing to: http://www.alearningoasis.com and clicking on the green “launch” button. If anyone has issues logging in, they can contact Sean Hanlon – shanlon@lexile.com http://www.alearningoasis.com

47. Reader Ability Temperature

48. Three well researched constructs  Reader ability  Text Complexity  Comprehension

49. Reading is a process in which information from the text and the knowledge possessed by the reader act together to produce meaning. Anderson, R.C., Hiebert, E.H., Scott, J.A., & Wilkinson, I.A.G. (1985) Becoming a nation of readers: The report of the Commission on Reading Urbana, IL: University of Illinois

50. An Equation = Reader Ability Text Complexity Comprehension - Conceptual Statistical Expected Raw Score = i e (RA – TC ) i 1 + e (RA – TC i ) RA = Reading Ability TC = Text Calibrations

51.  A causal model relating reader ability, text complexity, and comprehension.  Measures reader ability and text complexity, on a common scale—the Lexile scale.  Allows educators to forecast the level of success a reader is likely to experience with a particular text. The Lexile Framework for Reading

52. Eight Features of the Causal Model Relating Text Complexity, Reader Ability, Task Difficulty and Comprehension 1. The model is individual centered. The focus is on explaining variation within person over time. 2. In this framework the measurement mechanism is well specified and can be manipulated to produce predictable changes in measurement outcomes (e.g. percent correct). 3. Item parameters are supplied by substantive theory and, thus, person parameter estimates are generated without reference to or use of any data on other persons or populations. Therefore, effects of the examinee population have been completely eliminated from consideration in the estimation of person parameters for reader ability.

53. 4. The quantitivity hypothesis can be experimentally tested by evaluating the trade-off property for the individual case. A change in the person parameter can be off-set or traded-off for a compensating change in text complexity to hold comprehension constant. The trade-off is not just about the algebra. 5. When uncertainty in item difficulties is too large to ignore, individual item difficulties may be a poor choice to use as calibration parameters in causal models. As an alternative we recommend, when feasible, averaging over individual item difficulties to produce “ensemble” means. For example empirical text complexities can be excellent dependent variables for testing causal theories. Eight Features of the Causal Model cont’d.

54. 6. Causal Rasch models are individual centered and are explanatory at both within-subject and between-subject levels. The attribute on which I differ from myself a decade ago is the same attribute on which I differ from my brother today. 7. When data fit a Rasch model, differences between person measures are objective. When data fit a causal Rasch model absolute person measures (reader abilities) are objective (i.e. independent of instrument). 8. Causal Rasch models make possible the construction of generally objective growth trajectories. Each trajectory can be completely separated from the instruments used in its construction and from the performance of any other persons, whatsoever. Eight Features of the Causal Model cont’d.

55. May 2016 (12 th Grade) 1200 1000 1400 1600 Text Demands for College and Career May 2007 – April 2011 347 Encounters 138,695 Words 3,342 Items 983 Minutes Student 1528 7 th Grade Male Hispanic Paid Lunch Expected: 73.5% Observed: 71.7%

57. Theoretical versus Empirical Text Complexity for 719 Articles* Reliability = 0.997 SEM = 12.8L r = 0.968 r” = 0.969 R 2” = 0.938 RMSE” = 89.6L * Inclusion criteria: 50 encounters and 1,000 items

61. How Temperature and Pressure Relate Under Constant Volume TemperatureVolumePressure 2000°K20 Liters20.0 atm 1000°K20 Liters10.0 atm 500°K20 Liters5.0 atm 250°K20 Liters2.5 atm 125°K20 Liters1.25 atm

62. Comprehension Rates for Readers of the Same Ability with Texts of Different Complexity or How Reader Ability and Comprehension Rate Relate Under Varying Text Complexity Reader Ability Text Complexity Text Titles Comprehension Rates 1000L 500L 750L 1000L 1250L 1500L The Magic School Bus, Inside the Earth (Cole) The Martian Chronicles (Bradbury) The Reader’s Digest The Call of the Wild (London) On Equality Among Mankind (Rousseau) 96% 90% 75% 50% 25% Comprehension Rates for Fixed Reader Ability

63. Comprehension Rates for Readers of Different Ability with Texts of the Same Complexity or How Reader Ability and Comprehension Rate Relate Under Constant Text Complexity Reader AbilityClassroom TextbookComprehension Rates 500L 750L 1000L 1250L 1500L 1000L 25% 50% 75% 90% 96% Comprehension Rates for Fixed Text Complexity

64. How Many Ways Can We Say X Causes Y? X “elicited a greater” YX “impacts” Y X “accounts for” YX “has been linked to” Y Y “is the result of” XX “didn’t diminish” Y Y “because of” XY “depends on” X X “has led to” YX “largely motivates” Y Y “stemmed from” XX “proved critical to” Y X “fosters” YX “changes” Y X “triggers” YX “affects” Y

65. To causally explain a phenomenon [a measurement outcome] is to provide information about the factors [person processes and instrument mechanisms] on which it depends and to exhibit how it depends on those factors. This is exactly what the provision of counterfactual information…accomplishes: we see what factors some explanandum M [measurement outcome, raw score] depends on (and how it depends on those factors) when we have identified one or more variables such that changes in these (when produced by interventions) are associated with changes in M (Woodward, 2003, p.204).

66. Ensemble Calibration The Psychometric Engine of Oasis

67. Oasis Items  Single Instance  No individual item statistics  Theoretical estimates of text complexity become empirical estimates of text complexity  Protocol for determining response illustrations and foils  Machine-generated with no human review

68. Item Distributions  Items are presumed to be sampled from a distribution with a mean difficulty and an imputed spread  This distribution is called an ensemble

69. Traditional Rasch Algorithm  Iterative process  Two stages of iteration –Begin with asssumed starting values for person abilities and item difficulties (usually item difficulties) –Estimate person abilities given item difficulties. –Fix these estimated person abilities and estimate new item difficulties given these person abilities –Iterate until a stopping criterion is met

70. Ensemble Calibration Algorithm  Same iterative technique as traditional method –Fix person abilities to determine ensemble means –Fix resulting ensemble means to determine person abilities –Iterate until stopping criterion is met

71. What is Different?  Individual item difficulties neither available nor needed  Start iterative procedure with person abilities from Oasis  P-delta-sigma table to account for spread of distributions  Spread of ensemble effects success rate

72. P-delta-sigma table  A success rate (p) is determined by the difference in the ensemble mean for a distribution and the person ability (delta) and the spread of the ensemble (sigma)  Normal distribution assumption for the ensemble of item difficulties  Table used instead of integration over the random term for programming simplicity –Dimensions: (delta -1000 to +1000 by fives, Sigma 10 to 260 by fives)

73. Formula for p-delta-sigma p-delta-sigma = Δ=delta=difference between person ability and ensemble calibration σ=sigma=standard deviation of the ensemble u=variable of integration=random difference of item difficulty from ensemble mean

74. Spread of Ensemble  Ensembles vary in mean but not in SD  SD is uniform for this implementation  With embellishment a method for unequal variances could be derived

75. Closing No matter how it is sliced and diced, analyses of joint and conditional probability distributions yield no more than patterns of association. Nothing in the response data nor Rasch analyses of these data exposes the processes (features of the object of measurement) or mechanisms (features of the instrument) that are hypothesized to be conjointly causal on the measurement outcomes.

76. A. Jackson Stenner CEO, MetaMetrics jstenner@Lexile.com Contact Info: