1. College Readiness and Reading Proficiency: Using Lexiles to Investigate Reading Gaps in Texas
Commission for a
College Ready Texas
Houston - June 18, 2007
Thank you Dean, both a pleasure and an honor to participate in the important work of this Commission and to be here today…..
Presented by:
Eric Rolfhus, Ph.D.
Senior Researcher
Edvance Research, Inc.
Regional Educational Laboratory - Southwest
June 18, 2007

2. Why Lexiles?
As requested by the Commission as part of ELAR gap analysis:
Identifying gaps between TEKS and
“Reading demands associated with Lexile analysis and measures of text complexity and comprehensibility”
A Lexile is a widely used measure of Reading Comprehension Difficulty
MetaMetrics has conducted research on different populations of texts
Why are we talking about Lexiles today?
This Commission has requested an ELAR gap analysis from REL Southwest in preparation for the September report.
As part of this effort we’ve been requested to examine
“Reading demands associated with Lexile analysis and measures of text complexity and comprehensibility”
Lexiles are measures of Reading Comprehension levels derived from the characteristics of the text, and thus are independent from typical ‘expert evaluations’ – in Education research (as in other sciences) multiple methods are important to establish the validity of findings.
Reading obviously is a skill critical to all else – Reading First!
The Lexile Scale was developed by a private company (MetaMetrics), and is widely used by major test and book publishers.
The ultimate purpose of the scale is to faciliate match of readers to text, to optimize both learning and reading skill acquisition.
MetaMetrics has sample populations of texts from high-school, community college, and univerisity level, which provide independent estimates of skills needed in these environments.
Lexiles provide a uniques way to look at Curriculum Standards and Performance Criteria as they are independent estimates of required skills, using a different methodology.
Must introduce the scale before getting to the meat of it.
We must understand this new thermometer before using it.
Also Quantiles which are for another time
June 18, 2007
Edvance Research, Inc.

3. TAKS performance equated to Lexiles
Lexiles available on student reports
Lexiles available from most test publishers
Lexiles available for many textbook and curriculum materials
Lexile levels are reported on parent reports, are available from most test publishers, and many textbook and curriculum publishers.
The Lexile scale represents an independent yardstick, where materials from various sources can be placed on the same universal scale
What allows today’s talk, is the fact that the TAKS test has been equated to the Lexile scale for each grade.
Equating tables are publically available for the 2005 TAKS administration, so we’ll be discussing 2005 data this morning.
(unclear whether this is the only equating and we are supposed to accept this at face value for the subsequent years or not)
Equating = This is similar to lining up the Fahrenheit and Celsius scales, though a lot less precise
A quick display of what this means….Raw Total TAKS scores have a TAKS scaled score assigned, these TAKS scaled scores have been equated to the Lexile scale…(this is often done by)…this allows us to compare things in TAKS-world to things in Lexile world.
(from TEA;
June 18, 2007
Edvance Research, Inc.

4. How is a Lexile Calculated?
A regression equation was developed based on thousands of texts.
Think FICO (credit score) formula
(credit risk prediction from an equation)
Two factors included in the Lexile formula
Word Frequency
Length of Sentence
There is a lot of Psychometrics behind the Lexile Scale which we will gloss over today...a shame because Item-Response-Theory and correlations are beautiful things….you haven’t lived until you’ve spent Happy Hour with a bunch of statisticians…
Indices of Reading Comprehension, based on characteristics of the text, have been around since the 1930’s.
These have almost always required individuals to count/tally multiple dimension of the text (# of dependent clauses, number of syllables per sentence and so on)
Many many factors have been evaluated in constructing these indices, but almost all have included some variant of Word Freuency and Sentence Length as predictors of how difficult it is to comprehend a particular text.
Think of a regression equation with two predictors; the average of the frequency of each word in a text, and the average sentence length.
For example…the word “the” vs. “antidisestablishmentarianism” correlation between conceptual-level and commonality of word
These two factors can explain most of comprehension difficulty inherent in a text
Let’s go through some examples…simply explaining the thermometer is not enough, you need to experience it.
June 18, 2007
Edvance Research, Inc.

5. (from MetaMetrics, 2007 & Williamson, 2006a)
Workplace Lexiles
(from MetaMetrics, 2007 & Williamson, 2006a)
200 = 1st grade
1123 = 11-12th grade texts
1355 = 1st Year College texts
Forklift Operations Manual
Adobe PhotoShop 5.0 Tutorial 1080
Fine Homebuilding Magazine 1150
2008 Cadillac Eldorado Brochure 1150
Attendance Policy for Hourly Employees 1210
Federal Form W
Free Application for Federal Student Aid 1270
Aetna U.S. Healthcare - Open Choice PPO 1280
U.S. Dept. of Justice (INS)
- Employment Eligibility Verification 1340
Just as the Celsius scale is anchored at 0 degrees for freezing, and 100 degrees for boiling.
The Lexile scale originally had two anchors (200 for first grade primers, and 1200 for Groliers encyclopedia)
The numbers at the top of this slide represent some data we’ll look at in a minute.
Notice the levels we’re concerned with here….11-12th grade textbooks, and 1st/2nd year college texts….
June 18, 2007
Edvance Research, Inc.

6. (from MetaMetrics, 2007 & Williamson 2006a)
Newspaper Lexiles
(from MetaMetrics, 2007 & Williamson 2006a)
Miami Herald 1200
USA Today 1200
Chicago Sun-Times 1210
New York Daily News 1230
San Francisco Examiner 1230
San Jose Mercury 1230
New York Post 1280
Washington Times 1290
Associated Press 1310
Chicago Tribune 1310
Wall Street Journal 1320
Los Angeles Times 1330
Washington Post 1350
UPI
Bloomberg 1380
New York Times 1380
Reuters
200 = 1st grade
1123 = 11-12th grade texts
1355 = 1st Year College texts
On this slide…the same anchor points….and these examples….
June 18, 2007
Edvance Research, Inc.

7. Comparing Text Difficulty and Person Ability
Expected
Comprehension
Rate
Person
Ability
Text
Difficulty
1250
1250
25%
1000
1000
50%
750
750
75%
Both individuals and texts are assigned Lexile scores, placed on the same Yardstick…..
The scale is established to that if the Reader and the text share the same Lexile score, the Reading Comprehension rate is predicted to be 75%...this will vary of course for individual readers and texts….but is the best estimate…and what should be observed in a large sample of individuals and texts….
If the text is at a higher Lexile level than the reader, the comprehension rate is predicted to be lower.
If the text is at an easier point on the scale, the comprehension rate is predicted to be higher.
500
500
90%
250
250
95%
June 18, 2007
Edvance Research, Inc.

8. What does Comprehension-Rate mean?
This rose-bush, by a strange chance, has been kept alive in history; but whether it had merely survived out of the stern old wilderness, so long after the fall of the gigantic pines and oaks that originally overshadowed it, or whether, as there is fair authority for believing, it had sprung up under the footsteps of the sainted Ann Hutchinson as she entered the prison-door, we shall not take upon us to determine.
- The Scarlet Letter (1400L)
How is comprehension measured?
From a theoretical perspective… Imagine slicing a book into 250 word sections and asking a comprehension question about each section.
At a 75% comprehension level, 75% of these questions would be answered correctly.
This single sentence from The Scarlet Letter illustrates a number of Lexile issues. It has a very high Lexile value of 1400.
It has 73 words, 8 commas/semi-colons. It clearly illustrates that sentence length is a good surrogate for complexity.
Cloze item:
The rose-bush serves as a vibrant ______ for this tale.
June 18, 2007
Edvance Research, Inc.

9. Lexiles Chart – Handout
Another way to understand the scale can be found at the back of your handout….
June 18, 2007
Edvance Research, Inc.

10. Estimates of Reading Demands by Setting
(from Williamson, 2006a)
Percentile 5 25 50 75 95
To understand the comparison of Lexile difficulties between the different types of text…
This is a box-and-whisker plot
The number of different books sampled….
This is a distribution of difficulties….
Number of
texts used in
this estimate
June 18, 2007
Edvance Research, Inc.

11. Reading Demands in Secondary
And Beyond
(from Williamson, 2006a)
Self-explanatory
June 18, 2007
Edvance Research, Inc.

12. TAKS Performance Standards Overlay on Text Demands
(adapted from Williamson, 2006a)
Higher Ed:
2200 TAKS
1170 Lexile
Standard:
2100 TAKS
1015 Lexile
Commended:
2400 TAKS
1490 Lexile
On this slide we will look at the text demands from these different environments in relation to the TAKS performance standards
Your handout is a very busy…on screen we’ll walk through one piece at a time….
Note first that this slide contains additional types of text…Military, Citizenship, Workplace
There is a wide variety workplace materials depending on occupations
A quick review of TAKS Standard-setting
There are many approaches to Standard-Setting: Angoff, item-mapping approach….
Committees of experts, individually, take items for each grade that have been ordered by difficulty.
With definition of basic/Standard performance levels in mind, individual raters identified the point in the item-order
This was done for the Commended-level as well….
The Higher-Ed Readiness level was established based on research
extreme groups study
correlational study….TAKS – ACT Reading tests only correlated about .60….
Basically this standard lines up with a 50% probability of achieving a 20 on the ACT or a 500 on the SAT
Meeting this standard allows a student to bypass taking a Texas Success Initiative test….
ACT of 20 is considered basic complexity...
June 18, 2007 7

13. TAKS-Lexile Midpoints By Grade
(ELA - April 2005 Administration)
TAKS ELA
Commended
(1490 L)
TAKS ELA
Higher Education
Readiness (1170 L)
TAKS ELA Standard
Performance
Level (1015 L)
Lexile
Cross-sectional from grades 3-11 (exit-level)
Standard Performance…about 7th grade!
Higher Ed Standard…about 9th grade
Commended Performance Level truly maps to college-ready reading
Grade
June 18, 2007
Edvance Research, Inc. 8

14. April 2005 TAKS ELA Scaled Scores
with Lexile and Performance Standard Overlay (N=230,147)
Higher-Ed
Readiness+
71%
Standard+
85%
Commended+
16%
Text for Grades 11-12
76%
# of students at
TAKS score Level
Community College
41%
1st/2nd Year of College
35.5%
This slide is best experienced a piece at a time
TAKS Scaled Score
June 18, 2007
Edvance Research, Inc. 9

15. Conclusions
This gap (HS  4-yr College) appears to be built-in to typical high- school reading levels
University-level reading demands fall between TAKS Commended and Higher Education Readiness performance levels
TAKS Commended performance level lines up to about 75th percentile of university reading demands.
Only about 36% of those completing the TAKS in April 2005 are reading at a Lexile level that matches demands of college
(at a 75% comprehension level)
June 18, 2007
Edvance Research, Inc.

16. Examine Student Longitudinal Lexiles
Further Research
Examine Student Longitudinal Lexiles
(from Williamson, 2006b)
Actual Data
June 18, 2007
Edvance Research, Inc.

17. (Comparing NC longitudinal to TX cross-sectional)
Longitudinal Lexiles
(Comparing NC longitudinal to TX cross-sectional)
1st/2nd Year
College
Community
NC, TX
NAEP 2005
Results
4th/8th Grade
Nearly Identical
Lexile
Grade
June 18, 2007
Edvance Research, Inc.

18. References
Willamson, G. L. (2006a). Student Readiness for Postsecondary Endeavors. Paper presented at the Annual Meeting of the American Educational Research Association (AERA), San Francisco, CA.
Willamson, G. L. (2006b). Aligning the Journey with the Destination. Durham, NC: MetaMetrics, Inc.
June 18, 2007
Edvance Research, Inc.