1. Toward a Theory Relating Text Complexity, Reader Ability and Comprehension AERA New Orleans April 10, 2011
Jackson Stenner
Chairman & CEO, MetaMetrics

2. Three well researched constructs
Reader ability
Text Complexity
Comprehension
3 well researched variables
Let’s first put them together in a definition of reading, then put them in an equation.
Our focus today will be on text complexity but it is one of three variables that cooperate in a causal model.

3. Reading is a process in which information from the text and the knowledge possessed by the reader act together to produce meaning.
50 years later we revisit one of Rasch’s favorite constructs
Let’s put it all together – text, reader and comprehension
Ternary Structure (temperature, pressure, volume) (force, mass, acceleration)
Making meaning is comprehension
Let’s build a foundation for the science underlying The Lexile Framework.
Let’s start with a beautiful definition
Text and Reader act together to make meaning
How do we operationalize “act together”
The Lexile Framework uses an equation that specifies how reader measure and text measures cooperate or “act together” to produce comprehension
We use this equation to make text measures and reader measures.
Anderson, R.C., Hiebert, E.H., Scott, J.A., & Wilkinson, I.A.G. (1985)
Becoming a nation of readers: The report of the Commission on Reading
Urbana, IL: University of Illinois

4. An Equation e Conceptual - = = 1 + e Text Reader Complexity Ability
Comprehension
Statistical e
(RA – TC )
Raw
Score i =
1 + e
(RA – TC i ) i
Operationalizes the definition *Imagining a text to be a test. Key innovation.
Three inter-related but distinct constructs
The familiar Rasch model with one important addition. Item calibrations come from theory.
TCi’ s are either empirical or theoretical item calibrations (when making reader measures) or virtual item calibrations (when making text measures).
Same equation is used to make reader measures and text measures!!!
Major criticism is that this formulation is “just too simple.”
We have amassed a staggering array of successful predictions using this “too simple” model.
RA = Reading Ability
TC = Text Calibrations 4

5. Eight Features of the Causal Model Relating Text Complexity, Reader Ability and Comprehension
The model is individual centered. The focus is on explaining variation within persons over time.
In this framework the measurement mechanism is well specified and can be manipulated to produce predictable changes in measurement outcomes (e.g. percent correct).
Item parameters are supplied by substantive theory and, thus, person parameter estimates are generated without reference to or use of any data on other persons or populations. Therefore, effects of the examinee population have been completely eliminated from consideration in the estimation of person parameters for reader ability.
Don’t need metaphysical baggage – I mean X causes Y if intervention on X produces predicable changes in Y.
The trade-off
Revisit equation slide. Explain intervention on RA-TC and nextemp

6. Eight Features of the Causal Model cont’d.
The quantitivity hypothesis can be experimentally tested by evaluating the trade-off property for the individual case. A change in the person parameter can be off-set or traded-off for a compensating change in text complexity to hold comprehension constant. The trade-off is not just about the algebra.
When uncertainty in item difficulties is too large to ignore, individual item difficulties may be a poor choice to use as calibration parameters in causal models. As an alternative we recommend, when feasible, averaging over individual item difficulties to produce “ensemble” means. For example empirical text complexities can be excellent dependent variables for testing causal theories.

7. Eight Features of the Causal Model cont’d.
Causal Rasch models are individual centered and are explanatory at both within-subject and between-subject levels. The attribute on which I differ from myself a decade ago is the same attribute on which I differ from my brother today.
When data fit a Rasch model, differences between person measures are objective. When data fit a causal Rasch model absolute person measures (reader abilities) are objective (i.e. independent of instrument).
Causal Rasch models make possible the construction of generally objective growth trajectories. Each trajectory can be completely separated from the instruments used in its construction and from the performance of any other persons, whatsoever.

8. Text Demands for College and Career
Student 1528
6th Grade Male Hispanic Paid Lunch
May 2007 – Dec. 2009
284 Encounters 117,484 Words 2,894 Items 848 Minutes
Text Demands for College and Career
1200
1000
1400
1600
May 2016 (12th Grade) 8

9. Mythology Text Complexity Theoretical: 1300L Empirical: 1357L
Adapted from Oasis
Article courtesy of EBSCO Publishing
The study and interpretation of myth and the body of myths of a particular culture. Myth is a complex cultural phenomenon that can be approached from a number of viewpoints. As generally understood, a myth is a story or narrative that is traditional in a certain culture, having been passed down from early times and regarded as true. It may be said to 1 symbolically the origin of the basic elements and assumptions of a culture. Mythic narratives frequently revolve around the doings of gods or heroes, and may relate, for example, how the world began, how humans and animals came into being, or how certain customs, gestures, or forms of human activities 2. Almost all cultures possess or at one time possessed and lived in terms of myths.
immerse belittle portray contradict
2 originated adorned handicapped entwined
9999

10. Plot of Theoretical Text Complexity versus Empirical Text Complexity
for 446 passages

11. What could account for the 10% unexplained variance?
Missing Variables or Theory misspecification
Better Criterion Variable
Improved Proxies/Operationalizations
Expanded Error Model – Treat Item Type as Random
Rounding Error
Imperfections in Theory Implementation
Sarah Kershaw’s (FCRR) dissertation will check the most promising text variables not in the LF equation.
Very large text corpora (Google 500 billion) will permit very accurate frequency measures for 10,000s of words and word families. Also empirical Lexile word difficulties will be available in the next five years.
The task continuum promises new machine generated task types for measuring text complexity and reader ability.
MM is bringing on-line a new analyzer that does not round text measures.
Theory implementation compromises. Front to back vs. back to front text measurement gives us a way to estimate effects of theory implementation.

12. Closing
No matter how it is sliced and diced, analyses of joint and conditional probability distributions yield no more than patterns of association. Nothing in the response data nor Rasch analyses of these data exposes the processes (features of the object of measurement) or mechanisms (features of the instrument) that are hypothesized to be conjointly causal on the measurement outcomes.
In my view the agenda for a sixth decade of Rasch measurement practice should have at the top a focus on Causal Rasch Models.

13. Contact Info:
A. Jackson Stenner
CEO, MetaMetrics