1. Learning Progressions: Some Thoughts About What we do With and About Them Jim Pellegrino University of Illinois at Chicago

2. Learning is facilitated when new and existing knowledge is structured around big ideas or a conceptual framework rather than small, discrete bits of information. Learning develops as a continuous process with an individual continuously making links back and forth among ideas and not in linear, discrete steps. Learning difficult ideas takes time and often comes together as students work on a task that forces them to synthesize ideas. Yet, K – 12 science curricula are generally not structured to build and cycle back on ideas. Why Learning Progressions?

3. CurriculumInstruction Assessment Domain-Based Models of Learning & Understanding Importance of Domain-Based Models

4. CurriculumInstruction Domain-Based Models of Learning & Understanding Assessment ObservationsInterpretation Connecting Assessment to Curriculum & Instruction

5. A ssessment as a Process of Reasoning from Evidence cognition –model of how students represent knowledge & develop competence in the domain observations –tasks or situations that allow one to observe students’ performance interpretation –method for making sense of the data observationinterpretation cognition Must be coordinated!

6. Guiding Principles All models are wrong, some are useful »George Box Better an approximate answer to the right question than an accurate answer to the wrong question »John Tukey

7. Assessment Design Principles Assessment design should always be based upon a model of student learning and a clear sense of the inferences about student competence that are desired for the particular context of use. The model of student learning suggests the most important aspects of comptence that one would want to make inferences about and provides clues about the types of tasks that will elicit evidence to support those inferences.

8. Aspects of Student Models Domain specific and empirically based Identifies cognitive performances that differentiate expert and novice learners Lays out one or more typical progressions toward competence including milestones or landmark performances along the way. Can be at various levels of detail; grain size depends on assessment purpose

9. Contrasting & Complementary Perspectives on “Cognition” Targets for Explanation (& Assessment) Rationalist Perspective Sociocultural Perspective Nature of Performance Underlying Competencies Communal Practices Nature of Development Trajectories of Learning Trajectories of Participation Nature of Knowledge Mental Representations Forms of Mediated Activity

10. Assessment Learning Progression Observations Interpretation Model Assessment as a Process of Reasoning from Evidence

11. Interpretation -- Making Sense of the Observational Data Some of the ways in which individuals typically try to make sense of the observational data –Intuition -- gut reaction –Counting number correct (4/8); generating percent correct –Assigning points to answers or assigning letter grades Going deeper -- getting at student strengths & weaknesses –Focus on the nature of student thinking, including misunderstandings, not just correct responses –Create interpretive scales and rubrics that provide detail about multiple features of student competence Assessment isn’t just about assigning scores –It’s a meaning making process

12. Description of successively more sophisticated ways of thinking about a big idea Provide a framework for long-term development –Describes what it means to move towards more expert understanding in an area –Gauge increasing competence over time –A sequence of successively more complex ways of thinking about how an idea develops over time Consider how ideas build upon each other to form more complex practices or ideas Learning Progressions

13. The Researcher’s Model of Development/Change/Progress A central feature of all this work resides in the researcher’s beliefs about the nature of individual development or change -- IN ONE OR MORE DOMAINS!!!!; These beliefs affect the researcher’s choice of theoretical framing, choice of data, choice of measure, and choice of analysis tools; Choices interact with each other affecting the way claims are warranted and inferences are drawn;

14. Questions to Help Unpack Likely Assumptions Question 1: –Has change been framed as systematic or random? –All assessments have error. How does your model of change adjust for this error, or does it?

15. Questions to Help Unpack Likely Assumptions Question 2: Is development reversible? –Implications for the form of the measurement; –Implications for the functional form of the developmental trajectories; –The shape of the trajectory should be hypothesized before data collection; –The shape can be empirically tested;

16. Questions to Help Unpack Likely Assumptions Question 3: Is the change unitary or multi- path? Your theory of learning should be able to identify and describe why certain groups of students follow different growth trajectories;

17. Questions to Help Unpack Likely Assumptions Question 4: Can we consider change to: –be a continuous, gradual, quantitative phenomenon? –have large magnitude shifts on a quantitative variable? –Be a progression through as series of qualitatively distinct stages?

18. Questions to Help Unpack Likely Assumptions Question 5: Is change considered as differences in magnitude in: –an absolute sense? –calibration? –conceptualization?

19. Questions to Help Unpack Likely Assumptions Questions 6: Is change considered a shared characteristic of a group of individuals over time? –Is it what occurs within individuals over time? –Or both? –Keep in mind that both kinds of change can occur simultaneously!

20. Questions to Help Unpack Likely Assumptions Question 7: If we assume that all individuals have trajectories of the same functional form are there systematic inter-individual differences in the values of the individual growth parameters? –What would this mean theoretically?

21. Questions to Help Unpack Likely Assumptions Question 8: Are there cross domain relationships in change over time? –Is the relationship between inter-individual differences in intra-individual change over time (and the predictors of those differences) invariant across learning domains?

22. Questions to Help Unpack Likely Assumptions Question 9: Finally, is there invariance across groups with respect to the facet of change over time under investigation? –Put simply, are change patterns at the group level constant or invariant across groups?

23. Ravit’s Questions ++ Nature of progression: –Path/ paths/ landscapes? –Nature of movement -cycles, multiple states –Context dependence Nature of learning performances: –Integrate big ideas and practices –Quantifiable variables that measure learning outcomes Nature of evidence: –Can we really rely on short terms studies, will we (and if so when) need to actually follow student learning over grades? –Wont instruction fundamentally change what students can do, and therefore the progression Challenges for teaching (instructional practice) Challenges for assessment Challenges for curriculum design