1. Dr. Marilyn P. Carlson Research Based Tools and Insights for Improving Mathematics and Science Teachers’ Classroom Practices: Dr. Marilyn P. Carlson Arizona State University Principal Investigator: Project Pathways Professor, Department of Mathematics and Statistics Director, CRESMET Center for Research on Education in Science, Mathematics, Engineering, and Technology This work was supported, in part, by grant no. 9876127 from the National Science Foundation CRESMET

2. Project Pathways Partnership of ASU and five school districts ( ASU: Mathematicians, scientists, engineers, math and science educators, professional development experts) Primary Goal: To produce a research-developed, refined & tested model of inservice professional development for secondary mathematics and science teachers. Core Strategies: Four integrated math/science graduate courses + linked teacher professional learning communities (lesson study approach) Supporting Strategies: Motivational activities for students Tools for ELL students Information & networking for guidance counselors

3. Initial Assumptions: Many teachers have weak understanding of fundamental concepts they are expected to teach Teachers hold naïve beliefs about teaching and learning Teachers have little time to reflect on and modify their teaching practices Math and science teachers need support in learning to connect STEM concepts across disciplinary boundaries Short term interventions have little impact on teaching practice

4. Pathways Objectives for Teachers Deepen teachers’ understanding of foundational mathematics & science concepts and their connections (function, rate-of-change, covariation, force, pressure) Improve teachers’ reasoning abilities and STEM habits of mind (problem solving, scientific inquiry, engineering design) Support teachers in adopting “expert” beliefs about STEM learning, STEM teaching, and STEM methods (See STEM BILT Taxonomy) Support teachers in reflecting on and modifying their classroom instruction

5. Pathways Objectives for Students Increase student persistence in challenging course-taking Improve problem solving, scientific inquiry, engineering design practices Improve students’ understanding of foundational concepts and reasoning abilities that have been documented to be essential for continued STEM course taking Improve confidence and interest in math & science

6. Role of Frameworks--Linking Intervention and Assessment Drive curriculum development for all four courses and learning communities Guide classroom structure and practices Provides a lens for analyzing qualitative data Frames assessment tool development Analysis of qualitative data has resulted in development of new frameworks and adaptations of existing frameworks

7. The Reflexive Relationship Between Frameworks and Intervention One Example Carlson & Bloom Problem solving framework -Describes effective mathematical practices Conceptual Frameworks (e.g.,Function Taxonomy) -Characterizes understanding (e.g., reasoning abilities, connections, notational issues) Theoretical grounding for -Designing curricular modules -Determining course structure -Instrument development Lens for researching emerging practices and understandings Revise Cognitive Frameworks Design (Revise) Practices & Instruments

8. Some Insights About Effective Problem Solving Practices: The Problem Solving Cycle

9. Planning Phase  Behavior ResourcesHeuristicsAffectMonitoring Orienting  Sense making  Organizing  Constructing Mathematical concepts, facts and algorithms were accessed when attempting to make sense of the problem. The solver also scanned her/his knowledge base to categorize the problem. The solver often drew pictures, labeled unknowns, and classified the problem. (Solvers were sometimes observed saying, “this is an X kind of problem.”) Motivation to make sense of the problem was influenced by their strong curiosity and high interest. High confidence was consistently exhibited, as was strong mathematical integrity. Self-talk and reflective behaviors helped to keep their minds engaged. The solvers were observed asking “What does this mean?”; “How should I represent this?”; “What does that look like?”  Conjecturing  Imagining  Evaluating Conceptual knowledge and facts were accessed to construct conjectures and make informed decisions about strategies and approaches. Specific computational heuristics and geometric relationships were accessed and considered when determining a solution approach. Beliefs about the methods of mathematics and one’s abilities influenced the conjectures and decisions. Signs of intimacy, anxiety, and frustration were also displayed. Solvers reflected on the effectiveness of their strategies and plans. They frequently asked themselves questions such as, “Will this take me where I want to go?” and “How efficient will Approach X be?” Executing  Computing  Constructing Conceptual knowledge, facts, and algorithms were accessed when executing, computing, and constructing. Without conceptual knowledge, monitoring of constructions was misguided. Fluency with a wide repertoire of heuristics, algorithms, and computational approaches were needed for the efficient execution of a solution. Intimacy with the problem, integrity in constructions, frustration, joy, defense mechanisms, and concern for aesthetic solutions emerged in the context of constructing and computing. Conceptual understandings and numerical intuitions were employed to reflect on the reasonableness of the solution progress and products when constructing solution statements. Checking  Verifying  Decision making Resources, including well- connected conceptual knowledge, informed the solver as to the reasonableness or correctness of the solution attained. Computational and algorithmic shortcuts were used to verify the correctness of the answers and to ascertain the reasonableness of the computations. As with the other phases, many affective behaviors were displayed. It is at this phase that frustration sometimes overwhelmed the solver. Reflections on the efficiency, correctness, and aesthetic quality of the solution provided useful feedback to the solver. The Multidimensional Problem Solving Framework: A Characterization of Effective Problem Solving Practices  Assist teachers in learning to promote these practices in their students  Respect the process of change

10. STEM Beliefs Instrument about Learning and Teaching Mathematics (Instrument Taxonomy) Confidence (a)Teachers Confidence in their mathematical ability (b)Teachers confidence in their scientific reasoning ability (c)Teachers confidence in their pedagogical ability Methods of Mathematics and Science (a)Process of learning mathematics and science (b)Learnability of mathematics and science (c)Authority in learning mathematics and science III.Methods of Teaching Mathematics and Science (a)Instructional goals (b)The role of homework and exams (c)Attention to the learning process (d)Teaching Constraints (Factors of Resistance) (e)Instructional Approaches/Teacher Classroom Practices

11. Development of tools to assess teachers’ cognitive development, beliefs, values and classroom practices (e.g., Beliefs About Learning and Teaching Mathematics Taxonomy Understanding (Learning Dimension)  Understanding mathematics requires special abilities that only some people have.  Understanding mathematics requires the individual to engage in sense making to construct meaning. Authority (Methods of Learning Mathematics) Authority in learning mathematics and science  I try to get my students to rely on their own reasoning and logic when verifying the correctness of their solutions  It is important that students look to their teacher verify the correctness of their solutions.

12. (Teaching Dimension) Instructional goals  The primary goal of my instruction is to teach facts, skills/procedures that my students may need to score well on an exam.  The primary goal of my instruction is to promote deep and coherent understanding of central concepts The role of homework and exams  The primary goal of my exams is to assess if my students can memorize facts and carry out procedures. Attention to the learning process  It is important to understand what a student is thinking when s/he asks a question Teaching Constraints (Factors of Resistance)  The pace at which I must cover material does not allow me to teach ideas deeply  My school administrators value my efforts to get students to understand ideas deeply.  I am able to adapt my pacing so I can spend instructional time on central concepts of my courses

13. Discuss (with members of your partnership) the three major beliefs that your project is trying to change in either teachers or students. Write one likert style item for each belief that you identified.

14. Other Findings that are guiding Pathways Interventions  Dispositions that supported effective cognitive and metacognitive behaviors Expectation of Meaning Making  Persistence in sense making, initiative in stringing together a logical argument Mathematical Integrity (intellectual honesty)  All conjectures were based on a logical foundation  They did not pretend to know they didn’t

15. How these findings are informing the Pathways Project Informing the nature of our classroom and PLC curriculum  Decision to Establish Classroom Rules of Engagement  Individuals are engaged in meaning making through various instructional modes Allow time for individual construction through various instructional modes All individuals are held accountable for expressing meaning.

16. Classroom Rules of Engagement are Negotiated and Enforced by All Members of the Classroom Community Individuals are expected to: Persist in sense making Speak meaningfully Attempt to make logical connections Exhibit Mathematical Integrity  Don’t pretend to understand when you don’t  Base conjectures on a logical foundation Support peers in making their own constructions

17. Special Role of Instructor for Graduate Course and Facilitator of Professional Learning Community  Intervene as appropriate to reinforce classroom rules of engagement  Model effective mathematical practices  Promote opportunities for teachers to develop and value effective classroom practices  Assist teachers in learning to promote these practices in their students  Respect the process of change

18. Other Instruments for Assessing Pathways Progress and Effectiveness PCA (Precalculus Concept Assessment) Instrument  Previously developed and validated  Modified late in year 1 VAMS (Views About Mathematics Survey; previously developed)  STEM BILT (STEM Beliefs Instrument About Learning and Teaching); developed--early in the validation process RTOP (Reformed Teacher Observation Protocol)  Already validated and published Learning Community Observation Protocol (LCOP)  Early draft is available; still collecting qualitative data to identify critical variables

19. The Role of Concept Assessment Instruments in Evaluation Provides coherent assessment of central ideas and what is involved in understanding those ideas Provides valid and reliable assessment of student understanding Easy to administer to large populations Can assess course and instructional effectiveness May serve as a tool to assess readiness for a course

20. Limitations of Concept Assessment Instruments Do not reveal thinking of individual students Do not reveal insights into the process of learning Are not tied to specific instruction or course materials Limited in assessing creative abilities

21. Closing Remarks Use of individual cognitive models of learning or knowing, such as the Multidimensional Problem Solving Framework increase the purposefulness of instructional materials, instructional actions, and serve as guiding frameworks for instrument development.