Presentation on theme: "Dr. Robert Curtis, Director, IMSS Dr. Susan Tucker, Evaluation & Development Associates LLC Defining Effective STEM Teaching and Learning Through a Logic."— Presentation transcript:
Dr. Robert Curtis, Director, IMSS Dr. Susan Tucker, Evaluation & Development Associates LLC Defining Effective STEM Teaching and Learning Through a Logic Model Cycle
Agenda ① Setting the Context ② Logic Model Theories of Action ③ Logic Models Applied to MSP work ④ Emerging Questions and Learnings about Logic Models for our MSP
Integrated Middle School Science Project Goals ① Increase quality, quantity, and diversity of middle school science teachers by providing PD and coaching support so teachers improve science content knowledge, pedagogy, inquiry, and leadership ② Create online PD anchored in inquiry-based science instructional aligned with the Foundational Level General Science Credential, state and national science standards, & lesson study ③ Develop partnership infrastructure to support systemic change through a coherent focus on building science instructional leadership capacity.
Definition of Effective STEM Teaching integrates science and engineering practices, cross cutting ideas and content knowledge. focus and support inquiries and science practices while interacting with students orchestrate discourse among students about scientific ideas challenge students to accept and share responsibility for their own learning recognize and respond to student diversity and encourage all students to participate fully in science learning
STEM Learning: Scientific and Engineering Practices (NRC Framework for K-12 Science Education) 1. Asking questions (for science) and defining problems (for engineering) 2. Developing and using models 3. Planning and carrying out investigations 4. Analyzing and interpreting data
Scientific and Engineering Practices 5. Using mathematics, information and computer technology, and computational thinking 6. Constructing explanations (for science) and designing solutions (for engineering) 7. Engaging in argument from evidence 8. Obtaining, evaluating, and communicating information
Logic Model Theory of Action The logic model can serve as the basis for program evaluation, research and project management plans, involving stakeholders in logic modeling process so that buy-in and “ownership” occurs, which is central to the success of the subsequent evaluation and research efforts and use of results ( Torres, Hopson & Casey, 2008 )
Logic Model Uses & Benefits Facilitates conversation to negotiate common language & goals Represent program and theories graphically Align with budget to articulate & review priorities Shows how learning, action, & outcomes related Delineates & aligns indicators of achievement Enables a common framework and road map
Think-Pair-Share: Expectations Have you had experience using LM? If so, how have you used them? At this point, what do you want to get out of this session?
IMSS examples of LM Define effective STEM teaching and learning Refine and evaluate project goals outcomes and activities across various project components Inform and coordinate program management, research and evaluation Increase coherence across all partners Build capacities of all partners for inquiry-based teaching and learning and evaluation
Logic Model as Road Map Where are we going? How will you get there? What will tell you that you’ve arrived?
Generic logic model » InputsOutputsOutcomes ActivitiesParticipation »»»» A diagram of the theory of how a program is supposed to work A graphic depiction of relationships between activities and results StrategyImpact C O N T E X T Source: University of Wisconsin – Extension 12
EVALUATION: check and verify What do you want to know?How will you know it? PLANNING: start with the end in mind University of Wisconsin – Extension, Cooperative Extension, Program Development and Evaluation
Logic Model Process applied to NSF MSP Started with 5 year program logic model and then logic models developed by various project working groups. Focused on building capacity of partners as well as some subgroups that cut across partners to use logic modeling as a strategy to reveal and negotiate core assumptions. Individual logic models serve as a tool for inquiry into the various project components and groups. The process of modeling and the product of negotiated models serve multiple purposes: a project management tool, evaluation tool and reflective tool for each of our groups and the project as a whole.
INPUTSOUTPUTSOUTCOMES Program investments ActivitiesParticipationShortMedium What we invest What we do Who we reach Long- term Linear vs iterative projects Feedback loops and multi-dimensions University of Wisconsin – Extension, Cooperative Extension, Program Development and Evaluation What results
Starting is not always pretty
Phasing in LM ① during proposal writing: negotiated overall LM for grant & then generated annual LM’s ② During year 1-2: created LM’s for core components of grant: PD, coaching, research, preservice, afterschool program ③ Using constant comparison of LM revealed 7 core assumptions ④ Evaluator and project management team aligned core components by gap analysis & identified priorities ⑤ Evaluator facilitated quarterly data retreats to align data of R&E with LM plans
Project Overview Logic Map: Year 1
Project Overview Logic Map: Year 2
Coaching Logic Model: Year 2
Research Question 2 Logic Map Year 2: Leadership Team, Faculty, Coaches, CoPs & Evaluator
Think-Pair-Share: How could you use a logic model? What purpose(s): Evaluation, planning, management, collaboration? What groups could use LM and how? What project complexities do you need to consider: –Number of partners? –Diversity of partners? –Years of collaborating prior to onset of grant? –Number of skilled group facilitators across partnership? –Base of trust across partners? 27
Logic Model Outcomes Process helped each group identify clearer goals, theories of action and actionable outcomes. Focus priorities each year, and serve as a management tool to self-monitor progress. Process helped each group identify clearer goals, theories of action and actionable outcomes Network across groups to increase coherence across partners; build capacity and share learning's about effective STEM teaching Has helped examine distribution of leadership and build towards systemic instructional change in all partner organizations.
Emerging LM What is needed to facilitate LM across the diversity of our partnership? How does LM capacity evolve over time across various partners and subgroups? How does LM contribute to sustainable capacity building of various stakeholders regarding project planning, implementation and evaluation use? How does LM contribute to conceptualizing partnership?
Learnings Process of LM may be more important in the short term given shifts in first 1-2 years Collaborative structures in second year enable more focused & aligned questions and methods for evaluation, research and program across partners Ecological framework reflects the partnership’s evolution of core assumptions Authentic partnership requires critical reflection about unexamined assumptions & trust
Resources and References Torres, R. T., Hopson, R. K., & Casey, J. (2008). Case Study of LM Use: Building STEM Education and Evaluation Capacity Through Research on LM Use. AERA, Denver, Co. UW-Cooperative Extension: Program Development and Evaluation. www. uwex. edu/ ces/ pdande/ RAND Working Paper, Math Science Partnership of Southwest Pennsylvannia, Year Three Evaluation Report, 2006
Project Overview Logic Map: Year 3
Project Overview Logic Map: Year 4
Project Overview Logic Map: Year 5
Research Question 2 Logic Map Year 2: IHE Partners
Research Question 2 Logic Map Year 2: Evaluate Grant Partners
EXAMPLE: Do you really have a partnership? What strategies have helped explore assumptions in your project? partnerships are where partners are closely bound to one another, share common goals, share liability and risk, and have a mutual interest in adapting behavior to one another in the pursuit of shared objectives. Think-Pair-Share: Questioning Assumptions
Research says… organizations tend to require compelling environmental pressures to engage in such potentially invasive relationships (Aldrich, 1999) Assumption EXAMPLE: partnerships are where partners are closely bound to one another, share common goals, share liability and risk, and have a mutual interest in adapting behavior to one another in the pursuit of shared objectives.
Core Assumptions to examine: Continuous collective improvement Culturally responsive practice Building capacity for CoP Networked Learning Distributed Leadership Authentic Partnership Data enhanced systemic instructional change