1. How Logical Is Your Logic Model
How Logical Is Your Logic Model? Developing Useful and Robust Project Logic Models
2/27/2014

2. The work of TEAMS is supported with funding provided by the National Science Foundation, Award Number DRL Any opinions, suggestions, and conclusions or recommendations expressed in this presentation are those of the presenter and do not necessarily reflect the views of the National Science Foundation; NSF has not approved or endorsed its content.
2/27/2014

3. Strengthen the quality of the MSP project evaluation and build the capacity of the evaluators by strengthening their skills related to evaluation design, methodology, analysis, and reporting
2/27/2014

4. Online Help-Desk for submitting requests Website at: teams.mspnet.org
Webinar series targeted to specific evaluation topics
Tiered technical assistance for differentiated services
Instrument review and sharing
2/27/2014

5. RMC Research Corporation
How Logical Is Your Logic Model? Developing Useful and Robust Project Logic Models
Dave Weaver, Director
RMC Research Corporation
Portland, Oregon
February 27, 2014
2/27/2014

6. Webinar Objectives
Provide a rationale for the need for a well-defined logic model
Summarize characteristics of a well-defined theory of action and logic model
Provide an opportunity to examine your current theory of action and logic model
Discuss strategies for developing useful and robust logic models
2/27/2014

7. Why Are Logic Models Important?
Common Guidelines for Education Research and Development
U.S. Department of Education, National Science Foundation. (August, 2013).
“The proposal should include a description of the initial concept for the planned investigation, including a well-explicated theory of action or logic model. The concept and logic model should identify key components of the intervention and should describe their relationships, theoretically, and operationally.”
2/27/2014

8. What does it mean? “well-explicated theory of action or logic model”
2/27/2014

9. It starts with the Theory of Action A logic model is only as good as the theory of action that it is based on
2/27/2014

10. What Is a Theory of Action?
Confronting the Elephant in the Room
North Central RTI TA CollaborativeNext Steps in Building Collaborative Technical Assistance in the North Central Region
April 2011
February 2009February 2009
April 2009February 2009February 2009
What Is a Theory of Action?
Collective belief about causal relationships between action and desired impacts
Simple:
If . .   Then . . . 
Complex:
If  . . .  and . . .  and . . .  and . . . Then  . . . 
Collaborative interpretation of the literature
2/27/2014
NWRCC
Next Steps in Building Collaborative Technical Assistance in the North Central RegionNorth Central RTI Technicl Assistance Collabortive
National Center on Response to InterventionNext Steps in Building Collaborative Technical Assistance in the North Central RegionNorth Central RTI Technicl Assistance Collabortive 10 10

11. Characteristics of a Robust Theory of Action
Confronting the Elephant in the Room
April 2011
Characteristics of a Robust Theory of Action
Describes project impact as close to the primary target audience as possible
Example: A description of what students do to learn
Recognizable when it is going on
Observable
Defines fidelity of project implementation
Can be a testable hypothesis
Believable
Interpretation of current literature
2/27/2014
NWRCC

12. Math Example (Common Core State Standards)
If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:
Justifying—Explaining and justifying their reasoning mathematically
Generalizing—Identifying and verifying conjectures or predictions about the general case
Representing—Using representations (symbolic, notation, graphs, charts, tables, and diagrams) to communicate and explore mathematical ideas
Applying—Applying mathematical skills and concepts to real-world applications
Then student achievement and interest in mathematics will increase.
2/27/2014

13. Does it describe project impact as close to the primary target audience as possible?
If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:
Justifying—Explaining and justifying their reasoning mathematically
Generalizing—Identifying and verifying conjectures or predictions about the general case
Representing—Using representations (symbolic, notation, graphs, charts, tables, and diagrams) to communicate and explore mathematical ideas
Applying—Applying mathematical skills and concepts to real-world applications
Then student achievement and interest in mathematics will increase.
2/27/2014

14. Is it recognizable when it is going on?
If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:
Justifying—Explaining and justifying their reasoning mathematically
Generalizing—Identifying and verifying conjectures or predictions about the general case
Representing—Using representations (symbolic, notation, graphs, charts, tables, and diagrams) to communicate and explore mathematical ideas
Applying—Applying mathematical skills and concepts to real-world applications
Then student achievement and interest in mathematics will increase.
2/27/2014

15. Does it define fidelity of project implementation?
If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:
Justifying—Explaining and justifying their reasoning mathematically
Generalizing—Identifying and verifying conjectures or predictions about the general case
Representing—Using representations (symbolic, notation, graphs, charts, tables, and diagrams) to communicate and explore mathematical ideas
Applying—Applying mathematical skills and concepts to real-world applications
Then student achievement and interest in mathematics will increase.
2/27/2014

16. Can it be a testable hypothesis?
If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:
Justifying—Explaining and justifying their reasoning mathematically
Generalizing—Identifying and verifying conjectures or predictions about the general case
Representing—Using representations (symbolic, notation, graphs, charts, tables, and diagrams) to communicate and explore mathematical ideas
Applying—Applying mathematical skills and concepts to real-world applications
Then student achievement and interest in mathematics will increase.
2/27/2014

17. Is it believable?
If teachers use developmentally appropriate yet challenging tasks and activities that engage students in:
Justifying—Explaining and justifying their reasoning mathematically
Generalizing—Identifying and verifying conjectures or predictions about the general case
Representing—Using representations (symbolic, notation, graphs, charts, tables, and diagrams) to communicate and explore mathematical ideas
Applying—Applying mathematical skills and concepts to real-world applications
Then student achievement and interest in mathematics will increase.
2/27/2014

18. Science Example Students learn science when they:
Articulate their initial ideas,
Are intellectually engaged with important science content,
Confront their ideas with evidence,
Formulate new ideas based on that evidence, and
Reflect upon how their ideas have evolved
2/27/2014

19. Online Survey 1 Does your project have a theory of action statement?
If yes, to what extent does it meet the criteria?
Criteria for a robust theory of action:
Describes impact close to the primary audience
Is recognizable in practice
Defines fidelity of implementation
Can be a testable hypothesis
Is believable—Shared interpretation of literature
2/27/2014

20. Confronting the Elephant in the Room
April 2011
What Is a Logic Model?
A diagram that shows the logical connection between project resources, activities, outcomes, and expected impacts
Incorporates a primary theory of action
Can be viewed as a collection of theories of actions
Answers the question:
Why would the planned activities be expected to have the desired impacts?
2/27/2014
NWRCC

21. Basic Logic Model Black Box Issues Resources Distal Impacts Activities
Inputs
Activities
Outputs
Outcomes
Impacts
Resources
Distal Impacts
Activities
Proximal Impacts
2/27/2014

22. Logic Model Columns
Resources—What resources are or could reasonably be available?
Strategies and Activities—What will the activities, events, etc. be?
Outputs—What are the initial products of these activities?
Short-Term Outcomes—What changes are expected in the short-term?
Long-Term Outcomes—What changes are wanted after initial outcomes?
Impacts—What are hoped for changes over long haul?
2/27/2014

23. Characteristics of a Robust Logic Model
Incorporates a primary theory of action
Shows the logical connection between project resources, activities, outcomes, and expected impacts
Is a collection of If … Then… statements
Answers the question: Why would the planned activities be expected to have the desired impacts?
Serves as a framework for project evaluation
Guides fidelity of implementation
2/27/2014

24. Making Mathematical Reasoning Explicit (MMRE) Logic Model

25. Does it incorporate the primary theory of action for the project?
If teachers use rich mathematical tasks combined with purposeful and probing questions to engage students in discourse and the use of the tools (notation, symbolization, graphs, charts, etc.) of mathematics to:
Explain and justify their mathematical reasoning (justification)
Develop and verify mathematical generalizations
Then students’ reasoning skills and student achievement in mathematics will increase.

26. Does it show the logical connections?

27. Does it show why the planned activities would be expected to have the desired impacts?

28. Does it serve as a framework for project evaluation?

29. Does it guide fidelity of implementation?

30. Online Survey 2 Does your project have a logic model?
If yes, to what extent does it meet the criteria?
Criteria for a robust logic model:
Incorporates the project theory of action
Shows the logical connections between project resources, activities, outcomes, and expected impacts
Explains why the planned activities would be expected to have the desired impacts
Serve as a framework for the project evaluation
Guides fidelity of project implementation
2/27/2014

31. Strategies for Logic Model Development
Engage project stakeholders
Begin by drafting a primary theory of action statement
Go back to the characteristics of a robust theory of action
Ask probing questions to help stakeholders think about why activities should work
Take notes and work toward consensus
2/27/2014

32. Suggested Process First half-day session with stakeholders
Learn about project activities, objectives, and goals
Introduce logic model development process
Gather input for project theory of action
Between Sessions
First draft of Theory of Action and Logic Model
Second half-day session
Gather input on draft
Establish commitment to theory of action statement
“Do you believe that this statement accurately states your collective belief about why your project will achieve its desired impacts?”
2/27/2014

33. Go Back to the Characteristics of a Robust Theory of Action
Confronting the Elephant in the Room
April 2011
Go Back to the Characteristics of a Robust Theory of Action
Describes project impact as close to the primary target audience as possible
Example: A description of what students do to learn
Recognizable when it is going on
Observable
Defines fidelity of project implementation
Can be a testable hypothesis
Believable
Interpretation of current literature
2/27/2014
NWRCC

34. Some of My Favorite Questions!
What would it look like in the classroom if your professional development was effective?
What kind of cognitive engagement would students be involved in?
How would you know your project has been successful?
When this project is over what will you be able to point to as a legacy of the project?
What is it about your professional development that you think is effective?
2/27/2014

35. Using the Technology Taking Notes Drafting the Logic Model
Capture the language of the stakeholders
Don’t be afraid to paraphrase
Use outliner view
Drafting the Logic Model
Don’t use a table format—too linear and constraining
Use graphic software
Visio
MS Word
2/27/2014

36. Please use your chat box to submit questions for Dave
2/27/2014

37. RMC Research Corporation 633 17th Street Suite 2100
John T. Sutton, PI
Dave Weaver, Co-PI
RMC Research Corporation
633 17th Street
Suite 2100
Denver, CO
111 SW Columbia Street
Suite 1030
Portland, OR
Phone:
Toll Free:
Fax:
Phone:
Toll Free:
Fax:
2/27/2014