1 Evaluation Framework and Communication of Results

2 Evaluation Questions and Construction of Evaluation Framework Evaluation Questions—A Review Formative Summative Question: From WHERE do we derive our Evaluation Questions?

3 Evaluation Framework What are the key constructs that will be measured? What changes am I hoping to find? What do I need to know about the strategies and content used in professional development? Who has the information I need to collect? How will I gather the information I need from each source? What tools or processes will I use to gather the information I need?

4 Evaluation Framework Components Program Goals Measurable Objectives Information/Data Needed Data Source Data Collection Strategy Data Analysis Plan Timeline

5 Creation of Evaluation Framework Question: What do we consult to develop the Evaluation Framework?

6 Putting the Pieces Together Theory of ChangeLogic ModelEvaluation Framework

7 Questions to ask of your Theory of Change: What key concepts will be measured? How will those key concepts be measured? How will data from those measures be analyzed to construct the answer to your evaluation questions?

8 Evaluation Design: Experimental Quasi-experimental Descriptive Naturalistic Case studies Mixed-method

Communicating Results 9 Allow your data to tell your story Guiding questions or goals provide the basis of your narrative (NOT the survey) Be clear about what was studied Be concise! (Always include an executive summary) Be objective!

Creating Tables 10 Write table titles that report exactly what is in your table Label every column and every row Avoid using too many numbers in the table Report group sizes Report whole number percentages Present data in some sort of order (i.e., low to high)

11 Quantitative Data: Example

Descriptive Statistics Figure 1. Total Years of Teaching Experience for Teachers who completed the Teacher Survey

Frequency Distribution 13 Daily/WeeklySeldom/Never Activity%(n)% Manage student information100% (88)24%(21) Communicate with colleagues95% (84)15%(13) Develop instructional materials84% (73)23%(20) Communicate with parents87% (73)27%(23) Research to augment a lesson plan74% (65)58%(51) Post homework assignments72% (63)0%(0) Access student assignments65% (57)48%(42) Communicate with students52% (45)3%(3) Distribute and collect tests36% (32)17%(15) Table 6 Teacher ’ s Frequency of Technology Use in the Classroom by Activity

Data… 14 Figure 2. Frequency of Student Use of Laptop Computers by Activity

15 Tamara Otto Research and Evaluation Center for Technology in Education