MassCUE Evaluators Day 1 – January 11, 2012

Greetings! zLogistics and Locations yWiFi SSID and Password yBathrooms, etc. zTeams yWho are you? yTeam members? yBrief overview of your project

Your Facilitators zSun Associates yJeff Sun yJeanne Clark ywww.sun-associates.com/masscuewww.sun-associates.com/masscue zDamian Bebell - Boston College zShelley Chamberlain - MassCUE

Agenda and Expectations zReview the agenda zExpectations yAll team members will attend all 4 days ySome work will occur outside of these sessions yCollaborate! yProduce a concrete evaluation plan yProvide feedback to the process

Your Evaluation Plan Will… zBe organized around the goal(s) of your project zDefine success through the creation of indicators keyed to your project goal(s) zProduce data specifically targeted at measuring success relative to your project goal(s) zProvide feedback and recommendations for improvement

Why Evaluate? zDetermine if your investment in instructional technology is “paying off” zMeasure progress toward meeting your project goals zSupport action planning with data zIncrease eligibility for funding

Schematic View of the Evaluation Process

Goal What is your project aiming to accomplish? zThe goal should be big-picture and over- arching…encompassing the spirit of what you want to accomplish. yFurther detail on what it takes to meet the goal is covered in your indicators.

Indicators What does it look like when your goal is met? zIndicators are the organizing principle of your evaluation zIndicators should reflect your project’s unique goals and aspirations zIndicators should clearly describe what it looks like when/as you meet your goal(s) yIndicators need to be highly descriptive and can include both qualitative and quantitative measures ySome indicators are more “measurable” than others zIndicators guide your data collection

Data What do you need to examine in order to find out if you’re meeting your indicators? zCurriculum yReview of curriculum units/lesson plans yClassroom observations (of student impact) yStudent skills assessments yReview of student work zProfessional Development yTeacher interviews yProfessional development plans yReview of developed lesson plans, etc. yClassroom observation (of pedagogy) zInfrastructure yClassroom observations ySurveys yCost data yHelp desk data

Coordinating with Existing Data zWhat data is already collected and available for you to use? zOutcomes data on student achievement? zExisting student assessments? zYou may choose to use these other data sources in your analysis to address aspects of your indicators. zSometimes, this additional data will be one of the sources that you use to “triangulate” your findings.

Showing Impact and Change How do you determine that what you see happening in your program is different from the status quo? zIs there control/comparison group available for comparison? zAre pre-measures available for comparison?

Analysis and Findings zData analysis – comparing data to indicators -- will show the degree to which actions (PD, pedagogy, student work, infrastructure implementation) come together to produce the intended result. zFindings are the results of this analysis.

Recommendations zFindings (analysis) lead to recommendations and reporting. zRecommendations lead to new action items. yWhat are you going to do to implement the recommendations?

Developing Indicators zIndicators are the organizing principle of your evaluation zIndicators should reflect your project’s unique goals and aspirations zYour goals need to clearly state the purpose of your project yWhat is it that you intend to create with this project? zIndicators should clearly describe what it looks like when/as you meet your goal(s) yIndicators need to be highly descriptive and can include both qualitative and quantitative measures ySome indicators are more “measurable” than others zIndicators guide your data collection

Goal Statement: Project / PlanEvaluation What will your project be doing? What will be the desired outcome of these actions? What do these desired outcomes look like in the context of your project? What do you need to find out in order to assess how effectively your actions are producing the desired outcomes and fulfilling the indicators? How closely does your data match your indicators? Actions Desired OutcomesIndicator Data Avenues/S ources Questions/ Probes Data Analysis Curriculum PD Infrastructure Etc….

Day 1 Wrap-Up zQuestions? zDaily Evaluation zSee you tomorrow!