Based on the work of Vicki Bernhardt ITF January 27, 2009

 Replace hunches with facts concerning changes  Facilitate clear understanding of gaps between where school is and where it needs to be, and identify root causes-not treat symptoms  Provide information to eliminate ineffective practices  Ensure efficient use of $$$  School if school goals and objectives are being accomplished  Ascertain if school staffs are implementing their visions  Generate answers and effectively educate the community  Predict and prevent failures  Predict and ensure successes  Improve instruction  Provide students with feedback on their performance.

 Gain understanding of what quality is and how close we are to achieving it  Make sure students are not “falling through the cracks  Get the “root causes” of the problems  Guide curriculum development and revision  Meet state and federal requirements  Promote accountability

 In contrast to business, work culture in education usually doesn’t focus on data  Few are trained and few see it as part of their jobs  Schools don’t have data warehousing solutions that make it easier  Teachers are generally trained to be subject oriented, not data oriented  There’s a perception that data is gathered for other people not mine  The legislature keeps on changing the rules! (e.g. NHEIAP, NECAP)

 Data analysis should be  Systematic, Systemic All of the the processes and procedures that contribute to learning-the whole and the interrelationships of the parts of the whole to each other  Continuous Measuring and evaluating processes on an ongoing basis to identify and implement improvement “Upstream process improvement not downstream damage control” Teams and Tools (1991)

 One measure gives useful information, but…  Comprehensive, multiple measures give much richer info.

 Demographics (enrollment, drop out rate, gender, ethnicity)  Perceptions (values/beliefs, attitudes)  Student Learning (standardized tests, authentic assessments, teacher observations)  School Processes (how we teach children to read, our Math scope and sequence through the years)

Each of the four measures, by themselves, give valuable information. But the deeper we dig and the more levels we utilize, the more effective the resulting information.

 How many students are enrolled at MSS this year? (Demographics)  How satisfied are parents, students, and/or staff with the learning environment at HMS? (Perceptions)  How did students at the Hopkinton High School score on the SATs this year? (Student Learning)  What new courses are being offered by the high school this year? (School Processes)

 How has enrollment changed throughout the HSD over time? (Demographics)  How have student perceptions of the A.P. program changed over time? (Perceptions)  Are there differences in boy’s 6th grade math scores in the NHEIAP over time? (Student Learning)  What programs have been consistent at MSS over the last five years? (School Processes)

Looking at more than one type of data within each of the circles..  What percentage of students in the HSD are fluent speakers of languages other than English and are there equal numbers of males and females? (Demographics)  Are staff, students, and parent perceptions of the culture within HMS in agreement? (Perceptions)  Are student’s scores on the 6th grade NHEIAP consistent with K-6 assigned grades and performance assessment rubrics? (Student Learning)  What are the major instructional strategies used by the Math and Science departments at the high school? (School Processes)

 Similar to Level 3…just does the analysis over time…  For example: How has the enrollment of non-English speaking children changed in the last three years? (Demographic)

 Do students who have perfect attendance perform better on the NHEIAP than students who miss more than five days per month? (Demographics by Student Learning)  Is there a gender difference in students’ perceptions of The Senior Project? (Perceptions by Demographics)  Do students engaged in classrooms utilizing differentiated instruction perform better on the NHEIAP than in those classrooms that do not employ this strategy? (Student Learning by School Processes)

Same as Level 6, just over time, for example…  How have students of different ethnicities scored on the SAT over the past three years? (Demographics by Student Learning)

 Is there a difference in students’ reports of what they like most about MSS by whether or not they participate in extracurricular activities? Do these students have higher grades than those who do not participate in extracurricular activities? (Perceptions by Student Learning by School Processes)  What program at the high school is making the biggest difference with respect to student achievement for at-risk students this year and is one group of students responding better to the processes? (School Processes by Student Learning by Demographics)

Same as Level 7, but over time, for example…  What programs throughout the district do all types of students like the most every year? (Demographics by Perceptions by School Processes)

This is the ultimate analysis. It allows us to answer questions such as:  Are there differences in NWEA scores for 8th grade girls and boys who report they like school, by the type of program in which they are enrolled? (Demographics by Perceptions by School Processes by Student Learning)

Same as Level 9, but over time, for example…  Based on whom we have as students, how they prefer to learn, and what programs they are in, are all students learning at the same rate? (Student Learning by Demographics by Perceptions by School Processes)

Demographics, Perceptions, Student Learning, and School Processes

 Community  Location, history, population trends…  School District  Description and history, number of schools…  School  Grants and awards received, class sizes, facilities

 Students Over Time, and by Grade Level  Attendance, gender, race…  Staff Over Time  Years of experience, ethnicity, certifications…  Parents  Educational levels, involvement with child’s learning...

The separation of data into subgroups. Why is it important?  Disaggregation shows if there are differences in subgroups-there should be few differences.  Helps us find subgroups that are not responding to our techniques.

 A generally held view.  A belief stronger than impression and less strong than positive knowledge.

 Teachers  Parents  Students  Administrators  Community members

 Give parents a chance to experience the approach, e.g. parent night, open house.  Cognitive dissonance; the discomfort one feels when two thoughts, opinions, or ideas are inconsistent.

 Questionnaires  Interviews  Focus Groups (All three were done as part of the SPEDMIP report)

They must be:  Valid  Reliable  Understandable  Quick to complete

 Interview (face to face)  Telephone Interview (person to person)  Mailed  Paper  Online (recommended, e.g. SurveyMonkey.com

 Standardized tests  Norm-referenced, criterion referenced  Authentic assessments  Teacher-made test  Teacher-assigned grades  Performance assessments  Standards based assessments  Diagnostic testing

 Compares test performance of a school, group, or individual with the performance of a norming group (e.g. CAT, Iowa)

 Compares individuals performance to a set of standards and not to the performance of other test takers. (e.g. NHEIAP, NECAP)

 Tests given to students to know the nature of a student’s difficulty but not necessary the cause of that difficulty.(e.g. Sped testing)

 Measures skill and knowledge directly, e.g. if you want students to learn to write, assess it on a writing assignment  Advantages: measures process, can match state standards, is a learning tool in and of itself, provides reflection for students...

1. Identify desired results 2. Determine acceptable evidence 3. Plan learning experiences & instruction (from UbD PPT)

 The evidence should be credible & helpful.  Implications: the assessments should –  Be grounded in real-world applications, supplemented as needed by more traditional school evidence  Provide useful feedback to the learner, be transparent, and minimize secrecy  Be valid, reliable - aligned with the desired results of Stage 1 (and fair) (from UbD PPT)

 We need patterns that overcome inherent measurement error  Sound assessment (particularly of State Standards) requires multiple evidence over time - a photo album vs. a single snapshot (from UbD PPT)

 Varied types, over time:  authentic tasks and projects  academic exam questions, prompts, and problems  quizzes and test items  informal checks for understanding  student self-assessments (from UbD PPT)

 Programs  Reading Recovery, Write Traits, UBD  Practices  HBA, Mini-Mentors, Student Councils  Instructional strategies  Differentiated Instruction, Cooperative Learning

 What teachers and administrators do to achieve the vision of the school  What do we want students to be able to know and do?  How are we enabling students to learn in terms of Instructional strategies, learning strategies, instructional time, student-teacher ratio, philosophies of classroom management  How will we know if any given approach works?  What will we do with students who don’t learn this way?  How will the parts of the curriculum relate?  What learning strategies do successful learners use?

 Solid training in UbD  Conducting a School Portfolio  Setting up a systematic system for data collection and analysis  Selecting a data warehouse  Training the staff in general