Sample Outcome Data from Makes Sense Strategies Research Impact on WRITING Impact on VOCABULARY Impact on INFORMATION PROCESSING SKILLS Impact on CONTENT LEARNING Impact on CLASSROOM TEST SCORES MAKES SENSE STRATEGIES 4 Teaching & Learning

The study compared … Graphic Organizers depicting basic organization structures vs. Essential Understanding Smart-sheets vs. Generative Idea Smart-sheets The study measured… Breadth, depth, & accuracy of knowledge of 98 students Impact on learning when different kinds of cues are embedded into graphic organizers STUDY 1 16 high achieving students 16 typical achieving students 16 low achieving students 16 low achieving students w/LD

MAKES SENSE STRATEGIES 4 Teaching & Learning TEXT RESOURCES CLASS 1 GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic PHASE 1PHASE 2PHASE 3PHASE 4 Teacher employed a different strategy during each phase of instruction STUDY 1 SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic

MAKES SENSE STRATEGIES 4 Teaching & Learning Pre & Post measures of students’ Breadth of knowledge Depth of knowledge Accuracy of knowledge TEXT RESOURCES CLASS 1 GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic PHASE 1PHASE 2PHASE 3PHASE 4 STUDY 1 SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic Learning measured for each phase

MAKES SENSE STRATEGIES 4 Teaching & Learning TEXT RESOURCES CLASS 1 GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic PHASE 1PHASE 2PHASE 3PHASE 4 STUDY 1 SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic 2nd teacher used same strategies, but in a different order CLASS 2 SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic TEXT RESOURCES GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic

MAKES SENSE STRATEGIES 4 Teaching & Learning TEXT RESOURCES CLASS 1 GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic PHASE 1PHASE 2PHASE 3PHASE 4 STUDY 1 SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic CLASS 2 SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic TEXT RESOURCES GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic TEXT RESOURCES GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic CLASS 3 SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic TEXT RESOURCES CLASS 4 3rd & 4th teachers used same strategies, also in different orders

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 RESULTS 2-way analysis of variance to analyze interaction effects GROUP X TREATMENT Data analyzed using… Tukey’s Post-hoc analysis of significant interactions

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 Relative impact of strategy on knowledge gains RESULTS ACCURACY OF KNOWLEDGE Same pattern found, regardless of type of student high achieving typical achieving low achieving low achieving w/LD GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES Text Resources SMART-SHEETS featuring GENERATIVE IDEAS about a topic SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic NO statistically significant differences found between each instructional approach on students ’ accuracy of knowledge learned In other words, each treatment was equally effective (or not effective) for the different types of students on measures of accuracy

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 Relative impact of strategy on knowledge gains DEPTH OF KNOWLEDGE RESULTS

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 Text Resources Relative impact of strategy on knowledge gains GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic DEPTH OF KNOWLEDGE Again, each of type of student responded similarly to each type of instruction RESULTS Significant differences found between each instructional approach on students ’ DEPTH of knowledge

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 Text Resources Relative impact of strategy on knowledge gains GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic DEPTH OF KNOWLEDGE RESULTS Essential Understandings Most beneficial to ALL groups of students

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 Text Resources Relative impact of strategy on knowledge gains GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic DEPTH OF KNOWLEDGE RESULTS Basic Graphic Organizers pretty effective as well

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 Relative impact of strategy on knowledge gains BREADTH OF KNOWLEDGE RESULTS

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 Text Resources Relative impact of strategy on knowledge gains GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic BREADTH OF KNOWLEDGE RESULTS Significant differences found between each instructional approach on students ’ BREADTH of knowledge Again, each of type of student responded similarly to each type of instruction

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 Text Resources Relative impact of strategy on knowledge gains GRAPHIC ORGANIZERS depicting basic ORGANIZATION STRUCTURES SMART-SHEETS featuring GENERATIVE IDEAS about a topic SMART-SHEETS featuring ESSENTIAL UNDERSTANDINGS about a topic BREADTH OF KNOWLEDGE RESULTS Essential Understandings CLEARLY most beneficial

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 1 RESULTS So how well do the different treatments reduce the achievement gap b/w typical learners & students w/LD?

Impact of Graphic Organizers & Essential Understanding Smart-sheets on high school students’ BREADTH of relational knowledge via social studies instruction 21% less TAHA LA LD 29% less 27% less Traditional Text Materials High achievers mean level of BREADTH of knowledge Gap between high & typical achievers Gap between typical achievers & students w/ LD

21% less TA HA LA LD 29% less 27% less TA HA LA LD Impact of Graphic Organizers & Essential Understanding Smart-sheets on high school students’ BREADTH of relational knowledge via social studies instruction 21% less TAHA LA LD 29% less 27% less Traditional Text Materials Graphic Organizer Materials TA HA LA LD

Impact of Graphic Organizers & Essential Understanding Smart-sheets on high school students’ BREADTH of relational knowledge via social studies instruction 21% less TAHA LA LD 29% less 27% less Traditional Text Materials Graphic Organizer Materials TA HA LA LD 4 % less 11 % less 9 % less 16 % pt reduction in gap

Impact of Graphic Organizers & Essential Understanding Smart-sheets on high school students’ BREADTH of relational knowledge via social studies instruction 25 % less 21% less TAHA LA LD 29% less 27% less Traditional Text Materials Graphic Organizer Materials TA HA LA LD 4 % less Essential Understanding Smart-sheers 17 % less HA LA LD TA 9 % less 11 % less 13 % less TA HA LA LD

HA Impact of Graphic Organizers & Essential Understanding Smart-sheets on high school students’ DEPTH of relational knowledge via social studies instruction 21% less TA LA LD 29% less 27% less Traditional Text Materials

TA HA LA LD 21% less TA HA LA LD 29% less 27% less TA HA LA LD Impact of Graphic Organizers & Essential Understanding Smart-sheets on high school students’ DEPTH of relational knowledge via social studies instruction 21% less TAHA LA LD 29% less 27% less Traditional Text Materials Graphic Organizer Materials

TA HA LA LD Impact of Graphic Organizers & Essential Understanding Smart-sheets on high school students’ DEPTH of relational knowledge via social studies instruction 21% less TAHA LA LD 29% less 27% less Traditional Text Materials Graphic Organizer Materials TA HA LA LD 2 % less 20 % less 19 % less

TA HA LA LD Impact of Graphic Organizers & Essential Understanding Smart-sheets on high school students’ DEPTH of relational knowledge via social studies instruction 11% less 21% less TAHA LA LD 29% less 27% less Traditional Text Materials Graphic Organizer Materials TA HA LA LD 2 % less Essential Understanding Smart-sheers 16 % less HA LA LD TA 20 % less 11% less 19 % less

1 st place Teachers overwhelmingly preferred because… …Helped them focus on what was important …Made planning lessons easier …Resulted in more structured lessons & discussions …Liked how student elaboration was prompted, thereby making learning of history more applicable to students’ lives today and prompting students to practice higher order thinking. …Liked variety of uses and adaptability of the graphic organizers with topic- specific prompts across many classroom situations. …Perceive them as are of benefit to all students regardless of ability level or disability status. Most favored by teachers and students MAKES SENSE STRATEGIES 4 Teaching & Learning Essential understandings Smart-sheets STUDY 1 RESULTS

STUDENTS overwhelmingly preferred because… … Helped them focus on what was important … Found this type to be interesting … Valued the opportunities to elaborate … Perceived that they enhanced their learning & studying Most favored by teachers and students MAKES SENSE STRATEGIES 4 Teaching & Learning GOs containing topic- specific “essential understanding” prompts STUDY 1 RESULTS

5 th Grade Inclusive Classroom How well do what teachers say were key ideas and important details they hoped students learned match with what students actually reveal were learned? MSS impact on teachers’ ability to successfully identify and teach relevant main ideas and important details to students w/LD MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 2

Before MSS After MSS 04 Ave. number of main ideas about the lesson identified by students that matched with main ideas the teacher identified that she intended students to learn Ave. number of important facts (or details) students identified by students that matched with facts the teacher identified that she intended students to learn th grade students were interviewed immediately following social studies lesson. Interviewer webbed information as students identified ideas and directed interviewer where to note them on the web. 1.What was that whole lesson about? 2.What are some really important main ideas or different topics from the lesson? Why do you think they are important? 3.What are some specific things or facts from the lesson you think were really important? Why do you think they were important? MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 2

MAKES SENSE STRATEGIES 4 Teaching & Learning Impact on composition writing of 8th grade students with learning disabilities Typical 8 th grader N= words CONTROL group of students w/LD N = 28 EXPERIMENTAL group of students w/LD N = 28 FLUENCY PretestPost-testPretestPost-test Typical 8 th grader produces an average of 117 words on high- stakes essays 8 th students with LD typically produce about 27 words on the same task Experimental Group Students w/LD 29 words 26 words Control Group Students w/LD STUDY 3

MAKES SENSE STRATEGIES 4 Teaching & Learning Impact on composition writing of 8th grade students with learning disabilities Typical 8 th grader N= words CONTROL group of students w/LD EXPERIMENTAL group of students w/LD FLUENCY PretestPost-testPretestPost-test Experimental Group Students w/LD 29 words 26 words Control Group Students w/LD On the post test, the control group of students w/LD produced slightly fewer words than they did on the pretest 24 words STUDY 3

MAKES SENSE STRATEGIES 4 Teaching & Learning Impact on composition writing of 8th grade students with learning disabilities Typical 8 th grader N= words CONTROL group of students w/LD EXPERIMENTAL group of students w/LD FLUENCY PretestPost-testPretestPost-test 29 words 26 words 24 words Experimental group of students w/LD (received MSS instruction) 126 words +97 words more than pretest +9 words more than typical 8 th grader STUDY 3

9 Schools Extremely low performance 8 Schools Low performance 7 Schools Moderate performance 2 Schools Good performance MAKES SENSE STRATEGIES 4 Teaching & Learning High-Stakes Writing Assessment This study examined the impact of MSS on schools with varying degrees of prior performance This type of study shows what kind of school is likely to benefit the most from MSS STUDY 4 Following 1 year MSS implementation

9 Schools Extremely low performance 8 Schools Low performance 7 Schools Moderate performance 2 Schools Good performance 21.6% before MSS 38.83% before MSS 58.39% before MSS 73.02% before MSS MSS 62.41% pt. gain MSS 74.81% pt. gain MSS 49.24% pt. gain MSS 82.68% 9.79 pt. gain Lower-performing schools definitely demonstrate the most dramatic improvements MAKES SENSE STRATEGIES 4 Teaching & Learning High-Stakes Writing Assessment STUDY 4

9 Schools Extremely low performance 8 Schools Low performance MSS 62.41% 7 Schools Moderate performance MSS 74.81% 2 Schools Good performance MSS 49.24% MSS 82.68% 21.6% before MSS 38.83% before MSS 58.39% before MSS 73.02% before MSS pt. gain pt. gain pt. gain 9.79 pt. gain Moderate-performing schools closed the gap to within 9% pts of Good-performing schools MAKES SENSE STRATEGIES 4 Teaching & Learning High-Stakes Writing Assessment STUDY 4

23.08 pt. gain pt. gain pt. gain 9.79 pt. gain 9 Schools Extremely low performance 8 Schools Low performance MSS 62.41% 7 Schools Moderate performance MSS 74.81% 2 Schools Good performance MSS 49.24% MSS 82.68% 21.6% before MSS 38.83% before MSS 58.39% before MSS 73.02% before MSS pt. gain pt. gain pt. gain 9.79 pt. gain MSS school gains when compared to the state-wide average MAKES SENSE STRATEGIES 4 Teaching & Learning High-Stakes Writing Assessment STUDY 4 State-wide average =.05 pt. gain during same period

MAKES SENSE STRATEGIES 4 Teaching & Learning 7th grade English Language Learners / ESL Gains 24%75%51 pts Makes Sense Strategies implementation California Writing Assessment % students meeting standards STUDY 5

State Ave 30.90% % % MSS #1 36.1% 28.08%51.43% MSS implementation 1 ST YEAR pt gain! pt gain MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 6 Alabama Writing Assessment (High-stakes) Elementary 5th grades (semi-rural) % students meeting or exceeding standards

State Ave 30.90% % % % MSS #1 36.1% 28.08%51.43%81.0% MSS implementation 1 ST YEAR 2nd YEAR pt gain! 1.61 pt gain STUDY 6 MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 6 Alabama Writing Assessment (High-stakes) Elementary 5th grades (semi-rural) % students meeting or exceeding standards

State Ave 30.90% % % % nd YEAR 81.0% MSS #1 36.1% 28.08%51.43% MSS implementation 1 ST YEAR MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 6 Alabama Writing Assessment (High-stakes) Elementary 5th grades (semi-rural) % students meeting or exceeding standards

State Ave 30.90% % % % MSS #1 36.1% 28.08%51.43%81.0% MSS implementation MSS # % 10.0% 62.24% 1 ST YEAR 2nd YEAR pt gain! 71.0% 8.76 pt gain STUDY 6 MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 6 Alabama Writing Assessment (High-stakes) Elementary 5th grades (semi-rural) % students meeting or exceeding standards

State Ave 30.90% % % % MSS #1 36.1% 28.08%51.43%81.0% MSS implementation MSS # % 10.0% 62.24% 1 ST YEAR 2nd YEAR 71.0% STUDY 6 MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 6 Alabama Writing Assessment (High-stakes) Elementary 5th grades (semi-rural) % students meeting or exceeding standards

MSS #3 11.2%15.09%63.63% State Ave 30.90% % % % MSS #1 36.1% 28.08%51.43%81.0% MSS implementation MSS # % 10.0%62.24%71.0% 1 ST YEAR 2nd YEAR pt gain! MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 6 Alabama Writing Assessment (High-stakes) Elementary 5th grades (semi-rural) % students meeting or exceeding standards

34.0% MSS #3 11.2%15.09%63.63% State Ave 30.90% % % % MSS #1 36.1% 28.08%51.43%81.0% MSS implementation MSS # % 10.0%62.24%71.0% 1 ST YEAR 2nd YEAR pt LOSS! MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 6 Alabama Writing Assessment (High-stakes) Elementary 5th grades (semi-rural) % students meeting or exceeding standards

22.4% 34.0% MSS #3 11.2%15.09%63.63% State Ave 30.90% % % % MSS #1 36.1% 28.08%51.43%81.0% MSS implementation MSS # % 10.0%62.24%71.0% 1 ST YEAR 2nd YEAR pt LOSS! Why do you think this score dropped so drastically? School thought they had “made it” in writing so emphasis switched to other issues MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 6 Alabama Writing Assessment (High-stakes) Elementary 5th grades (semi-rural) % students meeting or exceeding standards

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY % 34.0% MSS #3 11.2%15.09%63.63% State Ave 30.90% % % % MSS #1 36.1% 28.08%51.43%81.0% MSS implementation MSS # % 10.0%62.24%71.0% 1 ST YEAR 2nd YEAR pt LOSS! Leadership & support for using MSS disappeared! Leadership expectations for fidelity & support must be on-going to ensure sustainability! IMPLICATIONS

MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 7 7th Grade Alabama Writing Assessment Gains Hi-performing school Comparison school %-2.17%43.79% Comparison school %-7.05%35.55% Makes Sense school 37.65%60.51%22.86% 92.64%85.08%-6.66%

7th Grade Alabama Writing Assessment Gains Hi-performing school Comparison school 1 Comparison school % 28.50% -2.17% -7.05% 43.79% 35.55% Makes Sense school 37.65%60.51%22.86% 92.64%85.08%-6.66% Hi-performing school Comparison school 1 Comparison school % 3.13% -3.79% -3.39% 19.35% 6.52% Makes Sense school 14.21%20.00% 5.79% 64.00%65.00% 1% Hi-performing school Comparison school 1 Comparison school % 23.40% -7.19% % 34.69% 35.55% Makes Sense school 27.08%43.83%20.75% NA

High School 10th grades (semi-rural) % students meeting or exceeding standards MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 8 53% MSS High School 77% 26 pt gain! 2 pt gain Non-MSS High School MSS implementation 51% Both groups performed at the same levels in

Action Research by Schools Alabama Writing Assessment % students meeting standards Gains Differentiating Making information more learnable Using brain- based pedagogy curriculum 32.71%57.84%25.13% Rural 7th Grade Suburb 7th Grade 38% 61%23.00%

96% POST 84% POST 85% PRE Control group of 6th grade typical achievers not receiving MSS 84% PRE Experimental group of 6th grade typical achievers receiving MSS 54% PRE Experimental group of 6th grade students w/LD receiving MSS 12 pt gain 87% POST 33 pt gain MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 9 Impact on vocabulary test scores 6th grade inclusion class

Concept test scores of students with LD Before After Gains 79% 92% 93% 14 pts 30 pts High Soc Economic 6 th grade 62% 63% 65% Mid Soc Economic 4/5 th grade Low Soc Economic 4/5 th grade MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 10

97% POST 81% PRE 11 TH grade typical achievers 71% PRE 11 th grade students w/LD 17 pt gain 26 pt gain 98% POST MAKES SENSE STRATEGIES 4 Teaching & Learning STUDY 11 Impact on concept test scores 11th grade inclusion social studies class