EVALUATING WEST VIRGINIAS EETT MODEL SCHOOLS PROJECT NECC 2006 San Diego, CA

West Virginia Statewide Educational Technology NAEP 50-State Measures of Student Computer Use (SOURCE: Jonathan Becker, Ph.D., Professor, Hofstra University, Hempstead, NY) *Only one state has had its students use of computers at the highest interval (a lot) exceed the national average in every assessed year from 1997 to 2003 (p = <.05) *That consistently high-use state is West Virginia.

West Virginia Statewide Achievement Gains Q: Can computer use improve student learning? A: YES

From BS/CE to TMS Question: When does technology improve achievement? Answer: When it is used by teachers and students. Question: How can more teachers be encouraged to use more technology? Answer: In-school technology integration specialists (TISs, the Technology Model Schools Program)

A Study Question of National Interest Question: Did the in-school, TIS strategy work and how do we know? Answers: Teacher computer useTeacher computer use Student computer useStudent computer use Achievement gainsAchievement gains

RESEARCH DESIGN

Experimental/Control & Pre/Post EXPERIMENTAL With TIS specialist CONTROL Without TIS specialist PRE Data at beginning of TIS year POST Data at end of TIS work with teachers and beyond

Using Technology To Measure Technology Meters in desktopsMeters in desktops –Software runs in the background 24/7 –Records file activity only –Installed on classroom computers only –Data ed directly to Interactive, Inc. –ZERO burden on teachers Pagers on teachers + web surveys –Free pager rings once per day, every other week –When pager rings, teacher and a randomly selected student take a web-survey –Pages are randomly scheduled during alternate weeks (not more than 5 per week)

USING TECHNOLOGY TO STUDY TECHNOLOGY: How well did it work (Year 1 – [ ])?

Pagers + Web-Surveys 2,100 pager-triggered responses from teachers.2,100 pager-triggered responses from teachers. 2,000+ responses from students.2,000+ responses from students. Responses from 110 different teachers.Responses from 110 different teachers. Responses from individual teachers ranged from 36 to 1.Responses from individual teachers ranged from 36 to 1.

Data from Metering Software Data from 184 computers in 25 schools.Data from 184 computers in 25 schools. Installed on 287 computers.Installed on 287 computers. 2 reports a day X 184 computers X 5 months2 reports a day X 184 computers X 5 months = 35,000 reports of computer use= 35,000 reports of computer use

RESULTS: Teacher and Student Computer Use (Web-Based Survey Data – Year 1 [ ])

More Teacher Computer Use In TIS Schools % of Yes Responses Treatment (n=933) 21 Control (n=646) 11 At the time the pager was activated, were YOU using computers?

More Teacher Productivity Use In TIS Schools Treatment % Control % Word Processing 3025 PowerPoint30 Spreadsheets103 MS Publisher 10 Compass / Riverdeep 43 Internet research Other2515 How were you using your computer?

More Student Computer Use In TIS Schools % Yes Technology Model Schools (n=941) 24% Comparison schools (n=649) 22% Another jurisdiction 0.1% At the time the pager was activated, were any of your students using computers?

RESULTS: Student Achievement (Year 1 [ ])

RESULTS: Achievement Gains Q: Can computer use improve student learning? A: YES: 4 th and 5 th Grade students in the TIS schools made (statistically significantly; p<.05) greater score gains (compared to similar students in non- TIS schools) on the states Math WESTEST ( ).

Summary (Year 1): IN THE TIS SCHOOLS Students used computers moreStudents used computers more Teachers used computers moreTeachers used computers more Teachers used computers more for productivity applicationsTeachers used computers more for productivity applications There were statistically significant gains in WESTEST Math scores from students in the 4 th and 5 th grades.There were statistically significant gains in WESTEST Math scores from students in the 4 th and 5 th grades.

Coming Attractions Year 2 [ ]

National (Year 2) Study Questions Since the TIS specialists completed their funding year (June 2005), what happened? Did teachers continue to use technology? Did achievement gains hold?Since the TIS specialists completed their funding year (June 2005), what happened? Did teachers continue to use technology? Did achievement gains hold?

Year 2 Data Collection Success 337 teacher responses to pager-inspired Web- surveys337 teacher responses to pager-inspired Web- surveys 297 student responses to pager-inspired Web- surveys297 student responses to pager-inspired Web- surveys 134 (of a possible 182) teachers in 48 schools completed the end-of-year questionnaire134 (of a possible 182) teachers in 48 schools completed the end-of-year questionnaire 45,434 s with data from TAM-monitored computers45,434 s with data from TAM-monitored computers 21 schools visited during academic year schools visited during academic year

(Very) Preliminary Conclusions Most places are: – –getting most teachers to be familiar with different applications – –and getting some teachers to mastery. West Virginia's multi-year effort has produced: – –near universal mastery with – –many teachers as fluent users. West Virginia's fluent users: – –pick and choose the strongest features for their student needs – –invent new ways to use technology – –emphasize small group and individual work. – –work collaboratively with other teachers to develop local applications.

Questions? Interactive, Inc. 61 Green Street Huntington, New York phone: fax: e: