1. Teachers' Use of Classroom Connectivity Technology: Log File Analysis Vehbi Aytekin Sanalan sanalan.1@osu.edu

2. 3/7/2008SITE 20082 Project team Principal Investigators Douglas T. Owens, OSU Stephen L. Pape, UF Karen E. Irving, OSU Louis Abrahamson, BEF Frank Demana, OSU Post doctoral researcher Vehbi Sanalan, OSU & Erzincan University, Turkey Consultants Christy Kim Boscardin, UCLA Joan Herman, UCLA Jeremy Roschelle, SRI Project Coordinator Mike Kositzke, OSU GRAs Sedat Ucar; Sukru Kaya; Melissa Shirley; Clare Bell; Ugur Baslanti; Sharilyn Owens; Hyesook Shin; Gonul Sakiz; Jonathan Bostic

3. 3/7/2008SITE 20083 CCMS Project Overview The research reported here was supported by the Institute of Education Sciences, U.S. Department of Education, through Grant R305K050045 to The Ohio State University. The opinions expressed are those of the authors and do not represent views of the U.S. Department of Education. Interdisciplinary professional development and research project Algebra I and Physical Science achievement Student self regulated learning skills Teaching strategy change in formative assessment practices Summer Institute – training T 3 conference follow-up (2 years) Classroom connectivity technology

4. 3/7/2008SITE 20084 Classroom Connectivity Technology Create a collaborative learning environment Engage in formative assessment by way of immediate feedback Enhance classroom management of TI graphing technology Quick Poll provides teacher understanding by receiving impromptu feedback The TI-Navigator System allows the teacher to:

5. 3/7/2008SITE 20085 Technology Components Quick Poll Screen Capture Activity Center Class Analysis/Learning Check

6. 3/7/2008SITE 20086 Implementation Fidelity Control/Treatment Group assignment is dichotomous Implementation is usually not uniform Data collection for implementation Teacher reports Observations Software tracing Issues with each procedure

7. 3/7/2008SITE 20087 Log File Literature Analysis of user tracing documents can provide information about actual use of technology (Saye & Brush, 2001; Nurmela, 1999; Reips & Stieger, 2004) It fails to provide detailed understanding of educational settings (Boving & Simonsen, 2004; Nyvang & Bygholm, 2005; Owston, 2000) Standard logging mechanism Timestamp of the event Type of event Source of event

8. 3/7/2008SITE 20088 Log File Structure Participant Teacher Time and means of submission Class session Class name Starting and ending time of the class Number of students logged in Component type Starting & ending times Number of Ss enrolled Quick Poll - question probe Class Analysis - test name Activity Center - type of info exchanged Screen Capture

9. 3/7/2008SITE 20089 Log File Structure Class and activities scheme School allocated time for class session (X=59’55, sd. 25’29) Class session duration from log files (X=52’38, sd. 13’58) Average QuickPoll usage (X=2’58, sd. 14’26) Average Activity Center Usage (X=4’31, sd. 7’58) Average ClassAnalysis usage (X=44’41, sd. 39’56)

10. 3/7/2008SITE 200810 Log File Data Preparation Preparing raw data for analysis Date and time limits Number of students Excessive class duration Activities without students

11. 3/7/2008SITE 200811 Phone Interview Data Rating Teachers were asked to; estimate the number of times they used the technology per week rate their use of each component (never, sometimes, often and every class session) Team of researchers used a coding scheme to establish the inter-rater reliability and finalized the rate for each teacher.

12. 3/7/2008SITE 200812 Congruence Average weekly use information from different sources are correlated (R=.466, p≤.001) but significantly different (t 44 =3.43, p ≤.001) Component use information is also correlated for Class Analysis and Screen Capture (R=.620 and.380)

13. 3/7/2008SITE 200813 Results - Participants Out of 98 total participants 50 teachers from 28 (21) different states Gender Female: 75% (76%) Male:25% (24%) Ethnicity White 87% (92%) African-American 9% (4%) Hispanic 4% (4%) First year vs. second year users 49% (50%) No significant difference between subgroups on average use of technology

14. 3/7/2008SITE 200814 Results – Total Use Average use per weekNPercentage (%) less than once1020 1-2 times1836 3-4 times918 More than 5 times1326

15. 3/7/2008SITE 200815 Results – Component Use

16. 3/7/2008SITE 200816 Results – Student Participation ComponentPercent Overall91% Quick poll85% Activity center93% Class analysis97%

17. 3/7/2008SITE 200817 Results – Tech Use Over a Year

18. 3/7/2008SITE 200818 Next Steps Analysis of content and log files in conjunction with classroom observations Include log file analysis in the implementation fidelity index derived from telephone interviews HLM analysis using log file data set

19. Thank you very much! For electronic copy of this presentation: http://www.ccms.osu.edu http://www.ccms.osu.edu or ask sanalan.1@osu.edusanalan.1@osu.edu (Please allow a couple of days until it shows up on the web)