Assessment of ICTs in Classroom Instruction Sarah McPherson New York Institute of Technology NECC 2009 Washington DC

Purpose To examine tools for assessing use of technology in the classroom To examine tools for assessing use of technology in the classroom To determine the validity of the instruments To determine the validity of the instruments To align the assessment with the professional development To align the assessment with the professional development To identify how the assessment results can be used to improve the professional development To identify how the assessment results can be used to improve the professional development To recommend tools and strategies for assessing effective uses of ICTs in instructional settings. To recommend tools and strategies for assessing effective uses of ICTs in instructional settings.

Background A STEM MSP grant in the Bronx To prepare teachers to use technology in STEM teaching and learning To prepare teachers to use technology in STEM teaching and learning To increase participants knowledge in STEM content To increase participants knowledge in STEM content To provide instructional strategies for teaching STEM in middle school and high school environments To provide instructional strategies for teaching STEM in middle school and high school environments To assess the extent of use of ICTs in STEM classrooms To assess the extent of use of ICTs in STEM classrooms

Perspective/Theoretical Framework Partnership for 21 st Skills Partnership for 21 st Skills education and society education and society education and learning sciences education and learning sciences education and learning tools education and learning tools

Partnership for 21 st C Skills Partnership for 21 st C Skills

Classrooms What do or could they look like? A Kansas classroom What do or could they look like? A Kansas classroom What do or could they look like? What do or could they look like? No Future Left Behind A New York project No Future Left Behind A New York project No Future Left Behind No Future Left Behind

Assessment of 21st Century Skills Route 21 Supports a variety of assessments Supports a variety of assessments Emphasizes useful feedback on student performance Emphasizes useful feedback on student performance Requires a balance of technology-enhanced, formative and summative assessments that measure student mastery Requires a balance of technology-enhanced, formative and summative assessments that measure student mastery Enables development of portfolios of student work to demonstrate high levels of student competency Enables development of portfolios of student work to demonstrate high levels of student competency

Route 21 Assessment Resources Route 21 Assessment Resources

The Reformed Teaching Observation Protocol (RTOP) The Reformed Teaching Observation Protocol (RTOP)

Categories in the Protocol Lesson Design and Implementation Lesson Design and Implementation Content: Propositional Pedagogic Knowledge Content: Propositional Pedagogic Knowledge Content: Procedural Pedagogic Knowledge Content: Procedural Pedagogic Knowledge Classroom Culture: Communicative Interactions Classroom Culture: Communicative Interactions Classroom Culture: Student/teacher Relationships Classroom Culture: Student/teacher Relationships

Characteristics of the learning environment and students Characteristics of the learning environment and students Teachers actions, (i.e. lecturing, discussion, interactive directions, etc.) Teachers actions, (i.e. lecturing, discussion, interactive directions, etc.) Student learning activities (i.e. creating presentations, researching, writing, drill and practice, etc.) Student learning activities (i.e. creating presentations, researching, writing, drill and practice, etc.) Technologies used by teacher and by students Technologies used by teacher and by students Time interval chart of use Time interval chart of use NETS – T addressed NETS – T addressed ISTE Classroom Observation Tool (ICOT) ISTE Classroom Observation Tool (ICOT)

ICOT Classroom Observation Classroom Observation Classroom Observation Classroom Observation

Other Assessment Tools Technology Inventories Technology Inventories checklists of available technologies checklists of available technologies teachers comfort level teachers comfort level E-Portfolios E-Portfolios Student work samples Student work samples Alignment with standards Alignment with standards Rubrics Rubrics

Questions to Consider How can classroom observation tools can be used to evaluate technology use in the classroom? How can classroom observation tools can be used to evaluate technology use in the classroom? How classroom assessment tools align with professional development (graduate and non-credit)? How classroom assessment tools align with professional development (graduate and non-credit)?

Grade Technology AvailableTechnology and barriers K Computer lab wiki SmartBoard, projector Filter blocked animal sounds 3 Sp Ed Laptop cart/projector website with video Directions in unison 3 Smartboards, webquests, laptopsPop-ups, Search skills 5 Inclusion Smartboard and Video on Internet Calibration of Smartboard Computer freezing 6LCD projector, laptop, PowerPointNo access to own laptop 7 Smartboard, laptops, overhead projector, flash video Video conferencing 7/8 Sp Ed Laptops on cart Google Images Hands-on – image as model 9 Pasco Probes/ Graphing Laptop, Smart Board, TI 83s, projector Computers to use with probes and calculators 10 Teacher station projector PowerPoint with Internet Note taking

Technology Inventory Results Access to desktop and/or laptop computers with internet access, SmartBoards, scanners, televisions, VCRs/DVD players, overhead and LCD projectors, printers, cameras, video cameras and math/science- related educational software. Access to desktop and/or laptop computers with internet access, SmartBoards, scanners, televisions, VCRs/DVD players, overhead and LCD projectors, printers, cameras, video cameras and math/science- related educational software. Access to laboratory equipment, such as calculators and thermometers, water supply, scales/balances, graduated cylinders/flasks/beakers, microscopes. Access to laboratory equipment, such as calculators and thermometers, water supply, scales/balances, graduated cylinders/flasks/beakers, microscopes.

Administrators Responses Difficulties in technology integration is due to teachers' low level of technology skills teachers' low level of technology skills Lack of knowledge and skill for integrating technology with content Lack of knowledge and skill for integrating technology with content Lack of time to explore resources and appropriate uses of technology Lack of time to explore resources and appropriate uses of technology

Limitations Actual classroom environment Actual classroom environment Student population Student population Classroom management practice Classroom management practice Technology available Technology available

Recommendations Regular access to technology for preparation Regular access to technology for preparation Access to the Internet Access to the Internet Reliable equipment Reliable equipment Professional development Professional development

Educational Importance Need for appropriate classroom observation metrics to assess implementation of ICTs Need for appropriate classroom observation metrics to assess implementation of ICTs RTOP lacks specific probes for technology use RTOP lacks specific probes for technology use Technology inventory is not real time and is self- report Technology inventory is not real time and is self- report Route 21 requires research, some are commercial and state-specific Route 21 requires research, some are commercial and state-specific ISTE ICOT Classroom observation, technology use in context ISTE ICOT Classroom observation, technology use in context Teacher and student portfolio and reflection on what worked Teacher and student portfolio and reflection on what worked

References Arizona Collaborative for Excellence in the Preparation of Teacher (2000). Reformed Teaching Observation Protocol (RTOP). Bloom, B. S. (1956). (Ed.) Taxonomy of educational objectives: the classification of educational goals. Susan Fauer Company, Inc. Dewey, J. (1980). The school and society. ISTE Classroom Observation Tool (ICOT), retrieved October 2, 2008 at Partnership for 21 st Century Skills (2007). 21 st century skills assessment. Piaget, J. (1928). The childs conception of the world. Routledge and Kegan Paul, London. Vygotsky, L. (1978). Mind in society: Development of higher psychological processes. Harvard University Press. Cambridge, MA.

Contact Information Sarah McPherson