Data Collection Considerations October Support Session

Welcome and Introductions Mary Russell Greg McKnight Dr. Jody Cleven

Norms of Virtual Support Minimize outside distractions. Be attentive and participate in all activities. Keep yourself muted when not talking. Notify the presenter if you are having technical difficulties by raising your hand. Use the chat bar on your dashboard to ask questions.

Meeting Outcomes During today’s session you will: review content from the September webinar; review upcoming module requirements and due dates; discuss data collection considerations - validity, reliability, and generalizability; and Identify the connections between validity, reliability, and generalizability and formative data.

Upcoming Module Requirements

Where Are You? All the module requirements complete and submitted. I feel confident Module requirements complete, but not submitted. I am feeling a little overwhelmed. I am behind and have questions. I am worried

A Video Tutorial

Triangulation

Questions to Consider 1. Who are the participants? 2. What data will be collected and from whom will it be collected? 3. Is this an area over which you control and can make changes? 4. Are the intended outcomes measurable? How will the researcher measure them? 6. What does the researcher need to do in preparation for data collection – are there any skills that the researcher or his/her participants need to have before this change or innovation? 7. What do you expect to happen? Have you considered unintended outcomes?

Our Focus Today Validity Reliability Generalizability Formative assessment connections  Let’s start with a brief pre-assessment about validity. Use the chat bar to post your answers to the following questions.

True or False Attention to validity, reliability, and generalizability will help ensure the quality of your action research.

True or False Internal validity refers to the degree to which results are not true for the participants in your study.

True or False External validity is the degree to which your study results are generalizable or applicable to groups outside your research setting.

True or False Internal validity refers to the degree to which your results are not true for the participants in your study.

Validity According to Mills (2014), validity is “how we know that the data we collect accurately gauge what we are trying to measure (p. 114).” Serves to convince the researcher that the results of the study are right, accurate, and could stand scrutiny from other researchers.

Ensuring Validity in Your Study Talk little; listen lot Record observations accurately Begin writing early Let readers see for themselves Report fully Be candid Seek feedback Write accurately

True or False Reliability is the degree to which a test inconsistently measures whatever it measures.

Reliability According to Mills (2014) the English definition of reliability; dependability and trustworthiness essentially means the same thing when collecting data for your action research plan. The more reliable a test it, the more confidence others will have in implementing your plan for the same results.

Reliability vs. Validity Reliability “The degree to which a test consistently measures whatever it is measuring” Validity “The degree to which a test measures what it is supposed to measure” “A reliable test may consistently measure the wrong thing.” “A valid that measures what it purports to measure will do so over time.” Mills, 2014

Generalizability According to Mills (2014) generalizability refers to the “applicability of findings to settings and contexts different from the one in which they were obtained (p. 120).” Tries to explain the behavior of a wider group of people based on the behavior of a small group of individuals.

Ensuring Generalizability in Your Study Limit personal bias – Conduct research in a systematic and disciplined manner – Remain objective – Reflect on what you actually see and not what you want to see It is easy to validate our existing practices so be mindful of the data you discard or how you triangulate data to support your thinking.

How does this Relate to Formative Assessment? What is formative assessment? “Formative assessment is a process used by teachers and students during instruction that provides feedback to adjust ongoing teaching and learning to improve intended instructional outcomes.” (CCSSO FAST SCASS, 2006).

The Formative Assessment Model

Formative Assessment Strategies Exit ticket My Favorite No Thumbs-up, Thumb-down Four Corners More strategies can be located here:

What types of FA are you using? When sharing, please address these questions. What is the formative assessment strategy? What have you discovered about your students’ learning? Has it been necessary to modify your innovation as the result of your formative assessment? Do you feel that the data is reliable, valid and can be generalized?

Questions?

Thank you for joining us! Our next webinar is scheduled for November 20 th. Your coach will the registration link will be ed to you.