1. Advancing Assessment Literacy Data Gathering IV: Collecting and Collating Data

2. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 2 Data Sources It is important to think about data sources as questions are being created. For some questions data may already be available, for others data may need to be collected. Data can be collated in a variety of ways. Some of the most accessible data collation and display tools are quality control tools such as run charts, scatter plot diagrams, and histograms.

3. 3 Run Charts Run charts are simple tables to gather and display data in one area over time. For example, data might be gathered on quizzes, number of students on time, number of students who participated in the breakfast program each week day, etc. Run charts enable analysis of trends over time. Assignment # Number Correct 1 2 3 4 5 6 7 8 1 2 3 4 5

4. 4 Scatter Plot Diagrams Scatter plot diagrams are used to collect and display data on performance, attainment, or usage by number of subjects. This is useful for charting the progress or actions of a group of students. Assignment # Number Correct 1 2 3 4 5 6 7 8 1 2 3 4 5

5. 5 Histogram A histogram is a means of collecting and displaying detailed data regarding the number of people who have attained a certain level of achievement or the frequency of an action. A histogram provides a clear representation of the distribution of data across a population group. Number of Questions Correct Number of People 1 2 3 4 5 6 7 8 1 2 3 4 5

6. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 6 Four Major Categories of Data Demographics –Descriptive information such as enrollment, attendance, ethnicity, grade level, etc. –Can disaggregate other data by demographic variables. Perceptions –Provides information regarding what students, parents, staff, and community think about school programs and processes. –This data is important because people act in congruence with what they believe. Student Learning –Describes outcomes in terms of standardized test results, grade averages, etc. School Processes –What the system and teachers are doing to get the results they are getting. –Includes programs, assessments, instructional strategies, and classroom practices. Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education.

7. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 7 Selecting Data Sources On the supplied template, indicate which data sources would best match each of the four categories of data. DemographicsPerceptions Student Learning School Processes

8. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 8 Other Data Sources Questionnaires Student Profiles CAT3 Assessment for Learning PISA, TIMSS Interviews Surveys Portfolios Classroom Assessments Archival Data –Previous Marks –Course Selection Demographic Data

9. Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. Demographics Enrollment, Attendance, Drop out Rate, Gender, Grade Perceptions Perceptions of Learning Environment Values & Beliefs Attitudes Observations Student Learning Standardized Tests, Norm/Criterion Referenced Tests Teacher Observations, Authentic Assessment School Processes School Programs And Processes

10. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 10 Triangulations Triangulation is the use of multiple data sources and types (quantitative & qualitative) to increase the validity of results. If two or more different data sources or types are giving the same information, it is more likely that what is being witnessed is true. Wellman, B. & Lipton, L. (2004). Data-driven dialogue. Mira Via, LLC.Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. The shaded area is the most valid.

11. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 11 Applying Data Collection and Triangulation Using the goal statements and initial questions previously created, complete a more detailed analysis thinking about the four categories of data available – demographics, perceptions, student learning, and school processes. You will be furnished with a data intersections template.

12. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 12 Refining Questions Take the initial questions and write them in the questions column. In the left column, use the diagram to identify the intersections or triangulations implied within the question. –What other intersections would increase the specificity of this question? –If necessary, rewrite the question to reflect the new intersections or triangulations. –Including an “over time” element to a question expands the data that might be accessed to answer the question. For each question identify the existing data source available or what tool will be required to collect it. Create new questions using a variety of the intersections or identified earlier.

13. 13 Original: In what classes is representation best taught? What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. Goal Statement: Teachers will better actualize the representation strand of the curriculum. Place the question in the questions column

14. 14 Original: In what classes is representation best taught? (Demographics (D) by Perception (P) ) What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. Goal Statement: Teachers will better actualize the representation strand of the curriculum. Identify the intersection within the question

15. 15 Original: In what classes is representation best taught? (D by P) Intersecting or triangulating with School Processes (SP) will add clarity What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. Goal Statement: Teachers will better actualize the representation strand of the curriculum. What other intersections would increase specificity?

16. 16 Original: In what classes is representation best taught? (D by P) Modified: In what classes is representation best taught and what methods are being used? (D by P by SP) What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. Goal Statement: Teachers will better actualize the representation strand of the curriculum. Rewrite the question to reflect the new triangulation

17. 17 Anecdotal data AFL data Lesson Plans Anecdotal Data AFL data Original: In what classes is representation best taught? (D by P) Modified: In what classes is representation best taught and what methods are being used? (D by P by SP) What data is available or needed to answer the question? What data collection tools will be required? Questions Intersections (Colour in and name the intersection) D P SL SP Adapted from: Bernhardt, V. L. (2004). Data analysis for continuous school improvement, 2 nd Edition. Larchmont, NY: Eye on Education. Goal Statement: Teachers will better actualize the representation strand of the curriculum. Identify Data Sources

18. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 18 Refining and Finalizing Questions Complete the process with existing questions and create new ones. When finished, evaluate the quality of the questions then decide which should go forward and which should be abandoned. On the sheet provided, write the goal statement and the refined questions that your group has decided to keep. These questions will be used to gather data as more in-depth goal statements are created.

19. Advancing Assessment Literacy Modules: Data Gathering IV (February 2008) 19 Reflection In what ways did this process refine the questions? Were any initial questions eliminated? Why?