1. PS397 Psychological Measurement & Testing

2. Educational Testing Service (ETS) Administers more than 11 million tests annually in 181 countries Administers more than 11 million tests annually in 181 countries Tests include: Tests include: SATSAT GREGRE

3. For the Love of Learning: Report of the Royal Commission on Learning, Ontario, 1995 Various forms of assessment have shown that those who are poor, members of some minority groups, or who are female perform less well than their knowledge or skills would warrant. Some communities complain that their students have been negatively streamed because of biased assessments. For example, more than a decade ago, a York University symposium on racial and ethnic relations in city school boards was told by Marcela Duran that “we were able to institute an experimental program, in co- operation with the Jamaican-Canadian Association, in which 100 West Indian children who had been placed in vocational schools were re-assessed, using different testing instruments. According to this process, 90 of these students were found to have been wrongly placed.”

4. Our students are tested to an extent that is unprecedented in American history and unparalleled anywhere in the world. Politicians and businesspeople, determined to get tough with students and teachers, have increased the pressure to raise standardized test scores. Unfortunately, the effort to do so typically comes at the expense of more meaningful forms of learning. That disturbing conclusion emerges from Alfie Kohn's devastating new indictment of standardized testing. Drawing from the latest research, he concisely explains just how little test results really tell us and just how harmful a test-driven curriculum can be.

5. Alfie Kohn’s Critique of Testing high scores often signify relatively superficial thinking a school that improves its test results may well have lowered its standards to do so far from helping to "close the gap," the use of standardized testing is most damaging for low-income and minority students as much as 90 percent of the variations in test scores among schools or states have nothing to do with the quality of instruction far more meaningful measures of student learning - or school quality - are available. Kohn's central message is that standardized tests are "not like the weather, something to which we must resign ourselves.... They are not a force of nature but a force of politics-and political decisions can be questioned, challenged, and ultimately reversed."

6. The EQAO (Education Quality & Accountability Office) Created in 1996 to oversee the provincial testing program instituted at that time Created in 1996 to oversee the provincial testing program instituted at that time All grade 3 & 6 students tested in reading, writing & math All grade 3 & 6 students tested in reading, writing & math All grade 9 students tested in mathematics All grade 9 students tested in mathematics All grade 10 students tested in literacy All grade 10 students tested in literacy Current budget: $50 million annually, over $250 million over the last five years Current budget: $50 million annually, over $250 million over the last five years

7. Ontario English Catholic Teachers Association, March 2002 How many schools would $250 million build? How many repairs would it accomplish on existing buildings? How many library books would it buy? Could every child have their own copy of their textbooks? How many assistants could there be for children with special needs? How many schools would $250 million build? How many repairs would it accomplish on existing buildings? How many library books would it buy? Could every child have their own copy of their textbooks? How many assistants could there be for children with special needs?

8. Other Criticisms EQAO does not publish detailed technical reports along with the results EQAO does not publish detailed technical reports along with the results The precise statistical procedures that were being used, the degree of reliability of markers evidenced in their scoring, and the confidence intervals of the scores, remain undisclosed The precise statistical procedures that were being used, the degree of reliability of markers evidenced in their scoring, and the confidence intervals of the scores, remain undisclosed

9. Course Objectives Help you understand some of the fundamental principles underlying psychological measurement and testing. Provide you with experience in constructing and evaluating psychological tests and measures. Familiarize you with many of the more commonly used psychological tests. Acquaint you with some of the important ethical and political issues relating to psychological testing.

10. Text Kaplan, R.M. & Sacuzzo, D.P. (2005). Psychological testing: Principles, applications, and issues (sixth edition). Pacific Grove, CA: Brooks/Cole. Additional readings may be assigned for some of the classes.

11. Course Requirements Brief assignments & class participation (10%) Brief assignments & class participation (10%) Mid-term test (30%) Mid-term test (30%) Test construction project (20%) Test construction project (20%) Final exam (40%) Final exam (40%)

12. Lectures & Course Components The course is divided into 2 parts: The course is divided into 2 parts: Part I (up until the mid-term)Part I (up until the mid-term) Principles of testing, psychometrics & test- related statistics, reliability, validity, test construction techniques Principles of testing, psychometrics & test- related statistics, reliability, validity, test construction techniques Part II (after the mid-term)Part II (after the mid-term) Applications & the ethics & politics of testing Applications & the ethics & politics of testing Review of major tests of intelligence, vocational interest, personality, etc. Review of major tests of intelligence, vocational interest, personality, etc.

13. Lectures & Class Notes For Part I of the course, I will not be using PowerPoint to present the course material For Part I of the course, I will not be using PowerPoint to present the course material For Part II of the course, I will be using PowerPoint, and the notes will be available from my web-site For Part II of the course, I will be using PowerPoint, and the notes will be available from my web-site

14. Key Dates February 8Test construction session February 15Mid-term test February 19-23Reading week March 1Group-constructed tests due for printing March 8Test administration session March 29Test report due April 9 - 27Exam period

15. Testing: Key Terms Test a measurement device or technique used to quantify behaviour or aid in the understanding & prediction of behaviour Item a specific stimulus to which a person responds overtly; this response can be scored or evaluated (for example, classified, graded on a scale or counted); the specific questions or problems that make up a test Psychological Test a set of items designed to measure characteristics of human beings that pertain to behaviour (overt or observable, or covert, such as feelings, thoughts & attitudes)

16. Key Terms (continued) individual tests given to one person at a time group tests administered to more than one person at a time

17. Types of Tests Ability Tests (measure skills in terms of speed, accuracy) Ability Tests (measure skills in terms of speed, accuracy) Achievement (measures previous learning)Achievement (measures previous learning) Aptitude (measures potential for acquiring a specific skill)Aptitude (measures potential for acquiring a specific skill) Intelligence (measures potential to solve problems, adapt to changing circumstances, profit from experience)Intelligence (measures potential to solve problems, adapt to changing circumstances, profit from experience)

18. Types of Tests (continued) Personality tests (measure typical behaviour – traits, temperaments, dispositions) Personality tests (measure typical behaviour – traits, temperaments, dispositions) Structured (objective) – provides a self- report statement to which a person responds “true” or “false”, “agree” or “disagree”Structured (objective) – provides a self- report statement to which a person responds “true” or “false”, “agree” or “disagree” Projective – provides an ambiguous test stimulus; response requirements are unclearProjective – provides an ambiguous test stimulus; response requirements are unclear

19. Individual Differences Most tests are designed to discover differences among individuals (“individual differences”) Most tests are designed to discover differences among individuals (“individual differences”) Basic assumption of testing is that differences in test results reflect real differences in individuals Basic assumption of testing is that differences in test results reflect real differences in individuals E.g., individuals who score higher on an IQ test assumed to be more intelligent E.g., individuals who score higher on an IQ test assumed to be more intelligent

20. A Brief History of Testing Chinese civil service tests (1368 to 1644 C.E.) Chinese civil service tests (1368 to 1644 C.E.) Darwin, Galton & individual differences (mid to late 1800’s) Darwin, Galton & individual differences (mid to late 1800’s) Binet & the intelligence test (1905) Binet & the intelligence test (1905) Stanford Achievement Test (1923) Stanford Achievement Test (1923) Sir Francis Galton Alfred Binet

21. History of Testing (cont’d) Personality tests (1930’s & 1940’s) Personality tests (1930’s & 1940’s) George Gallup & the opinion poll (1935) George Gallup & the opinion poll (1935) Hermann Rorschach (1884-1922) George Gallup (1901-1984)