1. Outline Ethics vs. Law Ethical use of tests – issues: Competence
Informed Consent
Knowledge of Results
Confidentiality
Test security
Divided Loyalties

2. Outline Professional Issues Social Issues Theoretical issues
Test adequacy
Actuarial vs. clinical prediction
Social Issues
Dehumanization
Access to benefits of psychological testing

3. Outline 6. Two examples of bad tests
Handwriting analysis
TOEFL
7. The CPA Code of Ethics (synopsis)
Respect for the dignity of persons
Responsible caring
Integrity in relationships
Responsibility to society

4. Ethics vs. Law
Ethics: what one should or should not do, according to principles or norms of conduct
Ethics codes are not produced by democratically-elected legislatures
Enforcement mechanisms are usually informal, may be complex, even unconscious

5. Ethics vs. Law
Law: what one must or must not do, according to legal dictates
Laws are created by democratically-elected legislatures in democracies such as Canada
Laws come with explicit penalties for infractions and a formal enforcement system
Legislatures that are not democratically-elected, in dictatorships, do not pass real laws of this sort

6. Ethical use of tests – issues:
Competence
Informed Consent
Knowledge of Results
Confidentiality
Test security
Divided Loyalties

7. 1. Competence develop competence in assessment concepts and methods
recognize boundaries of your competence
competence is very important if test result affects someone’s life
Any one engaged in psychological testing has a very important responsibility: to be competent. The importance of this responsibility varies with the importance of the test to the test-taker’s life. Thus, if the test is provided in a popular magazine essentially for entertainment purposes, a sensible person would not take it seriously and would not make life-altering decisions on the basis of the test result. In that case, the people developing or publishing the test have less responsibility, so long as they are careful to explain that the test is for entertainment purposes only.
In contrast, if the test is given as part of a meaningful process, such as an employee selection procedure, a university admissions procedure, or an assessment of an accused person’s ability to understand legal proceedings against them, then the responsibility is significant. It includes competence in relevant technical areas. The test-developer and the test-giver must be competent in assessment concepts and methods and they must be able to recognize the boundaries of their competence (that is, to know what they know and don’t know).

8. 2. Informed Consent
Generally, the client/subject must voluntarily consent
psychologist must inform client about nature and purpose of assessment in understandable language

9. Informed consent - exceptions
Sometimes, it is acceptable to test without getting consent – e.g., you have given implied consent to be tested by registering in this course, but I have never sought your explicit consent
Sometimes it is necessary to test without getting consent, even when consent is explicitly refused by person to be tested – e.g., when mandated by law

10. 3. Knowledge of Results
must fully disclose test results in understandable language
but do not do so in a way that reveals the content of test items, because that content is proprietary information (e.g., somebody owns it – it’s not yours to reveal)
From the joint CPA / test publishers’ statement: “To provide clients with test items, scoring criteria, and other test protocols would be to reveal confidential commercial information on which the scores are based and would render the Test Materials useless.”

11. Constraints on client access to results
PIPEDA Principal 4.9:
Upon request, a person must be informed of the existence, use, and disclosure of his or her personal information and must be given access if requested.
Section 9.3b says that access can be denied if it would mean revealing confidential commercial information (unless latter is severable).

12. Constraints on client access to results
Ontario Personal Health Information Act: access rights do not apply to a record that contains raw data from standardized psychological tests or assessments, unless those data are reasonably severable.
CPA and leading Canadian test publishers say that Test Materials are not “personal information” so should not be released.
CPA and test publishers also argue that their Test Materials are proprietary, copyrighted, confidential commercial information, analogous to trade secrets, and so refuse to release them – these materials fall under a PIPEDA exemption. Similar approach taken by test publishers in the US.

13. 4. Confidentiality Test results are confidential information
Release of results should only be made to another qualified professional after client’s consent

14. 5. Test Security Test materials must be kept secure
Test items are not revealed except in training programs and when mandated by law, to protect test integrity
Test items are private property

15. 6. Automated Scoring/Interpretation Systems
Psychologist is still responsible for proper interpretation of test results
Psychologists are professionals – they take personal responsibility for their work
The main difference between a professional and a non-professional is that professionals take personal responsibility for their work.

16. 7. Divided loyalties Who is the client?
The person being tested, or the institution you work for?
What if these parties have conflicting interests?
How do you maintain test security but also explain an adverse decision?

17. Divided loyalties
Suppose you are asked to test a child who might have disorder X or Y
Suppose X is very common and so the government provides resources to help schools deal with it
Suppose Y is not common and no resources are available in school to deal with it
If your testing suggests Y, will you report Y – or X, so the child gets some sort of help?

18. Professional Issues Theoretical issues Test adequacy
Actuarial vs. clinical prediction

19. 1. Theoretical Issues
Are you measuring a stable characteristic of the person being tested?
If so, differences in scores over time reflect measurement error or subject variables such as fatigue
Personality is about things that don’t change – but humans are great learners, and learning should change our responses.

20. 1. Theoretical Issues
Are you measuring a stable characteristic of the person being tested?
In fact, we have little evidence that this is true, so what is the value of your test result – will it still be true next year?
Especially problematic for personality tests
Personality is about things that don’t change – but humans are great learners, and learning should change our responses.

21. 1. Theoretical issues
Are you measuring a stable characteristic of the person being tested?
Suppose you test someone as part of a hiring process.
If the test result says that person does not have the characteristic you’re looking for, does that mean they could never acquire that characteristic?
This is a thorny problem – simply stopping the use of tests is no solution because we would still have to make decisions about people. The solution will be developed over time, through development of better theories of psychology in the various domains in which we use tests.

22. 2. Adequacy of tests How do we measure test adequacy?
(We’re supposed to be good at measuring things!)
What should go into an assessment of test adequacy?

23. 2. Adequacy of tests
So far, in considering the relative merits of various tests, we have asked whether they are psychometrically adequate
We haven’t asked: is the best test available good enough?
Society uses that standard when testing becomes a legal issue
But the answer is not psychometric

24. 3. Actuarial vs. clinical judgment
U.S. Supreme Court Justice Potter Stewart said in a 1964 case that he didn’t know how to define pornography, but he knew it when he saw it.
Is it possible for us to make good judgments on a question when we cannot articulate the basis for the judgment?

25. 3. Actuarial vs. clinical judgment
Actuarial judgment occurs when we feed test scores into statistical formulas to diagnose a psychological condition or predict future performance.
Clinical judgment occurs when we have a trained psychologist interpret test scores to diagnose a psychological condition or predict future performance.

26. 3. Actuarial vs. clinical judgment
In actuarial judgment, we cannot make accurate predictions tailored to individuals
Instead, our conclusion will be the same for every person with a given set of test scores
In clinical judgment, the claim is that you can determine “what caused what” in an individual’s person’s life (Dawes, 1994)
But clinical judgment does not improve with experience
A person giving a clinical judgment, in contrast, is saying, “I see something particular in this case, and I can use that particular bit of information to make an accurate diagnosis or prediction of outcome for this case which will be different from the diagnosis or prediction for other people who have the same set of test scores.”
As Dawes (1994) put it, the clinician claims “to be able to analyze “what caused what” in an individual’s life rather than to state what is in general true.” (p.80; emphasis in original.) That claim is not true (Dawes, Faust, & Meehl, 1989; Faust, Hart, Guilmette, & Arkes, 1988; Goldberg, 1991; Ruscio, 2002).
Dawes, R.M. (1994). House of Cards: Psychology and Psychotherpay Built on Myth. New York: Free Press.
Dawes, R.M. Faust, D. & Meehl, P.E. (1989). Clinical versus actuarial judgment. Science, 243,

27. 3. Actuarial vs. clinical prediction
Meehl’s work in the 1950s: feed test scores into statistical formula to predict future performance – or have trained psychologist interpret scores
statistical formula does a better job of prediction
still, someone has to be legally responsible – that has to be a trained professional
Faust, D., Hart, T., Guilmette, T., & Arkes, H. (1988). Neuropsychologists’ capacity to detect adolescent malingerers. Professional Psychology: Research and Practice, 19,
Goldberg, L.R. (1991). Human mind versus regression equation: Five contrasts. In D. Chicchetti & W. Grove (Eds.), Thinking Clearly about Psychology: Essays in Honor of Paul E. Meehl. 1, Minneapolis: University of Minnesota Press

28. Social issues Dehumanization
Access to benefits of psychological testing

29. 1. Dehumanization
Does computerized testing and analysis of test results create a danger of minimizing human uniqueness?
humans are very complex – which allows us to be individuals, different from each other
but testing and interpretation generalize

30. 1. Dehumanization Is generalization a problem?
Computer-scored and administered psychological tests might determine important aspects of your life
If you think this is a problem, why do you think so? What evidence do you have that this hurts people?
What alternative approach would you use?
Computer-administered and –scored tests might determine your future creditworthiness, educational opportunity, employment, psychological health, and much more

31. 2. Access to psychological testing
We’ve considered possibility of being hurt by tests – but what about possible benefits of tests? Who gets those benefits?
WAIS-III kit costs $775 per person – for tester to buy the kit. Cost must be passed on to someone. Who should that be?

32. 2. Access to psychological testing
Psychological services are not covered by OHIP; Psychiatric services are
Psychiatrists have opposed extending OHIP coverage to psychological testing
Is that just turf protection?
There is rivalry between the professions – but we could make a stronger case for billing OHIP if we used only actuarial judgment

33. Two examples of bad tests
Graphology (handwriting analysis)
TOEFL

34. Graphology (handwriting analysis)
Used in hiring decisions in business
Used to make assessments of personality type and to predict on-the-job performance
Simner & Goffin (2003): very little evidence of validity for graphology
studies using non- autobiographical writing are the most useful and show the least evidence of validity.
Simner & Goffin: neither applicant nor firm has much legal recourse if graphologist’s report is wrong.

35. Graphology (handwriting analysis)
Dean (1992) meta-analysis of handwriting studies showed an average correlation of .16 between graphological judgment and job performance, using autobiographical writing.
Correlation drops to .09 when using non- autobiographical writing.

36. Graphology (handwriting analysis)
Simner & Goffin: both general mental ability testing and structured employment interviews correlate around .50 with job performance. When used together, the two correlate .63 with job performance.
Thus, GMA testing and interviewing are much more likely to lead to sensible hiring decisions.

37. Test of English as a Foreign Language
Widely used to select among students who are not native speakers of English for admission to Canadian universities.
TOEFL manual presents a distorted picture of validity studies
Simner (1998) presents CPA’s recommendation – don’t use TOEFL for university admission decisions

38. TOEFL – Criterion Validity
TOEFL manual leaves out studies that show low to moderate correlations between TOEFL scores and teachers’ ratings of English language proficiency (.34 to .55)
TOEFL manual leaves out studies that show low to moderate correlations between TOEFL scores and oral interviews (.30 to .60)
Plus some of the evidence for criterion validity comes from correlations with tests such as MIchigan Test of English Language Proficiency which are not all that different from TOEFL. Thus, Simner says that these validity studies should be seen as parallel forms reliability studies instead.

39. TOEFL – Criterion validity
Generally, these studies assess concurrent validity, not more relevant predictive validity
Simner: some criterion validity tests for TOEFL involve the Michigan TELP, very similar to TOEFL – so this is really parallel forms reliability, not validity evidence
Plus some of the evidence for criterion validity comes from correlations with tests such as MIchigan Test of English Language Proficiency which are not all that different from TOEFL. Thus, Simner says that these validity studies should be seen as parallel forms reliability studies instead.

40. TOEFL – Construct validity
ETS: Native speakers of English do much better than non-natives on TOEFL.
Simner: in these studies, native speakers are a more highly-selected group.
Non-natives were unselected examinees from a range of backgrounds.
Natives were selected from among entering freshmen
Test anxiety differences
Anxiety – for the native speakers in these studies, nothing was at stake in the testing.

41. TOEFL – Construct validity
Unpublished ETS study: English speakers had an average score of 590, below the 600 cutoff used for non- natives at many Canadian universities.
If native English speakers can succeed with scores < 600 why can’t non-natives also succeed with scores below 600?

42. Test of English as a Foreign Language
Simner: Since 1995, 10 Ontario universities have raised their TOEFL cutoff from around 550 to This means that only the top 10% of examinees are accepted, instead of top 30%.
Taylor-Russell tables: r = .20, now 10% accepted, before 30%
Net gain in percentage likely to be successful is 4%, at a cost of rejecting an additional 20%.

43. TOEFL: Simner’s recommendations
make admissions decisions on the basis of all relevant information, not just TOEFL scores
do not use rigid cutoff scores
consider the kinds and levels of English proficiency required in different fields and levels of study
consider resources available for improving English language skills

44. CPA Code of Ethics
The following slides show the four basic principles advanced in the CPA Code of Ethics. All of the material on the next 15 slides is quoted directly from that Code.
There is significantly more information on each principle at this web address: dian%20Code%20of%20Ethics%20for%20Psycho.pdf
(Note: scroll down if the page looks blank)

45. CPA Principle I: Respect for the Dignity of Persons.
In the course of their work as scientists, practitioners, or scientist-practitioners, psychologists come into contact with many different individuals and groups, including: research participants; clients seeking help with individual, family, organizational, industrial, or community issues; students; trainees; supervisees; employees; business partners; business competitors; colleagues; employers; third party payers; and, the general public…

46. CPA Principle I: Respect for the Dignity of Persons.
In these contacts, psychologists accept as fundamental the principle of respect for the dignity of persons; that is, the belief that each person should be treated primarily as a person or an end in him/herself, not as an object or a means to an end…

47. CPA Principle II: Responsible Caring.
A basic ethical expectation of any discipline is that its activities will benefit members of society or, at least, do no harm. Therefore, psychologists demonstrate an active concern for the welfare of any individual, family, group, or community with whom they relate in their role as psychologists. This concern includes both those directly involved and those indirectly involved in their activities.

48. CPA Principle II: Responsible Caring.
However, as with Principle I, psychologists’ greatest responsibility is to protect the welfare of those in the most vulnerable position. Normally, persons directly involved in their activities (e.g., research participants, clients, students) are in such a position. Psychologists’ responsibility to those indirectly involved (e.g., employers, third party payers, the general public) normally is secondary.

49. CPA Principle III: Integrity in Relationships.
Psychologists are expected to demonstrate the highest integrity in all of their relationships. However, in rare circumstances, values such as openness and straightforwardness might need to be subordinated to the values contained in the Principles of Respect for the Dignity of Persons and Responsible Caring.

50. CPA Principle III: Integrity in Relationships.
The relationships formed by psychologists in the course of their work embody explicit and implicit mutual expectations of integrity that are vital to the advancement of scientific knowledge and to the maintenance of public confidence in the discipline of psychology. These expectations include: accuracy and honesty; straightforwardness and openness; the maximization of objectivity and minimization of bias; and, avoidance of conflicts of interest. Psychologists have a responsibility to meet these expectations and to encourage reciprocity.

51. CPA Principle III: Integrity in Relationships.
In addition to accuracy, honesty, and the obvious prohibitions of fraud or misrepresentation, meeting expectations of integrity is enhanced by self-knowledge and the use of critical analysis.

52. CPA Principle IV: Responsibility to Society.
Psychology functions as a discipline within the context of human society. Psychologists, both in their work and as private citizens, have responsibilities to the societies in which they live and work, such as the neighborhood or city, and to the welfare of all human beings in those societies.

53. CPA Principle IV: Responsibility to Society.
Two of the legitimate expectations of psychology as a science and a profession are that it will increase knowledge and that it will conduct its affairs in such ways that it will promote the welfare of all human beings.

54. CPA Principle IV: Responsibility to Society.
Freedom of enquiry and debate (including scientific and academic freedom) is a foundation of psychological education, science, and practice. In the context of society, the above expectations imply that psychologists will exercise this freedom through the use of activities and methods that are consistent with ethical requirements.

55. CPA Principle IV: Responsibility to Society.
The above expectations also imply that psychologists will do whatever they can to ensure that psychological knowledge, when used in the development of social structures and policies, will be used for beneficial purposes, and that the discipline’s own structures and policies will support those beneficial purposes. Within the context of this document, social structures and policies that have beneficial purposes are defined as those that more readily support and reflect respect for the dignity of persons, responsible caring, integrity in relationships, and responsibility to society.

56. CPA Principle IV: Responsibility to Society.
If psychological knowledge or structures are used against these purposes, psychologists have an ethical responsibility to try to draw attention to and correct the misuse. Although this is a collective responsibility, those psychologists having direct involvement in the structures of the discipline, in social development, or in the theoretical or research data base that is being used (e.g., through research, expert testimony, or policy advice) have the greatest responsibility to act. Other psychologists must decide for themselves the most appropriate and beneficial use of their time and talents to help meet this collective responsibility.

57. CPA Principle IV: Responsibility to Society.
When a person’s welfare appears to conflict with benefits to society, it is often possible to find ways of working for the benefit of society that do not violate respect and responsible caring for the person. However, if this is not possible, the dignity and well-being of a person should not be sacrificed to a vision of the greater good of society, and greater weight must be given to respect and responsible caring for the person.

58. CPA’s Caveat…
Even with the above ordering of the principles, psychologists will be faced with ethical dilemmas that are difficult to resolve. In these circumstances, psychologists are expected to engage in an ethical decision-making process that is explicit enough to bear public scrutiny. In some cases, resolution might be a matter of personal conscience.

59. CPA’s Caveat…
However, decisions of personal conscience are also expected to be the result of a decision-making process that is based on a reasonably coherent set of ethical principles and that can bear public scrutiny. If the psychologist can demonstrate that every reasonable effort was made to apply the ethical principles of this Code and resolution of the conflict has had to depend on the personal conscience of the psychologist, such a psychologist would be deemed to have followed this Code.