Presentation on theme: "1 Outline 1. Ethics vs. Law 2. Ethical use of tests – issues: Competence Informed Consent Knowledge of Results Confidentiality Test security."— Presentation transcript:
1 Outline 1. Ethics vs. Law 2. Ethical use of tests – issues: Competence Informed Consent Knowledge of Results Confidentiality Test security Divided Loyalties
2 Outline 4. Professional Issues Theoretical issues Test adequacy Actuarial vs. clinical prediction 5. Social Issues Dehumanization Access to benefits of psychological testing
3 Outline 6. Two examples of bad tests Handwriting analysis TOEFL 7. The CPA Code of Ethics (synopsis) Respect for the dignity of persons Responsible caring Integrity in relationships Responsibility to society
4 Ethics vs. Law Ethics: what one should or should not do, according to principles or norms of conduct Ethics codes are not produced by democratically-elected legislatures Enforcement mechanisms are usually informal, may be complex, even unconscious
5 Ethics vs. Law Law: what one must or must not do, according to legal dictates Laws are created by democratically-elected legislatures in democracies such as Canada Laws come with explicit penalties for infractions and a formal enforcement system
6 Ethical use of tests – issues: 1. Competence 2. Informed Consent 3. Knowledge of Results 4. Confidentiality 5. Test security 6. Divided Loyalties
7 1. Competence develop competence in assessment concepts and methods recognize boundaries of your competence competence is very important if test result affects someone’s life
8 2. Informed Consent Generally, the client/subject must voluntarily consent psychologist must inform client about nature and purpose of assessment in understandable language
9 Informed consent - exceptions Sometimes, it is acceptable to test without getting consent – e.g., you have given implied consent to be tested by registering in this course, but I have never sought your explicit consent Sometimes it is necessary to test without getting consent, even when consent is explicitly refused by person to be tested – e.g., when mandated by law
10 3. Knowledge of Results must fully disclose test results in understandable language but do not do so in a way that reveals the content of test items, because that content is proprietary information (e.g., somebody owns it – it’s not yours to reveal)
11 Constraints on client access to results PIPEDA Principal 4.9: Upon request, a person must be informed of the existence, use, and disclosure of his or her personal information and must be given access if requested. Section 9.3b says that access can be denied if it would mean revealing confidential commercial information (unless latter is severable).
12 Constraints on client access to results Ontario Personal Health Information Act: access rights do not apply to a record that contains raw data from standardized psychological tests or assessments, unless those data are reasonably severable. CPA and leading Canadian test publishers say that Test Materials are not “personal information” so should not be released.
13 4. Confidentiality Test results are confidential information Release of results should only be made to another qualified professional after client’s consent
14 5. Test Security Test materials must be kept secure Test items are not revealed except in training programs and when mandated by law, to protect test integrity Test items are private property
15 6. Automated Scoring/Interpretation Systems Psychologist is still responsible for proper interpretation of test results Psychologists are professionals – they take personal responsibility for their work
16 7. Divided loyalties Who is the client? The person being tested, or the institution you work for? What if these parties have conflicting interests? How do you maintain test security but also explain an adverse decision?
17 Divided loyalties Suppose you are asked to test a child who might have disorder X or Y Suppose X is very common and so the government provides resources to help schools deal with it Suppose Y is not common and no resources are available in school to deal with it If your testing suggests Y, will you report Y – or X, so the child gets some sort of help?
18 Professional Issues 1. Theoretical issues 2. Test adequacy 3. Actuarial vs. clinical prediction
19 1. Theoretical Issues Are you measuring a stable characteristic of the person being tested? If so, differences in scores over time reflect measurement error or subject variables such as fatigue
20 1. Theoretical Issues Are you measuring a stable characteristic of the person being tested? In fact, we have little evidence that this is true, so what is the value of your test result – will it still be true next year? Especially problematic for personality tests
21 1. Theoretical issues Are you measuring a stable characteristic of the person being tested? Suppose you test someone as part of a hiring process. If the test result says that person does not have the characteristic you’re looking for, does that mean they could never acquire that characteristic?
22 2. Adequacy of tests How do we measure test adequacy? (We’re supposed to be good at measuring things!) What should go into an assessment of test adequacy?
23 2. Adequacy of tests So far, in considering the relative merits of various tests, we have asked whether they are psychometrically adequate We haven’t asked: is the best test available good enough? Society uses that standard when testing becomes a legal issue But the answer is not psychometric
24 3. Actuarial vs. clinical judgment U.S. Supreme Court Justice Potter Stewart said in a 1964 case that he didn’t know how to define pornography, but he knew it when he saw it. Is it possible for us to make good judgments on a question when we cannot articulate the basis for the judgment?
25 3. Actuarial vs. clinical judgment Actuarial judgment occurs when we feed test scores into statistical formulas to diagnose a psychological condition or predict future performance. Clinical judgment occurs when we have a trained psychologist interpret test scores to diagnose a psychological condition or predict future performance.
26 3. Actuarial vs. clinical judgment In actuarial judgment, we cannot make accurate predictions tailored to individuals Instead, our conclusion will be the same for every person with a given set of test scores In clinical judgment, the claim is that you can determine “what caused what” in an individual’s person’s life (Dawes, 1994) But clinical judgment does not improve with experience
27 3. Actuarial vs. clinical prediction Meehl’s work in the 1950s: feed test scores into statistical formula to predict future performance – or have trained psychologist interpret scores statistical formula does a better job of prediction still, someone has to be legally responsible – that has to be a trained professional
28 Social issues 1. Dehumanization 2. Access to benefits of psychological testing
29 1. Dehumanization Does computerized testing and analysis of test results create a danger of minimizing human uniqueness? humans are very complex – which allows us to be individuals, different from each other but testing and interpretation generalize
30 1. Dehumanization Is generalization a problem? Computer-scored and administered psychological tests might determine important aspects of your life If you think this is a problem, why do you think so? What evidence do you have that this hurts people? What alternative approach would you use?
31 2. Access to psychological testing We’ve considered possibility of being hurt by tests – but what about possible benefits of tests? Who gets those benefits? WAIS-III kit costs $775 per person – for tester to buy the kit. Cost must be passed on to someone. Who should that be?
32 2. Access to psychological testing Psychological services are not covered by OHIP; Psychiatric services are Psychiatrists have opposed extending OHIP coverage to psychological testing Is that just turf protection? There is rivalry between the professions – but we could make a stronger case for billing OHIP if we used only actuarial judgment
33 Two examples of bad tests 1. Graphology (handwriting analysis) 2. TOEFL
34 Graphology (handwriting analysis) Used in hiring decisions in business Used to make assessments of personality type and to predict on-the-job performance Simner & Goffin (2003): very little evidence of validity for graphology studies using non- autobiographical writing are the most useful and show the least evidence of validity.
35 Graphology (handwriting analysis) Dean (1992) meta-analysis of handwriting studies showed an average correlation of.16 between graphological judgment and job performance, using autobiographical writing. Correlation drops to.09 when using non- autobiographical writing.
36 Graphology (handwriting analysis) Simner & Goffin: both general mental ability testing and structured employment interviews correlate around.50 with job performance. When used together, the two correlate.63 with job performance. Thus, GMA testing and interviewing are much more likely to lead to sensible hiring decisions.
37 Test of English as a Foreign Language Widely used to select among students who are not native speakers of English for admission to Canadian universities. TOEFL manual presents a distorted picture of validity studies Simner (1998) presents CPA’s recommendation – don’t use TOEFL for university admission decisions
38 TOEFL – Criterion Validity TOEFL manual leaves out studies that show low to moderate correlations between TOEFL scores and teachers’ ratings of English language proficiency (.34 to.55) TOEFL manual leaves out studies that show low to moderate correlations between TOEFL scores and oral interviews (.30 to.60)
39 TOEFL – Criterion validity Generally, these studies assess concurrent validity, not more relevant predictive validity Simner: some criterion validity tests for TOEFL involve the Michigan TELP, very similar to TOEFL – so this is really parallel forms reliability, not validity evidence
40 TOEFL – Construct validity ETS: Native speakers of English do much better than non-natives on TOEFL. Simner: in these studies, native speakers are a more highly-selected group. Non-natives were unselected examinees from a range of backgrounds. Natives were selected from among entering freshmen Test anxiety differences
41 TOEFL – Construct validity Unpublished ETS study: 5000 English speakers had an average score of 590, below the 600 cutoff used for non- natives at many Canadian universities. If native English speakers can succeed with scores < 600 why can’t non-natives also succeed with scores below 600?
42 Test of English as a Foreign Language Simner: Since 1995, 10 Ontario universities have raised their TOEFL cutoff from around 550 to 600. This means that only the top 10% of examinees are accepted, instead of top 30%. Taylor-Russell tables: r =.20, now 10% accepted, before 30% Net gain in percentage likely to be successful is 4%, at a cost of rejecting an additional 20%.
43 TOEFL: Simner’s recommendations make admissions decisions on the basis of all relevant information, not just TOEFL scores do not use rigid cutoff scores consider the kinds and levels of English proficiency required in different fields and levels of study consider resources available for improving English language skills
44 CPA Code of Ethics The following slides show the four basic principles advanced in the CPA Code of Ethics. All of the material on the next 15 slides is quoted directly from that Code. There is significantly more information on each principle at this web address: http://www.cpa.ca/cpasite/userfiles/Documents/Cana dian%20Code%20of%20Ethics%20for%20Psycho.pdf http://www.cpa.ca/cpasite/userfiles/Documents/Cana dian%20Code%20of%20Ethics%20for%20Psycho.pdf (Note: scroll down if the page looks blank)
45 CPA Principle I: Respect for the Dignity of Persons. In the course of their work as scientists, practitioners, or scientist-practitioners, psychologists come into contact with many different individuals and groups, including: research participants; clients seeking help with individual, family, organizational, industrial, or community issues; students; trainees; supervisees; employees; business partners; business competitors; colleagues; employers; third party payers; and, the general public…
46 CPA Principle I: Respect for the Dignity of Persons. In these contacts, psychologists accept as fundamental the principle of respect for the dignity of persons; that is, the belief that each person should be treated primarily as a person or an end in him/herself, not as an object or a means to an end…
47 CPA Principle II: Responsible Caring. A basic ethical expectation of any discipline is that its activities will benefit members of society or, at least, do no harm. Therefore, psychologists demonstrate an active concern for the welfare of any individual, family, group, or community with whom they relate in their role as psychologists. This concern includes both those directly involved and those indirectly involved in their activities.
48 CPA Principle II: Responsible Caring. However, as with Principle I, psychologists’ greatest responsibility is to protect the welfare of those in the most vulnerable position. Normally, persons directly involved in their activities (e.g., research participants, clients, students) are in such a position. Psychologists’ responsibility to those indirectly involved (e.g., employers, third party payers, the general public) normally is secondary.
49 CPA Principle III: Integrity in Relationships. Psychologists are expected to demonstrate the highest integrity in all of their relationships. However, in rare circumstances, values such as openness and straightforwardness might need to be subordinated to the values contained in the Principles of Respect for the Dignity of Persons and Responsible Caring.
50 CPA Principle III: Integrity in Relationships. The relationships formed by psychologists in the course of their work embody explicit and implicit mutual expectations of integrity that are vital to the advancement of scientific knowledge and to the maintenance of public confidence in the discipline of psychology. These expectations include: accuracy and honesty; straightforwardness and openness; the maximization of objectivity and minimization of bias; and, avoidance of conflicts of interest. Psychologists have a responsibility to meet these expectations and to encourage reciprocity.
51 CPA Principle III: Integrity in Relationships. In addition to accuracy, honesty, and the obvious prohibitions of fraud or misrepresentation, meeting expectations of integrity is enhanced by self-knowledge and the use of critical analysis.
52 CPA Principle IV: Responsibility to Society. Psychology functions as a discipline within the context of human society. Psychologists, both in their work and as private citizens, have responsibilities to the societies in which they live and work, such as the neighborhood or city, and to the welfare of all human beings in those societies.
53 CPA Principle IV: Responsibility to Society. Two of the legitimate expectations of psychology as a science and a profession are that it will increase knowledge and that it will conduct its affairs in such ways that it will promote the welfare of all human beings.
54 CPA Principle IV: Responsibility to Society. Freedom of enquiry and debate (including scientific and academic freedom) is a foundation of psychological education, science, and practice. In the context of society, the above expectations imply that psychologists will exercise this freedom through the use of activities and methods that are consistent with ethical requirements.
55 CPA Principle IV: Responsibility to Society. The above expectations also imply that psychologists will do whatever they can to ensure that psychological knowledge, when used in the development of social structures and policies, will be used for beneficial purposes, and that the discipline’s own structures and policies will support those beneficial purposes. Within the context of this document, social structures and policies that have beneficial purposes are defined as those that more readily support and reflect respect for the dignity of persons, responsible caring, integrity in relationships, and responsibility to society.
56 CPA Principle IV: Responsibility to Society. If psychological knowledge or structures are used against these purposes, psychologists have an ethical responsibility to try to draw attention to and correct the misuse. Although this is a collective responsibility, those psychologists having direct involvement in the structures of the discipline, in social development, or in the theoretical or research data base that is being used (e.g., through research, expert testimony, or policy advice) have the greatest responsibility to act. Other psychologists must decide for themselves the most appropriate and beneficial use of their time and talents to help meet this collective responsibility.
57 CPA Principle IV: Responsibility to Society. When a person’s welfare appears to conflict with benefits to society, it is often possible to find ways of working for the benefit of society that do not violate respect and responsible caring for the person. However, if this is not possible, the dignity and well-being of a person should not be sacrificed to a vision of the greater good of society, and greater weight must be given to respect and responsible caring for the person.
58 CPA’s Caveat… Even with the above ordering of the principles, psychologists will be faced with ethical dilemmas that are difficult to resolve. In these circumstances, psychologists are expected to engage in an ethical decision-making process that is explicit enough to bear public scrutiny. In some cases, resolution might be a matter of personal conscience.
59 CPA’s Caveat… However, decisions of personal conscience are also expected to be the result of a decision-making process that is based on a reasonably coherent set of ethical principles and that can bear public scrutiny. If the psychologist can demonstrate that every reasonable effort was made to apply the ethical principles of this Code and resolution of the conflict has had to depend on the personal conscience of the psychologist, such a psychologist would be deemed to have followed this Code.