DUMMIES RELIABILTY AND VALIDITY FOR By: Jeremy Starkey Lijia Zhang

Slides:

Advertisements

Similar presentations

Advertisements

FACULTY DEVELOPMENT PROFESSIONAL SERIES OFFICE OF MEDICAL EDUCATION TULANE UNIVERSITY SCHOOL OF MEDICINE Using Statistics to Evaluate Multiple Choice.

Presented by Eroika Jeniffer.  We want to set tasks that form a representative of the population of oral tasks that we expect candidates to be able to.

VALIDITY AND RELIABILITY

Part II Sigma Freud & Descriptive Statistics

General Information --- What is the purpose of the test? For what population is the designed? Is this population relevant to the people who will take your.

Chapter 4A Validity and Test Development. Basic Concepts of Validity Validity must be built into the test from the outset rather than being limited to.

Assessment: Reliability, Validity, and Absence of bias

Test Construction Processes 1- Determining the function and the form 2- Planning( Content: table of specification) 3- Preparing( Knowledge and experience)

Chapter 4 Validity.

Test Validity: What it is, and why we care.

RELIABILITY & VALIDITY

Developing a Hiring System Reliability of Measurement.

Lecture 7 Psyc 300A. Measurement Operational definitions should accurately reflect underlying variables and constructs When scores are influenced by other.

Characteristics of Sound Tests

Variables cont. Psych 231: Research Methods in Psychology.

Validity, Reliability, & Sampling

Chapter 7 Correlational Research Gay, Mills, and Airasian

Questions to check whether or not the test is well designed: 1. How do you know if a test is effective? 2. Can it be given within appropriate administrative.

Validity and Reliability Neither Valid nor Reliable Reliable but not Valid Valid & Reliable Fairly Valid but not very Reliable Think in terms of ‘the purpose.

Classroom Assessment A Practical Guide for Educators by Craig A. Mertler Chapter 9 Subjective Test Items.

Technical Issues Two concerns Validity Reliability

1 Item Analysis - Outline 1. Types of test items A. Selected response items B. Constructed response items 2. Parts of test items 3. Guidelines for writing.

RELIABILITY AND VALIDITY OF DATA COLLECTION. RELIABILITY OF MEASUREMENT Measurement is reliable when it yields the same values across repeated measures.

Sampling : Error and bias. Sampling definitions  Sampling universe  Sampling frame  Sampling unit  Basic sampling unit or elementary unit  Sampling.

C M Clarke-Hill1 Collecting Quantitative Data Samples Surveys Pitfalls etc... Research Methods.

Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.

Emily H. Wughalter, Ed.D. Measurement & Evaluation Spring 2010.

Technical Adequacy Session One Part Three.

Psychology 290 Lab #2 Sept. 26 – 28 Types & Parts of Articles Operational Definition Variables Reliability & Validity.

General Information Iowa Writing Assessment The Riverside Publishing Company, 1994 $39.00: 25 test booklets, 25 response sheets 40 minutes to plan, write.

Validity. Face Validity  The extent to which items on a test appear to be meaningful and relevant to the construct being measured.

Chap. 2 Principles of Language Assessment

Validity Is the Test Appropriate, Useful, and Meaningful?

Week 5 Lecture 4. Lecture’s objectives  Understand the principles of language assessment.  Use language assessment principles to evaluate existing tests.

Validity and Reliability Neither Valid nor Reliable Reliable but not Valid Valid & Reliable Fairly Valid but not very Reliable Think in terms of ‘the purpose.

1 Item Analysis - Outline 1. Types of test items A. Selected response items B. Constructed response items 2. Parts of test items 3. Guidelines for writing.

Validity Validity: A generic term used to define the degree to which the test measures what it claims to measure.

RELIABILITY AND VALIDITY OF ASSESSMENT

Presented By Dr / Said Said Elshama  Distinguish between validity and reliability.  Describe different evidences of validity.  Describe methods of.

Validity and Item Analysis Chapter 4.  Concerns what instrument measures and how well it does so  Not something instrument “has” or “does not have”

Assessment. Workshop Outline Testing and assessment Why assess? Types of tests Types of assessment Some assessment task types Backwash Qualities of a.

McGraw-Hill/Irwin © 2012 The McGraw-Hill Companies, Inc. All rights reserved. Obtaining Valid and Reliable Classroom Evidence Chapter 4:

Introduction to Item Analysis Objectives: To begin to understand how to identify items that should be improved or eliminated.

Review: Alternative Assessments Alternative/Authentic assessment Real-life setting Performance based Techniques: Observation Individual or Group Projects.

Nurhayati, M.Pd Indraprasta University Jakarta.  Validity : Does it measure what it is supposed to measure?  Reliability: How the representative is.

Chapter 6 - Standardized Measurement and Assessment

VALIDITY, RELIABILITY & PRACTICALITY Prof. Rosynella Cardozo Prof. Jonathan Magdalena.

Classroom Assessment Chapters 4 and 5 ELED 4050 Summer 2007.

TEST SCORES INTERPRETATION - is a process of assigning meaning and usefulness to the scores obtained from classroom test. - This is necessary because.

Evaluation and Assessment Evaluation is a broad term which involves the systematic way of gathering reliable and relevant information for the purpose.

Overview of Types of Measures Margaret Kasimatis, PhD VP for Academic Planning & Effectiveness.

Assessing Intelligence. Test Construction Standardization – defining the meaning of scores by comparing to a pretested “standard group”. Reliability –

1 Measurement Error All systematic effects acting to bias recorded results: -- Unclear Questions -- Ambiguous Questions -- Unclear Instructions -- Socially-acceptable.

Language Assessment.

Principles of Language Assessment

Writing Rubrics Module 5 Activity 4.

CHAPTER 3: Practical Measurement Concepts

Questions What are the sources of error in measurement?

Concept of Test Validity

EDU 385 Session 8 Writing Selection items

Validity and Reliability

Week 3 Class Discussion.

پرسشنامه کارگاه.

Reliability and Validity of Measurement

Chapter 4 Characteristics of a Good Test

Measurement Concepts and scale evaluation

Assessment in Language Learning

EDUC 2130 Quiz #10 W. Huitt.

Presentation transcript:

DUMMIES RELIABILTY AND VALIDITY FOR By: Jeremy Starkey Lijia Zhang November 4, 2002

RELIABILTY Reliability is the consistency of measurement from one test to another.

PRECISION Precision is defined as sharply or exactly defined; exact; particular.

VALIDITY Validity is the appropriateness of the interpretations made from test scores (measurements) and other evaluation results.

ACCURACY Accuracy is defined as done with care; exact; without error.

RELIABILITY Length of test Spread of scores Difficulty of test Factors that influence reliability are: Length of test Spread of scores Difficulty of test Objectivity of test

RELIABILITY Longer the test, then usually the higher the reliability. More questions on a test, then more reliable the test is.

RELIABILITY Larger the spread of scores, then usually the higher the reliability.

RELIABILITY Too easy or too hard tests results in low reliability because they restrict the spread of scores.

RELIABILITY Multiple-choice questions are highly objective and can not be influenced by the scorer, thus should have a high reliability.

VALIDITY Factors that influence validity: unclear directions reading vocabulary too difficult ambiguity inadequate time limits

VALIDITY Three categories for accumulating evidence of validity: Content Validity Criterion-related validity Construct Validity

VALIDITY Content Validity determines the extent to which a set of tasks provide both a relevant and representative sample of the domain of tasks about which the tests score are made.

VALIDITY Criterion-related evidence is the extent to which test performance is related to some other valued measure of performance.

VALIDITY Construct Validity interprets test performance in terms of some psychological quality.

RELIABILITY VS. VALIDITY If low reliability, then you can not have high validity. High reliability can either have high validity or low validity. You must have reliability of some sort before you have validity.

PRECISION VS. ACCURACY If you have poor precision, then you can not have good accuracy. If you have great precision, then you can either have good or poor accuracy. You must have some sort of precision before you can be accurate.

What a GREAT TARGET!!!!