Discussion of Issues and Approaches Presented by Templin and Teresi

Slides:



Advertisements
Similar presentations
Reading. 1: Is developing an interest in books Scale points 1 – 3 are based on the childrens achievement in their preferred language Looking for child.
Advertisements

Psychology Practical (Year 2) PS2001 Correlation and other topics.
Item Response Theory in a Multi-level Framework Saralyn Miller Meg Oliphint EDU 7309.
Structural Equation Modeling
Psychometric Aspects of Linking Tests to the CEF Norman Verhelst National Institute for Educational Measurement (Cito) Arnhem – The Netherlands.
Reliability and Validity checks S-005. Checking on reliability of the data we collect  Compare over time (test-retest)  Item analysis  Internal consistency.
Children’s subjective well-being Findings from national surveys in England International Society for Child Indicators Conference, 27 th July 2011.
Reliability & Validity.  Limits all inferences that can be drawn from later tests  If reliable and valid scale, can have confidence in findings  If.
Item Analysis: A Crash Course Lou Ann Cooper, PhD Master Educator Fellowship Program January 10, 2008.
Rasch analysis of the Roland-Morris Disability Questionnaire Megan Davidson, PhD School of Physiotherapy, La Trobe University, Melbourne.
Lecture 17. Survey validity Thanks to Dr. Neupert (my wife) for slides.
QUESTIONNAIRES ORANGE BOOK CHAPTER 9. WHAT DO QUESTIONNAIRES GATHER? BEHAVIOR ATTITUDES/BELIEFS/OPINIONS CHARACTERISTICS (AGE / MARITAL STATUS / EDUCATION.
Descriptive Statistical Analyses Reliability Analyses Review of Last Class.
Seven possibly controversial but hopefully useful rules Paul De Boeck K.U.Leuven.
Developing the Research Question
Nominal Level Measurement n numbers used as ways to identify or name categories n numbers do not indicate degrees of a variable but simple groupings of.
Multiple Regression Models Advantages of multiple regression Important preliminary analyses Parts of a multiple regression model & interpretation Differences.
A quick introduction to the analysis of questionnaire data John Richardson.
Ordinal Logistic Regression “Good, better, best; never let it rest till your good is better and your better is best” (Anonymous)
RESEARCH REPORT PREPARATION AND PRESENTATION
Multivariate Methods EPSY 5245 Michael C. Rodriguez.
Item Response Theory. What’s wrong with the old approach? Classical test theory –Sample dependent –Parallel test form issue Comparing examinee scores.
TRENDS AND CONSUMERISM IN HEALTH CARE Important Attributes Of Quality Health Care: Consumer Perspectives.
Development and Validation of Elder’s Satisfaction Questionnaire for patients Attending Dental Clinics By: Marjan Haghi PhD student in Gerontology University.
Development version 19/06/ of 48 Effectiveness of a postural care training programme © 2012 Effectiveness of a postural care education programme.
Construction and Evaluation of Multi-item Scales Ron D. Hays, Ph.D. RCMAR/EXPORT September 15, 2008, 3-4pm
Naglieri Nonverbal Ability Test (NNAT) Miami-Dade County Public Schools NNAT Workshop March 26, 28, & 29, 2007.
1 Psychology 2020 Measurement & Observing Behavior Unit 2.
Use of Health-Related Quality of Life Measures to Assess Individual Patients July 24, 2014 (1:00 – 2:00 PDT) Kaiser Permanente Methods Webinar Series Ron.
Presented by Mansour Ali Al-Zahrani Author: Ajibade Ayodeji Aibinu.
Developing a Tool to Measure Health Worker Motivation in District Hospitals in Kenya Patrick Mbindyo, Duane Blaauw, Lucy Gilson, Mike English.
Group 3 林正昀 Adam, 李燕俞 Amber, 李季樺 Gina, 徐家慧 Alice.
1 Finding and Verifying Item Clusters in Outcomes Research Using Confirmatory and Exploratory Analyses Ron D. Hays, PhD May 13, 2003; 1-4 pm; Factor
ITEC6310 Research Methods in Information Technology Instructor: Prof. Z. Yang Course Website: c6310.htm Office:
Q Rome Italy, July The Mysterious 36% Difference: Studies on a Measurement Error Anette Björnram, Boris Lorenc, Andreas Persson, Klas Wibell.
INDE 6335 ENGINEERING ADMINISTRATION SURVEY DESIGN Dr. Christopher A. Chung Dept. of Industrial Engineering.
Effects: Example Low High Variable 2.
Item Response Theory (IRT) Models for Questionnaire Evaluation: Response to Reeve Ron D. Hays October 22, 2009, ~3:45-4:05pm
Evaluating Self-Report Data Using Psychometric Methods Ron D. Hays, PhD February 6, 2008 (3:30-6:30pm) HS 249F.
Introduction to Psychology Class 4: The Scientific Method 1 June 15, 2006.
Multi-item Scale Evaluation Ron D. Hays, Ph.D. UCLA Division of General Internal Medicine/Health Services Research
Multitrait Scaling and IRT: Part I Ron D. Hays, Ph.D. Questionnaire Design and Testing.
Experimental Research Methods in Language Learning Chapter 12 Reliability and Reliability Analysis.
Multitrait Scaling and IRT: Part I Ron D. Hays, Ph.D. Questionnaire Design and Testing.
Psychometric Evaluation of Questionnaire Design and Testing Workshop December , 10:00-11:30 am Wilshire Suite 710 DATA.
Marshall University School of Medicine Department of Biochemistry and Microbiology BMS 617 Lecture 11: Models Marshall University Genomics Core Facility.
Evaluating Self-Report Data Using Psychometric Methods Ron D. Hays, PhD February 8, 2006 (3:00-6:00pm) HS 249F.
Social Capital [III] Exercise for the Research Master Multivariate Statistics W. M. van der Veld University of Amsterdam.
Sampling Design & Measurement Scaling
Multitrait Scaling and IRT: Part II Ron D. Hays, Ph.D. Questionnaire Design and Testing.
Reliability When a Measurement Procedure yields consistent scores when the phenomenon being measured is not changing. Degree to which scores are free of.
Multitrait Scaling and IRT: Part I Ron D. Hays, Ph.D. Questionnaire.
MYP Language Acquisition Objectives Phase 1 and
Evaluating Multi-item Scales Ron D. Hays, Ph.D. UCLA Division of General Internal Medicine/Health Services Research HS237A 11/10/09, 2-4pm,
Approaches to quantitative data analysis Lara Traeger, PhD Methods in Supportive Oncology Research.
Causes Identification of Employee's Politics and its Impact on Performance of Banking Industry in Pakistan Avinash Advani Independent Researcher Khadim.
Using statistics in the analysis of quantitative data A good way to use this material for detailed study is to print the whole file then to run the slide.
Quality instrument* Questions are determined by objectives Resist the temptation to ask questions that are interesting but not relevant to your hypothesis.
QDET2, Miami, FL, Hibiscus A
assessing scale reliability
Computing Reliability
Evaluating IRT Assumptions
Final Session of Summer: Psychometric Evaluation
Introduction Results Conclusions Hypotheses Method
Evaluating Multi-item Scales
Multitrait Scaling and IRT: Part I
Grace Orlyn SITOMPUL 5th ISC – Oct 30-31, 2017 APIU
Evaluating Multi-item Scales
Introduction to Psychometric Analysis of Survey Data
UCLA Department of Medicine
Presentation transcript:

Discussion of Issues and Approaches Presented by Templin and Teresi

Discussion Points Always use familiar along with unfamiliar (“more sophisticated”) methods Sophisticated approaches often may not lead to different conclusions but they could and you can often learn something unique All “quantitative” methods identify problems but supplemental approaches (“qualitative”) needed to find out why

Item-Scale Correlation Matrix-- Patient Satisfaction Questionnaire Technical Interpersonal Communication Financial Communication 10.58†0.59†0.61* †0.50†0.50* †0.66†0.63* †0.66†0.67* †0.71†0.70*0.25 Financial * * * * * * * * Note – Standard error of correlation is Technical = satisfaction with technical quality. Interpersonal = satisfaction with the interpersonal aspects. Communication = satisfaction with communication. Financial = satisfaction with financial arrangements. *Item-scale correlations for hypothesized scales (corrected for item overlap). †Correlation within two standard errors of the correlation of the item with its hypothesized scale. Cronbach's alpha

DIF Logistic (ordinal) regression Item response regressed on –Total scale score – Group –Interaction between total scale score and group

Hard time speaking with or understanding child’s dentist because you spoke different languages Complex item? Insufficient number of individuals in sample for which these item applies Something else?

Person Fit Large negative Z L values indicate misfit. Person responded to 14 items in physical functioning bank (Z L = -3.13) –For 13 items the person could do the activity (including running 5 miles) without any difficulty. –However, this person reported a little difficulty being out of bed for most of the day.