Data Analysis. Quantitative data: Reliability & Validity Reliability: the degree of consistency with which it measures the attribute it is supposed to.

Slides:



Advertisements
Similar presentations
Topics: Quality of Measurements
Advertisements

© 2006 The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill Validity and Reliability Chapter Eight.
Chapter 4 – Reliability Observed Scores and True Scores Error
Assessment Procedures for Counselors and Helping Professionals, 7e © 2010 Pearson Education, Inc. All rights reserved. Chapter 5 Reliability.
Reliability Analysis. Overview of Reliability What is Reliability? Ways to Measure Reliability Interpreting Test-Retest and Parallel Forms Measuring and.
Part II Sigma Freud & Descriptive Statistics
MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.
Methods for Estimating Reliability
Measurement. Scales of Measurement Stanley S. Stevens’ Five Criteria for Four Scales Nominal Scales –1. numbers are assigned to objects according to rules.
Reliability and Validity of Research Instruments
Part II Knowing How to Assess Chapter 5 Minimizing Error p115 Review of Appl 644 – Measurement Theory – Reliability – Validity Assessment is broader term.
Becoming Acquainted With Statistical Concepts CHAPTER CHAPTER 12.
Reliability Analysis. Overview of Reliability What is Reliability? Ways to Measure Reliability Interpreting Test-Retest and Parallel Forms Measuring and.
Concept of Measurement
Reliability n Consistent n Dependable n Replicable n Stable.
DATA ANALYSIS I MKT525. Plan of analysis What decision must be made? What are research objectives? What do you have to know to reach those objectives?
Chapter 13 Analyzing Quantitative data. LEVELS OF MEASUREMENT Nominal Measurement Ordinal Measurement Interval Measurement Ratio Measurement.
Chapter 14 Analyzing Quantitative Data. LEVELS OF MEASUREMENT Nominal Measurement Nominal Measurement Ordinal Measurement Ordinal Measurement Interval.
Analysis of Research Data
Chapter 8 Estimation: Single Population
Session 3 Normal Distribution Scores Reliability.
Statistical Evaluation of Data
Research Methods in MIS
Reliability of Selection Measures. Reliability Defined The degree of dependability, consistency, or stability of scores on measures used in selection.
Measurement and Data Quality
Instrumentation.
Foundations of Educational Measurement
Educational Research: Competencies for Analysis and Application, 9 th edition. Gay, Mills, & Airasian © 2009 Pearson Education, Inc. All rights reserved.
MEASUREMENT CHARACTERISTICS Error & Confidence Reliability, Validity, & Usability.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 14 Measurement and Data Quality.
McMillan Educational Research: Fundamentals for the Consumer, 6e © 2012 Pearson Education, Inc. All rights reserved. Educational Research: Fundamentals.
I/O Psychology Research Methods. What is Science? Science: Approach that involves the understanding, prediction, and control of some phenomenon of interest.
LECTURE 06B BEGINS HERE THIS IS WHERE MATERIAL FOR EXAM 3 BEGINS.
Basic Business Statistics, 11e © 2009 Prentice-Hall, Inc. Chap 8-1 Confidence Interval Estimation.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 17 Inferential Statistics.
Statistical Evaluation of Data
Instrumentation (cont.) February 28 Note: Measurement Plan Due Next Week.
Reliability Chapter 3. Classical Test Theory Every observed score is a combination of true score plus error. Obs. = T + E.
Reliability Chapter 3.  Every observed score is a combination of true score and error Obs. = T + E  Reliability = Classical Test Theory.
Reliability & Validity
1 Chapter 4 – Reliability 1. Observed Scores and True Scores 2. Error 3. How We Deal with Sources of Error: A. Domain sampling – test items B. Time sampling.
Counseling Research: Quantitative, Qualitative, and Mixed Methods, 1e © 2010 Pearson Education, Inc. All rights reserved. Basic Statistical Concepts Sang.
Tests and Measurements Intersession 2006.
Assessing Learners with Special Needs: An Applied Approach, 6e © 2009 Pearson Education, Inc. All rights reserved. Chapter 4:Reliability and Validity.
EDU 8603 Day 6. What do the following numbers mean?
Appraisal and Its Application to Counseling COUN 550 Saint Joseph College For Class # 3 Copyright © 2005 by R. Halstead. All rights reserved.
Determination of Sample Size: A Review of Statistical Theory
Copyright © 2008 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 17 Assessing Measurement Quality in Quantitative Studies.
JS Mrunalini Lecturer RAKMHSU Data Collection Considerations: Validity, Reliability, Generalizability, and Ethics.
Psychometrics. Goals of statistics Describe what is happening now –DESCRIPTIVE STATISTICS Determine what is probably happening or what might happen in.
Reliability and Validity in Testing. What is Reliability? Consistency Accuracy There is a value related to reliability that ranges from -1 to 1.
Reliability a measure is reliable if it gives the same information every time it is used. reliability is assessed by a number – typically a correlation.
Reliability When a Measurement Procedure yields consistent scores when the phenomenon being measured is not changing. Degree to which scores are free of.
EDU5950 SEM RELIABILITY ANALYSIS -CRONBACH ALPHA TEST FOR NORMALITY.
Dr. Jeffrey Oescher 27 January 2014 Technical Issues  Two technical issues  Validity  Reliability.
Copyright © 2014 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 11 Measurement and Data Quality.
NURS 306, Nursing Research Lisa Broughton, MSN, RN, CCRN RESEARCH STATISTICS.
5. Evaluation of measuring tools: reliability Psychometrics. 2011/12. Group A (English)
Chapter 2 Norms and Reliability. The essential objective of test standardization is to determine the distribution of raw scores in the norm group so that.
CRITICALLY APPRAISING EVIDENCE Lisa Broughton, PhD, RN, CCRN.
Measurement and Scaling Concepts
ESTABLISHING RELIABILITY AND VALIDITY OF RESEARCH TOOLS Prof. HCL Rawat Principal UCON,BFUHS Faridkot.
Lecture 5 Validity and Reliability
Reliability & Validity
مركز مطالعات و توسعه آموزش دانشگاه علوم پزشكي كرمان
Scoring: Measures of Central Tendency
Evaluation of measuring tools: reliability
The first test of validity
15.1 The Role of Statistics in the Research Process
Descriptive Statistics
Presentation transcript:

Data Analysis

Quantitative data: Reliability & Validity Reliability: the degree of consistency with which it measures the attribute it is supposed to measure Reliability: the degree of consistency with which it measures the attribute it is supposed to measure reliability measurement tools reliability measurement tools __ stability __ internal consistency __ internal consistency __ equivalence __ equivalence

Stability The extent to which the same results are obtained on repeated administration of the instrument The extent to which the same results are obtained on repeated administration of the instrument Obtained by test-retest reliability (same test for the same subjects in two occasions) Obtained by test-retest reliability (same test for the same subjects in two occasions) Reliability coefficient (the correlation between the two sets of testing; the test and the retest) Reliability coefficient (the correlation between the two sets of testing; the test and the retest)

Internal consistency All of the subparts of the instruments measure the same attribute All of the subparts of the instruments measure the same attribute Obtained by Split-half techniques (correlation between the two randomly split half of the instrument). Obtained by Split-half techniques (correlation between the two randomly split half of the instrument). Mostly reported by the values of cronbach’s alpha, and Kuder-Richardson formula 20 (KR20) Mostly reported by the values of cronbach’s alpha, and Kuder-Richardson formula 20 (KR20)

Equivalence Used when: Used when: 1. Different researchers used the same instrument to measure the same phenomenon 2. Two instruments are used at the same time Obtained by the methods of Interrater reliability (two well-trained observers to the same phenomenon simultaneously) Obtained by the methods of Interrater reliability (two well-trained observers to the same phenomenon simultaneously) Either by correlation or agreement formula Either by correlation or agreement formula

Validity The instrument measures what is suppose to measure The instrument measures what is suppose to measure Content validity Content validity Criterion-related validity Criterion-related validity Construct validity Construct validity Interpretation validity Interpretation validity

Qualitative data Credibility: confidence in the truth of data Credibility: confidence in the truth of data Transferability: Generalizability of the data Transferability: Generalizability of the data Dependability: stability of data over time and conditions Dependability: stability of data over time and conditions Confirmability: objectivity or neutrality of the data Confirmability: objectivity or neutrality of the data

Descriptive Statistics Levels of measurements Levels of measurements 1. Nominal 2. Ordinal 3. Interval 4. ratio

Shapes of distribution Shapes of distribution 1. Normal distribution 2. Skewed distribution (positive and negative) 3. unimodal __ bimodal __ multimodal

Central tendency : Central tendency :  Mode:  Median:  Mean: 5.4  Standard deviation: the average deviation of the data from its mean  Variance: SD 2  Range, inter-quartile range

Data Analysis-2

Data Interpretation Consideration: Consideration: 1. Accuracy - critical view of the data - investigating evidence of the results - investigating evidence of the results - consider other studies’ results - consider other studies’ results - peripheral data analysis - peripheral data analysis - conduct power analysis: type I & type II - conduct power analysis: type I & type II CorrectType-II Type -I Correct True False

alpha : the level of significance used for establishing type-I error alpha : the level of significance used for establishing type-I error β : the probability of type-II error β : the probability of type-II error 1 – β : is the probability of obtaining significance results ( power ) 1 – β : is the probability of obtaining significance results ( power ) Effect size: how much we can say that the intervention made a significance difference Effect size: how much we can say that the intervention made a significance difference

2. Meaning of the results - translation of the results and make it understandable - translation of the results and make it understandable 3. Importance: - translation of the significant findings into practical findings - translation of the significant findings into practical findings 4. Generalizability: - how can we make the findings useful for all the population - how can we make the findings useful for all the population 5. Implication: - what have we learned related to what has been used during study - what have we learned related to what has been used during study