Week 14 More Data Collection Techniques Chapter 5 Research Methods Week 14 More Data Collection Techniques Chapter 5

Review – Data Collection Primary Data Observations Interviews Personal Telephone Questionnaires Case Studies Secondary data

Measurement Techniques Scales for data Nominal Ordinal Interval Ratio

Measurement Techniques Nominal scale We use a number as a shorthand description Male = 1 Female = 2 Or Sultan Qaboos U = 2 University of Nizwa = 3 The number cannot be used mathematically. It makes no sense to divide Sultan Qaboos U by U Nizwa

Measurement Techniques Nominal scale – statistics possible on nominal data We can count We can calculate a mode (which number was the highest)

Measurement Techniques Ordinal scale – We align some results in order – Key success factors 1 – teamwork – 44 votes 2 – leader training – 43 votes 3 - team member training – 33 votes 4 – size of budget – 12 votes We have an order and can compare sizes (1 >2, 3 <2) We cannot assume size of difference – 1 may not have twice as many votes as 2.

Measurement Techniques Ordinal scale – Allowable statistical procedures – Count Median Key success factors 1 – teamwork – 44 votes 2 – leader training – 43 votes 3 - team member training – 33 votes 4 – size of budget – 12 votes

Measurement Techniques Interval Scale – There are equal distances between each number. But there is no absolute 0. A common example is temperature. Absolute 0 is only a theory for temperature. Most of our research data does have an absolute zero - 0 – number of hours studied 0 – number of errors in data entry 0 – cost of motivational incentives

Measurement Techniques Interval Scale – Allowable statistical procedures: Mean – 1.9 hours Standard deviation

Measurement Techniques Ratio Scale – The scale has regular intervals and an absolute 0. Example – number of hours each student used on-line learning 0 – 8 students 1 – 3 students 2 – 6 students 3 – 2 students 4 – 5 students 5 – 0 students

Tests of Measurement Quality Validity Tests Is the test “valid”? Do results represent the factor we are exploring? Example 1: One student gets a higher score on a test than another. Does the student with the higher score know the material better than the other student? What might influence the result?

Tests of Measurement Quality Validity Tests Is the test “valid”? Do results represent the factor we are exploring? Example 2: Project team 1 completes its project faster than Project team 2. Is project team 1 better than project team 2? What might influence the result? What factors other than speed might we test?

Tests of Measurement Quality Measures of Validity Content Validity – does the test adequately cover the subject under review? Example – reasons for outsourcing: 1- cost 2- speed 3- accuracy The number one result might be cost, but only because subjects were not able to give the real reason – lack of local talent, or low regard for a particular department.

Tests of Measurement Quality Measures of Validity Content Validity – does the test adequately cover the subject under review? Example – reasons for not using on-line learning 1- no access to computers 2- dislike for computers 3- poor grasp of English used in software What other reasons might there be for lack of usage? How would you know? What would you have to do to be sure your list was complete?

Tests of Measurement Quality Measures of Validity Predictive Validity – does the test predict the results we later find? For example, does the student with the higher grades actually know more when it comes time to use the knowledge? Example – A student who gets high grades in all accounting classes should easily pass the CPA exam. What if she doesn’t? Could the accounting class exams still be valid? How could you make the course exams more valid? Could there be other ways to find the predictive validity of course exams?

Tests of Measurement Quality Measures of Validity Predictive Validity – does the test predict the results we later find? Example 1: Your list of critical success factors shows that team training is the most important factor. How would you test the predictive validity of your results? Example 2: Your survey shows the top reason to outsource is the need for speed. How would you test the predictive validity of that result?

Tests of Measurement Quality Measures of Validity Concurrent Validity – does the test result agree with other findings? Example 1: Students taking your tests get As in your class, but they get Cs in other classes. Are the tests concurrently valid? How would you check the validity of the results?

Tests of Measurement Quality Measures of Validity Concurrent Validity – does the test result agree with other findings? Before you did your study, you read all prior research and you knew the results others had found. You find different results. Now what?

Tests of Measurement Quality Measures of Validity Construct validity: The degree to which your results match the results based on the primary theory upon which you based your research. Example – You predict that outsourced employees will be more productive because of Taylorism (they do the same job over and over and know it well), but your results show that their advantage is based on motivation– the need for money. Now what?

Tests of Measurement Quality Measures of Validity What would we have to do to determine if the results on last week’s midterm exam were valid? Content Validity Predictive Validity Concurrent Validity Construct Validity

Tests of Measurement Quality Measures of Reliability Reliability – the instrument produces consistent results. If we administer a questionnaire to the same people a week later, we would expect to get the same answers. What might cause the answers to change?

Tests of Measurement Quality Measures of Reliability Reliability – the instrument produces consistent results. If we administer a questionnaire to the same people a week later, we would expect to get the same answers. What might cause the answers to change?

Tests of Measurement Quality Measures of Reliability Reliability – the instrument produces consistent results. Stability – the same person gives the same answers. But this might not be the case. The person might be tired, busy, bored, angry with a co-worker. Since people are never truly stable, how much variation can you find and still determine a test is reliable?

Tests of Measurement Quality Measures of Reliability Reliability – the instrument produces consistent results. Equivalence – two test administrators should get the same results. My observations should be the same as yours. My interview results should be the same. What might cause the results to be different?

Tests of Measurement Quality Measures of Reliability Reliability – the instrument produces consistent results. Equivalence – two test administrators should get the same results. My observations should be the same as yours. My interview results should be the same. Problem – you find a questionnaire in a published study and administer it to 30 students, but you get different results than the people who created the survey. Now what?

Tests of Measurement Quality Measures of Practicality Practicality – we are looking for economy, convenience, and interpretability. Economy and convenience are obvious. Interpretability may be more important. When you get the results, will it be clear what they mean? Example – your results show that team members value leadership above all else. Can you define “leadership”? If you went to a CEO and said “ leadership is crucial to project success,” how would you answer if he said, “So what would make my managers better leaders?”

Tests of Measurement Quality Measures of Practicality Practicality – we are looking for economy, convenience, and interpretability. Interpretability - When you get the results, will it be clear what they mean? Example – You think students will be more likely to use on-line learning websites if the sites use images for clarity. How can you create your survey questions so the results are easy to interpret?

Tests of Measurement Quality Summary Measures of Validity Content Validity Predictive Validity Concurrent Validity Construct Validity Measures of reliability Stability Equivalence Measures of Practicality