1. Week 14 More Data Collection Techniques Chapter 5
Research Methods
Week 14
More Data Collection Techniques
Chapter 5

2. Review – Data Collection
Primary Data
Observations
Interviews
Personal
Telephone
Questionnaires
Case Studies
Secondary data

3. Measurement Techniques
Scales for data
Nominal
Ordinal
Interval
Ratio

4. Measurement Techniques
Nominal scale
We use a number as a shorthand description
Male = 1
Female = 2 Or
Sultan Qaboos U = 2
University of Nizwa = 3
The number cannot be used mathematically. It makes no sense to divide Sultan Qaboos U by U Nizwa

5. Measurement Techniques
Nominal scale –
statistics possible on nominal data
We can count
We can calculate a mode (which number was the highest)

6. Measurement Techniques
Ordinal scale –
We align some results in order –
Key success factors
1 – teamwork – 44 votes
2 – leader training – 43 votes
3 - team member training – 33 votes
4 – size of budget – 12 votes
We have an order and can compare sizes (1 >2, 3 <2)
We cannot assume size of difference – 1 may not have twice as many votes as 2.

7. Measurement Techniques
Ordinal scale –
Allowable statistical procedures –
Count
Median
Key success factors
1 – teamwork – 44 votes
2 – leader training – 43 votes
3 - team member training – 33 votes
4 – size of budget – 12 votes

8. Measurement Techniques
Interval Scale –
There are equal distances between each number. But there is no absolute 0. A common example is temperature. Absolute 0 is only a theory for temperature.
Most of our research data does have an absolute zero -
0 – number of hours studied
0 – number of errors in data entry
0 – cost of motivational incentives

9. Measurement Techniques
Interval Scale –
Allowable statistical procedures:
Mean – 1.9 hours
Standard deviation

10. Measurement Techniques
Ratio Scale –
The scale has regular intervals and an absolute 0.
Example – number of hours each student used on-line learning
0 – 8 students
1 – 3 students
2 – 6 students
3 – 2 students
4 – 5 students
5 – 0 students

11. Tests of Measurement Quality
Validity Tests Is the test “valid”? Do results represent the factor we are exploring? Example 1: One student gets a higher score on a test than another. Does the student with the higher score know the material better than the other student? What might influence the result?

12. Tests of Measurement Quality
Validity Tests Is the test “valid”? Do results represent the factor we are exploring? Example 2: Project team 1 completes its project faster than Project team 2. Is project team 1 better than project team 2? What might influence the result? What factors other than speed might we test?

13. Tests of Measurement Quality
Measures of Validity
Content Validity – does the test adequately cover the subject under review?
Example – reasons for outsourcing:
1- cost
2- speed
3- accuracy
The number one result might be cost, but only because subjects were not able to give the real reason – lack of local talent, or low regard for a particular department.

14. Tests of Measurement Quality
Measures of Validity
Content Validity – does the test adequately cover the subject under review?
Example – reasons for not using on-line learning
1- no access to computers
2- dislike for computers
3- poor grasp of English used in software
What other reasons might there be for lack of usage? How would you know? What would you have to do to be sure your list was complete?

15. Tests of Measurement Quality
Measures of Validity Predictive Validity – does the test predict the results we later find? For example, does the student with the higher grades actually know more when it comes time to use the knowledge? Example – A student who gets high grades in all accounting classes should easily pass the CPA exam. What if she doesn’t? Could the accounting class exams still be valid? How could you make the course exams more valid? Could there be other ways to find the predictive validity of course exams?

16. Tests of Measurement Quality
Measures of Validity Predictive Validity – does the test predict the results we later find? Example 1: Your list of critical success factors shows that team training is the most important factor. How would you test the predictive validity of your results? Example 2: Your survey shows the top reason to outsource is the need for speed. How would you test the predictive validity of that result?

17. Tests of Measurement Quality
Measures of Validity Concurrent Validity – does the test result agree with other findings? Example 1: Students taking your tests get As in your class, but they get Cs in other classes. Are the tests concurrently valid? How would you check the validity of the results?

18. Tests of Measurement Quality
Measures of Validity Concurrent Validity – does the test result agree with other findings? Before you did your study, you read all prior research and you knew the results others had found. You find different results. Now what?

19. Tests of Measurement Quality
Measures of Validity
Construct validity: The degree to which your results match the results based on the primary theory upon which you based your research.
Example – You predict that outsourced employees will be more productive because of Taylorism (they do the same job over and over and know it well), but your results show that their advantage is based on motivation– the need for money. Now what?

20. Tests of Measurement Quality
Measures of Validity
What would we have to do to determine if the results on last week’s midterm exam were valid?
Content Validity
Predictive Validity
Concurrent Validity
Construct Validity

21. Tests of Measurement Quality
Measures of Reliability Reliability – the instrument produces consistent results. If we administer a questionnaire to the same people a week later, we would expect to get the same answers. What might cause the answers to change?

22. Tests of Measurement Quality
Measures of Reliability Reliability – the instrument produces consistent results. If we administer a questionnaire to the same people a week later, we would expect to get the same answers. What might cause the answers to change?

23. Tests of Measurement Quality
Measures of Reliability
Reliability – the instrument produces consistent results.
Stability – the same person gives the same answers. But this might not be the case. The person might be tired, busy, bored, angry with a co-worker. Since people are never truly stable, how much variation can you find and still determine a test is reliable?

24. Tests of Measurement Quality
Measures of Reliability
Reliability – the instrument produces consistent results.
Equivalence – two test administrators should get the same results. My observations should be the same as yours. My interview results should be the same.
What might cause the results to be different?

25. Tests of Measurement Quality
Measures of Reliability
Reliability – the instrument produces consistent results.
Equivalence – two test administrators should get the same results. My observations should be the same as yours. My interview results should be the same.
Problem – you find a questionnaire in a published study and administer it to 30 students, but you get different results than the people who created the survey. Now what?

26. Tests of Measurement Quality
Measures of Practicality Practicality – we are looking for economy, convenience, and interpretability. Economy and convenience are obvious. Interpretability may be more important. When you get the results, will it be clear what they mean? Example – your results show that team members value leadership above all else. Can you define “leadership”? If you went to a CEO and said “ leadership is crucial to project success,” how would you answer if he said, “So what would make my managers better leaders?”

27. Tests of Measurement Quality
Measures of Practicality Practicality – we are looking for economy, convenience, and interpretability. Interpretability - When you get the results, will it be clear what they mean? Example – You think students will be more likely to use on-line learning websites if the sites use images for clarity. How can you create your survey questions so the results are easy to interpret?

28. Tests of Measurement Quality
Summary
Measures of Validity
Content Validity
Predictive Validity
Concurrent Validity
Construct Validity
Measures of reliability
Stability
Equivalence
Measures of Practicality