1. Evaluation Research Step by Step Step 1 – Formulate research question Step 2 – Operationalize concepts ◦ Valid and reliable indicators Step 3 – Decide on sampling technique ◦ Draw sample Step 4 – Select data collection technique ◦ Collect data Step 5 – Analyze data Step 6 – Write up the report

2. Evaluation Research Step by Step Steps 1 and 2 have already been covered in previous power points

3. Step 3 - Drawing A Sample Identify Population Decide on either random (probability) or nonrandom (nonprobability) sampling technique

4. Sampling Terms to know ◦ Population: Total groups to be studied ◦ Case: Units to be sampled  (individual, county, etc.) ◦ Sample: Group(s) to be interviewed ◦ Sampling Frame: List from which we obtain the sample  (a roster from a class, or a directory from an organization, etc.)

5. Sampling Types of Sampling Designs ◦ Simple Random Sampling  Single number assigned to each case  Table of random numbers is used  Assumes a complete list of the population  Laborious ◦ Systematic Sampling  Every “x”th element from a sample frame is selected  Determine size needed  Take random start  Select every “x”th element ◦ Probability Proportionate to Size (PPS)  Larger ones (clusters) have greater probability of being selected

6. Sampling Types of Sampling Designs ◦ Stratified Sampling  Sample separately from different key categories Example: Determine number needed from each racial/ethnic group (this number should be approximately the same for each group). Total ## Needed African Americans300100 Hispanics200100 Asian Americans100100 Anglo Americans500100 Total Population1000400 Sample separately from each group taking proportion needed to get approximately equal numbers from each racial/ethnic group African AmericansNeed 50% sample HispanicsNeed 50% sample Asian AmericansNeed 100% sample Anglo AmericansNeed 20% sample

7. Sampling Types of Sampling Designs ◦ Multistage Cluster Sampling  Used when don’t have a complete list  Sample in stages (units within units) Example of multistage random sample: (1) Universities USU U of UBYU (2) Colleges Engineering Agriculture (3) Departments Landscape, Architecture and Environmental Planning (LAEP) Plants, Soils and Climate Department (PSC) (4) Classes Theory of Design Landscape Materials (5) Students Molly Trish Jim

8. Determining Sample Size Appendix 2 Recommended sample sizes for two different precision levels Population Size 101027516374 151430017276 201932518077 252435018778 302837519480 353240020181 403642520782 454045021282 504447521883 554850022283 6052100028691 6556200033395 7059300035397 7563400036498 8066500037098 8570600037598 9073700037899 9576800038199 1008151900038399 125965610,00038599 1501106115,00039099 1751226420,000392100 2001346725,000394100 2251447050,000397100 25015472100,000398100 Sample Size +/- 5 % +/- 10% Population Size Sample Size +/- 5 % +/- 10% N D2D2.9604 N + 1 =

9. Nonrandom Sampling  Availability or Convenience Samples  Select those who are available or convenient  Quota Samples - select representative size of different types of respondents  Purposive Samples  Need specific types of information – select those who will have that information  Theoretical Samples  Theory specifies characteristics of your units of analysis. For instance, a theory of inequality may require you to have respondents at different levels of income.  Snowball Samples  Hard to Reach – Hard to identify populations

10. Nonrandom – Nonprobability Sampling  Availability or Convenience Samples  Sample classrooms of students here on campus because they are available and convenient  Availability or Convenience Using Quota  Sample classrooms of students here on campus because they are available and convenient, BUT  Select classes with approximately equal numbers of men and women because there are approximately equal numbers in university

11. Nonrandom - Nonprobability Sampling  Purposive Samples  Select key informants who can provide information you need  If you want to look at community leader’s attitudes toward homeless then you would select  Law enforcement  Civic leaders/policy makers  Heads of social service agencies  Three guidelines  Knowledge about the culture/situation or experience  Willing to talk  Representative of the range of points of view  Two tests  Completeness – overall sense of what they think  Saturation – not hearing anything new

12. Nonrandom – Nonprobability Sampling  Theoretical Samples Theory specifies characteristics of your units of analysis Example – Research on parent/teacher interaction using Expectation States Theory Research Questions - How do parents and teachers interact? How is interaction influenced by race and social class of parent? How are expectations influenced by socioeconomic characteristics of parents? Theory specifies that as inequality between parent and teacher increase, so does “expert power” of teacher and conformity of parent

13. NonRandom- Nonprobability Sampling  Snowball Samples Key informant or GATEKEEPER who then refers you to someone else Sample size “snowballs” or increases rapidly Used when you are working with groups that are reluctant to be studied, and you need to gain confidence of group members Example of groups where snowball samples are used– Gang members, Alcoholics Anonymous members, etc.

14. Ways that People Generally Gather Information – Data Collection Techniques Participant Observation Content Analysis Surveys/Intakes/Questionnaires This excludes non-empirical ways of collecting information (intuition, inspiration, 6 th sense, etc.)

15. Types of Data Collection Techniques Paired with Most Likely Sampling Technique Participant Observation – generally uses nonrandom sample Content Analysis – generally uses nonrandom sample Surveys/Intakes/Questionnaires – generally uses random sample This excludes non-empirical ways of collecting information (intuition, inspiration, 6 th sense, etc.)

16. Questions or comments? Please contact: Carol Albrecht Assessment Specialist USU Extension 979-777-2421 carol.albrecht@usu.edu