1. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Your Name Your Institution Date Research and Assessment Techniques: Surveys

2. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Overview

3. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Outcomes 1. Create a survey plan 2. Identify reliable scales 3. Develop valid survey items 4. Design a survey instrument

4. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Icebreaker What is the difference between a good survey question and a bad survey question? A. Good survey questions tell you exactly what you want to hear. B. Good survey questions tell a factual story. C. Good survey questions evaluate the opinions or perceptions of individuals.

5. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Purpose and Use A survey is used to: Provide a snapshot of how a group of people feel and/or perceive reality at a given point in time (perceptual/attitudinal data). A survey should not be used to: Describe or explain facts.

6. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Survey Characteristics

7. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Maintaining Survey Integrity Measurement Error Sampling Instrument design External threats Validity Face Content Construct Reliability Internal consistency

8. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Creating a Survey Plan 1. Purpose statement 2. Sampling design 3. Methodology 4. Instrumentation 5. Implementation

9. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Types of Scales Dichotomous Yes/no Gender Nominal Age Job classification Ordinal High to low 8:00 am – 10:00 pm Interval 3 point frequency 5 point agreement 7 point satisfaction 10 point importance Tips: Choose your scale before you write your survey questions. Limit your survey instrument to 2-3 scales.

10. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Interactive Activity Work with a partner to label the scale choices in the worksheet.

11. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Key Scale Attributes 1. Variance 2. Discrimination 3. Accuracy 4. Symmetry Tip: Use the smallest number of points in a scale to satisfy the four scale attributes.

12. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Interactive Activity Work with a partner to critique the sample scales using the four scale attributes as a guidelines.

13. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Types of Questions Multiple Choice Open-ended Dichotomous Ranking Demographic Checklist

14. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Key Question Attributes 1. Focus 2. Brevity 3. Clarity Tip: Keep It Simple, Sweetheart.

15. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Question Attributes Jargon vs. Clarity Use words that are common or frequently used. Lack of Knowledge vs. Knowledge Don’t ask more than the respondent knows; Assume nothing Complexity vs. Simplicity Keep questions short and focused on a single issue or topic

16. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Question Attributes Barreling vs. Specificity Avoid complex questions by splitting them into two or more questions Leading/Loaded Questions vs. Reasoning Prompt respondent to answer with a reasoned opinion rather than an emotional response Loose Bundling vs. Anchoring Anchor the concepts in specific behaviors or contexts

17. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Interactive Activity Work with a partner to critique the sample instrument using the question attributes as a guidelines.

18. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Question Anchoring Positive Attributes Accurate Concise Timely Varied Widespread outreach Negative Attributes Inaccurate/Misleading Confusing Untimely Singular Narrow outreach Topic: The Quality of Communication of College Events Behaviors – Skills – Contexts – Values – Attitudes Tip: Use a minimum of three questions/items to assess a single topic.

19. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Sample Questions/Items I receive information about events on campus in time for me to make a decision about whether to attend. I receive clear information about the campus events (e.g. time & location). I don’t receive enough information about events on campus.

20. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Question Anchoring Positive Attributes Negative Attributes Topic: Effective Professional Development Activities Behaviors – Skills – Contexts – Values – Attitudes

21. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Question Anchoring Positive Attributes Offered at convenient times Relevant topics/activities Well-organized Adequate variety Engaging delivery Broadly advertised Negative Attributes Offered at inconvenient times Irrelevant topics/activities Disorganized Insufficient variety Boring Not widely advertised Topic: Effective Professional Development Activities Behaviors – Skills – Contexts – Values – Attitudes

22. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Sample Questions/Items I received enough information about professional development activities so that I could make a decision about attending. The topics for professional development activities were relevant to what I teach. The facilitator of the professional development activities was engaging.

23. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Interactive Activity Question Development: Review survey plan. Identify a topic(s) and anchor the concept with positive and negative attributes. Write three questions per topic.

24. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges Instrument Design Introductory Paragraph Purpose of survey, how info will be used, who will use info, voluntary and confidential Question Sequencing Easy to difficult, general to specific Demographics first Limit Scale to Two Different Types Limit Forced-choice Items to 25 Limit Open-ended Question to 2

25. Bridging Research, Information and Culture An Initiative of the Research and Planning Group for California Community Colleges The BRIC Initiative BRIC: http://www.rpgroup.org/projects/BRIC.html http://www.rpgroup.org/projects/BRIC.html The RP Group: http://www.rpgroup.org/ Contact: Rob Johnstone Project Director, rjohnstone@rpgroup.orgrjohnstone@rpgroup.org Priyadarshini Chaplot Project Coordinator, pchaplot@rpgroup.orgpchaplot@rpgroup.org