1.  Cynthia J. Miller, Ph.D. Assistant Professor Dept. of Physiology & Biophysics University of Louisville

2. To reach a large number of participants To obtain baseline data and monitor changes To gauge motivations, beliefs, and attitudes To evaluate the effectiveness of teaching practices Time-efficient, convenient, and familiar To gather demographics To examine alignment between faculty and students

3.  Search for established and/or validated instruments o Consult the literature in your field o Verify that the instrument is beneficial for your use o Example: SALG (Student Assessment of their Learning Gains) Survey  If an appropriate instrument does not exist: o AKA “back to square one” o Example: CREATE Survey o Define your objectives o Work backwards-determine the data that you need to gather

4.  Forced vs. open-response items o Forced, Likert-scale most commonly used o Forced responses can be easier to analyze o Open responses can be useful due to multiple definitions/perceptions of questions, but also increases difficulty of analysis  Try to ensure anonymity if possible, guarantee confidentiality  Use simple terminology and unambiguous language  Check for bias-ensure you are not asking leading questions

5.  Ask questions from multiple perspectives and in different parts of the survey to ensure validity  Hoskins Paper o “The scientific literature is difficult to understand” o “I am confident in my ability to critically review scientific literature” o Negative statements are reverse-scored when quantifying data

6.  Revisit original goals/hypotheses and ensure alignment  Pilot questions with your research team, peers, a sample of target respondents, and a statistician

7. Pen-and-Paper  Benefits: Can ensure completion by respondents  Limitations: Must compile data by hand, lower perception of anonymity (particularly with open- response questions) Online  Programs such as SurveyMonkey, SurveyGizmo, Qualtrics  Most institutions have programs available for free or reduced prices  Benefits: Compile data, Produce graphs or charts, Statistical analysis, Larger potential sample sizes  Limitations: May lower response rates

8. Challenges  Low Response Rates o “Survey Fatigue” o Poor timing  Respondents not carefully completing survey Suggestions  Consider providing incentives for participation  Emphasize importance of results  Complete during class time or scheduled meetings  Endorsement by respected authorities

9.  Samples may not be representative of entire population  Self-reported data is often biased o Past events o Behaviors that may be negatively perceived  Respondents may feel the need to “tell you what you want to hear”

10.  Use multiple methods of data collection to analyze the same research question Faculty Surveys Student Surveys Exam Scores Student Focus Groups Faculty Surveys Classroom Observations

11. Student Surveys (Pre/Post) Mentioned past studies on critical- thinking skills, interviews, and validated surveys Student Surveys Student Exams (Pre/Post) Student Article Critiques Segura-Totten and Dalman Hoskins, Lopatto, and Stevens

12.  Survey Design o www.statpac.com/surveys www.statpac.com/surveys  Describing & Measuring Undergraduate Stem Teaching Practices o http://ccliconference.org/files/2013/11/Measuring-STEM-Teaching-Practices.pdf http://ccliconference.org/files/2013/11/Measuring-STEM-Teaching-Practices.pdf  Duke Initiative on Survey Methodology o http://dism.ssri.duke.edu/question_design.php http://dism.ssri.duke.edu/question_design.php  General Introduction o Fink AG. How to Conduct Surveys: A Step-by-Step Guide. Sage: Thousand Oaks, CA, 2008.  Question Development and Distribution Methods o DeVellis RF. Scale Development Theory and Application, 3 rd Edition. Sage: Thousand Oaks, CA, 2012. o Dillman DA. Mail and Internet Surveys: The Tailored Design Method, 2 nd Edition. John Wiley & Sons, Inc.: Hoboken, NJ, 2007.  Triangulation o Creswell N, Clark V. Designing and Conducting Mixed Methods Research. Sage: Thousand Oaks, CA, 2007.