Understanding Qualitative Assessment Skills Session #3 A-Team Training November 13, 2007

Session II Overview of Outcome Design Creating Outcomes Service, Learning, & Development Session III Critiquing Outcomes 3 M’s Intro: Assessment Steps Session I Outcomes & Overview What is Assessment Assessment Language Comp Model & Foundation SALDOs Session IV Assessment Steps (cont) Intro: Assessment Plans Session V Review of Assessment Plans Session VI Assessment Plan Presentation Session Evaluation Foundation Session Outcomes Demonstrate an understanding of “A” language & literature Define and plan an assessment project for your department Increase Technical Skills— Perseus, Report Writing, Qual/Quan Perform 1 + assessment projects Assessment Resource for department and team

What is Qualitative Assessment? Qualitative research seeks descriptively rich data from a small, purposeful sample with meaning and understanding as it end goals (Lincoln & Guba, 1985). “The detailed descriptions of situations, events, people, interactions, and observed behaviors; use of direct quotes from people about their experiences, attitudes, beliefs, and thoughts; and analysis of excerpts or entire passages from documents, journals, or case histories…” (Patton, 1990).

Differences from Quantitative Assessment Qualitative -Focus of research on Quality (nature, essence) -Key concepts of meaning, understanding, and description Quantitative - Focus of research on Quantity (how much, how many) - Key concepts of statistical relationships, prediction control, description, hypothesis testing

Why is Qualitative Assessment important to student affairs work?

Outcomes of Qualitative Assessment Detailed descriptions Direct quotations Analysis of documents Deeply rich content

Key Assumptions in Qualitative Assessment Meaning Making, understanding Not interested in applying broadly to situations outside the case being studied The researcher is the primary means by which data are collected and analyzed Inductive  data leads to a finding/theory Social reality is constructed by the participants

What kinds of questions would one use as a guide for Qualitative Assessment? Why How Process Perception Reflection of experiences, feelings

Common Types of Qual Research Basic Interpretative Study Case Study Ethnography Phenomenology

Qualitative Techniques Interviews Focus Groups Document Analysis Observations

Sampling Key features: not random, smaller than quantitative, provide rich information Several types: (Patton, 2003)  Homogeneous-common characteristics  Stratified-sample representative of different characteristics of population (i.e. demographics, participants vs. non- participants)  Criterion-based on predetermined criteria  Snowball-ask participants for other potential participants

Data Analysis  Constant Comparative Method (Glaser & Strauss, 1967) Constantly compare from the get-go Example: Researcher journal-hunches, interview, transcribe, process is constant Often Data compared across multiple forms of data (interviews, essays, observations)

Coding Open Coding-General Selective Coding-Categorizing Thematic Coding-Themes

Coding Exercise Candy Exercise Interview Coding Practice

Coding Tips After interviews, write hunches down in journal Pay attention to what you are hearing as you transcribe, write down hunches Don’t overanalyze during general coding Use whiteboard or flip chart to start organizing categories (from general codes) No formula, but set up in way that best suits your skills (Remember: You are the research instrument )

Ensuring Rigor Researcher Journal  Hunches  Decisions  Thoughts  Biases, Assumptions Triangulation  Data triangulation  Researcher Triangulation Peer Reviewers

Focus Group Focus group-Interview a small group of participants (10-12 participants) Advantages/Disadvantages  Permits large amount of data gathered in small amount of time  Challenging to moderate multiple opinions and personalities  TIP: Employ another colleague to take notes!  Read “Creating a Focus Group” p (Joel’s article).

Any Questions? For other resources, please visit our website.