Qualitative data is information which does not present itself in numerical form and is descriptive, appearing mostly in conversational or narrative form. Words, phrases, text…

 Notebooks  Open-ended questions  Papers  Journal entries  On-line discussions, blogs   Twitter/ ‘tweets’  Notes from observations  Responses from interviews and focus groups

Qualitative analysis is the “interplay between researchers and data.” Researcher and analysis are “inextricably linked. ”

 Inductive process ◦ Grounded Theory (“Open coding”)  Unsure of what you’re looking for, what you’ll find  No assumptions  No literature review at the beginning  Deductive process ◦ Theory driven  Know the categories or themes using rubric, taxonomy  Looking for confirming and disconfirming evidence  Question and analysis informed by the literature, “theory”

 Coding process: ◦ Conceptualizing, reducing, elaborating and relating text– i.e., words, phrases, sentences, paragraphs.  Building themes: ◦ Codes are categorized thematically to describe or explain phenomenon.

What have students learned from a particular activity? What is the mental process of a student engaged in a particular intervention? What are the key transitions that occur as undergraduates acquire metacognitive- regulation skills? (Stanton et al., 2015)

Read through the student reflection paper and highlight words, parts of sentences, and/or whole sentences with some “code” attached and identified to those sections.

Why?

Read through this reflection paper and code based on this question: What does the term "scientific research" mean?

Why?

What does the term "scientific research" mean to you? What do you think doing scientific research entails?

1. Several individuals code material (triangulation). 2. Meet to decide on common set of terms. 3. Code again using common set. 4. Can go through steps 1-3 several times. 5. Analyze codes: qualitatively and/or quantitatively.

What is the research question of this study? What are the key transitions that occur as undergraduates acquire metacognitive- regulation skills? (Stanton et al., 2015) Why is qualitative analysis suited for looking at this research question?

Background Self-regulated learning (metacognition imp): understand what the task involves ID strengths and weaknesses Create plan for completing task (planning) Monitor how well plan is working (monitoring) Evaluate and adjust plan as needed (evaluating)

Let’s look at the Stanton paper: What was the experimental approach of this study?  Self-evaluation assignments after two tests E1-SE E2-FT Prompts The questions attempt to get at whether students are planning, monitoring and evaluating. Code student answers.

Let’s look at the Stanton paper: How did the authors approach the coding of student answers? With “metacognitive regulation in mind”. Theory-driven, not open coding.

Let’s look at the Stanton paper: How did the authors approach the coding of student answers? E1-SE: They developed a coding system (“sufficient/provides evidence” or “insufficient/provides no evidence”). Applied this to planning, monitoring, evaluating (self-regulated learning/metacognition).

Let’s look at the Stanton paper: How did the authors approach the coding of student answers? E2-FT: Coded for evidence that students followed study plan developed. Coding system (“yes/followed plan” or “no/did not follow plan”). Applied this to planning, monitoring, evaluating (self-regulated learning/metacognition).

Let’s look at the Stanton paper: Example of results

Let’s look at the Stanton paper: Example of results

Let’s look at the Stanton paper: Analysis of student answers led authors to propose a continuum of metacognitive regulation

Let’s look at the Stanton paper:

Don’t be scared! Most projects do not require this type of depth of qualitative analysis. Many studies can benefit from mixed methods (qual & quant). If you want to know student opinions, analyze long answer questions in exams, or essays, you may want to consider qualitative analysis.

 Use mixed methods, multiple sources.  Triangulate your data whenever possible.  Ask others to review your design methodology, observations, data, analysis, and interpretations (e.g., inter-rater reliability).  Note limitations of your study whenever possible.

 Read the literature: What is accepted in your field?  Read books or articles on qualitative research  Collaborate with a qualitative researcher  Or, just pick their brain over coffee

Designing and Conducting Mixed Methods Research, Creswell, J.W., and Plano Clark, V.L., 2006, Sage Publications. Discipline-Based Education Research: A Scientist’s Guide, Slater, S.J., Slater, T.F., and Bailey, J.M., 2010, WH Freeman. Educational Researchers: Living with a Lesser Form of Knowledge, Labaree, D.L., 1998, Educational Researcher, 27(8), Processing Field Notes: Coding and Memoing. In Writing Ethnographic Fieldnotes, 1995, Emerson, R.M., Fretz, R.I., and Shaw, L.L. (pp ). Chicago: The University of Chicago Press.