Evaluation and Assessment of Instructional Design Module #4 Designing Effective Instructional Design Research Tools Part 2: Data-Collection Techniques

Optimizing Data Collections Armed with an understanding of how one can approach a research study effectively, the next step in the natural journey is to focus on the collection of data. Critical Focus of This Part of Module #4 –Methods & suggestions for effective research studies Qualitative Research Quantitative Research Mixed Methods Research –Research-specific suggestions –Reasoning behind efficient data collection practices

Common Methods to Collect Data for Qualitative Research Studies Interviews –A formal or informal way to gather data from a subject within a research study. More commonly structured to stay on task and point –Can include questions that are open and/or closed. Closed-ended questions = commonly Yes/No Open-ended questions = chance to speak mind –Types of Interviews One-on-one, one-to-many, focus groups Interview sets (two or three at most)

Common Methods to Collect Data for Qualitative Research Studies Observations –A formal method to physically see what a subject is or not is doing. Use of behavioral categories – coding schemes to track a subject’s behavior in a study. –Ways to record observations Frequency (number ‘x’ happens) Duration (amount of time ‘x’ happens) Intervals (time between two (2) ‘x’ observations)

Common Methods to Collect Data for Qualitative Research Studies Observations –Ways to sample observations Time sampling – specific period of time –i.e. 30 minute intervals; usually for a group Individual sampling – different than an interview –Work with one person for ‘x’ time and then move on. Event sampling – analyze and observe a specific situation –Lecture capturing – newer method that’s popular Recording – cameras, voice recorders; perm. record – Watt, Vajoczki, Vine, et. al. (2014)

Common Methods to Collect Data for Qualitative Research Studies Diaries and journals –Daily recording of an occurrence, thought, or action Can be written down, spoken, or recorded electronically (audio or visually) –Optimal for time-sequenced studies Longer periods of time –Usually a smaller sample size Most complex option to choose from given common qualitative research data collection methods

Common Methods to Collect Data for Qualitative Research Studies Advantages of qualitative research methods –Rich data samples that come from physical observations –More optimal for educational research studies when sample populations are smaller in size –Attempting to come up with specific conclusions or to create a new field of knowledge Disadvantages of qualitative research methods –Not optimal for larger population sizes or samples –Difficult to determine patterns in subjects within studies –Labor and time intensive

Quantitative Research Generalities Sampling techniques –Probability sampling – random chance that a subject will be picked Simple random – equal chance Stratified random – divide sample into groups and then select participants from subgroups –Nonprobability sampling – purposeful picking Statistical references to quantitative research –Mean, median, mode, & ranges –Correlational studies – ‘x’ causes ‘y’ Hypothesis testing Moving along to the next type of research studies – quantitative research. While sampling was mentioned as it relates to qualitative research, it is a bit more specific in quantitative studies as researchers have the power to randomly group participants or purposely put then in a control or treatment group. The statistical concepts of mean or average, median or the frequency of the middle value, mode or the most frequently seen result, and range or the ranking of values from smallest to largest are important. Hypothesis testing is also essential.

Common Methods to Collect Data for Quantitative Research Studies Counting Studies –Recording of an event that happens over a specific interval of time –Use of frequency distributions to determine trends Histograms and pie charts Correlations between specific events The number of ‘x’ occurrences leads to ‘y’ happening a specific number of times. –Optimal for small to moderate-sized population samples. – C.D.C (2008)

Common Methods to Collect Data for Quantitative Research Studies Questionnaires –A tool which measures specific behaviors or patterns related to a specific subject or topic –Commonly used for self-reflection and assessment –Are subset of a survey Surveys –Most common tool from research reviewed for course for instructional design –Scaled questions using Likert scales (1-7 choices) –Can be provided or given as part of an interview – Domun & Bahadur (2014); Kaden & Patterson (2014)

Common Methods to Collect Data for Quantitative Research Studies Advantages of quantitative research methods –Quick and low cost way to gather data –Can be easier to determine trends in data over time –Optimal for larger population samples Disadvantages of qualitative research methods –Information overload –Can be difficult to comprehend conclusions from data studies One’s conclusion is likely to be different for another –Although less expensive, can take more time to analyze

Mixed Method Research Studies At A Glance Mixed Methods Research Studies –Combination of both qualitative and quantitative research method elements –More common of a method than about years ago Trends show this is most popular method from increased use of technology in education –Optimal for moderately-complex studies –A common tool that could be used in educational research and instructional design. – Lund (2012); Venkatesh, Brown, & Bala (2013)

Brief Review of Research Methods Educational & Instructional Design Research Mixed Methods Research Qualitative Research Quantitative Research

Common Methods to Collect Data for Mixed Method Research Studies Depends on the study that is being conducted –Commonly used methods to measure data Observations Interviews Questionnaires Surveys Educational research and instructional design research trends More popular due to flexibility of the research method being executed.

Goal of Educational Research To prove a point and extend further the body of knowledge –Technology focus in the classroom Mobile learning, online learning –Collaboration and learner engagement Relevant course materials and methods Keeping attention of learners –Attrition control Dropouts and transfers to other schools / institutions Do it in a way that is socially acceptable and ethically plausible