Presentation on theme: "Using MIXED METHODS RESEARCH to Analyze Surveys"— Presentation transcript:
1 Using MIXED METHODS RESEARCH to Analyze Surveys Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
2 MMR itu apa? Sulit mendefiniskannya Contoh-contoh definisi The use of qualitative and quantitative techniques in both the collection and analysis of dataMixed Methods research is given a priority in the research and the integration of both the quantitative and qualitative results occurs at some point in the research processResearch that includes both quantitative and qualitative data in a single research study, and either the QUAN or QUAL data provides data that would not otherwise be obtainable when using only the primary methodMM Research is difficult to define because it is an emerging fieldHere I provide some examples of the different definitions. There are more. I like the third one here the best because it implies a practical reason for using the MM approach.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
3 Mengapa MMR penting? Answers questions that other modalities cannot Provides a deeper understanding of the examined behavior or a better idea of the meaning behind what is occurringThe inferences made with mixed methods research can be strongerMixed methods research allows for more divergent findingsMM research can include culture in the design by giving a voice to everyone involved in the behavior being examinedAn example of how MM research allows for more divergent findings is that it can simultaneously answer exploratory and confirmatory theoretical questions or it can answer the following questions: What is happening and Why is it happening?The examples shown here will show how all the voices were “heard” because even though most people were satisfied with the programs being evaluated there were still some identifiable issues when the MM piece was included in the analysis.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
4 MMR – KolaboratifBerupaya melibatkan stakeholders dalam disain dan proses penelitianCan be very beneficial when many of the stakeholders are more likely to be criticsIncludes less powerful groups and helps to ensure that they have an equitable impact on the researchCollaboration has the ability to stimulate ways of thinking that might not occur when working individually on a projectThis is something that can work for any research design. The difference with MM Research is that includes the MM component as well.Collaboratively developed research projects may be very common for many us. For me, I have often collaborate with many stakeholders and gather than input when conducting a study. They are more likely to be better consumers of the information when they are included in the process.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
5 Merancang MMRThe key to any study is the research question(s) because this dictates the selection of the research methodsIn designing a study the underlying purpose is the reason for doing it, and is a necessary componentWhy are we doing the study?The quality of the study and the meaningfulness of the results are enhanced if we are clear about the purposeOften, the underlying purpose for us is student success.The reason “why?” needs to connect the research question.The purpose is the reasons why the research is undertaking the study.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
6 Enam Kategori MMR Disain Eksplanatori Sekuensial Disain Exploratory SekuensialDisain Transformatif SekuensialDisain Triangulasi KonkurenConcurrent Nested DesignDisain Transformatif KonkurenI put these slides in here because I’m a researcher and I often like to categorize things. However, there aren’t any hard and fast rules in terms of the designs. Hopefully, these may help you to begin to think about how a MM Research Design might apply to some of your work.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
7 Desain Eksplanatori Sekuensial Collection and analysis of QUAN data followed by the collection and analysis of QUAL dataPriority is usually given to QUAN dataIntegration of QUAN and QUAL data usually occurs in the interpretation phase of the studyThe purpose is usually to use the QUAL results to help explain the QUAN resultsWITH the sequential designs, you may conduct a study which raises some additional questions, and conduct a second study to help answer those questions.One drawback here is that there are two separate phases.With the sequential explanatory design, the results from the first phase inform the development of the second phase.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
8 Desain Eksploratori Sekuensial Conducted in two phasesPriority is given to the first phase of QUAL data collectionThe second phase involves QUAN data collectionOverall priority is given to QUAL data collection and analysisThe findings are integrated in the interpretation phaseMost basic purpose is to use QUAN data to help interpret the results of the QUAL phaseWith the sequential exploratory design, the results from the first phase inform the development of the second phase.This type of design can help to identify or narrow the focus of possible variables with the QUAL phase, and inform the development of the QUAN phase.Can also help in the development of a new instrument.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
9 Desain Transformatif Sekuensial Mempunyai dua fase pengumpulan data yangt berbedaPerspektif teoritis dipakai untuk mengarahkan studiPurpose is to use methods that will best serve the theoretical perspective of the researcherWith the sequential transformative design, the results from the first phase inform the development of the second phase.Little research has been published using this type of design.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
10 Disain Triangulasi Konkuren This is probably the most familiar MM designThe QUAL and QUAN data collection are concurrent, and happen during one data collection phasePriority could be given to either QUAL or QUAN methods, but ideally the priority between the two methods would be equalTwo methods are integrated in the interpretation phaseThe integration focuses on how the results from both methods are similar or different, with the primary purpose being to support each otherSumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
11 Disain Nested Konkuren Gathers both QUAL and QUAN data during the same phaseEither QUAL or QUAN dominates the designThe analysis phase mixes both the QUAL and QUAN dataThe QUAL data is used to help explain or better understand the QUAN dataThis is the design demonstrated here and the one that I have found most useful.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
12 Disain Transformatif Konkuren Guided by a specific theoretical perspectiveThe QUAN and QUAL data are collected during the same phaseThe integration of data occurs during the analysis phaseThe integration of data could occur in the interpretation phaseAgain, the purpose is to use methods that will best serve the theoretical perspective of the researcherSumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
13 Proses Integrasi Data QUAL dan QUAN The process of integrating QUAL and QUAN research needs to be well thought out prior to the studyQUAL portion needs to be constructed in a way so that more novel information can be discoveredNeed to decide if QUAL portion is exploratory or confirmatoryIf exploratory, the purpose is to identify other dimensions that the QUAN portion is missingIf confirmatory, the purpose is to support the QUAN relationshipQUAL results can also be used to explain why there wasn’t a statistically significantly differenceIn examples shown here the purpose of QUAL was both exploratory and confirmatory. For instance, do the open-ended responses support the QUAN results and, if not, why not?May not know whether QUAL findings are confirmatory or exploratory until after the study is conducted.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
14 Arahan untuk Integrasi Hasil-hasil QUAL dan QUAN Selection of research methods need to be made after the research questions are askedSome methods work well in some domains and not in othersThere is no model of integration that is better than anotherWhen there are results that support each other, it is possible that both the QUAN and QUAL results are biased and both are not validThe main function of integration is to provide additional information where information obtained from one method only was is insufficientIf the results lead to divergent results, then more than one explanation is possibleNeed to first ask the research questions to know what best methods are. The research questions dictate the selection of research methods.XXXThe best model needs to be decided each time that a research project is conducted.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
15 Mengintegrasikan Data QUAL dan QUAN One process of incorporating QUAL data with QUAN data is known as quantitizing, or quantifying the open-ended responsesDummy Coding (i.e. binarizing) – refers to giving a code of 1 when a concept is present and a code of 0 if it is not presentA helpful tool for dummy coding open-ended responses is SPSS Text Analysis, now known as PASW Text Analysis for Surveys 3.0 run’s for $1,300.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
16 Menyajikan Temuan-temuan MMR As with any research findings, if they cannot be communicated to the people who can use the information than the findings are worthlessPresenting MM research can be more challenging because we are trying to communicate two types of information to readersFor instance, writing-up QUAN research is very well defined, and QUAL research is more often about discoverySumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
17 Menjamin relevansi temuan-temuan MMR Melibatkan semua stakeholders dalam merencanakan penelitianUsing MM research design may help a wider range of audiences connect to the materialMake sure to define the language used in the reportIt is important to decide how the MM research findings are going to be written: combined or separatelyDefining the language used in the report is important because MMs is a new field and there isn’t a lot of agreement on many of the definitions. For instance, triangulation may refer to methodology or different points of view.Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
18 Contoh MMR The form was not developed by IR Evaluated paid tutors on five job qualification areasJob skillsJob knowledgeWork habitsCommunication skillsAttitudeThree point rubric was used to evaluate paid tutorsDid not meet the requirementMet the requirementExceeded requirementsEvaluators were also asked to provide commentsMost closely matches a Concurrent Nested DesignGathers both QUAL and QUAN data during the same phaseEither QUAL or QUAN dominates the designThe analysis phase mixes both the QUAL and QUAN dataThe QUAL data is used to help explain or better understand the QUAN dataSumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
19 Contoh MMRHow did I combine the qualification ratings (QUAN) with the evaluator comments (QUAL)?Found an example of how to do this from Sandelowski (2003)Sandelowski provided an example where the QUAN responses were categorized and themes for each category were generated from the open-ended commentsSumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
20 Contoh MMR First step is to create the categories from the QUAN data This step involves being very familiar with your data, and also some creativityWith the paid tutor evaluation it was fairly easy to develop the categoriesPaid tutors who received a perfect rating in every category (n = 13)Paid tutors who had an average ranking equal to or above the mean (n = 5)Paid tutors who had an average below the mean (n = 11)Developing the categories for the paid tutor study was relatively easy. The two examples that I will go over later required a little more work.Overall average was computed by summing all of the job qualification category ratings and dividing by the number of job qualification categories (M = 2.51).Sumber: Keith Wurtz , Senior Research Analyst, Chaffey College.
21 Contoh MMR Mencampur data QUAL dan QUAN dalam fase analisis data After I created the three categories I printed out the comments associated with the paid tutors for each category and identified a theme for each oneSumber: Keith Wurtz , Senior Research Analyst, Chaffey College.