Research Methodology 尚惠芳 教授兼系主任 義守大學應用英語學系 98學年度第一學期 1

Outline Qualitative vs. Quantitative Research Literature Review and Research Problems Survey Research Qualitative Methods Mixed-Methods and Mixed-Model Designs Sampling Data Collection Data Analysis, Interpretation, and Reporting Writing Research Paper 2

Qualitative vs. Quantitative Research Section I Qualitative vs. Quantitative Research 3

Qualitative vs. Quantitative Research ( by James Neill) Qualitative Quantitative "All research ultimately has a qualitative grounding" by Donald Campbell "There's no such thing as qualitative data. Everything is either 1 or 0" by Fred Kerlinger The aim is a complete, detailed description. The aim is to classify features, count them, and construct statistical models in an attempt to explain what is observed. 4

Qualitative vs. Quantitative Research Researcher may only know roughly in advance what he/she is looking for. Researcher knows clearly in advance what he/she is looking for. Researcher is the data gathering instrument. Researcher uses tools, such as questionnaires or equipment to collect numerical data. 5

Qualitative vs. Quantitative Research Data is in the form of words, pictures or objects. Data is in the form of numbers and statistics. Subjective -- individual’s interpretation of events is important, e.g., uses participant observation, in-depth interviews, etc. Objective -- seeks precise measurement & analysis of target concepts, e.g., uses surveys, questionnaires, etc. 6

Qualitative vs. Quantitative Research Qualitative data is more 'rich', time consuming, and less able to be generalized.  Quantitative data is more efficient, able to test hypotheses, but may miss contextual details. Researcher tends to become subjectively immersed in the subject matter. Researcher tends to remain objectively separated from the subject matter. 7

Section II Literature Review and Research Problems 8

Literature Review and Research Problems I. Why Do a Literature Review? A. for planning primary research: 1. find a lack of consistency in reported results 2. find a flaw in research methods 3. conduct on a different population 4. find a solution for a problem 5. uncertainty about the interpretation 9

Literature Review and Research Problems B. As an end in itself: 1. inform practice (e.g. solve a problem) 2. provide understanding about the topic (longer process for integrative research review) 10

Literature Review and Research Problems II. What Is the Search Process? A. Nine steps: 1. Identify a research topic (cause → effect) 2. Review secondary sources to get an overview 3. Develop a search strategy (collect databases, titles, abstract, full texts) 4. Conduct the search (search the sources) 11

Literature Review and Research Problems 5. Get a copy of the article 6. Read and prepare bibliographic information and notes 7. Evaluate the research reports (identify major points) 8. Analyze the research findings and synthesize the results (narrative and statistical methods) 9. Use the literature review (based on major points) 12

Literature Review and Research Problems III. Research Questions and Hypotheses: A. four categories of research questions: 1. descriptive 2. normative 3. correlative 4. impact 13

Literature Review and Research Problems 1. Descriptive questions: Provide information about what is or has been happening related to the research topic (e.g., What is the major difficulty for EFL writers?) 2. Normative questions: Provide information compared with some standard or expected observation (e.g., Will EFL teachers achieve a better quality if they get a teaching certificate?) 14

Literature Review and Research Problems 3. Correlative questions: Identify relationship (e.g., What is the relationship between the year of training in writing and ss’ writing performance?) 4. Impact questions: Identify effect (e.g., What is the effect of teachers’ positive comments on ss’ writing performance?) 15

Literature Review and Research Problems 4-1. Two types of hypotheses: →directional hypothesis: can expect the outcome e.g., “If ss can get teachers’ positive comments, (then) they will have a better writing performance” →null hypothesis: can not expect the outcome e.g., “There is no difference between teachers’ positive comments and ss’ writing performance” 16

Literature Review and Research Problems IV. In Summary, the Purposes of Literature Review: Provide previous research results (big picture) Provide a need for additional research (problem) Develop a conceptual framework (from major points to generate research questions and hypotheses) Guide the research design and conduct the study 17

Section III Survey Research 18

Survey Research I. Main Purpose of Survey (Quantitative) Research: Collect data from a larger number of people Rely on individuals’ self-reports of their knowledge, attitudes, or behaviors II. Design Phase: Write the purpose of the survey (like a thesis statement) Write research questions based on the purpose (thesis) 19

Survey Research III. Three Design Considerations: Descriptive approach Cross-sectional approach Longitudinal approach 20

Survey Research Descriptive approach: A survey describing the characteristics of a sample at one point in time (e.g., senior students’ research paper writing difficulties in the Department of Applied English at I-Shou University in 2009) 21

Survey Research Cross-sectional approach: A survey examining several groups at one point in time (e.g., different grade level students’ writing difficulties in the AE Department at ISU in 2009) Pro: Compare responses across different grade levels in a shorter time Con: Not easy to make a comparison due to different subjects 22

Survey Research Longitudinal approach: A survey examining one group at different points in time (e.g., 1-year, 2- year-, and 3-year graduates’ perceptions of RP course in the AE Department at ISU in 2006-2008) Pro: Follow the same subjects over a period of time Con: Take a longer time 23

Survey Research IV. Data Collection Choices: Mail, telephone, personal interviews, email, web-based surveys, etc. Mail: Pro: Collect closed-ended information; low cost; enough time for the respondents Con: Lower response rates; no more in-depth information 24

Survey Research Phone interviews: Pro: Collect open-ended information; higher response rate; more additional information Con: More costly; can’t observe participants’ body language and facial expressions 25

Survey Research Personal interviews: Pro: Less structured approach; more conversational style; easy to get additional information; highest response rate Con: Take more time 26

Survey Research V. Sampling Plan: Identification of participants: Identify the participants who have the information you want or who have experienced the event Population definition: More specific about the participants’ sources 27

Survey Research Two Sampling decisions: 1. Probability sampling: Need to specify to whom the results will be generalized 1-1. Sampling frame: to give a list of people who have a chance to be selected 1-2. Only choose the population included in the sampling frame 2. Purposeful sampling: Need to provide sufficient details about the important characteristics of the participants 28

Survey Research Three sources of sampling errors: Coverage errors: (a) should be in the sampling frame, but not there, (b) should not be in the frame, but in there 2. Nonresponse errors: (a) refuse to be interviewed, (b) can’t complete questionnaire, (c) can’t be reached 3. Sampling errors: Use different (wrong) sampling strategies drawn from a population 29

Survey Research VI. Designing the Questionnaire: A. Review the literature: Borrow what has been designed before (with appropriate citations and permissions) B. Develop your own questionnaire: 1. follow Delphi Technique: Ask experts to provide ideas for better questionnaire, and then do a pilot testing with a larger sample 30

Survey Research 2. Explain to yourself why asking each question 3. Decide format: Open (answer questions in own words) or closed format (multiple-choice, true-false, checklist) 4. Do a pilot study to include all alternative responses 5. Avoid psychologically threatening (socially desirable) questions 6. Clear items: Provide a definition of the terms first 7. Short items 31

Survey Research 8. Avoid negative wording questions (e.g., Which of these are not ……?) 9. Avoid items that ask more than one idea 10.Use appropriate word level 11.Avoid leading questions 12.Emphasize critical words by using italics or underlining or bold letters 32

Survey Research VII. Formatting the Questionnaire: Make it attractive Organize and lay out the questions Number the items and pages Put the returned address and name at the beginning and end of the questionnaire Include brief, clear instructions 33

Survey Research 6. Use examples if necessary 7. Organize the questions in a logical sequence 8. Begin with a few interesting and nonthreatening items 9. Do not put the most important items at the end 10.Avoid using questionnaire or checklist on the form (use the title “Response Form”) 34

Survey Research VIII. Pilot Testing the Questionnaire: Select a sample similar to your population Give more spaces for writing comments Encourage pilot participants to give suggestions Follow the procedures that you plan to use in your study Add, change, or delete any questions if necessary 35

Survey Research IX. Special Types of Questions: 1. Demographic questions: 1-1. Background information (e.g., gender, age, etc.) 2. Nonthreatening behavioral questions: 2-1. Ask general questions on usual behavior first 2-2. Bounded recall: “e.g., In the last year……..” 2-3. Wording of question: “e.g., Are your difficulties in writing a research paper mainly due to your grammar problem?” 36

Survey Research 3. Threatening behavioral questions: Any questions elicit a defensive reaction in the participant; better to use : 3-1. open-ended questions 3-2. longer questions with explanations 3-3. familiar words 3-4. put answers into sealed envelopes 3-5. use appropriate wording (e.g., “Did you ever happen to ….”, “Such behaviors occur with different frequencies….”) 3-6. use multiple measures until getting the truth 37

Survey Research 4. Knowledge questions: Ask questions to test participants’ knowledge of a subject in school; better to use: 4-1. knowledge questions before asking attitude questions 4-2. appropriate level of difficulty 4-3. avoid asking “Do you happen to know …..?” 4-4. avoid using mail survey (may check up the answer) 38

Survey Research 5. Attitude questions: Should give a definition of the term or concept in advance; better to ask: 5-1. three kinds of attitudes: 5-1-1. affective (like vs. dislike): How does the person feel about this? 5-1-2. cognitive (knowledge): What does the person know about this? 5-1-3. action: What is the person willing to do about this? 39

Survey Research 5-2. assess attitude strength (e.g., How much do you like or dislike ….?) 5-3. avoid asking more than one concept (e.g., Would you vote for me and peace or my opponent and war?) 5-4. use bipolar questions (e.g., Are you satisfied or dissatisfied with …….?) 5-5. move on to measure degree of positive or negative attitude (e.g., Are you strongly satisfied or moderately satisfied with …….?) 5-6. Use five points/alternatives on the rating scale 40

Survey Research X. Letter of Transmittal: A cover letter to specify the purpose of the survey; hints to motivate the returned questionnaires Appeal to authority: Need the well-known person’s signature Appeal to self-interest: “You are one of the few people with the intelligence to be able to help us with this issue.” Appeal to professional interests: “This is a very important question in our field (society).” 41

Survey Research 4. Appeal to altruism: “The results of this survey will be used to solve one of our students’ writing problems.” 5. Appeal to curiosity: Offer to send a copy of the results 6. Appeal to greed: Offer to send a monetary incentive 7. Appeal to a sense of connection: Enclose a tea bag or a pencil (more details will be shown in the consent form) 42

Survey Research XI. Conducting the Survey: Send out an advance (cover) letter Enclose the questionnaire with the transmittal letter Supervise the data collection Send a follow-up to nonrespondents 4-1. send again 4-2. send a postcard reminder 4-3. make a call 5. Control processing errors 6. Enter the data into the database 7. Clean up the data before you begin analysis 43

Survey Research XII. Notes Specific to Personal Interviews: Hold an introductory meeting to share the purpose, and get assurance to participate Schedule the interview time Make a flexible interview Don’t use yes or no questions; plan to ask open-ended questions (e.g., How do you feel about …..? What is you opinion about …..? What do you think about ….?) Conclude with open-ended questions: (e.g., Is there anything that I didn’t ask about…..? Is there anything that you wanted to tell me….?) Pretest your interview procedures 44

Survey Research XIII. Starting and Conducting the Interview: Briefly review the purpose of the interview and information needed Focus on what the participant is saying Sequence the questions from general to specific Ask for specific examples Ask a variety of different kinds of questions Avoid asking “why” questions; better to use “how come” Use role play or simulation questions (If I were your …..) Record the interview and take notes 45

Section IV Qualitative Method 46

Qualitative Method I. Main Purpose of Qualitative Research: Provide in-depth descriptions to interpret something by using case study, personal experiences, interview, observation Complexity, contextual, exploration, discovery, inductive logic 47

Qualitative Method II. Strategies for Qualitative Research: 1. Ethnographic research: Describe and analyze practices and beliefs of cultures and communities by interacting with people in their everyday lives 2. Case study: Detailed study of one individual (of a group) through observation 48

Qualitative Method 3. Phenomenological research: Seek the individual’s perceptions and meaning of a phenomenon or experience; “what is the meaning of the experience of this phenomenon for this person or group of people?” 49

Qualitative Method 4. Grounded theory: Develop a theory after gathering and analyzing the data 5. Participatory research: Co-research the issue cooperatively 6. Clinical research: Understand a problem within a clinical context 7. Focus groups: Observe the interaction within the group (e.g., group interviews) 50

Qualitative Method III. General Methodological Guidelines: “Interview” A. The researcher is the instrument B. Speaking for the other C. Focus on an area of inquiry D. Explore research sites E. Gaining permission F. Negotiating entry G. Entering the field H. Role of the researcher 51

Qualitative Method The researcher is the “instrument”: 1. what questions to ask 2. in what order 3. what to observe 4. what to write down 5. key person to collect and interpret data 52

Qualitative Method B. Speaking for the other: Retell someone’s story or experiences in a new way (re-writing) C. Focus on an area of inquiry: Explore the topic based on literature review and research questions (tentative) D. Explore research sites: Collect information from the place (location) you do the study by piloting, sampling, etc. 53

Qualitative Method E. Gaining permission: Try to get permission from the “gatekeepers” (those with power and authority) F. Negotiating entry: How to get the true information with protection of human beings and their organization G. Entering the field: Make plans to enter the field in the least disruptive manner 54

Qualitative Method H. Role of the researcher: Unbiased, objective, discover multiple views of the participants; more interactive 1. Supervisor (with authority) 2. Leader (e.g., scout leaders) 3. Friend (with respect) 55

Qualitative Method IV. Critically Analyzing Qualitative Research: Credibility Transferability Dependability Confirmability Authenticity Emancipatory 56

Qualitative Method A. Credibility: Internal validity (a correspondence between how the researcher observes and participants express viewpoints) 1. Prolonged and substantial engagement: Give enough time staying at the site 2. Persistent observation: Sufficient observation until identifying clear issues/picture 3. Peer debreifing: Discuss with the disinterested peers for better research ideas 57

Qualitative Method 4. Negative case analysis: If the finding doesn’t fit the hypothesis, seek more information and analyze the negative case 5. Progressive subjectivity: Be open-minded and keep changing if necessary 6. Member checks: Share research results with peers for comment 7. Triangulation: Check information from different sources or methods to find consistency (e.g., quantitative data = qualitative findings) 58

Qualitative Method B. Transferability: Thick description (provide sufficient details such as time, place, context) and multiple cases for external validity (generalize the results to other similar situation) C. Dependability: Reliability (maintain the same result without change no matter tested how many times) D. Confirmability: Objectivity (provide evidences by synthesizing data instead of researcher’s imagination) 59

Qualitative Method E. Authenticity: Present a balanced view of all values and beliefs Fairness: Display different views (pros and cons) Ontological authenticity: Show individual’s real experiences Catalytic authenticity: Get evidences of participants’ views by the “inquiry” process 60

Qualitative Method F. Transformative Paradigm Criteria: Positionality or standpoint epistemology: The research result can not contain all universal truth because a lot of factors may influence the result; should avoid showing the position or standpoint of the author Community: Know the research site well to link the research results to positive action within that community 61

Qualitative Method Attention to voice: Speak for the minority Critical reflexivity: Be sensitive to understand the psychological state of participants Reciprocity: Develop trust with participants Sharing the perquisites of privilege: Share research results 62

Section V Mixed-Methods and Mixed-Model Designs 63

Mixed-Methods and Mixed-Model Designs Definition and Characteristics: A. Use both qualitative and quantitative methods in the design, data collection, and analysis B. Quantitative analysis first, and then followed by content analysis of qualitative data 64

Mixed-Methods and Mixed-Model Designs II. Importance in Educational and Psychological Research: A. Obtain a more complete picture of human behavior and experience B. Provide broader and deeper understanding of the issue 65

Mixed-Methods and Mixed-Model Designs III. Three paradigms for deeper understanding: A. Pragmatic paradigm: 1. Researcher determines which method to use to answer a specific research question (i.e., find a problem/issue and use a method for solutions) 2. Conduct the study and anticipate the results based on researcher’s own values and explanations (the result may match their value system) 66

Mixed-Methods and Mixed-Model Designs B. Transformative paradigm: Emphasize minority group’s values and viewpoints for “social change” (“Transformation” means “change”) C. Mixed-method design: 4 options to apply pragmatic and transformative paradigms 67

Mixed-Methods and Mixed-Model Designs IV. Mixed-Methods/Models Design: Apply the quantitative and qualitative methods simultaneously (parallel) or sequentially; 4 design options: A. Pragmatic parallel mixed-methods B. Pragmatic sequential mixed-methods C. Transformative parallel mixed-methods D. Transformative sequential mixed-methods 68

Mixed-Methods and Mixed-Model Designs A. Pragmatic parallel mixed-methods: 1. Collect qualitative and quantitative data to answer research questions 2. Collect two types of data simultaneously or with small time lag (almost simultaneously) e.g., examine 16 cases by doing a quantitative questionnaire study and qualitative open-ended interviews with two independent phases (no data analysis) 69

Mixed-Methods and Mixed-Model Designs B. Pragmatic sequential mixed-methods: 1. Collect and analyze one type of data (e.g. quantitative) as a basis for collecting another type of data (e.g., qualitative) 2. The second phase of research is to confirm or disconfirm the first phase e.g., questionnaires first; use the results to develop qualitative interviews 70

Mixed-Methods and Mixed-Model Designs C. Transformative parallel mixed-methods: 1. Use qualitative and quantitative methods to promote social or political changes 2. Explore the minority’s populations 3. Emphasize value-based and action-oriented purposes (make changes for improvement) 71

Mixed-Methods and Mixed-Model Designs D. Transformative sequential mixed-methods: 1. Focus on the minority’s populations and values 2. One type of data provides a basis for collection of another type of data 72

Mixed-Methods and Mixed-Model Designs V. Questions for Critically Analyzing Mixed-Methods Research: A. Consider inference quality = internal validity (quantitative) = trustworthiness (qualitative) = consistency between research purpose, research questions, and methodology used 73

Mixed-Methods and Mixed-Model Designs B. Conflicts between qualitative (small sample) and quantitative (large sample), so need to link both data C. Need to consider a lot of factors e.g., language, gender, reading or writing levels (demographic information) 74

Mixed-Methods and Mixed-Model Designs V. Questions for Critically Analyzing Mixed-Methods Research: What are your research purpose and questions? Have you matched the purpose and questions to appropriate methods? To what extent do you reach the criteria of the quality for the quantitative part of the study? (i.e., internal validity, external validity, reliability, objectivity) 75

Mixed-Methods and Mixed-Model Designs 4. To what extent do you reach the criteria of the quality for the qualitative part of the study? (i.e., credibility, transferability, dependability, confirmability, authenticity, transformative) 5. How do you solve the conflict between the design and implementation of the study? 6. What is the limitations for generalization? 76

Mixed-Methods and Mixed-Model Designs 7. How do you integrate the results from the mixed methods? How do you explain the conflicting findings if necessary? 8. How do you control and explain the factors of gender, language levels, etc? 77

Section VI Sampling 78

Sampling Definition: Select a given number of people from a population A. Probability sampling: Every member of the population has a possibility of being selected B. Nonprobability (purposeful) sampling: Select members who had the particular experience before; try to look for negative cases for better improvement 79

Sampling II. External Validity: Generalizability or Transferability A. Generalize findings to the larger (target) population, so need to provide detailed description of the case III. Sampling Strategies: A. Probability sampling B. Nonprobability (Purposeful) sampling C. Convenience sampling 80

Sampling Probability sampling: 1. Simple random sampling 2. Systematic sampling 3. Stratified sampling 4. Cluster sampling 5. Multistage sampling 81

Sampling 1. Simple random sampling: Each member has a number (code) and each population has an equal chance to be selected →Pro: simple process →Con: difficult to get a complete list of all populations; may include some “outsiders” 82

Sampling 2. Systematic sampling: Select every nth name from the list; so need to estimate the needed sample size →Pro: not every member needs to be numbered →Con: bias if members are arranged in a specific pattern (e.g., choose the last name with A in the given year) 83

Sampling 3. Stratified sampling: Divide the populations into subgroups (e.g., genders) or levels (e.g., proficiency levels) and then draw randomly from each subgroup →Pro: easy to compare each subgroup results →Con: must get information before dividing them 84

Sampling 4. Cluster sampling: Choose groups of individuals e.g., city blocks or classrooms in a school, and study all of the samples there →Pro: save time and money by collecting data at a limited number of sites →Con: small sample size, so less precision in estimating the effect 85

Sampling 5. Multistage sampling: Combine sampling strategies e.g., use cluster sampling to randomly select classrooms and use random sampling to select a sample with each classroom →Pro: more reliable →Con: complex calculations (regression analysis) 86

Sampling B. Nonprobability (Purposeful) sampling: Provide in-depth, interpretive information 1. Extreme or deviant cases 2. Intensity sampling 3. Maximum-variation sampling 4. Homogeneous sampling 5. Typical-case sampling 6. Stratified purposeful sampling 87

Sampling 7. Critical-case sampling 8. Snowball or chain sampling 9. Criterion sampling 10. Theory-based or operational construct sampling 11. Confirming and disconfirming cases 12. Opportunistic sampling 13. Purposeful random sampling 14. Sampling politically important cases 88

Sampling 1. Extreme or deviant cases: Study the extreme (unusual) case to illuminate the ordinary e.g., analyze an extremely best writer’s work and compare it with the poorest one 2. Intensity sampling: Similar to extreme cases, but less emphasis on extreme; explore rich information on typical cases; analyze why individuals meet the specified criterion 89

Sampling 3. Maximum-variation sampling: Maximize the variation within the samples and then indicate their unique (major) difference e.g., study students’ English abilities among different locations (rural, urban, and suburban areas) 4. Homogeneous sampling: Identify the samples who share similar characteristics e.g., rural teachers’ attitudes toward bilingual learning for kindergarten kids 90

Sampling 5. Typical-case sampling: Choose a case in which a program/instruction has been implemented to show this case is indeed average 6. Stratified purposeful sampling: Divide samples into subgroups and then select cases within each subgroup 7. Critical-case sampling: Study a very important, critical case and the effect should be representative: “if it happens there, it will happen anywhere.” 91

Sampling 8. Snowball or chain sampling: Start with a key person and introduces the next one to become a chain 9. Criterion sampling: Set up a criterion and identify cases that meet that criterion e. g., study cases that could pass TOEFL last semester 10. Theory-based or operational construct sampling: Define a theoretical construct (e.g., metacognitive learning on EFL reading development) 92

Sampling 11. Confirming and disconfirming cases: Form grounded theory by analyzing sample cases that fit (confirming) and do not fit (disconfirming) the major points in literature 12. Opportunistic sampling: Should take the opportunity to decide the sampling procedure or samples during the study 93

Sampling 13. Purposeful random sampling: Randomly select participants who had similar experiences in a very small sample 14. Sampling politically important cases: Use particular samples that someone expects to use the results 94

Sampling Overall strengths and weaknesses of purposeful sampling: Pro: 1. Less costly and time consuming 2. Ease of administration 3. Assures high participation rate 4. Generalization possible to similar subjects 5. Assures receipt of needed information Con:1. Difficult to generalize to other subjects 2. Less representative of an identified population 3. Greater subject bias 95

Sampling C. Convenience sampling: Choose the samples who are very available (easy to access), but can not generalize the results beyond the population poor Pro: 1. Less costly and time consuming 2. Ease of administration 3. Assures high participation rate 4. Generalization possible to similar subjects Con:1. Difficult to generalize to other subjects 2. Less representative of an identified population 3. Greater subject bias 96

Sampling IV. Access Issues: Consider how to obtain your participants A. Get permission and agreement with the appropriate person (e.g., school principal, classroom teacher, or parents) B. Obtain consent form from the participants C. How to “label” students if using stratified sampling D. Consider appropriate sample size 97

Sampling V. Sample size: A. Quantitative research rules: For survey research, 100 cases B. Qualitative research rules: For grounded theory, 30-50 interviews 98

Sampling VI. Consent form: A. Explain research purpose, duration, and procedures B. Describe any risk or discomfort C. Describe confidentiality or anonymity D. Provide the name of person to contact with E. Provide voluntary participation, and available to refuse or withdraw any time 99

Dear Principle of I-Shou International School, My name is Chiau-jen Zhu, and I’m a junior student of the Applied English Department at I-Shou University. In order to accomplish my required research, “The Effect of Reading Stories to Improve Elementary School Students’ Reading Ability”, I need your permission to interview students at your school. Reading stories has become a tendency to improve elementary school students’ reading ability. Therefore, my purpose is to interview students to get more ideas about the effect of reading literature. The interview includes three parts: past experience, the details of experience and personal opinions of participants. The participation is not compulsory, but I will appreciate your contribution if you agree with the interviews. Each interview will take 15 to 25 minutes, and five students are needed from the fifth to the sixth grade. In addition, the names of participants will be innominate for sure. As for the contents of interviews, the response will be summarized, and the raw data will not be presented in the final report. Besides, the participants have the rights to read and withdraw any time. The results of this study will be provided to my teachers and supervisors only, which will be very useful for their teaching. Thank you for your assistance in this study. Sincerely yours, Chiau-jen Zhu Junior, Applied English Department I-Shou University E-mail: gekkoh-159@yahoo.com.tw 100

101

Section VII Data Collection 102

Data Collection Why collect data? To know what certain people are thinking or doing by asking them questions Quantitative data collection: Use questionnaires and show “numbers” to inform the user the “information” A. Primary data: To serve our specific purpose B. Secondary data: Collected by others 103

Data Collection III. Asking questions: Get information (data) by filling in the questionnaire as the basis of data collection for email, postal and telephone surveys and structured and semi-structured interviews A. Structured interview = closed questions = answer predetermined items B. Semi-structured interview = open-ended questions = answer freely in own words (e.g., What were the main difficulties for RP writing?) 104

Data Collection 105

Data Collection 1. Classificational questions: To classify respondents by gender, age, occupation, etc. (personal information) e.g., Age last birthday? □ Under 18 □ 18-25 □ 26-30 □ 31-40 □ 41-50 106

Data Collection 2. List questions: To ask respondents to select one answer e.g., Specify the length of learning English. □ less than 3 years □ 4-6 years □ 7-9 years □ 10-12 years □ more than 12 years 107

Data Collection 3. Ranking questions: To give an order or preference e.g., Rank the following factors in order of importance if you want to be a good English writer. (from 5 the most important to 1 the least important) □ grammar □ vocabulary □ background knowledge □ reading ability □ Chinese language ability 108

Data Collection 4. Scale (rating) questions: To determine the strength of views or opinions e.g., How likely do you like (agree) grammar instruction? Very likely (Agree strongly) 5 Quite likely (Agree slightly) 4 Neither likely (Agree) nor unlikely (Disagree) 3 Quite unlikely (Disagree slightly) 2 Very unlikely (Disagree strongly) 1 109

Data Collection C. Framing questions: 1. Avoid bias and leading questions (e.g., Would you agree that ……?) 2. Avoid shorthand: e.g., Would an application of CALL …….? 3. Avoid ambiguous words: e.g., “usually, frequently, or recently” 4. Avoid long questions: Short and simple questions are more effective. 110

Data Collection 5. Avoid negative questions: e.g., Do you never ……..? 6. Avoid hypothetical questions: e.g., If you were a millionaire, would you …….? 7. Avoid irrelevant questions 8. Make respondents to feel easy to answer questions: e.g., What did you do last week? (too broad) Have you done any of the following things last week? (easier to answer) 111

Data Collection 9. Avoid yes-no questions 10. Provide “classification” type data (e.g., age, sex, previous training in writing, etc.) for demographic information 11. Add comments (e.g., In your opinion, what ….?) for in-depth information 112

Data Collection IV. Qualitative Data Collection: Collect data by interviewing and conducted individually or in a group 1. Focus group interviews: Fewer than 10 questions 2. Open-ended questions: Allow more self-reflections 3. Avoid using “why” questions: Use “how come” or “what” 4. Carefully develop the questions by doing pilot testing 5. Provide enough information for understanding 6. Arrange questions from general to specific 113

Data Collection 7. Listen more, talk less, and ask “real questions” (the interviewer doesn’t already know or anticipate the response) 8. Try to get “inner voice” instead of “superficial” information 9. Take notes and bring tape-recording: Easy to transcribe 10. Ask questions when you do not understand: Ask for examples or concrete details 114

Data Collection 11. Explore laughter, pauses, facial expressions 12. Ask participants to tell a story: To get a complete picture (transcription model 1 and model 2) 13. Follow your instincts: Try to feel participants’ feelings 115

Data Collection V. Benefits of using a questionnaire in interviewing: A. To ensure covering all of the questions B. Can explore more in-depth information within each of the questions C. Collect both quantitative and qualitative data altogether 116

Section VIII Data Analysis, Interpretation, and Reporting 117

Data Analysis, Interpretation, and Reporting I. Qualitative analytic strategies: A. Recursive process: → analyze cases→ generate findings→ draw conclusion → form grounded theory → write report B. Nine qualitative data analysis principles: 1. Collect the data at the site and carefully study all the data to seek similarities and differences, concepts and reflections 118

Data Analysis, Interpretation, and Reporting 2. Saturation and sufficiency of information: Never stop data analysis until the emergence of regularities; i.e., no new information emerges with additional data analysis 3. Accountability of information: Keep notes or transcripts if outsiders want to review the data analysis procedures and results 4. Divide the data (excerpts) into smaller units related to your major points 119

Data Analysis, Interpretation, and Reporting 5. Organize the smaller units into categories (based on major points) 6. Build conceptual similarities, find negative evidences, and discover patterns 7. Modify categories as further patterns occur 8. Analyze negative cases to reflect their perceptions 9. Synthesize the patterns into the grounded theory 120

121

Data Analysis, Interpretation, and Reporting * SHOULD BE: - connected with what is being discussed in the major points - exact wording (excerpts) used in the statement * SHOULD NOT BE: - based on interviewer’s personal opinions - irrelevant to the major points 122

Data Analysis, Interpretation, and Reporting C. Six Steps in Qualitative Data Analysis: Give codes from the notes (transcripts) Note personal reflections or other minority’s comments Sort the notes to identify similar and different relationships between patterns Identify these patterns, similarities, and differences Elaborate a small set of generalizations that cover the consistencies Examine those generalizations and form grounded theory (see “Content Analysis”) 123

Data Analysis, Interpretation, and Reporting Grounded Theory (紮根理論): → a process of constructing various data → inductive process by collecting, analyzing, and comparing data systematically → theory is grounded on data to explain the phenomena 124

Grounded Theory (by Marlene Pomrenke) Grounded theory is a method of social inquiry associated with a qualitative approach to research. This inductive research process utilizes generalized knowledge that is derived from specific observations of phenomena from the field. In turn, this can be used to build theory. For example, grounded theorists aim to create theoretical categories from collected data and then analyze relationships between key categories (Charmaz, 1990). Indeed, the main purpose of using a grounded theory approach is to develop theory through understanding concepts that are related by means of statements of relationships (Strauss & Corbin, 1990). Using the concepts from grounded theory, this study starts from understanding the experience of the research participants (i.e., how they construct their worlds). The data analysis stage focused on finding recurrent themes or issues in the data, and finally into developing or refining a theory about the phenomenon. 125

Data Analysis, Interpretation, and Reporting D. Grounded Theory Analysis Strategies: 1. Recur by moving back and forth with the data, analyzing, collecting more data, and analyzing some more until reaching conclusions 2. An interactional method of theory building by comparing and analyzing the data 126

Data Analysis, Interpretation, and Reporting 3. Three steps in the grounded theory analytic (coding) process: (1) Open coding: Break data into small parts → compare for similarities and differences → explain the meanings of the data by focusing on “who, when, where, what, how much, why” (ask questions to get a clear story) (2) Axial coding: After open coding, make connections (sort) between categories and confirm or disconfirm your hypotheses (3) Selective coding: Select the core category (match hypotheses) and explain the minor category (against hypotheses) with additional supporting data 127

Coding Process MP 1 MP 2 MP 3 Open Coding Axial Coding Selective Coding MP 1 MP 3 128

Data Analysis, Interpretation, and Reporting II. Interpretation Issues in Qualitative Data analysis: A. Triangulating data: Use multiple methods and data sources to support the strength of interpretations and conclusion (e.g., semi- structured interviews, consent form, grounded theory, etc.) 129

Data Analysis, Interpretation, and Reporting B. Audits: Questions to examine the data for interpretations and conclusion 1. Is sampling appropriate to ground the findings? 2. Are coding strategies applied correctly? 3. Is the category process appropriate? 4. Do the results link hypotheses? 5. Are the negative cases explained? 130

Data Analysis, Interpretation, and Reporting * Four steps of negative case testing: 1. Make a rough hypothesis 2. Conduct a thorough search 3. Discard or reformulate hypothesis 4. Examine all relevant cases 131

Data Analysis, Interpretation, and Reporting C. Cultural bias: Discuss cultural differences with different groups of participants (compare the differences between western and Taiwanese students’ attitudes) D. Generalization: Not appropriate for qualitative research 1. Case-to-case transferability by providing thick description to apply to another setting 2. Generalize the result to a broader theory (e.g., use deviant cases) 132

Data Analysis, Interpretation, and Reporting III. Writing Research Reports: A. Introduction B. Literature Review C. Methodology D. Results: Tie the results to study purpose (hypotheses) E. Discussions and Conclusion: Tie discussions to the literature; recommendations for practice; limitations of the study 133

Data Analysis, Interpretation, and Reporting Quantitative reports: Report results by the use of tables and graphs Avoid first-person pronoun Use passive voice Qualitative reports: Look for a deep description (narrative style) Look for well-grounded theory Seek contextual meaning by understanding demographic information (different experiences) 134

Writing the Research Paper Section IX Writing the Research Paper 135

Writing the Research Paper Chapter I: Introduction (Problem Statement) A. Area of Study: Provide a general introduction to the topic; outline the problem B. Definition of Terms: Define important terms and concepts C. The purpose of the study D. Assumptions: Make hypotheses for the proposed study 136

Writing the Research Paper Chapter 2: Literature Review A. History: Provide a review of the historical background B. Current Literature: Include a review of current relevant literature with the analysis of pros and cons C. Research Problem: Provide research questions 137

Writing the Research Paper Chapter 3: Methodology A. Research Questions and Hypotheses: Propose them based on literature review (major points) B. Research Design: * Quantitative research: Use a survey (three design considerations; discuss how to design questionnaire) * Qualitative research: Use interviews (present a rationale for the design of choice) 138

Writing the Research Paper C. Sample: * Quantitative research: Describe sampling techniques and rationale for the method used for selecting the sample and sample size * Qualitative research: Describe the criteria for the selection of the participants and setting 139

Writing the Research Paper D. Data Collection Procedures: * Quantitative research: Describe the procedures that the data will be collected by using a survey * Qualitative research: Describe the use of audiotapes, note-taking E. Pilot Testing: Describe the pilot study procedures and any adjustments 140

Writing the Research Paper F. Data Analysis Procedures: * Quantitative research: Describe how you handled the data; provide statistical procedures (e.g., compute mean score or frequency) * Qualitative research: Describe data analysis strategies; explain multiple sources of data G. Limitations of the Study: Explain anticipated limitations of the study 141

Writing the Research Paper Chapter 4: Results: Just report what the data say (you may give the subheading based on each hypothesis) A. Findings from the quantitative results (% or mean) B. Findings from the qualitative results (3 excerpts) C. Fit into your hypotheses or not (e.g., broadly/partially/do not fit) 142

Writing the Research Paper Chapter 5: Discussions and Conclusion A. Interpret the meanings of data (discuss “why” your participants would think/say so) B. Discuss whether the findings fit into the literature or not C. Form any grounded theory? D. The results will be useful to “whom”? E. Any limitation to reach the generalizability F. Draw conclusions and suggestions (about improvement) based on your or minority’s opinions 143

Writing the Research Paper Outline of Research Paper 144