1. 研究論文寫作 Research Paper Writing 尚惠芳 教授兼系主任 義守大學應用英語學系 1 98 學年度第二學期

2. 2 Outline I. Qualitative vs. Quantitative Research II. Literature Review and Research Problems III. Survey Research IV. Qualitative Methods V. Mixed-Methods and Mixed-Model Designs VI. Sampling VII. Data Collection VIII. Data Analysis, Interpretation, and Reporting IX. Writing Research Paper

3. 3 Qualitative vs. Quantitative Research Section I

4. 4 Qualitative vs. Quantitative Research ( by James Neill) QualitativeQuantitative "All research ultimately has a qualitative grounding" by Donald Campbell "There's no such thing as qualitative data. Everything is either 1 or 0" by Fred Kerlinger The aim is a complete, detailed description. The aim is to classify features, count them, and construct statistical models in an attempt to explain what is observed.

5. 5 QualitativeQuantitative Researcher may only know roughly in advance what he/she is looking for. Researcher knows clearly in advance what he/she is looking for. Researcher is the data gathering instrument. Researcher uses tools, such as questionnaires or equipment to collect numerical data. Qualitative vs. Quantitative Research

6. 6 QualitativeQuantitative Data is in the form of words, pictures or objects. Data is in the form of numbers and statistics. Subjective -- individual ’ s interpretation of events is important, e.g., uses participant observation, in- depth interviews, etc. Objective -- seeks precise measurement & analysis of target concepts, e.g., uses surveys, questionnaires, etc.

7. 7 Qualitative vs. Quantitative Research QualitativeQuantitative Qualitative data is more 'rich', time consuming, and less able to be generalized. Quantitative data is more efficient, able to test hypotheses, but may miss contextual details. Researcher tends to become subjectively immersed in the subject matter. Researcher tends to remain objectively separated from the subject matter.

8. 8 Literature Review and Research Problems Section II

9. 9 Literature Review and Research Problems I. Why Do a Literature Review? A. for planning primary research: 1. find a lack of consistency in reported results 2. find a flaw in research methods 3. conduct on a different population 4. find a solution for a problem 5. uncertainty about the interpretation

10. 10 B. As an end in itself: 1. inform practice (e.g. solve a problem) 2. provide understanding about the topic (longer process for integrative research review) Literature Review and Research Problems

11. 11 II. What Is the Search Process? A. Nine steps: 1. Identify a research topic (cause → effect) 2. Review secondary sources to get an overview (review article) 3. Develop a search strategy (collect databases, titles, abstract, full texts) 4. Conduct the search (search the sources) Literature Review and Research Problems

12. 12 5. Get a copy of the article 6. Read and prepare bibliographic information and notes 7. Evaluate the research reports (identify major points) 8. Analyze the research findings and synthesize the results (narrative and statistical methods) 9. Use the literature review (based on major points) Literature Review and Research Problems

13. 13 III. Research Questions and Hypotheses: A. four categories of research questions: 1. descriptive 2. normative 3. correlative 4. impact Literature Review and Research Problems

14. 14 1. Descriptive questions: Provide information about what is or has been happening related to the research topic (e.g., What is the major difficulty for EFL writers?) 2. Normative questions: Provide information compared with some standard or expected observation (e.g., Will EFL teachers achieve a better quality if they get a teaching certificate?) Literature Review and Research Problems

15. 15 3. Correlative questions: Identify relationship (e.g., What is the relationship between the year of training in writing and ss’ writing performance?) 4. Impact questions: Identify effect (e.g., What is the effect of teachers’ positive comments on ss’ writing performance?) Literature Review and Research Problems

16. 16 4-1. Two types of hypotheses: → directional hypothesis: can expect the outcome (from literature review) e.g., “If ss can get teachers’ positive comments, (then) they will have a better writing performance” → null hypothesis: can not expect the outcome e.g., “There is no difference between teachers’ positive comments and ss’ writing performance” Literature Review and Research Problems

17. 17 IV. In Summary, the Purposes of Literature Review: Provide previous research results (big picture) Provide a need for additional research (problem) Develop a conceptual framework (from major points to generate research questions and hypotheses) Guide the research design and conduct the study Literature Review and Research Problems

18. 18 Survey Research Section III

19. 19 Survey Research I. Main Purpose of Survey (Quantitative) Research: Collect data from a larger number of people Rely on individuals’ self-reports of their knowledge, attitudes, or behaviors II. Design Phase: Write the purpose of the survey (like a thesis statement) Write research questions based on the purpose (thesis)

20. 20 Survey Research III. Three Design Considerations: Descriptive approach Cross-sectional approach Longitudinal approach

21. 21 Survey Research Descriptive approach: A survey describing the characteristics of a sample at one point in time (e.g., senior students’ research paper writing difficulties in the Department of Applied English at I-Shou University in 2009)

22. 22 Survey Research Cross-sectional approach: A survey examining several groups at one point in time (e.g., different grade level students’ writing difficulties in the AE Department at ISU in 2009)  Pro: Compare responses across different grade levels in a shorter time  Con: Not easy to make a comparison due to different subjects

23. 23 Survey Research Longitudinal approach: A survey examining one group at different points in time (e.g., 1-year, 2- year-, and 3-year graduates’ perceptions of RP course in the AE Department at ISU in 2006-2008)  Pro: Follow the same subjects over a period of time  Con: Take a longer time

24. 24 Survey Research IV. Data Collection Choices: Mail, telephone, personal interviews, email, web-based surveys, etc. Mail:  Pro: Collect closed-ended information; low cost; enough time for the respondents  Con: Lower response rates; no more in-depth information

25. 25 Survey Research Phone interviews:  Pro: Collect open-ended information; higher response rate; more additional information  Con: More costly; can’t observe participants’ body language and facial expressions

26. 26 Survey Research Personal interviews:  Pro: Less structured approach; more conversational style; easy to get additional information; highest response rate  Con: Take more time

27. 27 Survey Research V. Sampling Plan: Identification of participants: Identify the participants who have the information you want or who have experienced the event Population definition: More specific about the participants’ sources

28. 28 Survey Research Two Sampling decisions: 1. Probability sampling: Need to specify to whom the results will be generalized 1-1. Sampling frame: to give a list of people who have a chance to be selected 1-2. Only choose the population included in the sampling frame 2. Purposeful sampling: Need to provide sufficient details about the important characteristics of the participants

29. 29 Survey Research Three sources of sampling errors: 1.Coverage errors: (a) should be in the sampling frame, but not there, (b) should not be in the frame, but in there 2. Nonresponse errors: (a) refuse to be interviewed, (b) can’t complete questionnaire, (c) can’t be reached 3. Sampling errors: Use different (wrong) sampling strategies drawn from a population

30. 30 Survey Research VI. Designing the Questionnaire: A. Review the literature: Borrow what has been designed before (with appropriate citations and permissions) B. Develop your own questionnaire: 1. follow Delphi Technique: Ask experts to provide ideas for better questionnaire, and then do a pilot testing with a larger sample

31. 31 Survey Research 2. Explain to yourself why asking each question 3. Decide format: Open (answer questions in own words) or closed format (multiple-choice, true-false, checklist) 4. Do a pilot study to include all alternative responses 5. Avoid psychologically threatening (socially desirable) questions 6. Clear items: Provide a definition of the terms first 7. Short items

32. 32 Survey Research 8. Avoid negative wording questions (e.g., Which of these are not ……?) 9. Avoid items that ask more than one idea 10.Use appropriate word level 11.Avoid leading questions 12.Emphasize critical words by using italics or underlining or bold letters

33. 33 Survey Research VII. Formatting the Questionnaire: 1.Make it attractive 2.Organize and lay out the questions 3.Number the items and pages 4.Put the returned address and name at the beginning and end of the questionnaire 5.Include brief, clear instructions

34. 34 Survey Research 6. Use examples if necessary 7. Organize the questions in a logical sequence (from general to specific) 8. Begin with a few interesting and nonthreatening items 9. Do not put the most important items at the end 10.Avoid using questionnaire or checklist on the form (use the title “Response Form”)

35. 35 Survey Research VIII. Pilot Testing the Questionnaire: 1.Select a sample similar to your population 2.Give more spaces for writing comments 3.Encourage pilot participants to give suggestions 4.Follow the procedures that you plan to use in your study 5.Add, change, or delete any questions if necessary

36. 36 IX. Special Types of Questions: 1. Demographic questions: 1-1. Background information (e.g., gender, age, etc.) 2. Nonthreatening behavioral questions: 2-1. Ask general questions on usual behavior first 2-2. Bounded recall: “e.g., In the last year……..” 2-3. Wording of question: “e.g., Are your difficulties in writing a research paper mainly due to your grammar problem?” Survey Research

37. 37 3. Threatening behavioral questions: Any questions elicit a defensive reaction in the participant; better to use : 3-1. open-ended questions 3-2. longer questions with explanations 3-3. familiar words 3-4. put answers into sealed envelopes 3-5. use appropriate wording (e.g., “Did you ever happen to ….”, “Such behaviors occur with different frequencies….”) 3-6. use multiple measures until getting the truth Survey Research

38. 38 4. Knowledge questions: Ask questions to test participants’ knowledge of a subject in school; better to use: 4-1. knowledge questions before asking attitude questions 4-2. appropriate level of difficulty 4-3. avoid asking “Do you happen to know …..?” 4-4. avoid using mail survey (may check up the answer) Survey Research

39. 39 Survey Research 5. Attitude questions: Should give a definition of the term or concept in advance; better to ask: 5-1. three kinds of attitudes: 5-1-1. affective (like vs. dislike): How does the person feel about this? 5-1-2. cognitive (knowledge): What does the person know about this? 5-1-3. action: What is the person willing to do about this?

40. 40 Survey Research 5-2. assess attitude strength (e.g., How much do you like or dislike ….?) 5-3. avoid asking more than one concept (e.g., Would you vote for me and peace or my opponent and war?) 5-4. use bipolar questions (e.g., Are you satisfied or dissatisfied with …….?) 5-5. move on to measure degree of positive or negative attitude (e.g., Are you strongly satisfied or moderately satisfied with …….?) 5-6. Use five points/alternatives on the rating scale

41. 41 Survey Research X. Letter of Transmittal: A cover letter to specify the purpose of the survey; hints to motivate the returned questionnaires 1.Appeal to authority: Need the well-known person’s signature 2.Appeal to self-interest: “You are one of the few people with the intelligence to be able to help us with this issue.” 3.Appeal to professional interests: “This is a very important question in our field (society).”

42. 42 Survey Research 4. Appeal to altruism: “The results of this survey will be used to solve one of our students’ writing problems.” 5. Appeal to curiosity: Offer to send a copy of the results 6. Appeal to greed: Offer to send a monetary incentive (a small gift) 7. Appeal to a sense of connection: Enclose a tea bag or a pencil (more details will be shown in the consent form)

43. 43 Survey Research XI. Conducting the Survey: 1.Send out an advance (cover) letter 2.Enclose the questionnaire with the transmittal letter 3.Supervise the data collection 4.Send a follow-up to nonrespondents 4-1. send again 4-2. send a postcard reminder 4-3. make a call 5. Control processing errors (eg., response rate) 6. Enter the data into the database 7. Clean up the data before you begin analysis

44. 44 Survey Research XII. Notes Specific to Personal Interviews: 1.Hold an introductory meeting to share the purpose, and get assurance to participate 2.Schedule the interview time 3.Make a flexible interview 4.Don’t use yes or no questions; plan to ask open-ended questions (e.g., How do you feel about …..? What is you opinion about …..? What do you think about ….?) 5.Conclude with open-ended questions: (e.g., Is there anything that I didn’t ask about…..? Is there anything that you wanted to tell me….?) 6.Pretest your interview procedures

45. 45 Survey Research XIII. Starting and Conducting the Interview: 1.Briefly review the purpose of the interview and information needed 2.Focus on what the participant is saying 3.Sequence the questions from general to specific 4.Ask for specific examples 5.Ask a variety of different kinds of questions 6.Avoid asking “why” questions; better to use “how come” 7.Use role play or simulation questions (If I were your …..) 8.Record the interview and take notes

46. 46 Qualitative Method Section IV

47. 47 Qualitative Method I. Main Purpose of Qualitative Research: Provide in-depth descriptions to interpret something by using case study, personal experiences, interview, observation Complexity, contextual, exploration, discovery, inductive logic

48. 48 II. Strategies for Qualitative Research: 1. Ethnographic research: Describe and analyze practices and beliefs of cultures and communities by interacting with people in their everyday lives 2. Case study: Detailed study of one individual (of a group) through observation Qualitative Method

49. 49 Qualitative Method 3. Phenomenological research: Seek the individual’s perceptions and meaning of a phenomenon or experience; “what is the meaning of the experience of this phenomenon for this person or group of people?”

50. 50 Qualitative Method 4. Grounded theory: Develop a theory after gathering and analyzing the data 5. Participatory research: Co-research the issue cooperatively 6. Clinical research: Understand a problem within a clinical context 7. Focus groups: Observe the interaction within the group (e.g., group interviews)

51. 51 III. General Methodological Guidelines: “Interview” A. The researcher is the instrument B. Speaking for the other C. Focus on an area of inquiry D. Explore research sites E. Gaining permission F. Negotiating entry G. Entering the field H. Role of the researcher Qualitative Method

52. 52 Qualitative Method A. The researcher is the “instrument”: 1. what questions to ask 2. in what order 3. what to observe 4. what to write down 5. key person to collect and interpret data

53. 53 Qualitative Method B. Speaking for the other: Retell someone’s story or experiences in a new way (re-writing) C. Focus on an area of inquiry: Explore the topic based on literature review and research questions (tentative) D. Explore research sites: Collect information from the place (location) you do the study by piloting, sampling, etc.

54. 54 Qualitative Method E. Gaining permission: Try to get permission from the “gatekeepers” (those with power and authority) F. Negotiating entry: How to get the true information with protection of human beings and their organization (eg., consent form) G. Entering the field: Make plans to enter the field in the least disruptive manner

55. 55 Qualitative Method H. Role of the researcher: Unbiased, objective, discover multiple views of the participants; more interactive 1. Supervisor (with authority) 2. Leader (e.g., scout leaders) 3. Friend (with respect)

56. 56 Qualitative Method IV. Critically Analyzing Qualitative Research: A. Credibility B. Transferability C. Dependability D. Confirmability E. Authenticity F. Emancipatory

57. 57 Qualitative Method A. Credibility: Internal validity (a correspondence between how the researcher observes and participants express viewpoints) 1. Prolonged and substantial engagement: Give enough time staying at the site 2. Persistent observation: Sufficient observation until identifying clear issues/picture (eg., observe facial expressions, pauses) 3. Peer debreifing: Discuss with the disinterested peers for better research ideas

58. 58 Qualitative Method 4. Negative case analysis: If the finding doesn’t fit the hypothesis, seek more information and analyze the negative case 5. Progressive subjectivity: Be open-minded and keep changing if necessary 6. Member checks: Share research results with peers for comment 7. Triangulation: Check information from different sources or methods to find consistency (e.g., quantitative data = qualitative findings)

59. 59 Qualitative Method B. Transferability: Thick description (provide sufficient details such as time, place, context) and multiple cases for external validity (generalize the results to other similar situation) C. Dependability: Reliability (maintain the same result without change no matter tested how many times) D. Confirmability: Objectivity (provide evidences by synthesizing data instead of researcher’s imagination)

60. 60 Qualitative Method E. Authenticity: Present a balanced view of all values and beliefs Fairness: Display different views (pros and cons) Ontological authenticity: Show individual’s real experiences Catalytic authenticity: Get evidences of participants’ views by the “inquiry” process

61. 61 Qualitative Method F. Transformative Paradigm Criteria: Positionality or standpoint epistemology: The research result can not contain all universal truth because a lot of factors may influence the result; should avoid showing the position or standpoint of the author Community: Know the research site well to link the research results to positive action within that community

62. 62 Qualitative Method Attention to voice: Speak for the minority Critical reflexivity: Be sensitive to understand the psychological state of participants Reciprocity: Develop trust with participants Sharing the perquisites of privilege: Share research results with participants

63. 63 Mixed-Methods and Mixed-Model Designs Section V

64. 64 I.Definition and Characteristics: A. Use both qualitative and quantitative methods in the design, data collection, and analysis B. Quantitative analysis first, and then followed by content analysis of qualitative data Mixed-Methods and Mixed-Model Designs

65. 65 II. Importance in Educational and Psychological Research: A. Obtain a more complete picture of human behavior and experience B. Provide broader and deeper understanding of the issue Mixed-Methods and Mixed-Model Designs

66. 66 III. Two paradigms for deeper understanding: A. Pragmatic paradigm: 1. Researcher determines which method to use to answer a specific research question (i.e., find a problem/issue and use a method for solutions) 2. Conduct the study and anticipate the results based on researcher’s own values and explanations (the result may match their value system) Mixed-Methods and Mixed-Model Designs

67. 67 Mixed-Methods and Mixed-Model Designs B. Transformative paradigm: Emphasize minority group’s values and viewpoints for “social change” (“Transformation” means “change”)

68. 68 Mixed-Methods and Mixed-Model Designs IV. Mixed-Methods/Models Design: Apply the quantitative and qualitative methods simultaneously (parallel) or sequentially; 4 design options: A. Pragmatic parallel mixed-methods B. Pragmatic sequential mixed-methods C. Transformative parallel mixed-methods D. Transformative sequential mixed-methods

69. 69 Mixed-Methods and Mixed-Model Designs A. Pragmatic parallel mixed-methods: 1. Collect qualitative and quantitative data to answer research questions 2. Collect two types of data simultaneously or with small time lag (almost simultaneously) e.g., examine 16 cases by doing a quantitative questionnaire study and qualitative open-ended interviews with two independent phases (no data analysis)

70. 70 Mixed-Methods and Mixed-Model Designs B. Pragmatic sequential mixed-methods: 1. Collect and analyze one type of data (e.g. quantitative) as a basis for collecting another type of data (e.g., qualitative) 2. The second phase of research is to confirm or disconfirm the first phase e.g., questionnaires first; use the results to develop qualitative interviews

71. 71 Mixed-Methods and Mixed-Model Designs C. Transformative parallel mixed-methods: 1. Use qualitative and quantitative methods to promote social or political changes 2. Explore the minority’s populations 3. Emphasize value-based and action-oriented purposes (make changes for improvement)

72. 72 Mixed-Methods and Mixed-Model Designs D. Transformative sequential mixed-methods: 1. Focus on the minority’s populations and values 2. One type of data provides a basis for collection of another type of data

73. 73 Mixed-Methods and Mixed-Model Designs V. Questions for Critically Analyzing Mixed-Methods Research: A. Consider inference quality = internal validity (quantitative) = trustworthiness (qualitative) = consistency between research purpose, research questions, and methodology used

74. 74 Mixed-Methods and Mixed-Model Designs B. Conflicts between qualitative (small sample) and quantitative (large sample), so need to link both data C. Need to consider a lot of factors e.g., language, gender, reading or writing levels (demographic information)

75. 75 Mixed-Methods and Mixed-Model Designs V. Questions for Critically Analyzing Mixed-Methods Research: 1.What are your research purpose and questions? 2.Have you matched the purpose and questions to appropriate methods? 3.To what extent do you reach the criteria of the quality for the quantitative part of the study? (i.e., internal validity, external validity, reliability, objectivity)

76. 76 4. To what extent do you reach the criteria of the quality for the qualitative part of the study? (i.e., credibility, transferability, dependability, confirmability, authenticity, transformative) 5. How do you solve the conflict between the design and implementation of the study? 6. What is the limitations for generalization? Mixed-Methods and Mixed-Model Designs

77. 77 7. How do you integrate the results from the mixed methods? How do you explain the conflicting findings if necessary? 8. How do you control and explain the factors of gender, language levels, etc? Mixed-Methods and Mixed-Model Designs

78. 78 Midterm Exam 1.What is the major difference between quantitative and qualitative research methods? 2.What are the four categories of research questions? 3.What are the two types of hypotheses? 4.What are the three survey design approaches? 5.What are the two sampling decisions?

79. 79 6. What is grounded theory? 7. What is pilot testing? 8. What are the four options in the mixed-method design? Midterm Exam

80. 80 Midterm Exam (answer key) 1.Quantitative research: Use survey, questionnaires to collect numbers; need large samples; more objective Qualitative research: Use observation, interviews for a complete and detailed description 2. descriptive, normative, correlative, impact 3. directional and null hypotheses 4. descriptive, cross-sectional, longitudinal approach

81. 81 Midterm Exam (answer key) 5. probability and nonprobability sampling 6. develop a theory after gathering and analyzing the data 7. select a sample similar to your population; give more spaces for comments; add, change, or delete any questions if necessary 8. pragmatic parallel, pragmatic sequential, transformative parallel, transformative sequential mixed-methods

82. 82 Sampling Section VI

83. 83 Sampling I.Definition: Select a given number of people from a population A. Probability sampling: Every member of the population has a possibility of being selected B. Nonprobability (purposeful) sampling: Select members who had the particular experience before; try to look for negative cases for better improvement

84. 84 Sampling II. External Validity: Generalizability or Transferability A. Generalize findings to the larger (target) population, so need to provide detailed description of the case III. Sampling Strategies: A. Probability sampling B. Nonprobability (Purposeful) sampling C. Convenience sampling

85. 85 Sampling A.Probability sampling: 1. Simple random sampling 2. Systematic sampling 3. Stratified sampling 4. Cluster sampling 5. Multistage sampling

86. 86 Sampling 1. Simple random sampling: Each member has a number (code) and each population has an equal chance to be selected →Pro: simple process →Con: difficult to get a complete list of all populations; may include some “outsiders”

87. 87 Sampling 2. Systematic sampling: Select every nth name from the list; so need to estimate the needed sample size →Pro: not every member needs to be numbered →Con: bias if members are arranged in a specific pattern (e.g., choose the last name with A in the given year)

88. 88 Sampling 3. Stratified sampling: Divide the populations into subgroups (e.g., genders) or levels (e.g., proficiency levels) and then draw randomly from each subgroup →Pro: easy to compare each subgroup results →Con: must get information (eg., demographic information, pretest scores) before dividing them

89. 89 Sampling 4. Cluster sampling: Choose groups of individuals e.g., city blocks or classrooms in a school, and study all of the samples there →Pro: save time and money by collecting data at a limited number of sites →Con: small sample size, so less precision in estimating the effect (hard to reach generalization)

90. 90 Sampling 5. Multistage sampling: Combine sampling strategies e.g., use cluster sampling to randomly select classrooms and use random sampling to select a sample with each classroom →Pro: more reliable →Con: complex calculations (regression analysis)

91. 91 Sampling B. Nonprobability (Purposeful) sampling: Provide in-depth, interpretive information 1. Extreme or deviant cases 2. Intensity sampling 3. Maximum-variation sampling 4. Homogeneous sampling 5. Typical-case sampling 6. Stratified purposeful sampling

92. 92 Sampling 7. Critical-case sampling 8. Snowball or chain sampling 9. Criterion sampling 10. Theory-based or operational construct sampling 11. Confirming and disconfirming cases 12. Opportunistic sampling 13. Purposeful random sampling 14. Sampling politically important cases

93. 93 Sampling 1. Extreme or deviant cases: Study the extreme (unusual) case to illuminate the ordinary e.g., analyze an extremely best writer’s work and compare it with the poorest one 2. Intensity sampling: Similar to extreme cases, but less emphasis on extreme; explore rich information on typical cases; analyze why individuals meet the specified criterion

94. 94 Sampling 3. Maximum-variation sampling: Maximize the variation within the samples and then indicate their unique (major) difference e.g., study students’ English abilities among different locations (rural, urban, and suburban areas) 4. Homogeneous sampling: Identify the samples who share similar characteristics e.g., rural teachers’ attitudes toward bilingual learning for kindergarten kids

95. 95 5. Typical-case sampling: Choose a case in which a program/instruction has been implemented to show this case is indeed average 6. Stratified purposeful sampling: Divide samples into subgroups and then select cases within each subgroup 7. Critical-case sampling: Study a very important, critical case and the effect should be representative: “if it happens there, it will happen anywhere.” Sampling

96. 96 8. Snowball or chain sampling: Start with a key person and introduces the next one to become a chain 9. Criterion sampling: Set up a criterion and identify cases that meet that criterion e. g., study cases that could pass TOEFL last semester 10. Theory-based or operational construct sampling: Define a theoretical construct (e.g., metacognitive learning on EFL reading development) Sampling

97. 97 Sampling 11. Confirming and disconfirming cases: Form grounded theory by analyzing sample cases that fit (confirming) and do not fit (disconfirming) the major points in literature 12. Opportunistic sampling: Researcher should take the opportunity to decide the sampling procedure or samples during the study

98. 98 Sampling 13. Purposeful random sampling: Randomly select participants who had similar experiences in a very small sample 14. Sampling politically important cases: Use particular samples that someone expects to use the results

99. 99 Sampling Overall strengths and weaknesses of purposeful sampling: Pro: 1. Less costly and time consuming 2. Ease of administration 3. Assures high participation rate 4. Generalization possible to similar subjects 5. Assures receipt of needed information Con:1. Difficult to generalize to other subjects 2. Less representative of an identified population 3. Greater subject bias

100. 100 Sampling C. Convenience sampling: Choose the samples who are very available (easy to access such as neighbor, classmate), but can not generalize the results beyond the population poor Pro: 1. Less costly and time consuming 2. Ease of administration 3. Assures high participation rate 4. Generalization possible to similar subjects Con:1. Difficult to generalize to other subjects 2. Less representative of an identified population 3. Greater subject bias

101. 101 Sampling IV. Access Issues: Consider how to obtain your participants A. Get permission and agreement with the appropriate person (e.g., school principal, classroom teacher, or parents) B. Obtain consent form from the participants C. How to “label” students if using stratified sampling D. Consider appropriate sample size

102. 102 Sampling V. Sample size: A. Quantitative research rules: For survey research, 100 cases B. Qualitative research rules: For grounded theory, 30-50 interviews

103. 103 Sampling VI. Consent form: A. Explain research purpose, duration, and procedures B. Describe any risk or discomfort C. Describe confidentiality or anonymity D. Provide the name of person to contact with E. Provide voluntary participation, and available to refuse or withdraw any time

104. 104 Dear Principle of I-Shou International School, My name is Chiau-jen Zhu, and I’m a junior student of the Applied English Department at I-Shou University. In order to accomplish my required research, “The Effect of Reading Stories to Improve Elementary School Students’ Reading Ability”, I need your permission to interview students at your school. Reading stories has become a tendency to improve elementary school students’ reading ability. Therefore, my purpose is to interview students to get more ideas about the effect of reading literature. The interview includes three parts: past experience, the details of experience and personal opinions of participants. The participation is not compulsory, but I will appreciate your contribution if you agree with the interviews. Each interview will take 15 to 25 minutes, and five students are needed from the fifth to the sixth grade. In addition, the names of participants will be innominate for sure. As for the contents of interviews, the response will be summarized, and the raw data will not be presented in the final report. Besides, the participants have the rights to read and withdraw any time. The results of this study will be provided to my teachers and supervisors only, which will be very useful for their teaching. Thank you for your assistance in this study. Sincerely yours, Chiau-jen Zhu Junior, Applied English Department I-Shou University E-mail: gekkoh-159@yahoo.com.tw

105. 105

106. 106 Data Collection Section VII

107. 107 Data Collection I.Why collect data? To know what certain people are thinking or doing by asking them questions II.Quantitative data collection: Use questionnaires and show “numbers” to inform the user the “information” A. Primary data: To serve our specific purpose (researcher collects the data himself/herself) B. Secondary data: Collected by others (literature review)

108. 108 III. Asking questions: Get information (data) by filling in the questionnaire as the basis of data collection for email, postal and telephone surveys and structured and semi-structured interviews A. Structured interview = closed questions = answer predetermined items B. Semi-structured interview = open-ended questions = answer freely in own words (e.g., What were the main difficulties for RP writing?) Data Collection

109. 109 Data Collection Type of Questions OpenClosed ClassificationalListsRankingScale

110. 110 1. Classificational questions: To classify respondents by gender, age, occupation, etc. (personal information) e.g., Age last birthday? □ Under 18 □ 18-25 □ 26-30 □ 31-40 □ 41-50 Data Collection

111. 111 Data Collection 2. List questions: To ask respondents to select one or more from the list of answers e.g., Specify the length of learning English. □ less than 3 years □ 4-6 years □ 7-9 years □ 10-12 years □ more than 12 years

112. 112 Data Collection 3. Ranking questions: To give an order or preference e.g., Rank the following factors in order of importance if you want to be a good English writer. (from 5 the most important to 1 the least important) □ grammar □ vocabulary □ background knowledge □ reading ability □ Chinese language ability

113. 113 Data Collection 4. Scale (rating) questions: To determine the strength of views or opinions e.g., How likely do you like (agree) grammar instruction? Very likely (Agree strongly) 5 Quite likely (Agree slightly) 4 Neither likely (Agree) nor unlikely (Disagree) 3 Quite unlikely (Disagree slightly) 2 Very unlikely (Disagree strongly) 1

114. 114 C. Framing questions: 1. Avoid bias and leading questions (e.g., Would you agree that ……?) 2. Avoid shorthand: e.g., Would an application of CALL …….? 3. Avoid ambiguous words: e.g., “usually, frequently, or recently” 4. Avoid long questions: Short and simple questions are more effective. Data Collection

115. 115 Data Collection 5. Avoid negative questions: e.g., Do you never ……..? 6. Avoid hypothetical questions: e.g., If you were a millionaire, would you …….? 7. Avoid irrelevant questions 8. Make respondents to feel easy to answer questions: e.g., What did you do last week? (too broad) Have you done any of the following things last week? (easier to answer)

116. 116 Data Collection 9. Avoid yes-no questions 10. Provide “classification” type data (e.g., age, sex, previous training in writing, etc.) for demographic information 11. Add comments (e.g., In your opinion, what ….?) for in-depth information

117. 117 IV. Qualitative Data Collection: Collect data by interviewing and conducted individually or in a group 1. Focus group interviews: Fewer than 10 questions 2. Open-ended questions: Allow more self-reflections 3. Avoid using “why” questions: Use “how come” or “what” 4. Carefully develop the questions by doing pilot testing 5. Provide enough information for understanding 6. Arrange questions from general to specific Data Collection

118. 118 Data Collection 7. Listen more, talk less, and ask “real questions” (the interviewer doesn’t already know or anticipate the response) 8. Try to get “inner voice” instead of “superficial” information 9. Take notes and bring tape-recording: Easy to transcribe 10. Ask questions when you do not understand: Ask for examples or concrete details (past, present, & opinions)

119. 119 Data Collection 11. Explore laughter, pauses, facial expressions 12. Ask participants to tell a story: To get a complete picture (transcription model 1 and model 2)model 1 model 2 13. Follow your instincts: Try to feel participants’ feelings

120. 120 Data Collection V. Benefits of using a questionnaire in interviewing: A. To ensure covering all of the questions B. Can explore more in-depth information within each of the questions C. Collect both quantitative and qualitative data altogether

121. 121 Data Analysis, Interpretation, and Reporting Section VIII

122. 122 Data Analysis, Interpretation, and Reporting I. Qualitative analytic strategies: A. Recursive process: → analyze cases→ generate findings→ draw conclusion → form grounded theory → write report B. Nine qualitative data analysis principles: 1. Collect the data at the site and carefully study all the data to seek similarities and differences, concepts and reflections

123. 123 Data Analysis, Interpretation, and Reporting 2. Saturation and sufficiency of information: Never stop data analysis until the emergence of regularities; i.e., no new information emerges with additional data analysis 3. Accountability of information: Keep notes or transcripts if outsiders want to review the data analysis procedures and results 4. Divide the data (excerpts) into smaller units related to your major points

124. 124 Data Analysis, Interpretation, and Reporting 5. Organize the smaller units into categories (based on major points) 6. Build conceptual similarities, find negative evidences, and discover patterns 7. Modify categories as further patterns occur 8. Analyze negative cases (minority’s opinions) to reflect their perceptions 9. Synthesize the patterns into the grounded theory

125. 125

126. 126 Data Analysis, Interpretation, and Reporting * SHOULD BE: - connected with what is being discussed in the major points - exact wording (excerpts) used in the statement * SHOULD NOT BE: - based on interviewer’s personal opinions - irrelevant to the major points

127. 127 Data Analysis, Interpretation, and Reporting C. Six Steps in Qualitative Data Analysis: 1.Give codes from the notes (transcripts) 2.Note personal reflections or other minority’s comments 3.Sort the notes to identify similar and different relationships between patterns 4.Identify these patterns, similarities, and differences 5.Elaborate a small set of generalizations that cover the consistencies 6.Examine those generalizations and form grounded theory (see “Content Analysis”)Content Analysis

128. 128 → a process of constructing various data → inductive process by collecting, analyzing, and comparing data systematically → theory is grounded on data to explain the phenomena Grounded Theory ( 紮根理論 ): Data Analysis, Interpretation, and Reporting

129. 129 Grounded Theory (by Marlene Pomrenke) Grounded theory is a method of social inquiry associated with a qualitative approach to research. This inductive research process utilizes generalized knowledge that is derived from specific observations of phenomena from the field. In turn, this can be used to build theory. For example, grounded theorists aim to create theoretical categories from collected data and then analyze relationships between key categories (Charmaz, 1990). Indeed, the main purpose of using a grounded theory approach is to develop theory through understanding concepts that are related by means of statements of relationships (Strauss & Corbin, 1990). Using the concepts from grounded theory, this study starts from understanding the experience of the research participants (i.e., how they construct their worlds). The data analysis stage focused on finding recurrent themes or issues in the data, and finally into developing or refining a theory about the phenomenon.

130. 130 Data Analysis, Interpretation, and Reporting D. Grounded Theory Analysis Strategies: 1. Recur by moving back and forth with the data, analyzing, collecting more data, and analyzing some more until reaching conclusions 2. An interactional method of theory building by comparing and analyzing the data

131. 131 Data Analysis, Interpretation, and Reporting 3. Three steps in the grounded theory analytic (coding) process: (1) Open coding: Break data into small parts → compare for similarities and differences → explain the meanings of the data by focusing on “who, when, where, what, how much, why” (ask questions to get a clear story) (2) Axial coding: After open coding, make connections (sort) between categories and confirm or disconfirm your hypotheses (3) Selective coding: Select the core category (match hypotheses) and explain the minor category (against hypotheses) with additional supporting data

132. 132 Coding Process Open Coding MP 1 MP 2 MP 3 Axial Coding Selective Coding MP 1 MP 3 (code) (category) (pattern)

133. 133 Data Analysis, Interpretation, and Reporting II. Interpretation Issues in Qualitative Data analysis: A. Triangulating data: Use multiple methods and data sources to support the strength of interpretations and conclusion (e.g., semi- structured interviews, consent form, grounded theory, etc.)

134. 134 Data Analysis, Interpretation, and Reporting B. Audits: Questions to examine the data for interpretations and conclusion 1. Is sampling appropriate to ground the findings? 2. Are coding strategies applied correctly? 3. Is the category process appropriate? 4. Do the results link hypotheses? 5. Are the negative cases explained?

135. 135 * Four steps of negative case testing: 1. Make a rough hypothesis 2. Conduct a thorough search 3. Discard or reformulate hypothesis 4. Examine all relevant cases Data Analysis, Interpretation, and Reporting

136. 136 Data Analysis, Interpretation, and Reporting C. Cultural bias: Discuss cultural differences with different groups of participants (compare the differences between western and Taiwanese students’ attitudes) D. Generalization: Not appropriate for qualitative research 1. Case-to-case transferability by providing thick description to apply to another setting 2. Generalize the result to a broader theory (e.g., use deviant cases)

137. 137 Data Analysis, Interpretation, and Reporting III. Writing Research Reports: A. Introduction B. Literature Review C. Methodology D. Results: Tie the results to study purpose (hypotheses) E. Discussions and Conclusion: Tie discussions to the literature; recommendations for practice; limitations of the study

138. 138 Data Analysis, Interpretation, and Reporting Quantitative reports: Report results by the use of tables and graphs Avoid first-person pronoun Use passive voice Qualitative reports: Look for a deep description (narrative style) Look for well-grounded theory Seek contextual meaning by understanding demographic information (different experiences)

139. 139 Writing the Research Paper Section IX

140. 140 Writing the Research Paper Chapter I: Introduction (Problem Statement) A. Area of Study: Provide a general introduction to the topic; outline the problem B. Definition of Terms: Define important terms and concepts C. The purpose of the study D. Assumptions: Make hypotheses for the proposed study

141. 141 Chapter 2: Literature Review A. History: Provide a review of the historical background B. Current Literature: Include a review of current relevant literature with the analysis of pros and cons C. Research Problem: Provide research questions Writing the Research Paper

142. 142 Writing the Research Paper Chapter 3: Methodology A. Research Questions and Hypotheses: Propose themResearch Questions and Hypotheses based on literature review (major points) B. Research Design:Research Design * Quantitative research: Use a survey (three designsurvey considerations; discuss how to design questionnaire) * Qualitative research: Use interviews (present ainterviews rationale for the design of choice)

143. 143 Writing the Research Paper C. Sample: * Quantitative research: Describe samplingsampling techniques and rationale for the method used for selecting the sample and sample size * Qualitative research: Describe the criteria for the selection of the participants and setting

144. 144 Writing the Research Paper D. Data Collection Procedures:Data Collection Procedures * Quantitative research: Describe the procedures that the data will be collected by using a survey * Qualitative research: Describe the use of audiotapes, note-taking E. Pilot Testing: Describe the pilot study procedures and any adjustments

145. 145 Writing the Research Paper F. Data Analysis Procedures:Data Analysis Procedures * Quantitative research: Describe how you handled the data; provide statistical procedures (e.g., compute mean score or frequency) * Qualitative research: Describe data analysis strategies; explain multiple sources of data G. Limitations of the Study: Explain anticipated limitations of the study

146. 146 Writing the Research Paper Chapter 4: Results: Just report what the data say (you may give the subheading based on each hypothesis) A. Findings from the quantitative results (% or mean) B. Findings from the qualitative results (3 excerpts) C. Fit into your hypotheses or not (e.g., broadly/partially/do not fit)

147. 147 Writing the Research Paper Chapter 5: Discussions and Conclusion A. Interpret the meanings of data (discuss “why” your participants would think/say so) B. Discuss whether the findings fit into the literature or not C. Form any grounded theory? D. The results will be useful to “whom”? E. Any limitation to reach the generalizability F. Draw conclusions and suggestions (about improvement) based on your or minority’s opinions

148. 148 Writing the Research Paper Outline of Research Paper

149. 149 Final Exam 1. What is simple random sampling? Each member has a number (code) and each population has an equal chance to be selected.

150. 150 Final Exam 2. What is systematic sampling? Select every nth name from the list; so need to estimate the needed sample size

151. 151 Final Exam 3. What is stratified sampling? Divide the populations into subgroups (e.g., genders) or levels (e.g., proficiency levels) and then draw randomly from each subgroup

152. 152 Final Exam 4. What is cluster sampling? Choose groups of individuals e.g., city blocks or classrooms in a school, and study all of the samples there

153. 153 Final Exam 5. What is stratified purposeful sampling? Divide samples into subgroups and then select cases within each subgroup

154. 154 Final Exam 6. What is confirming and disconfirming cases? Form grounded theory by analyzing sample cases that fit (confirming) and do not fit (disconfirming) the major points in literature

155. 155 Final Exam 7. What is purposeful random sampling? Randomly select participants who had similar experiences in a very small sample

156. 156 Final Exam Type of Questions OpenClosed ClassificationalListsRankingScale 8. What are some types of quantitative questions?

157. 157 Final Exam → a process of constructing various data → inductive process by collecting, analyzing, and comparing data systematically → theory is grounded on data to explain the phenomena 9. What is grounded theory?

158. 158 Final Exam 10. What are the three steps in the grounded theory analytic (coding) process? (1) Open coding: Break data into small parts (2) Axial coding: After open coding, make connections (sort) between categories (3) Selective coding: Select the core category (match hypotheses) and explain the minor category (against hypotheses) with additional supporting data