1. 1 Research Methodology General Review

2. 2 Benefits of research to whom As a graduate student... As a graduate student... To be able to read and understand the empirical literature in your field; to become a critical consumer of information. To be able to read and understand the empirical literature in your field; to become a critical consumer of information. As a graduate student preparing for a thesis or dissertation… As a graduate student preparing for a thesis or dissertation… To be able to both design and implement your thesis or dissertation as well as future studies that interest you. To be able to both design and implement your thesis or dissertation as well as future studies that interest you.

3. 3 Benefits to whom As a future practitioner… As a future practitioner… To be able to intelligently participate in research projects, evaluations, and studies undertaken by your institution. To be able to intelligently participate in research projects, evaluations, and studies undertaken by your institution. As an educated citizen... As an educated citizen... To understand the difference between scientifically acquired knowledge and other kinds of information. To understand the difference between scientifically acquired knowledge and other kinds of information.

4. 4 What Research Is Not Research isn’t information gathering: Research isn’t information gathering: Gathering information from resources such as books or magazines isn’t research. Gathering information from resources such as books or magazines isn’t research. No contribution to new knowledge. No contribution to new knowledge. Research isn’t the transportation of facts: Research isn’t the transportation of facts: Merely transporting facts from one resource to another doesn’t constitute research. Merely transporting facts from one resource to another doesn’t constitute research. No contribution to new knowledge although this might make existing knowledge more accessible. No contribution to new knowledge although this might make existing knowledge more accessible.

5. 5 What Research Is Research is: Research is: “…the systematic process of collecting and analyzing information (data) in order to increase our understanding of the phenomenon about which we are concerned or interested.”( Leedy and Ormrod, 2001 )

6. 6 What’s the Difference Between “Method” and “Methodology”? Method: Techniques for gathering evidence Techniques for gathering evidence The various ways of proceeding in gathering information The various ways of proceeding in gathering information Methodology: The underlying theory and analysis of how research does or should proceed, often influenced by discipline

7. 7 Epistemology, Methodology, and Method “a research method is a technique for (or way of proceeding in) gathering evidence" while “a research method is a technique for (or way of proceeding in) gathering evidence" while "methodology is a theory and analysis of how research does or should proceed" and "methodology is a theory and analysis of how research does or should proceed" and "an epistemology is a theory of knowledge" "an epistemology is a theory of knowledge"

8. 8 "It is the theory that decides what can be observed." Albert Einstein Albert Einstein

9. 9 Research Characteristics 1. Originates with a question or problem. 2. Requires clear articulation of a goal. 3. Follows a specific plan or procedure. 4. Often divides main problem into subproblems. 5. Guided by specific problem, question, or hypothesis. 6. Accepts certain critical assumptions. 7. Requires collection and interpretation of data. 8. Cyclical (helical) in nature.

10. 10 Research Projects Research begins with a problem. Research begins with a problem. This problem need not be Earth-shaking. This problem need not be Earth-shaking. Identifying this problem can actually be the hardest part of research. Identifying this problem can actually be the hardest part of research. In general, good research projects should: In general, good research projects should: Address an important question. Address an important question. Advance knowledge. Advance knowledge.

11. 11 Research Project Pitfalls The following kinds of projects usually don’t make for good research: The following kinds of projects usually don’t make for good research: Self-enlightenment. Self-enlightenment. Comparing data sets. Comparing data sets. Correlating data sets. Correlating data sets. Problems with yes / no answers. Problems with yes / no answers.

12. 12 High-Quality Research (1 of 2) Good research requires: Good research requires: The scope and limitations of the work to be clearly defined. The scope and limitations of the work to be clearly defined. The process to be clearly explained so that it can be reproduced and verified by other researchers. The process to be clearly explained so that it can be reproduced and verified by other researchers. A thoroughly planned design that is as objective as possible. A thoroughly planned design that is as objective as possible.

13. 13 High-Quality Research (2 of 2) Good research requires: Good research requires: Highly ethical standards be applied. Highly ethical standards be applied. All limitations be documented. All limitations be documented. Data be adequately analyzed and explained. Data be adequately analyzed and explained. All findings be presented unambiguously and all conclusions be justified by sufficient evidence. All findings be presented unambiguously and all conclusions be justified by sufficient evidence.

14. 14 Sources of Research Problems Observation. Observation. Literature reviews. Literature reviews. Professional conferences. Professional conferences. Experts. Experts.

15. 15 Stating the Research Problem Once you’ve identified a research problem: Once you’ve identified a research problem: State that problem clearly and completely. State that problem clearly and completely. Determine the feasibility of the research. Determine the feasibility of the research. Identify subproblems: Identify subproblems: Completely researchable units. Completely researchable units. Small in number. Small in number. Add up to the total problem. Add up to the total problem. Must be clearly tied to the interpretation of the data. Must be clearly tied to the interpretation of the data.

16. 16 Research Acquisition of Knowledge Acquisition of KnowledgeKnowledgev/sInformation (Theoretically, concerned with developing, exploring, or testing theories)

17. 17 Theory What exists? What exists? Why exists? Why exists? What will happen in future? What will happen in future?

18. 18 How to acquire Knowledge? Inductive Reasoning Inductive Reasoning ( works moving from specific observation to broader generalisation, bottom approach) Deductive Reasoning Deductive Reasoning ( more general to more specific or top down approach)

19. Deductive Reasoning 19 Theory Hypothesis Observation Confirmation

20. 20 Inductive Reasoning Observation Pattern Hypothesis Theory

21. 21 Positivism Goal of Knowledge is to describe the phenomena that are experienced, Goal of Knowledge is to describe the phenomena that are experienced, There is interdependence of observation and theory, our observations are theory laden There is interdependence of observation and theory, our observations are theory laden

22. 22 Scientific Thought

23. 23 Hypotheses Hypotheses are tentative, intelligent guesses as to the solution of the problem. Hypotheses are tentative, intelligent guesses as to the solution of the problem. There is often a 1-1 correspondence between a subproblem and a hypothesis. There is often a 1-1 correspondence between a subproblem and a hypothesis. Hypotheses can direct later research activities since they can help determine the nature of the research and methods applied. Hypotheses can direct later research activities since they can help determine the nature of the research and methods applied.

24. 24 Delimitations All research has limitations and thus certain work that will not be performed. All research has limitations and thus certain work that will not be performed. The work that will not be undertaken is described as the delimitations of the research. The work that will not be undertaken is described as the delimitations of the research.

25. 25 Definitions Define each technical term as it is used in relation to your research project. Define each technical term as it is used in relation to your research project. This helps remove significant ambiguity from the research itself by ensuring that reviewers, while they may not agree with your definitions, at least know what you’re talking about. This helps remove significant ambiguity from the research itself by ensuring that reviewers, while they may not agree with your definitions, at least know what you’re talking about.

26. 26 Assumptions Assumptions are those things that the researcher is taking for granted. Assumptions are those things that the researcher is taking for granted. For example: a given test instrument accurately and consistently measures the phenomenon in question. For example: a given test instrument accurately and consistently measures the phenomenon in question. As a general rule you’re better off documenting an assumption than ignoring it. As a general rule you’re better off documenting an assumption than ignoring it. Overlooked assumptions provide a prime source of debate about a research project’s results. Overlooked assumptions provide a prime source of debate about a research project’s results.

27. 27 Importance of the Study Many research problems have a kind of theoretical feel about them. Such projects often need to be justified: Many research problems have a kind of theoretical feel about them. Such projects often need to be justified: What is the research project’s practical value? What is the research project’s practical value? Without this justification, it will prove difficult to convince others that the problem in question is worth study. Without this justification, it will prove difficult to convince others that the problem in question is worth study.

28. 28 Research Proposals Research proposals are documents that describe the intended research including: Research proposals are documents that describe the intended research including: Problem and subproblems. Problem and subproblems. Hypotheses. Hypotheses. Delimitations. Delimitations. Definitions. Definitions. Assumptions. Assumptions. Importance. Importance. Literature review. Literature review.

29. 29 Literature Review A literature review is a necessity. A literature review is a necessity. Without this step, you won’t know if your problem has been solved or what related research is already underway. Without this step, you won’t know if your problem has been solved or what related research is already underway. When performing the review: When performing the review: Start searching professional journals. Start searching professional journals. Begin with the most recent articles you can find. Begin with the most recent articles you can find. Keep track of relevant articles in a bibliography. Keep track of relevant articles in a bibliography. Don’t be discouraged if work on the topic is already underway. Don’t be discouraged if work on the topic is already underway.

30. 30 Literature Review Pitfalls (1 of 2) Be very careful to check your sources when doing your literature review. Be very careful to check your sources when doing your literature review. Many trade magazines are not peer reviewed. Many trade magazines are not peer reviewed. Professional conferences and journals often have each article reviewed by multiple people before it is even recommended for publication. Professional conferences and journals often have each article reviewed by multiple people before it is even recommended for publication. The Cornell University Library website is a very good place to start looking for legitimate research. The Cornell University Library website is a very good place to start looking for legitimate research.

31. 31 Literature Review Pitfalls (2 of 2) The Internet can be a good source of information. It is also full of pseudo-science and poor research. The Internet can be a good source of information. It is also full of pseudo-science and poor research. Make sure you verify the claims of any documentation that has not been peer reviewed by other professionals in the related fields. Make sure you verify the claims of any documentation that has not been peer reviewed by other professionals in the related fields.

32. 32 Processes & Methodologies Research Process. Research Process. Common Methodologies. Common Methodologies. Methodology Comparison. Methodology Comparison.

33. 33 Research Process Research is an extremely cyclic process. Research is an extremely cyclic process. Later stages might necessitate a review of earlier work. Later stages might necessitate a review of earlier work. This isn’t a weakness of the process but is part of the built-in error correction machinery. This isn’t a weakness of the process but is part of the built-in error correction machinery. Because of the cyclic nature of research, it can be difficult to determine where to start and when to stop. Because of the cyclic nature of research, it can be difficult to determine where to start and when to stop.

34. 34 Step 1: A Question Is Raised A question occurs to or is posed to the researcher for which that researcher has no answer. A question occurs to or is posed to the researcher for which that researcher has no answer. This doesn’t mean that someone else doesn’t already have an answer. This doesn’t mean that someone else doesn’t already have an answer. The question needs to be converted to an appropriate problem statement like that documented in a research proposal. The question needs to be converted to an appropriate problem statement like that documented in a research proposal.

35. 35 Step 2: Suggest Hypotheses The researcher generates intermediate hypotheses to describe a solution to the problem. The researcher generates intermediate hypotheses to describe a solution to the problem. This is at best a temporary solution since there is as yet no evidence to support either the acceptance or rejection of these hypotheses. This is at best a temporary solution since there is as yet no evidence to support either the acceptance or rejection of these hypotheses.

36. 36 Step 3: Literature Review The available literature is reviewed to determine if there is already a solution to the problem. The available literature is reviewed to determine if there is already a solution to the problem. Existing solutions do not always explain new observations. Existing solutions do not always explain new observations. The existing solution might require some revision or even be discarded. The existing solution might require some revision or even be discarded.

37. 37 Step 4: Literature Evaluation It’s possible that the literature review has yielded a solution to the proposed problem. It’s possible that the literature review has yielded a solution to the proposed problem. This means that you haven’t really done research. This means that you haven’t really done research. On the other hand, if the literature review turns up nothing, then additional research activities are justified. On the other hand, if the literature review turns up nothing, then additional research activities are justified.

38. 38 Step 5: Acquire Data The researcher now begins to gather data relating to the research problem. The researcher now begins to gather data relating to the research problem. The means of data acquisition will often change based on the type of the research problem. The means of data acquisition will often change based on the type of the research problem. This might entail only data gathering, but it could also require the creation of new measurement instruments. This might entail only data gathering, but it could also require the creation of new measurement instruments.

39. 39 Step 6: Data Analysis The data that were gathered in the previous step are analyzed as a first step in ascertaining their meaning. The data that were gathered in the previous step are analyzed as a first step in ascertaining their meaning. As before, the analysis of the data does not constitute research. As before, the analysis of the data does not constitute research. This is basic number crunching. This is basic number crunching.

40. 40 Step 7: Data Interpretation The researcher interprets the newly analyzed data and suggests a conclusion. The researcher interprets the newly analyzed data and suggests a conclusion. This can be difficult. This can be difficult. Keep in mind that data analysis that suggests a correlation between two variables can’t automatically be interpreted as suggesting causality between those variables. Keep in mind that data analysis that suggests a correlation between two variables can’t automatically be interpreted as suggesting causality between those variables.

41. 41 Step 8: Hypothesis Support The data will either support the hypotheses or they won’t. The data will either support the hypotheses or they won’t. This may lead the researcher to cycle back to an earlier step in the process and begin again with a new hypothesis. This may lead the researcher to cycle back to an earlier step in the process and begin again with a new hypothesis. This is one of the self-correcting mechanisms associated with the scientific method. This is one of the self-correcting mechanisms associated with the scientific method.

42. 42 Common Methodologies Methodologies are high-level approaches to conducting research. Methodologies are high-level approaches to conducting research. The individual steps within the methodology might vary based on the research being performed. The individual steps within the methodology might vary based on the research being performed. Two commonly used research methodologies: Two commonly used research methodologies: Quantitative. Quantitative. Qualitative. Qualitative.

43. 43 Methodology Comparison Quantitative Explanation, prediction Explanation, prediction Test theories Test theories Known variables Known variables Large sample Large sample Standardized instruments Standardized instruments Deductive Deductive Qualitative Explanation, description Build theories Unknown variables Small sample Observations, interviews Inductive

44. 44 An Overview of Empirical Research Methods Descriptive (Qualitative) Ethnography Ethnography Case Study Case Study Suvey/Sampling Suvey/Sampling Focus Groups Focus Groups Discourse/Text Analysis Discourse/Text Analysis Quantitative Description Quantitative Description Prediction/Classification Prediction/Classification Experimental (Quantitative) True Experiment Quasi-Experiment Meta-Analysis

45. 45 Assessing Methods Research Question(s) is/are key Research Question(s) is/are key Methods must answer the research question(s) Methods must answer the research question(s) Methodology guides application Methodology guides application Epistemology guides analysis Epistemology guides analysis

46. 46 Ethnographies + Observational field work done in the actual context being studied + Focus on how individuals interrelate in their own environment (and the influence of this environment) - Difficult to interpret/analyze - Time consuming/expensive - Can influence subject behavior

47. 47 Case Studies + Focus is on individual or small group + Able to conduct a comprehensive analysis from a comparison of cases + Allows for identification of variables or phenomenon to be studied - Time consuming - Depth rather than breadth - Not necessarily representative

48. 48 Survey Research + An efficient means of gathering large amounts of data + Can be anonymous and inexpensive - Feedback often incomplete - Wording of instrument can bias feedback - Details often left out

49. 49 Focus Groups + Aid in understanding audience, group, users + Small group interaction more than individual response + Helps identify and fill gaps in current knowledge re: perceptions, attitudes, feelings, etc. - Does not give statistics - Marketing tools seen as “suspect” - Analysis subjective

50. 50 Discourse/Text Analysis + Examines actual discourse produced for a particular purpose (job, school) + Helps in understanding of context, production, audience, and text + Schedule for analysis not demanding - Labor intensive - Categories often fluid, making analysis difficult

51. 51 Quantitative Descriptive Studies + Isolates systematically the most important variables (often from case studies) and to quantify and interrelate them (often via survey or questionnaire) + Possible to collect large amounts of data + Not as disruptive + Biases not as likely - Data restricted to information available

52. 52 Discourse/Text Analysis + Examines actual discourse produced for a particular purpose (job, school) + Helps in understanding of context, production, audience, and text + Schedule for analysis not demanding - Labor intensive - Categories often fluid, making analysis difficult

53. 53 Quantitative Descriptive Studies + Isolates systematically the most important variables (often from case studies) and to quantify and interrelate them (often via survey or questionnaire) + Possible to collect large amounts of data + Not as disruptive + Biases not as likely - Data restricted to information available

54. 54 Prediction and Classification Studies Goal is to predict behaviors: Prediction forecasts and interval variable (Diagnostic/TAAS scores) Prediction forecasts and interval variable (Diagnostic/TAAS scores) Classification forecasts a nominal variable Classification forecasts a nominal variable + Important in industry, education to predict behaviors - Need substantial population - Restricted range of variables can cause misinterpretation - Variables cannot be added together; must be weighted and looked at in context of other variables

55. 55 Positive Aspects of Descriptive/Qualitative Research Naturalistic; allows for subjects to interact with environment Naturalistic; allows for subjects to interact with environment Can use statistical analysis Can use statistical analysis Seeks to further develop theory (not to influence action); Prescientific Seeks to further develop theory (not to influence action); Prescientific Coding schemes often arise from interplay between data and researcher’s knowledge of theory Coding schemes often arise from interplay between data and researcher’s knowledge of theory

56. 56 Problems with Descriptive/Qualitative Research Impossible to overlay structure Impossible to overlay structure Impossible to impose control Impossible to impose control Subject pool often limited, not representative Subject pool often limited, not representative Seen as more “subjective,” less rigorous Seen as more “subjective,” less rigorous Beneficial only in terms of initial investigation to form hypothesis Beneficial only in terms of initial investigation to form hypothesis

57. 57 Experimental Research: True Experiment + Random sampling, or selection, of subjects (which are also stratified) + Introduction of a treatment + Use of a control group for comparing subjects who don’t receive treatment with those who do - Adherence to scientific method (seen as positive, too) - Must have both internal and external validity - Treatment and control might seem artificial

58. 58 Experimental Research: Quasi- Experiment + Similar to Experiment, except that the subjects are not randomized. Intact groups are often used (for example, students in a classroom). + To draw more fully on the power of the experimental method, a pretest may be employed. + Employ treatment, control, and scientific method - Act of control and treatment makes situation artificial - Small subject pools

59. 59 Meta-Analysis + Takes the results of true and quasi-experiments and identifies interrelationships of conclusions + Systematic + Replicable + Summarizes overall results - C/C apples and oranges? - Quality of studies used?

60. 60 Positive Aspects of Experimental Research Tests the validity of generalizations Tests the validity of generalizations Seen as rigorous Seen as rigorous Identifies a cause-and-effect relationship Identifies a cause-and-effect relationship Seen as more objective, less subjective Seen as more objective, less subjective Can be predictive Can be predictive

61. 61 Problems with Experimental Research Generalizations need to be qualified according to limitation of research methods employed Generalizations need to be qualified according to limitation of research methods employed Controlled settings don’t mirror actual conditions; unnatural Controlled settings don’t mirror actual conditions; unnatural Difficult to isolate a single variable Difficult to isolate a single variable Doesn’t allow for self-reflection (built-in) Doesn’t allow for self-reflection (built-in)

62. 62 Testing the Waters How do you come up with a good research question? How do you come up with a good research question? How do you determine if the method you plan to use will answer your question? How do you determine if the method you plan to use will answer your question? What epistemology should you use to analyze data? What epistemology should you use to analyze data?

63. 63 Quantitative Methods Samplingst Samplingst Testing of Hypothesis Testing of Hypothesis Chi Square Test Chi Square Test ANOVA ANOVA Multivariate Analysis Multivariate Analysis

64. 64 Thanks Thanks