Research Design

Research Desgin It is the entire design of the research project It is said as the research proposal It involves deciding on all aspects of the research process That includes  Philosophical assumption  Research method  Data collection techniques  Data analysis  Publication outlet if possible It matters to get your proposal accepted or rejected

Research Process Problem Liter., Theory Propositions/ Hypothesis Research Design Data Collection & Analysis Data Interpretation

How to Knowledge Process Iterations Theory Inference Identify Problem & Motivate Define Problem Show Importance Define Objectives of a Solution What would a Better Artifact Accomplish? Design & Development Artifact Disciplinary Knowledge Metrics, Analysis Knowledge Demonstration Find Suitable context Use Artifact to Solve problem Evaluation Observe How Effective, efficient Iterate Back to design Communica- tion Scholarly Publications Professional Publications Design research process

Step 1: Problems/Questions A question/problem occurs or is posed to the researcher for which that researcher has no and can find no answer(s).

Example -1 User profiling on the Internet How can we support an individual user in providing information suited to their needs? There are many approaches to automated profiling of users based on their web habits, and occasionally they throw up some interesting information. Can we improve how users' information needs are identified, specified, and managed in specific domains, and in general.

Step 2: Literature, Theory, Logic Available literature/theory is reviewed to determine if there is already a solution to the problem. Existing solutions do not always explain new conditions. The existing solution might require some revision or even be discarded.

Example III – Interface Agents Interface agents have become a technology widely used to provide personalized assistance to users with their computer-based tasks. Interface agents are computer programs that have the ability to learn a user’s preferences and working habits, and to provide him proactive and reactive assistance in order to increase the user’s productivity. A commonly used metaphor for understanding interface agent paradigm is comparing them to a human secretary or personal assistant who is collaborating with the user in the same work environment (Maes, 1994).

Example – III Interface agent – Research gap Horvitz (1999) has pointed out some problems with the use of interface agents: poor guessing about the goals and needs of users, inadequate consideration of the costs and benefits of each agent action, poor timing of agent actions and inadequate attention to opportunities that allow a user to guide the invocation of agent services and to refine potentially suboptimal results provided by the agent. Research Gap we consider that interface agent developers have paid little attention to two key issues when developing agents that assist users: how to best interact with each user and how to provide them assistance of the right sort at the right time.

Hypothesis We have focused our attention on studying the interaction between a user and an agent, as well as on how a user wants to be assisted. We believe that, in the same way as each person works and interacts in a certain way with his/her personal assistant, each user interacts in a personal way with an interface agent. Conceptual Framework The proposed solution is by studying user and agent interaction, we can design a better agent that can serve the user to achieve his/her goals.

Step 3: Propositions/Hypotheses Application only for Quantitative Research The researcher generates intermediate propositions and/or hypotheses to gain a better understanding of the problem or describe a solution to the problem.  This is at best a temporary solution since there is as yet no evidence to support either the acceptance or rejection of these propositions/hypotheses.

Hypothesis Understanding user and agent interaction results in better agent design Another example  Graphical user Interface increases software learnability  Use of Could computing reduces system interruption  Use of Could computing increases organizational profitability

Step 4: Research Design Research methodologies reflect high-level approaches to conducting research.  The individual steps within the methodology might vary based on the research problem being performed. There are three broad approaches to be used in research design  Quantitative.  Qualitative.  Design science

Step 4: Design a Study QuantitativeQualitativeDesign science Explanation, prediction Explanation, description Motivated to solve problems Test/Modify theories Build theories Prescriptive – has heuristic nature Known variablesUnknown variables Known variables Large sampleSmall sample Small sample, case based Standardized instruments Observations, interviews Observation, interviews, etc Deductive Inductive

Step 4: Cont’d QuantitativeQualitativeDesign science Statistical generalization Analytic generalization Replicable to similar problems in similar contexts Use numbers Use textsTexts, codes and drawings Standard instrument to validation No standard instrument, triangulation Pragmatic validity Objective TruthSubjective truth Subjective Truth ??? Linear Process Iterative

Step 5: Acquire/Analyze Data The researcher collects data relating to the research problem.  The means of data acquisition will often change based on the type of the research problem.  This might entail only data gathering, but it could also require the creation of new measurement instruments.

Example III – Interface Agents To prove our hypothesis, we carried out an experiment with real users interacting with different interface agents, and we discovered several user–agent interaction issues that require personalization. Data collected about: 1. we studied what kind of software assistant different users want 2. we studied users’ preferences and assistance requirements concerning three assistance actions: warnings, suggestions and actions on the user’s behalf. 3. we analyzed users’ reactions towards interruptions, i.e. whether users object to being interrupted by their agents or not

Cont’d 4. we considered users’ tolerance of agents’ errors, since some users are more tolerant than others 5. we investigated how users feel about providing explicit feedback to train their agents. Not all users are willing to give explicit feedback, and their willingness may vary as they interact with the agent 6. we analyzed users’ opinions regarding agents’ ability to perform actions on behalf of users and the possibility of giving agents more autonomy

Step 5: Acquire/Analyze Data The data that were gathered in the previous step are analyzed as a first step in ascertaining their meaning. Involves statistical analyses, for the most part.  The specific statistical tests and procedures depend on the nature of the underlying data.  Qualitative researchers used words to understand and describe the phenomena

Step 6: Data Interpretation The researcher interprets the newly analyzed data and suggests a conclusion.  Interpretation means identifying relationship between variables to draw conclusions  Understand the meaning behind the data  The Data for SW success shows low figures, below our expectation. What it tells us! Does it mean the SW failed? What causes the Failure? How do we overcome those Failures?  Computer importation increased in Ethiopia. What does it mean?

Example: Data Interpretation -- Interface Agents As a result of our experiment we found that different users have different preferences and requirements regarding the interaction issues mentioned above. This is an indication that personalization is required if interface agents want to assist users as they expect. As a result of our experiment we found that different users have different preferences and requirements regarding the interaction issues mentioned above. This is an indication that personalization is required if interface agents want to assist users as they expect. Thus, we propose a definition for a user interaction profile that models a user’s interaction preferences, requirements and habits. The user interaction profile can be used by an interface agent developers to personalize its interaction with each user and to adapt its behavior to each user’s interaction and assistance requirements.

Step 6: Compare Results to Propositions/Hypotheses The data will either support the propositions and hypotheses or they won’t.  This may lead the researcher to cycle back to an earlier step in the process and begin again with a new hypothesis.  This is one of the self-correcting mechanisms associated with the scientific method.

Research Proposal

Structure of Research Proposal 1. Title 2. Abstract 3. Introduction 1. Research Problem 2. Research Objective 4. Literature Review 5. Research Method 6. Significance of the Study 7. Research Budget 8. Research Schedule 9. Bibliography

Title It should be concise and descriptive. For example, the phrase, "An investigation of..." could be omitted. Often titles are stated in terms of a functional relationship, because such titles clearly indicate the independent and dependent variables. Think of an informative but catchy title. An effective title not only pricks the reader's interest, but also predisposes him/her favorably towards the proposal. E.g. change without change

Abstract It is a brief summary of approximately 300 words. It should include the research question, the rationale for the study, the hypothesis (if any), the method and the main findings. Descriptions of the method may include the design, procedures, the sample and any instruments that will be used.

Introduction The introduction typically begins with a general statement of the problem area, with a focus on a specific research problem, to be followed by the rational or justification for the proposed study. The introduction generally covers the following elements: 1. State the research problem, which is often referred to as the purpose of the study. 2. Provide the context and set the stage for your research question in such a way as to show its necessity and importance. 3. Present the rationale of your proposed study and clearly indicate why it is worth doing.

Cont’d 4. Briefly describe the major issues and sub-problems to be addressed by your research. 5. Identify the key independent and dependent variables of your experiment. Alternatively, specify the phenomenon you want to study. 6. State your hypothesis or theory, if any. For exploratory or phenomenological research, you may not have any hypotheses. 7. Set the delimitation or boundaries of your proposed research in order to provide a clear focus. 8. Provide definitions of key concepts. (This is optional.)

Method – Example of Quantitative Design -Is it a questionnaire study or a laboratory experiment? What kind of design do you choose? Subjects or participants - Who will take part in your study ? What kind of sampling procedure do you use? Instruments - What kind of measuring instruments or questionnaires do you use? Why do you choose them? Are they valid and reliable? Procedure - How do you plan to carry out your study? What activities are involved? How long does it take?

Design Science What procedures to follow to design the new artifact For example for Software Development? Describe how you collect the Requirement? From whom you want to collect  Focused group, interview, questionnaire How you structure your requirement like class diagram or other tools What algorithm you want to develop Which language to use for your algorithm What you evaluate your design solutions? With which population group? Why you select that One? How long? Justify all your selection by citing other authors for using similar method to solve or its appropriateness

Results Obviously you do not have results at the proposal stage. However, you need to have some idea about what kind finding you get from your research. Describe the expected research or design outputs It is used to evaluate your success or failure

Failure to Proposal Writing Failure to provide the proper context to frame the research question. Failure to delimit the boundary conditions for your research. Failure to cite landmark studies. Failure to accurately present the theoretical and empirical contributions by other researchers. Failure to stay focused on the research question.

Cont’d Failure to develop a coherent and persuasive argument for the proposed research. Too much detail on minor issues, but not enough detail on major issues. Too much rambling -- going "all over the map" without a clear sense of direction. (The best proposals move forward with ease and grace like a seamless river.) Too many citation lapses and incorrect references. Too long or too short. Slopping writing.