1. Prof. A. Taleb-Bendiab Room 605 a.talebbendiab@livjm.ac.uk http://www.cms.livjm.ac.uk/taleb/lectures/cmssem003 A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 1 Survey Research

2. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 2 So far … #1  Introduction  Nature of research  Definitions  Research as a process Defining the research problem, Research protocol design, Undertake research, and document  Classifying Research  The field of research,  The purpose of research To review existing knowledge, to describe some situation or problems, the construction of something novel, explanation.  The approach to research, Theoretical, Laboratory experiment, field experiment, case- study and survey.  The nature of research Pure basic research, Testing of existing theory, description of the state-of-the-art, specific problem solution.

3. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 3 So far … #2  Types of research methods  Qualitative research  Survey research  Action research  Ethnographic research  Case-study research  Observation  Quantitative Research  Sampling  Measurement & Data Collection  Experimental Research

4. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 4 So Far … #3  Examined three types of experimental design:  Pre-Experimental Design  True Experimental Design  Quasi-Experimental Design

5. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 5 In This Session … Survey Research  Definition  Planning and undertaking survey  Information sources, search method  Guidelines  Documentation and presentation

6. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 6 Introduction  Survey research method  is a commonly used method in CS and IT research including; human factors concerns in IT, IS systems modelling, and evaluation.  It can be used as a research approach in its own right Or as a tool in conjunction with other forms of research methods.  Survey can be used in several ways:  Descriptive, the gathering of facts, what is questions,  Formative and summative  Relational, studying relationships between variables, this approach is most often combined with the descriptive, but the descriptive can stand on its own.  Validation/evaluation  reactions and impact  learning -- enhanced attitudes, perceptions or knowledge  changes in skills -- applied the learning to enhance behaviours.  Effectiveness and/or performance enhancement, etc.

7. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 7 Classification of Surveys  Generally there are two kinds of surveys:  population survey, involves using the whole of a specified population;  sample survey, involves using only a subset of the total population.  Scope of surveys -- Can be used for studies on both very large or very small populations.  Surveys can involve the use of different data gathering instruments:  interview, questionnaire,  controlled observation:  official records, standardised tests,  participant observation, non-participant observation  Case-study  Note -- good planning and design is essential

8. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 8 Basic Data Gathering Methods #1  Questionnaires  Purpose  To quickly and/or easily get lots of information from people in a non threatening way  Advantages  can complete anonymously, inexpensive to administer,  easy to compare and analyse, administer to many people,  can get lots of data, many sample questionnaires already exist  disadvantages  might not get careful feedback, wording can bias client's responses,  are impersonal, in surveys, may need sampling expert,  doesn't get full story

9. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 9 Basic Data Gathering Methods #2  Interviews  Purpose  To fully understand someone's impressions or experiences, or learn more about their answers to questionnaires  Advantages  get full range and depth of information, develops relationship with client,  can be flexible with client  disadvantages  can take much time,  can be hard to analyse and compare,  can be costly, interviewer can bias client's responses

10. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 10 Basic Data Gathering Methods #3  documentation review  Purpose  when want impression of how program operates without interrupting the program; is from review of applications, finances, memos, minutes, etc.  Advantages  get comprehensive and historical information, doesn't interrupt program or client's routine in program,  information already exists, few biases about information  disadvantages  often takes much time, info may be incomplete,  need to be quite clear about what looking for, not flexible means to get data; data restricted to what already exists

11. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 11 Basic Data Gathering Methods #4  Observation  Purpose  to gather accurate information about how a program actually operates, particularly about processes  +  view operations of a program as they are actually occurring,  can adapt to events as they occur  -  can be difficult to interpret seen behaviours  can be complex to categorize observations  can influence behaviours of program participants,  can be expensive

12. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 12 Basic Data Gathering Methods #5  Focus groups  Purpose  explore a topic in depth through group discussion, e.g., about reactions to an experience or suggestion, understanding common complaints, etc.; useful in evaluation and marketing  +  -quickly and reliably get common impressions -can be efficient way to get much range and depth of information in short time - can convey key information about programs  -  -can be hard to analyse responses -need good facilitator for safety and closure -difficult to schedule 6-8 people together

13. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 13 Basic Data Gathering Methods #6  case studies  Purpose  to understand and model a given system and its relationship to it external environment such a business process modelling, information and IT systems requirements.  To analyse and validate developed models, tools and techniques through for observing and examining a representative sample of the studied population.  +  fully depicts client's experience in program input, process and results,  powerful means to portray program to outsiders  -  usually quite time consuming to collect, organize and describe  represents depth of information, rather than breadth  Difficulty to access bias and subjectivity

14. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 14 Planning the Survey  Design:  the chosen research methods must be specifically tailored so that the information requirements of the study are translated into specific questionnaires, interview questions, scale tests, etc.  Pilot:  after initial design a pilot test is extremely useful as a check on the efficacy of the survey design.  Implementation:  after the pilot it may be necessary to incorporate amendments to the original design in order to achieve the final implementation of the survey.  Validation  how to analyse and interpret and report results.

15. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 15 Key Considerations #1  Purpose of the research  what do you want to be able to decide as a result of the research?  The audiences for the information from the research including; management, employees, customers, etc.  Requirements  Information  needed for the survey  Source should the information be collected, e.g., employees, customers, groups of employees or customers, certain documentation, etc.?  how much can be collected and analysed in a low-cost and practical manner, e.g., using questionnaires, surveys and checklists?  Accurate will the information be (reference the above table for disadvantages of methods)?  Will the methods get all of the needed information?

16. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 16 Key Considerations #2  Process  How can that information be collected in a reasonable fashion, e.g., questionnaires, interviews, examining documentation, observing staff and/or clients in the program, conducting focus groups among staff and/or clients, etc?  When is the information needed (so, by when must it be collected)?  What resources are available to collect the information?  Choice of sample:  rationale should be top down: identify the population first, if it is too large to survey then define a sample of that population;  is the objective to show: that the small sample differs from the general population, or that results from the small sample can be generalised to the larger population?

17. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 17 Key Considerations #3  Choice of Survey Methods  Interview, Questionnaire, Controlled Observation,  Or any combination may be used.  Consider the following questions:  What additional methods should and could be used if additional information is needed?  Will the information appear as credible to decision makers, and/or assessors?  Will the nature of the audience conform to the methods including; will they fill out questionnaires carefully, engage in interviews or focus groups, let you examine their documentations, etc.?  Who can we administer the methods now or is training required?  How can the information be analysed?  Sensitivity of the information sought may be a factor in our choice.  Available resources have a bearing on the choice!

18. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 18 Examples of IS Surveys  Human Factors  Users requirements for improvement of an existing system.  Users reactions to implementation of a new system.  Programmers with a new development environment.  System designers reaction to a new CASE tool.  Increase in workflow due to changes in a document management system

19. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 19 Next Session …  Project Planning and Management  Introduction  Project modelling and Network analysis  Etc.

20. A. Taleb-Bendiab, Module: Research Methods, Code: CMSSEM003, Lecture: 4, Page: 20 Tutorial  Example  Your company ResearchMethods Ltd. has just been awarded a feasibility study contract to develop a web-based groupware software to support distributed software engineering. To this end, you have persuaded your customer to conduct a survey prior to starting the software design and implementation.  As a team leader, your role is to plan, design and manage the proposed survey. Thus, you are required to submit a report outlining relevant information including;  Aim and objectives  Research methods to be used  Resources requirement including; HR, cost and information  Implementation plans  Analysis of the results  Case History