Presentation is loading. Please wait.

Presentation is loading. Please wait.

IS Research Development an International Perspective

Similar presentations

Presentation on theme: "IS Research Development an International Perspective"— Presentation transcript:

1 IS Research Development an International Perspective
Università degli Studi di Verona Facoltà di Economia Nicholas C. Romano, Jr. SDA Bocconi Information Systems Division Università Commerciale Luigi Bocconi Spears School of Business Management Science and Information Systems Oklahoma State University – Tulsa

2 Talk Outline Personal Introduction Business and IS Research
Research Defined and Ways of Knowing Scientific Method and Theory Research Process Research Paradigms Theory Construction IS Research Methods

3 1. Personal Introduction
Management Information Systems 1. Personal Introduction Nicholas C. Romano, Jr., Ph.D. Associate Professor, MSIS Spears School OSU Ph.D. from University of Arizona in MIS (1998) Family Man, Husband and Father Wife Rosalina, 2 Daughters Isabella (8) Gabriela (6) 1 Son Nico (3) Dog Osa AIS Council Member (Americas Representative – ) Very Active in AIS and the AMCIS and HICSS Conferences 20 years’ work experience, much of it in group support for teamwork and projects Former IBMer (Dad, Brother, Sister, Brother in law, Great Uncle, Second cousin, others I am sure) Occasional, but Terrible Golfer ( FORE!!!) – want to do it more.. Isabella 8 Gabriela 6 Nico 3

4 What is Business Research?
the “systematic and objective process of gathering, recording and analyzing data for aid in making business decisions” (Zikmund, Business Research Methods, 2002, p. 6) Systematic and Objective Distinguish Business Research Important tool for managers and decision-makers in corporate and non-corporate organizations

5 When is Business Research Used?
Business research methods used in situations of uncertainty, when decision-makers face two or more courses of action and seek to select the best possible alternative under the circumstances Aims to improve the quality of decision-making which, in turn, benefits the organization and helps ensure its continuity and efficiency

6 Typical Users of Business Research Methods
Businesses and Corporations Public-Sector Agencies Consulting Firms Research Institutes Non-Governmental Organizations Non-Profit Organizations Independent Researchers and Consultants

7 Fields Where Business Research is Often Used – (1)
General Business Conditions and Corporate Research Short- & Long-Range Forecasting, Business and Industry Trends Global Environments Inflation and Pricing Plant and Warehouse Location Acquisitions Financial and Accounting Research Forecasts of financial interest rate trends, Stock,bond and commodity value predictions capital formation alternatives mergers and acquisitions risk-return trade-offs portfolio analysis impact of taxes research on financial institutions expected rate of return capital asset pricing models credit risk cost analysis Management and Organizational Behaviour Research Total Quality Management Morale and Job Satisfaction Leadership Style Employee Productivity Organizational Effectiveness Structural ssues Absenteeism and turnover Organizational Climate

8 Fields Where Business Research is Often Used – (2)
Sales and Marketing Research Market Potentials Market Share Market segmentation Market characteristics Sales Analysis Establishment of sales quotas Distribution channels New product concepts Test markets Advertising research Buyer behaviour Customer satisfaction Website visitation rates Information Systems Research Knowledge and information needs assessment Computer information system use and evaluation Technical suppot satisfaction Database analysis Data mining Enterprise resource planning systems Customer relationship management systems Corporate Responsibility Research Ecological Impact Legal Constraints on advertising and promotion Sex, age and racial discrimination / worker equity Social values and ethics

9 The Value of Business Research for Managers – (1)
Uncertainty Reduction and improved decision-making quality with several consequent advantages (e.g. strategic, operational) and benefits for Firms Business Research Methods can be employed in 4 stages: Identification of problems and/or opportunities Useful for strategy planning, analysis of internal and external organizational environment

10 The Value of Business Research for Managers – (2)
(2) Diagnosis and Assessment of problems and/or opportunities Gain insight into underlying reasons and causes for the situation. If there is a problem, it asks what happened and why? If there is an opportunity, it seeks to explore, clarify and refine the nature of the opportunity and, in the case of multiple opportunities, seeks to set priorities (3) Selection and Implementation of Courses of Action After alternative courses of action have been determined, selection of the best possible course.

11 The Value of Business Research for Managers – (3)
An important consideration is the quality of forecasting which is an essential tool of research (4) Evaluation of Courses of Action Business Research Methods are used after a course of action has been implemented in order to determine whether activities have been properly implemented and have accomplished what they intended to do

12 The Value of Business Research for Managers – (4)
Evaluation Research Formal objective measurement and evaluation of the extent which an activity, project or program has achieved its goal, and the factors which influence performance (e.g. audits). Formal objective measurement and evaluation of the extent to which on-going activities, projects or programs are meeting their goals (performance-monitoring research)

13 When Should Business Research be Undertaken?
Is sufficient time available? Yes NO Is information inadequate? Do not undertake Business Research NO Yes High importance of decision? NO Yes NO Do Undertake Business Research Research benefits greater than costs? Yes

14 Value and Costs of Undertaking Business Research
Decreased Uncertainty Higher Likelihood of Correct Decisions Better Business performance Higher Profits Better Reputation Research Costs Delay in Making Business Decisions Disclosure of Information to Rivals Possibility of Error COSTS VALUE

15 Research Building Blocks
WISDOM Blend of Knowledge information, experience and in-sights that provides a framework that can be thoughtfully evaluated when assessing new information or evaluating relevant situations KNOWLEDGE Blend of information, experience and in-sights that provides a framework that can be thoughtfully evaluated when assessing new information or evaluating relevant situations Determination of relationship amongst data with a view to facilitating understanding of the phenomena, their relationships and decision-making (e.g. past and predicted future sales trends) INFORMATION Measurements of phenomena (e.g. sales statistics of a department store) DATA

16 Research Introduction
Definition Ways of Knowing The Scientific Method

17 Origin of the Word “Research”
From French word “recherche” to travel through or survey.

18 What is Research? “If research is to make the contribution to practice that is now possible, we must start with an adequate concept of the nature of research.” “Research is an unusually stubborn and persisting effort to think straight which involves the gathering and the intelligent use of relevant data” Hamlin, H. M. (1966) What is Research? American Vocational Journal, September See:

19 Pendulum of Scholarship in Business Management Schools
Social System of Science -Scientists -Graduate Schools -Research Institutes -Scholarly Societies Social System of Practice -Practitioners -Managers -Businesses -Trade Associations -Management Societies Management Consulting Disciplinary Science As you know, the gap between research and practice of management knowledge is a complex and controversial subject. Herbert Simon provided a useful way to frame the gap. He proposed that a basic challenge for scholars in professional schools is to conduct studies that contributed to both theory and practice – not either-or. Simon, advocated that researchers become involved in engaged scholarship. I adopt Ernest Boyer’s (1997) broad view of “scholarship” as including the scholarship of discovery, teaching, practice, and integration. Knowledge advancements in each of these domains are important, and we should not limit ourselves to one domain. The social systems of management and science are different communities of practice, and the only way to understand each community is to interact in each. Each represents a different and exclusive community of trained practitioners who apply specialized knowledge to perform socially-valued work, who identify with work, and share common values, norms, & procedures Interactions between scientists and practitioners decrease stereotypes, increase familiarity, respect, and trust, and set the stage for the emergence of a learning community. Professional Learning Community Adapted From Van de Ven, Engaged Scholarship

20 Elements of Research Not Frequently Discussed
Research is not a linear process It is just written up like it is. One study leads to others Research is a social process Not because research is social but because results must enter into a social “learned” society (be read and cited) Research value (impact) more a question of importance than volume But volume is a wonderful, simple measure of productivity But, one good published idea is worth more than 100 articles How do you know value? CITES!!!Citation Tools [isi web of science; Google Scholar, Citeseer, SSRN, Libra, Publish or Perish]

21 Little Discussed Elements (II)
Research is for posterity i.e., it has a different time scale than consulting Refereed archival journals versus the Internet Research builds upon the past …by tearing it down (theory building), or by supporting it (replication studies; theory extension) Research not published is virtually worthless The importance is more to be read than to read! Research demands special form of writing and language

22 Some Myths About Research
Purpose of research is to “Prove” or “Confirm” a theory Research findings are presented as “Complete” and “Conclusive” answers Research Scientists come to “Consensus” or “Agreements” on how things work (i.e. Global Warming; Pluto a planet) There is a hierarchy of research methodologies that places true “experimental” research at the top. NONE OF THESE ARE TRUE!

23 Key Terms Philosophy Epistemology
Distinguishing True (Real) Knowledge from False (Pseudo) Knowledge The Love of Knowledge

24 Different Ways of “Knowing”
Authority Because someone you respected told you so Tenacity Because it has withstood the test of time Serendipity discovery by accident Logic / Reason Because you figured it out with your mind Science (Research) We’ll get to that shortly…

25 Authority How do we know that the Earth is flat?
Right, but how do I know? But how do you know? Because Claudius Ptolemy said so. Because The Pope said so. (Pope Paul V) Because I’m in charge and I am putting you (Galileo)in prison!

26 Galileo Galilei (15641642) First to use telescope to study sky
Discovered Solar spots and Jupiter’s satellites (Galilean moons) Believed Earth moves around Sun In 1632 he was convicted of heresy. In 1992 it was officially stated by the Pope that Galileo was right. (360 years later) Authority is SLOW to change

27 Tenacity Grandpa, how do I know that I should drink 8 cups of water per day? But how did he know? But how did THEY KNOW?! Because that’s what my father did. Because that’s what his father did. Because that’s what his father did! Well you’re alive, aren’t you?

28 Serendipity Columbus is the archetype of surprising discoveries
In 1492 Columbus sailed the ocean blue In quest of a passage through The Indies and the orient too He discovered America, Serendipitous through and through.

29 Serendipity Isaac Newton's famed apple falling from a tree, led to his musings about the nature of gravitation. “In the year 1666 he retired again from Cambridge to his mother in Lincolnshire. Whilst he was pensively meandering in a garden it came into his thought that the power of gravity (which brought an apple from a tree to the ground) was not limited to a certain distance from earth, but that this power must extend much further than was usually thought. Why not as high as the Moon said he to himself & if so, that must influence her motion & perhaps retain her in her orbit, whereupon he fell a calculating what would be the effect of that supposition.“ John Conduitt, Newton's assistant at the royal mint

30 Serendipity The Post-it note was invented in 1968 by Dr Spencer Silver, a 3M scientist who stumbled upon a glue (Acrylate-copolymer microspheres [adhesive formula] ) that was not sticky enough. In 1968, Silver developed a high-quality but "low-tack" adhesive, made of tiny, indestructible acrylic spheres that would stick only where they were tangent to a given surface, rather than flat up against it. As a result, the adhesive's grip was strong enough to hold papers together, but weak enough to allow the papers to be pulled apart again without being torn. More importantly, the adhesive could be used again and again. Silver wanted to market the adhesive as a spray, or as a surface for bulletin boards on which temporary notices could be easily posted and then removed. Over the next five years, Silver tried to interest his colleagues at 3M, informally and in presentations. A marketable form of the product proved elusive however, until Arthur Fry attended one of Silver's seminars. Fry sang in his church choir. He was frustrated the paper bookmarks he used to mark the songs in his hymnal would not stay put. In a moment of insight, Fry realized that Silver's reusable adhesive would provide precisely what he needed. Fry wrote up his idea for a reusable bookmark and presented it to his supervisors. Initially, management was skeptical, but the staff could not get enough of the samples Fry was passing around. Soon, 3M gave the invention its full support. It took another five years to perfect and design machines to manufacture the product, but in 1980, Post-it® Notes were introduced nationwide. Within two years, the product became a necessity in the office, schools, labs, libraries, and even in homes.

31 Logic and Reasoning Understanding phenomena by analyzing with our minds what we observe with our senses. Syllogism A logical argument consisting of two premises and a conclusion. Example: Persons who smoke cigarettes have a high rate of lung cancer. Persons who do not smoke cigarettes have a low rate of lung cancer. Therefore, smoking cigarettes causes lung cancer.

32 Syllogism Persons who smoke cigarettes have a high rate of lung cancer and yellow teeth. Persons who do not smoke cigarettes have a low rate of lung cancer and yellow teeth. Therefore, smoking cigarettes causes lung cancer and yellow teeth.

33 Well that sounds pretty good, I’ll just use logic and reasoning for my research!
Problems with limiting our knowledge to what we can discover with logic and reason: Subjectivity (Bias) We do not observe “the whole picture” We have no external “check” on our logical thought processes Example: I observe that all stars follow a regular pattern of motion in the sky in relation to the Earth. Therefore the Earth is stationary and at the center of the universe.

34 Why not just rely on pure observation?
What one observes: May not be Quantifiable May Change over time May not be Reality Can be based on Misinformation or Bias

35 Why not just rely on pure observation?
Count the Black Dots….How many do you see?

36 Why not just rely on pure observation?
Are the Horizontal lines parallel or do they slope?

37 Actual building in Melbourne, Australia

38 Why not just rely on pure observation?
How many legs does this elephant have?

39 Are the two boys the same or different?
Why not just rely on pure observation? Are the two boys the same or different?

40 Count the men one last time Count the men Again
Why not just rely on pure observation? Count the men Count the men one last time Count the men Again

41 Is the wine glass on or off the tray
Is the wine glass on or off the tray? Is the water glass standing or laying down

42 Are the three purple Shapes Squares. Are their sides parallel
Are the three purple Shapes Squares? Are their sides parallel? Are they moving or still?

43 People seem to change size as they move around
The Ames Room People seem to change size as they move around Figure 3.25 from: Kassin, S. (2001). Psychology, third edition. Upper Saddle River, NJ: Prentice Hall.

44 How the Ames Room works Figure 3.25 from:
Kassin, S. (2001). Psychology, third edition. Upper Saddle River, NJ: Prentice Hall.

45 Actual Position of Person A
How the Ames Room works Actual Position of Person A Apparent Position of Person A Actual and Apparent Position of Person A Apparent Room shape Viewing Hole Figure 3.25 from: Kassin, S. (2001). Psychology, third edition. Upper Saddle River, NJ: Prentice Hall. Doctoral Seminar – MSIS 6333 Wednesday August 21st, 2009 Nicholas C. Romano, Jr. Oklahoma State University

46 Why not just rely on pure observation?
Subjectivity (Bias) “group A is nicer than group B” Recall (forgetfulness – selective memory) What did you say to me last week about topic X? Interpretations or conclusions that lack convincing support “most kids don’t care what their parents say”

47 Even reason, when applied with bias,
leads to irrationality and incorrect conclusions.

48 Logical Fallacies Fallacies occur when we reach wrong conclusions based on real observations or facts. Examples: Cum hoc ergo propter hoc (with this, therefore because of this) – Attributing causality based on correlation Converse accident – Generalizing to a group based on an individual (or a small set of individuals) Accident – Specifying to an individual based on a group Post hoc ergo propter hoc (after this, therefore because of this) – Attributing causality based on temporality See a list of 40 Fallacies at:

49 Penguins are Blank and White.
Old TV shows are Blank and White. We all know that Penguins cannot fly. Therefore some Penguins are Old TV Shows. Logic, one more thing that Penguins are not good at.

50 So I should just accept that ignorance is bliss and that I don’t know anything?
Maybe if you want to sit on your porch all day doing nothing, but ……. Authority isn’t Always Inaccurate We need to rely on knowledge our parents, teachers, government tells us Tradition is Important We all need a starting point and roots Logic and Reason are Powerful Tools Our brains are like supercomputers We couldn’t survive without Thinking Each way of knowing can only lead us so far We need a method to correct for weaknesses of other approaches

51 Scientific Method (or methods)

52 Science and other kinds of knowledge
Religious Knowledge Artistic/Mystic Knowledge Scientific Knowledge Bible-thumping fundamentalist or robe-draped monk; fond of Sunday-morning radio. Geek with pocket protector And calculator; watches Discovery Channel often. Crystal-hugging wearer Of tie-dyed T-shirts; listens to new-age music. Outrageous stereotype of user From ancient texts or revelations of inspired individuals. From evidence generated by observation of nature or by experimentation. How one discovers Knowledge From personal insight, Or insight of others Extent to which knowledge changes through time Little. May be considerable. Considerable. Extent to which Future changes in Knowledge are expected by user Can be expected, to the degree that the user Expects personal Development None. Considerable. Unchangeable except by reinterpretation by authorities, or by new inspired revelations, or by divergence of mavericks. How knowledge changes through time As user changes or as User encounters ideas of others By new observations or experiments, and/or by reinterpretation of existing data.

53 predictable, and explainable
Science and other kinds of knowledge (continued) Religious Knowledge Artistic/Mystic Knowledge Scientific Knowledge Certainty of the user High, given sufficient faith; Can be complete. High Dependent on quality and Extent of evidence; should never be complete. Assumptions Ancient texts or Inspired revelation have meaning to modern or future conditions. personal feelings And insights reflect nature. Nature has discernible, predictable, and explainable patterns of behavior. Where users put Their Faith In the supernatural beings That they worship or in the authorities who interpret Texts and events. In their own perceptions. In the honesty of people reporting scientific data (the incomes of whom depend on generation of that data), and in the human ability to Understand nature. Sources of Contradiction Between different religions; between different texts and/or authorities within one religion; within individual texts (as in the two accounts of human origin in the Judeo Christian Genesis). Between users, who Each draw on their own Personal Insights Across time, as understanding changes; between fields, which use different approaches and materials; and between individuals, who use different approaches and materials.

54 Religion and Science Science is based on skepticism and experiment
Religion is based on faith However Many scientists are religious Also many leaders of religion have been great scientists (Mendel – father of experimental Genetics - Monk) Science and Religion are simply different parts of our lives Science cannot disprove the idea of God Religion cannot prove that Science is wrong

55 Scientific Inquiry as a way of Knowing
Science is a disciplined, systematic way to understand the nature of the universe. Science uses empirical data to test falsifiable theories via a deductive method. WHAT THE HECK DOES THAT MEAN???

56 Science = order, explanation, rational methods, logic
The main purpose of science is to trace, within the chaos and flux of phenomena, a consistent structure with order and meaning. This is called the philosophy of rationalism, rational as in conforming with reason. And the purpose of scientific understanding is to coordinate our experiences and bring them into a logical system.

57 Science is also a Dialogue between Humankind and Nature.
Science is far from a perfect instrument of knowledge, but it provides something that other philosophies fail to, concrete results. Science is a “candle in the dark'' to illuminate irrational beliefs or superstitions

58 Six General Goals of Science
Organize & categorize things (typologies and ontologies) Explain Past Events Predict Future Events Control Future Events Provide a Sense of Understanding Generalize Results Adapted from: Reynolds, P. D. (1971). A primer in theory construction. Indianapolis, The Bobbs-Merrill Company. This paper proposes a new systems development methodology entitled heuristic dataflow diagrams (HDFDs). HDFD leverages group support systems (GSS) and heuristics to improve the creation of DFDs, which are frequently used design artifacts in systems development. We propose theory-based hypotheses to predict likely outcomes of using HDFD in non-GSS groups, GSS groups, and distributed GSS groups. These hypotheses were tested in a laboratory experiment using 123 subjects. The results indicate that HDFD performed with GSS provides process gains over HDFD performed without GSS. Further, distributed groups using GSS and HDFD are able to be as effective as face-to-face (FtF) GSS groups. Our GSS-based methodology of HDFD can be embraced by practitioners to improve the creation of DFDs in FtF and distributed teams and it can likely also be extended to other systems design methodologies, such as structured walkthroughs, entity-relationship diagrams, use cases, and sequence diagrams.

59 More Specific Goals of Science
Create Causal Models for phenomena of interest (Theory) Test the usefulness of our models (Experiments and other methods) Use those models to increase the likelihood people will survive and thrive. (Applications)

60 Science – a Definition Science, ... organized systematic enterprise that gathers knowledge about the world and condenses the knowledge into testable laws and principles. Diagnostic features of science that distinguish it from pseudoscience are: 1. Repeatability: The same phenomenon is sought again, preferably by Independent investigation, and the interpretation given to it is confirmed or discarded by means of novel analysis and experimentation. 2. Economy: Scientists attempt to abstract the information into the form that is both simplest and aesthetically most pleasing the combination Called Elegance while yielding the largest amount of information with the least amount of effort. 3. Mensuration: If some thing can be properly measured, using Universally accepted scales, generalizations about it are rendered unambiguous. 4. Heuristics: The best science stimulates further discovery, often in unpredictable New directions; and the new knowledge provides an additional test of the original principles that led to its discovery. 5. Consilience: The explanations of different phenomena most likely to survive are those that can be connected and proved consistent with one another. Edward O. Wilson (1998) American Scientist, 86(1) Jan/Feb P.6. This paper proposes a new systems development methodology entitled heuristic dataflow diagrams (HDFDs). HDFD leverages group support systems (GSS) and heuristics to improve the creation of DFDs, which are frequently used design artifacts in systems development. We propose theory-based hypotheses to predict likely outcomes of using HDFD in non-GSS groups, GSS groups, and distributed GSS groups. These hypotheses were tested in a laboratory experiment using 123 subjects. The results indicate that HDFD performed with GSS provides process gains over HDFD performed without GSS. Further, distributed groups using GSS and HDFD are able to be as effective as face-to-face (FtF) GSS groups. Our GSS-based methodology of HDFD can be embraced by practitioners to improve the creation of DFDs in FtF and distributed teams and it can likely also be extended to other systems design methodologies, such as structured walkthroughs, entity-relationship diagrams, use cases, and sequence diagrams.

61 The Research Process Pick a research topic.
Formulate an appropriate research question related to that topic. How do you do this? Pick an outcome you think is interesting and ask, “What do think caused that outcome? and Why?”

62 The Research Process 3. Refine the research question by hypothesizing relationship(s) between the variables

63 The Research Process 4. Operationalize the variables
The conversion of abstract concept into concrete terms. Measurement -- how do we know anything happened?

64 The Research Process 5. Select an appropriate research technique
Examples: 1. Experiments 2. Quasi-experiments 3. Surveys 4. Interviews 5. Unobtrusive Data Collection 6. Content Analysis Case Studies Action Research Design Science Simulation

65 The Research Process 6. Collect data – Measure attributes of the real world. Classifying things that actually happen in the world with your operational scheme and then recording that data. Things to consider: 1. Quantitative vs. Qualitative 2. Primary vs. Secondary 3. Sample vs. Population Selection of Cases Validity

66 The Research Process 7. Analyze Data Look for systematic differences

67 The Research Process 8. Interpretation of the results
What did you find? How do your findings relate to other findings? What are the theoretical implications, how will this impact other IS research? What are the Practical implications, how will this impact IS practice?

68 Study of Research Methods Theory Building Theory Testing
Exploratory Research Initial Literature Review “Value Creation” RESEARCH OUT COMES Why study Value? Answer Weaknesses of current solutions Unsolved issues PHASE Pre-understanding Study of Research Methods Theory Building Theory Testing Evaluation of the research Point of Departure Research tools 8 Case Studies Counting M. Feedback : consultants &conferences Application of R. Methods Evolution of Frameworks Acceptability of Frameworks Development of: Workshop Definition of the Scope of Lit. Rev. Review of Selected Literature Identification of initial findings- “Gap” Gap Definition Initial Research Objective Review of: Scientific Paradigm Research strategies Techniques among others Understanding of research methods Definition of Phenomenological Research Approach Controls Criteria to evaluate results and the whole research Taking the gap further .. Development of: The Value Matrix The 3rd. Dimension VM Hard & Soft Value Footprints Gap Asses. Tool Research Questions Value Matrix Value cube Footprints Identification of: Contributions to Knowledge and Theory Contributions to Practice Limitations of Framework/ Footprint Validation of Frameworks Answer to RQ

69 “Thus, the task is not so much to see what no one has yet seen, but to think what no one has yet thought about that which everybody sees.” Arthur Schopenhauer

70 Science tests all hypotheses, but some scientists summarily dismiss opposing views:             Science has proven itself to be an infallible tool for unlocking certain areas of knowledge, but it's not logical to conclude from this that all thinking by scientists is infallible.                                                                       Science can be used to discover many things, yet some scientists wrongly presume that all things can be discovered through it.                                      

71 What is Science? A set of facts and the theories that explain the facts. Whatever’s being done by institutions carrying on “scientific” activity. Science = A particular approach, the scientific method

72 Data Data are concrete facts, records or collections of information we gather about phenomena of interest. Usually expressed in numerical terms Data may express facts of individuals (blood pressure, disease status, response to survey questions) or geopolitical areas (crime rate, death rate, per capita income)

73 Empiricism An empirical (observational) approach to research is one that strives to be objective. It expresses concepts in concrete, tangible ways. Not fuzzy or abstract ways… Empiricism tests relationships between these concepts. Empiricism is closely tied to the type of data that you use.

74 Limitations of Empirical Data
Our research cannot attempt to make value judgments Cannot answer questions such as, “What is morally right?” or “Which drug is better?” Can analyze people’s opinions about such things Some concepts are difficult to measure with numbers If we wanted to know if “happier people” lived longer, how could we measure happiness? Sometimes we just can’t get the data

75 Popular Fictions The goal of science is to accumulate facts
Science distorts reality and can’t do justice to the fullness of human experience. Scientific knowledge is truth. Science is concerned primarily with solving practical and social problems.

76 So then, what is Science? “Science is neither a philosophy nor a belief system. It is a combination of mental operations that has become increasingly the habit of educated peoples, a culture of illuminations hit upon by a fortunate turn of history that yielded the most effective way of learning about the real world ever conceived.” Edward O. Wilson Consilience: The Unity of Knowledge

77 Why do we see what we do and not see something else?
The Heart of the Matter Why do we see what we do and not see something else? Paradigm, Ontology, Epistemology, Axiology

78 Paradigm

79 Some Fundamental Paradigms
Positivist Research Interpretivist Research Criticalist Research Design Research Action Research

80 understand where you fit
Research Taxonomies It is important to understand where you fit in regards to research

81 Ontology Epistemology Methodology Methods Sources What’s out there to know? What and how can we know about it? How can we go about acquiring that knowledge? Which precise procedures can we use to acquire it? Which data can we collect? The interrelationship between the building blocks of research (Grix 2004: 66)

82 Philosophical Assumptions
Research Perspectives Basic Belief Positivist Interpretivist Design Study that describes the nature of reality: for example, what is real and what is not, what is fundamental and what is derivative? Single Reality; Knowable, Probabilistic Multiple Contextually Situated alternative World States; Socio-technologically enabled Multiple Realities; Socially Constructed Ontology Objective; Dispassionate. Detached observer of truth Study that explores the nature of knowledge: for example, on what does knowledge depend and how can we be certain of what we know? Subjective, i.e. values and knowledge emerge from the researcher-participant interaction Knowing through making: Objectively constrained construction within a context. Iterative circumscription reveals meaning Epistemology Strategy or plan of action; Research Design – shapes our choice of methods and links that choice to the research outcomes. Participative; Qualitative; Hermeneutical; Dialectical. Observational; Quantitative; Statistical Developmental; Measure artifactual impacts one the composite system Methodology Finally, the philosophical perspective of the design researcher changes as progress is iteratively made through the phases of Figure 3. In some sense it is as if the design researcher creates a reality through constructive intervention, then reflectively becomes a positivist observer, recording the behavior of the system and comparing it to the predictions (theory) set out during the abductive phase. The observations are interpreted, become the basis for new theorizing and a new abductive, interventionist cycle begins. In this sense design research is very similar to the action research methodology of the interpretive paradigm, however, the time frame of design research construction is enormously foreshortened relative to the social group interactions typical of action research. Study of values: what values does an individual or group hold and why? Truth: Universal and Beautiful; Predictive. Understanding: Situated and Descriptive Control; Creation; Progress (i.e. improvement) Understanding Axiology Expected Functionality and performance Useful and easy to use Solves Problem at hand Study of how we evaluate our investigations. Internal validity; construct validity; external validity and reliability. Credibility: triangulation multiple data sources Confirmability Criteriology

83 Behavioral vs. Design Science (Hevner, et al. 2004)
Behavioural Science Research (BSR) Design Science Research (DSR) Origin Natural Science Engineering, Sciences of the Artificial Paradigm Problem understanding paradigm Problem solving paradigm Objective develop and justify theories which explain or predict organizational human phenomena surrounding the analysis, design, implementation, management, and use of information Systems create innovations that define ideas, practices, technical capabilities, and product through the analysis, design, implementation, management, and use of information systems Object human-computer-interaction IT artefact design

84 IS Research Cycle Behavioral Science Research Understanding, Truth
Justify Build Design, IS Artifacts Theory Building, Hypotheses Evaluate/ Apply Theorize Utility, Usage in Practice Design Science Research

85 Reassembling the Dimensions
A given research project is a point in multidimensional space. Some regions of this space are popular: These often go together as Qualitative research. These often go together as Quantitative research. topic Biophysical Psychosocial Before After Theory Sample Case(s) scope Quantitative Qualitative method Observational Interventionist mode Objective Subjective ideology Neutral Partisan politics This pigeonholing doesn’t apply to the novelty, technology and utility dimensions.

86 Diagnosing your research paradigm H&H p
Diagnosing your research paradigm H&H p.73 (A if you agree/D if you disagree) Quantitative data is more scientific than qualitative data It is important to state the hypotheses before data collection Surveys are probably the best way to investigate business issues Unless a phenomenon can be measured reliably, it cannot be investigated A good knowledge of statistics is essential for all approaches to business research

87 Diagnosing your Research Paradigm H&H p.73
6. Case studies should only be used as a pilot project before the main research is conducted 7. Using participant observation to collect data is of little value in business research 8. Laboratory experiments should be used more widely in business research 9. It is impossible to generate theories during the course of research into business issues 10. Researchers must remain objective and independent from the phenomena they are studying

88 Diagnosing your Research Paradigm H&H p.73
To score, count the number of As and Ds: More As than Ds – Positivist More Ds than As – phenomenological/Interpretivist/Subjectivist Equal – flexible (Post-Positivist)

89 Science and Scientific Method
Science “the methodological and systematic approach to acquisition of new knowledge” (Geoffrey Marcyzk, David DeMatteo, David Festinger, Essentials of Research Design and Methodology, John Wiley & Sons, 2005, p. 4) Scientific method, evolved since 13th century, concerns set of tools, techniques and procedures used by researchers to analyze and understand phenomena and support or discard prior conceptions

90 Essence of the Scientific Method
Characteristics of the Scientific Method Objectivity Systematic Analysis Logical Interpretation of Results Elements of the Scientific Method Empirical Approach Observations Questions Hypotheses Experiments Analysis Conclusion Replication General Laws Basic Research Scientific Method Applied Research Information or Ideas for alternative Courses of action

91 4 Scientific Argument Types
Deduction: Conclusion is drawn from a set of propositions (pure logic) Induction: One draws general conclusions from particular facts that appear to serve as evidence Probability: Passes from frequencies within a known domain to conclusions of stated likelihood, Statistical: On the average, a certain percentage of a set of entities will satisfy the stated conditions.

92 4 Scientific Arguments types
Deductive Inductive Probabilistic Statistical Mathematics Computer Simulations Temporal Data Spectral Data Images Temporal Data Sets of Data Logical/Rational Thought Correlations/Patterns Likelihood Trends Laws of Nature Rules of Nature Generalities of Nature Predictions of Nature The fact that scientific reasoning is so often successful is a remarkable property of the Universe, the dependability of Nature.

93 Inductivisim vs. Deductivism
Exploratory Starts by observing, ends with a theory May be necessary to uncover relationships when little is known about a phenomena (i.e., AIDS in 1980’s) Confirmatory Starts with a theory, ends with test results Is considered the gold standard for conducting scientific research

94 Qualitative vs. Quantitative
Theories Quantitative Confirmation “Deductive” (X) f1 (X) (X) f2 (X) ………… (X) fn (X) Qualitative Exploration “Inductive” Manipulation Logic and Math Observations Results Theory Experiment 1.414 1.418 1.732 1.725 2.236 2.237 Reasonable Agreement Adapted from, Kuhn, Thomas (1961) "The Function of Measurement in Modern Physical Science." in The Essential Tension. (1977) Chicago, IL: University of Chicago Press, pp

95 Deductive Reasoning Theory Hypotheses Observation Confirmation
Deductive reasoning starts with a Given Theory Hypotheses as the basis for which we develop Hypotheses Observation and then acquire Specific Data through Observation or Experimentation and finally check to see if the data confirms (supports) our hypotheses and theory or not (Is our theory valid or not?) Confirmation

96 Inductive Reasoning Observation Pattern Tentative Hypothesis Theory
Inductive reasoning starts with a Specific Observation Pattern as the basis for which we develop a General Pattern Tentative Hypothesis and Tentative Hypothesis As foundation for a Theory Theory

97 Flow of Research: Top to Bottom Approach

98 To support these methods, a scientist also uses a large amount of skepticism to search for any fallacies in hypothesis or scientific arguments. Note that there is an emphasis on falsification, not verification. If a theory passes any test then our confidence in the theory is reinforced, but it is never proven correct in a mathematical sense. Thus, a powerful hypothesis is one that is highly vulnerable to falsification and that can be tested in many ways.

99 Falsification For a theory to be scientific, its hypotheses must be falsifiable The possibility must exist for the data to prove you wrong When collecting data one must not collect data simply to support one’s hypothesis This is essentially what an inductive approach does, as the hypothesis is based on the data Difference between science and philosophy / religion

100 Pluto…a planet or not? In a move that's already generating controversy and will force textbooks to be rewritten, Pluto will now be dubbed a dwarf planet. But it's no longer part of an exclusive club, since there are more than 40 of these dwarfs A clear majority of researchers voted for the new definition at a meeting of the International Astronomical Union (IAU) in Prague, in the Czech Republic. The IAU decides the official names of all celestial bodies. The tough decision comes after a multiyear search for a scientific definition of the word "planet." The term never had an official meaning before. What Is a Planet Today? According to the new definition, a full-fledged planet is an object that orbits the sun and is large enough to have become round due to the force of its own gravity. In addition, a planet has to dominate the neighborhood around its orbit. Pluto has been demoted because it does not dominate its neighborhood. Charon, its large "moon," is only about half the size of Pluto, while all the true planets are far larger than their moons.

101 Tying it all Together Use theory to develop research questions
Formulate specific, empirical hypotheses that are falsifiable Select variables based on theory that are empirically measurable Try to prove your hypothesis wrong Test the falsification (null) hypothesis State the limitations of your research

102 Theory and Research What is Theory?
“…the language that allows us to move from observation to observation and to make sense of similarities and differences.” Rudestam & Newton, 2001, p. 10.

103 Relationship between theory and research
Consider the Research Process Wheel as proposed by Rudestam & Newton in Surviving Your Dissertation (2001).

104 Generalized Wheel of Science
General/Abstract Theoretical Deduction Nomothetic Empirical Generalization Verify Confirm Evaluate Falsify Hypothesis Testing Idiographic Induction Specific/Concrete Empirical

105 Information Systems Research & Practice
Leads to Theory: ideas Practice: use of ideas Leads to after Checkland & Holwell (1998)

106 REALLY What is a Theory? (1)
Zikmund (p. 41) has defined a theory as “a coherent set of general propositions, used as principles of explanation of the amount of the apparent relationships of certain observed phenomona” Concepts (or constructs) are the basic building blocks of theory development. A concept (or construct) is a generalized idea about a class of objects, attributes, occurrences, or processes that have been given a name. A concept (or construct) may vary in terms of the level of abstraction THEY ARE PART OF THEORY Examples: Productivity, Leadership, Morale, Assets, Inflation

107 What is a Theory? (2) A Proposition is a statement concerned with the relationship between concepts. It asserts a universal connection and logical linkage between concepts. Propositions are at a higher level of abstraction than concepts. THEY ARE PART OF THEORY Example: Smoking is injurious to health Hypotheses are propositions which are empirically testable. They are usually concerned with the relationships between variables. THEY ARE NOT PART OF A THEORY. Example: Increasing salary by 10% will double the production

108 Theory Construction Baconian Inductivism (Interpretivist) Problems
Start with what we observe (data) Look for patterns among data Create theory based on observed patterns Empirically test predictions of the theory Problems Not all phenomena can be observed Depends on large number of observations

109 Theory Construction, continued
Hypothetico – Deductivism (Positivist) Propose theory Either a pre-existing theory or one that logically makes sense Generate testable hypothesis Collect empirical data Test hypothesis, interpret results Problems As theories become outdated, knowledge derived from this method may become meaningless


111 Observations of Objects, Events and Occurrences (Reality)
Abstraction Ladder Theory Abstract Level Levels of Abstraction Propositions Concepts / Constructs Observations of Objects, Events and Occurrences (Reality) Empirical Level

112 Theory and Research Theory functions three ways in research:
Theories prevent our being taken in by flukes. Theories make sense of observed patterns in ways that can suggest other possibilities. Theories can direct research efforts, pointing toward likely discoveries through empirical observation.

113 Theory as Explanation Research questions call for explanations
Answers or Explanations come from theorie

114 Vocabulary Concept: “a word or a symbol to represent an idea”

115 Vocabulary Theory: “concepts and their interrelationships”

116 Vocabulary Model: “imitation of an existing object”

117 Logics Theory World Model Denotation Approximation {True, False}
(supported or not) {Good, Fair, Poor}

118 Vocabulary Hypothesis: “testable statement based on theory”
Prediction about what Patterns we will see in the world if our theory is correct

119 Vocabulary Operational Definition: (variable)
“concept at a level that is testable” (measurable)

120 Operational Definition (variables)
concept concept concept concept concept concept concept concept concept THEORY Hypotheses Operational Definition (variables)

121 Theory Based on well established facts, testable hypotheses are formed. The process of testing "leads scientists to accord a special dignity to those hypotheses that accumulate substantial observational or experimental support." This "special dignity" is denoted by the granting of the title "theory," which, when it "explains a large and diverse body of facts" is considered "robust" and if it "consistently predicts new phenomena that are subsequently observed," it is "reliable."

122 What Theory is NOT 1. References are not theory.
2. Data are not theory 3. Lists of Variables or Constructs are Not Theory 4. Diagrams are Not Theory 5. Hypothesis (or predictions) are not theory 6. Theory is not something one "adds" to data, or something that one transforms from weaker to stronger by means of graphics or references, or can be feigned by flashy conceptual performance. Sutton, R. I. and Staw, B. M. (1995) What theory is not. ASQ 40: Weick, Karl, E. What Theory is Not, Theorizing Is, ASQ, 1995, 40: Parts of an Article That are not theory 1. References are not theory. Lots of references to existing theories does not create a new theory. Sometimes references are a smoke screen or are "throw-away" references. Authors need to explain which concepts and arguments are adopted from cited sources and how they are linked to the developed theory Data are not theory Data describe which empirical patterns were observed and theory explains why empirical patterns were observed. Data provides support but does not constitute a theory. Those who use qualitative data must develop causal arguments to explain why findings are observed if they want to include theory Lists of Variables or Constructs are Not Theory Theory is not conceptual definitions. Lists of variables that cover all possible determinants help explain but is not theory by themselves. Comparative tests of variables is not a comparative test of theory. They key issue is again WHY certain variables are more important. 4. Diagrams are Not Theory Diagrams can help explain how a phenomenon is created, but they again don't explain why. Good theory is often representational and verbal. With a strong theory one can discern when a major hypothesis is most or least likely to hold. 5. Hypothesis (or predictions) are not theory A theoretical model is not simply a statement of hypotheses. Hypotheses are statements about what is expected to occur, not why it is expected to occur. Predictions without logic are not theory. Stong theory papers have both simplicity and connectedness.

Myth 1: Hypotheses Become Theories Which Become Laws Myth 2: A Hypothesis is an Educated Guess Myth 3: A General and Universal Scientific Method Exists Myth 4: Evidence Accumulated Carefully Will Result in Sure Knowledge Myth 5: Science and its Methods Provide Absolute Proof Myth 6: Science Is Procedural More Than Creative Myth 7: Science and its Methods Can Answer All Questions Myth 8. Scientists are Particularly Objective Myth 9: Experiments are the Principle Route to Scientific Knowledge Myth 10: All Work in Science is Reviewed to Keep the Process Honest. See: Myth 1: Hypotheses Become Theories Which Become Laws This myth deals with the general belief that with increased evidence there is a developmental sequence through which scientific ideas pass on their way to final acceptance. Many believe that scientific ideas pass through the hypothesis and theory stages and finally mature as laws. A former U.S. president showed his misunderstanding of science by saying that he was not troubled by the idea of evolution because it was "just a theory." The president's misstatement is the essence of this myth; that an idea is not worthy of consideration until "lawness" has been bestowed upon it. For instance, Newton described the relationship of mass and distance to gravitational attraction between objects with such precision that we can use the law of gravity to plan spaceflights. During the Apollo 8 mission, astronaut Bill Anders responded to the question of who was flying the spacecraft by saying, "I think that Issac Newton is doing most of the driving fight now." (Chaikin, 1994, p. 127). His response was understood by all to mean that the capsule was simply following the basic laws of physics described by Isaac Newton years centuries earlier. The problem created by the false hierarchical nature inherent in this myth is that theories and laws are very different kinds of knowledge. Of course there is a relationship between laws and theories, but one simply does not become the other--no matter how much empirical evidence is amassed. Laws are generalizations, principles or patterns in nature and theories are the explanations of those generalizations (Rhodes & Schaible, 1989; Homer & Rubba, 1979; Campbell, 1953). The more thorny, and many would say more interesting, issue with respect to gravity is the explanation for why the law operates as it does. At this point, there is no well. accepted theory of gravity. Some physicists suggest that gravity waves are the correct explanation for the law of gravity, but with clear confirmation and consensus lacking, most feel that the theory of gravity still eludes science. Interestingly, Newton addressed the distinction between law and theory with respect to gravity. Although he had discovered the law of gravity, he refrained from speculating publically about its cause. In Principial, Newton states" I have not been able to discover the cause of those properties of gravity from phenomena, and I frame no hypothesis . . ." " it is enough that gravity does really exist, and act according to the laws which we have explained . . ." (Newton, 1720/1946, p. 547). The definition of the term hypothesis has taken on an almost mantra- like life of its own in science classes. If a hypothesis is always an educated guess as students typically assert, the question remains, "an educated guess about what?" The best answer for this question must be, that without a clear view of the context in which the term is used, it is impossible to tell. Myth 2: A Hypothesis is an Educated Guess The term hypothesis has at least three definitions, and for that reason, should be abandoned, or at least used with caution. For instance, when Newton said that he framed no hypothesis as to the cause of gravity he was saying that he had no speculation about an explanation of why the law of gravity operates as it does. In this case, Newton used the term hypothesis to represent an immature theory. As a solution to the hypothesis problem, Sonleitner (1989) suggested that tentative or trial laws be called generalizing hypotheses with provisional theories referred to as explanatory hypotheses. Another approach would be to abandon the word hypothesis altogether in favor of terms such as speculative law or speculative theory. With evidence, generalizing hypotheses may become laws and speculative theories become theories, but under no circumstances do theories become laws. Finally, when students are asked to propose a hypothesis during a laboratory experience, the term now means a prediction. As for those hypotheses that are really forecasts, perhaps they should simply be called what they are, predictions. The notion that a common series of steps is followed by all research scientists must be among the most pervasive myths of science given the appearance of such a list in the introductory chapters of many precollege science texts. This myth has been part of the folklore of school science ever since its proposal by statistician Karl Pearson (1937). The steps listed for the scientific method vary from text to text but usually include, a) define the problem, b) gather background information, c) form a hypothesis, d) make observations, e) test the hypothesis, and f) draw conclusions. Some texts conclude their list of the steps of the scientific method by listing communication of results as the final ingredient. Myth 3: A General and Universal Scientific Method Exists Philosophers of science who have studied scientists at work have shown that no research method is applied universally (Carey, 1994; Gibbs & Lawson, 1992; Chalmers, 1990; Gjertsen, 1989). The notion of a single scientific method is so pervasive it seems certain that many students must be disappointed when they discover that scientists do not have a framed copy of the steps of the scientific method posted high above each laboratory workbench. One of the reasons for the widespread belief in a general scientific method may be the way in which results are presented for publication in research journals. The standardized style makes it appear that scientists follow a standard research plan. Medawar (1990) reacted to the common style exhibited by research papers by calling the scientific paper a fraud since the final journal report rarely outlines the actual way in which the problem was investigated. Close inspection will reveal that scientists approach and solve problems with imagination, creativity, prior knowledge and perseverance. These, of course, are the same methods used by all problem-solvers. The lesson to be learned is that science is no different from other human endeavors when puzzles are investigated. Fortunately, this is one myth that may eventually be displaced since many newer texts are abandoning or augmenting the list in favor of discussions of methods of science. Induction was first formalized by Frances Bacon in the 17th century. In his book, Novum Organum (1620/ 1952), Bacon advised that facts be assimilated without bias to reach a conclusion. The method of induction he suggested is the principal way in which humans traditionally have produced generalizations that permit predictions. What then is the problem with induction? All investigators, including scientists, collect and interpret empirical evidence through the process called induction. This is a technique by which individual pieces of evidence are collected and examined until a law is discovered or a theory is invented. Useful as this technique is, even a preponderance of evidence does not guarantee the production of valid knowledge because of what is called the problem of induction. Myth 4: Evidence Accumulated Carefully Will Result in Sure Knowledge It is both impossible to make all observations pertaining to a given situation and illogical to secure all relevant facts for all time, past, present and future. However, only by making all relevant observations throughout all time, could one say that a final valid conclusion had been made. This is the problem of induction. On a personal level, this problem is of little consequence, but in science the problem is significant. Scientists formulate laws and theories that are supposed to hold true in all places and for all time but the problem of induction makes such a guarantee impossible. The nature of induction itself is another interesting aspect associated with this myth. If we set aside the problem of induction momentarily, there is still the issue of how scientists make the final leap from the mass of evidence to the conclusion. In an idealized view of induction, the accumulated evidence will simply result in the production of a new law or theory in a procedural or mechanical fashion. In reality, there is no such method. The issue is far more complex -- and interesting --than that. The final creative leap from evidence to scientific knowledge is the focus of another myth of science. The proposal of a new law begins through induction as facts are heaped upon other relevant facts. Deduction is useful in checking the validity of a law. For example, if we postulate that all swans are white, we can evaluate the law by predicting that the next swan found will also be white. If it is, the law is supported, but not proved as will be seen in the discussion of another science myth. Locating even a single black swan will cause the law to be called into question. The problem of induction argues against proof in science, but there is another element of this myth worth exploring. In actuality, the only truly conclusive knowledge produced by science results when a notion is falsified. What this means is that no matter what scientific idea is considered, once evidence begins to accumulate, at least we know that the notion is untrue. Consider the example of the white swans discussed earlier. One could search the world and see only white swans, and arrive at the generalization that "all swans are white. " However, the discovery of one black swan has the potential to overturn, or at least result in modifications of, this proposed law of nature. However, whether scientists routinely try to falsify their notions and how much contrary evidence it takes for a scientist's mind to change are issues worth exploring. The general success of the scientific endeavor suggests that its products must be valid. However, a hallmark of scientific knowledge is that it is subject to revision when new information is presented. Tentativeness is one of the points that differentiates science from other forms of knowledge. Accumulated evidence can provide support, validation and substantiation for a law or theory, but will never prove those laws and theories to be true. This idea has been addressed by Homer and Rubba (1978) and Lopnshinsky (1993). Myth 5: Science and its Methods Provide Absolute Proof We accept that no single guaranteed method of science can account for the success of science, but realize that induction, the collection and interpretation of individual facts providing the raw materials for laws and theories, is at the foundation of most scientific endeavors. This awareness brings with it a paradox. If induction itself is not a guaranteed method for arriving at conclusions, how do scientists develop useful laws and theories? Myth 6: Science Is Procedural More Than Creative Induction makes use of individual facts that are collected, analyzed and examined. Some observers may perceive a pattern in these data and propose a law in response, but there is no logical or procedural method by which the pattern is suggested. With a theory, the issue is much the same. Only the creativity of the individual scientist permits the discovery of laws and the invention of theories. If there truly was a single scientific method, two individuals with the same expertise could review the same facts and reach identical conclusions. There is no guarantee of this because the range and nature of creativity is a personal attribute. Unfortunately, many common science teaching orientations and methods serve to work against the creative element in science. The majority of laboratory exercises, for instance, are verification activities. The teacher discusses what will happen in the laboratory, the manual provides step-by-step directions, and the student is expected to arrive at a particular answer. Not only is this approach the antithesis of the way in which science actually operates, but such a portrayal must seem dry, clinical and uninteresting to many students. In her book, They're Not Dumb, They're Different (1990) Shiela Tobias argues that many capable and clever students reject science as a career because they are not given an opportunity to see it as an exciting and creative pursuit. The moral in Tobias' thesis is that science itself may be impoverished when students who feel a need for a creative outlet eliminate it as a potential career because of the way it is taught. Philosophers of science have found it useful to refer to the work of Karl Popper (1968) and his principle of falsifiability to provide an operational definition of science. Popper believed that only those ideas that are potentially falsifiable are scientific ideas. Myth 7: Science and its Methods Can Answer All Questions. For instance, the law of gravity states that more massive objects exert a stronger gravitational attraction than do objects with less mass when distance is held constant. This is a scientific law because it could be falsified if newly-discovered objects operate differently with respect to gravitational attraction. In contrast, the core idea among creationists is that species were placed on earth fully-formed by some supernatural entity. Obviously, there is no scientific method by which such a belief could be shown to be false. Since this special creation view is impossible to falsify, it is not science at all and the term creation science is an oxymoron. Creation science is a religious belief and as such, does not require that it be falsifiable. Hundreds of years ago thoughtful theologians and scientists carved out their spheres of influence and have since coexisted with little acrimony. Today, only those who fail to understand the distinction between science and religion confuse the rules, roles, and limitations of these two important world views. It should now be clear that some questions simply must not be asked of scientists. During a recent creation science trial for instance, Nobel laureates were asked to sign a statement about the nature of science to provide some guidance to the court. These famous scientists responded resoundingly to support such a statement; after all they were experts in the realm of science (Klayman, Slocombe, Lehman, & Kaufman, 1986). Later, those interested in citing expert opinion in the abortion debate asked scientists to issue a statement regarding their feelings on this issue. Wisely, few participated. Science cannot answer the moral and ethical questions engendered by the matter of abortion. Of course, scientists as individuals have personal opinions about many issues, but as a group, they must remain silent if those issues are outside the realm of scientific inquiry. Science simply cannot address moral, ethical, aesthetic, social and metaphysical questions. Myth 8. Scientists are Particularly Objective Another aspect of the inability of scientists to be objective is found in theory-laden observation, a psychological notion (Hodson, 1986). Scientists, like all observers, hold a myriad of preconceptions and biases about the way the world operates. These notions, held in the subconscious, affect everyone's ability to make observations. It is impossible to collect and interpret facts without any bias. There have been countless cases in the history of science in which scientists have failed to include particular observations in their final analyses of phenomena. This occurs, not because of fraud or deceit, but because of the prior knowledge possessed by the individual. Certain facts either were not seen at all or were deemed unimportant based on the scientists's prior knowledge. In earlier discussions of induction, we postulated that two individuals reviewing the same data would not be expected to reach the same conclusions. Not only does individual creativity play a role, but the issue of personal theory-laden observation further complicates the situation. Many philosophers of science support Popper's (1963) view that science can advance only through a string of what he called conjectures and refutations. In other words, scientists should propose laws and theories as conjectures and then actively work to disprove or refute those ideas. Popper suggests that the absence of contrary evidence, demonstrated through an active program of refutation, will provide the best support available. It may seem like a strange way of thinking about verification, but the absence of disproof is considered support. There is one major problem with the idea of conjecture and refutation. Popper seems to have proposed it as a recommendation for scientists, not as a description of what scientists do. From a philosophical perspective the idea is sound, but there are no indications that scientists actively practice programs to search for disconfirming evidence. Scientists are no different in their level of objectivity than are other professionals. They are careful in the analysis of evidence and in the procedures applied to arrive at conclusions. With this admission, it may seem that this myth is valid, but contributions from both the philosophy of science and psychology reveal that there are at least three major reasons that make complete objectivity impossible. Related to the issue of theory-based observations is the allegiance to the paradigm. Thomas Kuhn (1970), in his ground-breaking analysis of the history of science, shows that scientists work within a research tradition called a paradigm. This research tradition, shared by those working in a given discipline, provides clues to the questions worth investigating, dictates what evidence is admissible and prescribes the tests and techniques that are reasonable. Although the paradigm provides direction to the research it may also stifle or limit investigation. Anything that confines the research endeavor necessarily limits objectivity. While there is no conscious desire on the part of scientists to limit discussion, it is likely that some new ideas in science are rejected because of the paradigm issue. When research reports are submitted for publication they are reviewed by other members of the discipline. Ideas from outside the paradigm are liable to be eliminated from consideration as crackpot or poor science and thus do not appear in print. This lesson has clear implications for science teaching. Teachers typically provide learning experiences for students without considering their prior knowledge. In the laboratory, for instance, students are asked to perform activities, make observations and then form conclusions. There is an expectation that the conclusions formed will be both self-evident and uniform. In other words, teachers anticipate that the data will lead all pupils to the same conclusion. This could only happen if each student had the same exact prior conceptions and made and evaluated observations using identical schemes. This does not happen in science nor does it occur in the science classroom. Examples of scientific ideas that were originally rejected because they fell outside the accepted paradigm include the sun-centered solar system, warm-bloodedness in dinosaurs, the germ-theory of disease, and continental drift. When first proposed early in this century by Alfred Wegener, the idea of moving continents, for example, was vigorously rejected. Scientists were not ready to embrace a notion so contrary to the traditional teachings of their discipline. Continental drift was finally accepted in the 1960s with the proposal of a mechanism or theory to explain how continental plates move (Hallam, 1975 and Menard, 1986). This fundamental change in the earth sciences, called a revolution by Kuhn, might have occurred decades earlier had it not been for the strength of the paradigm. Throughout their school science careers, students are encouraged to associate science with experimentation. Virtually all hands-on experiences that students have in science class is called experiments even if it would be more accurate to refer to these exercises as technical procedures, explorations or activities. True experiments involve carefully orchestrated procedures along with control and test groups usually with the goal of establishing a cause and effect relationship. Of course, true experimentation is a useful tool in science, but is not the sole route to knowledge. Myth 9: Experiments are the Principle Route to Scientific Knowledge It would be unwise to conclude a discussion of scientific paradigms on a negative note. Although the examples provided do show the contrary aspects associated with paradigm-fixity, Kuhn would argue that the blinders created by allegiance to the paradigm help keep scientists on track. His review of the history of science demonstrates that paradigms are responsible for far more successes in science than delays. Charles Darwin punctuated his career with an investigatory regime more similar to qualitative techniques used in the social sciences than the experimental techniques commonly associated with the natural sciences. For his most revolutionary discoveries, Darwin recorded his extensive observations in notebooks annotated by speculations and thoughts about those observations. Although Darwin supported the inductive method proposed by Bacon, he was aware that observation without speculation or prior understanding was both ineffective and impossible. The techniques advanced by Darwin have been widely used by scientists Goodall and Nossey in their primate studies. Scientific knowledge is gained in a variety of ways including observation, analysis, speculation, library investigation and experimentation. Many note-worthy scientists have used non-experimental techniques to advance knowledge. In fact, in a number of science disciplines, true experimentation is not possible because of the inability to control variables. Many fundamental discoveries in astronomy are based on extensive observations rather than experiments. Copernicus and Kepler changed our view of the solar system using observational evidence derived from lengthy and detailed observations frequently contributed by other scientists, but neither performed experiments. The result of the lack of oversight has recently put science itself under suspicion. With the pressures of academic tenure, personal competition and funding, it is not surprising that instances of outright scientific fraud do occur. However, even without fraud, the enormous amount of original scientific research published, and the pressure to produce new information rather than reproduce others' work dramatically increases the chance that errors will go unnoticed. Frequently, the final step in the traditional scientific method is that researchers communicate their results so that others may learn from and evaluate their research. When completing laboratory reports, students are frequently told to present their methods section so clearly that others could repeat the activity. The conclusion that students will likely draw from this request is that professional scientists are also constantly reviewing each other's experiments to check up on each other. Unfortunately, while such a check and balance system would be useful, the number of findings from one scientist checked by others is vanishingly small In reality, most scientists are simply too busy and research funds too limited for this type of review. Myth 10: All Work in Science is Reviewed to Keep the Process Honest. An interesting corollary to this myth is that scientists rarely report valid, but negative results. While this is understandable given the space limitations in scientific journals, the failure to report what did not work is a problem. Only when those working in a particular scientific discipline have access to all of the information regarding a phenomenon -- both positive and negative -- can the discipline progress.

124 “I grant you evolution was a theory to begin with… but it evolved into a fact a long time ago!“

125 Hypotheses; Theories; Laws Observations Develop Hypothesis
To Explain Observations Pass Test Hypothesis Fail Pass Many Tests Hypothesis Theory Pass Test Theory Fail Pass Many Tests Theory Law Fail

126 Phenomenon of Interest? Collections of Data
Observations by other scientists Inductive Reasoning applied (Specific to General) to develop General hypotheses Hypotheses Experiments designed Through deductive reasoning (General to Specific) Small dots represent observations Large dots represent Experimental Results Controlled experiments provide new data that is tested statistically for significance and falsification Theory Scientific Law Results of statistical tests on new data Add evidence to support, modify or falsify the theory (or more rarely the scientific law) Scientific Law Scientific Law Scientific Theories are relatively large, general concepts. Scientific Laws are smaller, mathematically precise concepts.

127 Scientific Law Theory C Theory B Scientific Law Theory A

128 Theory A Causal Model of the phenomenon-of-interest
Drives all subsequent Scientific Activity Hypotheses Experimental/Research Design Measures Analysis Conclusions Interpretations Limitations

129 Anything Missing? Truth

130 Positivist Perspective
Science <> True Science = Useful

131 A Useful Model is often Better Than Truth

132 Nor need it be to be useful
No scientific inquiry is ever complete, and no scientific theory is ever “final” Nor need it be to be useful A scientific theory in its current state can be very useful in the present even though it may later be or improved upon or even superceded

133 “I think you should be more explicit here in Step Two.”
Connecting outputs to outcomes is a challenge “I think you should be more explicit here in Step Two.” My Theory

134 Useful Is Better Than True

135 Name the Phenomenon Nickezite Block
Tie in a Who Cares argument to this story. Tie bobozite blocks to survive and thrive.

136 Describe the Phenomenon
A Nickezite Block B

137 Explore the Phenomenon
A Bobezite Block B

138 Explore the Phenomenon

139 Describe Phenomenon Dynamics
Nickezite Block B

140 A Useful Theory One Gear

141 Truth (Reality) Many Gears Belts and pulleys

142 The model becomes useful when you want to do something new

143 Therefore For matters of cause-and-effect A useful model (Theory)
Is/Can Be better than Truth (Reality)

144 What is a Theory? To this add “Theoretical Rationale”
“A set of interrelated constructs (variables), definitions, and propositions that presents a systematic view of phenomena by specifying relations among constructs, with the purpose of explaining natural phenomena.” Kerlinger, F. N. (1979) Behavioral Research: A conceptual Approach. New York, NY, USA: Holt, Rinehart and Winston. To this add “Theoretical Rationale” “Specifying how and why the constructs and relational statement are interrelated.” Labovitz and Hagedorn (1971) Introduction to Social Research. New York, NY, USA. McGraw-Hill.

145 A Good Theory Should explain existing data
Explain a range of related observations Allow statements to be made about the world Allow predictions about the future Have meaningful implications Taken from Davey et al. (2004)

146 What is a Theory? Causal Model Internally Consistent
Explains and/or predicts Proposes mechanisms of causation Testable

147 Structure of a Theory Axioms (Assumptions)
Propositions (Causes and Effects)

148 Substruction A strategy to help you understand the theory and methods (operational system) in a research study Applies to empirical, quantitative research studies There is no word, Substruction, in the dictionary. It has an inductive meaning, constructing and a deductive meaning, deconstructing Heuristic

149 Scaling/Data analysis
Substruction Theory (Theoretical system) Construct Concept Deductive (Qualitative) Methods (Operational System) Measures Scaling/Data analysis (Quantitative) Inductive

150 Substruction: Building Blocks or Statements of Relationships
Theoretical Model Construct Pain Axiom Quality of Life Concept Intensity Proposition Functional status Measure 10 cm scale Hypothesis mobility scale Measurement Model

151 Statements of Relationships
Construct: Postulate: Statement of relationship between a construct and concepts Pain consists of three concepts Concepts: Intensity Location Duration

152 Basic Concepts Hypothesis
States a relationship between two, or more, concepts and suggests that one has an impact on the other (Grix 2004:42) An Hypothesis is a provisional idea whose merit is to be evaluated. A hypothesis requires more work by the researcher in order to either confirm or disprove it. In the hypothetico-deductive method, a hypothesis should be falsifiable, meaning that it is possible that it be shown false, usually by observation. Note that, if confirmed, the hypothesis is not necessarily proven, but remains provisional. (Wikipedia)

153 Basic Concepts Propositions
Functional Statements of cause-and-effect that must be logically true if the axioms are true Examples P1: Effort toward group goal is a function of goal congruence P2. Group Productivity is an inverse function of distraction

154 Propositions must be... Causal Composed of Constructs
Without empirical content Logically derivable from axioms

155 Propositions of Direct Causation
+ + Goal Congruence Effort Productivity 2 1 - 3 Distraction Proposition 1: Productivity is a function of effort Proposition 2: Effort is a function of goal congruence Proposition 3: Effort is an inverse function of distraction

156 Theory Explication Example: What determines musical Taste?

157 Important Terms Theory Concept Variable Independent Dependent
Antecedent Intervening Mediating

158 The Simplest Diagram of a Theory
Independent Concept (Construct) Dependent Concept (Construct) Relationship

159 A Simple Theory Peer Group Musical Taste Influences

160 Adding an Antecedent Construct
Socioeconomic status Affects Peer Group Musical Taste Influences

161 Adding a Mediating Construct
Socioeconomic status Gender Affects Peer Group Impacts Musical Taste Influences

162 Adding an Intervening Construct
Socioeconomic status Gender Affects Peer Group Musical Taste Impacts Self image Leads to Influences

163 A Third Construct Explanation
Parental influence X Peer group Musical taste

164 Deriving Theory and Hypotheses
Variables Variables Empirical test of Theory Constructs Constructs Propositions Theory Logical Foundation for Propositions Axioms and Postulates Derived in part from Bacharach (1989).

165 Frequency of use H1a (-) H1b (-) Duration of use H1c (-)
“Eight-item scale adapted from Langfred” Score H1b (-) Duration of use H1c (-) Measurement Model (Empirical test of Theory) Percentage of system features used regularly H1d (-) Proportion of use Degree of Freedom P1 (-) Usage Intensity P4 (-) Independence P2 (-) Theoretical Model P5 (-) Usage Scope P3(-) Discretion P6 (-) Team Autonomy culture Axiom 1 (-) Individual Usage

166 Implicit Coordination Theory (ICT) Causal Process Model
Theoretical Assumptions Supported Constructs, Postulates and Axioms Theoretical Proposals Proposed Constructs and Propositions Parallelism Group Awareness A1(+) Response Bias P1 (+) P6(+) P7(+) A2(+) Shared Cognition Task Performance A3(+) P3(+/-) Implicit Coordination Self-Scribing Ability Formalized Group Memory P2 (+) P4 (+) P5 (+) A4(+) Heedful Interrelating H1(+); H5(+) SDT Givens Paul – in one of the earlier versions of the paper you wrote the following comment: “We can further frame parallelism, group memory, group awareness, and anonymity in terms of media richness theory (see ARS theory). However, we need to figure out how to apply signal-detection theory first and build everything around signal-detection theory.” That is what I am trying to do here. To focus things around SDT; then to try to bring in A and P; can you help with this? You musthave had somethingin mind here. Response Criterion Hit Rate (Detected Errors/ Total Errors) Shared Interface H4(+); H8a(+) (-) Discriminability (Sensitivity) False Alarm Rate (Non-Errors identified as errors/ Total Non-Errors) H3(+); H7(+) (-) H2(-); H6(-) Experimental Control : Dependent Variable Measurement Model : Independent Variables

167 Qualities of a Good Theory
Parsimony ( simple, small ) Explanatory/Predictive Bounded

168 “If you can't explain it simply, you don't understand it well enough.”
Albert Einstein

169 Pragmatic Theory Usually start with propositions and work backward to axioms Usually start poorly and get better Use someone else’s theory whenever you can Technology has No Place in your theory (if you tie technology to your theory, what will happen when technology changes?)

170 Pragmatic Theory A good theory will get you to the moon and back safely on the first try Good theory will do more to save you from drawing “bone-headed” conclusions than any other discipline of positivism Good theory will make you look like a genius

171 Explanation in Science
CAUSAL EXPLANATION Common variation The cause X and effect Y should vary together Order X precedes Y Third Factors The common variation of X and Y should not be due to a third factor Z Empirical Connection The connection between X and Y is empirical Theory The connection between X and Y should be deduced from a general theory Mechanism The mechanism that connects X to Y should be known

172 Scientific Method (Deductive)
Phenomenon Observe Record Analysis (Previous Theory) Synthesize (Cause -> Effect) Peculiar? Hypotheses Construct Theory Unify/Simplify Understand Underlying Domain

173 Scientific Method (more detail)
Interest Idea Theory Choice of Research Method Experiments Surveys Field Study Content Analysis Secondary data analysis Comparative Evaluation Design Population and Sampling Whom do we want to be able to draw conclusions about? Who will be observed for that purpose Conceptualization Specify the meaning Of the concepts and Variables to be studied Operationalization How will we actually measure the variables under study? Observations Collecting data for Analysis and interpretation Data Processing Transforming the data collected into a form appropriate to manipulation and analysis Scientific Method (more detail) Analysis Analyzing data and Drawing conclusions Application Reporting results and assessing their implications

174 Cycle of Research and Theory-Building
People notice phenomena. They gather information about the phenomena. The theory is strengthened. Yes They build a theory which explains and predicts it. People use the theory to write hypotheses. They share their theory with other people. People conduct studies. Are the hypotheses supported? No The theory is weakened. The researchers may suggest modifications.

175 Cycle of Research and Theory-Building Detailed
Worldview Theoretical Framework (Conceptual Scheme, Principles, mode of representation, Template) Empirical Methods Theoretical Methods Problem Body of Data Theoretical Model Comparison Possible Actions: Revise Model Reassess data Redefine Problem Reconsider Empirical Methods Review Theoretical Methods Reconstruct Framework Rethink Worldview

176 An Experiment without a theory is meaningless

177 Large, Odd-Smelling Boxes
Phenomena: Large, Odd-Smelling Boxes Put a who-cares on this.

178 Scientific Instrument:

179 Collecting Data Without A Theory

180 Collecting Data With a Theory

181 A Physicist Uses the ‘Elephant’ Theory
+ = Fission!

182 A Farmer Uses the ‘Elephant’ Theory

183 There is nothing more useful than A Good Theory

184 So, what is “Science”? Theory must be founded on natural laws.
Theory must be falsifiable. Theory must produce hypotheses that are corroborated by evidence. Disconfirmation is “overblown.” Most research progresses by solving puzzles using the ideas within the hard core of a research program. Rewards go to those who solve particularly hard puzzles.

185 Conditions of Science continued
Predictions of new facts that are then corroborated by evidence is the ideal. Scientific revolutions or paradigm shifts are rare. Challenging or amending the hard core is not what science is usually about. Changing “how we think about the universe” occurs at several levels, from resolving particularly difficult puzzles to developing a new paradigm.

186 Lessons for Your Research
Since science is a social enterprise, your work counts only by how it is received by the scientific community. This reception is partly subjective. You must argue your case in the face of sometimes conflicting and ambiguous criteria. Even though we may agree on the conditions that make a theory “better,” we can still disagree and, therefore, argue over which particular theory best fits those conditions. ( and still respect each other’s work and be friends.)

187 Valid Scientific Arguments
You are solving a genuine and significant puzzle within the field. The evidence corroborates your theory and hypotheses. This is a question of research design. The better your research design, the stronger your argument will be. Your amendment to the hard core is progressive. Resolving the puzzle uncovers additional implications that are also empirically corroborated.

188 Scientific Arguments continued
Since multiple theories may exist in the protective belt or positive heuristic, your theory is more elegant, broader in the range of phenomena its predicts/explains, and supported better by the evidence than its plausible rivals. In rare cases, you have sufficiently altered the hard core that you have created a new research program.

189 Consequences for Information Growth
AYER - LOGICAL POSITIVISM Theories confirmed & areas sewn up New areas investigated Less to investigate in each generation End of Science!

190 Consequences for Information Growth
POPPER - FALSIFICATION Theories not disproved All results contingent Each generation re-investigates results Exponential growth of science Philosophical underpinnings of science drive it forward and predispose it to exponential growth

191 Merton’s Norms Merton’s social norms of scientific conduct
Universalism: new work is assessed by universal impersonal criteria Communality: scientific knowledge should be common property Disinterestedness: prime concern is the advancement of knowledge Organized scepticism: knowledge should be continually subjected to critical scrutiny Reflects stated values rather than actual behaviour: what they do is not what they say. See Watson’s The Double Helix, for example

192 IS Research Methods

193 Doctoral Seminar – MSIS 6333 Wednesday August 21st, 2009
Bio-psycho-socio-cultureal disciplines and theories Positivism Focus on generalizeability and Causal Explanation Interpretativm Focus on relativism and understanding Epistemological Paradigms Addressing Research Questions (explanation or understanding) Hypothesis Testing (Aiming to establish, explain, Predict causal links Between key variables Methodologies/ Research Strategies Hermeneutic Inquiry (‘Thick description’ And in-depth Understanding) Survey Experiment Questionnaire Structured Observation Quantitative Analysis Case Study Discourse analysis Life History Ethnography phenomenology Participant Observation Focus Group Interviewing Grounded theory Action Research Methods (data collection And analysis) Doctoral Seminar – MSIS 6333 Wednesday August 21st, 2009 Nicholas C. Romano, Jr. Oklahoma State University Methodological field

194 The Research Process Onion
Positivism Research Philosophy Deductive Experiment Research Approaches Survey Sampling Secondary data Observation Interviews Questionnaires Cross sectional Case study Research Strategies Grounded theory Realism Action research Longitudinal Time Horizons Ethnography Inductive Data Collection Methods interpretivism

195 Alternative Motivations for Research
• ‘Pure Research' ‘because it’s there’ contribute to abstract, theoretical understanding • ‘Applied Research' ‘I have hammer, so find a nail’ • Instrumentalist Research ‘I see a problem, so find a solution’

196 Basic (pure) Applied Research Type Purpose Context Methods
Expand Knowledge Academic Setting Single Researcher Less Time/Cost Pressure Internal Validity Cause Single Level of Analysis Single Method Experimental Direct Observations Understand Specific Problem Real-World Setting Multiple Researchers More Time/Cost Pressure External Validity Effect Multiple Levels of Analysis Multiple Methods Quasi-Experimental Indirect Observations

197 Nature of Research Outcomes
• Descriptive Depiction of a behavior or a domain • Explanatory Systemic explanation of how behaviors arise ascription of causes to occurrences in the domain • Predictive Statement of: what behavior will arise, and how; what occurrences will arise within the domain; what effect will particular interventions have • Normative Declaration of interventions to a desired outcome

198 Three Types of Research
Descriptive research – finding out (What, Where, When) Explanatory research – explaining ; identifying causality; theory/model; prediction (HOW/WHY) Evaluative research – evaluation of strategies, policies, programs, practices (Value)

199 Types of Research Spectrum
To become familiar with phenomena; to gain new insights; to formulate a more specific research problem or research hypothesis. To portray accurately the incidence, distribution, and characteristics of a group or situation. (Usually not begun with specific hypothesis.) To investigate relationships between variables. (Begins with specific hypotheses.) To test hypotheses of causal relationships between variables. (Begins with specific hypotheses.) (Explore) (Describe) (Explain - Predict) (Control) Descriptive Research Correlational/Ex Post Facto Survey Research Exp./Quasi-Exp. Independent Variables Independent Var. (X) controlled by investigator Independent variables (X) not controlled by investigator

200 Research generates knowledge in order to:
Action Change Within A System Emancipate Through Action Expose And Change The Dominate System Build Broader Understanding Pave The Way For Change Basic Or Pure Research Applied / Evaluative Research Action Research Critical / Radical Ethnography Technical/ Practical Participatory/ Emancipatory

201 Research Approaches Two main classes of approaches:
Theory ‘testing’ – apply theory to ‘read’ the data Theory ‘emergent’ – look for ‘patterns’, understanding emerges from the data (Hirschheim, 2002)

202 Combining Approaches Case study/action research leads to
Research question Theory building Theory testing with lab. experiments and Theory testing with field experiments Theory extension and feedback loop to Theory testing

203 Research Approaches Mathematical approaches
Approaches studying reality Research stressing what is reality Conceptual-analytical approaches approaches for empirical studies theory-testing approaches theory-creating approaches Research stressing utility of artifacts artifact-building approaches artifacts-evaluating approaches (Järvinen & Järvinen, 1999)

204 Alternative Research Methods
Constructivist Methods (“Design”) conceptual development and technical development Nomothetic Methods (“Confirmatory”) field research, surveys, lab experiments … using the hypothetico-deductive method Idiographic Methods (“Exploratory”) case studies and action research (Hirschheim, 2002)

205 Research Methods Non-Empirical Techniques
Scientific Research Techniques Interpretivist Research Techniques Research Techniques at the Scientific/Interpretivist Boundary Engineering Research Techniques (Design Fits here as well)

206 Research Methods Non-empirical Techniques
The following techniques are detached from real-world data. This is not to say that they are necessarily totally remote or irrelevant, but rather that they are once-removed, depending on synthetic data, or on conceptual thinking about abstractions. The primary techniques are: Conceptual research. This is based on opinion and speculation, and comprises philosophical or 'armchair' analysis, and argumentative/dialectic analysis; theorem proof. This applies formal methods to mathematical abstractions, in order to demonstrate that, within a tightly defined model, a specific relationship exists among elements of that model; simulation. This is the study of a simplified, formal model of a complex environment, in order to perform experimentation not possible in a real-world setting; futures research, scenario-building, and game- or role-playing. Individuals interact in order to generate new ideas or gather new insights into relationships among variables. A specific instance that is often applied in the information systems discipline is the delphi technique (Delbecq et al., 1975); review of existing literature, or 'meta-analysis'. The literature examined in such research may include the opinions and speculations of theorists, the research methods adopted by empirical researchers, the reports of the outcomes of empirical research, and materials prepared for purposes other than research.

207 Research Methods Scientific Research Techniques
The following are common techniques that can be applied by information systems researchers within the scientific tradition: Forecasting. This technique involves the application of regression and time-series techniques, in order to extrapolate trends from past data; Field experimentation and quasi-experimental designs. Opportunities are sought in the real-world which enable many factors, which would otherwise confound the results, to be isolated, or controlled for (Cook & Campbell 1979); Laboratory experimentation. This involves the creation of an artificial environment, in order to isolate and control for potentially confounding variables (Hersen & Barlow 1976, Jarvenpaa et al. 1984, 1985, Jarvenpaa 1988, Benbasat 1990a, 1990b, DeSanctis 1990).

208 Research Methods Interpretivist Research Techniques
The following are techniques which are unequivocally interpretivist in their style: descriptive/interpretive research. In this techniques, empirical observation is subjected to limited formal rigour. Controls over the researcher's intuition include self-examination of the researcher's own pre-suppositions and biases, cycles of additional data collection and analysis, and peer review; focus group research. This involves the gathering of a group of people, commonly members of the public affected by a technology or application, to discuss a topic. Its purpose is to surface aspects, impacts and implications that are of concern. See Stewart & Shamdasani (1990) and Clarke (1999); action research. The researcher plays an active role in the object of study, e.g. by acting as a change-agent in relation to the process being researched. See Clark (1972), Susman & Evered (1978), Mansell (1991), Stringer (1996, 1999), Myers (1997a) and Baskerville & Wood-Harper (1998); ethnographic research. This technique applies insights from social and cultural anthropology to the direct observation of behaviour. See Harvey & Myers (1995) and Myers (1997a); grounded theory. This is a specific technique that it is claimed enables the disciplined extraction of a theory-based description of behaviour, based on empirical observations. See Glaser & Strauss (1967), Strauss & Corbin (1990) and Myers (1997a).

209 Research Methods Research Techniques at the Scientific/Interpretivist Boundary
Several techniques can be applied within either a scientific or an interpretivist context. field study. The object of study is subjected to direct observation by the researcher (Klein & Myers 1999); questionnaire-based survey. This involves the collection of written data from interviewees, or the collection of verbal responses to relatively structured questions. See Straub (1989), Kraemer (1991), Kraemer & Dutton (1991), Pinsonneault & Kraemer (1993), and Newsted et al. (1998); interview-based survey. This involves the recording of verbal data from interviewees, which arises in relatively unstructured interviews or meetings; case study. This involves the collection of considerable detail, from multiple sources, about a particular, contemporary phenomenon within its real-world setting. For guidance on the use of case studies within the scientific tradition, see Yin (1984, 1994), Benbasat et al. (1987) and Lee (1989); and for guidance on their use in an interpretivist manner, see Walsham (1995b) and Myers (1997b); secondary research. Rather than producing new data, this technique analyses the contents of existing documents. Commonly, this is data gathered by one or more prior researchers, and it is re-examined in the light of a different theoretical framework from that previously used. The documents may also include materials prepared for purposes other than scientific research.

210 Research Methods Engineering Research Techniques
Information systems research conducted within the computer science and engineering context uses two categories of research technique: construction. This approach involves the conception, design and creation (or 'prototyping') of an information technology artefact and/or technique (most commonly a computer program, but sometimes a physical device or a method). The new technology is designed to intervene in some setting, or to enable some function to be performed, or some aim to be realised. The design is usually based upon a body of theory, and the technology is usually subjected to some form of testing, in order to establish the extent to which it (and, by implication, the class of technologies to which it belongs) achieves its aims; destruction. In this case, new information is generated concerning the characteristics of an existing class of technologies. This is typically achieved through testing the technology, or applying it in new ways. The design is usually based upon a body of theory.

211 Research Types and Methods
Type/ Method Tests, Measurements Interviews Observations Surveys Documents Experimental P A Quasi-experimental Causal-comparative Correlational Descriptive Evaluation Ethnographic Action Case study P = primary method used; A = additional method that may be used.

212 Taxonomy of Research Methods and Appropriate Objects/levels of Analysis
Theorem Proof Laboratory Experiment Field Case Study Survey Forecasting and Future Research Simulation and Game/Role Playing Subjective/ Argumentative Descriptive/ Interpretive Action Research Society Organization/Group Individual Technology Methodology Theory Building Testing Extension Modes for Newer Approaches (Interpretations) Modes for Traditional Empirical Approaches (Observations) No No Possibly Possibly Yes Yes Possibly Yes Yes Possibly No Possibly (Small Groups) Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes Yes Possibly Possibly Possibly Yes Yes Possibly Yes Yes Yes No Possibly Yes Yes Possibly Possibly Possibly No No Yes Yes Yes Yes No Yes Yes Yes Yes Yes No No No Yes Yes Yes Yes Yes Yes Yes Yes Possibly Possibly No Possibly No Possibly Possibly Possibly Possibly Possibly Possibly Possibly No No No Possibly Possibly (Galliers 1990)

213 International IS Research Methods

214 Design Science IS Research Framework
Information Systems (IS) are complex, artificial, and purposefully designed. IS are composed of people, structures, technologies, and work systems. Two Basic IS Research Paradigms Behavioral Research – Goal is Knowledge Design Research – Goal is Utility Source Al Hevner

215 IS Research Cycle IS Artifacts Provide Utility Behavioral Design
Science Research Design Science Research IS Theories Provide Knowledge Adapted from Hevner

216 Systems Development in Information Systems Research
Theory Building Conceptual frameworks Mathematical models Methods Systems Development Prototyping Product development Technology transfer Experimentation Computer simulations Field experiments Lab experiments Observation Case studies Survey studies Field studies Adapted from Nunamaker, Chen and Purden JMIS (1991) 7(3).

217 Arizona SW Engineering Methodology

218 Arizona SW Engineering Research Cycle
Theory, Concept, Model Prototypes Field and Lab Research Product Adapted from Nunamaker

219 Lab & Field Study Objectives
Develop new Process and Tool Uses Develop Metrics for Process and Tool Use Evaluate usefulness of new Processes and Tools Identify Process and Tool Improvements Confirm Lab Results in the Field Gain best Practice from Lab and Field Use Source Nunamaker

220 Research Road Map Through the Last Research Mile
Identify a Real Problem Proof-of-Concept Prototype Proof-of-Value Prototype Proof of Self-Sustaining Use (Production System) Travel the Last Mile 3. POU 2. POV Real Problem 1. POC Adapted from Nunamaker

221 Design Science Design is a Artifact (Noun) Design is a Process (Verb)
Constructs Models Methods Instantiations Design is a Process (Verb) Build Evaluate Design is a Wicked Problem Unstable Requirements and Constraints Complex Interactions among Subcomponents of Problem and resulting Subcomponents of Solution Inherent Flexibility to Change Artifacts and Processes Dependence on Human Cognitive Abilities - Creativity Dependence on Human Social Abilities - Teamwork Source Al Hevner

222 Design Science IS Research Framework (Hevner et. al., MISQ, 2004) USA
Implementable, synthesize an existing body of research, [or] stimulate critical thinking. among IS practitioners. Achieved by appropriately applying existing foundations and methodologies. Relevance Rigor Vocabulary and Symbols that provide the language in which problems and solutions are defined and communicated. Environment IS Research Abstractions and Representations that use constructs to represent a real world situation-the design problem and its solution space Knowledge Base IS Design theories seek to prescribe effective development practices (methods) and a type of system solution. (instantiation) for a particular class of user requirements. (models) Algorithms and Practices that define processes and provide guidance on how to solve problems, that is, how to search the solution space. People Roles Capabilities Characteristics Organizations Strategies Structure and Culture Process Technology Infrastructure Applications Communications Architecture Development Capabilities Constructs Models Methods Instantiations Implemented and Prototype systems that show that constructs, models, or methods can be implemented in a working system. They demonstrate feasibility, enabling concrete assessment of an artifact’s suitability to its intended purpose Develop/Build Foundations Theories Frameworks Instruments Constructs Models Methods Instantiations Methodologies Data Analysis Techniques Formalisms Measures Validation Criteria Theories Artifacts Business Needs Applicable Knowledge Assess Refine Justify/Evaluate Analytical Case Study Experimental Field Study Simulation Additions to the Knowledge base Application in the Appropriate Environment Design Science IS Research Framework (Hevner et. al., MISQ, 2004) USA

223 Three Cycles of DS Research
Design Science Knowledge Base Environment Foundations Scientific Theories & Methods Application Domain People Organizational Systems Technical Systems Problems & Opportunities Build Design Artifacts & Processes Experience & Expertise Rigor Cycle Grounding Additions to KB Relevance Cycle Requirements Field Testing Design Cycle Evaluate Meta-Artifacts (Design Products & Design Processes) Adapted from Hevner

224 Consortial Research Method (St Gallen Switzerland)
Outline: Need Gap, Goal Consortium Agreement Research Plan State of Instantiations State of Models & Methods State of Theories & Constructs Analysis Domain Scientific Publication User Interface Design Software Engineering Practitioner Publication Practical Knowledge Business Models Processes & Structures Information Systems Information Technology Scientific Knowledge Instantiations Models Methods Theories Constructs Method Engineering Diffusion Design Teaching Materials Reference Modeling …. Roll-out Plan Evaluation Review Workshop Function Test Experiment Simulation Pilot Application

225 Organisational Forms of Knowledge Creation
Internal External With Exclusive Exploitation Rights Without Exclusive Exploitation Rights With Exclusive Exploitation Rights Without Exclusive Exploitation Rights Industrial Research 1 Collaborative Research 2 Contract Research 3 With Customers Or Suppliers 2a With Neutral Partners 2b With Competitors 2c (adapted from Brockhoff 1999)


227 A Research Framework for the Organizational laboratory
The framework is represented by a triangle Change Dotted lines inside the triangle represent research dynamics as movements towards (and away from) the ideal types. Points are Ideal Types; That is Weberian abstractions That are not attainable in practice. Research Praxis is represented by the constrained space of the triangle. The Points represent Intended Research Outcomes: Movement toward the change point is achieved through a process of intervention as typified by action research. Change is the outcome of interventionist modes of enquiry; successful interventions lead to improvements in the problem situation. intervention One implication for research praxis is that all three dynamics (reduction, interpretation, and intervention) are, regardless of the research method adopted, co-present, albeit with differing mixes and emphasis. Prediction is the outcome of positivist modes of enquiry; although a ‘good’ theory does indeed have explanatory power, the more significant outcome of positivist theories is the ability to control and predict. Understanding is the outcome of Interpretivist modes of enquiry; successful Interpretations bring out insider rationality and promote understanding. Interpretation Reduction Understanding Prediction For example: interpretivist research methods involve a reducing of the infinite range of factors that might be considered relevant to a particular inquiry, although such a ‘reduction’ is not rationalized through the application of the systematic procedures of positivism. Movement toward the understanding point through a process of interpretation is associated with greater richness of insight into the role of IS in organizational settings. Understanding is achieved typically through case studies informed by schools of sociological thought such as phenomenology, hermeneutics and ethnography. As the researcher moves towards the prediction point through a process of formalized reduction there should be greater explanatory and predictive power. The traditional approach to explanation and prediction is experimental method. Adapted from Braa and Vidgen =1997)

228 Mapping Research Methods into the Framework
an extension of laboratory experiments into an organizational context. There are lots of factors which the researcher cannot control but which could affect the outcomes. Field experiments aim at controlling a small number of variables which may then be studied intensively. A major advantage is that the experiment is conducted in a real-life setting. A major problem however is the difficulty of finding organizations prepared to be experimented on. Experimental control is essential and involves taking appropriate steps to eliminate ‘nuisance’ variables, which are factors other than the independent variables that might be responsible for observed changes in the dependent variable. ‘true’ experimental design which meets the criteria of multiple treatments (or one treatment and a control group), randomization, and experimental control; Empirical inquiry that ‘investigates a contem­porary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident’. Can be used in three modes: explanatory, descriptive, and exploratory . Applicable where the research question is of a ‘how’ or ‘why’ nature, where control over behavioural events is not needed, and where there is a focus on contemporary events. Allow reality to be captured in detail and many variables to be analyzed; from a positivist stance, problems with case studies include the difficulty of generalization, lack of control over variables, and different interpretations by different stakeholders An experimental design which does not meet the three criteria of multiple treatments (or one treatment and a control group), randomization, and experimental control; but rather attempts to preserve as many of the properties of true experimentation as possible, given the constraints of the research setting. These are much more common than ‘True’ experiments. Interpretivist approach is concerned with gaining understanding; generalization is the movement from a concrete situation to the social totality beyond the individual case. A soft case study based on ethnographic methods can involve a variety of data collection techniques, such as videotape, and data analy­sis might involve, for example, techniques from grounded theory From an interpretive position, the validity of an extrapolation from an individual case or cases depends not on the representativeness of such cases in a statistical sense, but on the plausibility and cogency of the logical reasoning used in describing the results from the cases, and in drawing conclusions from them. Change (Intervention) A hybrid between interpretation and intervention. A trade-off between being an observer who can make interpretations (understanding) and a researcher involved in creating change in practice. when doing case studies researchers contribute to change by questioning events and applying new concepts. On the other hand, full-scale action research projects are often not appropriate due to organizational constraints or the nature of the topic to be inves­tigated. Small scale intervention with a deep contextual understanding is one way of balancing this dilemma. Action Research Quasi experiment Action case Action research is a way of building theory and descriptions within the context of practice itself. Theories are tested through intervention in the organizational laboratory, that is, through experiments that bear the double burden of testing hypotheses and effecting some desirable change in the situation. Action research comes in many flavours, ranging from formal approaches through to less formalized, more reflective and personal approaches. Field Experiment Soft Case Hard Case Prediction (Reduction) Understanding (Interpretation) Adapted from Braa and Vidgen =1997)

229 Adapted from Baskerville and Stage
Cycle Process of Action Research DIAGNOSING Identifying or Defining a problem SPECIFIC LEARNING Identification of General Findings ACTION PLANNING Considering alterative Courses of Action To Solve a Problem Development of a Client-System Infrastructure EVALUATING Studying Consequences Of an Action ACTION TAKING Selecting a Course of Action Adapted from Baskerville and Stage

230 Questions?

Download ppt "IS Research Development an International Perspective"

Similar presentations

Ads by Google