1. HiST Tacitly developed research methods in ICS Ivar Tormod Berg Ørstavik

2. HiST Outline Introduction: Methodology in ICS (ICS = CS + SE + IS) KWIC: Where and how is ICS methodology discussed? Text classification: Research method per chapter in 96 PhDs at IDI, NTNU. Research method exemplification: ICS engineering as a research method.

3. HiST Introduction ICS as applied science “A large part of CS research consists of proposing new designs: systems, algorithms, and models. Such designs must be judged by whether they increase our knowledge about what are useful and cost-effective problem solutions” Tichy (1995: 9). “Software engineering is concerned with techniques useful for the development of effective software programs” (Zelkowitz 1997: 735). “A justified theory that is not useful for the environment contributes as little to the IS literature as an artifact that solves a nonexistent problem” Hevner (2004: 81).

4. HiST Introduction Methods’ status in ICS applied science Research method is a means to an end. From a strong applied research position, the internal workings of research processes are only indirectly relevant. “highly practical research in the field of information systems [to] simply ignore the research method issue” (Baskerville 1998: 90). “CS has not developed a concise taxonomy of methods applicable for demonstrating the validity of a new technique” (Zelkowitz 1997: 1). “a lack of methodology for study and research in scientific software development” (Diaconescu 2002: 72). “SE researchers rarely write explicitly about their paradigms of research and their standards for judging quality of research” (Shaw 2002: 1).

5. HiST Introduction Does ICS lack methods? NO! Missing is not the methods themselves, but discussions of them. Does ICS lack a methodology alltogether? No, in Software Engineering a vibrant discussion about the need for empirical experiments and observation as research methods in SE thrives. No, discussions of research methods in ICS pop up everywhere. Yes, in ICS and particularily CS the researcher’s own role and methods are often implicit, not discussed. Yes, a coherent methodological discussion in which all the three disciplines participate does not seem to be present. Does poor methodology equal poor methods? No, the ICS disciplines and many of their methods has clearly demonstrated their value! An explicitly discussed method does not necessarily equals a better method. But, by discussing strengths and weaknesses of methods openly, it is much easier to improve on them.

6. HiST KWIC

7. HiST KWIC From the mid-1990s onward, several theses mainly within SE devote lengthy discussions and entire chapters for discussions of research method. But still research methods remain implicit more than half the PhD theses. Some comments about the KWIC analysis: Each occurrence of “method” must be subjectively judged by what type of method it is. Not all “methods” are research methods. Here, “research method” is every “method” that is to be performed by the researcher(s) in their capacity as researcher (i.e. not a “system method” or “development methods”). Other terms such as ”approach” could be used to describe reserach method. KWIC gives an overall perspective, not exact, hard facts.

8. HiST Text classification

9. HiST Text classification The research methods are spread unevenly. As in the two original studies, most categories are practically not present in this material. Some categories are too broad and function as storage containers. Text classification is highly problematic! Research methods are not spread out 1:1 per article or PhD chapter. Research methods do (and should) not follow a rigid template. They are tailored to the individual projects needs and context. To classify a research method therefore often implies a subjective interpretation of each category that both varies from text to text and researcher to researcher. The resulting graph and research method taxonomies hide these essential interpretations. The existing taxonomies are biased. Some categories are highly nuanced, others wide open. In my material, far too many texts used methods not really part of the taxonomy.

10. HiST Research method exemplification ICS Engineering as a research method in two projects: “an ultrasound system”, (Lindseth 2002). “a database system”, (Thorbjørnsen 1995). Shared tacit assumptions underpinning the two projects. Solutions and designs are implementable in hardware and software. Both projects engineer solutions and only devote time to problems that can be described as solutions. Any analytical process (process that takes things apart) is subsumed in this constructive outlook. The projects goals are applied knowledge and are justified by their usefulness. The main source of insight is gained not from observing systems but from participation in their construction. The researcher himself is the brains behind many of the ideas and solutions discussed. The researcher’s role should be understood as subjective participation, not objective observation.

11. HiST Research method exemplification ICS Engineering as a research method in two projects: “the fishing boats with fishermen system”, (Digernes 1982). “The development process system”, (Opdahl 1992). These two projects extend tacit assumptions about ICS engineering. Diversity of problems is overcome through unity in solutions. Internal coherence is achieved through holistic solutions. The research method is optimistic. Research questions are not formulated skeptically (“Can X be solved?”), but optimistically (“What is the best solution for X?”). Problems pursued are solvable and implementable: unsolvable problems is not part of ICS engineering’s scope.

12. HiST Research method exemplification As a research method, ICS engineering is strongly solution oriented. The aim of ICS engineering is to develop applicable, useful solutions. This applied goal is propped up by an optimistic outlook that filters out problems that cannot be solved. While working with solvable problems, the active process of designing a solution is central. The researchers participate in these constructive processes and therefore act subjectively when they later analyze their own and others’ rival or allied technical concepts. Internal coherence in the research project is primarily achieved through a holistic solution.

13. HiST Future leads ICS research methods are worth studying! Before trends can be counted, they must first be described. Studies of research methods are likely to be qualitative and subjective, even when they are presented as quantitative and objective. As so many ICS research projects and methods use system construction and implementation so vibrantly, investigations about the role of design and implementation languages on research method and goals should be investigated.