1. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 1 Evaluation of a Business Application Framework Using Complexity and Functionality Metrics Hikaru Fujiwara †, Shinji Kusumoto †, Katsuro Inoue †, Toshifusa Ootsubo ‡, Katsuhiko Yuura ‡ † Graduate School of Engineering Science, Osaka University, Japan. ‡ Business Solution Systems Division, Hitachi Ltd. This research is partially supported by International Information Science Foundation

2. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 2 1. Background 2. Approach 2.1 Purpose 2.2 Reuse 3. Evaluation Method 4. Case Study 4.1 Case Study 1 4.2 Case Study 2 5. Conclusion Contents 1. Background

3. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 3 1. Background (1/4) It becomes important to develop high-quality software cost-effectively. Reuse is one of the most famous techniques to attain it. In object-oriented software development, developers reuse a particular library called framework. A framework is a collection of classes that provide a set of services for a particular domain.

4. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 4 1. Background (2/4) A department of Hitachi Ltd. develops application software for many local governments. The developers have been using the original reuse technique based on a conventional module-based reuse.

5. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 5 1. Background (3/4) 47 prefectures in Japan. 3229 local governments, 671 cities, 1991 towns, 567 villages. A department of Hitachi Ltd. develops application software for many local governments. The requirement of the application is different depending on local ordinance.

6. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 6 1. Background (4/4) The department is going to introduce the framework in order to improve the efficiency. It is difficult to transfer the new framework to the development. To motivate the developers, we need to show the benefit of using framework quantitatively.

7. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 7 1. Background 2. Approach 2.1 Purpose 2.2 Reuse 3. Evaluation Method 4. Case Study 4.1 Case Study 1 4.2 Case Study 2 5. Conclusion Contents 1. Background

8. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 8 2. Approach 2.1 Purpose We evaluate the usefulness of the framework quantitatively from two viewpoints. –saving cost –improving software quality We conducted two case studies to evaluate it. –Application : Applications for local governments –Language : Java

9. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 9 2.2 Reuse Conventional module-based reuse (1/2) An application consists of a main module and some screen control modules. The main module controls business logics and screen transitions of the application. Each screen control module corresponds to one screen. A ： Database renewal program X 1 ： Data Inquiry Y 1 ： Showing the Inquiry Result Z 1 ： Updating the Record (a) Health insurance application for a local government A Screen transition controller

10. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 10 2.2 Reuse Conventional module-based reuse (2/2) A ： Database renewal program X 1 ： Data Inquiry Y 1 ： Showing the Inquiry Result Z 1 ： Updating the Record (a) Health insurance application for a local government A Screen transition controller X 2 ： Data Inquiry Y 2 ： Showing the Inquiry Result Z 2 ： Updating the Record (b) Health insurance application for a local government B B ： Database renewal program Screen transition controller

11. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 11 2.2 Reuse Framework-based reuse The framework is intended to reuse the processing of the transition of the screens in addition to module-based reuse. –Typical transitions of screens are prepared. data inquiry, data renewal, data addition, data removal, etc. Database Renewal Program A Database renewal Framework Data Inquiry Showing the Inquiry Result Updating the Record F 1 ： Framework Specific Parameters to a Local Government A Specific Parameters to a Local Government B Database Renewal Program B

12. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 12 1. Background 2. Approach 2.1 Purpose 2.2 Reuse 3. Evaluation Method 4. Case Study 4.1 Case Study 1 4.2 Case Study 2 5. Conclusion Contents 1. Background

13. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 13 3. Evaluation Method We compare the framework-based reuse with the conventional module-based reuse from the viewpoints of cost and quality. (Case Study 1) Developing applications that have the same functions. Specification of function f a framework-based reuse conventional reuse CaCa FW PaPa FW ： framework

14. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 14 3. Evaluation Method (Case Study 2) Adding functions to the existing application. Specification of function f a CaCa FW PaPa framework-based reuse conventional reuse Specification of function f b C a+b FW P a+b Specification of function f c C a+b+c FW P a+b+c

15. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 15 3. Evaluation Method Metrics used in the case studies We could not collect the actual effort and the number of faults. We used following metrics to indirectly evaluate the productivity and quality. –OOFP (Object-Oriented Function Point) measures functionality. productivity. –C&K metrics (Chidamber and Kemerer’s metrics) measure complexity. quality.

16. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 16 3. Evaluation Method OOFP （ Object Oriented Function Points § ) OOFP is an adaptation of FP(Function Point) to enable the measurement of object-oriented analysis and design specifications. –FP is measured from logical files(Internal Logical Files:ILFs, External Interface Files:EIFs) and transactions (inputs, outputs, inquiries). –OOFP is measured from logical files (ILF, EIF) and transactions (Service Requests:SRs). Classes correspond to logical files Methods correspond to transactions §:G.Caldiera, G.Antoniol, R.Fiutem, C.Lokan, “Definition and Experimental Evaluation of Function Points for Object-Oriented Systems”, IEEE, Proc. of METRICS98, pp.167-178 (1998).

17. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 17 3. Evaluation Method C&K metrics (Chidamber and Kemerer’s metrics) viewpointsmetricshow to calculate metrics of a class C inheritanceDITthe depth of C in the inheritance tree NOCthe number of immediate sub-classes subordinated to C couplingCBOthe number of couplings between C and any other class RFC (the number of the methods in C) ＋ (the number of the methods called by C) methodWMCthe sum of the complexity of the methods in C LCOM Assume that methods M 1...M n ∈ C, I i is a set of instance variables used in M i, (the number of pairs (M k, M l ) such that I k ∩I l ＝ φ) － (the number of pairs (M k, M l ) such that I k ∩I l ≠φ)

18. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 18 1. Background 2. Approach 2.1 Purpose 2.2 Reuse 3. Evaluation Method 4. Case Study 4.1 Case Study 1 4.2 Case Study 2 5. Conclusion Contents 1. Background

19. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 19 4. Case Study 4.1 Case Study 1 Four applications C a, C b, C c, C d are developed using the framework-based reuse. Four applications P a, P b, P c, P d are developed using the conventional reuse. C i and P i (i = a,b,c,d) implement the same function f i. framework-basedconventional function fafa CaCa PaPa fbfb CbCb PbPb fcfc CcCc PcPc fdfd CdCd PdPd

20. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 20 4.1 Case Study 1 We compared C i with P i from the viewpoints of productivity and quality. We measured OOFP and C&K metrics from newly developed part of each application. FW ： framework : newly developed Specification of function f a CaCa FW PaPa framework-based reuse conventional reuse FW ： framework

21. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 21 4.1 Case Study 1 Result of case study 1 number of classes OOFPCBORFCWMCLCOM CaCbCcCdCaCbCcCd 5 11 8 176 180 418 252 3.8 5.8 5.4 4.1 14.4 18.2 33.1 17.8 7.4 7.6 8.1 6.4 21.4 23.2 28.7 13.8 PaPbPcPdPaPbPcPd 25 29 28 526 671 672 2.1 2.3 3.0 2.4 5.8 5.9 7.4 8.6 3.3 3.4 4.3 2.1 2.0 15.7 C&K metrics are the average values per class

22. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 22 4.1 Case Study 1 Analysis of OOFP framework- based (C) conventional (P) function fafa 176526 fbfb 180526 fcfc 418671 fdfd 252672 OOFP(C i ) < OOFP(P i ) Application development using framework reduces the effort of development.

23. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 23 4.1 Case Study 1 Analysis of OOFP To develop the framework, initial investment (effort) is needed. The OOFP of the framework FW is 1298. The framework-based reuse is about 2.5 times more effective than the conventional reuse. The department develops the similar applications repeatedly. It will save the effort after three or four applications have been developed, whereas the investment for the framework was spent.

24. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 24 4.1 Case Study 1 Analysis of CBO and RFC average CBOaverage RFC framework- based (C) conventional (P) framework- based (C) conventional (P) function fafa 3.82.114.45.8 fbfb 2.318.25.9 fcfc 5.43.033.17.4 fdfd 4.12.417.88.6 CBO(C i ) > CBO(P i ), RFC(C i ) > RFC(P i ) All of the methods called by C i are included in the framework. If the framework is high quality, the complexity does not affect the quality of the overall application program.

25. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 25 4.1 Case Study 1 Analysis of WMC and LCOM average WMCaverage LCOM framework- based (C) conventional (P) framework- based (C) conventional (P) function fafa 7.43.321.42.1 fbfb 7.63.323.22.1 fcfc 8.13.428.72.0 fdfd 6.44.313.815.7 WMC(C i ) > WMC(P i ), LCOM(C i ) > LCOM(P i ) There are many simple methods, that set/get the values of the attribute. The complexity does not affect the quality of the overall application program.

26. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 26 1. Background 2. Approach 2.1 Purpose 2.2 Reuse 3. Evaluation Method 4. Case Study 4.1 Case Study 1 4.2 Case Study 2 5. Conclusion Contents 1. Background

27. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 27 4.2 Case Study 2 At first, application C a and P a are developed. Function f b, f c, f d are continuously added to C a and P a. C i and P i (i = a,a+b,a+b+c,a+b+c+d) implement the same function f i. framework- based conventional function fafa CaCa PaPa f a+b C a+b P a+b f a+b+c C a+b+c P a+b+c f a+b+c+d C a+b+c+d P a+b+c+d

28. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 28 4.2 Case Study 2 We compared the differences between the two successive applications from the viewpoint of productivity and quality. We measured OOFP and C&K metrics from newly developed part of each application. Specification of function f a CaCa FW PaPa framework-based reuse conventional reuse Specification of function f b C a+b FW P a+b Specification of function f c C a+b+c FW P a+b+c

29. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 29 4.2 Case Study 2 Result of case study 2 number of classes OOFPCBORFCWMCLCOM C a C a+b C a+b+c C a+b+c+d 5 7 15 20 176 251 578 743 3.8 5.0 5.9 5.8 14.4 16.7 30.1 28.3 7.4 7.6 8.3 7.9 21.4 20.1 28.6 25.6 P a P a+b P a+b+c P a+b+c+d 25 29 39 46 526 615 849 1084 2.1 2.8 3.7 4.0 5.8 6.9 8.6 10.5 3.3 3.9 2.1 2.0 1.8 10.0 C&K metrics are the average values per class

30. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 30 4.2 Case Study 2 Analysis of OOFP framework- based (C) conventional (P) function fafa 176526 f a+b 251615 f a+b+c 578849 f a+b+c+d 7431084 75 327 165 89 234 235 OOFP(C a+b+c - C a+b ) > OOFP(P a+b+c - P a+b ) The adaptability of the framework to the function f c is not good. It is necessary to add new components for the functions like f c to the framework.

31. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 31 4.2 Case Study 2 Analysis of OOFP OOFP(C a+b+c+d ) < OOFP(P a+b+c+d ) framework- based (C) conventional (P) function fafa 176526 f a+b 251615 f a+b+c 578849 f a+b+c+d 7431084 The framework-based reuse is more effective to reduce the effort of development than the conventional reuse.

32. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 32 4.2 Case Study 2 Analysis of CBO and RFC average of CBOaverage of RFC framework- based (C) conventional (P) framework- based (C) conventional (P) function fafa 3.82.114.45.8 f a+b 5.02.816.76.9 f a+b+c 5.93.730.18.6 f a+b+c+d 5.84.028.310.5 1.2 0.9 -0.1 0.9 0.3 2.3 13.4 -1.8 1.1 1.7 1.9 RFC(C a+b+c - C a+b ) > RFC(P a+b+c - P a+b ) C a+b+c calls a lot of methods included in the framework, in order to handle many data items in the function f c. If the framework is high quality, the complexity does not affect the overall application program.

33. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 33 1. Background 2. Approach 2.1 Purpose 2.2 Reuse 3. Evaluation Method 4. Case Study 4.1 Case Study 1 4.2 Case Study 2 5. Conclusion Contents 1. Background

34. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 34 5. Conclusions We have experimentally evaluated the usefulness of the framework quantitatively from the viewpoints of quality and saving cost. As the result of the case studies, the framework-based reuse is more effective than the conventional reuse. In order to show the usefulness of the framework, we are going to apply the framework to many software development projects in future.

35. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 35

36. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 36 Appendix How to measure OOFP(1) OOFP=OOFP ILF +OOFP EIF +OOFP SR For each ILF, OOFP ILF is measured using DETs (Data Element Types) and RETs (Record Element Types). For each EIF, OOFP EIF is measured using DETs and RETs. For each SR, OOFP SR is measured using DETs and FTRs (File Types Referenced). Finally, OOFP is calculated by summing up OOFP ILF, OOFP EIF and OOFP SR.

37. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 37 Appendix How to measure OOFP (2) To calculate OOFP from Java source code, we suppose that the concepts in Java correspond to ones in OOFP as follows. OOFPJava Logical filesILFsClasses within the application. EIFsClasses outside of the application. DETsNumber of simple attributes (such as integers, strings) RETsNumber of complex attributes (such as objects) TransactionsSRsMethods within the application. DETsNumber of simple arguments, instance variables and class variables. FTRsNumber of complex arguments, instance variables, class variables and objects.

38. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 38 Appendix Case Study 2; Analysis of WMC and LCOM (1) average of WMCaverage of LCOM framework- based (C) conventional (P) framework- based (C) conventional (P) function fafa 7.43.321.42.1 f a+b 7.63.320.12.0 f a+b+c 8.33.328.61.8 f a+b+c+d 7.93.925.610.0 0.2 0.7 -0.4 0.0 0.6 -1.3 8.5 -3.0 -0.1 -0.2 8.2 LCOM(C a+b+c - C a+b ) > LCOM(P a+b+c - P a+b ) There are a lot of set/get methods in C a+b+c, in order to handle many data items in the function f c.

39. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 39 Appendix Case Study 2; Analysis of WMC and LCOM (2) average of WMCaverage of LCOM framework- based (C) conventional (P) framework- based (C) conventional (P) function fafa 7.43.321.42.1 f a+b 7.63.320.12.0 f a+b+c 8.33.328.61.8 f a+b+c+d 7.93.925.610.0 0.2 0.7 -0.4 0.0 0.6 -1.3 8.5 -3.0 -0.1 -0.2 8.2 LCOM(C a+b+c+d - C a+b+c ) < LCOM(P a+b+c+d - P a+b+c ) There are complex transitions of the screens in function fd. In P a+b+c+d, the LCOM values of the classes implement these transactions are high.

40. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 40 Appendix Case Study 2; Analysis of WMC and LCOM (3) average of WMCaverage of LCOM framework- based (C) conventional (P) framework- based (C) conventional (P) function fafa 7.43.321.42.1 f a+b 7.63.320.12.0 f a+b+c 8.33.328.61.8 f a+b+c+d 7.93.925.610.0 WMC(C i ) > WMC(P i ), LCOM(C i ) > LCOM(P i ) We examined the classes of C i that have high WMC and LCOM values. It is found that there are many simple methods, that set/get the values of the attribute.

41. Software Engineering Research Group, Graduate School of Engineering Science, Osaka University 41 Appendix Process of Case Study Development of the framework Case Study1 –Two developers –Develop C a, C b, C c, C d –Develop P a, P b, P c, P d Case Study2 –Two developers –Develop C a, C a+b, C a+b+c, C a+b+c+d –Develop P a, P a+b, P a+b+c, P a+b+c+d Apply the metrics to the applications. Analysis