1. A Neuro-Fuzzy Model with SEER-SEM for Software Effort Estimation Wei Lin Du, Danny Ho*, Luiz F. Capretz Software Engineering, University of Western Ontario, London, Ontario, Canada * NFA Estimation Inc., Richmond Hill, Ontario, Canada November 2010

2. Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

3. Purpose Integrate neuro-fuzzy (NF) technique with SEER-SEM Evaluate estimation performance of NF SEER-SEM versus SEER-SEM

4. Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

5. SEER-SEM SEER-SEM was trademarked by Galorath Associates, Inc. (GAI) in 1990 Effort estimation is one of the SEER- SEM algorithmic models SEER-SEM Estimation Processing Size Personnel Environment Complexity Constraints Effort Cost Schedule Risk Maintenance

6. SEER-SEM Effort Estimation Software Size Lines, function points, objects, use cases Technology and Environment Parameters Personal capabilities and experience (7) Development support environment (9) Product development requirements (5) Product reusability requirements (2) Development environment complexity (4) Target environment (7)

7. SEER-SEM Equations where:E Development effort K Total lifecycle effort including development and maintenance S e Effective size D Staffing complexity C te Effective technology C tb Basic technology

8. Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

9. NFA FM 2 … NFB 1 NFB N Algorithmic Model NFB 2 Output Metric Mo FM 1 FM N RF 2 RF 1 RF N ARF 1 ARF N Preprocessing Neuro-Fuzzy Inference System (PNFIS) ARF 2 … where N is the number of contributing factors, M is the number of other variables in the Algorithmic Model, RF is Factor Rating, ARF is Adjusted Factor Rating, NFB is the Neuro-Fuzzy Bank, FM is Numerical Factor/Multiplier for input to the Algorithmic Model, V is input to the Algorithmic Model, and Mo is Output Metric. USA Patent No. US-7328202-B2

10.    N N N A iN A i2 A i1  …… … ARF i FM i FMP i1 FMP iN FMP i2 w1w1 wNwN Layer1 Layer3Layer4Layer5 Layer2 NFB where ARFi is Adjusted Factor Rating for contributing factor i, is fuzzy set for the k-th rating level of contributing factor i, is firing strength of fuzzy rule k, is normalized firing strength of fuzzy rule k, is parameter value for the k-th rating level of contributing factor i, and is numerical value for contributing factor i.

11. NF SEER-SEM ACAP NF 1 NF 2 NF m … Software Estimation Algorithmic Model Effort Estimation SEER-SEM Effort Estimation Size, SIBR P1P1 P2P2 P 34 AEXP Complexity (Staffing)

12. Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

13. Performance Metrics Relative Error (RE) = (Est. Effort – Act. Effort) / Act. Effort Magnitude of Relative Error (MRE) = |Est. Effort – Act. Effort | / Act. Effort Mean Magnitude of Relative Error (MMRE) = (∑MRE) / n Prediction Level (PRED) PRED(L) = k / n

14. Design of Evaluation Case IDDescription C1No outliers C2Including all outliers C3Excluding part of outliers C4-175% for Learning, 25% for testing C4-250% for Learning, 50% for testing

15. MMRE Results Case ID MMRE (%) SEER-SEMValidationChange C184.3961.05-23.35 C284.3959.11-25.28 C384.3959.07-25.32 C4-150.4939.51-10.98 C4-242.0529.01-13.04 Negative value of MMRE change means improvement

16. MMRE Results

17. PRED Results SEER-SEM Average of Validation Change PRED(20%)39.76%27.48%-12.28% PRED(30%)49.27%36.46%-12.81% PRED(50%)62.02%55.35%-6.67% PRED(100%)85.55%97.69%12.14% Positive value of PRED change means improvement

18. Summary of Evaluation Results MMRE is improved in all cases, with the greatest improvement over 25% Average PRED(100%) is increased by 12% NF SEER-SEM improves MMRE by reducing large MREs

19. Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

20. NF with SEER-SEM improves estimation accuracy General soft computing framework works with various effort estimation algorithmic models

21. Future Directions Evaluate with original SEER-SEM dataset Evaluate general soft computing framework with: more complex algorithmic models other domains of estimation

22. THANKS !

23. Any Questions?