A Neuro-Fuzzy Model with SEER-SEM for Software Effort Estimation Wei Lin Du, Danny Ho*, Luiz F. Capretz Software Engineering, University of Western Ontario, London, Ontario, Canada * NFA Estimation Inc., Richmond Hill, Ontario, Canada November 2010

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

Purpose Integrate neuro-fuzzy (NF) technique with SEER-SEM Evaluate estimation performance of NF SEER-SEM versus SEER-SEM

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

SEER-SEM SEER-SEM was trademarked by Galorath Associates, Inc. (GAI) in 1990 Effort estimation is one of the SEER- SEM algorithmic models SEER-SEM Estimation Processing Size Personnel Environment Complexity Constraints Effort Cost Schedule Risk Maintenance

SEER-SEM Effort Estimation Software Size Lines, function points, objects, use cases Technology and Environment Parameters Personal capabilities and experience (7) Development support environment (9) Product development requirements (5) Product reusability requirements (2) Development environment complexity (4) Target environment (7)

SEER-SEM Equations where:E Development effort K Total lifecycle effort including development and maintenance S e Effective size D Staffing complexity C te Effective technology C tb Basic technology

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

NFA FM 2 … NFB 1 NFB N Algorithmic Model NFB 2 Output Metric Mo FM 1 FM N RF 2 RF 1 RF N ARF 1 ARF N Preprocessing Neuro-Fuzzy Inference System (PNFIS) ARF 2 … where N is the number of contributing factors, M is the number of other variables in the Algorithmic Model, RF is Factor Rating, ARF is Adjusted Factor Rating, NFB is the Neuro-Fuzzy Bank, FM is Numerical Factor/Multiplier for input to the Algorithmic Model, V is input to the Algorithmic Model, and Mo is Output Metric. USA Patent No. US B2

   N N N A iN A i2 A i1  …… … ARF i FM i FMP i1 FMP iN FMP i2 w1w1 wNwN Layer1 Layer3Layer4Layer5 Layer2 NFB where ARFi is Adjusted Factor Rating for contributing factor i, is fuzzy set for the k-th rating level of contributing factor i, is firing strength of fuzzy rule k, is normalized firing strength of fuzzy rule k, is parameter value for the k-th rating level of contributing factor i, and is numerical value for contributing factor i.

NF SEER-SEM ACAP NF 1 NF 2 NF m … Software Estimation Algorithmic Model Effort Estimation SEER-SEM Effort Estimation Size, SIBR P1P1 P2P2 P 34 AEXP Complexity (Staffing)

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

Performance Metrics Relative Error (RE) = (Est. Effort – Act. Effort) / Act. Effort Magnitude of Relative Error (MRE) = |Est. Effort – Act. Effort | / Act. Effort Mean Magnitude of Relative Error (MMRE) = (∑MRE) / n Prediction Level (PRED) PRED(L) = k / n

Design of Evaluation Case IDDescription C1No outliers C2Including all outliers C3Excluding part of outliers C4-175% for Learning, 25% for testing C4-250% for Learning, 50% for testing

MMRE Results Case ID MMRE (%) SEER-SEMValidationChange C C C C C Negative value of MMRE change means improvement

MMRE Results

PRED Results SEER-SEM Average of Validation Change PRED(20%)39.76%27.48%-12.28% PRED(30%)49.27%36.46%-12.81% PRED(50%)62.02%55.35%-6.67% PRED(100%)85.55%97.69%12.14% Positive value of PRED change means improvement

Summary of Evaluation Results MMRE is improved in all cases, with the greatest improvement over 25% Average PRED(100%) is increased by 12% NF SEER-SEM improves MMRE by reducing large MREs

Agenda Purpose SEER-SEM NF SEER-SEM Evaluation Conclusion

NF with SEER-SEM improves estimation accuracy General soft computing framework works with various effort estimation algorithmic models

Future Directions Evaluate with original SEER-SEM dataset Evaluate general soft computing framework with: more complex algorithmic models other domains of estimation

THANKS !

Any Questions?