Software Engineering Research & Development at A.U.Th Ioannis Stamelos Panagiotis Katsaros {stamelos,

18 Apr 2006Sweden2 Presentation Outline Introduction Introduction AUTH / Dept of Informatics AUTH / Dept of Informatics Software Engineering Research Group Software Engineering Research Group Software Quality, Management, Education Software Quality, Management, Education Open Source Software Engineering / Extreme Programming Open Source Software Engineering / Extreme Programming

18 Apr 2006Sweden3 Introduction

18 Apr 2006Sweden4

18 Apr 2006Sweden5 City of Thessaloniki Capital of Macedonia region, northern Greece Capital of Macedonia region, northern Greece ~1 M inhabitants, 2 nd greek city, major port of Balkans ~1 M inhabitants, 2 nd greek city, major port of Balkans 2300 years of continuous history, joined modern Greece in years of continuous history, joined modern Greece in 1912

18 Apr 2006Sweden6

18 Apr 2006Sweden7 AUTH / Dept of Informatics

18 Apr 2006Sweden8 Aristotle University of Thessaloniki, Greece First school (Philosophy) established in 1925 First school (Philosophy) established in 1925 Largest Greek University Largest Greek University –45 departments –> students – > staff / teachers

18 Apr 2006Sweden9 Department of Informatics School of Physical Science, plans to move to Polytechnic School School of Physical Science, plans to move to Polytechnic School Established in 1993 Established in academic staff members 22 academic staff members ~700 students ~700 students

18 Apr 2006Sweden10 4-years Degree Curriculum 8 semesters 8 semesters 5 semesters with obligatory courses 5 semesters with obligatory courses 3 semesters with options out of 4 directions: 3 semesters with options out of 4 directions: –Information Systems –Digital Media –Multimedia / Educational Technologies –Networks / Architecture / Communications Thesis Project Thesis Project

18 Apr 2006Sweden11 Master Courses MSc in Informatics (4 directions) MSc in Informatics (4 directions) MSc in Informatics & Management (with Dept. of Business Management) MSc in Informatics & Management (with Dept. of Business Management)Plus MSc in Medical Informatics MSc in Medical Informatics

18 Apr 2006Sweden12 Five Research Labs Programming Languages and Software Engineering – PLASE Programming Languages and Software Engineering – PLASE Data Technologies and Engineering -DELAB Data Technologies and Engineering -DELAB Information Processing and Artificial Intelligence Information Processing and Artificial Intelligence Multimedia (Educational Technologies) Multimedia (Educational Technologies) Networks, Communications and Architecture Networks, Communications and Architecture

18 Apr 2006Sweden13 Programming Languages and Software Engineering – PLASE 6 staff members 6 staff members 10 research associates 10 research associates 20 PhD students 20 PhD students Major research areas: Major research areas: –Intelligent Systems –Web Engineering –Software Engineering –Mathematics

18 Apr 2006Sweden14 Software Engineering Group (SWENG) 3 ac. staff members: 3 ac. staff members: –Assistant Professors I. Stamelos, L. Angelis / PLASE –P. Katsaros / DELAB 5 research associates 5 research associates 10 PhD students (joined the group during last 4 years) 10 PhD students (joined the group during last 4 years)

18 Apr 2006Sweden15 SWENG: Teaching Dept. of Informatics / Hellenic Open University (ODL) Dept. of Informatics / Hellenic Open University (ODL) –Mathematics, Graph Theory –Language Theory, Compilers –Introduction to Software Engineering –Object Oriented Analysis / Java / C++ Master Courses Master Courses –Software Project Management –Enterprise Information Systems

18 Apr 2006Sweden16 SWENG: R&D Activities Basic Research Basic Research Collaboration with other research teams in Greece and in EU: Collaboration with other research teams in Greece and in EU: –Politecnico di Torino (Prof. Morisio) –Univ. LAMSADE / Paris (Prof. Tsoukias) –Univ. Juan Carlos / Madrid (Prof. Barahona) Funded Applied Research & Development Funded Applied Research & Development –Greek Public Sector –Greek Secretariat for Research and Technology –Private Companies –EU

18 Apr 2006Sweden17 SWENG: Funding Software industry context: Software industry context: –Public Sector still developing IT infrastructure (e.g. transportation) –Private Sector companies enhancing IT infrastructure (banks, utilities, privatized public companies) –Most large SW Companies mainly acting as vendors (e.g. of ERP systems) –Various small but aggressive SW companies acting as developers Funding: limited but growing fast Funding: limited but growing fast –(in last 5 years, while SWENG group was growing) ~0,7 M euros –mostly from development projects –participated also to other PLASE projects (MATHIND, Go Net, …) Now shifting to research projects as well: Now shifting to research projects as well: –obtained ~0,5 M, expecting another 0,6 M in years (mainly from EU, GSRT) –see forthcoming research projects at Forthcoming Research Projects slide Forthcoming Research ProjectsForthcoming Research Projects

18 Apr 2006Sweden18 Examples of current / completed projects Expert System for Software Evaluation Expert System for Software Evaluation Automated Certification System (with data mining functionality) Automated Certification System (with data mining functionality) Enterprise Knowledge Management System Enterprise Knowledge Management System Specification of various public company information systems (e.g. ticketing) Specification of various public company information systems (e.g. ticketing)

18 Apr 2006Sweden19 SWENG: Major Research Areas Software Quality Software Quality –Dependability, Security Software Cost Estimation Software Cost Estimation Software Evaluation Software Evaluation Open Source Software Open Source Software Extreme Programming Extreme Programming Software Education Software Education

18 Apr 2006Sweden20 Software Quality, Management, Education

18 Apr 2006Sweden21 Software Quality (1) Problems: Problems: –how to quantify and predict / assess SW quality attributes –many SW quality metrics /methods have been proposed but have not been sufficiently validated Need for Need for –novel, intuitively confirmed measures –empirical evidence for assessing methods, tools, approaches

18 Apr 2006Sweden22 Software Quality (2) Formal experiments Formal experiments –Most powerful empirical investigation tool, producing generic conclusions A typical formal experiment setting: A typical formal experiment setting: –Two groups of subjects are given the same object, differing only in the controlled variable (e.g. design of the same system based on two different design approaches, see next two slides) –Subjects are asked to perform the same task –All other potentially affecting variables are kept constant (e.g. group experience) –The two groups’ performance is monitored accurately and statistical tests are run to detect statistical difference

18 Apr 2006Sweden23

18 Apr 2006Sweden24

18 Apr 2006Sweden25 Software Quality (3) Currently investigating quality assessment based on Heuristics Currently investigating quality assessment based on Heuristics - A Heuristic is a good practice (e.g. avoidance of ‘God’ classes in OO design, see previous slide) –Developed tools assessing compliance with Riel’s set of object- oriented heuristics (based on C++/Java code, UML class diagrams) Design of new methods / models for quality prediction Design of new methods / models for quality prediction –Software Defect Prediction Using Regression via Classification (AICSSA ’06)

18 Apr 2006Sweden26 Decision tree for defect prediction (for a specific data set)

18 Apr 2006Sweden27 Software Cost Estimation The problem: predict the human effort for developing a system The problem: predict the human effort for developing a system –Expert judgment –Analogy based method, project is compared with past projects, project cost is proportional to cost of ‘similar’ projects –Algorithmic / ad hoc models, a set of equations are used (e.g. the COCOMO models) Issues: Issues: –Wrong, inaccurate, missing past projects data –Uncertainty about new project –Many categorical values (e.g. type of system, language used) –Method accuracy –Calibration with local data –Method suitability to local situation (e.g. local project data may be not available or projects are not measured accordingly)

18 Apr 2006Sweden28 Software Cost Estimation (1) Interval estimation Interval estimation –A range of values is produced –More realistic approach w.r.t to a point estimate Interval incorporates model error Interval incorporates model error Interval accounts for estimation data inaccuracy Interval accounts for estimation data inaccuracy –May be easily transformed to a point estimate (e.g. take the mean value of the interval) A number of techniques has been proposed A number of techniques has been proposed

18 Apr 2006Sweden29 Software Cost Estimation (2) Improvement of Analogy Based Estimation (Bootstrap sampling method) Improvement of Analogy Based Estimation (Bootstrap sampling method) –To calibrate method parameters (e.g. number of similar projects used for the estimation) –To produce interval estimates Project portfolio estimation Project portfolio estimation –Combination of interval estimates from portfolio project members Categorical, multinomial, ordinal regression Categorical, multinomial, ordinal regression –Regression models that provide interval estimates –Intervals are often predefined by managers –Produced more accurate results than previously proposed methods (e.g. OLS)

18 Apr 2006Sweden30 BRACE: a tool for advanced estimation by analogy Screenshot of BRACE showing the results of a calibration session

18 Apr 2006Sweden31 BRACE: interval estimate generation New project is estimated to demand an effort of mh (prob. 65,1%), (prob. 23,1%), etc.

18 Apr 2006Sweden32 Software Cost Estimation (3) How to deal with missing data in your historical data base? How to deal with missing data in your historical data base? Imputation methods: replace missing values Imputation methods: replace missing values Proposed Multiple Logistic Regression as an imputation method for missing categorical values Proposed Multiple Logistic Regression as an imputation method for missing categorical values MLR found to perform better w.r.t. to previously proposed methods MLR found to perform better w.r.t. to previously proposed methods

18 Apr 2006Sweden33 Software Cost Estimation (4) Comparative assessment of proposed methods (focusing on AI methods) Comparative assessment of proposed methods (focusing on AI methods) –Association Rules, BBN, CART –AR are found to be the most accurate / promising approach

18 Apr 2006Sweden34 Example Association Rule for Productivity Estimation Association Rule SupportConfidenceRule BodyRule Head [ACAP_H]+[DATA_N]==> [PROD_4] Interpreted as follows: When the programmers’ analysis capability is high and the database size is nominal then the productivity is likely to be in the fourth category (100<PROD<160 KLOC/MM)

18 Apr 2006Sweden35 Software process modeling with iterative Bayesian Belief Networks

18 Apr 2006Sweden36 Software Evaluation Application of MCDA (Multiple Criteria Decision Aid) on software problem situations Application of MCDA (Multiple Criteria Decision Aid) on software problem situations

18 Apr 2006Sweden37 Profile evaluation of software artifacts / processes User defines profiles as sets of attribute values, then product / process is assessed through the Electre method to decide its profile value

18 Apr 2006Sweden38 Software Engineering Education Investigation of novel approaches in SE education through formal experiments Investigation of novel approaches in SE education through formal experiments –Distance learning of UML (sequence diagrams) –Use of a lesson sheet for teaching Usability

18 Apr 2006Sweden39 Open Source Software Engineering / Extreme Programming

18 Apr 2006Sweden40 Free / Open Source Software (1) Revolutionary software development Revolutionary software development Issues: Issues: –What is the quality of F/OSS products? –How can we assess an F/OSS project? Is it active? Is it active? Is it interesting? Is it interesting? –How can we predict an F/OSS project? How will it grow in terms of code size, number of programmers? How will it grow in terms of code size, number of programmers?

18 Apr 2006Sweden41 Free / Open Source Software (2) F/OSS product quality assessment F/OSS product quality assessment F/OSS process modeling F/OSS process modeling –Dynamic models for predicting F/OSS project behavior Knowledge sharing and learning in F/OSS Knowledge sharing and learning in F/OSS

18 Apr 2006Sweden42 F/OSS product quality assessment Case Studies Case Studies –Measured and assessed 100 LINUX applications: applications’ code found to be of at least equal quality w.r.t. to closed source code –Measured and assessed specific software systems’ quality (e.g. assessed COMPIERE code, an Open Source ERP system, found to be of excellent quality) Quality assessment horizontal studies Quality assessment horizontal studies –Measured, assessed 5 F/OSS projects and compared with closed source counterparts using Maintainability Index (SEI) –F/OSS systems found to exhibit superior maintainability, however F/OSS maintainability deteriorated in similar way as in closed source

18 Apr 2006Sweden43 F/OSS maintainability evolution

18 Apr 2006Sweden44 F/OSS process modeling Dynamical F/OSS process model Dynamical F/OSS process model Based on programmer motivation Based on programmer motivation Elaborates on programmer tasks (coding, debugging), modules, defects Elaborates on programmer tasks (coding, debugging), modules, defects Calibrated on typical F/OSS projects (e.g. Apache, Gnome) Calibrated on typical F/OSS projects (e.g. Apache, Gnome) Predicting curves for size (LOCs), number of programmers Predicting curves for size (LOCs), number of programmers

18 Apr 2006Sweden45 Knowledge sharing and learning in F/OSS Modeling of knowledge sharing in F/OSS Modeling of knowledge sharing in F/OSS –Knowledge sharing model –Mining F/OSS mail lists to determine knowledge brokers Learning mechanisms in F/OSS Learning mechanisms in F/OSS –Learning in F/OSS occurs following the constructivist model –Ultimate goal: use of F/OSS projects in SE teaching –Ran a pilot study: 15 students practiced software testing by participating in F/OSS projects, obtained promising results

18 Apr 2006Sweden46 F/OSS Knowledge Sharing Model Knowledge is generated, shared and acquired by traversing various paths of the graph model in many different ways

18 Apr 2006Sweden47 F/OSS Mailing List Social Network

18 Apr 2006Sweden48 Extreme Programming Surveying Agile Methods / XP spread in Greece Surveying Agile Methods / XP spread in Greece –SW SMEs use XP methods without knowing it! Investigation of human factors Investigation of human factors –Developer personality and temperament when forming a pair in pair programming –Conducted formal experiments with students –Found that mixed personalities and temperaments produce better results

18 Apr 2006Sweden49 Forthcoming Funded Research Projects SQO – OSS (IST/EU), Software quality observatory for Open Source, a system that will continuously monitor and evaluate OSS products SQO – OSS (IST/EU), Software quality observatory for Open Source, a system that will continuously monitor and evaluate OSS products FLOSSMETRICS (IST/EU), studies and models for OSS process management FLOSSMETRICS (IST/EU), studies and models for OSS process management F/OSS in education (MINERVA / EU), exploitation of OSS in informatics education F/OSS in education (MINERVA / EU), exploitation of OSS in informatics education DIERGASIA (means PROCESS in Greek) (TELETEL / GSRT), prediction models and procedures for advanced software quality assurance in a TLC systems company DIERGASIA (means PROCESS in Greek) (TELETEL / GSRT), prediction models and procedures for advanced software quality assurance in a TLC systems company Also bidding for funding for various development projects (training in basic computer skills using ODL, portal development for technology museums, …)

18 Apr 2006Sweden50 Current / Future Research topics F/OSS quality assessment through the use of heuristics F/OSS quality assessment through the use of heuristics Social Networks in F/OSS Social Networks in F/OSS –Indicators for community decay in F/OSS projects: when is the project endangered because of mentors / key developers abandonment? Software Architecture Software Architecture –Exploit architectural tactics in component specification to support reusability (ICSR’06, Torino) Management Anti-Patterns Management Anti-Patterns –Modeling with BBN –XP management anti-patterns Intellectual Software Capital Valuation Intellectual Software Capital Valuation –How much is a software artifact (component, method, piece of knowledge) worth? Investigate XP through experiments with professionals Investigate XP through experiments with professionals Introduction of F/OSS projects in SE education Introduction of F/OSS projects in SE education

18 Apr 2006Sweden51 Tactic driven process