1. 1 A Software Engineering Institute for the Victorian Software Industry A Re-useable Case Study*... Chair IEEE-Computer Society Tech. Council on Software Engineering Governor, IEEE-Computer Society(1997-1999,2000-2002), Director, Computer Sys. & Software Engineering Board, ACS, Department of Computer Science & Computer Engineering, La Trobe University Hon. Visiting Professor, Middlesex University *A summary presented to the Sheffield-Hallam Univ. Feb. 2001 by Assoc. Prof. Karl Reed,FACS, FIE-Aust., MSc,ARMIT

2. 2 Why Were We Proposing VSEI?  Promote government policy in information technology ensuring support for an already successful industry  Ensure that the industry has a major research institute  Ensure ‘world’s best practice’ is understood and adopted here  Twenty years from now  a permanent structure delivering an increased competitive edge to a major exporting industry!

3. 3 Australian Political Reality  Information Technology Sector --> Primary Industry share of GDP  Extensive Gov’t R &D for Primary Sector..  10 Divisions of CSIRO, State R & D inst., CRC’s  ~15 Govt.-Industry Funded R & D Corps for Primary Industry  Estimated relative short-fall ~A$700M.p.a (£300M.p.a)  Few Public sector large-scale “professional” research groups  Universities regarding R & D as their domain.. means of cross-subsidising teaching(?) How does this compare with the UK?

4. 4 What was our strategy?  “Forcing” argument based on comparison with Primary Industry and Govt. rhetoric  Linkage with industry  Credible proposal …. to deliver world’s best practice  Twenty years from now  a permanent structure delivering an increased competitive edge to a major exporting industry!

5. 5 Goals  Guarantee the long term technical viability of a major IT industry sector  the software and services industries  Provide equivalent R & D support as that available to overseas competitors  Provide professional research capability advancing the discipline

6. 6 Industry-Academia Working Party Meeting since May 1997  Chairman  Karl Reed, La Trobe University  Deputy Chairman  Paul Radford, Managing Director Charismatek  Working party  Silvio Salom, Managing Director Adacel  Laurie Lock Lee, Manager Planning and Development BHPIT  Robyn Lawrie, Technical Director Charismatek  Alex Sawicki, Department of State Development/Multimedia Victoria  Gary Stoneham, Marketing Manager MITS  Sally Duncan, Project Manager Megatec  Barbara O’Brien, Quality Manager Megatec  Bill Jacobs, Technical Director TUSC Computer Systems  David Cleary, La Trobe University  Special advisor  Dan Marantz, Cobol Digital  Observers  The Preston Group  Whittle Programming  KCS Computer Services  Clive Finkelstein, Information Engineering

7. 7 Academic Partners with industry links  Karl Reed, Amdahl Project (Case, Metrics, Evolvable Programming)  Tharam Dillon, La Trobe University OO, Relibility, Metrics  T.Y. Chen, Swinburne Univ. Tech., IVE/STC HK Testing and Software Quality  Paul Bailes, Univ. Queensland. Software Maintenance Centre  HK opportunity for participation

8. 8 Not Seeking Silver Bullets  We’ve all heard of  goto-less programming, structured analysis and design, CASE, object orientation, CMM, SPIN  ???  all trumpeted as ‘the solution’  This proposal  “a considered holistic response to a series of difficult problems”  not a “quick fix”!

9. 9 An Appropriate Solution A Small to Medium Enterprise Bias  Only one of its kind in the world, other SEIs  client domains generally not the software industry  focus on embedded systems, telecom and defense aerospace  pre-occupied with process issues  Our objectives  commercial outcomes, deliverables to industry  not just academic  Funding for technology transfer

10. 10 World’s Best Practice  Aggregated research teams ~ researchers 40 + ~20 PG's  Funded technology transfer  Industry driven collaboration (including management)  Cash driven budget (mostly Government)  Single point of control

11. 11 Scale Comparable to World's SEI's  Overall  A$2M pa to A$30M pa (combined Esprit is much larger)  Fraunhofer Institute for Experimental Software Engineering  DM20Mpa (goal)  Centre de Recherche de Montreal (CRIM)  C$18M pa (total)  C$3M pa (software engineering)

12. 12 International Funding Models  Overall  typically less than 50% non-government funding, often as low as 30%, sometimes zero  funds often obtained from government sources other than granting agency  funding models  dominant mode; core funding (up to 70%) guaranteed by government agency with allowances for ramp-up  100% government funding  grant based funding

13. 13 International Funding Models  Fraunhofer Institute for Experimental Software Engineering  Fraunhofer Gesellschaft 40%  regional government 10%  future goal; non-Fraunhofer funding to be 70% (ramp up conditions apply)  joint projects; seek 50% involvement (people preferred)

14. 14 SEIs Around the World a Brief Summary  Client domain  generally not targeted at the software industry  mainly ‘embedded’ systems, telecom and defense aerospace (Korea and Taiwan and some Esprit projects are software industry)  Modus Operandi  networks, multi-project granting (CNRC, CRIM), academic driven, research driven, captive-client (SEI), in house, large-scale (Fraunhofer), etc

15. 15 SEIs Around the World a Brief Summary  Areas of investigation  all areas of software engineering...  Technology transfer  seminars to technology trials and ‘leverage’ activities (often funded by agency), experiments (PIEs), intellectual property handover  Research styles  single project (Esprit), short, medium, long- term, team, consortia, distributed consortia

16. 16 SEIs Around the World a Brief Summary  Outcomes  process improvement, methodology, tools, some product (Esprit varies enormously), intellectual property  Intellectual property  all models; shared by partners, client owned

17. 17 SEIs Around the World a Brief Summary  Participants (primary funding sources)  single agencies (SEI, Fruanhofer, etc), multiple agencies (European SEI), state governments (Italy, Canada, Germany), multi-national programs (Europe), very few consortia with upfront industry funding

18. 18 SEIs Around the World a Brief Summary  Research agenda determination  ‘agenda setting’; agency attempts to inject new technology and best of breed practice (NSF, SEI, DARPA)  ‘responsive’; industry set research agendas  ‘academic interventionist’; academia propose research projects  ‘joint’; academia and industry  ‘empiricist’; agendas derived from study of practice

19. 19 SEIs Around the World Fraunhofer Institute for Experimental Software Engineering  Funding  Fraunhofer Gesellschaft 40%  regional government 10%  future goal; non-Fraunhofer funding to be 70%  local government 50% of new building  special ramp-up conditions apply  Scale goal DM20M pa

20. 20 SEIs Around the World, Centre de Recherche de Montreal (CRIM) Software Development Tools and Methods (SDTM)  Funding  about C$3M pa for software engineering  Quebec  Scale total CRIM C$18M pa  Client domain  embedded systems, aerospace, telecom, electricity supply, software tool builders

21. 21 SEIs Around the World, Centre de Recherche de Montreal (CRIM) Software Development Tools and Methods (SDTM)  Modus operandi  joint academic-industry projects  studies of tools, methods etc for clients  specific research projects.  participation sought  technology monitoring  projects seem small

22. 22 SEIs Around the World, Centre de Recherche de Montreal (CRIM) Software Development Tools and Methods (SDTM)  Areas of investigation  methodology improvement  re- and reverse engineering  improvement of practice, including metrics, standards and SQA  object orientation  domain-specific architectures, reuse  real-time systems, including formal methods

23. 23 Some ESPRIT Research Projects  Legacy Assessment Workbench (LAW)  areas (aerospace, nuclear)  reengineering, metrics workbench for C legacy code (safety critical systems)  symbolic execution, formal methods  outcomes  prototypes for evaluation and adoption

24. 24 Where Are We?  Bowsers that are limited  Time-To-Market web-application deployment  16 year old wunder-kinder throwing systems together  rapidly deployed functionality NO PROBLEMS MATE! DISASTER IN WAITING!  poorly designed functionality  rapid evolution of systems to meet customer needs  conventional approaches being left behind  the old do not understand the new Traditionalist’s View Modernist’s View

25. 25 “F1. Current software has too many surprises. The sources of surprise are poorly understood.” Sources of surprises... Real and apparent ambiguity in the means of representation of systems, e.i. Languages (cf 3 pages of c++ with 3 pages of government regulations)(Reed, 2000) “F2.Key sources of software surprise include immature or poorly integrated software domain sciences, construction (product) principles, and engineering processes. Software research emphases have swung from process to product research, with weak coverage of domain sciences and integration.” To many surprises….!!! (nsf report on s/w research 1998)

26. 26 “F1. Current software has too many surprises. The sources of surprise are poorly understood.” Sources of surprises... Real and apparent unpredictability in behaviour... No surprises….!!! (nsf report on s/w research 1998) “Teenagers have less trouble with PC software because they are adept at playing computer games” Charles Wright, editor Melbourne Age “green pages” computer section 2000 “Building ‘bots’ that play computer games with near human competence is not that hard” US researcher in AI….

27. 27 By way of Illustration...Some Contradictions…… and confusion 2. Software Process.. CMM vs fine-grained process independent, Time To Market vs Planned Process, Phase incompletedness, Extreme Programming. 3. Software Process... Often mandated, but not followed… few detailed studies similar to production engineering (see Hess) 4. Re-use… not successful, yet components industry emerging 5. Engineering & SE.. Poor choices of analogues from traditional domains, e.g. “immutable components” 1. Software Architecture.. ‘not immutable, not always determinable a’priori,multiple versions in one artefact, retrofitable…. Analog with “built” systems not clear.

28. 28 Some Contradictions…… and confusion (cont’d) 7. Prescriptive Design processes... only slowly beginning to appear, perhaps via UML. 8. Requirements Engineering... Cannot always be completed in advance..may be continuous part of the implementation process... 9. Software Crisis… yet increasingly, successful large-scale applications are ubiquitous 10. High Quality training for 30 yrs.. Yet each new s/w development wave starts with a blank mind, e.g. web-based computing 6. SWEBOK.. Organised body of knowledge opposed by leading SE players. 11. Documentation matters but.. It’s seldom actually done

29. 29 Approaching Software Developers…  Be able to show ROI after adoption costs (equipment + training) and productivity losses due to learning curves after adoption. (improved profit)  Show resolution of competitive advantage problems (beat off competitors, maintain market share)  Show new market opportunities due to new products/services / Technico-Commercial Drivers… the linkage / Show an economic benefit  The goal is to find a high-level, one-line statement of pressing commercial issue that maps directly on to a “technology acquisition” (research) agenda (map idea to common concept base accessible to highest management)

30. 30 Typical SE Research Agenda Australia ~ 1997 1.Re-engineering and Empirical Studies of s/w Practice, 2.Tools and Methodologies, and Design Representation, 3. Re-Use, 4. Evolving Software, 6. Object Oriented Dev. 7. Product Quality Measurement 8. Time-to-Market 9. Testing ¶ Impact of developments in run-time platforms ¶ Low-cost and evolving software ¶ User Interface Development ¶ Software Productivity ¶ Performance Predictability ¶ Software Product Quality Certification ¶ Time to Market Technico-commercial Drivers Research-Commercial Mapping… Defining Relevance

31. 31 / The ANSEI Technico- Comercial Driver to Research agenda mapping

32. 32 Organisational Proposal Government Industry Academia Research Experience Industry Experience Technical Support Bus Man CEO Technical Board Management Board Stakeholders Personnel

33. 33 Research Agendas Providing Solutions Over-Arching Goals  Our research outcomes are methodologies with the following properties, these in fact become research objectives  performance-based, predictable, development of systems to stated performance requirements  fine-grained-prescriptive, provide precise prescriptions for steps in development  improved technology and expertise for re- engineering of existing systems  study of existing systems and projects, using re- engineering, design recording

34. 34 Research Agendas Providing Solutions Over-Arching Goals  Issues of re-use, re-use intensive and re-use based methodologies  evolving software (current agenda of DARPA)  object oriented methodologies, will influence, and be influenced  engineering of interfaces, part of the methodology program  Plus, tools that reflect this!

35. 35 Characterising Time to Market  Delivery schedules severely truncated compared to ‘normal’ (less than 50%)  How would we deal with this currently?  RAD/JAD, prototyping, ‘super programmers’  What would the product be like?  slow, unreliable, incomplete expensive!  Research goal  convert this to a methodology capable of delivering quality products

36. 36 Characterising Time to Market  Research problems  for a given project  a minimum feasible delivery time t dmin  cost = k t dmin  quality greatly reduced  competitive position increased! -n

37. 37 Time to Market Project Attributes AttributeStandard Development (under control) Schedulecontrolled acceptable Costcontrolled acceptable Qualityhigh Customer high satisfaction Runtime minimal resources Designhigh quality Time to Market (current situation) truncated unpredictable uncontrollable high usually low usually resentful excessive poor Time to Market (target situation) truncated predictable acceptable predictable high predictable high

38. 38 Research Questions  Schedule  What is an optimal/reasonable project schedule?  Is it feasible to attempt a project within a specific timeframe?  Resources  How can available human resources best be allocated to ensure projects succeed?

39. 39 Research Questions  Factors determining schedules  estimating  staff quality and experience  methodology  re-use  tools

40. 40 Research Activities & Outcomes  Determine existing best practices  collaborative work with institute partners from industry and academia  Develop models, methods, tools and methodologies  Assess models, methods and tools  field assessments using real projects  internal assessments using institute projects  Disseminate knowledge gained  field application of methods and tools with institute partners, publication

41. 41 Research Overview IssueFactorsApproachOutcomes ScheduleImpact of scheduleData collectionCalibrated new reductionProcess recording exemplarestimating techniques projects Codification of known models (eg Microsoft) Risk assessmentIdentification of risk indicators Tool assessment Resource allocationImpact on projectData collectionImproved planning planningProcess recordingmethods Impact of decomposition models and parallel implementation StaffQualitySkill identification, fineImproved selection grained classificationtechniques ExperienceIdentification and recordingTraining need of experienceidentification Project structure

42. 42 Research Overview IssueFactorsApproachOutcomes MethodologyImprovedAnalysis RAD/JADHigh quality prototyping productivityConversion of prototyping to product development Identify essential features of usability Application generators Ultra high skill levels Re-useAreas ofHow to achieve this?Components applicationWhat is re-use?Plans Assess current re-user Designs practicesTest plans Importance of experience ToolsCASE versusWhat do we really need here?Tool set selection lightweightTool integrationNew tools Project managementAnalysis of existing toolsChoice and planningIdentify processes LanguagesReview and recommendation Process/design recording

43. 43 5. Current State of Knowledge and Practice-MS vs the Rest..(cont’d) history  Dijkstra and THE OS in the 70’s (the lesson?)...  “Five people as smart as Edgar Dijkstra can do anything” Reed, 1981  The first Unix effort …(but what did it take for product versions)  OS/360, PL/I (the lesson?) (60’s)..  Very large teams can build large systems very quickly..~ x1000 person years  Total volumes of functionality (e.g. OS/360) may allow partitioning..TTM issue obscured..

44. 44 / “Extreme programming”? System Test Programming Unit Test Program Design Systems Analysis Feasibility Study Requirements Analysis System Integration Optimal task allocation, observed <1970 one or two people Waterfall S/W Process Model / No need for ‘third- party” readable work products! / Private s/w process? (PeSP compliant?)

45. 45 Time to Market Conclusion  Massive competitive advantage from time to market products with traditional high quality  Necessary elements of time to market focussed processes may be accessible  Detailed ‘research’ by experienced staff with access to current practice is most likely to be successful

46. 46 The role of re-engineering.. S/W Archaeology and S/W Architecture.... / recovery of standard architectures / identification of s/w construction practices, e.g. shifts from one programming style to another § architectural styles / development of maintainability and evolvability classifications for -- / development of maintainability and evolvability classifications for architectural styles § design methodologies

47. 47 component semantics and concept extraction.. The role of re-engineering.. Architecture issues for the S/W Archaeologist / identification of design approaches which ensure that conceptual architectures are transferred to implementation / identification of standard mappings from conceptual to actual architectures which occur using different design approaches on different problems

48. 48 Conclusion  The proposal didn’t win..beaten by SEA.. (Problem ridden.. Consortium)  Is there a need for an SEI in the UK?  Surely more than one)  Do the arguments “map”?  How will the academic community react? (SE research in the US did not stop because CMU got the SEI…)

49. 49 Technico-Commercial Problems Facing the Software Industry  Hardware-software run-time environments-adding GUI etc to legacy systems  ‘Evolving’ software dynamically  ‘Engineered’ user interfaces  Software productivity  Design to predicted performance  Time to market  Software quality  Testing  Object orientation  ‘Net-centric’ systems- autonomous communicatings "agents"

50. 50 Technico-Commercial Problems Facing the Software Industry  Developments in hardware-software run- time environments  continuing improvements in price/performance + GUI  need to migrate legacy systems to these platforms  need to add functionality to legacy systems on existing platforms, e.g GUI, client-server  Re-engineering of legacy systems to conform to these constraints a major business opportunity

51. 51 Technico-Commercial Problems Facing the Software Industry  ‘Evolving’ software dynamically  software capable of rapid change to meet existing customers changing needs  software capable of rapid change to meet needs of new customers  New development methodologies which yield ‘evolving’ software

52. 52 Technico-Commercial Problems Facing the Software Industry  ‘Engineered’ user interfaces  UI design derived from data presentation and processing + ergonomics  Methodology supporting improved UI design

53. 53 Technico-Commercial Problems Facing the Software Industry  Software productivity  increase productivity to produce PC shrink- wrapped software  reduce costs for NC/style delivery platforms,  reduce maintenance costs  New diagramming systems  design recording  new methodologies enforcing re-use

54. 54 Technico-Commercial Problems Facing the Software Industry  Software Quality Testing and Certification  Improve efficiency of specfication-based testing  Develop means of code-quality control and asessment  Improve design to simplify "testablity" of code

55. 55 Research Agenda Providing Solutions Main Items  Our research agendas address the technico- commercial problems  reengineering and empirical studies of software practice  tools, methodologies and design representations  re-use  evolving software  engineering of user interfaces  object oriented development  product quality measures  time to market