1. T. E. Potok - University of Tennessee CS 594 Software Engineering Lecture 1 Dr. Thomas E. Potok potokte@ornl.gov 865-574-0834

2. 2 Software Engineering CS 594T. E. Potok - University of Tennessee Agenda  Class introduction  Why study software engineering  History  Evolving structure  Requirements

3. T. E. Potok - University of Tennessee Class Overview

4. 4 Software Engineering CS 594T. E. Potok - University of Tennessee Course Description  Overview of practical and theoretical software engineering techniques and approaches  2 Small Projects – Class formed into groups – Define requirements, design, prototype code – Simulate industrial software development – 1/2 hour of each class on project  Texts: – None

5. 5 Software Engineering CS 594T. E. Potok - University of Tennessee Course overview  Requirements Gathering and Analysis (1 week)  Project size estimation (3 weeks)  Planning (5 weeks)  Team interaction (2 weeks)  Methodology (3 weeks)  Runaway Projects (2 weeks)

6. 6 Software Engineering CS 594T. E. Potok - University of Tennessee Tests and Grading  Tests and Grading – Mid-term exam 30% – Final exam 30% – Homework (including projects) 40% – Last year roughly 30% of the class made an A, the rest received a B.  Office Hours – 1 hour before/after class by appointment – Email or phone calls are fine

7. 7 Software Engineering CS 594T. E. Potok - University of Tennessee Class format  Professional conduct – Class starts and ends on time – Contact me if you are going to miss a class – I will most likely miss one or two classes  Schedule – 50 Minutes lecture - 10 Minute break – 50 Minute lecture - 10 Minute break – 10 minute Summary – 30 minutes project time  Guest – There will be several guests in the class, I expect them to be treated with the utmost respect

8. 8 Software Engineering CS 594T. E. Potok - University of Tennessee Projects  The class will develop components of software to meet the needs of two real customers with hypothetical problems – Using the techniques learned in class – Working in groups – Simulating a realistic software development environment

9. 9 Software Engineering CS 594T. E. Potok - University of Tennessee Why take this class?  IEEE Computer 5/2000 “What Knowledge is Important to a Software Professional?” T. C. Lethbridge 1. Negotiation 2. User Interfaces 3. Leadership 4. Real-time system design 5. Management 6. Software Cost Estimation 7. Software Metrics 8. Software Reliability and fault tolerance 9. Ethics and professionalism 10. Requirements gather and analysis

10. 10 Software Engineering CS 594T. E. Potok - University of Tennessee What we will cover  Leadership - High  Management - High  Software Cost Estimation - High  Software Metrics - High  Requirements gather and analysis – High  Negotiation - Medium  User Interfaces - Medium  Software Reliability and fault tolerance – Medium  Ethics and professionalism – Low  Real-time system design - None

11. T. E. Potok - University of Tennessee Why Study Software Engineering?

12. 12 Software Engineering CS 594T. E. Potok - University of Tennessee Why study software development?  Just hire the best people you can, and let them go  Look at any successful project, and you will find that one key person who was willing to do what ever it takes to succeed.  You can document processes and techniques all day long, no one ever follows them.  “Software development is more of an art form than engineering”.

13. 13 Software Engineering CS 594T. E. Potok - University of Tennessee Have programmers gotten better?  Do you think that the value added to a corporation by the typical software engineer has increased in the last 20 years?  If so, by how much?  Abdel-Hamid notes spending on application development tools has increased at a 19 percent annual growth rate or higher,  However, looking at the value that was added per software developer adjusted for inflation, shows no improvement in the past two decades [Abdel-Hamid (1996)].

14. 14 Software Engineering CS 594T. E. Potok - University of Tennessee Productivity Crunch  70’s 3-5 year development cycles  80’s 2-3 year development cycles  90’s 12-18 months – 6 week feasibility studies are common – “Web year” 3 months – My two most recent software research projects were 4 months, and 9 months in duration

15. 15 Software Engineering CS 594T. E. Potok - University of Tennessee High Stakes Development  IRS - Computerworld: IRS wastes $50 billion per year resulting from repeated failures to integrate its "stovepipe" computer systems.  Denver baggage claim system 16 months late, 2 Billion dollars over budget  Air traffic control...

16. 16 Software Engineering CS 594T. E. Potok - University of Tennessee There has got to be a better way  Technology - Better tools and languages  Methodology - Structured analysis, Object-oriented, agent-oriented,...  People - Better trained, better paid (???)  Process - CMM, ISO 9000  Better management - Focus on deliverables, not people

17. 17 Software Engineering CS 594T. E. Potok - University of Tennessee What is software development?  Engineering?  Art?  Craft?  Science?

18. 18 Software Engineering CS 594T. E. Potok - University of Tennessee Software Development  Art – For entertainment and enjoyment – Can make a statement – Creator extremely significant – One of a kind, unrepeatable  Engineering – Based on scientific foundation – Can be very creative and innovative – Process followed is significant (patents) – A repeatable process can be define, followed, and standardized

19. 19 Software Engineering CS 594T. E. Potok - University of Tennessee Software Development  Craft – Mainly for function – Artistic features – Craftsman important – Elements are repeatable  Science – Mainly for discovery – Based on time honored method – Scientists reputation is very important – Discovery needs to be reproduced

20. 20 Software Engineering CS 594T. E. Potok - University of Tennessee Why most software is not art  Think of two outstanding programmers, and two great artists, Picaso and Monet. – The programmers can both produce software to solve a problem, and it will not be obvious who wrote what. – The artists can paint a vase, it is clear who painted what  A program that works better than others will replace the competition. Obsolete software is useless.  A Picaso and a Monet are not replaceable. They gain value over time.  A group of skilled programmers can duplicate any software currently on the market given time and money  A skilled artist can duplicate a Picaso, or a Monet, to fool most experts, but it has little value.

21. 21 Software Engineering CS 594T. E. Potok - University of Tennessee Software as craft  Similar to art, craft work is distinctive based on the talent of the craftsman  Handmade work is more valuable than machine made work  With software it is hard to tell who wrote what, or whether the code was hand crafted, or machine generated.  Are you interested in the credentials of the people who write the software you use?

22. 22 Software Engineering CS 594T. E. Potok - University of Tennessee Computer “Science”  Scientist are interested in problems that have not been solved  They show a solution to a problem which is often little more than a prototype  Most software solves a problems that have been solved many times before  Software that is not industrial strength is of little value

23. 23 Software Engineering CS 594T. E. Potok - University of Tennessee So Software Development is Engineering  What science underlies software development? – Mathematics?  Is standardization in progress? – Windows, HTML, Java…  Programmers seem more influenced by style than convention

24. 24 Software Engineering CS 594T. E. Potok - University of Tennessee So who cares?  If software is art – The artist’s creatively is the key to producing software  If a craft – Learning is done through apprenticeships – Ability is more important than education  If software in engineering – There is a process to it that can be Repeated, and can be Improved over time

25. 25 Software Engineering CS 594T. E. Potok - University of Tennessee Brief History of Software Engineering  Nato conference in 1968 – Software should follow an engineering paradigm  Brook’s Mythical Man Month – Experiences from the development of the IBM 360 operating system – Brook’s law

26. 26 Software Engineering CS 594T. E. Potok - University of Tennessee The 60’s  Remember ‘2001 a Space Odyssey’? One large intelligent computer running a spaceship...  In the 60’s and 70’s software was written to solve some very basic problems, such as how to tell the computer to manage it’s own resources.  The trick in those days was to actually get the software working.  Computer time was scarce and the computers themselves were quite limited, however, there seemed to be no bounds on what a computer could do.

27. 27 Software Engineering CS 594T. E. Potok - University of Tennessee Programming in the 70’s  Shift from mainframe computers to mini-computers.  A change in understanding what computer can really do. – The hardware gets faster, smaller, and cheaper, while the software becomes more complicated with less promise.  Software was written by elite, well trained, well paid programmers who worked for industry.  The most expensive part of developing software was now the programmer’s time, not the computer time.  Customers of this software were elated if you ignored only 90% of their requirements, and were a mere 6 months late

28. 28 Software Engineering CS 594T. E. Potok - University of Tennessee Programming in the 80’s  The start of the personal computer revolution, PC on desk tops, and even in homes.  A college dropout could work in his garage with a clone PC developing and sell software.  Virtually anybody with a home computer could develop software in his or her garage or carport and sell it on the open market.  The trick was no longer merely to get it working, but now to make it useful and useable.  Software was now sold in stores for the average user.

29. 29 Software Engineering CS 594T. E. Potok - University of Tennessee Programming in the 90s  Large and small powerful computers connected throughout the world that can communicate with each other.  Develop software faster, with more function, and cheaper than the competition.  Like in the old garage computing days, anyone can develop a software package, but now, they can do it anywhere in the world.  However, – there are not as many interesting problems to solve anymore. – Either they have been solved many times over, or no one can figure out how to solve them.

30. 30 Software Engineering CS 594T. E. Potok - University of Tennessee Summary  The software development environment has changed quite a bit over the past 30 years.  Lessons learned in the 60’s and 70’s may be irrelevant to current software development, or they may be fundamental principles of creating software.  Without a yardstick to measure the results of such concepts, it is merely one opinion against another.

31. T. E. Potok - University of Tennessee Software Process Improvement

32. 32 Software Engineering CS 594T. E. Potok - University of Tennessee History  Deming is legendary for advising the Japanese to focus on process to revolutionize their post-war manufacturing.  Japanese products out performed American products mainly due to quality and ability to meet customer needs  This approach was adopted by a variety of industries, and was seen as a way to revolutionize software development as well.  You define how you do something (a process), – Next, you repeat this process for every project that you develop – Then you measure and analyze the process, – Finally make any improvements necessary.  This should eventually improve the quality of the software produced and increasing the productivity of your programming teams.

33. 33 Software Engineering CS 594T. E. Potok - University of Tennessee Why Process?  Management wants a forecast of how many widgets will be produced  How can this be accurately done without: – Knowing the steps required to make a widget – The machine capacity needed – The skills needed – The material needed  It can not be, therefore any forecast is little more than a random number

34. 34 Software Engineering CS 594T. E. Potok - University of Tennessee Maximize output?  Once a process is understood, then what – Maximize process output? No – Maximize process consistency!  Is is better to consistently produce a widget in 8-16 days, or 5-20 days.  This consistency provides for greater control over the process.

35. 35 Software Engineering CS 594T. E. Potok - University of Tennessee Variance Reduction What curve do you want your team to follow?

36. 36 Software Engineering CS 594T. E. Potok - University of Tennessee SEI Capability Maturity Matrix  Broadly agree to define how a software organization matures and improves  Based on manufacturing process improvement and “best practices” from software engineering  Some dramatic successes...

37. 37 Software Engineering CS 594T. E. Potok - University of Tennessee Capability Maturity Matrix  Developed by the Software Engineering Institute (SEI) by Watts Humphry and Mark Paulk.  Five levels of maturity for an organization – Level 1 - Initial; – Level 2 - Repeatable; – Level 3 - Defined; – Level 4 - Managed; – Level 5 - Optimizing.

38. 38 Software Engineering CS 594T. E. Potok - University of Tennessee Initial  Poorly defined procedures and controls  No management mechanism to to ensure they are followed  Heroic efforts by one or two people saves the day.  Projects are late, crisis to crisis

39. 39 Software Engineering CS 594T. E. Potok - University of Tennessee Repeatable  Basic project controls  Quality problems  No framework for orderly improvement  Fault data is being collected

40. 40 Software Engineering CS 594T. E. Potok - University of Tennessee Defined  Commitment to software process evaluation and improvement  Appropriate software engineering standards and methods are in place  Strong qualitative understanding of the process

41. 41 Software Engineering CS 594T. E. Potok - University of Tennessee Managed  Process is quantified  Quality and productivity measured for each key task  Wide dissemination of process related information  Errors can be predicted with acceptable accuracy

42. 42 Software Engineering CS 594T. E. Potok - University of Tennessee Optimizing  Process improvement feed-back and feed-forward controls  Rigorous defect causal analysis and defect prevention  Proactive management

43. 43 Software Engineering CS 594T. E. Potok - University of Tennessee More on CMM  There are 18 key process areas defined by CMM, including – Requirements Management, – Software Configuration Management – Process Change Management – Defect Prevention  Each key process area has five common features: – 1) goals to be achieved; – 2) ability to perform; – 3) activities performed; – 4) measurement and analysis; – 5) verification

44. 44 Software Engineering CS 594T. E. Potok - University of Tennessee Level of an organization  Levels are determined by audit.  Like ISO 9000, the program can be of great value, or can be done to merely reach a level.

45. T. E. Potok - University of Tennessee Requirement Analysis

46. 46 Software Engineering CS 594T. E. Potok - University of Tennessee Three basic types of requirements  Requirements against an existing product – Make a change or modification to an existing product  A new project with requirements directly from a customer – Building a project for a specific customer  Market requirements for a new product – Building a new product that the market may need

47. 47 Software Engineering CS 594T. E. Potok - University of Tennessee Requirements against an existing product  A very structured environment  Requirements are usually clearly understood  Determining the priority of the requirements is the challenge. – Not implementing a much needed requirements can could a loss of market share – Implementing trivial requirements may result is a wasted investment – Assessing priorities will be covered later in the course

48. 48 Software Engineering CS 594T. E. Potok - University of Tennessee New project requirements  Customer need special software built for his or her need  This need is typically described informally to the programming team  The programming team then attempts to transform the requirements into running code

49. 49 Software Engineering CS 594T. E. Potok - University of Tennessee How to gather and record requirements  Traditional approach is the JAD (Joint Application Development) session  Domain experts are taught rudimentary data modeling and data flow diagramming techniques, then lead by an expert into developing a design  A beginning design can be developed in a few days

50. 50 Software Engineering CS 594T. E. Potok - University of Tennessee JAD Sessions  Entity relationship modeling is used to capture the data requirements – These ER models are then translated into a database definition  Data flow diagrams are used to capture the functions that are needed by the system – These DFDs are then translated into functional designs

51. 51 Software Engineering CS 594T. E. Potok - University of Tennessee Entity Relationship Modeling StudentClass 5,m1,m Teacher 1,m 1,1 Name ID number Classification Major Name Number Description Department Credit hours Student limit Name Department Grade Enrolls Teaches

52. 52 Software Engineering CS 594T. E. Potok - University of Tennessee Dataflow Diagrams Enrolls Student Class Class list Assign Teacher Class Enrollment Flag

53. 53 Software Engineering CS 594T. E. Potok - University of Tennessee OO approach  Another methods is the use of scenarios, or use cases to gather requirements  The customer deals in his or her domain describing what the computer system should do.  The programmer needs to understand the basics of the domain, and work through inconsistencies or problems in the scenarios.

54. 54 Software Engineering CS 594T. E. Potok - University of Tennessee Use Cases  Use cases can be derived from a requirements document, or with the help of the customer  The use cases stress – Inputs and outputs – What the computer does – How use cases relate to each other

55. 55 Software Engineering CS 594T. E. Potok - University of Tennessee Example Register Starts: When a student want to register Ends: When a student registers for a class Actors: Student Scenario: Enter the registration system (separate use case) Select register Enter student number select department the class is taught by select the specific class to register for receive confirmation of registration Teach Starts: When class begins Ends: When class ends Actors: Teachers and student Scenario: Link to course server Adjust live video view the class disconnect from server

56. 56 Software Engineering CS 594T. E. Potok - University of Tennessee New product  Survey the market to understand what is needed and how much it will sell for  Survey the competition, what are their strengths and weaknesses  Develop the product based on what is learned from this analysis

57. 57 Software Engineering CS 594T. E. Potok - University of Tennessee Common Problems  The customers does not know what they really want  The customers know what they want, but it is impossible, or impractical to achieve  The customers change what they want after development has started  The programmer do not understand the requirements

58. 58 Software Engineering CS 594T. E. Potok - University of Tennessee The customers does not know what they really want  Often business problems shift in importance or scale  A critical problem two months ago, may be uninteresting today  There may not be enough of an understanding of the problem to know what a good solution would be

59. 59 Software Engineering CS 594T. E. Potok - University of Tennessee The customers know what they want, but...  Many customers are not very familiar with software  Having a computer understand a paragraph of text does not seem too difficult  Or to have a computer learn over time

60. 60 Software Engineering CS 594T. E. Potok - University of Tennessee How to figure out what the customer wants  Interviews  Surveys  Market research  Prototypes

61. 61 Software Engineering CS 594T. E. Potok - University of Tennessee Summary  Software is probably most like engineering, but it is not an exact fit  Productivity demands of software development keeps increasing  CMM is a way of assessing and improving the way that software is developed  Requirements are essential to building effective software