1. Copyright 2003-2013, Dennis J. Frailey Software Engineering Best Practices – Things You Seldom Learn Much About in School Dennis J. Frailey ACM Distinguished Speaker Frailey@ACM.ORG This talk discusses several practices that make a big difference for real software development projects

2. Copyright 2003-2013, Dennis J. Frailey The Distinguished Speakers Program is made possible by For additional information, please visit http://dsp.acm.org/http://dsp.acm.org/

3. 3 Copyright 2003-2013, Dennis J. Frailey 3 Objective  To present some basic techniques used in professional software development projects –Some of these are not widely covered in academic computing programs

4. 4 Copyright 2003-2013, Dennis J. Frailey 4 Would You Buy This Car?  The price is 50% more than what you agreed to pay  Delivered to the dealer 3 months late  Not the right color  Some of the options you selected aren’t there ( “we’ll install them later” )  The speakers and the CD player are incompatible, so you can only use the CD player through headphones  Although “new”, it has 5,000 miles on it from the extensive testing that was done  Despite that testing, it still won’t start if it’s below freezing and it has a number of other problems

5. 5 Copyright 2003-2013, Dennis J. Frailey 5 Frequent Problems for Product Developers on Real Software Projects  Nobody knows what work must be done –Do we need to provide an installation guide? –Is there a warranty on this software?  and what should we do about it if there is? –In what format should we deliver the code? –Do we have to provide training for the end users?

6. 6 Copyright 2003-2013, Dennis J. Frailey 6 Frequent Problems on Real Software Projects (continued)  Vague understanding of the big picture –Has the customer used a computer before? –How does the software fit into the overall customer environment?  Are the networks and computers suitable for the software? –How will the customer use the software?

7. 7 Copyright 2003-2013, Dennis J. Frailey 7 Air Traffic Control Replacement System The New System The Old System

8. 8 Copyright 2003-2013, Dennis J. Frailey 8 Frequent Problems on Real Software Projects (continued)  Unclear understanding of how much has been accomplished and how much is left to do –“How much longer will it take?” –“How much more money is required?”

9. 9 Copyright 2003-2013, Dennis J. Frailey 9 Frequent Problems on Real Software Projects (continued)  Ignorance about risks –What are our biggest potential project problems? –Do we have any idea what the real risks are? –Do we know which ones to worry about the most? –Can we mitigate the risks? (make them less likely or lower their potential impact) –Do we have a contingency plan? (what to do if the risks happen?

10. 10 Copyright 2003-2013, Dennis J. Frailey 10 Frequent Problems on Real Software Projects (continued)  Confusion about different versions of software –Which version of the data base goes with the latest version of the user interface? –How come it does one thing in the customer’s site but something else in our testing lab? –Does the design match the code? If not, which one is right?

11. 11 Copyright 2003-2013, Dennis J. Frailey 11 Frequent Problems on Real Software Projects (continued)  Haphazard approaches to quality –“We didn’t have time to test it thoroughly” –“We tested it for weeks but it still had lots of bugs”

12. 12 Copyright 2003-2013, Dennis J. Frailey 12 Best Practices for Managing These Problems  The Work Breakdown Structure –Documenting and organizing the work to be done  Estimating and Tracking –So you know where you are  Systems Engineering –Designing the big system before designing the software  Risk Management  Configuration Management  Quality Assurance

13. 13 Copyright 2003-2013, Dennis J. Frailey 13 The Work Breakdown Structure Definition: A work breakdown structure is a hierarchical list of the work activities required to complete a project. This includes tasks for: - Software development - Management of software development - Support of software development - Any other activities required to meet customer requirements, such as: –creation of documents, –training programs, –tool development or acquisition, –travel, etc. Parser Code Generator File System Run Time System User Interface Manage Software Development Build “C” Compiler Build Test Suite Write Documentation Write Installation Software Software for “C” Compiler

14. 14 Copyright 2003-2013, Dennis J. Frailey 14 Sample Work Breakdown Structure for Taking a Course 1.Take a Course on Programming in Java 1.1 Do Assignment 1 – Simple Program 1.2 Do Assignment 2 – Program with Arrays 1.3 Take Midterm Exam 1.4 Do Assignment 3 – Program with Pointers 1.5 Do Assignment 4 – Program with I/O 1.6 Take Final Exam These are the tasks you must complete in order to finish the course

15. 15 Copyright 2003-2013, Dennis J. Frailey 15 Sample Work Breakdown Structure Hierarchy Allows Additional Detail 1.Take a Course on Programming in Java 1.1 Do Assignment 1 – Simple Program 1.1.1 Study Chapters 1 and 2 of textbook 1.1.2 Write a Simple Java Program 1.2 Do Assignment 2 – Program with Arrays 1.2.1 Study Chapters 3 and 4 1.2.2 Review Graded Assignment 1 1.2.3 Write Java Program with Arrays 1.3 Take Midterm Exam 1.3.1 Study Chapters 1-5 1.3.2 Meet with study group to review 1.3.3 Go to exam room at designated time

16. 16 Copyright 2003-2013, Dennis J. Frailey 16 Sample WBS for a Software Project (top levels only) 1.0 Software for Payroll system 1.1Build the Payroll software 1.1.1 Build a User Interface 1.1.2 Build a File System 1.1.3 Build a Data Base 1.1.4 Build a Table of Rules 1.1.5 Build a Check Printing System 1.2Build the Test Suite for the Software 1.2.x (detailed steps for building test suite) … 1.3Write Documentation 1.4Write Installation Software 1.5Manage the Above

17. 17 Copyright 2003-2013, Dennis J. Frailey 17 The WBS Is... A “table of contents” for the project.

18. 18 Copyright 2003-2013, Dennis J. Frailey 18 Top Level Role of WBS Source Documents (Statement of Work, Requirements, contract, test criteria, etc,) Historical Records (at end of project) WBS Cost Estimate (proposal &/ project start) Cost Tracking (during execution)

19. 19 Copyright 2003-2013, Dennis J. Frailey 19 Best Practices for Managing These Problems The Work Breakdown Structure –Documenting and organizing the work to be done  Estimating and Tracking –So you know where you are  Systems Engineering –Designing the big system before designing the software  Risk Management  Configuration Management  Quality Assurance

20. 20 Copyright 2003-2013, Dennis J. Frailey 20 Using the WBS to Estimate WBS # Task Estimated Hours Estimated Completion 1.1 Do Assignment 1 159/22 1.1.1 Read Chapters 1, 2 39/15 1.1.2 Write Simple Java Pgm 129/22 1.2 Do Assignment 2 219/30 1.2.1 Read Chapters 3, 4 49/23 1.2.2 Review Graded Assign. 1 19/23 1.2.3 Write Java Program with Arrays 169/30

21. 21 Copyright 2003-2013, Dennis J. Frailey 21 Using the WBS to Track WBS # Task Estimated Hours Actual Hours Estimated Completion Actual 1.1 Do Assignment 1 15 129/229/25 1.1.1 Read Chapters 1, 2 3 29/159/22 1.1.2 Write Simple Java Pgm 12109/229/25 1.2 Do Assignment 2 219/30 1.2.1 Read Chapters 3, 4 49/23 1.2.2 Review Graded Assign. 1 19/23 1.2.3 Write Java Pgm with Arrays 169/30 When the project is over, you have a detailed record of how much time and effort were required.

22. 22 Copyright 2003-2013, Dennis J. Frailey 22 Graph to Show Progress

23. 23 Copyright 2003-2013, Dennis J. Frailey 23 Using the WBS for Historical Records Previous WBS New WBS Facts about New Project Previous WBS Old Projects You can make more accurate estimates for the next project.

24. 24 Copyright 2003-2013, Dennis J. Frailey 24 Best Practices for Managing These Problems The Work Breakdown Structure –Documenting and organizing the work to be done Estimating and Tracking –So you know where you are  Systems Engineering –Designing the big system before designing the software  Risk Management  Configuration Management  Quality Assurance

25. 25 Copyright 2003-2013, Dennis J. Frailey 25 Systems Engineering A Sample System DisplayPrinterKeyboard Processor and Memory System Software Power Supply Computer System Unit

26. 26 Copyright 2003-2013, Dennis J. Frailey 26 Systems Engineering - Step 1 - Analyze the Requirements of the Total System  Determine what the customer really wants –>> Analysis of the problem and the need  Produce verifiable specifications (specifications may be in the form of simulations of expected behavior, use cases, specification documents, system requirements models, or other things)  Confirm the specifications are right by conferring with the customer

27. 27 Copyright 2003-2013, Dennis J. Frailey 27 Systems Engineering - Step 2 - Design the System  Determine the architecture of the system –>> Synthesis, not analysis –Done by Systems engineer and/or chief architect  Determine the major parts (functional decomposition into sub-systems)  Determine the interfaces between the subsystems

28. 28 Copyright 2003-2013, Dennis J. Frailey 28 Systems Engineering - Step 3 - Allocate the Requirements to the Components of the System  Determine what each part will do  Assign requirements to each part based on what it is supposed to do  This is how you get some of the software requirements when software is part of a larger system

29. 29 Copyright 2003-2013, Dennis J. Frailey 29 Typical Requirements Flowdown Transmission System Analysis & Design Allocated Requirements Automobile Drive TrainChassisEngine 100 pounds torque... 50 pounds2000 pounds500 pounds 250 hp... Original Requirements 3000 pounds 0-60mph in 9 sec

30. 30 Copyright 2003-2013, Dennis J. Frailey 30 When the Product is Finished Verify that it Satisfies the Requirements Requirements Development Testing or Other Verification Methods Verification Methods derived from requirements When the requirements (specifications) are verifiable by testing, you are much more likely to have effective tests

31. 31 Copyright 2003-2013, Dennis J. Frailey 31 Ideal Requirements (some but not all desirable characteristics)  Cohesive (each requirement addresses only one thing)  Complete (each requirement is fully stated; all requirements are known and allocated)  Sufficient (every required feature is specified)  Consistent (no requirements contradict each other)  Unambiguous (only one possible interpretation)  Verifiable (you can tell when they are met)

32. 32 Copyright 2003-2013, Dennis J. Frailey 32 Actual Requirements  Some requirements will be: –TBD (unknown - to be determined) –Wrong –Missing –Vague or unclear  Some requirements will –Contradict others –Not be readily verified –Change in mid- stream  Technology changes  Customer understanding of problem changes  Problem changes

33. 33 Copyright 2003-2013, Dennis J. Frailey 33 Risk Management for Imperfect Requirements  Define a lifecycle/process that is adaptable – it will readily accommodate requirements changes  Assign someone to be responsible to manage requirements changes –including involvement of software staff  Select a lifecycle based on level of expected requirements stability  Get ranges for “TBD” (to be determined) requirements  Work to eliminate wrong and missing requirements

34. 34 Copyright 2003-2013, Dennis J. Frailey 34 Interface Requirements  How does your software interface with other parts of the system?  This is just as important as other requirements  Make sure the interface requirements are –Documented –Under Control  If possible, requirements should be –Contained in (or controlled by) a SINGLE document or other control point

35. 35 Copyright 2003-2013, Dennis J. Frailey 35 Who is Establishing the Requirements? All Stakeholders!  End user -- will use the software  Champion -- will pay for it  Marketing -- will sell it  Accountants and Lawyers -- will protect but may not understand the product or the process  Political factors –Such as engineers -- who “know better” than the customer

36. 36 Copyright 2003-2013, Dennis J. Frailey 36 “V” Model of Software Development A General Model of Software Development Requirements Analysis Architecture Design Detailed Design Implementation Unit Test Integration Test Acceptance Test Test Cases Based on Requirements Test Cases Based on Architecture Test Cases Based on Design Test Planning Begins with Requirements! This model underlies almost all others

37. 37 Copyright 2003-2013, Dennis J. Frailey 37 Are we Asking the Right Question?  The usual question: –How can we verify the software (make sure we have satisfied the requirements)?  A better question: –How can we define requirements that are readily verified?  The requirement can be verified through one of these basic methods:  inspection,  demonstration,  test (instrumented) or  analysis (to include validated modeling & simulation).

38. 38 Copyright 2003-2013, Dennis J. Frailey 38 Best Practices for Managing These Problems The Work Breakdown Structure –Documenting and organizing the work to be done Estimating and Tracking –So you know where you are Systems Engineering –Designing the big system before designing the software  Risk Management  Configuration Management  Quality Assurance

39. 39 Copyright 2003-2013, Dennis J. Frailey 39 Risk Management  Risks are: –things that can go wrong  If it has already happened, or is certain to happen, it is an issue, not a risk! –You should be discussing your action plan for managing the problem

40. Copyright 2003-2013, Dennis J. Frailey Risk Management Phases 1)Risk Assessment – The things you do before you start the project to identify – What can go wrong? – How it can hurt you? – How likely is it to happen? – Why it may happen? – How you can mitigate (reduce) the risk? 2)Risk Control – The things you do during the project – Preventing the unexpected – Responding when you cannot prevent

41. 41 Copyright 2003-2013, Dennis J. Frailey 41 Risk Assessment Chart RiskEvidenceHow Likely ImpactPriorityMitigation Late project due to employee turnover Unplanned turnover rate has been running high 7749Employee bonuses if they stay to end of project Bottleneck at testing due to changing require- ments Past experience with this customer 8648Iterative development Customer workshops Etc.

42. 42 Copyright 2003-2013, Dennis J. Frailey 42 Risk Control A) Risk Monitoring - Watching to see if risks happen –Metrics to warn you of problems –Periodic reviews B) Risk Abatement - Counteracting risks - Taking contingency actions as needed

43. 43 Copyright 2003-2013, Dennis J. Frailey 43 Risk Control Chart RiskMonitoringCounteracting / Contingency Ongoing Mitigation Late project due to employee turnover Measure turnover rate and project schedule Get management help in identifying alternate employees Periodic employee mini- bonuses Bottleneck at QA due to changing require- ments Track requirements changes and QA backlog Requirements freeze; Multiple releases Feature prioritization Etc.

44. 44 Copyright 2003-2013, Dennis J. Frailey 44 Risk Management is Ongoing Risk Management (planning, analysis) Product Development Data and reports to identify and monitor risks Actions to mitigate or abate risks

45. 45 Copyright 2003-2013, Dennis J. Frailey 45 Example of Risk Monitoring: Requirements Stability Monitoring CDRPDR

46. 46 Copyright 2003-2013, Dennis J. Frailey 46 Setting Thresholds Expected Range of Values Abnormally High Values Abnormally Low Values

47. 47 Copyright 2003-2013, Dennis J. Frailey 47 Thresholds Help You Monitor Risks  Thresholds tell you when to act –Beware of the “contented frog” problem –Historical experience is a good basis to judge when things are getting out of hand It’s only a little hotter than yesterday!

48. 48 Copyright 2003-2013, Dennis J. Frailey 48 Best Practices for Managing These Problems The Work Breakdown Structure –Documenting and organizing the work to be done Estimating and Tracking –So you know where you are Systems Engineering –Designing the big system before designing the software Risk Management  Configuration Management  Quality Assurance

49. 49 Copyright 2003-2013, Dennis J. Frailey 49 What Is Software Configuration Management? Definition: Software Configuration Management is the process of identifying, organizing and managing modifications to software

50. 50 Copyright 2003-2013, Dennis J. Frailey 50 Purpose of Configuration Management The purpose of configuration management is to maintain integrity of the product by controlling change to the product

51. 51 Copyright 2003-2013, Dennis J. Frailey 51  All components that make up or relate to a version of a product –The proper version of each: What Is The Configuration? document compiler used to develop development tool data set used for simulating special equipment operating system test code test data library procedure hardware etc. code module component data file macro simulator specification requirement If any of the above is wrong, the product lacks integrity and may not work

52. 52 Copyright 2003-2013, Dennis J. Frailey 52 What is Integrity? Having the right parts that all fit together to form a correct product. RequirementsUser’s Guide Main, rel 2.3 Sub1, rel 1.7 Sub2, rel 3.1... Object code

53. 53 Copyright 2003-2013, Dennis J. Frailey 53 Why is SW Configuration Management Needed?  Because software is flexible, changeable, and intangible  It is too easy to lose integrity of software because of these factors Unmanaged change is very possibly the largest single cause of failure to deliver systems on time and within budget. NASA SCM Guidebook, August, 1995 Unmanaged change is very possibly the largest single cause of failure to deliver systems on time and within budget. NASA SCM Guidebook, August, 1995

54. 54 Copyright 2003-2013, Dennis J. Frailey 54 Examples of Loss of Integrity  Performance 2 years ago cannot be duplicated with the current release  Source code will not compile without errors  Compiled code will not execute with the same results as the released code  It worked in the factory but not in the field  All records of a particular feature are gone, and the programmer/designer left the company

55. 55 Copyright 2003-2013, Dennis J. Frailey 55 More Examples of Loss of Integrity  Compiled code has different object code or code size from released code  A bug that we fixed once before has reappeared  Code does not match the design specification  A feature that we added and fully tested has suddenly stopped working  We cannot figure out which version of each module is needed to reproduce a problem we found in the field

56. 56 Copyright 2003-2013, Dennis J. Frailey 56 Configuration Management Has Five Major Functions 1) Configuration Identification –A unique name for every version of every part –A record of which versions of the parts are combined in what way to make a given release/version of the product [known as a Version Description Document] 2) Configuration Control –Rules for making changes to software  So two people don’t change it at the same time 3) Change Authorization –Deciding who is allowed to make changes and when? 4) Status accounting –Knowing where to find the current version of each component? 5) Configuration Authentication –Verifying that everything is right

57. 57 Copyright 2003-2013, Dennis J. Frailey 57 Best Practices for Managing These Problems The Work Breakdown Structure –Documenting and organizing the work to be done Estimating and Tracking –So you know where you are Systems Engineering –Designing the big system before designing the software Risk Management Configuration Management  Quality Assurance

58. 58 Copyright 2003-2013, Dennis J. Frailey 58 Software Quality Assurance Purposes: –To prevent defective products from being shipped to the customer –To arbitrate disputes when there are debates about whether the product is “good enough” –To provide managers and developers with visibility into the process being followed and the products being built  So they can manage the outcome

59. 59 Copyright 2003-2013, Dennis J. Frailey 59 Quality Assurance Looks at the Entire Process Requirements Development Testing and Inspection Pass Fail Standards of Quality Process and Design Standards Tests derived from requirements

60. 60 Copyright 2003-2013, Dennis J. Frailey 60 Software Quality Assurance Typical Practices: –Inspections –Reviews –Audits –Communication –Measurements –Independent confirmation of compliance  Standards, requirements, procedures

61. 61 Copyright 2003-2013, Dennis J. Frailey 61 Professional Software Engineering Is... Building Software Systems that People can Rely On “The application of a systematic, disciplined, quantifiable approach to the development, operation and maintenance of software; that is, the application of engineering to software.” IEEE Standard Glossary of Software Engineering Terminology

62. 62 Copyright 2003-2013, Dennis J. Frailey 62 Where Are The Computing Jobs? Applications of Computers & Electronics to Problems in Other Disciplines Research Jobs Systems Jobs (tool development, operating systems, etc.) All of the Other Jobs

63. 63 Copyright 2003-2013, Dennis J. Frailey 63 What Kinds of Jobs does a Computing Graduate Do?  Write Software or Build Hardware  Fix Bugs in Other Peoples’ Software/Hardware  Test Software or Hardware  Design Software or Hardware  Understand Customers and Document Their Requirements  Design Systems to Solve Customer Problems  Control Changes to Software or Hardware  Assure Quality in Software or Hardware  Plan Software or Hardware Projects  Manage Software or Hardware Projects Career Progression

64. 64 Copyright 2003-2013, Dennis J. Frailey 64 Spectrum of Computing Jobs Theory, Principles, Innovation Application, Deployment, Configuration Computer Hardware Organization; Information Systems Application Technologies Software Methods & Technologies Systems Infrastructure (Source: Computing Curriculum 2005, sponsored by ACM and IEEE Computer Society)

65. 65 Copyright 2003-2013, Dennis J. Frailey 65 Spectrum of Computing Jobs Theory, Principles, Innovation Application, Deployment, Configuration Computer Hardware Organization; Information Systems Application Technologies Software Methods & Technologies Systems Infrastructure Computer Engineer Build Hardware that Works

66. 66 Copyright 2003-2013, Dennis J. Frailey 66 Spectrum of Computing Jobs Theory, Principles, Innovation Application, Deployment, Configuration Computer Hardware Organization; Information Systems Application Technologies Software Methods & Technologies Systems Infrastructure Software Engineer Build Software that You Can Depend On

67. 67 Copyright 2003-2013, Dennis J. Frailey 67 Spectrum of Computing Jobs Theory, Principles, Innovation Application, Deployment, Configuration Computer Hardware Organization; Information Systems Application Technologies Software Methods & Technologies Systems Infrastructure Computer Scientist Find New Ways to Solve Computing Problems

68. 68 Copyright 2003-2013, Dennis J. Frailey 68 Spectrum of Computing Jobs Theory, Principles, Innovation Application, Deployment, Configuration Computer Hardware Organization; Information Systems Application Technologies Software Methods & Technologies Systems Infrastructure Information Systems Design Computer Systems that Solve Organization Problems

69. 69 Copyright 2003-2013, Dennis J. Frailey 69 Spectrum of Computing Jobs Theory, Principles, Innovation Application, Deployment, Configuration Computer Hardware Organization; Information Systems Application Technologies Software Methods & Technologies Systems Infrastructure Information Technology Build Computer Systems that Solve Organizational Problems

70. 70 Copyright 2003-2013, Dennis J. Frailey 70 Some Things are Learned On the Job Foundation Skills Mathematics Science Communication Computing Skills Requirements DesignProgramming Testing Management Planning Application Skills (depends on application area) Job Experience College Education

71. 71 Copyright 2003-2013, Dennis J. Frailey 71 The Changes in Computing

72. Copyright 2003-2013, Dennis J. Frailey About ACM ACM, the Association for Computing Machinery is the world’s largest educational and scientific computing society, uniting educators, researchers and professionals to inspire dialogue, share resources and address the field’s challenges. ACM strengthens the computing profession’s collective voice through strong leadership, promotion of the highest standards, and recognition of technical excellence. ACM supports the professional growth of its members by providing opportunities for life-long learning, career development, and professional networking. With over 100,000 members from over 100 countries, ACM works to advance computing as a science and a profession. www.acm.orgwww.acm.org

73. 73 Copyright 2003-2013, Dennis J. Frailey 73 Questions?