1. CSC444F'05Lecture 11 CSC444 Software Engineering Prof. David A. Penny Lectures Will start at 7:10 pm Please come out now to purchase a book for $40. If you don’t have the cash, give me your name instead and you can owe me.

2. CSC444F'05Lecture 12 CSC444 Software Engineering Lectures: Prof. David A. Penny Practicum: Jorge Aranda

3. CSC444F'05Lecture 13 This Course Specially commissioned by Dept. of EE. Designed to lessen the gap experienced by students when they enter industry. Throws out traditional software engineering curriculum. –(Unchanged since I took this course in 1984). Updates it to be relevant to software professionals today working in challenging environments under great pressure. Professional Knowledge University Taught

4. CSC444F'05Lecture 14 About Prof. Penny Graduated B.Sc. in CS UofT 8T5, Ph.D 9T3 –OOT IDE, Polyx, MiniTunis, CE,... IBM Labs 1992 – 1994 –C++ IDE for AIX Algorithmics 1994 – 1999 –VP Software Development –RiskWatch > $500M in revenues to-date Consultant 1999 – 2003 –Software management consulting (~10 engagements) UofT CS 2000 – 2003 –Associate Professor Electronics Workbench 2003 – 2005 –VP R&D – Acquired by National Instruments Ceryx 2005 – present –CIO

5. CSC444F'05Lecture 15 Lectures I wrote a manuscript for you guys: –Professional Software Development, 2005. –I will be following it closely –Tentative Schedule (first time course - may vary significantly) LectureDateTopicsChapters 1Sep 12Top-10 Practices, Introduction to Planning1,2 2Sep 19Release Planning Overview, Capacity Constraint3,4 3Sep 26Discipline in Programming (practicum intro) 4Oct 3Quantitative Capacity Constraint, Sample RP5,A,C Oct 10Thanksgiving – no lecture 5Oct 17Stochastic Capacity Constraint, Sample SRP6,B,C 6Oct 24Releases, Versions7,8 7Oct 31MIDTERMSource Control, Build, Testing9,10 8Nov 7Defect & Feature Tracking11,12 9Nov 14Spillover / Process Control13 10Nov 21Architectural Clarity14 11Nov 28Business Aspects15,16

6. CSC444F'05Lecture 16 Practicum Will be conducted entirely by Jorge Aranda and his TA’s. Will do exercises from –A Discipline for Software Engineering Watts S. Humphrey Addison-Wesley Professional, 1994 Introductory Lecture (I must be out-of-town): –Sep.26 7-9 pm –Will overview the entire series –Will get your started on A1 –Discuss marking scheme Subsequent Lectures –Oct.3 – Nov.28 Except for Oct.10 (Thanksgiving), Oct.31 (Midterm) Progressive Assignment due each week (other than Oct.31) –Last is a term report

7. CSC444F'05Lecture 17 Practicum (cont’d) Exercises will involve: –Programming smallish C programs –Keeping track of Detailed time log Detailed defect log Sizing and Estimation of your own work Correlate back... Motivation –I did the exercises myself early in my career –They were a revelation to me and a big help in my professional career Purpose –To bring engineering discipline to programming –To demonstrate to you that it is possible to do so –To have you see first hand the benefits –So you can practice this yourself when you start working as a professional –So you can encourage others as a colleague and set an example –So you can ask this of staff you may have occasion to manage

8. CSC444F'05Lecture 18 Grades Practicum – 35% –6 Programming / tracking assignments –Hand in completed program, logs, data, and analysis –1 Final report –5% each –Late Policy: Hand in during practicum slot, 6-7pm DO NOT HAND IN TO ME (I cannot accept it – no easy way of getting it to Jorge) 20% penalty if handed in <= 1 week late Not accepted after that (will get too far behind otherwise) Midterm – 25% –Oct.31, 6-7pm –Closed book (I want you to study!) –Covers all lecture, practicum, and assigned reading covered up to and including Oct.24 th Exam – 40% –Closed book (ditto) –Covers all lectures, practicum, and assigned reading

9. CSC444F'05Lecture 19 Professional Practices The software development industry is not yet mature. One is more likely than not to find: –Missed dates –Poor quality software –Badly-designed features –Poor user documentation –Poor architecture and architectural documentation –Dysfunctional professional relationships between “The Business Side” and Software Development Due to lack of application of basic professional practices –Not consistently taught at Universities (or anywhere else)

10. CSC444F'05Lecture 110 Experience Need –Formal education in the computing sciences –Professional experience Build software that lots of people pay money to buy –Not just “are you paid” Make certain decisions for v1 of a product Live with your mistakes through v2, v3, v4,... Make fewer mistakes next time around We try to fill the gap a bit –Lessons coming out of extensive professional experience Not all professionals agree on what constitute “basic professional practices” –Characteristic of an immature industry –But can agree on the problems we are trying to solve –One (informed) opinion will be presented here

11. CSC444F'05Lecture 111 Intended Audience Commercial software vendor environment –Not open source, internal IT, ASP, NASA,... Who –Individual contributors, Technical leaders, First-line managers, Directors, VP’s, CTO’s Next release –Not initial release –“Green fields” is 80% inspiration, 20% process –“Next Release” is 80% process –Next release development is more important to businesses Initial release development –Innovation is clearly also important –Innovation is less amenable to help from process –Should set things up to be sustainable

12. CSC444F'05Lecture 112 New Product Versus Established One New product –1 yr. to develop –3 coders, 1 tester, 1 documenter –Cost = 1 x 5 x $100,000 = $500,000 Established Product –5 years later –20 coders, 10 testers/build, 5 documenters –Cost to date = $10,000,000 –Ongoing cost = $3,500,000 / year Improve productivity by 10% –New product: save $50,000 –Established Product: save $1,000,000 to date, $350,000/year Next release development is more economically important.

13. CSC444F'05Lecture 113 Top-10 Essential Practices Crystallized for me whenever I enter into a new engagement. If any of these are missing, I know I have something to fix. These are all important It will take more than this course to cover them all You will agree that all suggestions are sensible and will probably vow to carry them out –On your first job, you’ll focus on code and test and forget most of them –You’ll be bitten in the ass –You’ll re-commit to the ideas (if you’re good) Simple but hard –Trust me: make sure these things are done and everything will go ok –Very hard to change behaviour –Need to be dogged and determined and tricky

14. CSC444F'05Lecture 114 1. Source Code Control Central repository –Everybody knows where to find what they are looking for –Secure, backed-up storage Defines module architectural structure –hierearchy Complete change history –Can back up and find where problems are first introduced Multiple maintenance streams –Work on next release while maintaining previous releases Patches –Can go back and patch any release in the field Enables Team development “Interface” to coordinate Dev and QA/Build “Guard” against bad changes

15. CSC444F'05Lecture 115 2. Defect / Feature Tracking Keeps track of all defects found or new features desired –Won’t forget any Coordinates a workflow for writing / fixing them –Won’t skip steps Provides management visibility into progress Enables effective prioritization Enables metrics gathering

16. CSC444F'05Lecture 116 3. Reproducible Builds Check out of source control and one command to build the product Required for a consistent experience across all developers, QA/Build, customers Dev builds –For coding and testing Production builds –Includes creation of install image –And creation of ISO-Image –Should also be fully automated

17. CSC444F'05Lecture 117 4. Automated Regression Testing Scripts that run after every QA/Build dev build to test as much functionality as possible Critical to improving software quality Prevents errors with previously seen symptoms from recurring –A very common thing to happen Enables coders to change tricky bits with confidence Enables finding problems closer to their injection –Earlier you can find an issue the less costly it is to fix. Enables fixing last problems prior to shipping with confidence –Can release with fewer known defects –Can release on time

18. CSC444F'05Lecture 118 5. Release Planning After the previous basics are in place this is the most important practice –Will spend 1/3 of the course on this Determining –What goes into the next release –By when will it will be done –Using what resources Tracking that throughout the release Adjusting as necessary Enables business side to do their jobs –Good relationships Enables quality –By maintaining the test/debug period Provides elbow room –To improve productivity

19. CSC444F'05Lecture 119 6. Feature Specifications Complicated features require them –Need to make this determination Needed to keep release plan on track –Better estimates if know what we are doing in more detail Enables a better end-user feature Eliminates unanticipated integration problems Best place to introduce reviews

20. CSC444F'05Lecture 120 7. Architectural Control Must maintain a clean architecture even in the face of –Many coders working on the code –Frequent feature additions That the software was not designed for initially –Frequent defect corrections By inexperienced coders who do not understand the architecture Architectural documentation Review of designs and code for conformance Chief Architect Automated architectural checking tools

21. CSC444F'05Lecture 121 8. Effort Tracking Need to know how much staff time is spent on –Each new feature –Correcting defects –Other Can improve estimation accuracy Can improve estimates of staff time available for next release Can monitor effectiveness of initiatives to free up coder time for more coding

22. CSC444F'05Lecture 122 9. Process Control Written process for the release cycle Gets everybody on the same page –Can train new staff Enables systematic definition / collection of metrics Can monitor process for compliance Can consider changes to the process from a stable baseline

23. CSC444F'05Lecture 123 10. Business Planning Development occurs within a business context If not understood and managed, will sink the project more surely than technical shortcomings Writing effective proposals Integrating into the budget cycle.

24. CSC444F'05Lecture 124 Summary 1.Source Code Control 2.Defect / Feature Tracking 3.Reproducible Builds 4.Automated Regression Testing 5.Release Planning 6.Feature Specifications 7.Architectural Control 8.Effort Tracking 9.Process Control 10.Business Planning Do all these things, and you’re doing well.