1. Derek Mathieson, James Purvis, Rostislav Titov CERN
Software Team Development Practices based on Java Development Teams at CERN
Derek Mathieson, James Purvis, Rostislav Titov CERN

2. The reality today Failure
31.1% of IT projects will be canceled before they ever get completed
52.7% of projects will cost 189% of their original estimate
More than 50% of software projects fail today.
Success
Only 16.2% software projects that are completed on time and on budget
In the large companies only 9% of their projects come in on time and on budget.
Catastrophe
Ariane 5 (7bn dev, 500 million) – numerical conversion error
Mars Climate Orbiter ($125 million) – metric conversion error
Mars Polar Lander ($165 million) – design error
A Standish Group research report shows a staggering 31.1% of projects will be canceled before they ever get completed. Further results indicate 52.7% of projects will cost 189% of their original estimate
More than 50% of software projects fail today.
On the success side, the average is only 16.2% for software projects that are completed on time and on budget. In the larger companies, the news is even worse: only 9% of their projects come in on time and on budget. And, even when these projects are completed, many are no more than a mere shadow of their original specification requirements. Projects completed by the largest American companies have only approximately 42% of the originally proposed features and functions. Smaller companies do much better. A total of 78.4% of their software projects get deployed with at least 74.2% of their original features and functions
On June 4, 1996 an unmanned Ariane 5 rocket launched by the European Space Agency exploded just forty seconds after its lift-off from Kourou, French Guiana. The rocket was on its first voyage, after a decade of development costing $7 billion. The destroyed rocket and its cargo were valued at $500 million. A board of inquiry investigated the causes of the explosion and in two weeks issued a report. It turned out that the cause of the failure was a software error in the inertial reference system. Specifically a 64 bit floating point number relating to the horizontal velocity of the rocket with respect to the platform was converted to a 16 bit signed integer. The number was larger than 32,767, the largest integer storeable in a 16 bit signed integer, and thus the conversion failed.

3. Typical Failures User requirements not Met Software unreliable
Too late (You took so long that our requirements have changed…)
It works great for me, now deploy it for 10,000 users…
You did what I asked. but I didn’t say what I meant…
Projects completed by the largest American companies have only approximately 42% of the originally proposed features and functions.

4. Project Success factors
Project Success Factors and % of Responses 1
User Involvement
15.90% 2
Executive Management Support
13.90% 3
Clear Statement of Requirements
13.00% 4
Proper Planning
9.60% 5
Realistic Expectations
8.20% 6
Smaller Project Milestones % 7
Competent Staff
7.20% 8
Ownership
5.30% 9
Clear Vision & Objectives
2.90% 10
Hardworking, Focused Staff
2.40%
Other
Project Challenged Factors and % of Responses
Lack of User Inputs
12.80%
Incomplete Requirements & Specifications
12.30%
Changing Requirements & Specifications
11.80%
Lack of Executive Support
7.50%
Technology Incompetence
7.00%
Lack of Resources
6.40%
Unrealistic Expectations
5.90%
Unclear Objectives
23.00%

5. Factors Cost Quality Four interdependent factors
Not possible to have best of all four
Cannot have cheap, high quality, little risk & built quickly
You can achieve 2 successfully and other 2 have to be managed
Risk and quality are most important: The system must work and successfully meet user requirements. This leaves speed (time) and cost (money) to be adjusted accordingly
Risk
Speed
Cheap
Good
Fast
Choose two
In any systems project, there are four interdependent factors:
1.       Cost
2.       Quality
3.       Speed
4.       Risk
It is not possible to have the best of all four factors. Specifically, you cannot have a system built inexpensively, of high quality, built quickly and with little or no risk of failure. Most discussions of these factors only include the first three. It is possible to build a high-quality system quickly, at a relatively low cost by cutting corners, and doing little or no testing. However, the risk of such a system failing increases dramatically. Of these four factors, in any project, two are always possible to achieve successfully, leaving the other two to be managed.
Of these four factors, the two most important are risk and quality. The system must work and successfully meet user requirements. This leaves speed (time) and cost (money) to be adjusted accordingly. If you insist on fast development time or low cost, then quality and risk will shift accordingly. I have had many arguments with managers over this principle. I have been told “If we use Product X and Methodology Y, we can have the system built quickly at minimal cost.” This is not a realistic or tenable position. You can insist on low risk and high quality, recognizing that time and money must be adjusted to achieve these goals.

6. The answer
Q :What is the key to running a successful software project?
A: Don’t believe in Magic!
There is no silver bullet
…But there are some step that can be carried out at each step in the process…

7. Software Development Cycle
Analysis
“Failing to build the right thing”
Design
“We didn't have time to do a design."
Implementation
“Do you want it on time or documented?”
Test
“It's not a bug, it's a feature”
Whatever software project methodology – some common components & they include analysis, design, implementation & test

8. Requirements Analysis
“Writing software from Requirements is like walking on water – its easier when frozen”

9. Analysis State Goal Specify the problem not the solution
“send a man to the moon before end of the decade & return him safely to Earth”, JF Kennedy
Specify the problem not the solution
Classification
M Must o S Should C Could 0 W Would
Concept of Operations document
“This is my understanding of what you want”
Beware of requirements ‘gold plating’
80/20 rule
Verifiable : use-cases
1 Scope
2 Referenced documents
3 Background
4 Current situation
5 Required changes
6 Proposed capability
7 Operational scenarios
8 Support concept and scenarios
9 Impact summary
10 Analysis of proposed capability
11 Verification & validation

10. Design
One of the things that tools can do is to help bad designers create ghastly designs much more quickly than they ever could in the past

11. Example Failures Ariane 5 (half billion dollars) Y2K bug
F-16 & equator
Mars Climate Orbiter
Mars Polar Lander
Just a few years ago, the failure of two multi-million space missions to Mars was prominent in the news. Both failures were probably due to software problems, but in both cases the problem was not with an incorrect program as such. In September 1999, the Mars Climate Orbiter burned up in the Martian atmosphere because data that was expressed in English units of measurement (such as feet and pounds) was entered into a computer program that was designed to use metric units (such as centimeters and grams). A few months later, the Mars Polar Lander probably crashed because its software turned off its landing engines too soon. The program was supposed to detect the bump when the spacecraft landed and turn off the engines then. It has been determined that deployment of the landing gear might have jarred the spacecraft enough to activate the program, causing it to turn off the engines when the spacecraft was still in the air. The unpowered spacecraft would then have fallen to the Martian surface. A more robust system would have checked the altitude before turning off the engines!
There are many equally dramatic stories of problems caused by incorrect or poorly written software. Let's look at a few incidents recounted in the book Computer Ethics by Tom Forester and Perry Morrison. (This book covers various ethical issues in computing. It, or something like it, is essential reading for any student of computer science.)
In 1985 and 1986, one person was killed and several were injured by excess radiation, while undergoing radiation treatments by a mis-programmed computerized radiation machine. In another case, over a ten-year period ending in 1992, almost 1,000 cancer patients received radiation dosages that were 30% less than prescribed because of a programming error.
In 1985, a computer at the Bank of New York started destroying records of on-going security transactions because of an error in a program. It took less than 24 hours to fix the program, but by that time, the bank was out $5,000,000 in overnight interest payments on funds that it had to borrow to cover the problem.
The programming of the inertial guidance system of the F-16 fighter plane would have turned the plane upside-down when it crossed the equator, if the problem had not been discovered in simulation. The Mariner 18 space probe was lost because of an error in one line of a program. The Gemini V space capsule missed its scheduled landing target by a hundred miles, because a programmer forgot to take into account the rotation of the Earth.
In 1990, AT&T's long-distance telephone service was disrupted throughout the United States when a newly loaded computer program proved to contain a bug.

12. Jumping into the coding
Reasons
Too complex
“We didn't have time to do a design."
Jumping into the coding
Architect: Someone who knows the difference between that which could be done and that which should be done.
One of the things that tools can do is to help bad designers create ghastly designs much more quickly than they ever could in the past
Just a few years ago, the failure of two multi-million space missions to Mars was prominent in the news. Both failures were probably due to software problems, but in both cases the problem was not with an incorrect program as such. In September 1999, the Mars Climate Orbiter burned up in the Martian atmosphere because data that was expressed in English units of measurement (such as feet and pounds) was entered into a computer program that was designed to use metric units (such as centimeters and grams). A few months later, the Mars Polar Lander probably crashed because its software turned off its landing engines too soon. The program was supposed to detect the bump when the spacecraft landed and turn off the engines then. It has been determined that deployment of the landing gear might have jarred the spacecraft enough to activate the program, causing it to turn off the engines when the spacecraft was still in the air. The unpowered spacecraft would then have fallen to the Martian surface. A more robust system would have checked the altitude before turning off the engines!
There are many equally dramatic stories of problems caused by incorrect or poorly written software. Let's look at a few incidents recounted in the book Computer Ethics by Tom Forester and Perry Morrison. (This book covers various ethical issues in computing. It, or something like it, is essential reading for any student of computer science.)
In 1985 and 1986, one person was killed and several were injured by excess radiation, while undergoing radiation treatments by a mis-programmed computerized radiation machine. In another case, over a ten-year period ending in 1992, almost 1,000 cancer patients received radiation dosages that were 30% less than prescribed because of a programming error.
In 1985, a computer at the Bank of New York started destroying records of on-going security transactions because of an error in a program. It took less than 24 hours to fix the program, but by that time, the bank was out $5,000,000 in overnight interest payments on funds that it had to borrow to cover the problem.
The programming of the inertial guidance system of the F-16 fighter plane would have turned the plane upside-down when it crossed the equator, if the problem had not been discovered in simulation. The Mariner 18 space probe was lost because of an error in one line of a program. The Gemini V space capsule missed its scheduled landing target by a hundred miles, because a programmer forgot to take into account the rotation of the Earth.
In 1990, AT&T's long-distance telephone service was disrupted throughout the United States when a newly loaded computer program proved to contain a bug.

13. Good Design Principles
Consider alternative approaches
Not tunnel vision
Traceable to the requirements
Correct & complete
Not reinvent the wheel
Use a pattern
Adaptability Accommodate Change
Maximize Cohesion
Minimize Coupling
Understandabilty
A design must be understandable if it is to support modification.
Cohesion describes how well the contents of a module cohere (stick together). A component should implement a single logical function  or should implement a single logical entity.
Coupling describes how modules interact. Systems should be loosely coupled. Highly coupled systems have strong interconnections with units dependent on each other. Loosely coupled systems are made up of components which are independent or almost independent.

14. Good & Bad design Good Design
Change in one part of the system doesn't always require a change in another part of the system.
Every piece of logic has one and one home.
The logic is near the data it operates on.
System can be extended with changes in only one place.
Simplicity
Bad Design
One conceptual change requires changes to many parts of the system.
Logic has to be duplicated.
Cost of a bad design becomes overwhelming.
Can't remember where all the implicitly linked changes have to take place.
Can't add a new function without breaking an existing function.
Complexity

15. The Cost of Change
Cost
Time
CERN
AIS

16. Design First Design team (>1) Maxims
Design requires different skills
Collaborative
Informal design reviews
Standards based
Patterns
Maxims
Storyboard
Metaphor – vision
Class Responsibility

17. Lessons Learned It’s not always right first time
Refactoring is OK
Keep the vision simple
Automate repetitive tasks
TEST the design
By modifying the problem

18. Coding
“It doesn’t matter if I write poor code… the compiler will tell me if there is problem”
“It doesn’t matter if I make a mistake… it will come out in testing”

19. Bugs
Experienced software engineers inject one defect in about every 10 lines of code.
The programmers aren't incompetent or lazy - they're just human.
All humans make mistakes, but in software, these mistakes result in defects.
This means that a modest-size program of 100,000 lines of code typically would start with about 10,000 defects.
Examples :
INTEL Pentium : no more than Bugs
Cell Phone ( loc) up to 600 errors.
Windows-95: 10 Mill. loc: up to errors.
Xerox Corp. measured the time it takes to find and fix a defect at each stage:
• Code reviews, in which programmers analyze their own programs: three minutes. • Code inspections, in which several programmers look for problems: 25 minutes. • Module testing, in which programmers test small program modules: 32 minutes. • System testing, in which the modules are combined so the whole system is tested: 1,400 minutes.

20. Code Review Example And identify the defect?
Can you understand the following?
public class ba {
public static final String cur = “USD";
public void dep (int i) { bal -=i; }
public void wit (int i) { bal +=i;}
public String get () { return Integer.toString (bal) + " " + cur }
private int bal; }
And identify the defect?
Like about C5s is “dive straight in”
Use egg timer
How many programme in Java?
How many understand 5 lines of simple java / no complex expressions?
Imagine you are working in medium size s/w proj. Bug has been identified by user and you know its in 5 lines of java code. You go in to take a look...
Can you find the bug?

21. The same code public class BankAccount {
public static final String CURRENCY = “USD";
private int m_balance;
public void deposit (int amount)
balance = balance - amount; }
public void withdraw (int amount)
balance = balance + amount;
public String getBalance ()
return Integer.toString (bal) + " " + CURRENCY;
Code read moe times than written. Important readable not wrteable
Same code. Just more meaningful variable names
Now find bug?
How many had tech student and said ‘write doc & test’, but have you looked at his/her code?
Code is bread & butter, hear of system.
Professional Coding Practices are key.. We go on to discuss this approach

22. Now imagine you have
A multi-domain, multi-lingual horizontal software application supporting 10,500 users in 42 countries composed of :
1.2 million Lines of Code
6, Java classes
10,000 HTML templates
many other files
Welcome to EDH!

23. EDH: Good Old Days (1991-98) “Imagination rules the world”
Mac or PC or Unix?
C or C++ or ?
University atmosphere
Freedom & Individualism
No advantage to collaborate
Choice, choice, choice...

24. Results Healthy outside, but unhealthy inside
Evolution from freedom to Chaos!
Development Platform : Mac, PC & Unix
Code : C,C++,Python, Prolog, ProC, PL/SQL, Perl...
Comments : Spanish, Italian, French & English...
Bugs : “Y2K don’t care”
Obvious code never reviewed : Why would you show your code to anybody? Never did at university... Results count!
Consequence : Maintenance became the primary resource-consuming activity

25. The “authoritarian” new days (1998...)
Autocratic
You will use a PC
You will use the chosen IDE
You will use Version Control
You will develop in Java
You will adhere to coding standards
You will show your code to others
Production Environment is Sacred
Scheduled Deploys
Team decision, No individual actions
No unreviewed code goes live
Doesn’t sound very motivating? Ironically it is more motivating!

26. From University to Industry
Individual Development Practices
Team Development Practices
Freedom of Choice for Development Environment
Uniform development environment
Free selection of tools
Common set of tools
Choice of language & technology
Single technology choice
Individual Code Responsibility (& blame)
Common Code Ownership (& learning)
Quality of the Product
Quality of the Process
... driven by the members of the team
(not management imposed)

27. Requires Concrete Practices
Code Reviews
Coding Standards
Design Reviews
Mentoring
Uniform development environment
Coherent set of development & deployment tools
Single language & Technology
Test procedures, unit testing & usability studies

28. Coding Standards – why? Why ?
80% of the lifetime cost of a piece of software goes to maintenance.
Hardly any software is maintained for its whole life by the original author.
Code conventions improve the readability of the software, allowing engineers to understand new code more quickly and thoroughly.
Cannot review unless you have standards...
endless debate – was driving too fast? Cannot answer without defined speed limits
Recommend best practices, avoid bad practices
Maintainable & reliable software is key
Produces
Common Code Ownership
Why
Reasons provide by java.sun.com
We have different reasons
Leave system anecdote
Believe me – we tried it!
Endless debate – driving too fast

29. Code Reviews - guidelines
Form
Product is guilty until proven innocent
Producer is innocent because he/she is not on trial
More likely to find bugs if you assume they are there
Evaluate product not producer
Emphasize "review" aspect; do not "fix it here".
Raise problems. Do not discuss solutions

30. Code Reviews - guidelines
Format
Three people minimum, seven people maximum
Roles :
Author
Moderator
Scribe
Reviewer(s)
Preparation
Short & businesslike (2 hrs max)
Not off-site. No telephones or interruptions
Donuts or other home goodies
Dismiss the guru who wants to demonstrate that their way is better

31. Code Reviews - guidelines
Management Involvement
NONE!
Not a manager's status meeting
Management is not represented during inspections
Inspections must not be used as a tool to evaluate workers
… Not a committee, not a working group, not a status report & not an appraisal instrument …

32. Benefits Primary objective Other benefits :
remove defects as early as possible in the development process
Other benefits :
Early Testing
Project Tracking
Educational – share best practices
Training of new/junior programmers
Improved Communication
Improved Individual Quality
Cross-training
Process-improvement
Shared Responsibility – no individual blame

33. The “Yes, buts...” I don’t have time for this...
Good programmer’s code doesn’t need reviewing
Its only a ‘minor’ piece of code
Code Changes, then what?
2nd pair eyes rule
Pair programming

34. Coding: Tools Atlassian JIRA Atlassian GreenHopper
Issue tracking and project tracking
EDH: every change must have a JIRA
Process should be as lightweight as possible
Atlassian GreenHopper
Agile project management (Scrum)
EDH: 2-week sprints
Atlassian Confluence
Documentation (WIKI style)

35. Coding: Tools (2) Atlassian Crucible Atlassian FishEye
Online code reviews
EDH: Every production line of code must be reviewed
Atlassian FishEye
Browse version control repository (CVS, SVN)
Real-time notifications of code changes
Web-based reporting, visualisation and code sharing
Atlassian Bamboo
Continuous integration
Atlassian Clover
Java code coverage metrics

36. Coding: Tools (3)

37. Testing
Bugs
Standard Software: 25 bugs per 1000 lines of program. Good Software: 2 errors per 1000 lines. Space Shuttle Software: < 1 errors per lines. Example Handy (Cellular Phone): lines of program: up to 600 errors. Windows-95: 10 Mill. lines: up to errors.
Sept – Jpeg buffer overrun bug in MS windows
“an attacker who successfully exploited this vulnerability could take complete control of an affected system, including installing programs; viewing, changing, or deleting data; or creating new accounts with full privileges”
Defects
Testing ≠ Debugging
You may have zero bugs, but s/w may not meet requirements, scale, respond-in time…

38. What do you test? Correctness testing Security testing
Reliability testing
Stress testing
Scalability testing
Performance testing
Usability testing

39. Testing
Software testing is an art : requires a tester's creativity, experience and intuition, together with proper techniques.
Most of the testing methods and practices are not very different from 20 years ago.
Testing is more than just debugging.
Testing is expensive. Automation helps.
Complete testing is infeasible. Tradeoff
Testing, while essential, may not be the most effective method to improve software quality.

40. And after that? Sofware Maintenance 80% of lifetime of software
The Legacy Crisis
The relative cost for maintaining software and managing its evolution now represents more than 90% of its total cost.
Example costs
Annual software maintenance cost in USA has been estimated to be more than $70 billion
Y2K : $8.38 billion US dollar, $90 million for Nokia
50% of their time is spent in the process of understanding the code!!!

41. Legacy Code
An average Fortune 100 company maintains 35 million lines of code
These companies add in average 10% each year only in enhancements
As a result, the amount of code maintained doubles in size every 7 years
E.g. 70% or more of the still active business applications are written in COBOL (at least 200 billion lines of COBOL-code still existing in mainframe computers alone)
Data based on variety of studies

42. Types of Maintenance Corrective Maintenance (21%)
A process that includes diagnosis and correction of errors.
Adaptive Maintenance (25%)
Activity that modifies software to properly interface with a changing environment (hardware and software).
Perfective Maintenance (50%)
Activity for adding new capabilities, modifying existing functions and making general enhancements.
This accounts for the majority of all effort expended on maintenance.
Preventive Maintenance (4%)
Activity which changes software to improve future maintainability or reliability or to provide a better basis for future enhancements.
Still relatively rare.

43. Results Maintenance Costs
Typical software organizations spend anywhere from 40 to 70 percent of all funds conducting maintenance.
Maintenance-bound organizations result : loss or postponement of development opportunities.
Customer dissatisfaction when requests cannot be addressed.
Reduction in overall software quality as a result of changes that introduce latent errors in the maintained software.

44. Your challenge ! Come in the 9% of projects on time & in budget
Engineer your software (design, review & maintenance in mind)
Are you an artist, scientist or engineer?
Art or Science?
Control the spiraling IT costs & improve the reputation of the industry

45. Thank You For More Information E-mail: Rostislav.Titov@cern.ch
Thank you very much for your attention:
For more information please look at our web-site:
Or send me an (in English – please)