1. CS 501: Software Engineering
Lecture 22
Delivering the System

2. Administration Final Presentation Sign up for the presentation.
The presentation should include a demonstration of your software in operation.
You will need a rehearsal.

3. CS 501 Final Presentation: What do You want to Achieve?
• Personal and team satisfaction in handing over a good piece of work to the client
• Complete the course in good style with good grade
• A clean handover without loose ends
Perhaps: a good basis for future involvement with the
client, team, or this project

4. Final Presentation: Who is the Audience? What do they Want?
Clients
The clients have invested effort in this project:
• Is it ready for production?
• Should they invest more effort to bring it into production?
• Should they abandon the project?
Course team
• What has been accomplished? What has been learned?
• Is the client satisfied?
• Are you handing over a maintainable system?

5. How much Time do You Have? How much can You Cover?
Plan for 45 minutes total. You should cover:
Presentation:
• Brief review of goals
• Honest summary of achievements and gaps
• Summary of what is being delivered
Demonstration of operational system:
• Show the system in operation
• Be honest about gaps, weaknesses, etc.
Time for discussion

6. Regression Testing
Regression Testing is one of the key techniques of Software Engineering
When software is modified regression testing is to provide confidence that modifications behave as intended and do not adversely affect the behavior of unmodified code.
• Basic technique is to repeat entire testing process
after every change, however small.

7. Regression Testing: Program Testing
1. Collect a suite of test cases, each with its expected behavior.
2. Create scripts to run all test cases and compare with expected behavior. (Scripts may be automatic or have human interaction.)
3. When a change is made to the system, however small (e.g., a bug is fixed), add a new test case that illustrates the change (e.g., a test case that revealed the bug).
4. Before releasing the changed code, rerun the entire test suite.

8. Documentation of Testing
Testing should be documented for thoroughness,
visibility and for maintenance
(a) Test plan
Test specification and evaluation
Test suite and description
(d) Test analysis report

9. A Note on User Interface Testing
User interfaces need two categories of testing.
• During the design phase, user interface testing is carried out with trial users to ensure that the design is usable. Design testing is also used to develop graphical elements and to validate the requirements.
• During the implementation phase, the user interface goes through the standard steps of unit and system testing to check the reliability of the implementation.
Acceptance testing is then carried out with users, on the
complete system.

10. A CS 501 Project: Methodology How we’re user testing:
The next few slides are from a CS 501 presentation (second milestone)
How we’re user testing:
One-on-one, min user tests with staff levels
Specific tasks to complete
No prior demonstration or training
Pre-planned questions designed to stimulate feedback
Emphasis on testing system, not the stakeholder!
Standardized tasks / questions among all testers

11. A CS 501 Project: Methodology
How we’re user testing:
Types of questions we asked:
Which labels, keywords were confusing?
What was the hardest task?
What did you like, that should not be changed?
If you were us, what would you change?
How does this system compare to your paper based system
How useful do you find the new report layout? (admin)
Do you have any other comments or questions about the system? (open ended)

12. What we’ve found: Issue #1, Search Form Confusion!
A CS 501 Project: Results
What we’ve found: Issue #1, Search Form Confusion!

13. What we’ve found: Issue #2, Inconspicuous Edit/ Confirmations!
A CS 501 Project: Results
What we’ve found: Issue #2, Inconspicuous Edit/ Confirmations!

14. What we’ve found: Issue #3, Confirmation Terms
A CS 501 Project: Results
What we’ve found: Issue #3, Confirmation Terms

15. What we’ve found: Issue #4, Entry Semantics
A CS 501 Project: Results
What we’ve found: Issue #4, Entry Semantics

16. What we’ve found: #5, Search Results Disambiguation & Semantics
A CS 501 Project: Results
What we’ve found: #5, Search Results Disambiguation & Semantics

17. Fixing Bugs Isolate the bug Intermittent --> repeatable
Complex example --> simple example
Understand the bug
Root cause
Dependencies
Structural interactions
Fix the bug
Design changes
Documentation changes
Code changes

18. Moving the Bugs Around Fixing bugs is an error-prone process!
• When you fix a bug, fix its environment
• Bug fixes need static and dynamic testing
• Repeat all tests that have the slightest relevance (regression testing)
Bugs have a habit of returning!
• When a bug is fixed, add the failure case to the test suite for the future.

19. Maintenance
Most production programs are maintained by people other than the programmers who originally wrote them.
(a) What factors make a program easy for somebody else to maintain?
(b) What factors make a program hard for somebody else to maintain?

20. CS 501: Software Engineering
Lecture 22
Delivering the System

21. Delivery: Summary A good delivery package results in: • happy client
• happy users
• less expense in support or maintenance
But most projects rush the packaging, help systems, and training materials, give them to the least experienced members of the team, do not test them properly, and generally neglect this part of the software process.

22. Delivering the System: Themes
• Different categories of software product need different packaging and delivery procedures.
• Packaging, support, maintenance and major failures are expensive. Trade-offs must be made.
• Pressures to get products to market and in operation often lead to bad decisions.
(In my experience, the pain of being late is often less than the pain of putting a bad system into operation.)
• Services, such as installation, training, configuration, etc. may be paid for separately.

23. Delivery of Software: Categories of Product
Shrink-Wrapped Package
• Installation scripts
-- automatic
-- varieties of hardware and operating systems
-- uninstall, reinstall, etc.
• Support (very expensive when it requires staff)
-- staff training
-- documentation (user, system administrator, expert user)
• Maintenance
-- client does not have source code
-- no bug fixing except with new release

24. Delivery of Software: Categories of Product
Data Processing System
• Acceptance
-- acceptance period may cover several months
-- client should be comfortable with complete system
• Support
-- client should be self-sufficient
-- documentation and training for system administrators
and operators
-- well organized source code for maintenance
-- maintenance and support contracts

25. Delivery of Software: Categories of Product
Embedded System
• Acceptance
-- hardware and software developed together
-- acceptance tests combination
• Maintenance
-- bug fixes require servicing the hardware
-- errors may be expensive or dangerous
• Support
-- training for support personnel
-- documentation and training for users

26. Training Time and money spent on training is usually well spent:
• one-on-one
• in-house training
• training courses
• distance education
• online tutorials
Development team needs to provide training materials:
• users (perhaps several categories)
• system administrators
• system maintainers
• trainers

27. Training and Usability
A well-designed system needs less training
• good conceptual model
• intuitive interfaces
Different skill levels need different types of training
• skilled users work from the conceptual model
• less-skilled users prefer cookbook sets of instructions
• occasional users will forget complex details, but remember general structure

28. Help Systems Resources
• A good help system is a major sub-project (time-consuming, expensive)
• A good help system saves user time and support staff (time-saving, cost-saving)
Help system design
• Users need many routes to find information (index by many terms, examples, mini-tutorials, etc.)
• Help systems need to be tested with real users

29. Documentation Online documentation • Much cheaper than print
• Fewer restrictions on numbers of pages, colors, etc.
• Easy to update (e.g., over the Internet)
but... Cannot be used if the user's system is down

30. Categories of Documentation
Software development
• Requirements, design
• Source code, test plans and results
User
• Introductory (various audiences)
• User manual
• Web site of known problems, FAQ, etc.
System administrator and operator
• System manuals
Business
• License, contract, etc.

31. Installation Tools
Creating installation scripts may be a major sub-project
• Different scripts, tools and procedures for different categories of software
• Testing must be extensive with real users in their own environment

32. CS 501: What Materials should you Deliver?
When you leave, all that the client has is your documentation and your software. Imagine that you work for the client and are asked to take over this system. What would you want?
• Materials can be in any format, need not be huge, but must be current.
• Place your materials on your GForge or other project site.

33. Delivery: Check List for CS 501 Projects
Documentation
• Requirements, updated to reflect delivered system
• System and program design, updated to reflect delivered system
• Instructions for: users, administrators, operators
• Presentation slides, updated to reflect delivered system
• Business documentation, e.g., copyright license
System
• Source code and matching binary for all programs
• Installation scripts, etc.
• Test scripts, test data, and test reports
Different projects will have different deliverables