Presentation is loading. Please wait.

Presentation is loading. Please wait.

April 3, 2015 IDS SEPG PROCESS … The GLUE that holds Software Projects Together A Focus on Software Testing Dennis A. Morrison Senior Principal Engineer.

Similar presentations


Presentation on theme: "April 3, 2015 IDS SEPG PROCESS … The GLUE that holds Software Projects Together A Focus on Software Testing Dennis A. Morrison Senior Principal Engineer."— Presentation transcript:

1 April 3, 2015 IDS SEPG PROCESS … The GLUE that holds Software Projects Together A Focus on Software Testing Dennis A. Morrison Senior Principal Engineer DD(X) System Architecture

2 April 3, 2015 IDS SEPG Presentation Outline Software Process Overview Software Testing Overview Source Code Control Trouble Reporting and Bug tracking Summary

3 April 3, 2015 IDS SEPG Software Phases (Waterfall Model) Requirements Design Conceive Code Test Build Deploy Support Deliver

4 April 3, 2015 IDS SEPG Software Phases (Process Glue) Requirements Design Code Test Deploy Support Requirements Review – Are the Requirements testable Artifacts – Review documentation; Requirements placed under documentation control Design Review – Does the design support requirements? Artifacts – Review documentation Code Review – Does the code support requirements and implement the design?? Artifacts – Review documentation; Code under source code control Test Review – Test plans Test Results – Did the tests pass? Artifacts – Review documentation; Test plans; System Trouble Reports (STR) Test Review – Customer Acceptance Test Test Results – Did the tests pass? Artifacts – Review documentation; STRs Customer Survey – Is the system meeting customer expectations Artifacts - Software Updates; STRs

5 April 3, 2015 IDS SEPG Software Phases and Loops Requirements Design Code Test Deploy Support Phases where most rework occurs Design Errors New feature requests or feature refinement Software Bugs New Feature Request Requirement Clarification or changes

6 April 3, 2015 IDS SEPG Software Testing An Art or a Science ? Testing is a phase of software development that can become, or seem, endless –Classically, the most expensive part of the project –Schedule overruns are often attributed to the testing cycle –Testers are usually squeezed into smaller and smaller test cycles How much testing is enough? How do you know when you are finished testing? How good is the product when you are finished?

7 April 3, 2015 IDS SEPG Testing Has Many Stages Test Unit Test White Box Testing Integration Test Grey Box Testing System Test Black Box Testing Acceptance Test Subset of Black Box Testing

8 April 3, 2015 IDS SEPG Phase = Unit Test Testing is focused on internal operations of the software component –Inputs/outputs –Logical operations –Limit testing –Limited functional testing with stubs Testing methodologies –Formal testing with test plan and review –Ad-hoc random –Typically, tests are manually executed Unit Test = White Box Testing

9 April 3, 2015 IDS SEPG Phase = Integration Test Testing is focused on functional aspects of an integrated set of software components –Inputs/outputs –Logical operations from outside of the software components –Limit testing (induced error paths from external stimuli) –Stubs are replaced by real software as they become available Testing methodologies –Formal testing with test plan and review –Semi-automated and manually executed –Automation yields regression test capability Integration Test = Grey Box Testing

10 April 3, 2015 IDS SEPG Phase = System Test Testing is focused on system level requirement verification –User Interface (Human to Computer Interface) –Requirement verification –Limit and normal operational testing –Stress and longevity testing Tests are executed on target hardware and software Testing methodologies –Formal testing with test plan and review –Typically, tests are semi automated and manually executed –Automation yields regression test capability System Test = Black Box Testing

11 April 3, 2015 IDS SEPG Phase = Acceptance Test Testing is focused on externally visible features –User Interface (Human to Computer Interface) –Operational focus –Interfaces with external systems in customer domain Other collaborative systems Customer network, etc. Tests are executed on target hardware and software usually at the customer’s site Testing methodologies –Formal testing with test plan and review –Typically, tests are manually executed Acceptance Test = Subset of Black Box Testing

12 April 3, 2015 IDS SEPG Some Software Testing Models Developer level testing –Applicable to small teams Community testing –Beta versions of a product are released to a volunteer community for functionality and user level testing Used by gaming community prior to product releases Microsoft uses this approach as one of their test phases Collaborative testing –Developers and customers test together Project level testing –Used by most large projects

13 April 3, 2015 IDS SEPG When Should Software Testing Begin? The software test cycle should begin during the project requirements phase Requirements review should result in the first level of test specifications Similar to the “Agile” software development process, test cases should be written and reviewed before the software is developed

14 April 3, 2015 IDS SEPG When is Testing Finished? When the schedule says you’re finished? When you run out of money? When the boss says to release it? When the tester’s say you’re done? When the customer says you’re done? When the law suits are filed?

15 April 3, 2015 IDS SEPG Requirements Verification Raytheon uses requirement verification and statistical mechanisms to determine when testing is complete Requirements are decomposed and allocated to software components Requirements are “sold off” with during formal testing events with stakeholders (customers) present

16 April 3, 2015 IDS SEPG Something you have Token Reader Something you know Pin Reader Something you are Biometric Reader Requirement Decomposition Req 1 Req 1.3Req 1.2Req 1.1 Req 1.3.2Req 1.3.1Req 1.2.1Req 1.1.2Req Operational Requirement Parent requirement decomposes into three child requirements The system shall provide three factor user authentication

17 April 3, 2015 IDS SEPG Requirements Role in Testing Software is tested against requirements –Before a parent requirement can be “sold off”, all children requirements must be fulfilled –Before a child requirement can be “sold off”, all of its siblings must themselves be “sold off” –The failure of any requirements will prevent the parent from being fulfilled

18 April 3, 2015 IDS SEPG Something you have Token Reader Something you know Pin Reader Something you are Biometric Reader Requirement Decomposition Req 1 Req 1.3Req 1.2Req 1.1 Req 1.3.2Req 1.3.1Req 1.2.1Req 1.1.2Req Operational Requirement Parent requirement decomposes into three child requirements The system shall provide three factor user authentication Req 1

19 April 3, 2015 IDS SEPG Requirements Traceability Software test cases are built from system use cases –Scenarios (rainy day and sunny day) are used to test various threads through the software –Every scenario is traced to parent requirements A scenario may have more than one parent requirement –Test cases must demonstrate interface level testing All system and software level interfaces are documented and tracked These are known as IRLLs, IRLEs, and IRLMs –IRLL = Interface Requirement Logical Level –IRLE = Interface Requirements Electrical Level –IRLM = Interface Requirements Mechanical Level Test cases and requirements are stored in DOORS

20 April 3, 2015 IDS SEPG Software Source Code Control The ability to save and retrieve versions of software is a key component to success in medium to large software projects –Software can be “versioned” for new features or software load lines –Changes made to software modules due to bug fixes can be isolated –Enables developers to work on next release while maintaining current versions –Software builds can include or exclude changes

21 April 3, 2015 IDS SEPG Using Source Control Systems Source Code Version 1 Source Code Control System Source Code Version 1 Put (ModA V1) get(ModA V1) Source Code Version 2 put(ModA V2) Mod A-V1.1 Mod X-V7 Mod Z-V2 SW Build Specification Software Build Source Code Version 1.1 Source Code Version 1 get(ModA V1) put(ModA V1.1) New Feature Bug Fix

22 April 3, 2015 IDS SEPG Trouble Reporting and Bug Tracking Software bugs are documented using formal mechanisms –CMTS – Change Management Tracking System (MRs) –WSTR – Web System Trouble Report (STRs) Bugs are considered addressed when software has been modified, tested, and submitted to the load-line Bugs are considered fixed when they have been verified Bug fixes are pulled into the software load via the source code control system Bug reports and time based metrics guide management in software release decisions

23 April 3, 2015 IDS SEPG

24 April 3, 2015 IDS SEPG Test Methods Manual –Usually applied during unit test Semi Automated –Applied during integration test Fully Automated –System test

25 April 3, 2015 IDS SEPG Test Automation Test automation can provide a project with huge dividends Test scripts –Drive test automation engines –Can be used in unit testing –Can be reused by multiple test organizations Regression tests can be constructed from reusable test scripts

26 April 3, 2015 IDS SEPG Metrics Drive Behavior An organization or individual’s behavior is directly related to the metrics they are measured by –Defects found and fixed –On time delivery –Productivity SLOC counts Defect rate at delivery –Complexity indices –Customer satisfaction

27 April 3, 2015 IDS SEPG Best Practices? No real silver bullets Process is the glue that holds a project together The right solution depends on the project but is often a combination of methods/processes and tools Just remember that customer satisfaction trumps all!!

28 April 3, 2015 IDS SEPG BACKUP SLIDES

29 April 3, 2015 IDS SEPG In the world of software development, process is the “glue” that holds the project and team together. Too little, the pieces fall apart; too much and it can become a sticky quagmire resulting wasted effort, time, and money. The challenge facing many projects is finding just the right balance point on the process continuum. As a project progresses through its life cycle, the balance point will shift and the project will need to continuously review and adjust to maintain optimal performance. A typical software project life cycle is comprised of six distinct phases: Requirements, Design, Code, Test, Deployment, and Support. Most projects place a significant amount of emphasis on the front-end phases of the life cycle e.g. requirements, and design. However, industry data clearly demonstrates that the largest percentage of a projects budget is spent during the final phases, namely test, deployment, and support. These phases encapsulate much of the “rework” experienced by the project. This talk will describe the process philosophies and tools used by Raytheon to tackle large-scale multisided software development projects. ABSTRACT

30 April 3, 2015 IDS SEPG Tools The right tool can result in huge pay backs Source Code Control Tools Trouble Report tracking Stress testing Performance testing

31 April 3, 2015 IDS SEPG Testing Methodologies Ad-hoc –Non structured testing with or without test artifacts –No test plans –Non repeatable tests Formal testing –Test plans –Test Artifacts –Defect tracking

32 April 3, 2015 IDS SEPG Software Phases (Spiral Model) Spiral Development: Experience, Principles, and Refinements Spiral Development Workshop February 9, 2000 Barry Boehm, edited by Wilfred J. Hansen

33 April 3, 2015 IDS SEPG The NATO Sea Sparrow Surface Missile System (NSSMS) is a quick reaction, point defense system designed to protect the ship it is on. It is very, very effective against enemy aircraft, low flying sea skimming missile and even bomb laden inflatable boats. It was first developed by Raytheon in the '70's and installed on over 150 ships world-wide. In the mid '90's the Navy contracted Raytheon to re-architect the system so that it would be capable of firing the Navy's newest missile the Evolved Sea Sparrow Missile (ESSM). The Navy's brand new nuclear powered aircraft carrier, the USS Ronald Reagan, has 6 Seasparrow radars & 3 Seasparrow missile launchers. The movie shows the Navy doing Qualification Testing on these systems.

34 April 3, 2015 IDS SEPG Software Phases (Process Glue) Requirements Design Code Test Deploy Support Requirements Review – Do the requirements support the customer’s intent? Are they testable? Are they complete (sunny day/ rainy day) Artifacts – Review documentation; Requirements placed under documentation control Design Review – Does the design support requirements? Artifacts – Review documentation Code Review – Does the code support requirements and implement the design?? Artifacts – Review documentation; Code under source code control Test Review – Test plans – Do the tests adequately exercise the requirements? Test Results – Did the tests pass? Artifacts – Review documentation; Test plans; System Trouble Reports (STR) or Modification Requests (MR) Test Review – Customer Acceptance Test Test Results – Did the tests pass? Artifacts – Review documentation; System Trouble Reports (STR) or Modification Requests (MR) Customer Survey – Is the system meeting customer expectations Artifacts - Software Updates; STRs


Download ppt "April 3, 2015 IDS SEPG PROCESS … The GLUE that holds Software Projects Together A Focus on Software Testing Dennis A. Morrison Senior Principal Engineer."

Similar presentations


Ads by Google