1. Requirement & release management Marko ”NarsuMan” Rintamäki 2012

3. What means Requirement Manangement? What means a feature, requirement, use cases, and user story

4. Different points of view on IFDK system Seller User Developer

5. Scalability Stability Performance Security Performance Stress Usabilty Different points of view on system Requirement Category‘s Requirements from product development point of view „scalability “ „Stability “ „Functionality“

6. Between 40% and 60% of software failures and defects are the result of poor software management and requirements definition. In plain English, this means that about half of the problems encountered could have been avoided by making it clear, from the very beginning, what the customer expected from the respective project. This is to say that the programming was fine and the developers did their job well – only they did a different job from what they were supposed to. http://www.softwareprojects.org Read also http://www.projectshrink.com/why-requirements-change-270.html

7. ● You can find requirements from several sources: Users, Stakeholders, Business and Development team and many others ● Requirements are trying to define nature of feature/system/solution more specific than common written document does. This information is helping development team to design a solution for a need ● There is several common methods to define and gather requirements. – Traditional Requirement modeling (http://en.wikipedia.org/wiki/Requirements_management) – RUP/UML based Use Case modeling (http://en.wikipedia.org/wiki/IBM_Rational_Unified_Process) – Agile XP oriented User Story’s (http://en.wikipedia.org/wiki/Agile_software_development) ● Your task is to figure out a small difference between them Read more http://en.wikipedia.org/wiki/Requirements_management

8. System Testing Integration Testing Unit Testing Customer Requirements System Requirements Design Requirements Component Requirements Acceptance Testing Why we need requirements from testing point of view? Implementation „Developer's Area“ „Test Engineers Area“ ”Traditional Testing Levels”

9. Feature Feature is functionality of product/software which can be seen as one module of whole product Internal Flame kit has WLAN support Internal Flame Kit has touch screen user interface Feature is functionality of product/software which can be seen as one module of whole product Internal Flame kit has WLAN support Internal Flame Kit has touch screen user interface Do some googling!! Create a wiki page!! AboutUserStory Do some googling!! Create a wiki page!! AboutUserStory

10. Idea#1 Who is defining a Feature? Customer I would like to have Internal Flame Drum Kit Could you deliver it to us? I would like to have Internal Flame Drum Kit Could you deliver it to us? Actually We have several features for it here Actually We have several features for it here Ok! What's a plan Ok! What's a plan Nice looking feature propoals. We have to do some evaluation Nice looking feature propoals. We have to do some evaluation Idea#2 Idea#3 Idea#4 Idea#1 -Technology? -Knowledge -Resource -Solution? -Priority?

11. Calory Counter Drum Metronome Table Drum Mode Standby Mode MIDI Support Touch Screen with single tap

12. Core Software Calory Counter Drum Metronome Table Drum Mode Standby Mode MIDI Support Touch Screen with single tap

13. Play problem domain card game with team to search for features?

14. Feature X * n Calory Counter: Player can measure calories during training session. This can be seen as exercise result in web service eg. Facebook application Energy usage

15. Table Drummer: Player drums table board instead of drum can. IFDK kit is able to use DSP algorithm to detect correct drum sound from environment. In training mode IFDK is trained to detect drum sounds for environment. Table Drum Mode DSP Algorithm DSP Algorithm

16. Calory Counter Drum Metronome Table Drum Mode Simple Training Mode MIDI Support Touch Screen with single tap Customer Type 1 Customer Type 2 Customer Type 3Customer Type 4 Who are our target customers?

17. Drum Metronome Table Drum Mode Simple Training Mode MIDI Support Touch Screen with single tap Customer Type 1 Customer Type 2 Customer Type 3Customer Type 4 What is our key customer? Primary Target Calory Counter Secondary Target

18. Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1User Story #1 User Story #2 User Story #3 Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1 User Story #1 Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1User Story #1 User Story #2 User Story #3 Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1 User Story #1 User Story #2 User Story #3 Requirement USE CASE #2 USE CASE #1 USE CASE #3 Requirement USE CASE #1 Requirement USE CASE #2 USE CASE #1 Requirement USE CASE #1 Features and release planning Release 0.1 Release 1.1 Release 1.2 Feature: Simple Training Mode Feature: Table Drum mode Feature Touch Screen with single tap Release 1.0 TIME TO MARKET!! For Target Group 3 CORE/Platform Software Development TIME TO MARKET!! For Target Group 2 TIME TO MARKET!! For Target Group 1

19. Defines Feature X * n Functional Requirements Functional Requirements Non-Functional Requirements Non-Functional Requirements Use Cases Vision of product Simple Requirement Management Process Problem DomainSolution Domain Solution Proposal Customer/Marketing/ business User Storys FEATURE VISION/NEED/PROPOSAL FEATURE VISION/NEED/PROPOSAL YOU! Design documents & implementation Test Case

20. A requirement shall be a complete sentence. Sentence has to be understandable, measurable and testable ReqId1 - Tractor has four wheels ReqId2 - Tractor has one exhaust pipe ReqId3 - Engine of tractor is capable of use flexi fuel ReqId4 – The tractor has a hook for trailer ReqId5- The tractor shall have a enhanced driving system ReqId1 - Tractor has four wheels ReqId2 - Tractor has one exhaust pipe ReqId3 - Engine of tractor is capable of use flexi fuel ReqId4 – The tractor has a hook for trailer ReqId5- The tractor shall have a enhanced driving system Google: requirement specification template and SRS Software Requirement Specification

21. Functional Requirements Functional Requirements Non-Functional Requirements Non-Functional Requirements Functional Requirement "User can select application from ui by using wheel button ” ”Tractor can be driven both directions” Functional Requirement "User can select application from ui by using wheel button ” ”Tractor can be driven both directions” Non-Functional Requirement "Performance Requirement" ”Tractor Startup should take minimum 10 seconds” ”Usability Requirement” ”User interface should be able to control using simple wheel quide” ”The hook can last max 20Kkg trailer load” Non-Functional Requirement "Performance Requirement" ”Tractor Startup should take minimum 10 seconds” ”Usability Requirement” ”User interface should be able to control using simple wheel quide” ”The hook can last max 20Kkg trailer load” How it works? How fast it is? How stable it is? Do some googling!! Create a wiki page!! AboutUserStory Do some googling!! Create a wiki page!! AboutUserStory

22. Functional Requirements Functional Requirements Non-Functional Requirements Non-Functional Requirements Functionality Stability Security Performance Usability User Interface Design? Feature X Traditional Requirement Modelling and Features.......

23. Functional Requirements Non-Functional Requirements

24. Scalability How our implementation is scaling in situation X? Scalability How our implementation is scaling in situation X? Stability Is our implementation stable on situation like zzzZZZ? Stability Is our implementation stable on situation like zzzZZZ? Functionality Implementation should work like this way Functionality Implementation should work like this way Security Is our implementation secure enough against attack type xxx? Security Is our implementation secure enough against attack type xxx? Performance How good performance our implementation provides against competitor ? Performance How good performance our implementation provides against competitor ? Stress How much we can stress our implentation without a problems? Stress How much we can stress our implentation without a problems? Usabilty Is implementation usable for target customer? Usabilty Is implementation usable for target customer? Maintenance Is implementation easy to maintain? Maintenance Is implementation easy to maintain?

25. IDEAL Requirements help Design Requirements help Design REQ-X REQ-Y REQ-Z REQ-O REQ

26. Defines Feature X * n Functional Requirements Functional Requirements Non-Functional Requirements Non-Functional Requirements Use Cases Vision of product Simple Requirement Management Process Problem DomainSolution Domain Solution Proposal Customer User Storys FEATURE VISION/NEED/PROPOSAL FEATURE VISION/NEED/PROPOSAL YOU! Design documents & implementation Test Case

27. ● Defined by Ivar Jacobson http://en.wikipedia.org/wiki/Use_case ● Used with UML (Unified Modeling Language) http://en.wikipedia.org/wiki/Unified_Modeling_Language http://en.wikipedia.org/wiki/Unified_Modeling_Language ● RUP (Rational Unified Process) ● http://fi.wikipedia.org/wiki/RUP

28. Use cases are used for define functional requirements!

29. USE CASE Written scenario for action. Also execeptions included Use Case: Open IFDK Main Application Actor: IFDK User Step1: Gadget User touches home button Step2: UI wakeup initiated (if standby) Step3: Home screen is activated Setp4: User browses applications specific icons using wheel button Step5. Icons are moving on screen left and right Step6: User selects application by pushing wheel button Step7: Application starts up <4 seconds Execptions: 1. If application cannot start there will be note on screen about problem USE CASE Written scenario for action. Also execeptions included Use Case: Open IFDK Main Application Actor: IFDK User Step1: Gadget User touches home button Step2: UI wakeup initiated (if standby) Step3: Home screen is activated Setp4: User browses applications specific icons using wheel button Step5. Icons are moving on screen left and right Step6: User selects application by pushing wheel button Step7: Application starts up <4 seconds Execptions: 1. If application cannot start there will be note on screen about problem Use Case A use case in software engineering and systems engineering, is a description of steps or actions between a user (or "actor") and a software system which leads the user towards something useful. [1]software engineeringsystems engineeringactor [1] Wikipedia

30. ACTOR Use Case SYSTEM Actor: System: IFDK kit Use Cacenario: Standby mode after boot 1.User turn’s on IFDK by pressing red button on front panel 2. Screen wil flash and show welcome text ”Hello my friend!” 3.User interface opens after seconds 4.Screen will show three selection buttons 5.After 30 seconds user inteface goes to standby mode Exeption: 1. If user activates screenby tapping it standby counter will be reseted Actor: System: IFDK kit Use Cacenario: Standby mode after boot 1.User turn’s on IFDK by pressing red button on front panel 2. Screen wil flash and show welcome text ”Hello my friend!” 3.User interface opens after seconds 4.Screen will show three selection buttons 5.After 30 seconds user inteface goes to standby mode Exeption: 1. If user activates screenby tapping it standby counter will be reseted

31. USE CASE:_____________________________________________ Actor:__________________________________ Scenario: Execeptions: USE CASE:_____________________________________________ Actor:__________________________________ Scenario: Execeptions:

32. ● http://en.wikipedia.org/wiki/User_story http://en.wikipedia.org/wiki/User_story

33. ● Used in agile development process for requirement definition and gathering ● Backlog Item ● Simple way to describe needed functionality

34. Defines Feature X * n Functional Requirements Functional Requirements Non-Functional Requirements Non-Functional Requirements Use Cases Vision of product Simple Requirement Management Process Problem DomainSolution Domain Solution Proposal Customer User Storys FEATURE VISION/NEED/PROPOSAL FEATURE VISION/NEED/PROPOSAL YOU! Design documents & implementation Test Case

35. USER STORY Examples Simple phrase describes a need. This can lead to several other storys! "As user I would like to open application easily” As a user I would like to select application from icon list on screen As a user I would like to configure amount of application icons on screen "As a user I would like to use wheel for speeding up selection process" "As a user I would like to initate application fast enough" "As a tractor driver I would like to have enhanced driving system” USER STORY Examples Simple phrase describes a need. This can lead to several other storys! "As user I would like to open application easily” As a user I would like to select application from icon list on screen As a user I would like to configure amount of application icons on screen "As a user I would like to use wheel for speeding up selection process" "As a user I would like to initate application fast enough" "As a tractor driver I would like to have enhanced driving system” User Story

36. USER STORY: As a bad behavin person I cannot access IFDK using wlan without encryption How to test? Acceptance Criteria? USER STORY: As a bad behavin person I cannot access IFDK using wlan without encryption How to test? Acceptance Criteria? USER STORY: As a member of audience I can hear effect sound that player is having electrical shocks How to test? Acceptance Criteria? USER STORY: As a member of audience I can hear effect sound that player is having electrical shocks How to test? Acceptance Criteria?

37. Agile Requirement Management Epic Story User Story UserStory0001 RequirementId0002 RequirementId0003 EpicStory0001 As a user I would like to use product Which is fast to power on As a user I would like to use product Which is fast to power on As a Customer I would like to have top quality product As a Customer I would like to have top quality product NOTE: Gadget should have >30fps UI performace NOTE: Gadget should have >30fps UI performace NOTE: Gadget should Startup <5seconds NOTE: Gadget should Startup <5seconds Acceptance Criterias?

38. EPIC STORY: As a user I would like to use product which is fast to startup Story Points: Notes: Gadget should startup <5seconds How to show it is tested? (eg. Acceptance Criteria) Use timer to verify startup time. Measure time from power on to main screen activated. Time EPIC STORY: As a user I would like to use product which is fast to startup Story Points: Notes: Gadget should startup <5seconds How to show it is tested? (eg. Acceptance Criteria) Use timer to verify startup time. Measure time from power on to main screen activated. Time USER STORY: As user I would like to see startup message on screen, so I could be sure a gadget is started up Story Points: Notes: Gadget should startup <5seconds How to show it is tested? (eg. Acceptance Criteria) Use timer to verify startup time. Measure time from power on to main screen activated. Time USER STORY: As user I would like to see startup message on screen, so I could be sure a gadget is started up Story Points: Notes: Gadget should startup <5seconds How to show it is tested? (eg. Acceptance Criteria) Use timer to verify startup time. Measure time from power on to main screen activated. Time

39. USER STORY: Story Points: Notes: How to show it is tested? (eg. Acceptance Criteria) USER STORY: Story Points: Notes: How to show it is tested? (eg. Acceptance Criteria) USER STORY: Story Points: Notes: How to show it is tested? (eg. Acceptance Criteria) USER STORY: Story Points: Notes: How to show it is tested? (eg. Acceptance Criteria)

40. Requirement Measurable Testable Requirement Measurable Testable Use Case, User Story, Requirement USE CASE Written scenario for action. Also execeptions included Use Case: Open Application Actor: Gadget User Step1: Gadget User touches home button Step2: UI wakeup initiated (if standby) Step3: Home screen is activated Setp4: User browses applications specific icons using wheel button Step5. Icons are moving on screen left and right Step6: User selects application by pushing wheel button Step7: Application starts up <4 seconds Execptions: 1. If application cannot start there will be note on screen about problem USE CASE Written scenario for action. Also execeptions included Use Case: Open Application Actor: Gadget User Step1: Gadget User touches home button Step2: UI wakeup initiated (if standby) Step3: Home screen is activated Setp4: User browses applications specific icons using wheel button Step5. Icons are moving on screen left and right Step6: User selects application by pushing wheel button Step7: Application starts up <4 seconds Execptions: 1. If application cannot start there will be note on screen about problem USER STORY Simple phrase describes a need. This can lead to several other storys! "As user I would like to open application easily" "As a user I would like to use wheel for simplify ui interaction" "As a user I would like to initate application fast enough" USER STORY Simple phrase describes a need. This can lead to several other storys! "As user I would like to open application easily" "As a user I would like to use wheel for simplify ui interaction" "As a user I would like to initate application fast enough" Non Functional Requirement "Performance Requirement" "Application Startup should take minimum 4 seconds" Non Functional Requirement "Performance Requirement" "Application Startup should take minimum 4 seconds" Functional Requirements "User can select application from ui by using wheel button " Functional Requirements "User can select application from ui by using wheel button "

41. Requirements and Architecture Design Business and customer Development Team Business Requirements Business Requirements Technical Requirements Technical Requirements NEED SOLUTION Design & Architecture Agreement “One of sides should not dominate in design process”

42. http://www.sysml.org/ UML OMT System Engineering Google: SysML, UML, Systems engineering, Software Design, Code Software Engineering - Architecture SysML UML OMT.NET, JAVA, C++ Software Engineering - Design Requirements http://www.sysml.org/

44. http://www.opfro.org/ http://www.google.fi/url?sa=t&rct=j&q=requiremen t%20specification%20example&source=web&cd=1 &ved=0CBgQFjAA&url=http%3A%2F%2Fwww.opfro. org%2FComponents%2FWorkProducts%2FRequire mentsSet%2FSystemRequirementsSpecification%2F SystemRequirementsSpecificationExample.doc&ei= aiWETqjnG- P04QTMzNivDw&usg=AFQjCNFd4r5LLJgQj3foixq_jx ZHjk78pQ&sig2=ILMCdBta4j8JVTCbQfgZgQ&cad=rja Interesting Links ?

45. Features Test Case Use Cases User Storys Product Design & Implementation Requirements Test Case Ready to test Testing & Quality Assurance Can we deliver Product Ready to Deliver ? Customer Not ready to deliver

46. SW Development Process (Waterfall) Requirement Gathering/Evaluation Requirement Gathering/Evaluation Design Implementation Verification & Validation Verification & Validation Maintenance Error Managment Process Error Managment Process Task1 Mile Stone 1 Mile Stone 2 Mile Stone 3 Task1

47. SW Development Process (Agile) User Story Y Sprint Task1 Task2 Task3 Sprint Task4 Task5 Task6 User Story X Design Implementation Verification Design Implementation Verification Product Backlog Design Implementation Verification Design Implementation Verification Design Implementation Verification Design Implementation Verification Design Implementation Verification Design Implementation Verification User Story Z Task7 Task8 Task9 User Story Z

48. Customer/Business Requirements Customer/Business Requirements Sub System Requirements Sub System Requirements Component Requirements Component Requirements Component / Unit Testing Component / Unit Testing Integration Testing Integration Testing System Testing System Testing Acceptance Testing Acceptance Testing System Requirements System Requirements IFDK System Verification and Validation IFDK System Verification and Validation Use Cases User Storys Features Requirements Validation = Are we building the right product? Verification = Are we building the product right? Validation = Are we building the right product? Verification = Are we building the product right? Architecture& Design& Implementation Architecture& Design& Implementation Product VALIDATION VERIFICATION IFDK Product Ideas

49. Customer Requirements Sub System Requirements Component RequirementsComponent/Unit Testing Integration Testing System Testing Acceptance Testing System Requirements Verification and Validation Verification = Are we building the product right? Validation = Are we building the right product?

50. Component Design Class Attributes Class Methods Class Attributes Class Methods

51. ● Testing according ISEB standard Functional Testing Testing based on an analysis of the specification of the functionality of a component or system. See also black box testing.) Non-Functional Testing Testing the attributes of a component or system that do not relate to functionality, e.g. scalabilty, stability, reliability, efficiency, usability, maintainability and portability. http://www.bcs.org/upload/pdf/glossary-current.pdf

52. Black Box vs White Box Testing Unit Testing is White Box testing System Testing is Black Box Testing System Testing is Black Box Testing ? ? ? ?

53. We need to capsule all projects as sub projects, which have more independence Project Teams has to have own test engineer –> Integration Test Engineer this lead’s to better defect prevention Integration Test Engineer is part of project team, part of team at start of project Test Engineer provides valuable information for design and enables better testability for project product Define new roles for validation engineers Check next page Lower level for test automation Enables regression test (method to block new implementation defects) Support for executing more complex test scenarios Continuous test execution (stability, performance gain etc) ● Notes

54. Test Level1 – Component/Unit testing Component/Class level unit testing (eg. xUnit framework) Test Level2 - Integration Testing (Feature, Application) Integration test for feature/application with “stub” interface components Functionality SW Testing Non-Functional SW Testing according needs (eg. feature/application specific performance, stability) Test Level3 – Regression Testing (Target HW+Android platform, tool, terminal etc.) Functional SW testing for all possible features Non-Functional SW testing according needs Non-Functional HW Testing according needs Test Level4 – System Integration Testing (Target Platform, Tool, Terminal etc) Functional SW testing for all new features Non-Functional SW Performance, Stability, Scalability Test Level5 – Acceptance Testing Functional/Non-Functional test according customer requirements HWT1 – Hardware Testing HW Integration test with limited environment. Conducted performance verified ● Testing Level descriptions

55. ● Testing Levels (with TAF = test automation framework ) IFDK Release v1.0 Feature Project 2 Regression Testing (ISEB) System Integration TAF Component/ Unit Testing (ISEB) Integration Testing (ISEB) System Testing (ISEB) Acceptance Testing (ISEB) Feature Project 1 TAF TAF Project TAF enables TAF Supports Verification and Validation

56. Refrerences & Links http://www.rbcs-us.com/images/documents/The-ISTQB-Advanced- Syllabus.pdf

57. VERIFICATION & VALIDATION Validation = Are we building the right product? Verification = Are we building the product right?

58. Create a Test Case! Functional? Your Sources For Test Case Functional Test Case: ● Verify functionality of XXXX Non-Functional Test Cases ● Verify Stability of XXXX ● Verify Performance of XXX ● Verify Security of XXXX ● Verify Usability of XXXX ● Verify Scalability of XXXX ● etc... Non-Functional? Which Type? ● Requirement ● Use Case ● Feature ● User Story ● Customer's Idea ● Brainstorm ● Intitution ● Exploratory Check also..... ● Correct functionality path ● Miss-usage of functionality ● Boundary Check Check also..... ● Check Possiblity to automated testing? How to create Test Case??? Regression Test Case?? Write a Case

59. Verify what? Using configuration? With tools? What is verdict? ● Verify drum track player pause mode functionality. ● Do this with IFDK software release X and playing song ”Show must go on by Freddy Mercury” ● Test should be done using android emulator environment and using your hands, ears and eyes” Add Information about case ● Pre State: ● Android emulator is running ● Release X is installed on emulator ● Test Case Steps: ● 1. Open drum kit player application ● 2. Select song ”Show must go on” ● 3. Start to play ● 4. Press Pause and check song is paused ● 5. Check memory usage from system application ● 6. Press Play ● 7. jump to 4 several time (<10) ● 8. Listen song to the end ● 9. Exit player using ”exit button” ● End State: ● IFDK Kit in main screen mode ● Test Case Id ● Test Case owner/writer ● Date ● comments ● If Pause is working result is PASS. If Pause mode failed result is FAIL Define pre-state Define Steps Define end-state What Information Test Case should contain?

60. Component /Unit Testing Class Attributes Class Methods Class Attributes Class Methods Class Attributes Class Methods Class Attributes TestClass Methods Unit Test Frame Work Result

61. Integration testing using emulator

63. Test Case SUT/DUT IFDK What is generated as results from test case execution LOG FILE EVENTS NOTIFICATIONS SUT = System Under Test DUT = Device Under Test ENVIRONMENT ANDROID EMULATOR TOOLS scripts/grep TEST CASE

64. ● TEST CASE ID XXXXX ● Step1 ● Step2 ● Step3. ● Step4. INCIDENT (Huomio) Bug/Defect (Vika) Defect Database IFDK System EXECUTE TEST ! Test Engineer Notes Reports What means error reporting?

65. Component Testing Customer/Business Requirements Customer/Business Requirements Sub System Requirements Sub System Requirements Component Requirements Component Requirements Component / Unit Testing Component / Unit Testing Integration Testing Integration Testing System Testing System Testing Acceptance Testing Acceptance Testing System Requirements System Requirements Architecture& Design& Implementatio n Architecture& Design& Implementatio n Product VALIDATION VERIFICATION

66. How to verify component implementation -Unit Testing -Code Coverage -Branch Coverage -Complexity Analyse -Unit Testing -Code Coverage -Branch Coverage -Complexity Analyse

67. Code Complexity Example tool CCCC

68. Branch coverage The percentage of branches that have been exercised by a test suite. 100% branch coverage implies both 100% decision coverage and 100% statement coverage.

69. Code Coverage An analysis method that determines which parts of the software have been executed (covered) by the test suite and which parts have not been executed, e.g. statement coverage, decision coverage or condition coverage. http://en.wikipedia.org/wiki/Code_coverage http://www.atlassian.com/software/clover/

70. Release/Configuration Management & Integration Testing Day 6

71. Putting all tools together! Continous Integration http://hudson-ci.org/

72. Design verification - Unit Testing

73. Unit Test example Re-run with Meego Example of xUnit in QT environment

74. Integration Testing Customer/Business Requirements Sub System Requirements Component Requirements Component / Unit Testing Integration Testing System Testing Acceptance Testing System Requirements Architecture& Design& Implementation Product VALIDATION VERIFICATION

75. Why Integrate first? Avoid Big Bang! HW Component Data Base Component/Application 10% tested Component/Application 10% tested Web Service Tested Component/Application

76. Integration Test with stubs Tested Component/Application Log STUB/MOCK Component Scripted STUB Interface Control Configure Control Configure Simulated Interface Simulated Interface Messages/Events STUB/MOCK Component Control Interface

77. in practice #1 IFDK android setup Tested Component Application Tested Component Application Activate/Control STUB/MOCK Component Scripted STUB Interface Control Configure Control Configure Simulated Interface Simulated Interface Messages/Events WEB SERVER simulating Facebook interface WEB SERVER simulating Facebook interface Control Interface Trace/Log

78. in practice #2 server component testing Tested Component Application Tested Component Application Trace/Log Activate/Control Mock Server/Daemon Scripted STUB Interface Automated Test Interface Automated Test Interface Simulated Interface Simulated Interface Messages/Events WEB SERVER Control Interface Operating System Needed Fake Application Needed Fake Application Junit Scripted Interface

79. Release Management and Integration Testing IFDK Application Facebook Web Service HW Component Calore Meter Enabled Drum Stick HW Component Calore Meter Enabled Drum Stick Calore Meter SW Component 10% tested Calore Meter SW Component 10% tested Data Base Schema Design Stubs neede d Stubs neede d Stubs neede d Stubs neede d Stubs neede d Stubs neede d Stubs neede d Stubs neede d First System Test With all components First System Test With all components

80. Trace/Log as feedback Log contains important information. Log Simple tool for log analysing in Linux ”grep” command, TAIL -F /var/log/messages | grep error -Specific Inhouse log capturing and analyse tools

81. Agenda Brief guidance for NEST Project Platform 1.3 Background story for reference project IFDK Some Definitions SW Development Process Release + Configuration Management Testing Levels and Error Management Hands On: Test Link + Bugzilla Error Reporting, Metrics and daily usage Closed Software Project vs Open Source Project Discussion

82. Error/Bug/Defect Report ● Defect/Burg/Error ID ● Reporter ● Time ● Founded where ● Which way? ● Test Case ● Test Setup/Configuration ● Describe scenario? ● Attachements? Picture/Log/etc.. ● Defect/Burg/Error ID ● Reporter ● Time ● Founded where ● Which way? ● Test Case ● Test Setup/Configuration ● Describe scenario? ● Attachements? Picture/Log/etc..

83. About Error Management Requirement Management Requirement Management Implementation Process Implementation Process Verificati on& Validation Verificati on& Validation Failure Report Failure Report

84. Definitions Failure - Fault, Defect, Bug - Incident, Failure, Error Example forum thread: http://www.allinterview.com/showanswers/36257.htmlhttp://www.allinterview.com/showanswers/36257.html ISTQB syllabus

86. Release Management Version 0.1 Version 0.2 Version 0.3 Version 0.2.1 Version 0.2.2.1 Version 0.2.2 Version 0.4 Trunk Customer 1 Version 0.2.3 Version 0.2.2.2 Version 0.2.2.3

87. Release & Configuration Managmement Version 0.1 Version 0.2 Version 0.3 Version 0.2.1 Version 0.2.2.1 Version 0.2.2 Version 0.4 Trunk Customer 1 Version 0.2.3 Version 0.2.2.2 Version 0.2.2.3 Feature s Release 1.0 Feature s

88. Release Planning

90. Validaton& Verificaton (Testing) Management Version 0.4 Version 0.2.2.2 Version 0.2.2 Test Plan Test Cases For Features Test Plan Test Cases For Features Tested Release/configuration Error/De fect Report Error/De fect Report Error/De fect Report Error/De fect Report Error/De fect Report Error/De fect Report

91. Feature Pack Project Manager Designer/Coder Integration Test Engineer Integration Test Engineer Test Manager System Testing IFDK System Testing Feature Unit/Integration Testing IFDK System Acceptance Testing System Test Engineer System Test Engineer Test Automation Engineer Test Automation Engineer Acceptance Test Engineer Acceptance Test Engineer Validation Verification IFDK Verification/Validation (Testing Organization) Regression Testing Integration Testing System Testing Acceptance Testing Test Automation Unit Testing Product Release

93. Example Sources for error report CRM Field Testing Testing Process Customer Feedback / Customer Feedback / Error Report Change Request? Change Request? N x Incidents

94. Change Management Sometimes founded defect can lead to change Bug? Change Request? Not Clear Requirements Not Clear Requirements Feature ?

95. Error Reporting and Process

96. Hands On: Bugzilla Error Database http://www.bugzilla.org/ http://www.bugzilla.org/installati on-list/ http://www.bugzilla.org/ http://www.bugzilla.org/installati on-list/ What is Bugzilla? Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe. What is Bugzilla? Bugzilla is a "Defect Tracking System" or "Bug-Tracking System". Defect Tracking Systems allow individual or groups of developers to keep track of outstanding bugs in their product effectively. Most commercial defect-tracking software vendors charge enormous licensing fees. Despite being "free", Bugzilla has many features its expensive counterparts lack. Consequently, Bugzilla has quickly become a favorite of thousands of organizations across the globe.

97. Other Defect Database Solutions JIRA – Commercial Requisite Pro – Commercial Rational Synergy - Commercial Mantis – Open Source

99. Bugzilla

100. Reporting, Metrics and daily usage

101. CMMI Process framework CMMI – covers ”error management” in several process areas SP3.2 Analyze Verification results Typical Work Products: Trouble reports - Analyze the verificationd ata on defects - Record all results of the analysis in a report - Provide infromation on how defects can be resolved (including verfiocation methods, criteria, and verification environment) and initiate corrective action Project Monitoring and Control SG2 Manage Corrective Action to Closure SP2.1 Analyze Issues SP2.2 Take corrective Action SP2.3 Manage corrective Action SG2 Validate Product or Product Components SP2.2 Analyze Validation Results. Change request management & Configuration Management process SG2 Track and Control Changes SP2.1 Track Change Requests SP2.2 Control Configuration Items

102. Traditional SW Project vs Open Source Project Open Source – Crowd Sourcing SW Relase tested without coordination by group of volunteers Release tested by customer Field Testing Test Group