1.  Black Box Testing Techniques © Sheridan College SYST30009-Engineering Quality Software 4

2. Today o Software Testing perspectives o Focus primarily on black box testing Black box test design techniques Use case testing Equivalence partitioning Boundary Value Analysis Decision table testing State table testing o Note Reference Materials SLATE Required reading o Quiz No 1 - Today © Sheridan College SYST30009-Engineering Quality Software 2

3. © Sheridan College SYST30009-Engineering Quality Software 3

4.  Consider the following statements: o “We’ve used the Simplex HR software in our Human Resources Dept. for about three years and we have never had a software failure.” o “I started to use Simplex HR two months ago; we had so many failures that we are considering replacing the package.” o “We have been using the same software package for almost four years. We were very satisfied throughout the period until the last few months, when we suddenly faced several severe failures. The Support Center of the software vendor from which we bought the package claims that they have never encountered failures of the type we experienced even though they serve about 700 customers who utilize Simplex HR.”  Why is it possible for such a variation in users’ experience with failure to appear with the same software package?  How can a software package that successfully served an organization for a long period “suddenly” change its nature (quality) and become “bugged”? © Sheridan College SYST30009-Engineering Quality Software 4

5.  Do all software faults end with software failures? o Example: The “Meteoro-X” meteorological equipment firmware. The software requirements for “Meteoro-X” meteorological equipment firmware (software embedded in the product) were meant to block the equipment’s operation when it’s internal temperature rose above 60 degrees Celcius. A programmer error resulted in a software fault when the temperature limit was coded as 160 degrees. This fault could cause damage when the equipment was subjected to temperatures higher than 60 degrees. Because the equipment was used only in coastal areas where the temperature never exceeded 60 degrees, the software fault never turned into a software failure. © Sheridan College SYST30009-Engineering Quality Software 5

6.  Faulty definition of Requirements Absence of vital requirements Incomplete definition of requirements Inclusion of unnecessary requirements  Client-Developer communication failures Misunderstanding of the client’s instructions as stated in the requirement document. Misunderstanding of the client’s requirements changes presented to the developer in written form during the development period. Misunderstanding of the client’s requirements changes presented to the developer orally during the development period. Misunderstanding of the client’s responses to the design problems presented by the developer. Lack of attention to client messages referring to requirements changes and to client responses to questions raised by the developer on the part of the developer. © Sheridan College SYST30009-Engineering Quality Software 6

7.  Deliberate deviations from software requirements. o Developer re-uses software modules from another project without sufficient analysis of the changes and adaptations needed to correctly fulfill all of the requirements. o Due to time or budget pressures, the developer decides to omit part of the required functions to cope with these pressures. o Developer-initiated, unapproved improvements to the software, introduced without the client’s approval, frequently disregard requirements that seem minor to the developer. Such minor changes may, eventually cause software errors. © Sheridan College SYST30009-Engineering Quality Software 7

8.  Logical Design Errors o Process definitions that contain sequencing errors. o Erroneous definitions of boundary conditions.  Coding Errors  Non-compliance with documentation and coding instructions. o Some team members not following standards © Sheridan College SYST30009-Engineering Quality Software 8

9.  Shortcomings of the testing process o Incomplete test plans leave untreated portions of the software. o Failure to document and report detected errors and faults. o Failure to properly correct detected software faults as a result of inappropriate indications of the reasons or the fault. o Incomplete correction of detected errors due to negligence or time pressure. © Sheridan College SYST30009-Engineering Quality Software 9

10.  User Interface and procedure errors o User procedures direct the user with respect to the sequence of required activities of data collection and processing. o Main reasons for these types of errors Misunderstanding of the implementation process Failure to communicate the detail of the solution steps required to perform the application  Documentation errors © Sheridan College SYST30009-Engineering Quality Software 10

11. o Black Box o White(Glass) Box o Exploratory Experienced based © Sheridan College SYST30009-Engineering Quality Software 11

12.  Consider this every day example… o A lock & key © Sheridan College SYST30009-Engineering Quality Software 12

13. © Sheridan College SYST30009-Engineering Quality Software 13 FunctionalityWhat you need to know to use Features of a lock It is made of metal, has a hole provision to lock, has a facility to insert the key, and the keyhole ability to turn clockwise or anticlockwise. Features of a key It is made of metal and created to fit into a particular lock’s keyhole. Actions performed Key inserted and turned clockwise to lock. Key inserted and turned anticlockwise to unlock. StatesLocked/Unlocked InputsKey turned clockwise or anticlockwise Expected outcomeLocking Unlocking

14.  Black box testing requires a functional knowledge of the product to be tested. It does not mandate the knowledge of the internal logic of the system nor does it mandate the knowledge of the programming language used to build the product.  Tests are focused towards testing the features of the product (example: lock and key), the different states.  You may check if the lock works with some other key (other than its own). © Sheridan College SYST30009-Engineering Quality Software 14

15.  Black box testing is done based on requirements It helps in identifying any incomplete, inconsistent requirement as well as any issues involved when the system is tested as a complete entity.  Black box testing addresses the stated requirements as well as implied requirements Not all the requirements are stated explicitly, but are deemed implicit. For example, inclusion of dates, page header, and footer may not be explicitly stated in the report generation requirements specification. However, these would need to be included while providing the product to the customer to enable better readability and usability.  Black box testing encompasses the end user perspectives Since we want to test the behaviour of a product from an external perspective, end-user perspectives are an integral part of black box testing.  Black box testing handles valid and invalid inputs It is natural for users to make errors while using a product. Hence, it is not sufficient for black box testing to simply handle valid inputs. Testing from the end-user perspective includes testing for these error or invalid conditions. This ensures that the product behaves as expected in a valid situation and does not hang or crash when provided with an invalid input. These are called positive and negative test cases. The tester may or may not know the technology or the internal logic of the product. However, knowing the technology and the system internals helps in constructing test cases specific to the error-prone areas. Test scenarios can be generated as soon as the specifications are ready. Since requirements specifications are the major inputs for black box testing, test design can be started early in the cycle. © Sheridan College SYST30009-Engineering Quality Software 15

16. © Sheridan College SYST30009-Engineering Quality Software 16

17.  Typical black box test design techniques include: o Use case testing o Equivalence partitioning o Boundary value analysis o Decision table testing o State transition tables © Sheridan College SYST30009-Engineering Quality Software 17

18.  Fancy terms for a simple idea  For a given input value determine the set of valid values and set of invalid ones.  Pick a value from the valid set and the invalid set to test.  “bugs lurk in corners and congregate on boundaries”- programmers often make mistakes on the boundaries of the valid and invalid sets and hence we have to focus testing at these boundaries.  This type of testing is called Boundary Value Analysis. © Sheridan College SYST30009-Engineering Quality Software 18

19.  Consider how interest is calculated on a savings account at the bank o A balance in the range of $0 to $100 earns 3% o A balance over $100 and up to $1000 earns 5% o Balances over $1000 earn 7% How would you test to verify that the right interest rate has been paid? © Sheridan College SYST30009-Engineering Quality Software 19

20.  Boundary Value Analysis is based on testing at the boundaries between partitions-in other words range checking.  When we say invalid, it doesn’t mean that it represents a value that cannot be entered by a user or a value the user isn’t supposed to enter. It means that it is not one of the expected inputs for this particular field. The software should correctly handle values from this invalid partition (or set) by replying with an appropriate message.  Equivalence partitioning and boundary value analysis is a sampling strategy to cope with the problem of too many possible tests (the economics of testing).  Traditional domain analysis considers numeric input and output fields.  Boundary analysis is optimized to expose a few types of errors such as miscoding of boundaries or ambiguity in definition of the valid/invalid sets. © Sheridan College SYST30009-Engineering Quality Software 20

21. © Sheridan College SYST30009-Engineering Quality Software 21