GUI testing There are two major sets of GUI testing 1.“Usability” of the GUI Looks (aesthetic, visual appeal) artistic characteristic –Color –Shapes –Icons and Positions –Acronym/ Spelling/ Grammar Navigation (natural flow) business flow characteristic –Past cultural and work experience (e.g. accessibility from menu or tool bar)) –Amount of (or number) of choices and combination of choices –Shortcuts and options for experienced user 2.“Logic” of GUI Coverage of the data inputs/outputs of GUI fields –Input box –Check box –Drop down window list –Button choices –Etc. We will focus on the Functional Logic of GUI fields and the coverage of data inputs in this lecture.

But first - - -Testing the Usability of GUI Usability is largely based on – user perceptions and thus based on user feedback. Should not wait until the end of the development cycle: –Perform “paper” inspection of interface requirements/design (with users) –Testing “early prototypes” with users to: Explore users’ mental models (looks, flow, etc.) Evaluate alternatives Validate the choice (in terms of speed, accuracy, etc.) –Testing with functional code (incrementally with the interfaces.) –Perform a, possibly, formal usability test with : Good statistical sampling of users Usability expert to design the test and evaluate the results

Testing GUI Fields In order to test a program or system, it usually requires : 1.Starting the system (program) 2.Inputting some data 3.Let it execute 4.Observe the results (outputs and state) What data do we “input?” –Note that inputting data is not limited to keying in information. It includes clicking on a “button’ or making a choice from a “drop down” list. For example: Clicking on a button is a binary event – clicked or not clicked If there are n buttons on a screen, then we have 2 n permutations of “inputs” that need to be considered.

Consider an Example Key in # of people (max. 12): Cruise Meal Reservations Breakfast LunchDinner For this meal reservation, we have 4 input fields. - key in number of people - 3 buttons to click for the meal or meals What type and how many “inputs should we consider for the test ? Note: how do you think error or any message would/should be handled here?

Example (cont.) Breakfast button Dinner button Lunch button True False X X X X X X X X X X X X X X X X X X X X X X X X -A) First, the numerical input field allows “integer” inputs. -B) Second, the integer must not be greater than 12 -C) Third, there are 2 3 = 8 permutations for the 3 meal buttons These 8 combinations equates to 8 input test cases. But rows are the 3 logical ones?

“Partitioning” Numerical Field Test (from example) 1.The “illegal” input test for numerical field. In this case “number of people” has to be positive integers. Input tests should include: –Non-integer (e.g. decimal number) –Negative integer –non-numeric (characters) 2.The “legal” integer is “upper” bounded by 12: –Integer 12 (inclusive boundary) –Integer 13 (one over the boundary) –Integer 11 (one inside the boundary) 3.Is there a “lower” bound? not clear –Assume integer 1 (inclusive boundary) –0 (one outside the boundary) –2 (inside the boundary) The different inputs considered for the numerical field equates to 9 test cases. special kind of “out of bounds” test more conventional boundary value test

More on the Example Should we just add up the two separate sets of test cases and get 17 input tests? –8 from the 3 buttons –9 from the numerical field We can probably reduce some from the numerical field tests: –Just do one inside the boundary instead of both sides of the boundary. –One outside of the lower bound instead of 0 and a negative number. Do we need to ask if there is any relationship among the input fields? –Should we ensure that the combination of legitimate numerical input (e.g. 8 people) is combined with all three buttons not clicked to test the inter- relationship? –What type of error message, if any, should be issued if the user keys in 0 people and no buttons clicked? –Should we worry about some other possible relationship that we did not consider and thus perform 8 X 9 = 72 test cases? to see the different or consistency of “error” messages? Message box Message content Opportunity to recover

1-Dimensional Boundary Value Analysis Our previous example of numerical field had an upper and a lower bound of 12 and 1 respectively. There are two way to consider the boundary value analysis. 1.Legitimate to illegitimate includes : a) 2 (legitimate-inside boundary) b) 1 (legitimate – on boundary) c) 0 (illegitimate – outside the boundary) d) 11 (legitimate –inside the boundary) e) 12 (legitimate – on boundary) f) 13 (illegitimate – outside the boundary) legitimate 2. Illegitimate to legitimate includes : a) -1 (illegitimate-inside boundary) b) 0 (illegitimate – on boundary) c) 1 (legitimate – outside the boundary) d) 14 (illegitimate – inside the boundary) e) 13 (illegitimate – on boundary) f) 12 (legitimate – outside the boundary) illegitimate May choose to reduce to the set with 5 inputs { 0, 1, 2, 12, 13} May choose to reduce to the set with 5 inputs {-1, 0, 1, 13, 14 } or with only 4 inputs { 0, 1, 12, 13} uncomfortable ?

Multi-Dimensional Boundary Value Analysis (Case 1 – Partially Dependent) Book check-out Date; Book check-in Date: - Assume that for these two fields we have performed the boundary analysis and estimated the test cases: - check-out date : n1 input test cases - check-in date : n2 input test cases -Then the total test cases will be (n1 + n2). - But is there any inter-relationship? - Yes, book checkout date must be before or same as book check-in (return) date. - ensure that that the 3 cases of a) =, b) check-out > check-in, and c) check-out< check-in are in the ( n1+n2 ) input tests or one needs to add these.

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) Flight Time : Flight Date : Consider the situation where a discount is given to flights between: - 6 PM to 12AM (inclusive) and - 6/1/2005 through 12/31/2005 (inclusive) Aside from the individual field level 1-dimensional boundary value analysis, we need to take into account the “AND” logic for getting a discount. Individually, this logic requires: - {5:59pm, 6:00pm, 6:01pm, 11:59PM, 12:00AM, 12:01AM} for Flight Time - {5/31/2005, 6/1/2005, 6/2/2005, 12/30/2005, 12/31/2005, 1/1/2006} for Flight Dates ( note that we have reduced the one duplication inside the boundary )

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) – cont. Because of the ‘AND’ logic for the time and dates, we would consider the (5 “ time” inputs X 5 “date” inputs) or a total of 25 combinations. 5:59 PM 6:00 PM 6:01 PM 12:00 AM 12:01 AM 5/31/05 6/01/05 6/02/05 12/31/05 1/01/06 (ill, ill) (ill, leg) (ill, ill) (leg, ill) (ill, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (ill, leg) (ill, ill) the entries are in the form : (date, time) where - ill= illegitimate - leg = legitimate

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) – cont. 5:59 PM 6:00 PM 6:01 PM 12:00 AM 12:01 AM 5/31/05 6/01/05 6/02/05 12/31/05 1/01/06 (ill, ill) (ill, leg) (ill, ill) (leg, ill) (ill, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (ill, leg) (ill, ill) Do we need to run all 25 combinations ---- can we reduce the test cases? Cover all the borders scheme with a very limited saving - 24 test cases: a) (ill,ill)  (leg,leg); b) (ill, leg)  (leg,leg); c) (leg,ill)  (leg,leg)

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) – cont. 5:59 PM 6:00 PM 6:01 PM 12:00 AM 12:01 AM 5/31/05 6/01/05 6/02/05 12/31/05 1/01/06 (ill, ill) (ill, leg) (ill, ill) (leg, ill) (ill, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (ill, leg) (ill, ill) Do we need to run all 25 combinations ---- can we reduce the test cases? Cover some of the borders scheme with more savings – 12 test cases: (ill,ill)  (leg,leg); (ill, leg)  (leg,leg); (leg,ill)  (leg,leg)

Multi-Dimensional Boundary Value Analysis (Case 2 – Dependent) – cont. 5:59 PM 6:00 PM 6:01 PM 12:00 AM 12:01 AM 5/31/05 6/01/05 6/02/05 12/31/05 1/01/06 (ill, ill) (ill, leg) (ill, ill) (leg, ill) (ill, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (leg, leg) (leg, ill) (ill, leg) (ill, ill) Do we need to run all 25 combinations ---- can we reduce the test cases more? Try Partitioned Data sets of (ill, ill); (ill, leg); (leg,ill); (leg,leg) – 4 test cases: ( NOT a good choice to use partitioned data sets on this 2-dimensional boundary value analysis )

Multi-Dimensional Boundary Value Analysis We have seen in our example of 2 inter-related data fields, each having 5 potential inputs from boundary value analysis, would result in 5 x 5 = 25 test cases (even though the number may be reduced with risk) In general, if we have Z inter-related data fields, each with n test cases from individual boundary value analysis, then we would have: – n x n x x n (Z times) or – n Z test cases if n= 5 and z = 3, that would mean 5 3 = 125 test cases If we have three independent screens like this it would mean 3 x 125 = 375 test cases ! But if the 3 screens are dependent on each other, we may have as much as 125 x 125 x 125 = 1,953,125 test cases! (imagine that!)

An Interesting Multi-Scenario Test Analysis Flight Number : OR Enter Departing Loc. : Arriving Loc. : Approximate Arrival Time: Check Flight Status

An Interesting Multi-Scenario Analysis Flight Number XYZ-126 schedule dept-timeactual dept-timescheduled arr-timeactual arr-time 2:00PM 6:00pm Flight Number XYZ-126 schedule dept-timeactual dept-timescheduled arr-timeactual arr-time 2:00PM 2:15PM 6:00pm Test for : Prior to flight takes off Need to test for : after flight takes off What do you need to do to test this ? How about actual arrival time ? ---- relation to dept time ?