1. Active Design Reviews Doug Paida Roy Mammen Sharan Mudgal Jerry Cheng

2. Overview What is design review? What is design review? Taxonomy Taxonomy Limitations of Conventional approach Limitations of Conventional approach Walking through the Active Review Process using an example Walking through the Active Review Process using an example Conclusions Conclusions

3. What is a design review? A process of identifying faults in the design of a software program A process of identifying faults in the design of a software program Review should uncover both errors made in producing design documents, and errors made earlier in the design process Review should uncover both errors made in producing design documents, and errors made earlier in the design process

4. Taxonomy Structured Walkthrough author of the design and one or more other developers familiar with the development activity are involved Code Reading two or more developers read the code independently Discuss with the author

5. Taxonomy Code Inspection The process definition included planning, preparation, inspection, rework and follow up with specific roles define for the participants author/reader paraphrases the material, while the participants ask question that may lead to the discovery of defects Software Review The reviewers go through the material individually using a checklist The author collects the checklists and consolidates the results.

6. Taxonomy Active Design Review The questionnaire focuses the reviewer's attention on a particular set of issue reviewers concentrate on different sets of issues. The author meets with the reviewers individually to review their findings. Defect-Based Reading possible defects in requirements documents For each defect class a set of questions was developed that would characterize the defect class The questions also characterize a set of steps that should be performed while reading called scenarios. the reader tries to answer the questions provided in the scenario.

7. Taxonomy Meetingless Reviews As effective as meeting-based ones Individual strategies find more issues to resolve but these have a higher false positive and duplication rates The duplication rates can be reduced by specializing the roles during the process. Computer-mediated Technical Reviews supporting clerical functions of distribution and defect report collection and rework tracking. Aimed at synchronous distributed reviews where the review is not required to be in the room.

8. Limitations of Conventional approach Too much information to go through, and not enough time to do it thoroughly Too much information to go through, and not enough time to do it thoroughly Unfamiliarity of individual reviewers with the overall goals of the design Unfamiliarity of individual reviewers with the overall goals of the design No single part of the design gets a thorough and complete evaluation No single part of the design gets a thorough and complete evaluation

9. (continued) (continued) Burden is on reviewer to initiate action Burden is on reviewer to initiate action One-on-one interaction between individual reviewers and design team is limited One-on-one interaction between individual reviewers and design team is limited No systematic procedure – generally more free-form type of format No systematic procedure – generally more free-form type of format

10. Active Design Review Process Change from “general” review to a set of more focused reviews Change from “general” review to a set of more focused reviews Use questionnaires to actively engage the reviewer in using the design Use questionnaires to actively engage the reviewer in using the design More opportunities for one-on-one discussion between reviewer and designer More opportunities for one-on-one discussion between reviewer and designer

11. A design example We have been asked to review a design for a module which is part of an order processing system for a hospital. We have been asked to review a design for a module which is part of an order processing system for a hospital. The order processing system allows users to order items for patients, such as tests or medications. The order processing system allows users to order items for patients, such as tests or medications. The module we are reviewing acts as an interface between the order database and other modules in the system, hiding details of the actual database tables used. The module we are reviewing acts as an interface between the order database and other modules in the system, hiding details of the actual database tables used.

12. Order Processing System Add/Cancel OrderPrint/Display Orders Database Interface Database Design Example (cont.) Orders Order Item

13. Module requirements An order must contain at least these data items: An order must contain at least these data items: Patient ID Patient ID List of items being ordered. There must always be at least one item in every order. List of items being ordered. There must always be at least one item in every order. Date/time the items are being ordered for Date/time the items are being ordered for Status of the order (active or cancelled) Status of the order (active or cancelled) An item in an order contains these data items: An item in an order contains these data items: Item ID Item ID Quantity ordered Quantity ordered

14. Module requirements Basic functionality required Basic functionality required Add a new order for a patient Add a new order for a patient Change an order’s status to “cancelled” Change an order’s status to “cancelled” Obtain the number of orders a patient has Obtain the number of orders a patient has Iterate through all orders for a patient Iterate through all orders for a patient Iterate through all items within an order Iterate through all items within an order

15. Module requirements Actions that should be prohibited Actions that should be prohibited Making order active again once it is cancelled Making order active again once it is cancelled Adding items to an order after it’s been posted to the database Adding items to an order after it’s been posted to the database

16. Module specification Data in the Order class Data in the Order class int patientID int patientID int orderID int orderID Date orderDateTime Date orderDateTime int status int status Vector items Vector items

17. Module specification Access functions for Order class Access functions for Order class void addItem(Item i) void addItem(Item i) Item getFirstItem() Item getFirstItem() Item getNextItem() Item getNextItem() int numItems() int numItems() void setStatusToCancelled() void setStatusToCancelled() Date getOrderDateTime() Date getOrderDateTime() int getStatus() int getStatus()

18. Module specification Data for Item class Data for Item class int itemID int itemID int quantity int quantity Access functions for Item class Access functions for Item class int getItemID() int getItemID() int getQuantity() int getQuantity()

19. Module specification Access functions for Orders class Access functions for Orders class void submitOrder(Order o) void submitOrder(Order o) void updateOrder(Order o) void updateOrder(Order o) Order getOrder(int orderID) Order getOrder(int orderID) Order getFirstOrder(int patientID) Order getFirstOrder(int patientID) Order getNextOrder() Order getNextOrder() int numOrders(int patientID) int numOrders(int patientID)

20. Module specification Undesired Events Undesired Events getNextOrder() called before getFirstOrder() getNextOrder() called before getFirstOrder() addItem() called on an order that’s already been posted to the database addItem() called on an order that’s already been posted to the database

21. Steps for Active Design Review Prepare design and documentation for review Prepare design and documentation for review Identify for specialized reviews Identify for specialized reviews Identify the reviewers needed Identify the reviewers needed Design the questionnaires Design the questionnaires Conduct the review Conduct the review

22. Active Design Review Process Step 1: Prepare the design for review Think about what criteria reviewers will use: Think about what criteria reviewers will use: Well-structured Well-structured Simple Simple Efficient Efficient Adequate Adequate Flexible Flexible Practical Practical Implementable Implementable Standardized Standardized

23. Active Design Review Process Step 1: Prepare the documentation for review Make assumptions explicit Make assumptions explicit Module can record the order pertaining to a patient. Module can record the order pertaining to a patient. It is possible to obtain all the orders for a patient. It is possible to obtain all the orders for a patient. Module can determine & change the status of an order. Module can determine & change the status of an order. The order always contains at least one item The order always contains at least one item The status of an order is always in one of the two states i.e active or cancelled. The status of an order is always in one of the two states i.e active or cancelled.

24. Active Design Review Process Step 1: Prepare the documentation for review Incorrect Usage Assumptions (UE) Incorrect Usage Assumptions (UE) Cannot add or remove items once the order is placed. Cannot add or remove items once the order is placed. Once an order is cancelled, the status cannot be set to active again. Once an order is cancelled, the status cannot be set to active again. An item is always added with respect to an order. An item is always added with respect to an order.

25. Active Design Review Process Step 2: Identify the specialized reviews Focus the reviewer’s attention on specific properties of the design Focus the reviewer’s attention on specific properties of the design Data Access Sufficiency. Data Access Sufficiency. Provides all data required by the other modules ? Provides all data required by the other modules ? Assumption Sufficiency. Assumption Sufficiency. Contains all assumptions needed by the user program to efficiently access the DB ? Contains all assumptions needed by the user program to efficiently access the DB ? Consistency between Assumptions and Functions. Consistency between Assumptions and Functions. Assumptions? Access Functions? UE?

26. Active Design Review Process Step 3: Identify the reviewers needed People with different perspectives and expertise are needed as reviewers People with different perspectives and expertise are needed as reviewers Programmers and analysts who worked on the other modules in this order processing system. Programmers and analysts who worked on the other modules in this order processing system. Programmers and analysts familiar with hospital information systems in general. Programmers and analysts familiar with hospital information systems in general.

27. Active Design Review Process Step 4: Design the questionnaires Make reviewers take an active role Make reviewers take an active role Make them use the documentation Make them use the documentation Phrase questions in an active way Phrase questions in an active way “Write down the exceptions that can occur” rather than “Are exceptions defined for every program?” “Write down the exceptions that can occur” rather than “Are exceptions defined for every program?”

28. Active Design Review Process Step 5: Conduct the review Present an overview of the module Present an overview of the module Assign reviews to reviewers Assign reviews to reviewers Reviewers complete their reviews, meeting with designers as needed Reviewers complete their reviews, meeting with designers as needed Designers review completed questionnaires, and meet with reviewers to resolve questions Designers review completed questionnaires, and meet with reviewers to resolve questions Designers produce new version of the documentation Designers produce new version of the documentation

29. Sample Review: Data Access Sufficiency the access functions should be evaluated to ensure that access is provided to all data required by the other modules the access functions should be evaluated to ensure that access is provided to all data required by the other modules piece of data about an order that cannot be retrieved through an access function but needed reflects error in design. piece of data about an order that cannot be retrieved through an access function but needed reflects error in design. possible to modify the value of any piece of data in violation of the stated requirements, then there is a design error. possible to modify the value of any piece of data in violation of the stated requirements, then there is a design error. The module specifications should be reviewed by programmers of the other order processing system modules for their ability to satisfy these criteria. The module specifications should be reviewed by programmers of the other order processing system modules for their ability to satisfy these criteria.

30. Example Questionnaire The following questions should be used to aid in this review Using the access functions provided in the module specification, write the code you would use to obtain the date/time, item number and quantity ordered for all active orders for a patient. Using the access functions provided in the module specification, write the code you would use to obtain the date/time, item number and quantity ordered for all active orders for a patient. Attempt to come up with a sequence of access function calls that would allow a program to set a cancelled order’s status to active. Attempt to come up with a sequence of access function calls that would allow a program to set a cancelled order’s status to active. For each access function provided, write down the specific requirements from the requirements list that you believe the function was designed to meet. Make note of any functions that do not appear to satisfy any specific requirements For each access function provided, write down the specific requirements from the requirements list that you believe the function was designed to meet. Make note of any functions that do not appear to satisfy any specific requirements

31. Conclusions Reviewers focus on those areas they are best suited to evaluate Reviewers focus on those areas they are best suited to evaluate Time is used more wisely for all participants Time is used more wisely for all participants More errors are likely to be found More errors are likely to be found One-on-one communication with designers makes it easier for people to speak up One-on-one communication with designers makes it easier for people to speak up Few errors found not necessarily indicate that the design is good, but that the review process was not as effective as it could have been Few errors found not necessarily indicate that the design is good, but that the review process was not as effective as it could have been