Presentation on theme: "Sept. 9, 2009 Seminar at Beijing University 1 Deriving and Analysing the Quality of Software Architectural Designs Hong Zhu Department of Computing, Oxford."— Presentation transcript:
Sept. 9, 2009 Seminar at Beijing University 1 Deriving and Analysing the Quality of Software Architectural Designs Hong Zhu Department of Computing, Oxford Brookes University, Oxford OX33 1HX, UK
Sept. 9, Seminar at Beijing University Outline Overview of existing work and open problems Our approach Graphic model of software quality Derivation of software quality model from design Analysis of quality models Conclusion Comparison with related works Future work
Sept. 9, Seminar at Beijing University The Notion of Quality Quality, in particular software quality, is an elusive concept and varying from people to people. Garvins definitions of quality Transcendent view: Quality is universally recognisable. It is related to a comparison of features and characteristics of products. Product-based view: Quality is a precise and measurable variable. Differences in quality reflect the differences in quantities of some product attributes. User-based view: Quality is the fitness of intended uses. Manufacturing-based view: Quality is conformation to the specifications. Value-based view: A quality product is one that provides performance at an acceptable price or conformance at an acceptable cost.
Sept. 9, Seminar at Beijing University Software Quality Models Models about software quality in terms of The factors that affect software quality Attributes directly indicate the quality of he system, e.g. reliability, correctness, user friendliness, etc. Attributes indirectly related to quality, e.g. internal complexity, etc. The interrelations between the factors Causality models: Causal relationship, in terms of stereo-type relations Quantitative models: Quantitative relations expressed as numerical functions, Numerical values of the basic attributes are given, e.g. through using metrics/measurements The overall quality in an attribute on a more high level of abstraction is calculated numerically
Sept. 9, Seminar at Beijing University Hierarchical Models of SQ A set of quality related properties organised into a hierarchical structure E.g. decompose quality into a number of quality attributes and attributes into a number of factors, and then a number of measures, etc. Positive causal relationship is represented Examples: McCall model (1977) Boehm model (1978) ISO model (1992) Bansiya-Davis model of OO designs (2002), etc.
Sept. 9, Seminar at Beijing University McCalls Model Product operations Product transition Product revision Maintainability: Can I fix it? Flexibility: Can I change it? Testability: Can I test it? Portability: Will I be able to use it on another machine? Reusability: Will I be able to reuse some of the software? Interoperatability: Will I be able to interface it with another machine / software? Correctness: Does it do what I want? Reliability: Does it do it accurately all the time? Efficiency: Will it run on my machine as well as it can? Integrity: Is it secure? Usability: Can I run it?
Sept. 9, Seminar at Beijing University ISO 9126 Model Software quality Reliability Maturity Fault tolerance Recoverability Portability Adaptability Installability Conformance Replaceability Maintainability Analysability Changeability Stability Testability Efficiency Time behaviour Resource behaviour Usability Understandability Learnability Operability Functionality Suitability Accuracy Interoperability Compliance Security
Sept. 9, Seminar at Beijing University Relational Models of SQ A quality model consists of: A number of quality attributes and factors, etc. A set of stereo types of relationships between them, normally include Positive relation: High on one aspect of quality implies inevitably high on the other Negative relation: High on one aspect of quality implies inevitably low on the other Neutral relation: High or low on one aspect of quality does not imply on the other aspect the quality is high or low. Examples: Perry model (1991) Gillies model (1992, 1997)
Sept. 9, Seminar at Beijing University Perry s Model
Sept. 9, Seminar at Beijing University Roles in Software Development Understanding software quality To measure an existing systems quality, To predict a systems quality during development To analyse quality problems during development and/or in operation of a systems Guidelines to development activities To elicit users requirements on software quality To perform quality assurance activities in software development and maintenance
Sept. 9, Seminar at Beijing University Key Features of Existing SQ Models FeatureProsCons UniversalApplicable to all software systems Not address system specific issues SimplicityStereo types of relationships between quality attributes Incapable of dealing with complicated relationships between quality attributes Black-boxTreat software systems as black-boxes Provide little help to the designers to consider systems structure EmpiricalDeveloped-based on empirical knowledge from experiences Cannot be validated or verified of its correctness
Sept. 9, Seminar at Beijing University Our Approach Representing software quality models in a diagrammatic notation (Zhang, Zhu, et al. 2002) to improve expressiveness of quality models to represent complicated relationships between factors to associate quality features with the structure of the system Deriving software quality models from architectural designs (Zhang, et al. 2002, Zhu, 2005) to systematically derive quality models from software architectural designs by employing extended hazard analysis concepts and techniques Automating the analysis of quality models to help software designers (Zhang, et al, 2006a, 2006b) to perform various analysis tasks based on graphic quality models by developing and using automated software tools
Sept. 9, Seminar at Beijing University Graphic Notation for Modelling SQ Subject [+|-] Property Phenomenon Node: representing quality carrying properties and quality attributes and observable phenomena Link without annotation: representing logic relations between the nodes. Link with Annotation: providing additional information of the rationale of the relation between nodes. [+ | - ] Annotation of Reasons [+ | - ] Interface is not displayed properly Client side - Compatibility Font size too small to be illegible. System - Usability Difficult to operate Node A Node B Link
Sept. 9, Seminar at Beijing University Example: Web-based Applications HTML files + Well Structured Large size HTML files + Navigability Small number of hyperlinks System + Usability Easy to find required info Web Server - Responsiveness Long response time HTML files - Correctness Contains broken links Online Help - Availability Not available Client side - Compatibility Font size too small to be illegible when displayed users platform System - Usability Cannot find required info Server side - Performance Execution speed is slow Server side + Load Highly demanded Files are considered as unavailable when time-out Simpler hyperlink network usually easier to navigate Less nodes means less links Need long time to transmit the files Web page cannot be displayed properly Unable to obtain files through hyperlinks Unable to get help when experiencing difficulty
Sept. 9, Seminar at Beijing University Deriving Quality Models: The HASARD Methods HASARD: Hazard Analysis of Software ARchitectural Design Input: A software architectural model Output: A quality model in graphic notation Process: Hazard identification A hazard identification method is applied to identify all quality sensitive observable phenomena of the components, connectors, the system, etc. Cause-consequence analysis The causal relationships between the identified hazards are recognized Model assembling The information obtained in the previous steps are assembled together and represented in the graphical notation Quality concern recognition The quality carrying properties/quality attributes that a phenomenon demonstrates are recognized according to the nature of the phenomenon
Sept. 9, Seminar at Beijing University Hazard Identification Extended Hazard and Operability Studies (HAZOP) The method relies on determining answers to questions of what-if nature. By applying a set of guide words systematically to develop a collection of what-if questions. They are applied to the attributes of various components and connectors of the system being studied. If a deviation from the normal working of the component is credible, the behaviour of the component is considered as a possible hazard.
Sept. 9, Seminar at Beijing University Guide Words for Software Hazard Identification Guide word Applicable attribute Interpretations NoDate/control signals No data / control signal exchanged; No data / control signals produced/received. Component property/ function The component / connector does not have the designed property / function. Component / connector The system does not contain the component / connector. MoreQuantitative parameters The value of the parameter is too large. LessQuantitative parameters The value of the parameter is too small.
Sept. 9, Seminar at Beijing University As well as Event or activityRedundant data sent, or event/activity occurred in addition to the intended ones. Component / connector Other components / connectors are added in addition to the intended ones. Part ofStructured dataOnly a part of the data produced, stored or received. Structure eventsOnly a part of the events happened. ReverseDirection of flowThe information flow in the opposite direction. EventThe opposite event happened. Other than Data/control signals, quantitative/qualitative parameters Incorrect data / control signals produced; The parameter has a value different from the design. Component/systems function or property The component has a functionality / property different from the designed. Component/ connectorOther kind of component / connector is contained.
Sept. 9, Seminar at Beijing University EarlyPeriodical eventsThe event happened earlier than expected. LatePeriodical eventsThe event happened later than expected. BeforeTemporal ordersThe event happened in the order earlier than designed. AfterTemporal ordersThe event happened in the order later than designed.
Sept. 9, Seminar at Beijing University Example: Internet Connection Guide word Hazard/Failure modeCausesConsequences NoThe internet connection passes no messages between the client and server. Physically disconnected; Traffic jam; Software failure; Network server down. Client cannot communicate with the server. MoreMore messages are delivered than the clients and the server send out, e.g. duplicated messages. Hackers attack; Heavy traffic on the Internet caused resending packages. System clash; Overload on the server or the client. LessFewer messages are delivered than the server and the clients send out, i.e. lost messages. Discontinued Internet connection; Heavy traffic on the Internet; Software failure. Incomplete transactions; System clash; Damage the integrity of the data and program on the server and/or client.
Sept. 9, Seminar at Beijing University As well as Messages are delivered to other destinations in addition to the designated receiver. Hackers attack; Software failure. Leak of sensitive information. Part ofOnly a part of the packets of a message is delivered to the destination client or server. Discontinued Internet connection; Heavy traffic on the Internet; Software failure. Software failure; Production of incorrect computation results if incompleteness is not detected. Other than A message not from the client (or the server) is passed to the server (or client). Hackers attack; Other software systems failure System failure; Damage the integrity of the data and the program. Other than The message is in a different format. The client or the server is modified; Fault in the software. System failure.
Sept. 9, Seminar at Beijing University Cause-Consequence Analysis It aims at understanding the causal relationships between hazards. It can be performed backward or forward, or a combination of both. Forward analysis Search for the potential effects, i.e. consequences, of a hazard until the consequence is terminal. A hazard or failure mode is terminal: if it does not affect any other component of the system or If It does not cause any other hazards/failures. if we are not interested in its further consequences. Backward analysis Search for the causes of a hazard until the hazard is primitive. A hazard or failure mode is primitive: if its causes cannot be further identified without additional knowledge about the system if we are not interested in its causes
Sept. 9, Seminar at Beijing University Results of Cause-Consequence Analysis
Sept. 9, Seminar at Beijing University Constructing Graphic Quality Model Input: The records of the cause-consequence analysis Output: A partially completed graphical representation of quality model The transformation: Each record of the hazard or failure mode becomes a node with the component and phenomenon as specified in the record. Each row in the record becomes a link from the node that represents the cause to the node that represents the consequence. The explanation column of the row forms the reason of the link.
Sept. 9, Seminar at Beijing University Example:
Sept. 9, Seminar at Beijing University Identification of Quality Concerns For each node in the diagram Identify the quality attribute or quality carrying property that the phenomenon demonstrates Comparing the observable phenomenon with the definitions of a set of quality attributes and quality- carrying properties Recognising a new attribute or property, if no existing one matches. Fill the identified quality attribute into the slot of the node Examples, a hyperlink is broken demonstrates the quality attribute correctness of the HTML file. Server is down is related to the availability of the server.
Sept. 9, Seminar at Beijing University Analysis of Graphic Quality Models Contribution factors What are the factors that affect a specific quality issue that the user (designer) is interested in? HTML files Well Structured Large size Web Server - Responsiveness Long response time Server side - Performance Execution speed is slow Server side - Load balance Some servers have high demand, but some not Need long time to transmit the files Example: Factors of servers responsiveness
Sept. 9, Seminar at Beijing University Impacts of design decisions What are the consequences of a design decision? Simpler hyperlink network usually easier to navigate HTML files + Navigability Small number of hyperlinks Web Server - Responsiveness Long response time System - Usability Cannot find required information System + Usability Easy to find required information Less nodes means less links Need long time to transmit the files Files are considered as unavailable when time-out HTML files + Well Structured Large size Example: The impacts of HTML files are of large sizes
Sept. 9, Seminar at Beijing University Quality risks When a negative quality property is present, where does the risk come from? And what are the implications of the problem? Example: (partial model) The risk of long response time in a web-based application: its causes and implications HTML files + Well Structured Large size Web Server - Responsiveness Long response time System - Usability Cannot find required info Server side - Performance Execution speed is slow Server side - Load balance Some servers have high demand, but some not Files are considered as unavailable when time-out Need long time to transmit the files
Sept. 9, Seminar at Beijing University Relationships between quality issues How two quality issues are interrelated? Example: Relationships between flexibility and performance
Sept. 9, Seminar at Beijing University Trade-off points: When a design decision has positive impact on one quality attribute but negative impact on another quality attribute, a trade-off must be made to balance between the two. Where are the points in a complicated design that need to make trade-offs? Or simply, where are the trade-off points? Example: EJB components size is a trade-off point in systems implemented using EJB.
Sept. 9, Seminar at Beijing University Automation of Quality Analysis SQUARE: Software QUality and ARchitecture modeling Environment. Tools to support software architectural modelling Tools to support the derivation of software quality models from architectural design Graph algorithms are designed and implemented to automate the analysis of software quality; Hazard Analysis Tools Quality Analysis Tools Quality Model Editor Architecture Model Editor SA Model Quality Model Quality Analysis Results Model Repository Project Manager Overall structure of SQUARE toolkits
Sept. 9, Seminar at Beijing University Screen Snapshot of SQUARE toolkits
Sept. 9, Seminar at Beijing University Case Study The subject: A real e-commerce system of online trading of medicine. The system is operated by the local medicine trading regulation authority who supplies medicines to all state- owned hospitals in the province. Its main functions: customer relationship management, product catalogue management, online trade management, online auction of medicine supplies, online information advertisement, a search engine for medicine information, and so on. The system is implemented in J2EE The system has been in operation for more than one year at the time of case study
Sept. 9, Seminar at Beijing University Process of The Case Study Step 1: Construct the architectural model It is a reverse engineering process: Review of the design documents The systems design document consists of four main parts: (a) a simple and schematic J2EE architecture showing that the system uses J2EE as implementation technology; (b) interface design, which consists of a lot of HTML files; (c) database design, which consists of about 63 database tables; (d) a simple UML class diagram that contains a dozen of classes and shows the logical view of the system. Review of parts of the source code. When design information was not well documented, parts of the source code was reviewed Construction of architectural model Confirmation of architectural model by some of the chief developers of the system The draft architectural model was reviewed and corrected The final version of the model was approved of its accuracy
Sept. 9, Seminar at Beijing University Architecture of CRM Subsystem
Sept. 9, Seminar at Beijing University Step 2: Construction of Quality Model Employed the HASARD method and the SQUARE toolkits The quality model was reviews and corrected in several iterations with the collaboration with the developers The final version was confirmed for its accuracy and correctness by the developers The final result: 48 nodes 60 links between the nodes.
Sept. 9, Seminar at Beijing University Step 3: Analysis of the Quality Model The quality model was analysed using the SQUARE analysis tools Identification of quality risks and quality trade-off points Derivation of the impacts of design decision and the contribution factors on certain quality attributes The results were feed back to the developers A workshop was run to validate the outcomes All our findings were consistent with what has been observed in the development and operation of the system. Some of the phenomena observed in the operation of the system were first time satisfactorily explained A number of specific suggestions on the improvement of the systems architecture were made Some were taken by the development team in the development of the new release of the system. Some would result in major changes of systems architecture and regrettably cannot be implemented within the budget of the new releases.
Sept. 9, Seminar at Beijing University Example 1: Causes of usability problem The quality problem concerned: Users often cannot find desired information on the website Analysis goal: Find the factors that may cause the problem Method: Select the node that contains user as the entity and Cannot find info as its phenomenon Use the tool to find the contribution factors to this node Result: The tool generated a sub-diagram that contains 25 nodes out of the 48 nodes in the quality model. Interpretation of the result: Most components of the system affect usability of the system Usability is a very sensitive quality issue in the design The outcome: Details are provided by the tool about what affect usability Useful direction to enhance usability
Sept. 9, Seminar at Beijing University Example 2: Contribution Factors to Servers Availability Problem to be analysed: What are the factors that affect the servers availability? Method: Select the node that contains server as entity and availability as property. Use the tool to find the contribution factors. Result: A sub-diagram is generated, that shows that the factors include hardware reliability, software reliability, power supply, system security, and maintenance.sub-diagram Outcome: Suggested measures to improve availability To prevent hackers from attacking the server To ensure a reliable power supply To use reliable hardware to run the server To use fault tolerance to avoid software crashes on invalid input To provide maintenance tools to enable online maintenance To reduce the time that the system has to be shut down for maintenance
Sept. 9, Seminar at Beijing University Quality factors that affect servers availability
Sept. 9, Seminar at Beijing University Example 3: Identify the quality trade-off points Problem to be analysed: What are the trade-off points in the design? Method: Select a node and to find the consequences of the node using the tool Identify whether is leads to both positive result and negative result Examples: The size of HTML files is a trade-off point. (See diagram on slide 25)See diagram on slide 25 The granularity of the session beans.
Sept. 9, Seminar at Beijing University Comparison with related works Scenario-based analysis of software architectures SAAM, ATAM, etc.: Builds no quality model, not automated Our approach: More systematic to derive quality model from design, Can be automated for analysis once a model is constructed, Models can be relatively easily validated Hazard analysis of software system HAZOP, FMEA, etc.: Focused on safety, Systematic, But not interrelationships between quality attributes Our approach: More general rather than just for safety, Focuses on the relationships between attributes
Sept. 9, Seminar at Beijing University Future work Combining with scenario-based methods to construct graphic quality models to represent results of scenario analysis Analysis other quality features, e.g. Process-oriented quality features: how does a design affect the software development process? Such as reusability, modifiability, portability, etc. More case studies Existing case study is on e-Commerce web-based applications Work in progress: real-time and embedded systems
Sept. 9, Seminar at Beijing University References H. Zhu, Y. Zhang, Q. Huo and S. Greenwood, Application of Hazard Analysis to Software Quality Modelling, in Proc. of COMPSAC02, pp , IEEE CS, H. Zhu, Software Design Methodology: From Principles to Architectural Styles, Elsevier, Q. Zhang, J. Wu, and H. Zhu, Tool Support to Model- based Quality Analysis of Software Architectures, Proc. Of COMPSAC06, IEEE CS, Q. Zhang and H. Zhu, Automated Analysis of Software Designs with Graphic Quality Models, WSEAS Transactions on Computers, Vol. 5, Issue 10, Oct. 2006, ISSN , pp
Sept. 9, Seminar at Beijing University Acknowledgement The work reported in this talk is based on the research collaborated with Dr. Yanlong Zhang of Manchester Metropolitan University, UK, Dr. Sue Greenwood of Oxford Brookes University, UK, Ms. Qian Zhang and Mr. Jian Wu of National University of Defence Technology, China.