Software Architecture in Practice Introduction to Performance Engineering

Credits Jens Edlef Møller –Executive IT Architect in IBM Global Business Services. –Typical role as lead architect on complex delivery projects around the world. –IBMs Nordic leader of performance engineering consultancy. –M.Sc. in computer science from University of Southern Denmark (1993).

Agenda : - Introduction to Performance Engineering - Mathematical models of performance and scalability - Exercise (theoretical) : Lunch (sponsored by the institute) : - Architecture and factors influencing performance - Performance test and analysis - Exercise (practical) - Designing for performance - Introduction to mandatory project

What is Performance? Definition: –Performance: The degree to which a system or component accomplishes its designated functions within given constraints, such as speed, accuracy, or memory usage. [IEEE ] In general: –Predictability and timeliness of response is what performance is about. –A “faster” system is not always enough; for example, a real-time system requires repeatable, predictable results

Performance Requirements You need to understand what the true requirements are. A simple general statement such as “response times below 1 second” is not sufficient. You must clarify exactly what it means –What is the requirement? (Response time, transaction time or throughput?) –Under which circumstances? (24*7 ?) –How and where is it measured? (end user or server side?) –For which workload is it valid? (transactions and data) –For which users or systems? (how many users?) –Which evidence is needed? (95% percentile? Average?) Even if your client doesn’t give you well defined requirements he will most likely not accept a solution with poor performance !

Performance Terminology The terms most often used –Response time Amount of time system takes to process a transaction This is typically what people perceives as the performance! –Througput Transactions per second your application can handle (online) Number of records your application can handle over time (batch) … but these should also be considered –Capacity The amount of work that can be completed over a period of time –Utilization The current amount of work being completed divided by the capacity –Scalability The ability to support more work by increasing the ressources in the system –Latency The amount of time a message takes to traverse a system

Terminology – a highway analogy The response time is the time required for me to drive from Aarhus to the office in Copenhagen. (Measured in time units) The capacity is the maximum number of cars per hour the E45 can handle without congestion. (Measured in cars per time unit) The utilization is the throughput divided by the capacity. (Measured as percentage) The throughput is the number of cars passing E45 in a certain hour. (Measured in cars per time unit)

Balance performance and other QAs

End users’ perception of performance Interaction perception: –People treat computers as people too. –The steps in people’s sense of time are as follows: Instantaneous: < 0.2 seconds (for example, typing) Immediate: < 1 second (like-minded colleague / aggressive stranger) Fast: < 2 seconds (frequent or standard request) Normal: 3 seconds (typical conversation) Extended: 5 seconds (slow speaker - ideally needs to make sound in the long pauses to show they are still speaking) Too long: about 17 seconds (Other party gives up. “Get back to me when you have an answer”) (Miller, 1968: “Response time in man-computer conversational transactions”) Performance does not mean very high speed. It means an acceptable and predictable level of responsiveness

Cheaper to address performance early "Software Defect Reduction Top 10 List," Basili and Boehm, The IEEE Computing Journal, IEEE Computer Society, Vol. 34, No. 1, January 2001

Performance Engineering “Performance engineering within systems engineering, encompasses the set of roles, skills, activities, practices, tools, and deliverables applied at every phase of the Systems Development Life Cycle which ensures that a solution will be designed, implemented, and operationally supported to meet the non-functional performance requirements defined for the solution.” –Wikipedia Performance should never come as a surprise at the end of the project. The two approaches to managing system performance are: –1. Cross your fingers, close your eyes, and hope. –2. Consider performance from the start. Always take the second approach!

Good performance engineering begins early Can I tune my way out of this production performance problem? Can I reduce performance problems in production by better testing? Can I reduce performance problems in test by better design and development? Can I prevent performance issues by good NFRs, architectural decisions and planning? The sooner I start thinking about performance the better Solution timeline FeasibilityDesignDevelopmentTestRun & Maintain

Performance Engineering Process Performance Engineering is a full lifecycle process –from project launch and until operation IBM “Performance Engineering and Management Method” (PEMM)

Performance Engineering Process Know what to do in the different phases –Architecture and Design Ensure well defined and reasonable performance requirements Estimation and performance modeling Design the system to meet the performance requirements –Development and Test Code with good performance in mind. Good performance doesn’t come accidentially! Do the right amount of performance testing in due time to allow for optimizations –Operations Monitor the performance of the system Implement capacity and performance management processes

Testing and Optimization An iterative problem-solving process … which must follow a structured approach Beyond performance testing, part 2:

Performance Engineering must continue into the operation of the system Mature Capacity Management processes comprise three sub- processes, according to ITIL –Business Capacity Management Purpose: To translate business needs and plans into capacity and performance requirements for services and IT infrastructure and to ensure that future capacity and performance needs can be fulfilled. –Service Capacity Management Purpose: To manage, control and predict the performance and capacity of operational services. This includes initiating proactive and reactive action to ensure that the performances and capacities of services meet their agreed targets. –Resource Capacity Management Purpose: To manage, control and predict the performance, utilization and capacity of IT resources and individual IT components

But don’t consider it too late! To minimise the risk of performance issues, capacity management practices must interlock with delivery projects The ITIL® Application Management life cycle

Web Layer Architecture Browser Simplified view - no network equipment - no storage - single string - … Web Server Application Server Database Server

Web Layer Architecture Browser End user response time is the sum of all elements Web Server Application Server Database Server App Server DB Server Web Server LAN WAN Browser t Service time Wait time

Improving performance App Server DB Server Web Server LAN WAN Browser t Service time Wait time Improving performance is typically a matter of reducing the response time of one or more parts of the total chain

Every system has a limit When the load on the system is increased it will eventually reach a point where performance degrades Wait times grow in a busy system (More on this in next session) Beyound this point non-linear scalability

In a scalable system the limit can be moved Scalability is the ability to support more work by increasing the ressources in the system –Typically by upgrading/adding hardware SW must scale as well as HW –Logical’ resources such as locks, shared data structures, thread pools, and connection pools can be critical Application design often limits scale-out goals, e.g. because of: –Single threaded designs –Session state management –Shared data access

2 types of scalability Vertical Scaling (Scaling up) –Upgrade the configuration of nodes in the system Processors Memory I/O Storage Horizontal Scaling (Scaling out) –Parallellize processing Additional servers Load balancing Must design scalability in from the start –Difficult to retrofit during build! –Very difficult in live system … Browser Web Server DB Server Browser Web Server DB Server Browser Web Server DB Server Browser Web Server DB Server Web Server Load Balancer

Scalability and the other NFR’s Typically, horizontal scaling increases the availability of the clustered application at the cost of increased maintenance Active-Active clustering –All nodes in the cluster participate in servicing requests –Typically requires load balancer to distribute load Active-Passive clustering –One (primary) node service requests –Another (secondary) node is standby to take over if primary node fails –Typically requires shared storage or synchronization In a complex system a combination is often used