Performance Testing Back to Basics May 2013
Who Am I?
Who Are You?
Performance testers?
Who Are You? Performance testers Testers/QA?
Who Are You? Performance testers Testers/QA Developers?
Who Are You? Performance testers Testers/QA Developers CFR Cluj Supporters?
Today 4 sessions about an hour each Some practical work in groups Any questions/comments speak up Please share your thoughts/experiences
Conversation NOT lecture
Performance Testing Back to Basics
What is Performance Testing? ?
performance testing is in general testing performed to determine how a system performs in terms of responsiveness and stability under a particular workload? Wikipedia 2013
What is Performance Testing? Load Testing Stress Testing Soak Testing Spike Testing Configuration Testing Isolation Testing
Some History
Traditional Performance Testing understand the volumes Model the business processes Run experiments (tests) Measure performance indicators (response time, cpu etc.) Tune if needed Sign off report
Some History
From This
To This
Applications have changed More complex functionality Minimal defined business processes Diverse Public Internet Use of services (homegrown and third party) Cacheing Increased Client functionality (javascript etc.)
Applications have changed Cloud computing Multiple client platforms (tablet, mobile etc.) Push technology
Traditional Performance Testing Focus on IT/IS requirements Too difficult to model the entire application Length of time it takes to run a performance test Models have too many variables to be reliable Problems
New Approaches to Performance As applications have increased in complexity and application usage has become more diverse we have developed new tools and approaches to help us predict and manage the performance of applications
New Approaches to Performance Synthetic testing (traditional perf testing) A/B testing Instrumentation/Analytics (RUM) React Fast Over_Provision Third Party tools
Performance Testing Back To Basics
Rank Sites by Performance News related websites news.bbc.co.uk tv.msnbc.com
Rank Sites by Performance News related websites by response time news.bbc.co.uk (0.7) (3.3) (3.5) (3.9) tv.msnbc.com (5.5) (6.3) Gomez April 2013
Rank Sites by Performance News related websites by popularity tv.msnbc.com news.bbc.co.uk
The Five W's
WHO?
The Five W's WHO? WHAT?
The Five W's WHO? WHAT? WHY?
The Five W's WHO? WHAT? WHY? WHEN?
The Five W's WHO? WHAT? WHY? WHEN? WHERE?
Who is interested in performance? IS/IT Department
Who is interested in performance? IS/IT Department Protection of production Stability Capacity Resource
Who is interested in performance? Customer
Who is interested in performance? Customer Stability (of Browser) Freezing Consistency Ease of use (fewest steps to get job done)
Who is interested in performance? Business
Who is interested in performance? Business Revenue Unique monthly visitors Conversion rates Site Abandonment rate
Who is interested in performance? As Application complexity has increased so too has the amount of performance related information available As a performance tester it is important to be able to interpret the data and be able to present information that is useful and relevant to people.
Test models Choose 2 key business processes for each site
Test models Choose 2 key business processes for each site
Test models Choose 2 key business processes for each site
Test models Choose 2 key business processes for each site
What to Test? Information used to prioritize testing
What to Test? Information used to prioritize testing New technology Heavy usage Spikes in usage Key revenue generators Customer goals (or hot buttons)
What to Test? As it has become harder to model and test an entire application, being able to identify which areas of an application are key for performance is now more important than ever
Test models Choose 2 key business processes for each site
Why Do we Test? ?
Prevent problems Validate improvements Reproduce problems Aid in decision making
Why Do we Test? What Does a Successful Test look like? Setting goals Convert business goals into technical model Volumes (Average, Peak)
Understanding business process Using Webpagetest Free tool to allow you to see underlying calls made when you execute a business process Identify Key calls for performance
Understanding business process Using Webpagetest /09/waterfalls-101/
Using Webpagetest News related websites news.bbc.co.uk tv.msnbc.com
Understanding business process Identify Key calls for performance Based on who is Interested Based on what the key business processes are Based on why you are testing
Why do we Test? Integrate business goals with technical and customer goals Be wary of using design/developers as way of understanding what the application really does
When to Test? Traditionally performance testing was done just prior to release. Reasons for this: Models take time to build Need for stable baseline to compare against Site must be functional before perf testing
When to Test? Rise of Agile requires new approaches Split up into smaller component models to allow testing during development Clustering production allows testing in production with incremental rollout Test in design
Web Sequence diagrams Example for Facebook Login diagrams-examples.html
Where to Test? Dedicated Performance environment Prod Disaster recovery QA/Dev Environment Preprod Isolated components
Where to Test? Dont forget the surrounding infrastructure Network Load Balancers Caches CDN Third Party services
How? Lots of Vendors promising magic. Get demos and proof of concept for your site Only once you have a good understanding of the basics for your site.
How? Dont forget the basic free tools available WebPageTest Browser developer tools Curl SoapUI/LoadUI Many others
Questions