Download presentation
Published byRoderick Fisher Modified over 7 years ago
1
Agenda Introductions 10m Cavisson Intro 10m Product Presentation 30m
Next steps and Q&A 10m
2
Cavisson Systems, Inc What we do?
For web based applications, Our Products are aimed at Optimizing User Experience Perform Performance analysis Provide Capacity Planning US Patents pending technology
3
Cavisson Systems, Inc Sample Customers
Wells Fargo bank, One of the top 5 global banks Macys: US top retailer with over US$ 25 Billion in annual sales Oracle Corporation: Global software giant Renesas: Japanese electronic component conglomerate A10 networks: US based Bleeding edge Network component provider
4
Cavisson Systems, Inc Who we are? Leading edge technology provider
HQ in California, USA Sales office in New York, USA Development office in Noida, India Core team From top colleges (IIT’s, UC Berkley, Stanford) History of building industry success products (CDOT, India; AT&T; BEA Systems) Solid experience with all aspects of web technology
5
Cavisson Systems, Inc What’s unique about us!
We provide unique simulation solutions that replicate the production environment in the lab. Thus Provide user experience measure Optimize the applications for optimal resource utilization under real-world usage Help solve real-world production issues by replicating them in the lab
6
Issues with emerging new e-solutions
Failure to Plan is planning to fail Emerging e-solution just focus on development Little focus on deployment! Established e-solutions understands the need of deployment planning and hence load testing In western world, Almost all enterprise class web applications load tested In India – still emerging Much publicized election commission site failed during recent Lok Sabha elections. Because high visibility, became major news headline
7
Cavisson Systems, Inc Because of unique capabilities
Customers like Macys switched from legacy products to cavissson IBM & Oracle sidestepped their own tools to use cavisson solution on some their projects
8
Typical Web Application
Internet Web Application Multi-tier Servers Web Users
9
Web Application Load & Performance Testing
Internet Web Application Multi-tier Servers Web Users Emulated by NetStorm
10
Web Application Load & Performance Testing
Web Application Multi-tier Servers Emulated by NetStorm Emulates Internet & Web Users
11
Why NetStorm & NetOcean?
High performance Real-world emulation Emulate faithfully deployment environment in the Lab Provide Real-life view of web system Capacity & performance Extremely powerful – avoid network and server cluttering
12
Geographically Distributed Users With Diverse Network Access
Internet A New York Web User with DSL modem Web Application hosted in San Jose
13
A Typical Web transaction
Internet New York Web User points browser to a web site hosted in San Jose Web Application hosted in San Jose
14
A Typical Web transaction
Browser Opens several TCP connections to download a page Web Application hosted in San Jose Internet
15
A Typical Web transaction
Internet All Parallel connections share same modem bandwidth Web Application hosted in San Jose
16
A Typical Web transaction
All Network Communication faces SJ-NY WAN path Internet Web Application hosted in San Jose
17
A Typical Web transaction
Internet Web Application hosted in San Jose Server Gets request in Internet Random pattern
18
A Typical Web transaction
Server faces Web User Community behavior – Reloads on slow, sudden burst Internet Web Application hosted in San Jose
19
A Typical Web transaction
Internet Web Application hosted in San Jose All discussed elements play vital role in end-user response time & the server load
20
NetStorm: Emulates all web elements
NetStorm provides Real world view of End-user performance & Web System Load Web Application Multi-tier Servers NetStorm Appliance Emulates Internet & Web Users
21
NetStorm: Internet in a box!
Emulates geographically distributed web users Emulates network latency, jitter, packet drop and bandwidth impairments Unique capability to emulate user modem Internet traffic emulation
22
Lab View Vs Real-World View
Things That Matter Lab View (As Noted by Legacy tools: Load Runner ) Real-World View (As noted by Real users and NetStorm) Why Different? Response Time 1 Second 6 Sec (For Broadband Users averaged across USA) 14 Sec (For Dial-up users averaged across USA) Lacking simulation of Internet, Traffic pattern, User Community (Network Access, Location, browser and behavior) Performance Optimization Server service time optimized by 50% Server service time optimized by 3% User Response optimized by 8% User Response optimized by 30% Trying to optimize a component without understating big picture Overall response depends upon a complex mix of variables System capacity 60% 500 session/Hr 100% 500 session/Hr Network Concurrency is over 100X in real word Perceived Problem DB Server major cause Number of threads at App Server Major cause Focusing only on small sub-set of real-world setup
23
Overall View of System Performance including SUT
24
Co-Relate Server Performance with End User Response
25
Page Download Response Time for Geographic users
26
Would the response time for Geographic users meet SLA?
27
Page download time drill-down
28
NetStorm: More for Le$$
29
Why NetStorm & NetOcean?
Designed for Real-World load modeling of Enterprise scale in a small hardware footprint (order of magnitude small compared to Legacy systems such as Load Runner) Real-world User modeling for accurate production system predictions Provides Real-world view of system performance as opposed Lab-View of system performance provided by Legacy Systems Owing to its deep understanding and simulation of Network and Application layers, NetStorm has Solved real-life performance and production problems in large enterprises environments (Such as Macys, Wells Fargo, Oracle, National City Bank, Renesas,A10 Networks…). Replaced Network or Application centric tools such as IXIA, Load Runner as they fall short in identifying the root cause Companies like IBM Global services and Oracle choose to use NetStorm, sidestepping their own Load Generation tools
30
Case Study: Wells Fargo Bank
Planned to Deploy Load Balancer (SLB) Tested in the lab with legacy solutions Result suggested no change in performance Load balancer deployed Monitoring suggested increase in response time Rested – No clue about bad performance Tested with NetStorm NetStorm simulated the Real-world environment Displayed higher response with SLB Identified problem
31
Case Study: Macys 2007 Disaster – Session DB Growth
Holiday season nightmare Webload could not predict Band-Aid solutions LoadRunner bought, no new prediction NetStorm identified root cause Multiple WebSphere session getting created in single user session Owed to cookie not set for new users and thus creating new session on user navigations Concurrent user model employed by Load Runner could not maintain arrival rate and thus the pace of session creations Daily Session count creations now down 10 folds for same user sessions
32
Case Study: Macys Holiday stability Preparations
Many of the past instability related to OutOfMemory JVM Effort setup to make memory enhancements IBM’s Rational Load tester was employed from IBM Labs to create production sized load – Did not dent the system LoadRunner 8000 VU employed with 18 Load generators from IBM labs. Did not cause any serious impact on JVM memory Predict the new release with memory optimizations worse in performance Employed NetStorm Single Hardware box Created OutofMemory condition Predicted - better heap utilization with new release and no worse performance with new release After deployment, Production stats vindicate NetStorm prediction
33
Case Study: Macys Server monitors at work!
NetStorm collects server signatures Helped pin-point the trouble to changes in server configuration when Webspehere upgrade resulted in increased session count. NetStorm’s JAVA GC monitor helped establish the stable configuration Helped identify the root cause of Enterprise service bus seen in production. NetStorm Monitors pin-pointed the problem to server memory leak. Pin-pointed the root cause of server crash during Webspehere upgrade. Found a file descriptor leak. Identified the exact file name and process causing the leak.
34
Case Study: Macys Solid time savings
compact hardware foot-print (of the order of 1:20), test setup extremely reliable. Earlier used Load testing solution required reboot quite often and needed a lot of attention because of so many moving parts. NetStorm not rebooted even once since it was installed more than a year back. Test Comparison extremely fast. With Earlier load testing solutions, Test comparison was almost never employed because of long time for test comparison. NetStorm does it in a a couple of minutes using in-memory database and hashes as opposed to what took several hours to dissect and compare tests. NetStorm’s test suite automation used effectively to tune Java Virtual Machine (JVM) Performance. Tuning exercises were never conducted effectively earlier because of too much human time involvement.
35
Summary Credible Stress Environment Load Generation Why NetStorm?
Sized to replicate & stress Production System load Ability to replicate geographically distributed Users Ability to emulate Internet community behavior Minimize moving parts to increase test reliability and cutting corners Why NetStorm? Designed specifically to deal with Web Based systems, built on years of research Meets the goals & Objectives of Load Testing, Performance analysis & Capacity estimation Extremely powerful to simulate 100s of thousands real web users using single hardware box Solution of choice! Solved real-life performance and production problems in large enterprises environments (Such as Macys, Wells Fargo, Oracle, National City Bank, Renesas, A10 Networks…). Replaced Network or Application centric tools such as IXIA, Load Runner as they fall short in identifying the root cause Companies like IBM Global services and Oracle choose to use NetStorm, sidestepping their own Load Generation tools
36
Next Steps Websites Are there other websites under MP LUN?
for e-tenders      for  shop on-line sales Are there other websites under MP LUN? Who maintains (makes changes/updates) to these sites? MP LUN or some contracting company? What kind of servers are used for hosting/running these websites? Where are production server(s) hosted? Who maintain and monitors these servers? What are the Servers for testing and where they are located? And how test server configuration compares? Are they similar to one used for production system? Web site traffic details in hits /day? What is maximum hit rate observed on these sites?
37
Next Steps Websites Are there other websites under MP LUN?
for e-tenders      for  shop on-line sales Are there other websites under MP LUN? Who maintains (makes changes/updates) to these sites? MP LUN or some contracting company? What kind of servers are used for hosting/running these websites? Where are production server(s) hosted? Who maintain and monitors these servers? What are the Servers for testing and where they are located? And how test server configuration compares? Are they similar to one used for production system? Web site traffic details in hits /day? What is maximum hit rate observed on these sites?
38
Next Steps
39
Available Load test tools
40
NetStorm & NetOcean Web-based Application Load Testing Solution
Cavisson Systems Inc. Cavisson Proprietary and Confidential
41
Real-Life Network Cache
Internet R2 Network caches are on the network edge Main URL served by WF server. Embedded objects may be served by WF server or by nearest network cache (depending upon the main URL response HTML). Nearest network cache is usually nearer to user and hence usually better response if served by network cache. Network cache usually serve objects faster because of physical proximity to users. R1 > R2 Network Cache Origin server
42
Network Cache: Lab Test Setup
Internet Emulation R2 Network caches are on the network edge Main URL served by WF server. Embedded objects may be served by WF server or by nearest network cache (depending upon the main URL response HTML). Nearest network cache is usually nearer to user and hence usually better response if served by network cache. NetStorm emulates Geographically distributed users with Origin server at fixed location and Network cache at network edge for all users LAB NetOcean acting as Network cache Origin server
43
Web Application Load & Performance Testing
Origin Servers Emulated by NetStorm Emulates Internet & Web Users Network Cache Server emulated by NetStorm
44
Case study: Wells Fargo Bank Online Banking Authentication
Decides to use Load Balancer Pre-production tests used legacy load testing tools, saw no difference Before After
45
Case study:Wells Fargo Bank External monitoring Complains!
Response time gone up: 5.6 -> 6.2 sec Mostly extra time gone in SSL SLB Deployed
46
Case study:Wells Fargo Bank NetStorm used for Analysis!
Metric Direct With SLB Sign-On Load 1.212 1.29 Sign-On 3.601 3.875 Sign-Off 0.836 1.015 Total Sign On 5.649 6.18 New SSL Session/Sec 4.5 73.6 SSL Reuse Request/Sec 95.9 SSL Reused/Sec 26.7 Connections/Sec 100
47
Root Cause No SLB Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
48
Root Cause No SLB Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1 resolves online.wellsfargo.com (WF) to WS1 IP Address C4 WS3 Web Server Farm Web Clients
49
Root Cause No SLB Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1 Resolves online.wellsfargo.com (WF) to WS1 IP Address Client C1 Makes TCP connection to WS1. C4 WS3 Web Server Farm Web Clients
50
Root Cause No F5 Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WS1 SSID = 100 C1-WS1 SSID = 100 No F5 Case C1 WS1 C2 WS2 C3 C1 resolves online.wellsfargo.com (WF) to WS1 IP Address Client C1 Makes TCP connection to WS1. SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (ID assigned by WS1) C4 WS3 Web Server Farm Web Clients
51
Root Cause No F5 Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WS1 SSID = 100 C1-WS1 SSID = 100 No F5 Case C1 WS1 C2 WS2 C3 Web Client C1 Resolves online.wellsfargo.com to WS1 IP Address Client C1 Makes TCP connection to WS1. SSL handshake between Client C1 t & WS1. Both have reference to newly created SSL session ID = 100 (ID assigned by WS1) Client opens another connection in same web session to online.wellsfargo.com (DNS cached resolution points to WS1). C4 WS3 Web Server Farm Web Clients
52
Root Cause No F5 Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WS1 SSID = 100 C1-WS1 SSID = 100 No F5 Case C1 WS1 C2 WS2 C3 C1 resolves online.wellsfargo.com (WF) to WS1 IP Address Client C1 Makes TCP connection to WS1. SSL handshake between Client C1 t & WS1. Both have reference to newly created SSL session ID = 100 (ID assigned by WS1) Client opens another connection in same web session to online.wellsfargo.com (DNS cached resolution points to WS1). Client request to reuse session ID 100 (pre-negotiated SSL). WS1 is happy to reuse session ID 100 C4 WS3 Web Server Farm Web Clients
53
Root Cause With SLB Web Server Farm We Clients C1 WS1 C2 WS2 C3 C4 WS3
C1 Resolves online.wellsfargo.com (WF) to SLB IP Address C3 C4 WS3 Web Server Farm We Clients
54
Root Cause With SLB Web Server Farm We Clients C1 WS1 C2 WS2 C3 C4 WS3
C1 Resolves online.wellsfargo.com (WF) to SLB IP Address Client C1 makes TCP connection to SLB that is forwarded to WS1. C3 C4 WS3 Web Server Farm We Clients
55
Root Cause With SLB Web Server Farm We Clients C1 WS1 C2 WS2 C3 C4 WS3
C1-WF SSID=100 C1-WF SSID=100 With SLB C1 WS1 C2 WS2 Web Client C1 Resolves online.wellsfargo.com to SLB IP Address Client C1 Makes TCP connection to SLB that is forwarded to WS1. SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (ID assigned by WS1) C3 C4 WS3 Web Server Farm We Clients
56
Root Cause With SLB Web Server Farm We Clients C1 WS1 C2 WS2 C3 C4 WS3
Web Client C1 Resolves online.wellsfargo.com to SLB IP Address Client C1 Makes TCP connection to SLB that is forwarded to WS1. SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (SSID assigned by WS1) Client Makes another request in same web session to SLB that is forwarded to WS3 to load balance C3 C4 WS3 Web Server Farm We Clients
57
Root Cause With SLB Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WF SSID=100 C1-WF SSID=200 With SLB C1 WS1 C2 WS2 C1 Resolves online.wellsfargo.com (WF) to SLB IP Address Client C1 Makes TCP connection to SLB that is forwarded to WS1. SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (SSID assigned by WS1) Client Makes another request in same web session to SLB that is forwarded to WS3 to load balance Client request to reuse 100 (pre-negotiated SSL). WS3 has no knowledge of SSID = 100 and forces a new handshake with id 200 C3 C1-WF SSID=200 C4 WS3 Web Server Farm Web Clients
58
Root Cause With SLB Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WF SSID=100 C1-WF SSID=200 With SLB C1 WS1 C2 WS2 C1 Resolves online.wellsfargo.com (WF) to SLB IP Address Client C1 Makes TCP connection to SLB that is forwarded to WS1. SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (SSID assigned by WS1) Client Makes another request in same web session to SLB that is forwarded to WS2 to load balance Client request to reuse 100 (pre-negotiated SSL). WS3 has no knowledge of SSID = 100 and forces a new handshake with id 200 For a third new connection, Client will now request SSL reuse with SSID=200. If the new connection happen to served by any other web server than WS3, server will force new SSL handshake. C3 C1-WF SSID=200 C4 WS3 Web Server Farm Web Clients
59
Investigate options with NS: Keep-Alive is the winner!
Metric Direct With SLB SSL Sticky Keep-Alive One Teir Sign-On Load 1.212 1.29 1.294 1.142 1.221 Sign-On 3.601 3.875 3.788 2.861 2.815 Sign-Off 0.836 1.015 0.84 0.426 0.363 Total Sign On 5.649 6.18 5.922 4.429 4.399 New SSL Session/Sec 4.5 73.6 22.4 6.7 SSL Reuse Request/Sec 95.9 24.6 22.3 SSL Reused/Sec 26.7 6.6 Connections/Sec 100 29
60
NetStorm at Wells: 60 Sec Keep-Alive wins!
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.