1. Agenda Introductions 10m Cavisson Intro 10m Product Presentation 30m
Next steps and Q&A 10m

2. Cavisson Systems, Inc What we do?
For web based applications, Our Products are aimed at
Optimizing User Experience
Perform Performance analysis
Provide Capacity Planning
US Patents pending technology

3. Cavisson Systems, Inc Sample Customers
Wells Fargo bank, One of the top 5 global banks
Macys: US top retailer with over US$ 25 Billion in annual sales
Oracle Corporation: Global software giant
Renesas: Japanese electronic component conglomerate
A10 networks: US based Bleeding edge Network component provider

4. Cavisson Systems, Inc Who we are? Leading edge technology provider
HQ in California, USA
Sales office in New York, USA
Development office in Noida, India
Core team
From top colleges (IIT’s, UC Berkley, Stanford)
History of building industry success products (CDOT, India; AT&T; BEA Systems)
Solid experience with all aspects of web technology

5. Cavisson Systems, Inc What’s unique about us!
We provide unique simulation solutions that replicate the production environment in the lab. Thus
Provide user experience measure
Optimize the applications for optimal resource utilization under real-world usage
Help solve real-world production issues by replicating them in the lab

6. Issues with emerging new e-solutions
Failure to Plan is planning to fail
Emerging e-solution just focus on development
Little focus on deployment!
Established e-solutions understands the need of deployment planning and hence load testing
In western world, Almost all enterprise class web applications load tested
In India – still emerging
Much publicized election commission site failed during recent Lok Sabha elections.
Because high visibility, became major news headline

7. Cavisson Systems, Inc Because of unique capabilities
Customers like Macys switched from legacy products to cavissson
IBM & Oracle sidestepped their own tools to use cavisson solution on some their projects

8. Typical Web Application
Internet
Web Application Multi-tier Servers
Web Users

9. Web Application Load & Performance Testing
Internet
Web Application Multi-tier Servers
Web Users
Emulated by NetStorm

10. Web Application Load & Performance Testing
Web Application Multi-tier Servers
Emulated by NetStorm
Emulates Internet & Web Users

11. Why NetStorm & NetOcean?
High performance Real-world emulation
Emulate faithfully deployment environment in the Lab
Provide Real-life view of web system Capacity & performance
Extremely powerful – avoid network and server cluttering

12. Geographically Distributed Users With Diverse Network Access
Internet
A New York Web User with DSL modem
Web Application hosted in San Jose

13. A Typical Web transaction
Internet
New York Web User points browser to a web site hosted in San Jose
Web Application hosted in San Jose

14. A Typical Web transaction
Browser Opens several TCP connections to download a page
Web Application hosted in San Jose
Internet

15. A Typical Web transaction
Internet
All Parallel connections share same modem bandwidth
Web Application hosted in San Jose

16. A Typical Web transaction
All Network Communication faces SJ-NY WAN path
Internet
Web Application hosted in San Jose

17. A Typical Web transaction
Internet
Web Application hosted in San Jose
Server Gets request in Internet Random pattern

18. A Typical Web transaction
Server faces Web User Community behavior – Reloads on slow, sudden burst
Internet
Web Application hosted in San Jose

19. A Typical Web transaction
Internet
Web Application hosted in San Jose
All discussed elements play vital role in end-user response time & the server load

20. NetStorm: Emulates all web elements
NetStorm provides Real world view of End-user performance & Web System Load
Web Application Multi-tier Servers
NetStorm Appliance
Emulates Internet & Web Users

21. NetStorm: Internet in a box!
Emulates geographically distributed web users
Emulates network latency, jitter, packet drop and bandwidth impairments
Unique capability to emulate user modem
Internet traffic emulation

22. Lab View Vs Real-World View
Things That Matter
Lab View (As Noted by Legacy tools: Load Runner )
Real-World View (As noted by Real users and NetStorm)
Why Different?
Response Time
1 Second
6 Sec (For Broadband Users averaged across USA)
14 Sec (For Dial-up users averaged across USA)
Lacking simulation of Internet, Traffic pattern, User Community (Network Access, Location, browser and behavior)
Performance Optimization
Server service time optimized by 50%
Server service time optimized by 3%
User Response optimized by 8%
User Response optimized by 30%
Trying to optimize a component without understating big picture
Overall response depends upon a complex mix of variables
System capacity
60% 500 session/Hr
100% 500 session/Hr
Network Concurrency is over 100X in real word
Perceived Problem
DB Server major cause
Number of threads at App Server Major cause
Focusing only on small sub-set of real-world setup

23. Overall View of System Performance including SUT

24. Co-Relate Server Performance with End User Response

25. Page Download Response Time for Geographic users

26. Would the response time for Geographic users meet SLA?

27. Page download time drill-down

28. NetStorm: More for Le$$

29. Why NetStorm & NetOcean?
Designed for Real-World load modeling of Enterprise scale in a small hardware footprint (order of magnitude small compared to Legacy systems such as Load Runner)
Real-world User modeling for accurate production system predictions
Provides Real-world view of system performance as opposed Lab-View of system performance provided by Legacy Systems
Owing to its deep understanding and simulation of Network and Application layers, NetStorm has
Solved real-life performance and production problems in large enterprises environments (Such as Macys, Wells Fargo, Oracle, National City Bank, Renesas,A10 Networks…).
Replaced Network or Application centric tools such as IXIA, Load Runner as they fall short in identifying the root cause
Companies like IBM Global services and Oracle choose to use NetStorm, sidestepping their own Load Generation tools

30. Case Study: Wells Fargo Bank
Planned to Deploy Load Balancer (SLB)
Tested in the lab with legacy solutions
Result suggested no change in performance
Load balancer deployed
Monitoring suggested increase in response time
Rested – No clue about bad performance
Tested with NetStorm
NetStorm simulated the Real-world environment
Displayed higher response with SLB
Identified problem

31. Case Study: Macys 2007 Disaster – Session DB Growth
Holiday season nightmare
Webload could not predict
Band-Aid solutions
LoadRunner bought, no new prediction
NetStorm identified root cause
Multiple WebSphere session getting created in single user session
Owed to cookie not set for new users and thus creating new session on user navigations
Concurrent user model employed by Load Runner could not maintain arrival rate and thus the pace of session creations
Daily Session count creations now down 10 folds for same user sessions

32. Case Study: Macys Holiday stability Preparations
Many of the past instability related to OutOfMemory JVM
Effort setup to make memory enhancements
IBM’s Rational Load tester was employed from IBM Labs to create production sized load – Did not dent the system
LoadRunner 8000 VU employed with 18 Load generators from IBM labs.
Did not cause any serious impact on JVM memory
Predict the new release with memory optimizations worse in performance
Employed NetStorm
Single Hardware box
Created OutofMemory condition
Predicted - better heap utilization with new release and no worse performance with new release
After deployment, Production stats vindicate NetStorm prediction

33. Case Study: Macys Server monitors at work!
NetStorm collects server signatures
Helped pin-point the trouble to changes in server configuration when Webspehere upgrade resulted in increased session count.
NetStorm’s JAVA GC monitor helped establish the stable configuration
Helped identify the root cause of Enterprise service bus seen in production. NetStorm Monitors pin-pointed the problem to server memory leak.
Pin-pointed the root cause of server crash during Webspehere upgrade. Found a file descriptor leak. Identified the exact file name and process causing the leak.

34. Case Study: Macys Solid time savings
compact hardware foot-print (of the order of 1:20), test setup extremely reliable. Earlier used Load testing solution required reboot quite often and needed a lot of attention because of so many moving parts. NetStorm not rebooted even once since it was installed more than a year back.
Test Comparison extremely fast. With Earlier load testing solutions, Test comparison was almost never employed because of long time for test comparison. NetStorm does it in a a couple of minutes using in-memory database and hashes as opposed to what took several hours to dissect and compare tests.
NetStorm’s test suite automation used effectively to tune Java Virtual Machine (JVM) Performance. Tuning exercises were never conducted effectively earlier because of too much human time involvement.

35. Summary Credible Stress Environment Load Generation Why NetStorm?
Sized to replicate & stress Production System load
Ability to replicate geographically distributed Users
Ability to emulate Internet community behavior
Minimize moving parts to increase test reliability and cutting corners
Why NetStorm?
Designed specifically to deal with Web Based systems, built on years of research
Meets the goals & Objectives of Load Testing, Performance analysis & Capacity estimation
Extremely powerful to simulate 100s of thousands real web users using single hardware box
Solution of choice!
Solved real-life performance and production problems in large enterprises environments (Such as Macys, Wells Fargo, Oracle, National City Bank, Renesas, A10 Networks…).
Replaced Network or Application centric tools such as IXIA, Load Runner as they fall short in identifying the root cause
Companies like IBM Global services and Oracle choose to use NetStorm, sidestepping their own Load Generation tools

36. Next Steps Websites Are there other websites under MP LUN?
for e-tenders
     for  shop on-line sales
Are there other websites under MP LUN?
Who maintains (makes changes/updates) to these sites? MP LUN or some contracting company?
What kind of servers are used for hosting/running these websites? Where are  production server(s) hosted? Who maintain and monitors these servers?
What are the Servers for testing and where they are located? And how test server configuration compares? Are they similar to one used for production system?
Web site traffic details in hits /day? What is maximum hit rate observed on these sites?

37. Next Steps Websites Are there other websites under MP LUN?
for e-tenders
     for  shop on-line sales
Are there other websites under MP LUN?
Who maintains (makes changes/updates) to these sites? MP LUN or some contracting company?
What kind of servers are used for hosting/running these websites? Where are  production server(s) hosted? Who maintain and monitors these servers?
What are the Servers for testing and where they are located? And how test server configuration compares? Are they similar to one used for production system?
Web site traffic details in hits /day? What is maximum hit rate observed on these sites?

38. Next Steps

39. Available Load test tools

40. NetStorm & NetOcean Web-based Application Load Testing Solution
Cavisson Systems Inc.
Cavisson Proprietary and Confidential

41. Real-Life Network Cache
Internet R2
Network caches are on the network edge
Main URL served by WF server. Embedded objects may be served by WF server or by nearest network cache (depending upon the main URL response HTML).
Nearest network cache is usually nearer to user and hence usually better response if served by network cache.
Network cache usually serve objects faster because of physical proximity to users.
R1 > R2
Network Cache
Origin server

42. Network Cache: Lab Test Setup
Internet Emulation R2
Network caches are on the network edge
Main URL served by WF server. Embedded objects may be served by WF server or by nearest network cache (depending upon the main URL response HTML).
Nearest network cache is usually nearer to user and hence usually better response if served by network cache.
NetStorm emulates Geographically distributed users with Origin server at fixed location and Network cache at network edge for all users
LAB
NetOcean acting as Network cache
Origin server

43. Web Application Load & Performance Testing
Origin Servers
Emulated by NetStorm
Emulates Internet & Web Users
Network Cache Server emulated by NetStorm

44. Case study: Wells Fargo Bank Online Banking Authentication
Decides to use Load Balancer
Pre-production tests used legacy load testing tools, saw no difference
Before After

45. Case study:Wells Fargo Bank External monitoring Complains!
Response time gone up: 5.6 -> 6.2 sec
Mostly extra time gone in SSL
SLB Deployed

46. Case study:Wells Fargo Bank NetStorm used for Analysis!
Metric
Direct
With SLB
Sign-On Load
1.212
1.29
Sign-On
3.601
3.875
Sign-Off
0.836
1.015
Total Sign On
5.649
6.18
New SSL Session/Sec
4.5
73.6
SSL Reuse Request/Sec
95.9
SSL Reused/Sec
26.7
Connections/Sec
100

47. Root Cause No SLB Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4

48. Root Cause No SLB Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1 resolves online.wellsfargo.com (WF) to WS1 IP Address C4
WS3
Web Server Farm
Web Clients

49. Root Cause No SLB Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1 Resolves online.wellsfargo.com (WF) to WS1 IP Address
Client C1 Makes TCP connection to WS1. C4
WS3
Web Server Farm
Web Clients

50. Root Cause No F5 Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WS1 SSID = 100
C1-WS1 SSID = 100
No F5 Case C1
WS1 C2
WS2 C3
C1 resolves online.wellsfargo.com (WF) to WS1 IP Address
Client C1 Makes TCP connection to WS1.
SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (ID assigned by WS1) C4
WS3
Web Server Farm
Web Clients

51. Root Cause No F5 Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WS1 SSID = 100
C1-WS1 SSID = 100
No F5 Case C1
WS1 C2
WS2 C3
Web Client C1 Resolves online.wellsfargo.com to WS1 IP Address
Client C1 Makes TCP connection to WS1.
SSL handshake between Client C1 t & WS1. Both have reference to newly created SSL session ID = 100 (ID assigned by WS1)
Client opens another connection in same web session to online.wellsfargo.com (DNS cached resolution points to WS1). C4
WS3
Web Server Farm
Web Clients

52. Root Cause No F5 Case Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WS1 SSID = 100
C1-WS1 SSID = 100
No F5 Case C1
WS1 C2
WS2 C3
C1 resolves online.wellsfargo.com (WF) to WS1 IP Address
Client C1 Makes TCP connection to WS1.
SSL handshake between Client C1 t & WS1. Both have reference to newly created SSL session ID = 100 (ID assigned by WS1)
Client opens another connection in same web session to online.wellsfargo.com (DNS cached resolution points to WS1).
Client request to reuse session ID 100 (pre-negotiated SSL). WS1 is happy to reuse session ID 100 C4
WS3
Web Server Farm
Web Clients

53. Root Cause With SLB Web Server Farm We Clients C1 WS1 C2 WS2 C3 C4 WS3
C1 Resolves online.wellsfargo.com (WF) to SLB IP Address C3 C4
WS3
Web Server Farm
We Clients

54. Root Cause With SLB Web Server Farm We Clients C1 WS1 C2 WS2 C3 C4 WS3
C1 Resolves online.wellsfargo.com (WF) to SLB IP Address
Client C1 makes TCP connection to SLB that is forwarded to WS1. C3 C4
WS3
Web Server Farm
We Clients

55. Root Cause With SLB Web Server Farm We Clients C1 WS1 C2 WS2 C3 C4 WS3
C1-WF SSID=100
C1-WF SSID=100
With SLB C1
WS1 C2
WS2
Web Client C1 Resolves online.wellsfargo.com to SLB IP Address
Client C1 Makes TCP connection to SLB that is forwarded to WS1.
SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (ID assigned by WS1) C3 C4
WS3
Web Server Farm
We Clients

56. Root Cause With SLB Web Server Farm We Clients C1 WS1 C2 WS2 C3 C4 WS3
Web Client C1 Resolves online.wellsfargo.com to SLB IP Address
Client C1 Makes TCP connection to SLB that is forwarded to WS1.
SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (SSID assigned by WS1)
Client Makes another request in same web session to SLB that is forwarded to WS3 to load balance C3 C4
WS3
Web Server Farm
We Clients

57. Root Cause With SLB Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WF SSID=100
C1-WF SSID=200
With SLB C1
WS1 C2
WS2
C1 Resolves online.wellsfargo.com (WF) to SLB IP Address
Client C1 Makes TCP connection to SLB that is forwarded to WS1.
SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (SSID assigned by WS1)
Client Makes another request in same web session to SLB that is forwarded to WS3 to load balance
Client request to reuse 100 (pre-negotiated SSL). WS3 has no knowledge of SSID = 100 and forces a new handshake with id 200 C3
C1-WF SSID=200 C4
WS3
Web Server Farm
Web Clients

58. Root Cause With SLB Web Server Farm Web Clients C1 WS1 C2 WS2 C3 C4
C1-WF SSID=100
C1-WF SSID=200
With SLB C1
WS1 C2
WS2
C1 Resolves online.wellsfargo.com (WF) to SLB IP Address
Client C1 Makes TCP connection to SLB that is forwarded to WS1.
SSL handshake between Client C1 & WS1. Both have reference to newly created SSL session ID = 100 (SSID assigned by WS1)
Client Makes another request in same web session to SLB that is forwarded to WS2 to load balance
Client request to reuse 100 (pre-negotiated SSL). WS3 has no knowledge of SSID = 100 and forces a new handshake with id 200
For a third new connection, Client will now request SSL reuse with SSID=200. If the new connection happen to served by any other web server than WS3, server will force new SSL handshake. C3
C1-WF SSID=200 C4
WS3
Web Server Farm
Web Clients

59. Investigate options with NS: Keep-Alive is the winner!
Metric
Direct
With SLB
SSL Sticky
Keep-Alive
One Teir
Sign-On Load
1.212
1.29
1.294
1.142
1.221
Sign-On
3.601
3.875
3.788
2.861
2.815
Sign-Off
0.836
1.015
0.84
0.426
0.363
Total Sign On
5.649
6.18
5.922
4.429
4.399
New SSL Session/Sec
4.5
73.6
22.4
6.7
SSL Reuse Request/Sec
95.9
24.6
22.3
SSL Reused/Sec
26.7
6.6
Connections/Sec
100 29

60. NetStorm at Wells: 60 Sec Keep-Alive wins!