1. 4/26/2017
Use Cloud-Based Load Testing Service to Find Scale and Performance Bottlenecks
Randy Pagels
Sr. Developer Technology Specialist
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Agenda Why load testing? Cloud load testing Application insights
4/26/2017
Agenda
Why load testing?
Cloud load testing
Application insights
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. Why Load Testing? Performance testing Load testing Stress testing
How fast will the application code perform?
Load testing
How will the application behave in front of the expected users load?
Stress testing
What is point of users-load that will break your application?
Capacity planning
Will the application be able to scale up to the expected future capacity?

4. Load Test Agent Architecture-On Prem
Visual Studio Enterprise
Unlimited Load Test Virtual Users
Test Controller
Agent
Agent
Agent
Agent
Collect ASP.NET profiler traces
Target Server
Target Server
Collector Agent
Target Server

5. Demo On Prem Load Testing

6. What is needed to make that work?
Buy appropriate computers
Net load testing can be:
Expensive to setup
Provisioning is complex
Slow to scale
Costly to maintain
Physically setup computers
Install your controllers/agents
Replicate this to all the computers

7. Demo Cloud based Load Testing

8. Architecture Visual Studio Test Agent Pool - Dynamic
Load Test Web Service
Worker
Azure Tables
Visual Studio
Results database
Azure Blobs

9. Benefits of cloud load testing
No need to setup your own infrastructure and maintain it.
Get infrastructure in cloud when you need it.
Use the same load test project which you use for on premises today.
Scale out easily in case of doing stress testing.

10. Different ways for Running Performance Tests

11. VSO-Quick Performance Test
Quickly define and configure simple load tests
Use the elastic scale of VSO to simulate hundreds or thousands of virtual users
Measure the underlying system using Application Insights

12. Azure – Performance Test Web App

13. Visual Studio – Author Scenarios

14. Performance Testing

15. Cloud Load Testing Applications behind the firewall
4/26/2017
Cloud Load Testing Applications behind the firewall
Load testing Internal Apps require the load test agents in the Cloud-based Load Testing Service to reach them in a secure way. One way to accomplish this scenario is to use 'Trusted IP' as described below.
Load testing Applications behind a Firewall
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

16. DevOps Modern app development
4/26/2017
DevOps
Modern app development
Continuous value delivery and rapid response to change
There are two important considerations when creating modern applications:
continuously delivering value to your customers and rapidly responding to changes in the marketplace.
Delivering at the pace your customers expect is all about decreasing cycle time to incorporate additional feedback loops. A good application lifecycle management solution will help you coordinate your software development efforts from ideation through delivery, and incorporate customer feedback and user insights into your next iteration. This enables you to drive down your cycle times and close your feedback loops; making you better equipped to operate in today’s ever-changing marketplace.
Quotes to look for:
Narrative guidance:
Modern application development boils down to two key concepts. Continuously deliver value to your customers and rapidly respond to changes in the marketplace. Delivering at the pace your customers expect is all about cycle time and feedback loops. With a good application lifecycle management solution, you can coordinate your software development efforts from ideation to delivery to incorporating customer feedback into the next iteration. This will enable you to drive down your cycle times and close your feedback loops; making you better equipped to operate in today’s ever changing marketplace.
Transition to next slide:
This isn’t a new story. ALM has been the norm for software development for the last 10 years. Now let’s take a closer look at some emerging trends in the software development industry.
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

17. Visual Studio Online: DevOps Workflow
Code Repository
Team Build
Deploy to Slot
UI & Load Testing
Swap Slots
Why Azure App Service builds fully managed DevOps solutions: so you can manage business goals rather than infrastructure.
Azure
Visual Studio
Eclipse / XCode
Application Insights and Hockey App

18. Demo Automate Cloud-based Load Test for your continuous integration pipeline

19. Add “Cloud-based Load Test” Build Tasks

20. Add as part of the Release Pipeline

21. Deeper Analysis Measure the underlying system using Application Insights

22. Microsoft Ignite 2015
4/26/2017 9:16 PM 3
Application Insights
Get a 360° view of the application including availability, performance and usage patterns 1
Telemetry is collected at each tier: mobile applications, server applications and browser 2
We do that as part of a service we provide called Application Insights. Whether you are in production or development Application Insights for Visual Studio Online offers a set of developer services to give you actionable insight into the availability, performance and usage of your applications.
Telemetry arrives in the Application Insights service in the cloud where it is processed & stored
© 2015 Microsoft Corporation. All rights reserved. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. Application Insights Global System Monitoring
REQUIREMENTS
BACKLOG
RELEASE
Operate
Construct
WORKING SOFTWARE
Build | Measure | Learn
Collaboration
Early discovery with Alerts
Application Performance Monitoring
Actionable data w/ Log Mining
Customer Usage Monitoring

24. Customer Usage Monitoring

26. Demo Application Insights

27. Trend Analysis Reporting with Excel

28. Run Performance Report
Load Test Analysis Built into Excel for run comparison

29. Run Performance Report – Compare Runs
Load Test Analysis for Baseline Runs

30. Who can use cloud load testing?
One who has VS Enterprise and an account with VSO
Single URL to get started:
Application is available on the internet
Can use IP Filtering for applications behind firewall
First 20,000 virtual user minutes/month (VUM): Free!
$ / VUM for 20,001-2 M VUM
$ / VUM for 2,000, M VUM
$ / VUM for usage beyond 10 M VUM/month
Support
Contact Visual Studio Online support, for direct contact with the team
Visual Studio Online Pricing Details

31. What’s new for VS 2015?
Single Scale to 100 cores (1,000 cores when using parallel runs!!)
Author and initiate load tests from a browser experience
Run load tests up to the 72 hours
Collect performance data from production applications with Application Insights
Specify where your load test traffic is generated from

33. Cloud Based Load Tests Cloud load testing
4/26/2017
Cloud Based Load Tests
Cloud load testing
Performance test your code with hundreds of thousands of users
Author load tests from Visual Studio then use the cloud for the right load mix
Configuration options include number of virtual users, browser distribution and more
Test from where your users are
Run tests from one of many global Azure datacenter locations to minimize latency and simulate users' real-world conditions
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

34. Visual Studio Online Free Subscription
Visual Studio Online benefits are included in your MSDN Subscription
Create Team Projects for up to 5 users for free
Get started for free

35. References
When you run your first load test you will receive an like this from the Cloud Based Load Testing Team. This has several helpful links .
[CLICK]
The most helpful link is the additional resources link which takes you to the following web page.

36. References

37. References

38. Questions?
Randy Pagels

39. © 2014 Microsoft Corporation. All rights reserved
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.