Presentation is loading. Please wait.

Presentation is loading. Please wait.

How everyone should test their BizTalk based solutions Darren Jefford Solution Architect Microsoft UK

Similar presentations


Presentation on theme: "How everyone should test their BizTalk based solutions Darren Jefford Solution Architect Microsoft UK"— Presentation transcript:

1 How everyone should test their BizTalk based solutions Darren Jefford Solution Architect Microsoft UK

2 Objectives And Takeaways Objective(s): Some complex BizTalk solutions run into problems NONE should! Raise awareness of key tools and techniques to dramatically improve customers experience Based on real world experience Drive the quality of enterprise-class BizTalk solutions up Administration is also key..

3 BizTalk, Testing, No Time! We all know testing is important for every solution Customers rarely test well enough... but get away with it Solutions using BizTalk tend to be extremely business critical and tolerate no downtime Poor testing runs the risk of huge technical and political problems Extensive testing must always be performed, no excuses!

4 Testing approach Test early, often and comprehensively Ensure that a demonstrator is created early on to prove concepts, patterns and indicative performance Too many solutions are found to perform badly in the last few weeks for any number of reasons Highlights sub-optimal design up-front Identifies where the bottlenecks are likely to be Increases confidence

5 We sometimes hear... it doesnt scale it doesnt meet our performance requirements we were told it would do XX msgs/sec BizTalk is not A universal panacea Optimised for every customer scenario out-of- the-box Able to make bad code, good code BizTalk is part of the overall end-to-end solution

6 A great case study [BizUnit,LoadGen,MSTest] provides the ability to reliably and automatically execute hundreds of test cases that otherwise would have to be performed manually improved turn around times by a factor of upwards of 5. For a test pack containing 100 test cases which took up to 3 days to complete manually we can now execute in approx 3 hrs

7 Laying the foundations Critical to achieving representative performance within your environment Benchmark your environment File Adapter -> File Adapter (3K) message should get ~500msgs/sec Dont run perf tests on workstation hardware Beware of virtual machines Invest in SQL Server Hardware and Disks Beware of UNC FILE Adapter locations Message Tracking Remember Virus Scanners

8 Foundation Checklist Databases and Logs separated onto separate drive letters and therefore separate physical hard drives? Are the drives used for Databases and Logs configured on a fast disk- array and with RAID10/0+1? Next generation SAN technology tends to use huge caches so RAID choice is becoming unimportant Default Database Autogrowth settings changes or AutoGrowth disabled? SQL Server Agent Running? Have you created separate Receive, Orchestration and Send Hosts and moved the appropriate Adapters and Orchestrations to the appropriate host? Concurrent Web Service Calls configuration file setting changed?

9 PerfMon Counters..... ?? A huge selection of performance counters are made available by the BizTalk engine It can be time consuming to identify the counters you need especially when you have multiple machines Collection of counters is key

10 Perfmon Counters Excel spreadsheet to define counter collection for multiple machines

11 Visual Studio Profiler New with Visual Studio Team Developer Ideal for profiling custom code: custom pipeline components,.NET class libraries Profiler can be installed standalone for servers.. vs_profiler.exe located in the \vs\wcu\profiler directory of the VSTS media Must be driven via the command line to attach to a service

12 Configuring the VS Profiler VSPerfClrEnv /globalsampleon VSPerfCmd /start:sample /output:c:\dir\profileroutput" VSPerfCmd /attach: VSPerfCmd /detach: VSPerfCmd /shutdown VSPerfClrEnv /globaloff

13

14 Code Coverage Enables you to understand what code has been executed during your test run Identifies holes in your testing Ideal for custom pipeline components, helper assemblies, web services, etc. vsinstr –coverage YOURASSEMBLY vsperfmon –coverage –output filename.coverage vsperfcmd /shutdown

15 How do I test this? BAMPrimaryImportBAMPrimaryImport BizTalk: DarrenJ Travel Services Hotel Orchestration Flight Orchestration Hire Car Orchestration Travel Insurance Orchestration OrchestrationEventStreamOrchestrationEventStream Wiley Flight Services Booking Web Service DirectEventStreamDirectEventStream Itinerary Message [ASMX] [ASMX] Itinerary Orchestration MSMQ

16 BizUnit Test Cases are constructed from generic and reusable test steps Test cases described using an XML syntax Driven from NUNIT or MSTest Model every use case within your solution Run a test If you get a green light your solution functions at a requirements level Incredibly powerful, not just for the development lifecycle but for changes, new releases..... Not just about BizTalk

17 Test Steps available in box BAMDeploymentStep CheckPop3MailStep ContextManipulatorStep CrossReferenceData- VerificationStep DatabaseDeleteStep DatabaseRowCountStep DBQueryStep DelayStep DotNetObjectInvokerStep EventLogCheckStep EventLogClearStep ExecuteCommandStep FileCreateStepFileMultiValidateStepFilesExistStepFilesMoveStepFileDeleteMultipleStep FileDeleteStepFileMoveStepFileValidateStepHttpPostStepHttpRequestResponseStepMQSeriesClearQueueStepMQSeriesGetStepMQSeriesPutStepMSMQCreateQueueStepMSMQDeleteQueueStepMSMQQueuePurgeStepMSMQReadStepMSMQWriteStepOrchestrationConductorStepReceiveLocationEnabledStepReceivePortConductorStepRenameDirectoryStepRuleEngineStepSMTPReadStep SOAPHTTPRequest- ResponseStep WaitOnFileStepOutlookReadStep Your own..... Your own.....

18 BizUnit Test Case Format Setup Initialise the platform ready to perform the test Execution The execution of the test Cleanup Restore the platform to its state prior to test execution

19 Invoke a web service MyWebServiceName SubmitItinerary Itinerary ItineraryMessage.xml //ItineraryReceived/attribute::ItineraryReference

20 Validate a MSMQ message.\Private$\HotelQ HotelFragment.xml true

21 Validate database rows SELECT * FROM dbo.bam_Itinerary_Completed WHERE ActivityID='{0} Darren Jefford

22 BizUnit Writing a test for a end-to-end scenario

23 Orchestration Profiler Code Coverage tools dont really help with Orchestrations How do you understand where time is being spend inside Orchestrations? Enabling tracking and using HAT is the only real way right now No way to understand which Orchestrations are not being fully tested Edge Scenarios Error paths

24 Orchestration Profiler Code Coverage your orchestrations Identify slow performing shapes and orchestrations

25 LoadGen BizTalk Load Generation Tool New 2007 release provides WCF support Generate messages to test how servers handle normal and extreme workload conditions Data Generation Load Throttling Find the maximum sustainable rate of your rig, automatically. Multithreaded Throw away all those custom test harnesses, please...

26 in-box Load Generators FILEOutputs files in a destination folder HTTPSupports both asynchronous and request-response messaging. SOAPSupports both asynchronous and request-response messaging MSMQSupports both asynchronous and request-response messaging MQSeriesMessage can be local or remote, in-order or unordered, persistent or non-persistent. WSESupports both asynchronous and request-response messaging. WSSWindows® SharePoint® Services. CustomBase your own on the FileTransport sample

27 In-box Message Creators CustomMC Create new messages based on a template Replace Items within the message with differing data SizeMC Explodes a sample message to a certain size (big batch files)

28 In-box Throttle Monitors SQLSpool Ensure the Spool depth remains within a certain range Rate 100 msgs/sec.. CPU Keep the BizTalk servers within 80-90% CPU utilisation PerfMon Monitor any PerfMon counter

29 LoadGen Generating load into your BizTalk solutions

30 LoadGen and BizUnit LoadGen Fantastic load generation tool Generate increasing load Find the maximum sustainable point of your solution BizUnit Extensive library of Test Steps Collecting PerfMon and EventViewer Logs Storing Test Information Validating Test Results BizUnit can drive LoadGen Automate all of your performance testing end- to-end!!

31 LoadGen and BizUnit Driving test cases using LoadGen and BizUnit

32 Shameless Plug Further Reading In depth chapters covering Testing, Performance & Scalability..

33 Summary Get the environment right before any testing Start testing early, and often. A demonstrator is key for complex scenarios Use BizUnit for modelling use cases Use LoadGen for generating load Use Orchestration Profiler to ensure your touching all parts of your solution during testing Use BizUnit and LoadGen together to automate your tests

34 Tools PerfMon Counter spreadsheet Orchestration Profiler BizTalk Documenter LoadGen 2007 BizUnit

35 © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.


Download ppt "How everyone should test their BizTalk based solutions Darren Jefford Solution Architect Microsoft UK"

Similar presentations


Ads by Google