Presentation is loading. Please wait.

Presentation is loading. Please wait.

How everyone should test their BizTalk based solutions

Similar presentations


Presentation on theme: "How everyone should test their BizTalk based solutions"— Presentation transcript:

1 How everyone should test their BizTalk based solutions
4/1/2017 6:56 AM How everyone should test their BizTalk based solutions Darren Jefford Solution Architect Microsoft UK © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2 Objectives And Takeaways
4/1/2017 6:56 AM Objectives And Takeaways Objective(s): Some complex BizTalk solutions run into problems NONE should! Raise awareness of key tools and techniques to dramatically improve customers experience Based on real world experience Drive the quality of enterprise-class BizTalk solutions up Administration is also key.. © 2005 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.

3 BizTalk, Testing, No Time!
We all know testing is important for every solution Customers rarely test well enough... but get away with it Solutions using BizTalk tend to be extremely business critical and tolerate no downtime Poor testing runs the risk of huge technical and political problems Extensive testing must always be performed, no excuses!

4 Testing approach “Test early, often and comprehensively”
Ensure that a demonstrator is created early on to prove concepts, patterns and indicative performance Too many solutions are found to perform badly in the last few weeks for any number of reasons Highlights sub-optimal design up-front Identifies where the bottlenecks are likely to be Increases confidence

5 We sometimes hear... “it doesn’t scale”
“it doesn’t meet our performance requirements” “we were told it would do XX msgs/sec” BizTalk is not A universal panacea Optimised for every customer scenario out-of-the-box Able to make bad code, good code BizTalk is part of the overall end-to-end solution

6 A great “case study” “ [BizUnit,LoadGen,MSTest] provides the ability to reliably and automatically execute hundreds of test cases that otherwise would have to be performed manually” “improved turn around times by a factor of upwards of 5. For a test pack containing 100 test cases which took up to 3 days to complete manually we can now execute in approx 3 hrs”

7 Laying the foundations
Critical to achieving representative performance within your environment Benchmark your environment File Adapter -> File Adapter (3K) message should get ~500msgs/sec Don’t run perf tests on workstation hardware Beware of virtual machines Invest in SQL Server Hardware and Disks Beware of UNC FILE Adapter locations Message Tracking Remember Virus Scanners

8 Foundation Checklist Databases and Logs separated onto separate drive letters and therefore separate physical hard drives? Are the drives used for Databases and Logs configured on a fast disk-array and with RAID10/0+1? Next generation SAN technology tends to use huge caches so RAID choice is becoming unimportant Default Database Autogrowth settings changes or AutoGrowth disabled? SQL Server Agent Running? Have you created separate Receive, Orchestration and Send Hosts and moved the appropriate Adapters and Orchestrations to the appropriate host? Concurrent Web Service Calls configuration file setting changed? <system.net> <connectionManagement> <add address="*” maxconnection="80"/> </connectionManagement> </system.net>

9 PerfMon Counters..... ?? A huge selection of performance counters are made available by the BizTalk engine It can be time consuming to identify the counters you need especially when you have multiple machines Collection of counters is key

10 Excel spreadsheet to define counter collection for multiple machines
4/1/2017 6:56 AM demo Perfmon Counters Excel spreadsheet to define counter collection for multiple machines Logman create counter MYCOUNTERNAME –cf YOURLISTOFCOUNTERS.TXT © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

11 Visual Studio Profiler
New with Visual Studio Team Developer Ideal for profiling custom code: custom pipeline components, .NET class libraries Profiler can be installed standalone for servers.. vs_profiler.exe located in the \vs\wcu\profiler directory of the VSTS media Must be driven via the command line to attach to a service

12 Configuring the VS Profiler
VSPerfClrEnv /globalsampleon VSPerfCmd /start:sample /output:“c:\dir\profileroutput" VSPerfCmd /attach:<PID> VSPerfCmd /detach:<PID> VSPerfCmd /shutdown VSPerfClrEnv /globaloff

13

14 Code Coverage Enables you to understand what code has been executed during your test run Identifies holes in your testing Ideal for custom pipeline components, helper assemblies, web services, etc. vsinstr –coverage “YOURASSEMBLY” vsperfmon –coverage –output “filename.coverage” vsperfcmd /shutdown

15 How do I test this? MSMQ BizTalk: DarrenJ Travel Services
Wiley Flight Services Booking Web Service Flight Orchestration Hotel Orchestration Itinerary Orchestration Itinerary Message [ASMX] MSMQ Hire Car Orchestration Travel Insurance Orchestration DirectEventStream OrchestrationEventStream BAMPrimaryImport

16 BizUnit Test Cases are constructed from generic and reusable test steps Test cases described using an XML syntax Driven from NUNIT or MSTest Model every “use case” within your solution Run a test If you get a “green light” your solution functions at a requirements level Incredibly powerful, not just for the development lifecycle but for changes, new releases..... Not just about BizTalk

17 Test Steps available “in box”
BAMDeploymentStep CheckPop3MailStep ContextManipulatorStep CrossReferenceData- VerificationStep DatabaseDeleteStep DatabaseRowCountStep DBQueryStep DelayStep DotNetObjectInvokerStep EventLogCheckStep EventLogClearStep ExecuteCommandStep FileCreateStep FileMultiValidateStep FilesExistStep FilesMoveStep FileDeleteMultipleStep FileDeleteStep FileMoveStep FileValidateStep HttpPostStep HttpRequestResponseStep MQSeriesClearQueueStep MQSeriesGetStep MQSeriesPutStep MSMQCreateQueueStep MSMQDeleteQueueStep MSMQQueuePurgeStep MSMQReadStep MSMQWriteStep OrchestrationConductorStep ReceiveLocationEnabledStep ReceivePortConductorStep RenameDirectoryStep RuleEngineStep SMTPReadStep SOAPHTTPRequest-ResponseStep WaitOnFileStep OutlookReadStep Your own.....

18 BizUnit Test Case Format
Setup Initialise the platform ready to perform the test Execution The execution of the test Cleanup Restore the platform to its state prior to test execution

19 Invoke a web service <TestStep assemblyPath="" typeName="SOAPHTTPRequestResponseStep"> <WebServiceWSDLURL> <ServiceName>MyWebServiceName</ServiceName> <WebMethod>SubmitItinerary</WebMethod> <InputMessageTypeName>Itinerary</InputMessageTypeName> <MessagePayload>ItineraryMessage.xml</MessagePayload> <ContextLoaderStep assemblyPath="" typeName="XmlContextLoader"> <XPath contextKey="itineraryReference"> //ItineraryReceived/attribute::ItineraryReference </XPath> </ContextLoaderStep> </TestStep>

20 Validate a MSMQ message
<TestStep assemblyPath="" typeName=“MSMQReadStep"> <QueuePath>.\Private$\HotelQ</QueuePath> <Timeout>10000</Timeout> <ValidationStep assemblyPath="“ typeName=“BinaryValidationStep"> <ComparisonDataPath> HotelFragment.xml </ComparisonDataPath> <CompareAsUTF8>true</CompareAsUTF8> </ValidationStep> </TestStep>

21 Validate database rows
<TestStep assemblyPath="" typeName=“DBQueryStep"> <DelayBeforeCheck>25</DelayBeforeCheck> <ConnectionString>...</ConnectionString> <SQLQuery> <RawSQLQuery> SELECT * FROM dbo.bam_Itinerary_Completed WHERE ActivityID='{0}‘ </RawSQLQuery> <SQLQueryParams> <SQLQueryParam takeFromCtx="itineraryReference“/> </SQLQueryParams> </SQLQuery> <Rows><Columns> <CustomerName>Darren Jefford</CustomerName> </Columns></Rows> </TestStep>

22 Writing a test for a end-to-end scenario
4/1/2017 6:56 AM demo BizUnit Writing a test for a end-to-end scenario © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23 Orchestration Profiler
Code Coverage tools don’t really help with Orchestrations How do you understand where time is being spend inside Orchestrations? Enabling tracking and using HAT is the only real way right now No way to understand which Orchestrations are not being fully tested Edge Scenarios Error paths

24 Orchestration Profiler
4/1/2017 6:56 AM demo Orchestration Profiler “Code Coverage” your orchestrations Identify slow performing shapes and orchestrations © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

25 LoadGen BizTalk Load Generation Tool
New 2007 release provides WCF support Generate messages to test how servers handle normal and extreme workload conditions Data Generation Load Throttling Find the maximum sustainable rate of your rig, automatically. Multithreaded “Throw away all those custom test harnesses”, please... 

26 “in-box” Load Generators
FILE Outputs files in a destination folder HTTP Supports both asynchronous and request-response messaging. SOAP Supports both asynchronous and request-response messaging MSMQ MQSeries Message can be local or remote, in-order or unordered, persistent or non-persistent. WSE WSS Windows® SharePoint® Services. Custom Base your own on the FileTransport sample

27 In-box Message Creators
CustomMC Create new messages based on a template Replace Items within the message with differing data SizeMC Explodes a sample message to a certain size (big batch files)

28 In-box Throttle Monitors
SQLSpool Ensure the Spool depth remains within a certain range Rate 100 msgs/sec.. CPU Keep the BizTalk servers within 80-90% CPU utilisation PerfMon Monitor any PerfMon counter

29 Generating load into your BizTalk solutions
4/1/2017 6:56 AM demo LoadGen Generating load into your BizTalk solutions © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

30 LoadGen and BizUnit LoadGen BizUnit BizUnit can drive LoadGen
Fantastic load generation tool Generate increasing load Find the maximum sustainable point of your solution BizUnit Extensive library of Test Steps Collecting PerfMon and EventViewer Logs Storing Test Information Validating Test Results BizUnit can drive LoadGen Automate all of your performance testing end-to-end!!

31 Driving test cases using LoadGen and BizUnit
4/1/2017 6:56 AM demo LoadGen and BizUnit Driving test cases using LoadGen and BizUnit © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

32 Shameless Plug Further Reading
In depth chapters covering Testing, Performance & Scalability..

33 Summary Get the environment right before any testing
Start testing early, and often. A “demonstrator” is key for complex scenarios Use BizUnit for modelling “use cases” Use LoadGen for generating load Use Orchestration Profiler to ensure your “touching” all parts of your solution during testing Use BizUnit and LoadGen together to automate your tests

34 Tools PerfMon Counter spreadsheet Orchestration Profiler
BizTalk Documenter LoadGen 2007 BizUnit

35 4/1/2017 6:56 AM © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION. © 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.


Download ppt "How everyone should test their BizTalk based solutions"

Similar presentations


Ads by Google