how everyone should test their biztalk based solutions darren jefford solution architect microsoft...
TRANSCRIPT
How everyone should test their BizTalk based solutions
Darren JeffordSolution ArchitectMicrosoft [email protected]://blogs.msdn.com/darrenj
Objectives And Takeaways
Objective(s): Some complex BizTalk solutions run into problemsNONE should!
Raise awareness of key tools and techniques to dramatically improve customers experience
Based on real world experienceDrive the quality of enterprise-class BizTalk solutions up
Administration is also key..
BizTalk, Testing, No Time!
We all know testing is important for every solutionCustomers rarely test well enough... but get away with itSolutions using BizTalk tend to be extremely business critical and tolerate no downtimePoor testing runs the risk of huge technical and political problemsExtensive testing must always be performed, no excuses!
Testing approach
“Test early, often and comprehensively”Ensure that a demonstrator is created early on to prove concepts, patterns and indicative performanceToo many solutions are found to perform badly in the last few weeks for any number of reasons
Highlights sub-optimal design up-frontIdentifies where the bottlenecks are likely to beIncreases confidence
We sometimes hear...
“it doesn’t scale”“it doesn’t meet our performance requirements”“we were told it would do XX msgs/sec”BizTalk is not
A universal panaceaOptimised for every customer scenario out-of-the-boxAble to make bad code, good code
BizTalk is part of the overall end-to-end solution
A great “case study”
“ [BizUnit,LoadGen,MSTest] provides the ability to reliably and automatically execute hundreds of test cases that otherwise would have to be performed manually”
“improved turn around times by a factor of upwards of 5. For a test pack containing 100 test cases which took up to 3 days to complete manually we can now execute in approx 3 hrs”
Laying the foundations
Critical to achieving representative performance within your environment
Benchmark your environmentFile Adapter -> File Adapter (3K) message should get ~500msgs/sec
Don’t run perf tests on workstation hardwareBeware of virtual machinesInvest in SQL Server Hardware and DisksBeware of UNC FILE Adapter locationsMessage TrackingRemember Virus Scanners
Foundation Checklist Databases and Logs separated onto separate drive letters and
therefore separate physical hard drives? Are the drives used for Databases and Logs configured on a
fast disk-array and with RAID10/0+1? Next generation SAN technology tends to use huge
caches so RAID choice is becoming unimportant Default Database Autogrowth settings changes or AutoGrowth
disabled? SQL Server Agent Running? Have you created separate Receive, Orchestration and Send
Hosts and moved the appropriate Adapters and Orchestrations to the appropriate host?
Concurrent Web Service Calls configuration file setting changed?
<system.net> <connectionManagement> <add address="*” maxconnection="80"/> </connectionManagement></system.net>
PerfMon Counters..... ??
A huge selection of performance counters are made available by the BizTalk engineIt can be time consuming to identify the counters you need especially when you have multiple machinesCollection of counters is key
Perfmon Countersdemo
Excel spreadsheet to define counter collection for multiple machines
Visual Studio Profiler
New with Visual Studio Team DeveloperIdeal for profiling custom code: custom pipeline components, .NET class librariesProfiler can be installed standalone for servers..
vs_profiler.exe located in the \vs\wcu\profiler directory of the VSTS media
Must be driven via the command line to attach to a service
Configuring the VS ProfilerVSPerfClrEnv /globalsampleon
VSPerfCmd /start:sample /output:“c:\dir\profileroutput"
VSPerfCmd /attach:<PID>VSPerfCmd /detach:<PID>VSPerfCmd /shutdown
VSPerfClrEnv /globaloff
Code Coverage
Enables you to understand what code has been executed during your test run
Identifies holes in your testingIdeal for custom pipeline components, helper assemblies, web services, etc.vsinstr –coverage “YOURASSEMBLY”
vsperfmon –coverage –output “filename.coverage”
vsperfcmd /shutdown
How do I test this?
BAMPrimaryImport
BizTalk: DarrenJ Travel Services
Hotel Orchestratio
n
Flight Orchestratio
n
Hire Car Orchestratio
nTravel
Insurance Orchestratio
n
OrchestrationEventStream
Wiley Flight Services Booking
Web Service
DirectEventStream
Itinerary Message[ASMX]
Itinerary Orchestratio
n MSMQ
BizUnitTest Cases are constructed from generic and reusable test stepsTest cases described using an XML syntax
Driven from NUNIT or MSTestModel every “use case” within your solution
Run a testIf you get a “green light” your solution functions at a requirements levelIncredibly powerful, not just for the development lifecycle but for changes, new releases.....
Not just about BizTalk
Test Steps available “in box”BAMDeploymentStepCheckPop3MailStepContextManipulatorStepCrossReferenceData-
VerificationStepDatabaseDeleteStepDatabaseRowCountStepDBQueryStepDelayStepDotNetObjectInvokerStepEventLogCheckStepEventLogClearStepExecuteCommandStepFileCreateStepFileMultiValidateStepFilesExistStepFilesMoveStepFileDeleteMultipleStep
FileDeleteStepFileMoveStepFileValidateStepHttpPostStepHttpRequestResponseStepMQSeriesClearQueueStepMQSeriesGetStepMQSeriesPutStepMSMQCreateQueueStepMSMQDeleteQueueStepMSMQQueuePurgeStepMSMQReadStepMSMQWriteStepOrchestrationConductorStepReceiveLocationEnabledSte
pReceivePortConductorStep
RenameDirectoryStepRuleEngineStepSMTPReadStepSOAPHTTPRequest-
ResponseStepWaitOnFileStep OutlookReadStep
Your own.....
BizUnit Test Case Format
SetupInitialise the platform ready to perform the test
ExecutionThe execution of the test
CleanupRestore the platform to its state prior to test execution
TestSetup Stage
TestCleanup Stage
TestExecution Stage
Test Step: MSMQWriteStep
Test Step: MSMQReadStep
Test Step: DBQueryStep
Validation Step: XmlValidationStep
Test Step: ExecuteCommandStep
Test Step: MSMQQueuePurgeStep
Invoke a web service
<TestStep assemblyPath="" typeName="SOAPHTTPRequestResponseStep">
<WebServiceWSDLURL>http://..</WebServiceWSDLURL> <ServiceName>MyWebServiceName</ServiceName> <WebMethod>SubmitItinerary</WebMethod> <InputMessageTypeName>Itinerary</InputMessageTypeName> <MessagePayload>ItineraryMessage.xml</MessagePayload>
<ContextLoaderStep assemblyPath="" typeName="XmlContextLoader">
<XPath contextKey="itineraryReference"> //ItineraryReceived/attribute::ItineraryReference </XPath> </ContextLoaderStep></TestStep>
Validate a MSMQ message
<TestStep assemblyPath="" typeName=“MSMQReadStep">
<QueuePath>.\Private$\HotelQ</QueuePath> <Timeout>10000</Timeout>
<ValidationStep assemblyPath="“ typeName=“BinaryValidationStep"> <ComparisonDataPath>
HotelFragment.xml </ComparisonDataPath> <CompareAsUTF8>true</CompareAsUTF8> </ValidationStep>
</TestStep>
Validate database rows<TestStep assemblyPath="" typeName=“DBQueryStep"> <DelayBeforeCheck>25</DelayBeforeCheck> <ConnectionString>...</ConnectionString> <SQLQuery> <RawSQLQuery> SELECT * FROM dbo.bam_Itinerary_Completed WHERE ActivityID='{0}‘ </RawSQLQuery> <SQLQueryParams> <SQLQueryParam takeFromCtx="itineraryReference“/> </SQLQueryParams> </SQLQuery> <Rows><Columns>
<CustomerName>Darren Jefford</CustomerName> </Columns></Rows></TestStep>
BizUnitdemo
Writing a test for a end-to-end scenario
Orchestration Profiler
Code Coverage tools don’t really help with OrchestrationsHow do you understand where time is being spend inside Orchestrations?
Enabling tracking and using HAT is the only real way right now
No way to understand which Orchestrations are not being fully tested
Edge ScenariosError paths
Orchestration Profilerdemo
“Code Coverage” your orchestrations
Identify slow performing shapes and orchestrations
LoadGenBizTalk Load Generation Tool
New 2007 release provides WCF supportGenerate messages to test how servers handle normal and extreme workload conditionsData GenerationLoad Throttling
Find the maximum sustainable rate of your rig, automatically.
Multithreaded“Throw away all those custom test harnesses”, please...
“in-box” Load Generators
FILE Outputs files in a destination folderHTTP Supports both asynchronous and request-response messaging.SOAP Supports both asynchronous and request-response messagingMSMQ Supports both asynchronous and request-response messagingMQSeries Message can be local or remote, in-order or unordered, persistent or
non-persistent.WSE Supports both asynchronous and request-response messaging.WSS Windows® SharePoint® Services.Custom Base your own on the FileTransport sample
In-box Message Creators
CustomMCCreate new messages based on a templateReplace Items within the message with differing data
SizeMCExplodes a sample message to a certain size (big batch files)
In-box Throttle Monitors
SQLSpoolEnsure the Spool depth remains within a certain range
Rate100 msgs/sec..
CPUKeep the BizTalk servers within 80-90% CPU utilisation
PerfMonMonitor any PerfMon counter
LoadGendemo
Generating load into your BizTalk solutions
LoadGen and BizUnit
LoadGenFantastic load generation tool
Generate increasing loadFind the maximum sustainable point of your solution
BizUnitExtensive library of Test Steps
Collecting PerfMon and EventViewer LogsStoring Test InformationValidating Test Results
BizUnit can drive LoadGenAutomate all of your performance testing end-to-end!!
LoadGen and BizUnitdemo
Driving test cases using LoadGen and BizUnit
•Shameless Plug Further ReadingIn depth chapters covering Testing,
Performance & Scalability..
Summary
Get the environment right before any testingStart testing early, and often. A “demonstrator” is key for complex scenarios Use BizUnit for modelling “use cases”Use LoadGen for generating loadUse Orchestration Profiler to ensure your “touching” all parts of your solution during testingUse BizUnit and LoadGen together to automate your tests
Tools
PerfMon Counter spreadsheetOrchestration ProfilerBizTalk DocumenterLoadGen 2007BizUnit
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after
the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.