quoc buidownload.microsoft.com/documents/hk/technet/techdays2009/... · 2018-12-05 · optimization...

Quoc BuiSenior Program ManagerMicrosoft, Customer Advisory Team (CAT)Session Code: DEV 319

Session Objectives

Session Objective(s): Understand architectural, design, and development patterns to improve BizTalk Server performance

Provide information on the tools, techniques and processes used by the Rangers to run effective BizTalk performance labs.

Answer BizTalk performance related questions.

Microsoft Confidential

Agenda


BizTalk Rangers Performance LabResults for BizTalk Server 2009 Scale Out Lab

Coding PatternsDEV313 (15:15) Today

Optimization Highlights & Drill Down

Methodology

Performance limitations should be viewed as a series of obstacles: removing one bottleneck usually is not enough to boost your performance, you have to remove all of them.

Make an inventory of your system configuration before applying any change to the platform.

Use a bottom-up approach when tuning your platform.

After every modification you should retest your application since a change in one part of your solution could lead to problems or bottlenecks in another component/subsystem.

Performance Scale Out Goals Provide sizing and scaling guidance for all BizTalk:

Quantify scale out capabilities of BizTalk 1,2,4 servers

Quantify scale out capabilities of MsgBox 1,3 scenarios

Document this scenario in the BizTalk 2009 Performance Guide

Provide marketing with compete collateral

Understand how fast BizTalk can go on current hardware

Scenario Summaries

Messaging

Orchestration

Scale out BizTalk (1-4)

Scale out MessageBox (1,3)

Scenario 1:Messaging Message Flow

1. Loadgen generates a new XML message and sends it over NetTCP

2. A WCF-NetTCP Receive Location receives a the xml document from Loadgen.

3. The PassThruReceive pipeline performs no processing and the message is published by the EPM to the MessageBox.

4. The WCF One-Way Send Port, which is the only subscriber to the message, retrieves the message from the MessageBox

5. The PassThruTransmit pipeline provides no additional processing

6. The message is delivered to the back end WCF service by the WCF NetTCP adapter

Scenario 2:Orchestration Message Flow

1. Loadgen generates a new XML message and sends it over NetTCP

2. A WCF-NetTCP Receive Location receives a the xml document from Loadgen.

3. The XMLReceive pipeline performs no processing and the message is published by the EPM to the MessageBox.

4. The message is delivered to a simple Orchestration which consists of a receive location and a send port

5. The WCF One-Way Send Port, which is the only subscriber to the Orchestration message, retrieves the message from the MessageBox

6. The PassThruTransmit pipeline provides no additional processing

7. The message is delivered to the back end WCF service by the WCF NetTCP adapter

XMLReceive

Receive Port

WCF One-Way Receive Location

BizTalkMsgBoxDb

Messages

PassThruTransmit

WCF One-Way Send Port

Load Generating

Application Back End WCF Service

BizTalk Server 2009

XML

Message

(2 KB) WCF

Adapter

XML

Message

(2 KB)WCF

Adapter

BizTalk In Process 64 Bit Host: RxHost BizTalk In Process 64 Bit Host: TxHost

Orchestration

BizTalk In Process 64 Bit Host: PxHost

1 2 3 5 6 7

4

0200400600800

1000120014001600180020002200

BizTalk: 1BTS(1 RxHost, 1TxHost) SQL:

1 SQL


3 SQL


3 SQL


3 SQL

90.9 73.8 66.9 46.3

1088.3

1459.2

2102.6 2069.6

BizTalk Key Performance Indicators

AvgProcessor

Utilization(across allBTS boxes)

Throughput/ Documents

processedper second

Results Scenario 1: Messaging Single and Multi Message Box

WCF-NetTcp Recv + WCF-Nettcp Send Doc Processed/Sec BTS CPU AVG SQL CPU AVG

BizTalk: 1BTS (1 RxHost, 1 TxHost) SQL: 1 SQL

1088.390.9 39.5

BizTalk: 2 BTS SQL: 3 SQL

1459.273.8 24.9

BizTalk: 3 BTSSQL: 3 SQL

2102.666.9 36.5

BizTalk: 4BTSSQL: 3SQL

2069.646.3 36.4

0.0

500.0

1000.0

1500.0

2000.0

2500.0

BizTalk:1BTS (1

RxHost, 1TxHost) SQL:

1 SQL

BizTalk:2BTS (1


3 SQL

BizTalk:3BTS (1


3 SQL

BizTalk:4BTS (1


3 SQL

39.5 24.9 36.5 36.4

1088.3

1459.2

2102.6 2069.6

SQL Key Performance Indicators

SQL ProcessorUtilization %

Throughput /Documents

processed persecond

WCF-NetTcp Recv + Orch + WCF-Nettcp SendDoc Processed/Sec

BTS CPU AVG SQL CPU AVG

BizTalk: 1BTS (1 RxHost, 1 PxHosts, 1 TxHost)SQL: 1SQL

433.25 91.2 37.8

BizTalk: 2 BTS (1 RxHost, 1 PxHosts, 1 TxHost) SQL: 1SQL

675.88 74.6 60.3

BizTalk: 3 BTS (1 RxHost, 1 PxHosts, 1 TxHost) SQL: 1 SQL 645.31 28.8 49.3

BizTalk: 4 BTS (1 RxHost, 2 PxHosts, 1 TxHost) SQL: 1 SQL

561.63 22.8 60.2

Results Scenario 2: Orchestration Single Message Box

0.0

100.0

200.0

300.0

400.0

500.0

600.0

700.0

BizTalk:1BTS (1


1 SQL

BizTalk:2BTS (1


1 SQL

BizTalk:3BTS (1


1 SQL

BizTalk:4BTS (1


1 SQL

37.8 60.3 49.3 60.2

433.3

675.9645.3

561.6




processed persecond

0

100

200

300

400

500

600

700


1 SQL


1 SQL


1 SQL


1 SQL

91.2 74.628.8 22.8

433.3

675.9645.3

561.6


AvgProcessor


Throughput/ Documents

processedper second

Results Scenario 2: Orchestration Three Message BoxWCF-NetTcp Recv + Orch + WCF-Nettcp Send Doc Processed/Sec BTS CPU AVG SQL CPU AVG


581.5 78.3 27.3


829.81 52.8 44.0


1005.13 66.3 52.6

0

200

400

600

800

1000

BizTalk: 2BTS (1RxHost, 1

TxHost) SQL: 3SQL


TxHost) SQL: 3SQL


TxHost) SQL: 3SQL

78.3 52.8 66.3

581.5

829.81005.1


AvgProcessor


0.0

200.0

400.0

600.0

800.0

1000.0


TxHost) SQL: 3SQL


TxHost) SQL: 3SQL


TxHost) SQL: 3SQL

27.3 44.0 52.6

581.5

829.81005.1




processed persecond

Lab Hardware Kit - Overview

Detailed SAN Configuration

SAN LUN Configuration

CAT Team LUN configuration

Distribute MessageBox artifacts using automated script (shipped in BTS09 Perf Guide)

Server Name Drive Letter Volume Name Files Lun Size (GB) Cluster Size

Tap4x-O01/Tap4x-

O05/Tap4x-M01

F Data_Sys MASTER, and MSDB

data files

10 64Kb

G Logs_Sys MASTER, and MSDB

transaction log files

10 64KB

H Data_TempDb TempDB Data 50 64Kb

I Logs_TempDb TempDB Log 50 64KB

J Data_BtsMsgBox BizTalkMsgBoxDb data file

100 64Kb

K Logs_BtsMsgBox BiztalkMsgBoxDB log file

100 64KB

L Data_Spare MessageBox Files 10 64Kb

M Data_BizTalkDataBases Other BizTalk Database Files

20 64Kb

N Logs_BizTalkDatabases Other BizTalk Database Log Files

20 64KB

P Misc Misc 20 64KB

Optimizing LUNs for Capacity versus Performance

Capacity OptimizedOptimized for disk capacity

Pro: Efficient use of capacity

Con: LUN’s compete for I/O

Typical SAN configuration

Performance OptimizedOptimized for performance

Pro: Dedicated I/O of spindles to the LUN

Con: Less efficient with capacity

LUN1 LUN2 LUN3 LUN4I/O’s per second 200 200 200 I/O’s per second 600Capacity in GB’s 500 500 500 Capacity in GB’s 1500

Example: Example:

Agenda





Architectural Design Decisions

Architecturally elegantLoosely coupled

Itinerary processing

Performance-orientedLow-latency

High throughput

Trade-offs to meet both requirements

PatternsWhat is this workflow trying to do?

What’s wrong with it?

Child Orchestration gets called

Message gets submitted to LOB

Loopwhile Finish=false

Message is received from LOB

Finish=true

Finish=false

Sequential Ordered Message Delivery

Sequential Ordered Message Delivery

(Improving Ordered Message Delivery in BizTalk Server During Concurrent Processing)

http://msdn.microsoft.com/en-us/library/bb851740.aspx




Architecturally Elegant

Performance-oriented

Tightly-coupledOrchestrations call each other using Call shape

Inline sends

Debatching inbound in Pipelines

Allows parallel processing of individual messages

Prevents looping in orchestrations, which can be problematic

Can reduce memory consumption

Pipeline components

Stream-based approach to pipeline componentsProcessing in the stream implementation instead of the Execute method

Take advantage of BizTalk’s pipeline componentsVirtualStream and readOnlySeekableStream

Also in source-code in the BizTalk SDK

XPathMutatorStream

NamespaceTranslatorStream and XmlTranslatorStream

VirtualStream with ReadOnlySeekableStream

int bufferSize = 0x280;

int thresholdSize = 0x100000;

Stream vStream = new VirtualStream(bufferSize, thresholdSize);

Stream seekStream = new ReadOnlySeekableStream(inboundStream, vStream, bufferSize);

Orchestrations

Avoid loading messages in XmlDocument

Use XLANGMessagepublic void ProcessMessage(XLANGMessage message)

{

try

{

using (XmlReader reader = message[0].RetrieveAs(typeof(XmlReader)) as XmlReader)

if (reader != null)

{

...

}

}

finally

{

message.Dispose();

}

}

BizTalk Design Patterns

Enterprise Service Bus

Integrating the Health Enterprise

Parallel Convoy

Low Latency

Message sequence delivery

Async Aggregation

Inverse Direct Bound Port

First In First Out

Splitter

Interrupter Pattern

Terminate

Retry Pattern

Filter

Sequential Convoy Aggregator

Most of these are available on codeplex.com!

Agenda





Adjusted Windows Server 2008 for best performance

Disabled SynAttackProtect (Denial of Service checking)

Placed the PAGEFILE on a separate local disk

Changed the Speed & Duplex property of each NIC from “Auto” to 1 Gigabit Full Duplex

Platform Optimization Highlights

Stopped and disabled unused windows services on all BizTalk Server and SQL Server nodes

Disabled the debug flag in the web.configconfiguration files

Increased the maxconnection settings from the default value of 2 to 50 on BizTalk and Front-End web farm

Some settings did not improve performance on Windows Server 2008 which has a self tuning network stack:

Modified or created several registry entries regarding TCP/IP, Network, Session Manager, LanManServer, FileSystem settings.

Platform Optimization Highlights

Before baseline:

Installed CU package 9 and installed Performance Dashboard

Pre-allocate space (size) for all data / log files

Fixed autogrowth value to a fixed amount

Granted Windows ‘Lock Pages In Memory’ and ‘Perform Volume Maintenance Task’

Standard SQL optimizations:

Moved Data and Log files to separate LUNs

Output all Log/Full backups to dedicated LUN

Split TempDB into multiple equal sized files

SQL Optimization Highlights

Relieving contention on MessageBox:

R2 BizTalk stored procedures - Hotfix KB944234

Spread tables and indexes of the BizTalkMsgBoxDbacross multiple filegroups / multiple LUNs

Changed schedule of BizTalk ‘Mark Log’ backup job from 15 to 5 minutes

Scaling out:

Created dedicated SQL Server instance for BizTalkMsgBoxDb

Moved BizTalkMsgBoxDb to a dedicated Server

SQL Optimization Highlights

R2 64 bit BizTalk hosts take ~1 minute to restart Installed KB943165

Separated hosts (including dedicated tracking host)

Disabled Message Delivery throttling

Disabled global tracking as in the production site.

BizTalk Optimization Highlights

Reduced the MaxReceiveInterval to 50 ms (default 500 ms) in the adm_ServiceClass table

Increased the Internal Message Queue Size from the default value 100 to 1000 for all the BizTalk Hosts

Disabled orchestration tracking events

Disabled BAM

BizTalk Optimization Highlights

Increased the .NET CLR worker and IO threads for all the in-process hosts

The Microsoft Distributed Transaction Coordinator (MS DTC) log file can become a disk I/O bottleneck in heavy-transaction environments.

This is especially true when using transactional adapters (SQL, MSMQ, MQSeries) or in a multi-MessageBox environment.

BizTalk HTTP connections increased from 2 to 50

Minimize Persistence pointsOrchestration instance serialization includes all messages and variables instantiated in the orchestration.

Code Optimization Highlights

Avoid using XMLDocument object model

Serialize XSD into .NET classesXSD tool (.NET Framework 2.0)

SVCUTIL (.NET Framework 3.0)

Avoid using orchestration based long running transactions for due dates

Use SQL Server Service Broker, Query Activation Services

http://msdn.microsoft.com/en-us/library/t9x04ed2.aspx

Code Optimization Highlights




WCF-* AdaptersWhenever possible, use one of the following WCF-* Adapters.

WCF-NetTcp Adapter

WCF-Custom Adapter + CustomBinding with the binaryMessageEncoder and tcpTransport Binding Elements

Add the serviceThrottling service behavior to a WCF-* Receive Location and use the following settings for a

Parameter Default Suggested

tcpTransport->ConnectionPoolSettings->maxOutboundConnections 10 try changing to 20

tcpTransport->maxPendingAccepts 1 try changing to 20

tcpTransport->maxPendingConnections 10 try changing to 20

tcpTransport->listenBacklog 10 try changing to 20


tcpTransport->maxPendingAccepts 1 try changing to 20

tcpTransport->maxPendingConnections 10 try changing to 200

tcpTransport->listenBacklog 10 try changing to 20

Send Side (BizTalk Send Port):

Receive Side (BizTalk Receive Location):


serviceThrottling->maxConcurrentCalls 16 try changing to 200

serviceThrottling->MaxConcurrentSessions 10 try changing to 20

MSDN guides

BizTalk Server Performance Optimization Guidehttp://msdn.microsoft.com/en-us/library/ee377064(BTS.10).aspx

BizTalk Server Operations Guidehttp://msdn.microsoft.com/en-us/library/ee308947(BTS.10).aspx

BizTalk Server Database Optimizationhttp://msdn.microsoft.com/en-us/library/bb743398.aspx

//www.microsoft.com/biztalk/evaluation

//www.microsoft.com/technet/prodtechnol/biztalk

//microsoft.com/technet/community/newsgroups

//gotdotnet.com/team/wsservers

//msdn.microsoft.com/virtuallabs/biztalk and//www.microsoft.com/biztalk/techinfo/virtual-labs.mspx

//msdn.microsoft.com/biztalkDeveloper Center

Virtual Labs

Community

Newsgroups

Tech Center

Product Information

BizTalk Server Resources

//www.microsoft.com/biztalkProduct Website

© 2009 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS,

IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.