integration approach for mes
Post on 06-May-2015
340 Views
Preview:
DESCRIPTION
TRANSCRIPT
Table of Contents1. Asset Bounds....................................................................................................................................3
2. Asset Introduction...........................................................................................................................3
3. Advantages of BizTalk..........................................................................................................................4
4. Integration Process..............................................................................................................................5
4.1 File or Web Service......................................................................................................................6
5. Adapter and Integration Solutions.......................................................................................................7
6. BIZTALK Best Practices.........................................................................................................................8
a. Design Best Practices.......................................................................................................................8
b. Schema Creation Best Practices.......................................................................................................9
c. Map Creation Best Practices..........................................................................................................10
d. Orchestration Best Practices..........................................................................................................11
e. Best Practices for Performance.....................................................................................................15
1
Integration Reusable Assets - Systems Integration Approach for MES
Date 27/1/2014
Owner Vinod kumar kodatham
Description This approach will help to change the traditional approach of point-to-point communication in Manufacturing Execution Systems (MES) to using BizTalk server as a middleware to Integrate several systems. Multiple channels of several systems can be plugged into the centralized BizTalk server. Enterprise Service Bus (BizTalk server) with its inbuilt BAM and ESB capabilities will help especially for reporting Application tracking and application failures in respective BAM and ESB portals respectively. These are totally reusable and isolated which can be used by any application across the industry which uses BizTalk as the middle ware platform
OBJECT OVERVIEW
Technical NameIntegration Approach for MES.
Description
The asset will provide high level design considerations which enable seam less integration cross platform and systems using BizTalk framework which developers can modify as per the requirement.
2
1. Asset Bounds
This asset can be used for the several simple to complex Integration scenarios that needs to scale from low to high volumes with low latency and need of fault tolerance that has got redundancy, scalable and message retry features and governance over the solution. The conceptual architecture of a solution is done by even taking into account non-functional requirements like performance, availability, extensibility, resilience, robustness, and scalability and ensure they will be properly addressed during the technical design phase. This gives you out of the box features to develop/customize all these functionalities. The asset is independent of Source and Target System(s)/ Application.
2. Asset Introduction
This document describes the idea of systems Integration using a centralized BizTalk application i.e. using BizTalk framework as an Integration platform for integrating systems (process in MES System) like POM, RIM, IOM, RCM, QOM, SAP and PCS (Process Control Systems). BizTalk framework orchestrates the integration process by maintaining the performance, robustness and availability. This framework will describe the design decision made and development task required to build the BizTalk application’s components.
Any Business scenario which operates with integration has the above typical interface.
1. Business has a need to transfer information from one application to a different application or one system to a different system that are residing in a single or different multiple servers. These applications can be either in a same server or a different servers
2. These applications can be either an application server which uses Webservices or a database server. Now this source data needs to be migrated to the target system meeting the business requirements which is called as transformations.
3. In order to integrate these two different systems so that they can communicate each other business requires a Middleware i.e. BizTalk server.
There are lots of Middleware technologies available in market but if you need to go for the best then we say its BizTalk server as it is one of the most successful product in the market. It’s a Microsoft product. The benefits of this tool are mention in the very next section.
3
3. Advantages of BizTalkFigure 2: Orchestrated Data transfer Process
While planning for the integration scenario BizTalk takes care about tasks like Gathering information, Defining naming conventions, planning team development, setting up and working with source control.
Advantages of using BizTalk are.
1. Recoverable Interchange. In BizTalk, an Interchange can contain two or more messages, such as a batch. Recoverable Interchange was introduced in BizTalk Server 2006, only messages will be suspended if they fail validation, and when corrected they can be resumed after the error.
2. Failed message routing. This functionality is available to enable failed messages to be subscribed by orchestration and send ports. When used appropriately, failed message routing will be used to notify users about the failed messages or to handle error properly and repairing the messages if required.
3. Lowest total cost of ownership (TCO). BizTalk Server reduces the cost and complexity of managing business processes and its automation with a single, unified solution for Enterprise Application Integration (EAI), Business-to-Business integration (B2Bi), and Business Process Management (BPM).
4. BizTalk Integrated management and development tools will enhance productivity.5. Using TAPI it also supports Computer Telephony Interface (CTI).6. Powerful, familiar tools to design, develop, deploy, will enable to manage those
processes easily
4
4. Integration Process
Figure 3: A typical Integration Scenario
In this scenario, information will be transferred from multiple source systems like Enterprise Resource Planning (ERP), Line of Business (LOB) systems or any other third party vendor systems to Target system (ERP, LOB or a third party vendor). Messages inside BizTalk are through the XML documents and defined with the XML schemas in XSD standard. Maps are implemented with the XSLT standard. Orchestrations or mappings are implemented to meet the business needs while transferring data from source to target. Schemas, maps, pipelines and orchestrations are created visually using graphical tools within Microsoft Visual Studio. The additional functionality can be delivered by .NET assemblies that can be called from existing modules—including, for instance, orchestrations, maps, pipelines, business rules.
BizTalk allows creating 'send' and 'receive' port through which external applications can exchange messages .The ports can be configured to use a set of transport protocols and application adapters that are shipped with BizTalk Server. With these adapters, BizTalk can exchange messages over multiple channels such as files, MSMQ, HTTP (Hypertext Transfer Protocol), SOAP, IBM MQ, etc. Application adapters, which can directly interact with packaged applications like SAP, Oracle Apps, etc., are also available.
5
4.1 File or Web Service
Data is retrieved from a third party web service or a file based where the business rules are applied in and will transform the data to the target system. There are different ways to expose and consume Webservices from BizTalk Server.
BizTalk can understand both positional/delimited and XML message formats. Biztalk can transform messages from one format to another.
Since BizTalk has HTTP and SOAP Adapters, it can interact with web-based applications, especially web services. The BizTalk Server interacts with services in the following two ways:
1. BizTalk process orchestrations can be exposed as Web services. In such a case, external clients can invoke the orchestration by making SOAP/HTTP calls.
2. BizTalk can consume Web services by using the SOAP Adapter.
6
5. Adapter and Integration Solutions
Adapter for BizTalk Server seamlessly integrates BizTalk with enterprise databases, applications, transactions, business packages, EDI, and more.
Adapters allow connecting Microsoft BizTalk server to Enterprise Information Systems, including homegrown applications, databases like DB2, legacy data stores, transaction systems, e-business formats such as EDI, HIPAA and HL7, and packaged applications.
The Adapter Offering for BizTalk Server:
1. Provides easy administration and configuration through existing BizTalk Server Administration Console
2. Complete, Proven, High-Performance Adapter Set3. Relational and legacy database systems4. Transaction processors5. Custom applications6. Object models like COM objects, Enterprise JavaBeans, and others7. Messaging environments like IBM MQSeries,Oracle AQ,TIBCO EMS Q8. Terminal-based systems like mainframes and AS/400s9. Packaged applications such as Enterprise Resource Planning (ERP) systems, Supply
Chain Management (SCM) systems, and Customer Relationship Management (CRM) systems
10. Electronic Data Interchange (EDI) systems11. e-Business XML formats such as SWIFT, FIX, HIPAA, ebXML, BOD, and cXML
7
Each adapter masks the complexities of its underlying source, so developers can use industry-standard XML, SQL, or Stored Procedures to work with all types of information. Also, each adapter is highly optimized for its target, so developers don't require specialized knowledge to take full advantage of special features and syntax for unusual data structures, applications, and transactions. All adapters provide standard metadata about available functions and fields, return values, error codes, and more.
6. BIZTALK Best Practices
Some of the BizTalk design best practices to be followed are.
a. Design Best PracticesCategory Technique Reference
Design Group the projects as per artifacts type.
It is a best to group the
projects as per the kind of
artifacts it contains. Schemas
are broken into Internal, plus
either External, Import and
Export, or even separated by
adapter: SQL, Oracle,
etc. Orchestrations or
Processes should be in one
project. Avoid using
Transforms in Orchestrations.
Put maps in the Ports.
Pipelines, if you need them, in
a separate project.
Maps or Transforms in a
project.
Design Promote only those properties that are required for routing or tracking. Use distinguished property for expression shape usage.
Performance
More processing required to find the promoted properties from the message. These properties are loaded in the memory, hence having a lot of them can put the system under extreme loads. Also, they are
8
written to the database for tracking taking up disk space and more CPU cycles.
b. Schema Creation Best Practices
Category Technique Reference
Schemas Always Use Separate Internal and External Schemas
Always transform messages
into canonical schema—
regardless of how simple the
schema is or who the source
is. This can reduce the number
of maps you need. Suppose
you need to map three types of
inbound messages to three
types of outbound messages
(perhaps not today, but maybe
in the future). Applying this
technique, you create a map to
your canonical schema for
each inbound schema and then
a map from your canonical
schema to each outbound
schema. That’s only six maps
to build and maintain, not
nine.
Schemas Always validate xml message against the standard schema.
N/A
9
c. Map Creation Best Practices
Category Technique Reference
Transformation Use Transformation on Receive and Send Ports when you have multiple partners sending multiple message formats and you need to convert these messages to a canonical format.
Business reason. Embed the map in the schedule and the partner changes the format, not only do you have to rebuild the map, you have to rebuild the schedule to use the new version of the map.
Transformation Use transformation in your orchestration schedule When you need to generate a new message in the schedule and use the modified (mapped) contents of an existing message as the base. When you want to map multiple parts of a message into one outbound message (this type of mapping cannot be done in receive / send port). There are performance gains which come from doing mappings in receive ports sometimes, but they are mostly around how many persisted messages your scenario generates and it is a bit complicated to explain.
Performance.
Transformation Pure XSLT is much faster than XSLT which uses inline script or referenced assemblies.
Performance
Use xslt as much as possible in
mapping.
Transformation Use external XSLT or Maintainability.
10
serializable classes in case of complex maps with 1000+ connections.
Transformation Do not use .NET objects directly inside the map. Use external assemblies instead.
Maintainability and Performance.
Transformation To minimize memory consumption, we recommend that you put any map that uses these functoids into a small assembly. Then, reference that assembly.
Performance.
The
System.Policy.Security.Evidence
object is often used in transforms
and can consume a lot of
memory. Whenever a map
contains a scripting functoid that
uses inline C# (or any other inline
language), the assembly is created
in memory. The
System.Policy.Security.Evidence
object uses the object of the actual
calling assembly. This situation
creates a rooted object that is not
deleted until the BizTalk service
is restarted.
d. Orchestration Best Practices
Category Technique Reference
Orchestration Eliminate orchestrations for messaging only patterns
When possible minimize the use of orchestrations to increase overall throughput and reduce the latency of business processes. If there is
11
no need to run long running transactions and there is no need to invoke several systems for each request, then consider eliminating orchestrations and moving business logic to Receive and Send Ports to reduce the total amount of roundtrips to the BizTalkMsgBoxDb and decrease the latency due to database access. In this case, implement custom pipelines and reuse helper classes that were previously invoked from orchestrations. Use orchestrations only when strictly needed to implement design patterns as Scatter and Gather or Convoys.
Orchestration Always Use Multi-Part Message Types
If your logical port and Receive/Send shape are using your multi-part message type, and you can easily change the underlying schema in the future without having to disconnect the port from the receive shape.
Orchestration Optimize orchestration latency by reducing the number of persistence points.
Best practice
Orchestration Minimize orchestration complexity to improve performance
Best practice
Orchestration Use the Call Orchestration shape versus the Start Orchestration shape to improve performance
Best practice
Orchestration Use XmlReader with XLANGMessage versus using XmlReader with
Performance
12
XmlDocument to improve orchestration performance
Orchestration minimizing the use of logical ports bound to physical ports
Performance
Orchestration to extract or set properties used with business logic in an orchestration
If you are using a map to extract or set properties used with business logic in an orchestration, use distinguished fields or promoted properties. This practice should be followed because when extracting or setting values with a map the document is loaded into memory however when using distinguished fields or promoted properties, the orchestration engine accesses the message context and does not load the document into memory.
Orchestration aggregate several fields into one field
If you are using a map to aggregate several fields into one field, use distinguished fields or promoted properties with an orchestration variable to accumulate the result set.
Orchestration Use Orchestration Design Patterns when required
Performance and maintainability
Orchestration Make appropriate use of .NET classes in orchestrations to maximize performance
Performance and maintainability
Orchestration Orchestration Exception Handler Blocks
When using Orchestration exception Handler blocks, include all orchestration shapes in one or multiple scopes (non transactional scopes whenever possible) and create at least the following exception handler blocks:
13
An exception handler block for handling a generic System.Exception error.An exception handler block for handling a general exception in order to catch and handle possible unmanaged errors, such as COM exceptions.
Orchestration Always Try to Design Orchestrations with Direct-Bound Ports
You’ve got three Direct-
Bound port types: Message
Box Routing, Self-Correlating,
and Orchestration-to-
Orchestration (also called
Partner Ports). Direct-Bound
ports are self-configured,
without sacrificing flexibility
or performance. A handy
benefit of Direct-Bound ports
is that neither the developer
nor the administrator needs to
create corresponding physical
ports in BizTalk Explorer or
BizTalk Administration
Console. So Direct-Bound
ports yield orchestrations that
are more self-contained, and
therefore more reusable and
easier to redeploy
independently.
Orchestration Publish Your External Schema, Not Your Orchestration
It provides loosely coupled
mechanism for exposing the
services. This buys you more
freedom to change your
orchestration without breaking
14
the caller
e. Best Practices for Performance
Category Technique Reference
Performance Optimize the BizTalk Registry for Web Services.
If your BizTalk orchestration is published as a Web service, you’ll definitely want to tweak some of the default ASP.NET parameters used by BizTalk. Below are the recommended settings.
Windows Registry Editor version 5.00
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$BTSHOST\CLR Hosting]
“MaxIOThreads”=dword:00000064
“MaxWorkerThreads”=dword:00000064
“MinIOThreads”=dword:00000019
“MinWorkerThreads”=dword:00000019
Please refer http://msdn2.microsoft.com/en-us/library/aa561380.aspx for more details.
Performance Configure Parameters Used for Low Latency Performance Tuning
Please refer below link for more information.
http://msdn.microsoft.com/en-us/library/aa475435.aspx
Performance Create separate hosts for orchestration, Receive and
Creating separate hosts allows BizTalk to implement throttle mechanism and
15
Send Ports help it to load balance the incoming messages.
High Availability Cluster the resources for high availability.
Clustering of MSDTC, SSO, Disk and BizTalk for High availability.
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=eb437722-2828-4cbb-84c3-17556b4df26b
High Availability Use database mirroring for BizTalk Server databases high availability.
Database mirroring of BizTalk databases helps to achieve the High Availability for Database servers and helps to recover from Failover scenarios.
Appendix
Reference used for Best Practices:
http://social.technet.microsoft.com/wiki/contents/articles/5210.biztalk-server-2010-orchestration-best-practices.aspx
16
top related