improving scalability with objectgrid partitions

8/14/2019 Improving scalability with ObjectGrid partitions

1/42

Processing complex business events withWebSphere eXtreme Scale, Part 2: Improvingscalability with ObjectGrid partitionsSkill Level: Intermediate

Kenji Kojima ([email protected])DeveloperIBM

12 Nov 2008

This two-part series shows how IBM WebSphere eXtreme Scale can be used toimprove performance and scalability when processing complex business events. Part1 introduces a traffic monitoring scenario and uses an ObjectGrid on a stand-aloneserver to show how the scenario works. eXtreme Scale is used to pass high-volumeraw events to a prefiltering application, which significantly reduces the processingload on WebSphere Business Events. Part 2 builds on this scenario and uses a

distributed Object Grid to partition the events based on the Business Events contextID. This introduces high availability and scalability, as events can be processed bymultiple Business Events instances.

Introduction

This article is the second in a series on how WebSphere Business Events (hereafterreferred to as Business Events) and WebSphere eXtreme Scale (hereafter referredto as eXtreme Scale) can be used to improve scalability and high availability in acomplex event processing solution. Part 1 used eXtreme Scale to preprocess a

speed calculation, filter high-volume raw events by a given threshold, and then passthis subset of raw events, called actionable events, to Business Events. Using thisdesign pattern can significantly increase the rate of event processing.

In part 1 Business Events was running as a single instance on a stand-aloneapplication server. eXtreme Scale was configured with a distributed ObjectGrid, butwith a single partition. Realistically, as your application and system grow, you need

Improving scalability with ObjectGrid partitions Copyright IBM Corporation 1994, 2008. All rights reserved. Page 1 of 42
mailto:[email protected]://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlmailto:[email protected]


2/42

both a highly available and a scalable operating environment. Business Events v6.1doesn't support the existing clustering and high availability features of WebSphereApplication Server (hereafter referred to as Application Server). However, we canachieve this goal by combining the existing features of Application Server, BusinessEvents and eXtreme Scale. This article shows you how to modify the application and

configuration used in part 1 to answer several questions from a scalability andhigh-availability point of view:

How do clustered application servers interact with distributed ObjectGridand multiple partitions?

How do multiple Business Events instances and eXtreme Scale serversconnect by using the Service Integration Bus (SIBus)?

How do we modify Business Events interaction sets and eXtreme Scaleapplications appropriate to the new system environment for part 2?

How do we install and configure multiple Business Events instances andconnectors run on multiple nodes?

The scenario described here was built using Red Hat Enterprise Linux ESrelease 4, but the artifacts supplied in Downloads can be used on either Windows orLinux. The following software levels were used:

IBM Rational Application Developer V7.0.0.7

WebSphere Business Events V6.1 Fix Pack 1 for Linux

IBM DB2 V9.5 Fix Pack 1 for Linux

WebSphere eXtreme Scale V6.1.0.3

WebSphere Application Server Network Deployment V6.1.0.5

Planning for scalability and high availability

An event is the representation of an activity in a system. The correlation of theactivity across disparate events is known as Complex Events Processing (CEP).Business Events is specifically designed to orchestrate CEP, which evaluates notonly individual events but also patterns of events at run time. This powerful feature

also requires careful planning when building a scalable solution with multipleBusiness Events instances.

Part 1 of this series describes the filter used to determine which event or actionpatterns have occurred. The filter can be a defined series of events; based on theresults of the filter, Business Events governs the interaction between events at runtime. For example, if the LowSpeedEvent occurs more than nine times within a

developerWorks ibm.com/developerWorks

Improving scalability with ObjectGrid partitionsPage 2 of 42 Copyright IBM Corporation 1994, 2008. All rights reserved.
http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtml


3/42

one-minute period, the condition of the MultipleLowSpeeds filter is met. The filteralso checks that there haven't been any occurrences of the LowSpeedAlert in thelast five minutes.

Figure 1. MultipleLowSpeeds filter

This type of a filter is considered a CEP filter because it evaluates whether an eventhas taken place in a given context. The MultipleLowSpeeds filter shown in Figure 1is used in the ManyLowSpeeds interaction set shown in Figure 2. An interaction setcan have only one event, which in our case is the LowSpeedEvent. Notice that theSpeedLoc.location is specified in the Related by field. This becomes a key thatBusiness Events uses to detect that the event pattern has occurred. This is called aBusiness Events context. (In reality, a Business Events context is a field reference

from an intermediate object, associated with the interaction set by the "Related by"relationship.)

Figure 2. ManyLowSpeeds interaction set

Because we defined the context relationship to be SpeedLoc.location, the interactionset and filters are evaluated on a per-location basis in our scenario. As you know,Business Events evaluates not only individual events but also flows of events in acontext. When configuring multiple Business Events instances for scalability, wemust ensure that events in the same context are always sent to the same BusinessEvents instance. Each subsequent event that is part of the context must beprocessed on the same Business Events instance with the affinity for that context.

Therefore, our solution must provide context-based event routing from EJB clients toBusiness Events instances.

To achieve EJB scalability we want to use the EJB clustering support of ApplicationServer. Therefore, we need to install and configure Business Events on the top ofApplication Server. eXtreme Scale supports both scalability and high availability forpartitions. Although we used a single ObjectGrid partition for the high-volume raw

ibm.com/developerWorks developerWorks



4/42

events store in part 1 (which was neither highly available nor scalable in itsconfiguration), we can gain a scalable and highly-available event store with somesimple eXtreme Scale configuration changes from a single partition to multiplepartitions. We use this partitioning support provided by eXtreme Scale to ensure thatall events with the same Business Events context are always routed to the same

Business Events instance. There is no need to buy any additional products ordevelop any new code.

Business Events v6.1 doesn't support high availability. There is a single point offailure (SPOF) with respect to the Business Events instance and its repositorydatabase. You can use a High Availability (HA) solution such as IBM HighAvailability Cluster Multiprocessing (HACMP) or Oracle Real Application Clusters(RAC) to eliminate SPOF for repository database, but this is outside the scope ofthis article.

The solution architectureThe previous section introduced considerations for developing a scalable and highlyavailable architecture. This section outlines the solution architecture that satisfiesthose considerations.

EJB clustering is used to provide scalability for our stateless session EJB.

ObjectGrid key-based partitioned routing is used so that each subsequentevent in the Business Events context can be stored in a specific partition.This requires that the key of Business Events context is used as the key

of ObjectGrid entity model. This ensures that Business Events maintainsconsistency because it receives all the events in a context from theObjectGrid listener.

When a partition is activated as a primary, theObjectGridEventListener implementation class receives theactivate shard event and starts its worker threads that listen for objectsadded to this partition. These threads access a partition using First InFirst Out (FIFO) because ObjectGrid tracks the insertion order for apartition. When an object is read from the ObjectGrid partition, theObjectGrid listener publishes an event to the Business Events action topicfor this partition on SIBus using the JMS API. Threads are always running

to be collocated with a primary partition. If a partition is stopped, thethreads are deactivated and started again on a primary partition on othereXtreme Scale servers. More information on this is provided in Listing 6and Listing 7 of part 1.

Multiple Business Events instances can share a DB2 instance where theBusiness Events repository database is defined. Business Events


http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing6http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing7http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing7http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing6


5/42

communicates with DB2 using the DB2 JDBC driver.

Figure 3. Solution architecture for part 2

Next, we will map the components used in part 1 to the solution architecture above.The new solution is shown in Figure 4. The Member1 and Member2 applicationservers are defined as cluster members in the cluster where the TrafficAlerter andTrafficAlerterFilter EAR files are deployed. Because all of the application servers aremembers of a bus called WbeBus, the TrafficAlerterFilter can readily access theBusiness Events outbound action topic.

Figure 4. Solution components for part 2




6/42

ObjectGrid provides a public Java API for FIFO access to object maps. By using theObjectMap.getNext method within the TrafficAlerterFilter.ear file, we can easilyaccess maps like a FIFO queue. The getNextKey method returns the key of thenext available map entry in the map, which can then be removed from the queueusing the returned key. The TAFilterWorker.run method for using the FIFO API

is shown in Listing 7 of part 1. Because we use the local ObjectGrid referenceprovided to the ObjectGridListener we always read from the local partition. This isthe behavior we need to ensure that the same Business Events context is alwayssent to the same Business Events instance.

Implement changes for a multiple-partition solution

Several changes are required to implement the multiple-partition solution:

Create a schema root definition for distributed ObjectGrid partitions Change the entity model for partitioned data routing

Change the ObjectGrid transaction strategy

Map the partition ID to the JMS JNDI name

Change the configuration for multiple Business Events connectors on


http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing7http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing7


7/42

multiple nodes

Configure the SIBus and add a Business Events instance

Create a schema root definition for distributed ObjectGrid partitions

ObjectGrid partitions are established by splitting the data described by theuser-defined entity model in the entity.xml file. In a distributed ObjectGridenvironment a client can access all the defined entities through the root entity. Inpart 1, ObjectGrid was used with distributed and stand-alone mode, and we had onlyone partition so we didn't need a schema root definition in the entity.xml file. In part2, multiple partitions are deployed to multiple eXtreme Scale servers and the EJBworks as an ObjectGrid remote client. This means that a request must be routed to aspecific partition (possibly across the network), therefore requiring a schemaRootdefinition in our objectGrid.xml file.

We must also define two ObjectGrid spaces because one ObjectGrid space can

have only one schema root. Recall that we have two data types that we want tostore in ObjectGrid. The first type is DataPacket, used to calculate the speed of anRFID. The second type is SpeedAtLocation, which stores the speeds in theObjectGrid FIFO queue and is read by the TrafficAlerterFilter application. Thereforewe must define two object grids, TrafficAlerterOG and TrafficAlerterFilterOG, whichhave DataPacket and SpeedAtLocation entities defined as schema roots. SeeListing 1 and Listing 2 for the TrafficAlerterOG; Listing 4 and Listing 5 for theTrafficAlerterFilterOG.

Listing 1. The objectGrid.xml (TrafficAlerterOG)

Listing 2. The entity.xml file (TrafficAlerterOG)

"This is the DataPacket class"

Notice that we specify numberOfPartitions="2" and numInitialContainers="2" in the




8/42

objectGridDeployment.xml file in Listing 3 and Listing 6. This means that we havetwo partitions on the clustered Member1 and Member2 servers and we also have anasynchronous replica.

Listing 3. The objectGridDeployment.xml file (TrafficAlerterOG)

Listing 4. The objectGrid.xml file (TrafficAlerterFilterOG)

Listing 5. The entity.xml file (TrafficAlerterFilterOG)

"Location"

"Speed at a location"




9/42

Listing 6. The objectGridDeployment.xml file (TrafficAlerterFilterOG)

Change the entity model for the partitioned data routing

The entity model used in part 1 must be changed so that ObjectGrid automaticallyroutes a request to the correct server. Recall that we defined a unique id as a key forthe SpeedAtLocation entity. In this case, ObjectGrid routes a request to a partitionby using an algorithm that divides a hash value of a key by the number of partitions.This means that we cant control the partitioned data routing in distributed andmultiple partitions because the routing key is unique.

In our multiple-partition solution, an event for each location must get sent to thesame partition, so that the ObjectGrid listener for that partition sends the event to thesame Business Events instance. To do this we define a new entity model as shownin Figure 5. A Location entity has a reference to one or more SpeedAtLocationentities. Each SpeedAtLocation entity has a reference to a single Location entity.There are bidirectional relationships between the Location and the SpeedAtLocation.The locationId of the Location entity is used as a key for ObjectGrid partition routing.As the locationId is also a Business Events context in our interaction set, an eventwill get forwarded to the correct Business Events instance.

Figure 5. Entity model for multiple-partition solution

Figure 6. Entity model for single-partition solution




10/42

Because we changed the entity model, we need to modify the sendDataPacketmethod (Listing 7) as follows:

1. The ejbCreate method invokes this method to create an ObjectGridmanager, a client cluster context and get references to the

TrafficAlerterOG and TrafficAlerterFilterOG. TheCATALOG_SERVER_PROP should include the hostname andORB_LISTENER_ADDRESS port of the deployment manager.

2. Get a reference to the EntityManager for the TrafficAlerterFilterOG.

3. Begin a transaction for the TrafficAlerterFilterOG.

4. Find a location entity by the location ID as a key.

5. If there is no entry with this location ID in the TrafficAlerterFilterOG, then

create a new Location and SpeedAtLocation entity, associate them andstore two related entities.

6. Create a new SpeedAtLocation entity, add to the Location entity, andcommit them.

Listing 7. Modified sendDataPacket method

private ObjectGridManager ogManager;private ObjectGrid og;private ObjectGrid ogf;

1. private void initialiseOG() {

ogManager = ObjectGridManagerFactory.getObjectGridManager();ClientClusterContext context =

ogManager.connect(TrafficAlerterConstants.CATALOG_SERVER_PROP,null ,null);og = ogManager.getObjectGrid(context,TrafficAlerterConstants.TA_OG_NAME);ogf = ogManager.getObjectGrid(context,TrafficAlerterConstants.TAF_OG_NAME);

}

public void sendDataPacket(DataPacket data) {EntityManager em = getEntityManager();em.getTransaction().begin();try{




11/42

DataPacket currentData = (DataPacket) em.find(DataPacket.class, data.RFID);if (currentData == null) {

currentData = new DataPacket();currentData.locationId = data.locationId;currentData.RFID = data.RFID;currentData.timeInMillis = data.timeInMillis;em.persist(currentData);em.getTransaction().commit();

} else {BigDecimal diff = new BigDecimal(data.timeInMillis

currentData.timeInMillis);BigDecimal timeInHours = diff.divide(millisInHour,10,

RoundingMode.HALF_UP);BigDecimal speedBig = halfMile.divide(timeInHours, 0,

RoundingMode.HALF_UP);int speed = speedBig.intValue();currentData.locationId = data.locationId;currentData.timeInMillis = data.timeInMillis;em.persist(currentData);em.getTransaction().commit();

2. EntityManager emf = getEntityManagerForFilter();3. emf.getTransaction().begin();4. Location currentLoc = (Location) emf.findForUpdate(Location.class,

data.locationId);if (currentLoc == null) {5. Location loc = new Location();

loc.locationId = data.locationId;loc.speedAtLocations = new ArrayList();SpeedAtLocation sp = new SpeedAtLocation();sp.location=loc;sp.speed = speed;sp.UUID = UUID.randomUUID().toString();loc.speedAtLocations.add(sp);emf.persist(loc);emf.getTransaction().commit();

}else{6. SpeedAtLocation sp = new SpeedAtLocation();

sp.location=currentLoc;sp.speed=speed;sp.UUID = UUID.randomUUID().toString();

currentLoc.speedAtLocations.add(sp);boolean persisted = false;

while (! persisted){try {

emf.getTransaction().commit();persisted = true;} catch (EntityExistsException eee) {

sp.UUID = UUID.randomUUID().toString();}

}}

}} catch (Exception e) {

e.printStackTrace();}}

package com.ibm.ta.util;public class TrafficAlerterConstants {

public static final String TA_OG_NAME = "TrafficAlerterOG";public static final String TAF_OG_NAME = "TrafficAlerterFilterOG";public static final String CATALOG_SERVER_PROP =

"wstiblade9.rtp.raleigh.ibm.com:9100";}

Change the ObjectGrid transaction strategy




12/42

In part 1 we did not specify a locking strategy in the objectGrid.xml file. By default,ObjectGrid uses an optimistic lock strategy. This works fine for a stand-aloneeXtreme Scale server and a single partition (with only one ObjectGrid near cache,there is no chance for a collision error). However for multiple eXtreme Scale servers,collision errors can occur as illustrated in Figure 7.

1. The EJB client invokes a method on the Member1 server. The requestcontains values for the RFID, locationId and timeInMillis parameters of theDataPacket entity model. For example, the EJB client sends theparameters RFID=111 and locationId=1 at Location 1 and then the EJBcommits them to the TrafficAlerterOG object grid. In the meantime,ObjectGrid keeps them in the near cache of the Member1 server.

2. Next, a client moves to Location 2 and sends the parameters RFID=111and locationId=2. This request is dispatched to the Member2 server bythe EJB clustering function. The EJB can commit from the Member2server because we use a distributed ObjectGrid. At this point, theObjectGrid near cache of the Member1 server is stale.

3. After that, a client moves to Location 3 and sends parameters RFID=111and locationId=3. If this request is dispatched to the Member1 server, acollision error occurs when the client tries to commit it.

Figure 7. Collision error situation

See Listing 5 for more information on the sendDataPacket method used in part 1.This logic works fine when using a local ObjectGrid with an optimistic strategy, butwhen we have more than one eXtreme Scale server we need to specify apessimistic strategy in the objectGrid.xml file (Listing 1 and Listing 4).


http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing5http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing5


13/42

Map the partition ID to the JMS JNDI name

This section shows you how we make sure that each ObjectGrid partition sendsevents to the same Business Events instance. The TAOGListener andTAFilterWorker classes are explained in Listing 6 and Listing 7 of part 1. The

TAFilterWorker class retrieves an event from each partition and forwards it to aspecific Business Events server. To do this, add a new method to theTAFilterWorker class called getWbeResourceNames (Listing 8). This methodtakes a partition number and calculates which JMS connection factory and topic itshould use. This establishes affinity between the event and a Business Eventsinstance.

1. The method gets the JMS JNDI name corresponding to the ObjectGridpartition ID and assigns it to the CF_name and TOPIC_name variables.

2. The sendEventToWBE method sends the actionable event to the

Business Events instance having affinity with the current ObjectGridpartition ID.

3. This method initializes a lookup table (map) which maps a constantcomprised of a resource prefix concatenated with the partition ID to thedesired resource JNDI name.

Listing 8. The TAFilterWorker.run method

private String CF_name;private String TOPIC_name;

public void run() {initialiseOG();Session session = getSession();EntityManager em = session.getEntityManager();try {

ObjectMap map = session.getMap("SpeedAtLocation");BackingMap bMap = og.getMap("SpeedAtLocation");

1. getWbeResourceNames(bMap.getPartitionId());

while (!appStopping.get()) {em.getTransaction().begin();Object msgKey = map.getNextKey(2000);if (msgKey != null) {

SpeedAtLocation sp =(SpeedAtLocation) em.find(SpeedAtLocation.class, msgKey);if (sp.location.locationId.equals("-1")) {

sendTestFinishedEvent();

}else {

// now consume the messageif (sp.speed < Constants.TA_MIN_SPEED) {

2. sendEventToWBE(sp);}

}// need to remove itmap.remove(msgKey);

}em.getTransaction().commit();


http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing6http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing7http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing7http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#listing6


14/42

}}

private void getWbeResourceNames(int partitionID) {CF_name = wbeResourceMap.get("tcf"+partitionID);TOPIC_name = wbeResourceMap.get("topic"+partitionID);

}

public TAFilterWorker(ObjectGrid og) {super();this.og = og;if (wbeResourceMap.isEmpty()) {

initializeMap();}

}

3. private void initializeMap(){wbeResourceMap.put("tcf0", TrafficAlerterConstants.WBE1_TCF_PROP);wbeResourceMap.put("tcf1", TrafficAlerterConstants.WBE2_TCF_PROP);wbeResourceMap.put("topic0", TrafficAlerterConstants.WBE1_TC_PROP);wbeResourceMap.put("topic1", TrafficAlerterConstants.WBE2_TC_PROP);

}

package com.ibm.ta.util;

public class TrafficAlerterConstants {public static final String WBE1_TCF_PROP ="cell/nodes/wstiblade9Node01/servers/WBE-1/jms/WbeTopicConnectionFactory";

public static final String WBE1_TC_PROP ="cell/nodes/wstiblade9Node01/servers/WBE-1/jms/eventTopic";public static final String WBE2_TCF_PROP =

"cell/nodes/wstiblade6Node01/servers/WBE-2/jms/WbeTopicConnectionFactory2";public static final String WBE2_TC_PROP ="cell/nodes/wstiblade6Node01/servers/WBE-2/jms/eventTopic2";

}

Change the configuration for multiple Business Events connectors on multiplenodes

In our scenario, we defined two types of action connectors that can send an action toBusiness Events touchpoints (Figure 8). One is the file system connection for theLowSpeedAlert action to the HighwaysAgency touchpoint. Another is the messagequeue connection for the AllEventsProcessed action to the WXSServer touchpoint.

Figure 8. Action connectors on Business Events touchpoints




15/42

Figure 9 shows the Business Events connectors topology. The JMS queue ofAllEventsProcessed is used to determine when Business Events has processed allthe events. The EJB client application waits to receive the last message from thisqueue after all the raw events are sent. In part 1, the AllEventsProcessed queue wasdefined on the WXSServer. In part 2, because our eXtreme Scale servers areclustered we define an additional application server for this message processingcalled the Test1 server; multiple connectors can share its JMS queue.

Figure 9. Business Events connectors topology




16/42

To define JMS resources on the Test1 server, use the WebSphere administrativeconsole to do the following:

1. Create a bus destination called TEST_FINISHED.

a. Select Service integration => Buses and click WbeBus.

b. Click Destinations.

c. Click New.

d. Select the Queue checkbox, and click Next.

e. Specify TEST_FINISHED as the Identifier and click Next.

f. Select Test1 as the bus member and click Next.

g. Click Finish.

2. Create a JMS queue connection factory called AllEventsProcessedQCF.

a. Select Resources => JMS => Queue connection factories.

b. Select the first node as the scope and click New.

c. Select the Default messaging provider checkbox and click OK.

d. Specify AllEventsProcessedQCF as the Name and




17/42

jms/AllEventsProcessedQCF as the JNDI name.

e. Select WbeBus as the Bus name and specify the messagingengine name as the Target for the connection. You can find themessaging engine name by selecting Application servers =>

Test1 => Messaging engines.

f. Select Messaging engine name as the Target type and specifythe host name and port number as the Provider endpoints. Youcan find the value of SIB_ENDPOINT_ADDRESS by selectingApplication servers => Test1 => Ports and clicking OK.

3. Create a JMS queue called AllEventsProcessed.

a. Select Resources => JMS => Queues.

b. Select the first node as the scope and click New.

c. Select the Default messaging provider checkbox and click OK.

d. Specify AllEventsProcessed as the Name andjms/AllEventsProcessed as the JNDI name.

e. Select WbeBus as the Bus name and TEST_FINISHED as theQueue name for the connection, and click OK.

We added two properties in the trafficAlerter.props file: AllEventsProcessedQCF andAllEventsProcessedQueue. We can define compound JNDI names for JMS

resources. The providerURL should include the hostname andBOOTSTRAP_ADDRESS port of the deployment manager.

Listing 9. The trafficAlerter.props file

providerURL=iiop://wstiblade9:9809AllEventsProcessedQCF=cell/nodes/wstiblade9Node01/servers/Test1/jms/AllEventsProcessedQCFAllEventsProcessedQueue=cell/nodes/wstiblade9Node01/servers/Test1/jms/AllEventsProcessed

We also changed the configuration for these connectors. For the message queueconnections, use the Design Data UI to edit properties so that the Provider URL

matches your system environment. The port number should be theBOOTSTRAP_ADDRESS port of the Test1 server. See Figure 5 in part 1. For thefile system connections, edit the properties so that the directory name doesntconflict with the directory name for another Business Events connector. Use theDesign Data UI to change the directory path in the TAProject-partition.xml file for theWBE-2 server as shown in Figure 10. There are two TAProject-partition.xml files(one for each Business Events instance), so we need to make similar changes to theWBE-1 TAProject-partition.xml file.


http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#fig5http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#fig5


18/42

Figure 10. Edit properties for the file system connection

Build the topology and configure SIBus

This section describes the required WebSphere profiles needed to build the solutiontopology. We assume you are familiar with using the WebSphere administrativeconsole to configure application servers and J2EE applications. Because we want tobuild a mission-critical system that can handle high-volume raw events, we use adeployment manager profile and a custom profile (both enabled for eXtreme Scale)instead of a stand-alone profile. (In the profile management tool, chooseWebSphere Extended Deployment Data Grid in the Environment Selectiondialog, and then choose Deployment Manager in the Profile Type Selectiondialog.) All the application servers are defined within a single cell, as shown in

Figure 11. This configuration can provide central management capabilities formultiple application servers. First, we define a deployment manager profile and thencreate a custom profile and federate the custom profile to the cell. After thefederation, we can further configure the topology by adding application servers andclusters from the deployment manager using WebSphere administrative console.

Figure 11. WebSphere profiles and application server topology




19/42

We created a cluster and five application servers in a cell. The application clusterAppCluster consists of a pair of application servers: Member1 on Node 1 andMember2 on Node 2. Due to an eXtreme Scale product issue at the time of thetesting, we needed to use the same profile names for the AppCluster members on

both nodes. If the Member1 server fails, EJB requests are routed to the Member-2server. The Test1 server is used by our test application so that it knows when allevents have been processed by Business Events. (See Test Finished processing inpart 1.) The WBE-1 and WBE-2 servers are installed with the Business Eventsrun-time code (WBERuntimeEAR).

To define this system topology, use the WebSphere administrative console to do thefollowing:

1. Create a new server cluster called AppCluster.

a. Select Servers => Clusters and click New.

b. Specify AppCluster as the cluster name and uncheck the checkboxof prefer local, and click Next.

c. Specify Member1 as the member name and the first node as theselect node, and click Next.


http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#testFinishedhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#testFinished


20/42

d. Specify Member2 as the member name and the second node asthe select node, and click Next.

e. Click Finish.

2. Change the classloader policy as server specific application settings.

a. Select Servers => Application servers and click Member1.

b. Select Single as the classloader policy and click OK.Repeat this step to configure Member2.

3. Create shared libraries for the TrafficAlerterUtil.jar file.

a. Select Environment => Shared Libraries and select the first nodeas a scope, and click New.

b. Specify TrafficAlerterUtil as the name and//TrafficAlerterUtil.jar as the classpath, and click New.Repeat this step to configure the second node.

4. Configure class loaders for the Member1 and Member2 servers.

a. Select Servers => Application servers and click Member1.

b. Select Server Infrastructure => Java and Process Management=> Class loader and click New.

c. Select Classes loaded with parent class loader first and clickOK.

d. Click the created classloader.

e. Click Additional Properties => Shared library references.

f. Click Add and select TrafficAlerterUtil as the library name, andclick OK.Repeat this step to configure for the Member2 server.

Configure the Service Integration Bus (SIBus)

The SIBus configuration is very adaptable. You can have individual messagingengines (MEs) in an application server, or clustered multiple MEs for sharingworkloads. You can also configure the ME to be highly available. We defined a busmember by adding all the servers to a bus called WbeBus (Figure 12). The WbeBusis created and a server is added as a member of the bus using the configure_bus.sh




21/42

script when we install and configure a Business Events server. (The WBE-1 andWBE-2 servers are added to the bus when those servers are configured.) See theWebSphere Business Events Installation and Operations Guide in Resources forinformation on installing WebSphere Business Events on an existing WebSphereServer instance.

To define the SIBus topology, do the following:

1. Add the Test1 server and AppCluster as new members of the WbeBususing the WebSphere administrative console.

a. Select Service integration => Buses and click WbeBus.

b. Click Topology => Bus members.

c. Click Add.

d. Select the radio button for the cluster, select AppCluster, and clickNext.

e. Select the radio button for the file store and click Next.

f. Specify the log directory path and the permanent store directorypath, and click Next.

g. Click Finish.

2. Repeat this step for the Test1 server.Figure 12. Bus members on the WbeBus




22/42

3. By default, when you add a cluster to a bus, Application Server V6.1 doesME failover. It will ensure that only one instance of the ME is running atany given time within a cluster. Additional configuration steps are neededfor each cluster member to have its own ME. To do this, we need to add

the ME into the AppCluster bus member and create a static policy for theindividual ME. The steps are:

a. Click Add messaging engine to add an ME into the AppCluster.Figure 13. Add a messaging engine into the AppCluster

b. Click New to create a new static policy.Figure 14. Create a new static policy

c. Specify the name of a static policy and click the Match criteria linkto define the new static policy.Figure 15. Define the new static policy




23/42

d. Define two match criteria in the policy.Figure 16. Define the match criteria

e. Finally, click the Static group servers link. Add the Member1server into the static group servers of this policy as shown in Figure17.Figure 17. Add the Member1 server to the static group servers




24/42

Repeat Step 3 to configure Member2.

Install and configure the second Business Events server

This section describes the step-by-step procedure for installing and configuring thesecond Business Events server such as WBE-2 on a different node. It describes

how to run the supplied custom script files and how to define a DB2 database as aBusiness Events repository.

1. Create the repository database for the WBE-2 server on the first nodewhere you installed DB2.

a. Switch the db2 instance user ID.$ su - [userid]

b. Create the database. We will use the name of the

to define Business Events databasesettings at step 19.$ db2 create db

2. Copy the db2jcc.jar file from the first node where we created the BusinessEvents repository database to the second node.

3. Install Business Events on the second node.




25/42

a. Run the Business Events installer.

b. Select Custom on the Choose Install Set dialog and click Next.

c. Select WebSphere Business Events Technology Connectorsand Database Server and click Next.

d. Click Next on the Choose Install Folder dialog.

e. Select IBM DB2 V9.5 and click Next.

f. Click Choose and specify the JDBC driver location.

g. Enter the DB2 connection information for the first node.

h. Next, run the Business Events Fix Pack 1 installer.

4. Using the WebSphere administrative console, create a new applicationserver named WBE-2.

a. Select Application servers and click New.

b. Select the second node, enter server name WBE-2, and click Next.

c. Select a default template and click Next.

d. Select a checkbox to generate unique ports and click Next.

e. Click Finish.

5. Add host aliases.

a. Select Environment => Virtual Hosts and click default_host.

b. Click Host Aliases.

c. Click New.

d. Specify the WC_defaulthost port number as the Port and click OK.

You can find the port value for WC_defaulthost by selectingApplication servers => WBE-2 => Ports.

6. Navigate to the /config/was directoryand edit the setenv.sh script file as follows:

a. Specify the existing Application Server binary directory, forexample:




26/42

WASHOME=/opt/IBM/WebSphere/AppServer

b. Specify the deployment managers hostname, for example:WASADMINHOST=wstiblade9.rtp.raleigh.ibm.com

c. Specify the deployment managers bootstrap port number, forexample:WASBOOTSTRAPPORT=9809

d. Specify a profile name where you want to install the secondBusiness Events server, for example:WASPROFILE=Custom01

e. Specify the application server name where the Business Eventsrun-time EAR file is to be installed.WASSERVER=WBE-2

7. Copy the /config/WbeConfig.jar file tothe Application Server library directory.

8. Run the start_was.sh script to start the WBE-2 server.

9. Run the configure_server.sh script to configure the WBE-2 server.

10. Run the configure_bus.sh script to configure SIBus for the WBE-2 server.

11. Edit the configure_messaging.py script file as follows:

a. Add the following line to define a topic space for the WBE-2 server.topicSpaceName = topicSpaceName + '2'

b. Add the following line to define name variables for the WBE-2server.name = name + '2'

c. Comment out the following line:# description = topics.get(name)

d. Change the description for the WBE-2 server.'-description', "For WBE-2"]

12. Run the configure_messaging.sh script to configure JMS administrativeobjects for the WBE-2 server.

13. Specify the target messaging engine name for the queue connectionfactory using the WebSphere administrative console.




27/42

a. Select Resources => JMS => Queue connection factories.

b. Click WbeQueueConnectionFactory2.

c. Specify the messaging engine name as Target for the connectionand click OK. You can find the messaging engine name byselecting Application servers => WBE-2 => Messaging engines.

14. Specify the target messaging engine name for the topic connectionfactory.

a. Select Resources => JMS => Topic connection factories.

b. Click WbeTopicConnectionFactory2.

c. Specify the messaging engine name as Target for the connection

and click OK. You can find the messaging engine name byselecting Application servers => WBE-2 => Messaging engines.

15. Repeat steps 12 and 13 to configure for theWbeQueueConnectionFactory and WbeTopicConnectionFactory on theWBE-1 server.

16. Edit the install_app.py script file.

a. Change the following line for the WBE-2 server.jdbcDriverLibrary = 'WbeJdbcDriver2'

b. Add the following application name parameter.'-appname', "WBERuntimeEAR2", \

c. Change parameters as follows.parameters += ['-MapSharedLibForMod',

[['WBERuntimeEAR2', \

17. Run the install_app.sh script for installing the Business Events run-timeEAR file into the WBE-2 server.

18. Navigate to the /director/bin directory and

edit the connector.sh script file. We specify the from Step 2.export CLASSPATH=/db2jcc.jar

19. Next, specify the configuration properties using the Business Eventsproperties UI.




28/42

a. Start the Business Events properties UI.

b. Specify values for the Hostname, Instance, Username andPassword fields. For the Instance field, we specified, which is defined in Step 1.Figure 18. Business Events Properties UI for DatabaseSettings

c. Click Next and specify values for the Store URL, Topic ConnectionFactory and Queue Connection Factory fields. For the Store URLfield, we specified the WBE-2 bootstrap port.Figure 19. Business Events Properties UI for JMS Settings




29/42

d. Click Full Configurator.

e. Click the Common section and specify values for Subsection:Default Database Settings and Subsection: Asset Repositoryas shown in Figure 20.Figure 20. Business Events Properties UI for Asset RepositorySettings

f. Click the JMS section and specify values for Subsection: Topics.




30/42

Figure 21. Business Events Properties UI for JMS TopicsSettings

g. Click the Logging section and specify values forlog4j.logger.com, log4j.logger.event,log4j.appender.JMS1.TopicBindingName andlog4j.appender.JMS2.TopicBindingName as shown in Figure 22.Figure 22. Business Events Properties UI for Logging Settings




31/42

h. Click the Server section and specify values foras.director.server.instanceName,as.director.server.ruleProcessorInstances andas.director.server.preProcessor.instances.Also specify values for Subsection: Timebased RepositorySettings and Subsection: Eventflow Database Settings asshown in Figure 23.Figure 23. Business Events Properties UI for TimebasedRepository and Eventflow Database Settings




32/42

i. Click File and select Save Properties.

20. Start the WBE-2 server using the WebSphere administrative console.

21. Deploy the TAProject-Partition.xml file using the Business Events DesignData tool.

Run the application on the Business Events multiple instances




33/42

To run the application on multiple instances, perform the following steps:

1. Start a DB2 instance.$ su - db2inst1

$ db2start

2. Start the WBE-1, WBE-2, and Test1 servers by using the WebSphereadministrative console.

3. Start the AppCluster.

4. Navigate to the /opt/IBM/WBE/director/bin directory on the first node andstart the Business Events connector-1.$ connectors.sh

5. Navigate to the /opt/IBM/WBE/director/bin directory on the second node

and start the Business Events connector-2.$ connectors.sh

6. Run the launchClient.sh script to start the client application. ThetrafficAlerter.props file is described in Table 3 in part 1. We defined thedeployment managers bootstrap port as providerURL in thetrafficAlerter.props file.$ launchClient.sh ClientEAR.ear

-CCclasspath=TrafficAlerterUtil.jar

trafficAlerter.props

7. After the client application is finished, navigate to the/opt/TrafficAlerts/WBE-1 directory which we specified for the file systemconnection in Figure 10. Check the action XML files. They should containa sequence of odd numbers as a location id if our scenario is workingwell. To do this, confirm the value of the XML tag . Similarly, the action XML file inthe /opt/TrafficAlerts/WBE-2 directory should contain even numbers as alocation id.

Listing 10. Action XML files in the WBE-1 directory

2130

wstiblade9.rtp.raleigh.ibm.com


http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#table3http://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html#table3


34/42

2008-09-23T18:16:48.797-0400Generated from IBM WebSphere Business Events

2530

wstiblade9.rtp.raleigh.ibm.com2008-09-23T18:17:29.911-0400Generated from IBM WebSphere Business Events

2930


Listing 11. Action XML files in the WBE-2 directory

28

30


230





35/42

2230

wstiblade9.rtp.raleigh.ibm.com

2008-09-23T17:48:53.661-0400Generated from IBM WebSphere Business Events

Performance Results

To check the effectiveness of our solution, we measured the performance data forthree different system topologies (patterns 1, 2, and 3). Each pattern consists of asingle cell containing two nodes; each node runs on a separate machine (Figure 24,Figure 25, and Figure 26).

Look carefully at Figure 26 (pattern 3). Because ObjectGrid partitions are defined astwo partitions on each of two application servers, each application server has aprimary partition and an asynchronous replica. As EJB requests from a client arespread over both servers by the EJB clustering function, the EJB accesses either alocal or a remote partition. Stateless EJB instances arent always collocated with aprimary partition. Accessing a local partition is very fast because it is in memory; butwhen the EJB accesses a remote partition (that another server is running as aprimary), performance suffers because the access is across a network. Therefore,the primary consideration in a performance test should be the number ofremote-access paths to the ObjectGrid partitions. This means that we shouldnt

compare the performance results of the part 2 solution against part 1 the singlepartition in part 1 is always local and is neither highly available nor scalable. Tocorrectly determine performance behavior in a distributed system environment, wechose to build the three system topologies and compare the results for the threesystems.

As shown in Figure 24, pattern 1 is running the TrafficAlerterFilter application andthe WBE-1 server on a single node and the TrafficAlerter application is on anothernode. The important point to note in pattern 1 is that the EJB will remotely accessthe partition of the TrafficAlerterFilterOG over a network. Pattern 2 is running theTrafficAlerterFilter application and two Business Server instances across two nodes.

Pattern 2 uses clustering for the TrafficAlerterFilter applications and multipleBusiness Events instances. In pattern 3 both the TrafficAlerter andTrafficAlerterFilter applications run in the cluster, and each is configured to use twopartitions.

Figure 24. Pattern 1 topology




36/42


37/42

Figure 26. Pattern 3 topology




38/42

We used the same code for all the patterns. We ran performance tests to change thenumber of total events that Business Events processed. For the purposes of theperformance test, the WBERatio parameter in the TrafficAlerter.props file was set to1 and every message was forwarded to the Business Events instances without

applying the prefilter logic. The performance results are shown in Figure 27. Theresults show that the throughput is approximately 1.7 times better for patterns 2 and3 than for pattern 1. This proves the effectiveness of our solution design forintegrating Business Events with eXtreme Scale.

Figure 27. Performance results




39/42

Summary

This article describes important design considerations for building a scalable andhighly available solution by using Business Events and eXtreme Scale. eXtremeScale is used for high-volume raw events storage. A prefiltering application reduces

the processing load on Business Events. Using context-based event routing, eventsare routed to appropriate partitions in the distributed ObjectGrid. Performanceresults show a significant improvement in throughput when clustering and distributedpartitions are used. You learned how to implement this solution using the ObjectGridEntity manager API, the ObjectGrid FIFO API, and the JMS API on the SIBus. Youlearned how to configure multiple Business Events instances and to define a DB2database as a Business Events repository.

Acknowledgments

The author would like to thank Gale Botwick, Jeffrey Garratt, and Daniel McGinnesfor their careful review, comments, and feedback on this article.




40/42

Downloads

Description Name Size Download method

Project XML file TAProject-Partition.zip4KB HTTP

Deployable EAR files deployableEARFiles.zip111KB HTTP

Custom scripts for Linux and Windows customScripts.zip5KB HTTP

Source code for part 2 part2Source.zip 44KB HTTP

Information about download methods


http://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/0811_kojima/TAProject-Partition.ziphttp://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/0811_kojima/deployableEARFiles.ziphttp://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/0811_kojima/customScripts.ziphttp://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/0811_kojima/part2Source.ziphttp://www.ibm.com/developerworks/library/whichmethod.htmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/library/whichmethod.htmlhttp://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/0811_kojima/part2Source.ziphttp://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/0811_kojima/customScripts.ziphttp://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/0811_kojima/deployableEARFiles.ziphttp://download.boulder.ibm.com/ibmdl/pub/software/dw/wes/0811_kojima/TAProject-Partition.zip


41/42

Resources

Learn

Read part 1 of this article series Processing complex business events with

WebSphere eXtreme Scale, Part 1: Reducing the load on WebSphere BusinessEvents.

Learn the basic concepts of ObjectGrid in "Build a scalable, resilient, highperformance database alternative with the ObjectGrid component ofWebSphere Extended Deployment" (developerWorks, November, 2007).

Find details in the WebSphere eXtreme Scale and ObjectGrid V6.1 User Guide.

Read the WebSphere eXtreme Scale Information Center.

Read the Business Events Installation and Operations Guide.

Check out the WebSphere Business Events V6.1 Information Center. Read the series "Business Event processing with WebSphere Business Events

" (developerWorks, September, 2008) to learn about how Business Eventsinteroperate with other IBM products.

Get products and technologies

Download the latest Fix Packs for WebSphere Application Server.

Download the latest Fix Packs for WebSphere eXtreme Scale.

Discuss

Check out developerWorks blogs and get involved in the developerWorkscommunity.

About the author

Kenji Kojima

Kenji Kojima is a Developer at IBM at the WebSphere Technical Institute in ResearchTriangle Park, NC. He works on the integration of WebSphere Business Events withWebSphere eXtreme Scale. He previously worked as a Certified Consulting IT

Specialist providing consultancy to IBM customers in Japan.

Trademarks


http://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/techjournal/0711_chambers/0711_chambers.htmlhttp://www.ibm.com/developerworks/websphere/techjournal/0711_chambers/0711_chambers.htmlhttp://www.ibm.com/developerworks/websphere/techjournal/0711_chambers/0711_chambers.htmlhttp://www.ibm.com/developerworks/websphere/techjournal/0711_chambers/0711_chambers.htmlhttp://www.ibm.com/developerworks/wikis/display/objectgrid/Getting+startedhttp://publib.boulder.ibm.com/infocenter/wxdinfo/v6r1/index.jsp?topic=/com.ibm.websphere.dataint.doc/info/welcome_61_dgxd.htmlhttp://publib.boulder.ibm.com/infocenter/wbevents/v6r1m0/topic/com.ibm.wbe.pdfs.doc/C3469690.pdfhttp://publib.boulder.ibm.com/infocenter/wbevents/v6r1m0/index.jsphttp://www.ibm.com/developerworks/views/websphere/libraryview.jsp?search_by=Business+event+processing+with+WebSphere+Business+Events,+Parthttp://www.ibm.com/developerworks/views/websphere/libraryview.jsp?search_by=Business+event+processing+with+WebSphere+Business+Events,+Parthttp://www-1.ibm.com/support/docview.wss?rs=180&uid=swg27009661http://www.ibm.com/developerworks/wikis/display/objectgridprog/Release+noteshttp://www.ibm.com/developerworks/blogs/http://www.ibm.com/developerworks/communityhttp://www.ibm.com/developerworks/communityhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/legal/copytrade.shtmlhttp://www.ibm.com/developerworks/communityhttp://www.ibm.com/developerworks/communityhttp://www.ibm.com/developerworks/blogs/http://www.ibm.com/developerworks/wikis/display/objectgridprog/Release+noteshttp://www-1.ibm.com/support/docview.wss?rs=180&uid=swg27009661http://www.ibm.com/developerworks/views/websphere/libraryview.jsp?search_by=Business+event+processing+with+WebSphere+Business+Events,+Parthttp://publib.boulder.ibm.com/infocenter/wbevents/v6r1m0/index.jsphttp://publib.boulder.ibm.com/infocenter/wbevents/v6r1m0/topic/com.ibm.wbe.pdfs.doc/C3469690.pdfhttp://publib.boulder.ibm.com/infocenter/wxdinfo/v6r1/index.jsp?topic=/com.ibm.websphere.dataint.doc/info/welcome_61_dgxd.htmlhttp://www.ibm.com/developerworks/wikis/display/objectgrid/Getting+startedhttp://www.ibm.com/developerworks/websphere/techjournal/0711_chambers/0711_chambers.htmlhttp://www.ibm.com/developerworks/websphere/techjournal/0711_chambers/0711_chambers.htmlhttp://www.ibm.com/developerworks/websphere/techjournal/0711_chambers/0711_chambers.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.htmlhttp://www.ibm.com/developerworks/websphere/library/techarticles/0810_mcginnes/0810_mcginnes.html


42/42

DB2, IBM, and WebSphere are trademarks or registered trademarks of IBMCorporation in the United States, other countries, or both.

Microsoft and Windows are registered trademarks of Microsoft Corporation in theUnited States, other countries, or both.

Red Hat and Red Hat Enterprise Linux are registered trademarks of Red Hat, Inc. in

the U.S. and other countries.Linux is a trademark of Linus Torvalds in the United States, other countries, or both.

Java and all Java-based trademarks and logos are trademarks or registeredtrademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

Oracle is a registered trademark of Oracle Corporation and/or its affiliates.

Other company, product, and service names may be trademarks or service marks ofothers.


improving scalability with objectgrid partitions

Documents