4126StarTeam SDK Advanced

Programming Topics

Ron SauersPrinciple Architect

Borland Software Corporation

StarTeam SDKAdvanced Programming Topics

Performance TuningMPXInstant RefreshEvent HandlingPerformance Tuning Revisited

Performance TuningTips to help identify and eliminate

performance bottlenecks

Typical SDK ApplicationSDK apps are client/server apps:

– App requests some information via an SDK API.

– SDK determines whether the information is cached.

– If not, SDK issues a command across the network to the StarTeam server.

– Server retrieves the information, and ships it back across the network to the SDK.

– SDK caches it locally, and returns it to the app.

Server Commands

Most SDK applications spend the majority of time executing server commands.

For this reason, performance tuning usually means:– Minimizing the number of server

commands issued– Minimizing the amount of data sent

and received

Net Monitor

Monitors the command traffic between the client application and the StarTeam server.

For each command, it displays:– A descriptive name for the command; – Number of bytes sent; – Number of bytes received; – Execution time, in milliseconds.

Enabling NetMonitor

// Enables NetMonitor, // sending output to the NetMonitor window.public static void on(); // Enables NetMonitor, // sending output to the system console.public static void onConsole(); // Enables NetMonitor, sending output to the given file.public static void onFile(java.io.File file);

Sample NetMonitor Output Start: (rev 100) SRVR_CMD_GET_PROJECT_LIST  Time: 16 millis; Sent: 42 bytes; Got: 214 bytes  Start: (rev 100) SRVR_CMD_GET_PROJECT_VIEWS  Time: 16 millis; Sent: 46 bytes; Got: 335 bytes  Start: (rev 100) SRVR_CMD_PROJECT_OPEN  Time: 47 millis; Sent: 70 bytes; Got: 104 bytes  Start: (rev 100) PROJ_CMD_GET_VIEW_PROPERTIES  Time: 15 millis; Sent: 42 bytes; Got: 16983 bytes …

Sample NetMonitor Output

…  Start: (rev 100) PROJ_CMD_GET_FOLDERS  Time: 0 millis; Sent: 42 bytes; Got: 769 bytes  Start: (rev 100) PROJ_CMD_GET_FOLDER_ITEMS  Time: 31 millis; Sent: 106 bytes; Got: 620 bytes  Start: (rev 100) PROJ_CMD_REFRESH_ITEMS  Time: 125 millis; Sent: 766 bytes; Got: 36715 bytes

Item Properties

The SDK uses a simple caching mechanism to improve performance.

When an app retrieves a property, the SDK checks to see if its value has been cached.

If not, then the SDK issues a server command to fetch all of the properties of that item.

If the app is fetching properties for thousands of items, this will result in thousands of server commands.

Folder.populateNow()Retrieves properties of many items with one server

command. Parameters are:– typeName - for example, "File“– propertyNames - the property names to be

retrieved. Null retrieves all properties. – depth - Allows recursive populate.

Depth = 0 populates only items in the folder.Depth = 1 populates all items in the folder, and all

items in the folder’s immediate subfolders.Depth = -1 populates the folder and all subfolders

at any depth.

ItemList

Provides finer control over property retrieval.

App specifies a collection of items and a collection of properties.

Use this to retrieve properties of multiple historical revisions (of one item or multiple items).

Large Item Lists

There is a point at which adding more items to a PROJ_CMD_REFRESH_ITEMS command actually degrades performance.

The SDK internally may issue multiple PROJ_CMD_REFRESH_ITEMS commands for one call to populateNow().

Don’t panic; this is normal.

Identifying Unpopulated Properties

It is difficult to predict which item properties your application really needs.

When a request for a specific item property triggers a server command, a message is written to the NetMonitor output.

Message shows which property caused the fetch.

Identifying Unpopulated Properties

Start: (rev 100) PROJ_CMD_GET_FOLDER_ITEMS  Time: 16 millis; Sent: 106 bytes; Got: 620 bytes  Start: (rev 100) PROJ_CMD_REFRESH_ITEMS  Time: 125 millis; Sent: 566 bytes; Got: 5547 bytes  >> File.get("ModifiedUserID") triggers a fetch from the server.  Start: (rev 100) PROJ_CMD_GET_ITEMS_VERSIONS  Time: 47 millis; Sent: 282 bytes; Got: 779 bytes

StarTeamMPX

A framework for

publish/subscribe messaging.

StarTeamMPX

Improves performance of client apps.

Extends scalability of the server.  

Benefits for SDK apps:  – Instant Refresh– Event Handling

To use these features, the app must explicitly enable MPX.

Server.enableMPX()

// Enables MPX using the server's default client MPX profile. public void enableMPX();  // Enables MPX using the MPX profile with a given name. public void enableMPX(String strProfileName);  // Enables MPX using a given MPX profile. // The profile may be stored persistently on the StarTeam server,  // or created programmatically by the client application.  public void enableMPX(EventHandlerProfile p);

Server Constructor

New constructor: Server(ServerInfo)

ServerInfo includes an optional MPX profile name.

MPX is automatically enabled when Server.connect() is called.

If enableMPX() fails during the connect(), the MPXException is ignored. Use isMPXEnabled().

StarTeam URLs

Pass URL to StarTeamFinder.openServer(), etc.

Two forms of StarTeam URL:user : password @ address : portuser : password @ servername

No port number? Look up name in ServerList.

If the ServerInfo specifies an MPX profile, then StarTeamFinder will automatically enable MPX.

If your app uses StarTeam URLs, then it already supports MPX!

 

Instant Refresh

Instant Refresh

StarTeamMPX broadcasts information about changes to the repository through an encrypted publish/subscribe channel.  

SDK uses the information in MPX messages to keep its internal caches up-to-date.

Refresh operations serviced with little overhead.

For apps that refresh frequently, the resulting performance benefits are significant.

refresh()

// Populate all Items of a given type in the View. f = view.getRootFolder(); f.populateNow(typename, null, -1);  . . . // Refresh Items to reflect recent changes. f.refreshItems(typename, null, -1);

refresh()

In a non-MPX application, refresh could be a very expensive operation.

In an MPX-enabled application, the refresh is nearly instantaneous.

If nothing has changed, refresh() is nearly a no-op.

Important Note: refresh() modifies existing objects in place whenever possible!

isRefreshRequired()

Each refresh() has a corresponding isRefreshRequired().

Returns true if refresh() might have resulted in changes.

May return false positives.

If MPX is not enabled, always returns true.

isRefreshRequired()

// An inefficient use of isRefreshRequired().  if (f.isRefreshItemsRequired(typename, null, -1)) {      f.refreshItems(typename, null, -1);  }

// Proper use of isRefreshRequired().  if (f.isRefreshItemsRequired(typename, null, -1)) {      f.refreshItems(typename, null, -1);      doSomethingExpensive();  }

discard() vs. refresh()

// Bad. f.discardItems(typename, -1);  f.populateNow(typename, null, -1);

// Good. f.refreshItems(typename, null, -1);

discard() vs. refresh()

Never strictly correct, but harmless in non-MPX apps.

discard() always flushes the SDK's internal cache, even when MPX is enabled.

With MPX, refresh() issues fewer server commands, and will usually fetch a much smaller set of data.

Be sure to use discard() for memory management only.

copy() and isEqualTo()

refresh() modifies existing objects in place whenever possible.

Now a documented part of the APIs.

In older, non-MPX versions of the SDK, refresh() always created new objects.

Applications that incorrectly rely on this behavior will no longer work.

copy() and isEqualTo()

// Find a specific item.  item1 = view.findItem(type, id);   

 // Refresh, then find it again.  f = view.getRootFolder();  f.refreshItems(type.getName(), null, -1);   

 item2 = view.findItem(type, id);   if (myItemHasChanged(item1, item2)) {      // We never get here, because the two items     // are the same physical object!  }

copy() and isEqualTo()// Find an item and make a copy.  item1 = view.findItem(type, id);  item2 = item1.copy();     // Refresh the items.  f = view.getRootFolder();  f.refreshItems(type.getName(), null, -1);     if (!item1.isEqualTo(item2)) {      // We'll get here if the item changed      // during the refresh operation.  }

MPX Event Handling

MPX Event Handling

Subscribe to specific server eventsProvide event-handlers that are invoked

when they occur.   Supports a new class of app that would

otherwise be difficult to write and expensive to run. 

Java - listener modelCOM - connectable objects.NET - events / delegates

Event-Handling in Java Use the java listener model.

Event-handlers are abstract methods defined in an interface.

Listener interfaces extend java.util.EventListener.  

Event-handlers take a single parameter, derived from java.util.EventObject.   

Apps implement the listener interface, and register via addListener(). 

SDK implements an adapter class for each defined listener interface.   

Event-Handling in Java

server.addProjectListener(new ProjectAdapter() {      public void projectAdded(ProjectEvent event) {          Project p = event.getProject();          System.out.println("Added: " + p.getName());      }  });

Scope

Listeners are registered with an object that defines the scope of interest.

In some cases, a listener can be registered at different levels of the SDK object hierarchy.

This provides the app with a great deal of flexibility in defining the scope.

Scope: View

ItemListener myListener = new ItemAdapter() {      public void itemAdded(ItemEvent event) {          System.out.println("Item Added.");      }  };

view.addItemListener(myListener, type);

Scope: Folder(s)

ItemListener myListener = new ItemAdapter() {      public void itemAdded(ItemEvent event) {          System.out.println("Item Added.");      }  };

folder.addItemListener(myListener, type, depth);

Scope: ItemListManager

ItemListener myListener = new ItemAdapter() {      public void itemAdded(ItemEvent event) {          System.out.println("Item Added.");      }  };

itemListManager.addItemListener(myListener);

Security

Event-handlers are always attached to an object that defines the scope of interest.

The scope also defines the security context.

There is no way for an app to retrieve any data via an event-handler that it would not normally be able to retrieve outside the event-handler in the same security context.  

Item Listeners

In some cases, the SDK defines a listener interface that is itself a marker interface.

Multiple listener interfaces that extend the marker interface are supported.  

The most important example: IItemListener.

IItemListeners

ItemListener: Provides the most detailed information about new, moved, changed and removed Items.  Before and after items provided.

ItemListListener: When item list changes, provides the item type and parent folder.

ItemIDListener: Similar to ItemListener, provides only an ItemID instead of an item. The app can call View.getDisembodiedItem(), View.findItem() and so on.

NotificationListener: Provides "notifications" from the server.  

Writing Event Handlers APIs designed to make it as easy as possible for an

app to write event handlers. Event objects always describe events using SDK

objects that StarTeam developers are already familiar with.  

Very few restrictions on what can be done in an event handler.

For example: for ItemEvents …– Item is always a fully functional Item object. – Item is attached to a valid parent Folder.– Folder is a member of a valid Folder tree.– Attached to a valid View.

Example 1

What if new Folders are added to the View?What if new Items are then added to the new Folders?

It is the SDK's responsibility to ensure that it works.The app doesn’t do anything special.

view.addItemListener(new ItemAdapter() {      public void itemAdded(ItemEvent event) {          System.out.println("Item Added.");  }, type);

Example 2

Is there a small window of time where the Project exists, without a default View?

Can the app safely access the default View and its root Folder in the projectAdded() event handler? 

Again, it is the SDK's responsibility to make it work.

server.addProjectListener(new ProjectAdapter() {      public void projectAdded(ProjectEvent event) {          Project p = event.getProject();          Folder f = p.getDefaultView().getRootFolder();          System.out.println("Root folder: " + f.getName());  }, type);

Threading Model Handlers invoked from a separate thread.   One thread per Server object.   Events are queued, and handled

asynchronously in the order they are received.

The precise order of closely timed events is not completely predictable.  

A poorly written event-handler will block other events, but will not break the SDK.

Threading Model

For convenience: Server.handleEvents()

Places the current thread in an event-handling state.

The thread wakes occasionally to ping the server (keeping the connection alive).

The event-handling loop continues until Server.interruptHandleEvents().

Design Considerations

Is this an appropriate use of the event-handling APIs?

view.addItemListener(new ItemAdapter() {      public void itemAdded(ItemEvent event) {          Folder f = view.getRootFolder();          f.refreshItems(type.getName(), null, -1);  }, type);

Design Considerations

The event-handling mechanisms maintain their own state information.   

Internally, the SDK has its own event-handlers, implemented at a lower level and wired directly into the SDK's caching mechanisms.

These are running whether or not the app has registered its own event handlers.  

Design Considerations

Alternative approaches:– Invoke Folder.refreshItems() invoke it

periodically in the idle loop, it on a timer, etc.  

– Implement ItemListListener instead of ItemListener.  

Carefully consider design options.

Consider prototyping.

Event-Handling in COM

Connectable object: an object that exposes events (connection points).

The app implements event handlers, and registers them by connecting the object to its event handlers.  

Exactly how an object is connected to its event handlers varies from one programming environment to another.

Example: Visual Basic

Dim WithEvents myWidget As Widget

Private Sub myWidget_changed()  . . .  End Sub

Restrictions of COM

The connectable object implies the scope of interest.

When the event handlers are connected, there is no way to provide parameters to refine the scope.  

Each connectable object supports a single event interface (the default source dspinterface).

Event Sources

One EventSource type in COM for each listener interface in Java.

EventSources created by factory methods defined in the relevant parent class.

Factory methods have parameters to refine the scope of interest.

A class can have one factory method for each type of EventSource.

Example 1: ItemEventSource

Private Sub source_itemAdded(ByVal arg As IStItemEvent)      . . .  End Sub

Dim WithEvents source As StItemEventSource  Set source = view.newItemEventSource(type)

Example 2: Folder Scope

Dim WithEvents source As StItemEventSource  Set source = folder.newItemEventSource(type, depth)

Private Sub source_itemAdded(ByVal arg As IStItemEvent)      . . .  End Sub

Example 3: ItemListEventSource

Dim WithEvents source2 As StItemListEventSource  Set source2 = folder.newItemListEventSource(type, depth)

Private Sub source2_itemsChanged( ByVal arg As IStItemListEvent)      . . .  End Sub

Event Handlers in COM

COM or Java: same principles apply.

Use the same SDK object model.

No significant constraints that limit what can be done within an event-handler.

It is the SDK's responsibility to make it work.

Event-Handling in .NET

Events are first-class entities.

Any object can expose events.

Client event handler is a delegate.

C# language syntax for attaching a delegate to an event.

Restrictions of .NET

.NET has many of the same restrictions as COM.Object exposing the event implies the scope of

interest. No syntactic mechanism to provide parameters that

refine the scope.   StarTeam event-handling APIs in .NET use

EventSource objects similar to those used in COM.

EventSource defines a delegate type that specifies the calling conventions of the corresponding event handlers.

Example: OnItemAdded

void ListenForFilesAdded(View view) {      Server s = view.Server;      Type type = s.TypeForName(s.TypeNames.FILE);       // Create an event source via a factory method on the view.     // The view and item type define the scope of interest.     ItemEventSource source = view.NewItemEventSource(type);       // Attach an application event handler.     source.OnItemAdded += new ItemEventSource.Handler(OnFileAdded);  }

Example: OnItemAdded

// The event handler. private void OnFileAdded( ItemEventSource source, ItemEventArgs args ) {      File file = (File)args.NewItem;      Console.WriteLine("File Added: " + file.Name);  }

Performance Tuning Revisited

Performance Tuning Revisited

NetMonitor used to identify server commands that are performance bottlenecks.

What if the NetMonitor output shows server commands that you do not recognize?

Often difficult to deduce which API call resulted in a particular server command.

Even more difficult in an event-handling app.

Command may be from an event handler, responding to an event initiated by an app on some other workstation.

Events in NetMonitor

New in StarTeam SDK 2005.

NetMonitor provides event-handling APIs.

App can listen for server command events.

Server command event handlers are triggered synchronously, from within the thread that issued the command.

Use debugger to examine the call stack.

ServerCommandEventsNetMonitor.addServerCommandListener(     new ServerCommandAdapter() {         public void end(ServerCommandEvent event) {             String name = event.getCommandName();             long time = event.getCommandTime();             if (name.equals("PROJ_CMD_REFRESH_ITEMS")) {                 if (time > 250) {                     System.out.println("Found the bottleneck!");                 }             }         }     } );

Questions?

Thank You

4126

StarTeam SDK Advanced Programming Topics

Please fill out the speaker evaluation

You can contact me further at …Ron.Sauers@borland.com