Customization Tips and Techniques for Administering MicroStrategy Narrowcast Server

Bryan BrandowSr. MicroStrategy DeveloperBluegreen Corp.bryan.brandow@bluegreencorp.com

January 16th, 2008

About Bluegreen

History

Founded in 1966 in Massachusetts, the company’s core business wasbuying, improving, selling and financing recreational land in more than32 states. In 1986 the company was listed on the New York StockExchange and in 1991 it was relocated to Boca Raton, Fla.

Bluegreen Corporation (NYSE:BXG) is a leading provider of Colorful Places to Live and Play®. Headquartered in Boca Raton, Fla., the company has over 6,000 associates working in its Bluegreen ResortsSM and Bluegreen Communities® divisions.

Bluegreen Resorts markets and sells a flexible, real estate-based vacation ownership plan with access to over 40 resorts and an exchange network of over 3,700 resorts and other leisure products such as cruises and hotel stays. In 2006 Bluegreen Resorts delivered nearly 300,000 unique vacation experiences to over 170,000 owners.

Overview

Where We Came From

• Static web based reports executing from copies of the transactional tables would take 15-30mins and upwards of 2-3hrs for yearly reports.

• Delays in the queue would start in the morning and on busy report days, a request could take hours to be fulfilled.

• In order to be prepared for important Monday meetings, users had to come into the office earlier and earlier to beat the morning report rush.• Users had to print large binders of reports and prepare them for daily and weekly usage by Site Directors.

• No real data cleansing processes.

New System Requirements

Our users needed a more scalable and flexible tool, one that could support our growing company’s report and analysis demands.

Old New

• Slow execution (minutes to hours)• Static report.

• Fast execution (seconds, regardless of load)• More flexible in prompts and drill downs.• Scheduling

At Bluegreen, we exclusively use the Web Deliveries Integration as all subscriptions are created from pre-existing reports that users run or create themselves in MicroStrategy Web. This provides users with a single interface to execute reports and manage their subscriptions.

Web Deliveries is a relatively new feature, debuting in 8.0. During our implementation and roll out, we encountered some challenges. This presentation takes you on a tour of some of the things we faced and our solutions to overcome them.

Narrowcast at Bluegreen

Where Did This Come From?

Problem:

Users would receive subscriptions and have questions about them but not know who to ask. They would ask IT, but we didn’t have a way to track an email back to a subscription. Users were confused and IT was frustrated!

Solution:

Adding Information Objects for the owner of the subscription and the schedule that it is associated with will help in the debugging process and let the users know who to contact if they have questions.

Smart Reply To’s

Continued:

We also modified the emails sent from Narrowcast to use a dynamic Reply To. The users would see the normal “Business Intelligence” address as the sender, but if they clicked “Reply”, it would magically change to the owner of that subscription.

• Email is received from the standard company distribution box.

• Clicking reply replaces the name with the user who created the subscription.

Exception Reporting

Problem:

Some users have created exception based reports to email them when certain criteria are met. However, an email is sent regardless of report content and the user is given a daily “No Data Returned” email.

Solution:

Edit the Information Object for the report data and either change the message to something nicer or don’t send an email at all.

Timeouts!

Problem:

As our subscription base got higher, so did the load on our server. Some services started to struggle to finish and timeout.

Quick Narrowcast Lesson

Service

Execution

Execution

SegmentsSegments

All of the information needed to process the subscription (Who, What, When).

Individual instance that a Service is executed.

Subset of subscriptions broken out for each execution for use in load balancing between multiple Execution Engines.

Timeouts!

Problem:

As our subscription base got higher, so did the load on our server. Some services started to struggle to finish and timeout.

Solution:

Proper usage of the Segment Size, Execution Timeout and Work Allocation settings helped improve system stability and performance.

Administrative Notifications

Be the First to Know!

• Select either all notifications or specific ones. Careful though, sometimes errors that are logged aren’t necessarily important ones! Some are simply optional pieces of a subscription that a user chose to leave blank.

• Here, you can customize your notifications. Just put in your email address, select the section that sounds like what you’re after, and click the Governing and Filtering button.

What Really Happened …

At this point in the project, we found that we were mostly in the dark about what was happening. We employed several different solutions to get a better grasp on what was happening …

Problem:

Users would claim that they didn’t receive a subscription, or there was a problem with the report. How are we to know what really happened?

Solution:

• This is also a great way to do debugging in Production. If there is a service giving you problems, but you don’t want to spam the users, change this setting to “Save to File” and the users won’t receive any emails while you test!

Saved Mail – A Life Saver!Saved Mail

This setting on the Information Transmitter will save a copy of every email that it sends. This can be a life saver for debugging what’s going on. Just remember to periodically clean it out because it gets very large!

Spy on the Inbox

•Setup a dedicated email box for BI reports to be sent from, and add it to your outlook so that you can see its return messages.

• You’ll be able to see what happens to emails AFTER they’ve left the hands of Narrowcast. For example, if someone’s email box is full or their account is misspelled or non-existent.

Log Files …

• Be sure to edit your log governing settings! By default, they are 5mb and 10 files, that’s not enough!

• Log files hold the keys to every mystery. Be sure to enable additional pertinent information so that it is available when needed.

Oh, there’s the problem …

Narrowcast Log Analyzer

• NCLog is a Visual Basic.NET application written to help parse and analyze Narrowcast log files.

• Instead of the normal log file being stored chronologically, NCLog allows you to browse the file logically.

• Predefined user alerts are flagged with a red bulb and bubbled up to the top, indicating that a particular service contained an alert. You can now drill into a service and see only information related to the service you are interested in.

Narrowcast Log Analyzer – Demo!

Narrowcast Log Analyzer – Other Toys

• Copy the full log file line to clipboard.• Export this branch to file.

• Dynamic grouping options that are pulled from the log file.

• Ignore List throws out any line that contains one of these key words.

• Filter List to only include lines that contain these keywords.

• Alert List to highlight with the red bulb what you want to jump out.

I Need More Information!

Problem:

Even with all of the new initiatives, proactive support was still difficult:

• We were relying on mining the log files to see what was happening each day.

• We were unable to link a subscription to the physical report, which was causing a lot of problems when trying to troubleshoot subscriptions that were having problems.

• We had no idea how many people were using the system aside from counting the number of emails in the Saved Mail folder.

We needed even more information!

Narrowcast Transmission Recording

• This feature will create tables in your Narrowcast Metadata repository that log everything that Narrowcast does. This includes details on every subscription that is executed.

TN6200-072-0277

Enterprise Manager Integration

• For exception reporting and analysis, we need to extend the Enterprise Manager project to include the Narrowcast Transmission Data.

• With this you can do analysis on what has been happening with Narrowcast and create exception reports that will either email on occurrence or just give daily status updates.

Narrowcast Schema

This isn’t a comprehensive schema and only includes the items that were necessary for our project. You could also easily replace a complicated data model with free form SQL reports written to serve specific purposes.

Reports - Execution Log

This report is able to perform a lookup to the iServer metadata and retrieve the name of the Report that is subscribed to. This is information that is not available in any Narrowcast Log files or the Administration console. It is invaluable for tracking down errors.

Reports – Subscription Log

This report includes the Title that the user gave when setting up the subscription. This can help to differentiate between multiple subscriptions with the same name. This information is mined out of the Subscription Preferences XML stored in the Subscription Book Repository.

Reports – Other Reports

These are some additional reports that were created in this project that are used daily.

Reports – Other Reports

Narrowcast Status Report

Daily quick overview of how many subscriptions ran yesterday.

Reports – Other Reports

Narrowcast Daily Failures

A list of subscriptions that failed to deliver today, usually due to excessive file size or incorrect email addresses.

Reports – Other Reports

Deleted Narrowcast Users

Users that were deleted from iServer but still have active subscriptions in Narrowcast.

Reports – Other Reports

Subscriptions to Deleted Objects

Users that have subscriptions to reports that no longer exist.

Timeouts 2: The Sequel

Problem:

We started to have timeout problems again, but this time no amount of teasing the execution settings was helping to resolve it. Our project had grown and since our schedules required that all subscriptions run at the same time, this resulted in ~1,000 jobs trying to run at the same time and bottlenecking.

Solution:

Staggering the services seemed to help resolve the issue by lightening the load at any one time. But this provided another problem … if we stagger the schedules, and ETL runs long, then we are sending out subscriptions before the warehouse is finished loading!

Narrowcast Triggers

NCTriggers is a command line program that utilizes MicroStrategy Command Manager to remotely execute Narrowcast schedules. Subscriptions are executed based on timing built into the application and called by a completed ETL system.

The business need for this application is to stagger the execution of subscription sets so that they aren’t all run at the same time. This eases the load on Narrowcast server and gives you full control over when the subscriptions are executed.

Narrowcast Triggers - Components

An XML file is passed as a command line property to the program to support multiple instances from different projects. The file contains the Command Manager SCP script file which has the actual triggers to execute the Narrowcast service, the number of minutes it should wait before executing the next item, and the schedule it should run on.

Narrowcast Triggers – Log Files

The left file is an example of the program’s log that is created. This helps in checking later in the morning what it tried to run. The right file is the standard Command Manager output text which shows each service that is executed.

Narrowcast Triggers – Notification

When NCTriggers is finished executing a job, it will scan the Command Manager output file for any errors. If any are found, an Email notification is sent to the administrative email address with the text.

Takeaways

• Knowledge is Power: Proactive alerts and solving issues before they’re real issues. Build a system that will call for help when it needs it.

• The Job is Never Done: There is always more to tweak, stay on top of, and analyze when it comes to a growing environment. Either by analyzing common table hits to find aggregate opportunities or improving ways that you troubleshoot and diagnose an issue, there is always room for improvement.

“If it’s worth doing, it’s worth building a tool to do it.”

• A Happy System is a Stable System: Make sure the proper configurations are applied for your environment and monitor that everything is executing when it should and in the time frame it should.

If you only learned three things:

Thanks for Watching

Bryan BrandowSr. MicroStrategy DeveloperBluegreen Corp.bryan.brandow@bluegreencorp.com

January 16th, 2008

www.bluegreencorp.com