cognos content store survival guide

T H E C O M P L E T E M A N U A LT O S U R V I V E A N D M A N A G E T H E I B M C O G N O S C O N T E N T S T O R E

C O N T E N T S T O R E S U R V I V A L G U I D E ™ 2 of 22

All trademarks mentioned herein are the property of their respective owners. ©2011 Envisn Information Solutions. All rights reserved.

T a b l e o f C o n t e n t sEXECUTIVE SUMMARY

THE ROLE OF THE IBM COGNOS STORE

CONTENT STORE TABLE STRUCTURE AND ACCESS

TOP 10 KEY METRICS FOR CONTENT STORE ADMINISTRATION

MANAGING CONTENT STORE GROWTH

MANAGING MULTIPLE CONTENT STORES

CONSOLIDATING AND MERGING COGNOS BI ENVIRONMENTS

3

4

6

8

12

14

19

I N T R O D U C T I O NIn the IBM Cognos world nothing is more shrouded in mystery than the Content Store. While every Cognos administrator knows its importance to the Business Intelligence (BI) environment, few completely understand its role in the BI delivery process.

The least understood aspect of the Content Store is that it must be managed if the BI environment is going to remain healthy over time. This survival guide provides working knowledge about the IBM Cognos Content Store and its management.



E x e c u t i v e S u m m a r y

The IBM Cognos Content Store acts as a repository for both content and metadata for the BI environment.

Managing it presents some challenges particularly as the environment evolves over time and through different stages of growth. Administrators must develop and pay attention to key metrics of each Content Store and successfully manage them. The larger the Cognos environment gets the more complex the administration becomes, and while the magnitude of problems does not always scale along with size, it does often enough to cause real pain and sometimes disastrous results.

The lack of tools can contribute to administration issues. Off-the-shelf solutions are available to address this and can greatly aid in administering the Content Store. In addition, there are several key metrics that, when tracked appropriately, decrease the administration effort including that of growth control. The consistent use of best practices mitigates much of the risk of administering and managing Content Store environments primarily because when problem issues are addressed early their impact can be reduced or eliminated. This survival guide will help you identify what you need to manage and what best

practices might be appropriate for you and your installation.

So what are the keys to successfully managing the Content Store over time? The most obvious ones are:

Effective user training. The BI user community must be trained on how to effectively utilize the IBM Cognos BI tools and best practices.

Administrators need to track and manage key metrics that relate to their user base and the Content Store.

The use of effective tools by the administrators to:

Manage change – lifecycle managementManage contentIdentify and address key issuesManage security



T h e R o l e o f t h e I B M C o g n o s C o n t e n t S t o r eAt a recent IBM Cognos session focused on the Content Store, over 100 attendees were polled to determine the level of familiarity with the Content Store. Less than 1 in 5 had any working knowledge of Content Store functions or the amount of content it contained. This is not a surprise since for most users the Content Store is simply a part of the Cognos environment.

The Cognos Content Store has two essential roles: 1. To act as a repository for Cognos BI metadata. 2. To act as a storehouse for user output.

In other words, the Content Store stores everything about every object in Cognos Connection. In the Figure 1 graphic where we say that the metadata in the Content Store is ‘mostly static’ it’s important to understand that this is a relative term. It’s relative to saved output that typically grows in size more than it changes.



Objects in the Content Store are always changing as new objects are added, objects are deleted and objects are changed. Even a simple report can have multiple attributes. And these attributes have sub-attributes which also have sub-attributes, etc. A recent count we did came up with nearly 400 attributes for a report across different levels. And many of these are things that relate to jobs, schedules, triggers, etc. Many of these attributes can be important at different times for administrators and being able to access data about them can be critical. Knowing which jobs have triggers if some reports are not executing on a timely basis is something administrators need to know about.

The Content Store is expressly designed and optimized for high speed reporting. However, it does not provide administrators with many tools to effectively manage the Cognos BI environment.

The following information can be difficult to obtain without specialized tools:

Size of the Content StoreMake-up of object mixAmount of saved outputRate of growthAreas of growth

Unfortunately, for many Cognos administrators the lack of tools to determine these metrics can go unrecognized until problems occur.

While many of these numbers can be retrieved with individual queries into the Content Store, it is an inefficient process to use over time. There are no IBM Cognos tools for managing or monitoring metadata such as dependencies and relationships on a large scale. It is also difficult to find and use information to easily assess the impact of database changes on reports, queries, or report version model. In addition, there are many other activities that must be addressed by Cognos administrators, authors, or developers.

There are two ways to address this issue:

1. Build a solution in-house.2. Consider an off-the-shelf solution like NetVisn

Option 2 may be the only practical way to provide the necessary tools to administrators if the financial resources are available. Option 1 requires time, resources, and gets messy over time when new versions of Cognos BI are released.

http://www.envisn.com/products/netvisn/




C o n t e n t S t o r e T a b l e S t r u c t u r e a n d A c c e s s

The metadata in the Content Store is organized in a set of database tables. At last count there were approximately 200 of these. The tables are created when the Cognos Configuration Manager is implemented for the first time. As users are added to the environment and more content is created the Content Store begins to grow in size.

Again, this often goes unnoticed until a problem occurs. As mentioned in the previous section, there are no simple or convenient processes to retrieve basic metrics for size, item count, speed of growth and others. Direct queries into the Cognos Content Store can retrieve this information but this is tedious and time consuming.

How large is a typical Content Store? It depends on the age of the environment, user population, number of objects stored, and many more variables. For example, a Cognos environment of 800 to 1,000 users that has been in place for a year or two may have 12,000 to 20,000 objects or more. Each of these objects can have multiple (20+) properties that relate to security, schedules, jobs, and other properties, increasing the level of complexity.



A basic tool to get data out of the Content Store is adequate but to address issues such as dependencies, relationships, permissions, and changes a more systematic approach is needed.

The Content Manager Browser Tool is available for download on the IBM Cognos web site and can be used by administrators to access this information.

This tool provides some basic, though limited, capabilities:

Summary of the number and size of the various objects in a simple display format.

Ability to query folders and types of objects.

In order to see meaningful trends, the user would have to perform these steps on a regular basis and then build a basic data repository to monitor the information over time. While this approach will certainly work it provides only limited information. Determining why and where the trends are occurring adds another level of complexity.

While there are other ways to directly query these tables for the occasional data need, building ad hoc tools for this can have only limited success:

IBM Cognos reserves the right to change these tables and their structure at any time. Query routines or applications may be incompatible with new versions.

IBM Cognos recommends using their SDK Validation Tool for systematic access to the Content Store.

As mentioned earlier in this guide, a robust third party solution is a better alternative for broad and deep BI administration tools.

http://www.ibm.com/developerworks/data/library/techarticle/dm-0808masters/index.html

http://www.ibm.com/developerworks/data/library/techarticle/dm-0808masters/index.html

http://www.ibm.com/developerworks/data/library/techarticle/dm-0808sleigh/index.html






T o p 1 0 K e y M e t r i c s f o r C o n t e n t S t o r e A d m i n i s t r a t i o n

Managing the Cognos Content Store starts with knowing what’s in it, how large it is, how fast it’s growing, and where the growth is taking place. The metrics discussed here may not be appropriate for all environments. But it is a place to start the process of aligning the Content Store with the BI strategy and plans.

These are the key metrics that administrators should monitor in terms of production of the Cognos Content Store. Many can directly impact overall performance and user satisfaction.

1. SIZEUnimportant as a single metric.

Best used in conjunction with other metrics.

2. RATE OF GROWTHTrack in terms of number of objects as well as saved output.

More critical in environments that are large and/or growing rapidly.

Can be a key indicator of real or potential problems in the Content Store.

Tracking growth should be done at least monthly.



3. NUMBER OF OBJECTS BY TYPEUseful metric for determining the mix of stored objects.

It’s helpful to track the relationship between the number of reports and queries relative to the number of packages or models over time.

Best Practice: Create specific packages for individual areas such as sales, logistics, and marketing instead of relying on one or two very large packages for everything.

Example: Figure 1 shows an object count summary by type for a sample Production Content Store.

Figure 1: Object Summary Count



4. SAVED OUTPUTAmount of saved output in mid size or large environments can be significantly larger in terms of file size rather than the number of objects.

Users may be saving output from very large reports over time.

Best Practice: Have a policy in place for saving output.

Example: Figure 2 is showing total saved output and the detail for Sales folder in a sample Cognos Connection.

5. EXCESSIVE REPORT CREATIONIndicates users may not understand bursting or prompts.

Figure 2: Saved Output Detail



6. VERSION CREEPKeeping versions of models, packages, or other objects far beyond the needed time period can result in poor performance.

Turn off versioning if it’s not needed.

7. MISSING OBJECTSCan result in a null response when users attempt to run or receive a scheduled report.

Examples:

• User creates a view of a report but report is somehow deleted.

• Distribution list mistakenly deleted and the users do not receive reports.

8. UNUSED OBJECTSTrack usage of both content and users across the Cognos environment.

Content that has not been used for a period of time (2 – 3 months or more) should be a candidate for removal.

Purging public folders of unused content makes searches easier.

Unused user licenses can be reassigned to new users.

9. OBJECTS WITH BROKEN LINEAGELineage breaks occur when a package is modified and published without consideration for reports or queries using it.

Lineage breaks also occur when a data item(s) no longer exists.

At any given time the percentage of objects with broken lineage in a typical production Content Store is 10% to 15%.

Identification and resolution of objects with broken lineage before users are impacted reduces calls to the help desk and the number of frustrated users.

10. DUPLICATE OBJECTSIdentifying two or more copies of the same object across the environment may indicate a need for better user training.



In the five plus years that IBM Cognos 8 and 10 have been available there have been a number of trends underway in terms of how it is used. The most interesting one is the dramatic increase in the amount of overall content in the Cognos environment and the accompanying increase in the size of the Content Store. It is not uncommon to see Content Stores today that are over 100 Gb in size.

Causes of Dramatic GrowthBack in the days of Cognos Series 7, and even into the early days of ReportNet and Cognos 8, many Cognos BI installations developed a standard suite of reports intended to cover everything they thought the users required for reporting. Reports were created by in-house or third party developers for marketing, sales,

finance, and other departments or groups. This strategy really did not work too well. Users needs, never clearly defined at the beginning, changed over time along with the needs of the business.

Users began creating their own reports to more accurately fit their requirements. However, uneven and ineffective user training began to cause reporting problems. As advanced as BI tools are today, it is still a challenge for users to create the reports they need, especially if creation involves the proper use of prompts and/or bursting. Users will often experiment in creating reports until a satisfactory version is developed, but saving output from preceding versions can quickly increase growth and eventually impact performance.

In Cognos 8 and 10 users have personal folders (My Folders) to create, schedule, and share their own reports. They can also create views on public reports to develop a custom version of a standard report. This, plus a lack of rules on saving report output also increases content rapidly. All of this has resulted in an explosion of content over time. In large Content Stores the number of Cognos objects such as reports, models, and packages, is typically dwarfed by the amount of saved output. It is not unusual for a 100 Gb Content Store to be comprised of 20 Gb in Cognos objects and 80 Gb in saved output.

M a n a g i n g C o n t e n t S t o r e G r o w t h



Controlling GrowthHere are specific practices to keep growth manageable:

Monitor Content Store growth on a regular basis using the IBM Cognos utility or a product like NetVisn that will provide broad insight into causes of growth.

Training Users - with emphasis on best practices

Administrators and modelers need to know how growth issues can occur.

•Example: inadvertently or intentionally turning on package versioning on large models.

Set and enforce retention policies

• Set rules on saved output retention, both amount and length of time.• Obtain user input before setting rules to balance needs across the user base.

Archive content that must be retained

Delete orphan content

• In any large environment users are being added and deleted on a daily basis.

• Ensure that content from deleted users is also deleted

A large Content Store in and of itself is not necessarily cause for alarm. But size can become a problem when it begins to affect performance. For example, a user backs up a very large model each week to a different file location thinking he’s doing the right thing. It soon has a noticeable impact on overall performance. If users do not understand the impact of doing similar things, the Content Store is going to grow but not perform as needed. Growth should be organic as the result of new users and new content focused on real user needs.





Multiple Content Stores also have a role in the typical IBM Cognos Business Intelligence installation. There are similarities and differences between store types and usage depending on the organization.

As customers move to take advantage of what’s new in Cognos 10, the role of the Content Store is even more critical. It has more capabilities and more users relying on it. Most Cognos installations will typically have three distinct environments:

Production: Deliver BI content to end users.

Test or Pre-Production: Validate the functional readiness of new or changed content.

Development: Develop new content or modify existing content.

Some installations may have additional environments in the BI delivery chain; some may have fewer. In small to mid-size installations, for example, it is not uncommon to see only a production and a development environment where development also serves as the test environment.

There may only be a production environment. In this case both Development and Testing co-exist with production in

a single environment. It should be noted that this is a high risk practice and is not recommended. Content that has a different purpose at a given point in time should be in a different environment. Using a single environment for multiple activities significantly increases the likelihood of problems with security, data sources, and more.

Best PracticesThere are some themes that run through the best practices of all Content Store environments:

Clearly defined policies and procedures in place:

What data are used for each Content Store.

Security profiles for each environment are in place

Rules defined for when content is ready to be moved.

Deployment history of all content moved from one environment to the next. This needs to be a complete record of what transpired. (who, what, when)

M a n a g i n g M u l t i p l e C o n t e n t S t o r e s



Tools should be available and enable administrators to work both in and across all environments simultaneously.

Monitoring of key metrics:

May differ from one installation to another

Must include:

•Usage by content and users•Growth in content and users

Production Content StoreIn a production environment there is no room for error. When there are hundreds or thousands of users there must be policies and procedures in place which control working in the production environment.

Production Best Practices:Organize content for easy user access

Limit folder depth and number of clicks required for content access.

Clearly label content.

Create a clean structure for easy navigation.

Develop criteria for production ready content.

Create rules for managing My Folders.

Mandate and routinely test backup and recovery procedures.

Manage and validate security in real-time.

Validate data sources on an on-going basis.

Remove unused content and excess output.

Tools in place for administering the environment.

Implement schedules, triggers, distribution lists, etc.

Monitor content and user usage on an on-going basis



Test or Pre-Production Content Store Best Practices:The Test or Pre-production environment is similar to production but there are differences.

Security should represent the production environment.

Use active rather than static data sources.

Ability to compare to production environment for:•Security •Metadata•Data sources•Content counts (deleting old or unneeded test content)

Develop test criteria for production readiness.

Keep history of completed testing.


Maintain a history of items moved to production (including who-what-when). See Figure 3.

Practice content version control.

Make change management a major focus.

Mandate and routinely test backup and recovery procedures.

Track usage (Is the content really being tested?)

Figure 3: Log File of Content Promoted to Production



Development Content StoreThe development Content Store is where all new content is authored or created.

Development Best Practices:Security model should reflect only the development environment.

Design policies and procedures specifically for development.

Data sources are typically static but representative of production.

Determine criteria for moving content to Test or Pre-production.

Save a history of what has been moved to Test or Pre-production.

Design folder structure to meet author/developer requirements.


Tools in place to assist authors/developers in diagnosing problems in the development content. See Figure 4.

Make change management a major focus.

Figure 4: Broken Data Lineage



• Security should reflect Dev only

• Static data sources but represent Production

• Structure & use for Dev only and designed for authors/developers

• Readiness criteria in place before moving content to Test

• Metadata management & diagnostic tools in place

• Automated logging of content moved to Test

• Version control in place

• Security should represent Production environment

• Use active data sources

• Criteria in place for moving content to production

• Automated logging of content moved to Production


• Backup & recovery procedures in place & tested regularly

• Regular validation that content is actually being tested

• Metadata management & diagnostic tools in place

• Organize content for easy user access:

• Limit folder depth• Clearly label content

• Manage & validate security real time

• Validate data sources on a regular basis

• Rules in place for My Folders

• Metadata management tools in place

• Remove unused content and excess output

• Implement schedules, triggers, distrib. lists, etc.


C o n t e n t S t o r e s B e s t P r a c t i c e s



Merging or consolidating Cognos environments can be a challenge even in a small environment. In a large environment the difficulties are magnified. Creating a plan that reflects the specific needs when merging environments is essential. Using tools that make the task easier and minimize risk can help insure success and reduce the time and effort involved.

Managers or administrators are often faced with the task of consolidating or merging Cognos Business Intelligence environments. This may be due to mergers, a need for cost savings, or simply a desire to reduce the number of environments. Invariably the consolidation involves users and content that will be merged into a new or existing environment. This requires careful planning and flawless execution to succeed.

Merging environments is a major project. It involves merging one existing environment into another or merging two or more environments into a new one. An example would be merging two Cognos 8 environments into a new Cognos 10 environment.

Step 1: Determine Scope of MergerDetermine the scope of the environments to be merged. This requires detailed analyses to discover the data needed for planning. Needed information includes:

Extent of similarities and dissimilarities between environments.

Number of users.

Amount of content.

Duplication of users and/or content between environments.

Differences in security model (if any) between source and target.

Identify and eliminate unused content if possible.

Identify special issues depending on the specific needs of the environments.

C o n s o l i d a t i n g a n d M e r g i n g C o g n o s B u s i n e s s I n t e l l i g e n c e E n v i r o n m e n t s



If the merger involves more than two environments this information is required for all environments under consideration. In many respects the task will be significantly easier if there is no overlap in users and content between the environments to be merged.

Separate and Distinct Environments: Source and target environments that are dissimilar and have no overlap reduce the tasks involved. See adjacent:

Similar or Overlapping Environments: When the source and target environments are similar there are more tasks to address before merging the two environments.

Here both source and target are similar raising the possibility for multiple issues on common data sources, folders, and models. Identifying and resolving these issues is absolutely necessary for a successful merger.

Step 2: Planning and ExecutionAfter determining the scope of the merger using the information gleaned from the environments, identify and layout the key tasks in the right sequence. Most problems in a merger occur when key steps are missed or performed in the wrong order. Planning includes:

Collect required information in detail.

Decide how to handle common issues.

Decide how to handle unused content.

Modify (if necessary) the security model in the target environment.

Map the tasks to the timeline.

Review the plan with key constituencies.

Testing and validation.

Communicate the plan to users and set expectations.

Execute the plan.



To avoid a messy deployment you can use a product like NetVisn to perform the merger in a more controlled manner.

Advantages:Allows content to move from the source to the target with known outcomes.

Greatly reduces risk if hundreds or thousands of users must be moved.

Infinite level of selectivity around objects: reports, personal folders and content, etc.

Validates all objects as they are being moved to insure that all dependent objects are moved together and will work in the new environment.

Minimizes or eliminates any orphan content.

Alternatively you can use a deployment package instead, but this is an “all or nothing” approach that can instill fear in even the most careful planners. With a deployment package there is no selectivity below the folder level and personal reports cannot be dealt with at all unless you take all of the personal reports that are in the Content Store. But for many this may be the only alternative.

Remember, planning is the single most important part of the entire effort in merging or consolidating environments. The time you spend here has a huge payback in terms or risk avoidance and minimizing problems.





©Envisn Information Solutionswww.envisn.com

Envisn's IBM Cognos Blog

233 Ayer RoadHarvard, MA 01451 USA

T: 978-779-0400F: 978-772-0985

E: [email protected]

S U M M A R YThe IBM Cognos Content Store plays a vital role in the administration and productivity of the Cognos Business Intelligence Solution. In a well managed installation with a focus on best practices, and the right tools to work with, the Content Store(s) can function flawlessly. It does not require constant attention but there are some things you need to pay attention to on a regular basis. If you do this you can eliminate pain and risk from your installation and focus on giving your BI users what they need with high uptime.

FB TW LIN RSS

http://www.facebook.com/pages/Envisn/406857649128


http://twitter.com/%23!/Envisn


http://www.linkedin.com/pub/rick-ryan/14/250/a9


http://feeds.feedburner.com/EnvisnsIbmCognosBlog


http://www.envisn.com

http://www.envisn.com

http://www.envisn.com/envisn-cognos-blog/

http://www.envisn.com/envisn-cognos-blog/

mailto:[email protected]?subject=Content%20Store%20Survival%20Guide

mailto:[email protected]?subject=Content%20Store%20Survival%20Guide









cognos content store survival guide

Documents