integrating sharepoint with documentum - dell€¦ · 2010 emc proven professional knowledge...
TRANSCRIPT
EMC Proven Professional Knowledge Sharing 2010
Integrating SharePoint with Documentum
Rami A Al Ghanim and Sara A Al Mansour
Rami A Al GhanimSystems AnalystSaudi [email protected]
Sara A Al MansourSystems AnalystSaudi [email protected]
2010 EMC Proven Professional Knowledge Sharing 2
Table of Contents Introduction ................................................................................................................................... 3
Company Overview ....................................................................................................................... 4
History of Enterprise Content Management in Saudi Aramco IT, Dhahran ............................... 5
Our Journey with the Integration ................................................................................................... 8
How it started ............................................................................................................................ 8
EMC Documentum Repository Services for Microsoft SharePoint (EDRSMS) ....................... 10
Challenges .................................................................................................................................. 14
Future Goals and Integration Road Map ..................................................................................... 15
Disclaimer: The views, processes, or methodologies published in this compilation are those of the authors. They do not
necessarily reflect EMC Corporation’s views, processes, or methodologies.
2010 EMC Proven Professional Knowledge Sharing 3
Introduction When users browse any Enterprise Content Management (ECM) system, they have certain
expectations of where, what, when, and how to find the right information in the right time. Users
expect that a number of employees can work together to manage, version, edit, and store
documents. Information access is controlled, based on user roles. Users expect to be able to
easily store and retrieve their information. Users do not expect duplication or redundancy of
their documents or efforts. The feeling is, if the information exists and the users have access to
it, they, as users of the ECM system, have the right to simply find it. Users anticipate improved
communication and collaboration. And last but not least, users expect all of this to be done
through a user-friendly interface.
Saudi Aramco employees have the same expectations, and would like to use one system that
would make all their document-related issues go away. We, as service providers, could give this
to the Saudi Aramco users but we have one simple dilemma; we have two systems for us and
the user to choose from, with pros and cons to each. A way to combine the two systems would
be wonderful.
This article starts with a brief overview of Saudi Aramco and the history of ECM in Saudi
Aramco. Next, we will walk you through the journey of integration; why and how it started and
how we used EMC Documentum® Repository Services for Microsoft SharePoint (EDRSMS) to
resolve this situation. Finally, we will discuss our challenges, future goals, and the integration
road map.
2010 EMC Proven Professional Knowledge Sharing 4
Company Overview Owned by the Saudi Arabian Government, Saudi Aramco — with over 54,000 employees — is a
fully-integrated, global petroleum enterprise and a world leader in exploration and producing,
refining, distribution, shipping, and marketing. The Company manages the largest proven
reserves of conventional crude oil, 260 billion barrels, and manages the fourth-largest gas
reserves in the world, 263 trillion cubic feet. Saudi Aramco, through its affiliate, Vela Marine
International Ltd, owns and operates the world's second largest tanker fleet to transport its
crude oil production, which amounted to 1.7 million barrels per day in 2008. In addition to its
headquarters in Dhahran, Saudi Arabia, Saudi Aramco, through its affiliates, has joint ventures
and subsidiary offices in China, Japan, India, the Netherlands, the Republic of Korea, Malaysia,
Singapore, the United Arab Emirates, the United Kingdom, and the United States. Saudi
Aramco also refines and distributes oil products throughout the Kingdom of Saudi Arabia to
meet domestic daily energy demands. For more information, visit www.saudiaramco.com.
2010 EMC Proven Professional Knowledge Sharing 5
History of Enterprise Content Management in Saudi Aramco IT, Dhahran To better appreciate our current situation, and what we have been through regarding ECM, we
need to take you back to the year 2000! It was around that time when web applications were
becoming very popular in the region and Saudi Aramco was searching for a user friendly, web-
based document management system. At that time, the only one that fit Saudi Aramco’s
requirements was Livelink provided by Opentext. Livelink was acquired and rolled out
companywide as a document management and sharing application. It was used for uploading
and sharing product documentation, submitting status reports, project collaboration, simple
business process automation, and various other purposes. It was also around that time when
Documentum was acquired but used in a much smaller scale for specific applications.
As time passed, users became used to document management solutions and started to have
more challenging requirements. Documentum started to be used more widely and became more
popular within Saudi Aramco. Still, Documentum was used for specific applications that were
designed and used at a corporate level. Of course, Livelink was still there and very mature,
stable, and reliable. Livelink was very user friendly and provided most of what our users
needed. For a few years, we continued to provide Livelink as is and made a customized
application on top of Documentum.
Saudi Aramco uses SAP for business process automation. Some of these processes needed a
complex, secured document management system to store and manage the documents
associated with these business processes. So a project was started to look into the best product
in the market that can provide the needed requirements. We had two document management
solutions, Documentum and Livelink; so a technical decision had to be made. At that time,
Documentum was the only document management product certified by SAP among the two; so
the decision was to integrate SAP with Documentum. After that project was completed,
Documentum became the repository for all SAP documents. Of course, Livelink became
certified shortly after.
We also started getting requests for specific needs and requirements from the Livelink users
but, since we were only providing Livelink out of the box, we couldn’t fulfill those requests.
2010 EMC Proven Professional Knowledge Sharing 6
However, we were able to satisfy those needs with Documentum. This introduced a new
document management and sharing application along with a few questions; “Why do you have
the two applications?” “Why do they both provide almost the same features?” “Which one
should I use?” All valid questions from our customers and management. The answer was to go
with the application on which we invested the greatest amount of time and effort. So we now
had two systems; neither of them cheap in any way. We had to think right and fast. Saudi
Aramco conducted an application road map to decide on the future of many of its major
applications. Documentum was selected to be the solution for Saudi Aramco document
management since it was used in many corporate customized applications and was the
repository for Saudi Aramco SAP.
Once this decision was made, another set of questions were raised, “What to do with the
Livelink Content?” and “How do we manage the heavy users of Livelink?” We faced the
challenge of moving the Livelink Content to Documentum and converting all the Livelink users to
Documentum users. We looked at many solutions and nothing met our specific needs. We
developed our own migration tool and migrated all of our Livelink content to the customized
Documentum application, called e-Cabinet. A major awareness and change management
campaign was conducted. The result was that in 2006, Documentum became Saudi Aramco’s
document management system.
At around the same time, SharePoint 2007 Beta Release was distributed by Microsoft. We got a
request to establish a Knowledge Sharing System on top of SharePoint. Of course we all saw
the amazing features and user friendliness of SharePoint, but from a system perspective, we
didn’t want to add another application. Our users had a number of issues that a knowledge
sharing system would resolve. One of the issues that Saudi Aramco faced was a huge
retirement rush. All the extremely knowledgeable employees who have been working for the
company were scheduled to leave within the next five years. Their knowledge needed to be
captured and saved for current and future employees. Another issue was the duplication of
efforts when it came to analyzing and solving similar problems. As an example, when an
engineer working in Plant A notices a major problem with one of the machines in the plant, he or
she needs to look into the problem and analyze what could be causing it. This engineer found
the solution and proceeded to fix the problem. Solving this problem could take hours, days, or
weeks, depending on the situation. Now, if another engineer in Plant B faces the same problem.
what does he or she do? They go through the same process of analyzing and solving the issue.
2010 EMC Proven Professional Knowledge Sharing 7
This could be done more efficiently if the engineer had a place to go where he or she could
search for similar types of issues or even connect with the engineer who faced this similar
problem. We also had an issue with the huge number of information or organizational web sites
throughout the company and the need to consolidate those in one area. It was very clear that
we could resolve those problems using SharePoint collaboration and Content Management
features. So we spent some time looking into the Beta and eventually went ahead with
SharePoint 2007.
During that time, Livelink was fully migrated to e-Cabinet, the customized application on top of
Documentum. So, after all of that we are back to square one. We have two very similar
applications that provide very similar features. “Where do we put our stuff?” “Where can I find
this?” “What’s the difference between Knowledge and Content?” All were valid questions that
started to be repeated. The answers were a bit more difficult this time around. Why did we
provide the two systems? Looking into the situation and analyzing it further we discovered that
we did need the two systems. We discovered that each had its own areas that were extremely
efficient. Having the two ECM systems made it very difficult for us and our users. So we knew
what we had to do, but had no idea how to do it. The two systems–Documentum and
SharePoint–had to remain but only one would be apparent to users. Since we had invested a lot
in Documentum’s Enterprise Content Management features and modules, and after studying the
situation and looking into the market trends, we decided to keep SharePoint 2007 as the user-
friendly interface to Documentum; it would be our Knowledge and Content Management
System. That starts our story with integrating SharePoint with Documentum.
2010 EMC Proven Professional Knowledge Sharing 8
Our Journey with the Integration
How it started Now that we knew exactly what we wanted to do, we thought the rest would be easy …
WRONG!!! Due to the growing popularity of SharePoint, everyone provided some sort of
integration. We started looking at all of them!! And I do mean ALL of them. We had specific
requirements that needed to be met for us to take advantage of both of our systems; otherwise,
what was the point? We wanted to have the user continue working on SharePoint as is, with the
same look and feel of SharePoint. At the same time, we wanted all the documents to reside in
Documentum. Due to Microsoft’s recommended 100 GB limitation of the SQL database, we
knew we would face an issue there. Additionally, because we had invested in other EMC
Documentum ECM features such as Records Management, we wanted to be able to continue
using them after the integration took place.
None of the products we looked at fit our specific requirements. Most of them focused on the
idea of web-parts, which would provide a window view to Documentum from within SharePoint.
Even though that was very easy to implement and provided a very similar document view and
update interface to the one provided by SharePoint, that wasn’t acceptable to us or our users.
One of the products provided by EMC – Archive Services for SharePoint – seemed to have the
potential, so we decided to invest more time and effort on that product. We felt that Archive
Services would mature into a product that would fit our needs based on our testing and its road
map. Two of Saudi Aramco’s employees (who happen to be the two writing this article) were
sent on a year and a half business assignment in EMC. We were sent to gain knowledge about
EMC’s SharePoint integration products and actually helped build one of them. We weren’t spies
but we did learn a lot. We had the chance to work with the designers of Archive Services as it
matured and – of course – changed its name quite a few times.
Regardless of the name changes, we felt that the product we were following had reached a
point where it provided exactly what we were looking for; EMC called it EMC Documentum
Repository Services for Microsoft SharePoint (EDRSMS). To confirm our belief, a project team
was formed consisting of one project manager, two Documentum Administrators, two
2010 EMC Proven Professional Knowledge Sharing 9
SharePoint Administrators, and two Documentum Analysts. This project team was tasked with
evaluating the controlled release of the EDRSMS through the Controlled Release Program
(CRP). The CRP was very useful to Saudi Aramco, as it gave us early access to the product
and allowed us to collaborate with other companies and the product design team during the
entire process of planning, installing, and testing EDRSMS. After the completion of the CRP, it
was clear to us that it did provide exactly what we were looking for. Then the real fun started!!
2010 EMC Proven Professional Knowledge Sharing 10
EMC Documentum Repository Services for Microsoft SharePoint (EDRSMS) After evaluating the product, a quote provided by Saudi Aramco for use in the product rollout
best expresses our feeling about the product and why we decided to go with it. "EDRSMS
creates a true partnership, like Yin & Yang, SharePoint having the friendly face for everyone to
see and Documentum providing the muscle for everyone to use. EDRSMS combines best-of-
breed solutions allowing users to access their documents from the more user friendly interface
(SharePoint) while storing and managing them in a robust content management platform
(Documentum)."
EDRSMS journals SharePoint content into Documentum and provides seamless integration
between SharePoint and Documentum. Before going into how EDRSMS works, let’s go over the
points that led to choosing EDRSMS. First of all, the integration was transparent to the users. All
they had to do was log in to SharePoint to upload, update, or view their documents. Although it
appeared as if they were in SharePoint, they were all actually in Documentum. Second, it
relieved SQL Server bloat by having the content now stored in Documentum instead of the SQL
Server; this led to easier manageability of the storage. Third, it improved productivity since the
users would not need to access the two systems. Finally, it would allow us to capitalize on our
investment in the ECM features of Documentum and its add-on modules such as Records
Management (as promised in the future release of EDRSMS.)
Before installing EDRSMS, both existing SharePoint and Documentum environments needed to
be studied to properly design how the integration would fit within both environments. Taking a
closer look at our Documentum environment (Figure 1) illustrates how our web clients connect
to Documentum through the J2EE or IIS Servers which go through the connection brokers to
our Documentum Content Servers. We have five content servers hosting a number of
Repositories. The Content Server we are targeting for the integration is the one we call PDW. It
hosts the e-Cabinet Repository.
2010 EMC Proven Professional Knowledge Sharing 11
On the other side of it, we have our Microsoft SharePoint environment. Figure 2 shows how our
web clients would connect through the web interface to one of our Web Front Ends. In this case,
we have three Web Front End Servers; one of them hosts the Central Administration. All the
content and metadata is stored in the SQL Server databases.
2010 EMC Proven Professional Knowledge Sharing 12
To integrate SharePoint with Documentum, we needed to install and configure the needed
components of the product where they fit, as illustrated in Figure 3. We first needed to create a
new Repository (Rep_EDRSMS) that would host the SharePoint Content. Optionally, you could
also install an EDRSMS DAR file that would take care of Advanced Mapping. This means it
would bring a set of optional SharePoint properties back to Documentum such as SharePoint
Location, SharePoint Created By, SharePoint Modified By, SharePoint Deleted, etc. This would
be useful for reuse of the content or if you want to query Documentum for some specific
information based on this mapping. We decided to go with Advanced Mapping. We would then
need to install the Admin Component on the server that hosts Central Administration. This
would be used to configure and monitor EDRSMS through the EDRSMS Configuration page.
You would then need to install the EDRSMS Component on EACH Web Front End Server. This
Component consists of the Journaling Service (JME) and the Cleanup Service (CCS). The CCS
identifies the items that have been deleted from SharePoint and also need to be deleted from
Documentum. The Journaling Service does the actual content journaling and deletion in
2010 EMC Proven Professional Knowledge Sharing 13
Documentum. Once the Journaling Service is installed, you need to create the EDRSMS
database on the SQL Server (preferably) in its own instance. This database will host the
configuration information and contain the blob ID information for the content that is stored in
Documentum. After you have done all of this, your content will now be stored in the
Documentum Repository and only the blob information will reside in the EDRSMS database.
2010 EMC Proven Professional Knowledge Sharing 14
Challenges Our requirements didn’t end there. To be successful, we needed to consider existing content in
SharePoint and Documentum. Unfortunately, EDRSMS does not consider existing SharePoint
content, and would only integrate new content. After collaboration with the EDRSMS designers
and project team, we decided to take two approaches to this.
1- Forward Approach: We would do nothing. Once the user updated their content in
SharePoint, it would go though the journaling process. We sometimes called this the
“Lazy Approach.”
2- Backward Approach: We built a program that would force the content through the
journaling process. We would use this approach for SharePoint sites with large-sized
content that should move ASAP.
We also needed to consider the Documentum Content that already existed in e-Cabinet. If we
truly wanted to benefit from the integration and give the user one point of access, we needed to
migrate that content to SharePoint. Due to the integration, that content would go back to
Documentum. Technically, this change could be done in one step but we have to consider
change management and awareness of the e-Cabinet users. A phased approach was adopted
to minimize the impact of the change on the users.
We also had some backup/restore requirements that EDRSMS didn’t address. For example, we
needed to be able to copy or move our SharePoint environment to a new one for upgrades or in
case of disaster. Although possible, it had to be performed without the out-of-the-box solutions
provided by SharePoint. Therefore, extra consideration needed to be taken while attempting the
backup/restore operations.
Another challenge that we faced was not a technical one; it was related to segregation of the
administration function. We currently have two separate entities that administer each system
(SharePoint – Documentum). The EDRSMS configuration page is part of the SharePoint
Administration page and it is mainly used for the connection to Documentum, which requires a
Documentum administrator to set up. For the time being, we devised our own method until this
issue is addressed in future releases.
2010 EMC Proven Professional Knowledge Sharing 15
Future Goals and Integration Road Map Once we had the integration in place, our first phase was complete. The driver behind phase
one was to use the power of Documentum and enhance what we currently have in SharePoint.
Phase two was to take advantage of EMC Documentum’s features, e.g. Records Management,
through the integration. Although this is currently not achievable in the current release of the
product, it will be available in future releases of EDRSMS. With over six Terabytes of electronic
content (and growing), we cannot possibly keep this content forever and needed a solution that
would apply retention policies to these documents based on their type. We have already started
planning for this as the second phase of the integration.
Once we have completed all of this, we will have accomplished our Enterprise Content
Management goals and we can truly say that we have provided our users and our company with
what they require from content management and knowledge management.