WE ARE AN ENTERPRISE TECHNOLOGY PARTNER

Coders Center sp. z o.o.

Plac Solny 15

50-062 Wroclaw/Polandcontact@coders.center

+48 71 300 1245

Sitecore xDB – Architecture and Configuration

2

Tomasz Juranek – Sitecore Developer/Architect @ Coders CenterCertificated Sitecore Developer since 2012. For last 5 years has worked on severalSitecore project for big brands around the Europe.

What is Experience Platform?Sitecore Experience Platform is (customer or developer POV):

• Mongo

• No SQL, xDB

• Content personalization

• Analytics Component

• Marketing Tools

• Customer Engagement

• Reports

• User Profiles

• Big Data

• Framework

• Architecture

• New DMS, CEP

• Different approach?

3

What Experience Platform do, what is the role of xDB

Experience Platform:

The combination of Sitecore CMS, xDB, and other experience marketing applications.

xDB:

The Sitecore Experience Database (xDB) collects all your customer interactions from allchannel sources in a central, big data repository.

xDB connects interaction data to create a comprehensive, unified view of each individualcustomer, and makes the data available to marketers to manage the customerexperience in real time.

4Experience Platform Glossary: https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/experience_platform_glossary

How Experience Platform works from visitor perspective

When a contact interacts with the website, the xDB collects all contact and the interaction data. This data flows through the xDB in the following sequence:

5

1. A contact decides to interact with a website.2. Depending on the geographic location of the contact, the xDB

redirects their visit to the closest cluster (data center) through DNS configuration.

3. Load-balancing software directs the contact to an appropriate Sitecore content delivery server.

4. Contacts stay connected to the same cluster even if they switch devices or use another browser (session transfer).

5. Sitecore uses session state for storing information about the current contact interaction, contact state, and any other related data

6. On session end, Sitecore flushes this data to the collection database and schedules it for processing.

Session transfer:https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/server_considerations/server_clusters_and_transferring_contact_sessions

Session states in Experience Platform

• Private session state collects data of the interactions (visted pages) and the devices used by the contact.• Shared session state data includes all data that is unique to a contact and that can be shared across simultaneous sessions, such as contact

details (available for personalization) and any triggered engagement automation states.• Private and shared session providers are configured in web.config and can be InProcess (by default ASP.Net in memory), MongoDB or SQL

Server• On single standalone CD server shared session state is still useful for handling concurrent visits, by the same contact from different devices• On CM server use inProc

6Session states:https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/session_state/session_state

time

Session of Visit 2(private)

Shared session state for Contact session

Session of Visit 1 (private)

sessions

{

timeout

timeout

Expires 1 minute afterlast session ends

{

{

Contact Identification

New visit starts with unidentified contact (Anonymous in Experience Profile): • Identifier (GUID) and device/browser data only.• Connection between contact and device is stored in a cookie. Clearing cookies in the browser breaks the connection.

Unidentified contacts became identified by providing more data e.g.• Filling WFFM forms• Clicking EXM email link• Login form or other custom code, using Sitecore.Analytics.Tracker.Current.Session.Identify(string id);

When xDB identifies a contact, it merges the unidentified contact with the identified contact.• All the information stored in these contact records merge into one record. The old contact record is removed and replaced

with a reference to the updated contact record.• Typical scenario: Double Opt-In, where user registers on one device and clicks confirmation link on different one.

7Contact merge:https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/contacts/merge_contacts

Mongo Collections - Contacts

8

own

List Manager entry, e.g. for EXM

Personal Data from WFFM form

• xDB stores all visitors including anonymous• Contact vs User from Membership database• Contact can be easily extended with custom fields (Contact Facets)

How to create custom contact facet:https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/contacts/create_a_custom_contact_facet

Mongo Collections - FormData

9

• WFFM has built-in actions to save contact data into xDB and link with User in Membership database.• Form data can be exported to Excel or CSV in Sitecore back-office.• Form fields on website can be prefilled with data from xDB Contact (check „Rules” in Form Designer).

Mongo Collections - Interactions

10

• To speed up the testing set short session timeout, or use Session.Abandon();

Mongo Collections – Interactions (Pages)

11

xDB Data Aggregation1. After session ends, the latest interactions are saved to the collection database.

12Architecture overview:

https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/platform/architecture_overview

2. Interaction data is added to the processing pool for aggregation.3. An agent worker picks up the interaction from the processing pool and hands it on to an

aggregator.4. The aggregator pushes the interaction through the aggregation pipeline and converts the

data into a form suitable for the reporting database (reduced and easy to query).5. Once the data is converted into the correct format, it is merged into existing reporting

data that is stored in the SQL reporting database, keeping the reporting database continuously in sync with the latest interactions on the website

7. Gathered data is presented to marketers in Sitecore reporting applications via Reporting API

Data for Experience Platform reports (Reporting API)

13Reporting API descriptionhttps://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/platform/reporting_architecture

• Reporting database – a Microsoft SQL Server database that contains fact and dimension tables of aggregated data from the collection database

• Reporting Service – queries the reporting database(SQL) and the collection database (Mongo) to fetch report data

• Reporting server – a Sitecore server that hosts the Reporting Service

* Names may be confused: Legacy Analytics SQL db vs Analytics Collection in Mongo vs Reporting connection string

Data for Experience Platform reports (Reporting Apps)

14

Loaded from Analytics index:

Loaded directly from Mongo DB:

Loaded from Analytics index, SQL Reporting db:

Where Experience Platform stores visitor data - Summary

Cookie

• A cookie is used to identify the device that a contact uses to interact with a website.

In this way, you can identify the device if the contact returns using the same device.

Private and shared session:

• Interactions and devices

• Contact Personal information and identifier

• Contact Behavior profile (Contact's past profiles and personas matches)

• Key Behavior Cache in Sitecore 8.1 (Recent X contact’s activities)

• Data from WFFM forms

• Engagement automation states

Mongo DB

• All the visit and contact data persisted in collections

SQL Reporting Db

• Aggregated and grouped analytics data

Analytics Index

• Most important data is indexed for quick access

15Key Behavior Cache:https://doc.sitecore.net/sitecore_experience_platform/digital_marketing/personalization/historical_personalization/key_behavior_cache

16

Typical Sitecore CMS infrastracture

Customers typically don’t use analytics features from previous Sitecore versions. But they want to migrate to Sitecore 8.x because of Experience Platform

17

Recommended Experience Platform infrastracture

Good starting point, divide into roles to allow scaling

18

Scaled Experience Platform infrastracture

To fulfil SLA and performance requirements

19

Minimal Experience Platform infrastracture

Standalone, all-in-one configuration same as for typical Sitecore CMS

Scalability options:https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/platform/scalability_options

Experience Platform servers configuration

Possible Sitecore server roles (features) in the platform:

• Content Delivery Server

• Content Management Server

• Processing Server

• Reporting Server

Each server role it’s a Sitecore instance with different configuration:

• Several configuration files in /Include folder have to be enabled/disabled per server role

• Different database access per server role

• Server roles can be mixed, eg Reporting/CM, or Processing/Reporting on same server

• It’s flexible, can be easily extended horizontally (add more servers to perform each activity)

Additionally setup Mongo DB for Collection and Session databases (optionally on SQL Server)

20Server configuration:

https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/configuring_servers/configuring_servers

21

Experience Platform possible problems• Limited developers and community knowledge comparing to CMS

• Limited customer knowledge

• Typically compare Sitecore to their previous CMS• Don’t know what Experience Platform has to offer

• Customer budget and project plan:

• Usually CMS focused• Experience features need to be developed as well!

• Responsibility for analytics/personalization/marketing features:

• Customer (they know their business), or implementation partner? • Hard to say, due to limited knowledge, budget and wrong planning

• Sitecore upgrades are more complicated comparing to CMS-only:

• Experience Platform is changing a lot, comparing to stable CMS• A lot of different config files per server role• Related with WFFM and EXM modules

• More complicated architecture:

• Higher hardware costs • No MongoDB experience in typical corporate IT department• Solr is must on scaled environment

Solr on processing server:https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/configuring_servers/configure_a_processing_server

Thank you very much for your attention

22