Effective Search Deployment and Operations in SharePoint 2013Knut Brandrud Darrin AllredProgram ManagerSolution Architect

SPC095

Today you’re going to learn……about the SharePoint 2013 Search Topology…how it differs from SharePoint 2010…how to deploy a search topology with HA…how to keep your search topology healthy…about day-to-day operations of the search service

SharePoint 2013 Search Topology

SharePoint 2013 Search Architecture

SearchAdmin

Content UXCrawl

ContentProcessing Index

QueryProcessing WFE

API

AnalyticsProcessing

Crawl

Search Admin

Link

Analytics Reporting

FAST Search Index

SharePointSP AppsDevicesNon-SP UX

HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS

Public API

Search topology components

Content Query

New-SPEnterpriseSearchTopology New-SPEnterpriseSearchCrawlComponent New-SPEnterpriseSearchIndexComponent … Set-SPEnterpriseSearchTopology

Server Name Admin Crawler Content Analytics Query Index PartitionProcessing Processing Processing 0

SharePoint Search TopologyVital (and large) part of your farm!

Scalable – content and query load

Fault-tolerant

Appears as a SharePoint Service Application (Search Service Application)

Manage using PowerShell

Status from UI/PowerShell

Main changes from SharePoint 2010… from SharePoint Server 2010 Search

Crawl Component

Query Component

SharePoint 2010 Search Service Application

Crawl Indexing Engine

Query Engine

Search Admin

Property Store (SQL)

Content

UXWFE

Main changes from SharePoint 2010… from FAST Search Server 2010 for SharePoint

FAST Content SSA

FAST Query SSA

FAST back-end components(managed separately)

Extensibility:• Sandbox• Entity

Extraction

CrawlIndexing Engine

Query Engine

Content Pipeline

Analysis Engine

Query Pipeline

Search Admin

Content

UXWFE

Main changes from SharePoint 2010… to SharePoint Server 2013 SearchSharePoint 2013 Search Service

ApplicationIndex Component

Query Engine

Content Pipeline

Content ProcessingComponent

CrawlComponent

Query ProcessingComponent

AnalyticsProcessingComponent

Query Pipeline

Search Admin

Admin Component

Entire index on local disk

Property Store (SQL)

Content

UXWFE

Analysis Engine

CrawlIndexing Engine

More details: SPC172 -SPC202 - Search Architecture in SharePoint 2013

Link/query analysis &

recommendations

Separate crawl and indexing

Extensibility:• Web

callout• Entity

Extraction

Where does Search live in the farm?Windows servicesSharePoint Search Host Controller service

Runtime/lifecycle control of search components (except crawler) hostcontrollerservice.exe

SharePoint Server Search serviceCrawl Component mssearch.exe mssdmn.exe

ProcessesNoderunner.exe

Runtime environment for search components (except crawler)

msseearch.exe

mssdmn.exe

CrawlComponen

t

noderunner.exe

Search Runtime Environment

hostcontrollerservice.exe

Host Controller

Sh

are

Poin

t A

pp

Serv

er

Admin entitiesSearch Service Instance: Provisioning of the search service on each boxSearch Service Application: SharePoint Configuration entity

Still there, but only Crawl Component

noderunner.exe

noderunner.exe

noderunner.exe

noderunner.exe

Admin Component

Query Processing Component

Content Processing Component

IndexComponent

Analytics ProcessingComponent

Search High-AvailabilityComponent HA mode of operation

Crawl Component Load-sharing, state in DB

Index ComponentPrimary/Secondary per partition (dynamic selection)

Query load-sharingPrimary selected based on index generation status

Content/Query Processing components Load-sharing, stateless

Analytics Processing Component Distributed map/reduce, state in DB

Admin Component Primary/Secondary (dynamic selection)Hot standby, state in DB

More details: SPC172 - Overview of Capacity Planning, Sizing and High Availability for Search in SharePoint 2013

Host Controller (for search dictionary repository)

Primary/Secondary (cmdlet controlled)Lost primary –no newdictionaries deployed

> New-SPEnterpriseSearchTopology> New-SPEnterpriseSearchIndexComponent, New-

SPEnterpriseSearchQueryProcessingComponent, … > Set-SPEnterpriseSearchTopology

Set up your initial search topology- You probably want HAPrepare servers• HW/SW reqs!• Install SharePoint• Add to farm• Start Search

Service Instance• Create SSA

1

Create a new topology object for the SSA

2

Add search components to the topology

Assign components to servers

3

Activate the new topology

This will replace the single-server topology with a scaled-out topology

4

Need more performance or capacity?Scale out your search farm

Create an edit copy of your active topology> New-SPEnterpriseSearchTopology -CloneYou never modify the active topology object

Add before removeAdd components Activate Green state Remove components ActivateEnsures HA during the scale out operation

Small search topologyOne index PartitionPrepared for 10 Mill indexed documents

Fault tolerance 2 physical servers – 4 VMs

Which components on same server?

Search admin db Link db

Analytics dbCrawl db

All SharePoint databases All SharePoint databases

Analytics

Replica ReplicaIndex partition 0

Query Processing Query Processing

Crawl

Admin

Content processing

Analytics

Crawl

Admin

Content processing

Application Server Application Server

Application Server Application Server

Redundant copies of all databases

VM01

VM03

VM02

VM04

Host server A

Host server D

Host server C

Host server B

Crawl

Content Processing

Index

Analytics Processing

Admin

Query Processing

More details: SPC172 - Overview of Capacity Planning, Sizing and High Availability for Search in SharePoint 2013

Index PartitionReplica Replica

Query ProcessingQuery Processing

Demo

Deploying a small search topology with HA

Location of files on disk matters!Local disk for index files!Index operations are I/O demanding

At install timeYou can specify the data folder for all search components (including index location)

Per Index ComponentSeparate data folder for the index filesWhen creating a new index componentNew-SPEnterpriseSearchIndexComponent

-RootDirectory E:\Index

ULS logsPrevent disk contention/floodingSet-SPDiagnosticConfig -LogLocation F:\Logs

Note change from SP 2010 Set index location per SSI not supportedSet-SPEnterpriseSearchServiceInstance

-DefaultIndexLocation

Scale out by re-partitioning the indexWhen each partition exceeds 10 million itemsStateless partitioning using consistent hashingPause query and feedingStart with a symmetric index layout

Index

Index

Index

Index

Content Processing

Partitions

Index

Index

Replicas

Video

Scale out to a two partition search topology

Regional farm 2Regional farm 1

Index

Regional farm 3

Crawler

Crawler

Crawler Index

Index

Regional farm 2Regional farm 1

Index

Office 365 farm

Crawler

Crawler

Crawler Index

Index

Federation

Regional farm 2

Central Search Farm

Regional farm 1

Search service

CrawlerCrawler

Crawler

Crawler

Regional farm 3

Large enterprise: Multi-farm deploymentsCentral search farm Serving multiple content farmsEasier administrationBetter relevancyBetter end-user experience

Distributed deploymentSearch index on each farmSearch results using federationResult blocks in the search result

HybridOn premise and O365 – Federation Stepwise migration to the cloud

When to choose federation?Slow WAN between remote farmsRegulatory restrictionsMost users want search results from local sourcesHybrid (On premise and O365)

Keeping your search topology healthy

Search Diagnostic SourcesSystem/Component stateCentral Admin SSA Search Administration (UI)

Health information on SSA levelHealth state for search topology components

PowerShell: Get-SPEnterpriseSearchStatusSearch component status including detailed diagnostics info for issue resolution

Search health reports (UI)Content feeding performance, query rate/latency

TelemetryLogs and performance countersULS events and associated rollup in SCOM, including alerts

Server Name Admin Crawler Content Analytics Query Index PartitionProcessing Processing Processing 0

Understanding component issuesYellow (degraded)Transient component lifecycle states – Restarts and topology changesIndex replica not in sync

Seeding: Copy index to new replicaCatching up: Journal replay to a replica that is out of sync with latest updates

Red (failed / unknown)Indicates that server, or search component, is unreachableHW/Network issuesCmdlet: Will normally appear as ‘Unknown’.

Get-SPEnterpriseSearchStatus gives more details

Get-SPEnterpriseSearchStatusComponent state for each search componentActive / degraded / failed / unknown (not reachable)Extended info with additional data when yellow/red

Primary/secondary role for stateful HA componentsAdmin component, index component (per partition)

Status for analytics background jobsUsage analytics / Search analytics

Detailed diagnostic data for indexerIndex housekeeping (seeding, master merge, catch up)Diagnostic data for issue resolution

Get-SearchTopologyState.ps1One stop shop for search statusConsolidates data from multiple cmdlets and sources

All data from Get-SPEnterpriseSearchStatusCrawl status (crawling/idle/paused)# of indexed documentsHA status for topologyIndex size alertHost controller dictionary repository

Download: See TechNet Wiki

Demo

Get-SearchTopologyState.ps1

Search Health Reports - QuerySearch Administration DiagnosticsQuery performanceQuery latency Query rate

FederationPer result source

Query/feedingQuery latency / feeding rate correlation

Search Health Reports - Content

IndexContent Processing

Crawler

Search Administration Reports

Usage Database

Crawler Flow

Crawl LogErrors by type

Crawl RateDocuments/sec

By content source

Crawl LatencyTime spent in

content proc. and indexing

Crawl LoadCPU and memory

load

What about my feeding performance?

More details: SPC044 - Crawl and Index all Enterprise Content with SharePoint 2013 Search

What happened tomy document?

Server view Component view

Monitoring using SCOMArchitectural health view

Microsoft.Office.Server.Search

Search Service ApplicationHost Controller

Index Component

Index1

Index2Server Server

Admin Component

Monitoring using SCOMSCOM MonitorsUses events, performance data, and other information to assess the health state

SCOM Monitor examplesAnalytics analysis: failed to start warning – search analytics

Possible cause: Performance issue – analysis takes more than 24h – Scale outIndex: Missing partition

Content processing is not able to reach all partitions – Check component stateSearch Admin Platform Services: Repository Replication

Local repository (search dictionaries) not able to obtain the latest repository version, resulting in no new dictionaries being deployed to this server – Check host controller status (PowerShell)

TechNet articles - http://technet.microsoft.com/en-us/library/ee513082(v=office.15).aspx

Key day-to-day search operation

Your schedule as a search adminDailyCheck search topology UI at start of dayMonitor content freshness and query latency – Search health reportsAny component not green – More details using Get-SPEnterpriseSearchStatus

RegularlyVerify # of documents per partition and disk space on index and analytics componentsProduct backupSecurity updatesCumulative updatesReplace failed HW

Backup and Restore

Everything but the index is in databasesRestore possible to a different deploymentMay use B/R for standby DR locationRestore location must have the same topology

Ballpark figures8 hours to backup 80M document index: 2TB of data, 6 hours to restore

Backup and restoreSharePoint farmRecommended approachPart of SharePoint Backup/Restore

DB-only backup/restorePrimarily when moving DB servers

Volume Shadow Copy (VSS)Customer application controls backupSelective snapshot, including the indexSearch is down for a longer period of time

How does the standard backup work?Consistent across all DBs and indexIndex backup phase 1Backup the index – indexing paused – feeding continues – content updates persisted to the journal

Index backup phase 2Backup the journal – content updates since phase 1 started – feeding paused

Continued operations during backupQueries continue to work throughout the backupCrawl is paused for ~10% of the overall backup time

Patching and updatesSequence is important!For Windows Update, CUs, SPsTo minimize query downtimePassword Roll cannot be sequenced – Consider timing

Define your update groups1st half of components – Ensure HA during patching2nd half of componentsPrimary Admin Component – admin failover

Implies a short query downtime (few seconds)

Search admin db Link db

Analytics dbCrawl db

All SharePoint databases All SharePoint databases

Analytics

Replica ReplicaIndex partition 0

Query Processing Query Processing

Crawl

Admin

Content processing

Analytics

Crawl

Admin

Content processing

Application Server Application Server

Application Server Application Server

Group 1 Group 2 Group 3

Replace a failed server (with HA)

Add replacement server to topology

Clone active topologyAdd componentsActivate topology

1

Wait until added components are green

For index replica: Seeding must be completed

2

Remove old server from topology

Clone, remove, activate

Before you remove server from the farm! 3

Verify Host Controller repository status

Needs action if primary host controller was on the failed server

4

Verify status of primary host controller – in case it was on the failed server> Get-SPEnterpriseSearchHostController

If primary host controller was on the failed server:> Set-SPEnterpriseSearchPrimaryHostController

Survival Guide (TechNet Wiki)http://social.technet.microsoft.com/wiki/contents/articles/14382.survival-guide-sharepoint-2013-and-enterprise-search-sp2013es.aspx

TechNet Documentationhttp://technet.microsoft.com/en-us/library/cc303422(v=office.15).aspx

Related Resources

TechNet contentSearch AdministrationAdminister search in SharePoint Server 2013

Search TopologyScale search for performance and availability in SharePoint Server 2013Technical diagrams for SharePoint 2013Manage the search topology in SharePoint Server 2013Get-SPEnterpriseSearchStatus

Backup/RestoreBack up Search service applications in SharePoint 2013Overview of SharePoint Foundation and the Volume Shadow Copy Service

Search HOLs and events @ SPCHOL031 – Introduction to Search in SharePoint 2013HOL034 – Exploring Search Query Rules in SharePoint 2013HOL032 – Extending the Search experience in SharePoint 2013HOL033 – People Search in SharePoint 2013HOL035 – SharePoint Server 2013 Search Connectors and Using BCS

Meet a Search SME

Ask questions, meet the community and share knowledge!

Mon-Thu @ Exhibit Hall

Hands on Labs

Daily 10:30am-6:30pm @ HOL Lab Lounge

Ask the Experts

Discuss search!

Wed 6:15PM @ Ask the Experts Lounge

Related Search Sessions @ SPC

Wed 3:15pm - SPC049 - Custom Security Trimming for Search in SharePoint 2013Speakers: Morgan Larsson, Sveinar Rasmussen

Wed 3:15pm - SPC150 - Microsoft Early Learning: Moving Search to O365 and Building a Hybrid Experience - Speaker: Rene Sanchez Almaguer

SPC202 - Search Architecture in SharePoint 2013Speakers: Thomas Molbach, Rune Zakariassen

SPC259 - What's New in Search for SharePoint 2013Speakers: Glen Anderson, Cem Aykan

SPC172 - Overview of Capacity Planning, Sizing and High Availability for Search in SharePoint 2013 - Speakers: Barry Waldbaum, Olaf Birkeland

Wed 10:30am - SPC117 - How to Manage and Troubleshoot Search: A Practical Guide Speakers: Darrin Allred, Ed Barnett, Agnes Molnar

Questions?

Evaluate this session now on MySPC using your laptop or mobile device: http://myspc.sharepointconference.com

MySPC

© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.