Effective Search Deployment and Operations in SharePoint 2013Knut Brandrud Darrin AllredProgram ManagerSolution Architect
SPC095
Today you’re going to learn……about the SharePoint 2013 Search Topology…how it differs from SharePoint 2010…how to deploy a search topology with HA…how to keep your search topology healthy…about day-to-day operations of the search service
SharePoint 2013 Search Topology
SharePoint 2013 Search Architecture
SearchAdmin
Content UXCrawl
ContentProcessing Index
QueryProcessing WFE
API
AnalyticsProcessing
Crawl
Search Admin
Link
Analytics Reporting
FAST Search Index
SharePointSP AppsDevicesNon-SP UX
HTTPFile sharesSharePointUser profilesLotus Notes DocumentumExchange foldersCustom - BCS
Public API
Search topology components
Content Query
New-SPEnterpriseSearchTopology New-SPEnterpriseSearchCrawlComponent New-SPEnterpriseSearchIndexComponent … Set-SPEnterpriseSearchTopology
Server Name Admin Crawler Content Analytics Query Index PartitionProcessing Processing Processing 0
SharePoint Search TopologyVital (and large) part of your farm!
Scalable – content and query load
Fault-tolerant
Appears as a SharePoint Service Application (Search Service Application)
Manage using PowerShell
Status from UI/PowerShell
Main changes from SharePoint 2010… from SharePoint Server 2010 Search
Crawl Component
Query Component
SharePoint 2010 Search Service Application
Crawl Indexing Engine
Query Engine
Search Admin
Property Store (SQL)
Content
UXWFE
Main changes from SharePoint 2010… from FAST Search Server 2010 for SharePoint
FAST Content SSA
FAST Query SSA
FAST back-end components(managed separately)
Extensibility:• Sandbox• Entity
Extraction
CrawlIndexing Engine
Query Engine
Content Pipeline
Analysis Engine
Query Pipeline
Search Admin
Content
UXWFE
Main changes from SharePoint 2010… to SharePoint Server 2013 SearchSharePoint 2013 Search Service
ApplicationIndex Component
Query Engine
Content Pipeline
Content ProcessingComponent
CrawlComponent
Query ProcessingComponent
AnalyticsProcessingComponent
Query Pipeline
Search Admin
Admin Component
Entire index on local disk
Property Store (SQL)
Content
UXWFE
Analysis Engine
CrawlIndexing Engine
More details: SPC172 -SPC202 - Search Architecture in SharePoint 2013
Link/query analysis &
recommendations
Separate crawl and indexing
Extensibility:• Web
callout• Entity
Extraction
Where does Search live in the farm?Windows servicesSharePoint Search Host Controller service
Runtime/lifecycle control of search components (except crawler) hostcontrollerservice.exe
SharePoint Server Search serviceCrawl Component mssearch.exe mssdmn.exe
ProcessesNoderunner.exe
Runtime environment for search components (except crawler)
msseearch.exe
mssdmn.exe
CrawlComponen
t
noderunner.exe
Search Runtime Environment
hostcontrollerservice.exe
Host Controller
Sh
are
Poin
t A
pp
Serv
er
Admin entitiesSearch Service Instance: Provisioning of the search service on each boxSearch Service Application: SharePoint Configuration entity
Still there, but only Crawl Component
noderunner.exe
noderunner.exe
noderunner.exe
noderunner.exe
Admin Component
Query Processing Component
Content Processing Component
IndexComponent
Analytics ProcessingComponent
Search High-AvailabilityComponent HA mode of operation
Crawl Component Load-sharing, state in DB
Index ComponentPrimary/Secondary per partition (dynamic selection)
Query load-sharingPrimary selected based on index generation status
Content/Query Processing components Load-sharing, stateless
Analytics Processing Component Distributed map/reduce, state in DB
Admin Component Primary/Secondary (dynamic selection)Hot standby, state in DB
More details: SPC172 - Overview of Capacity Planning, Sizing and High Availability for Search in SharePoint 2013
Host Controller (for search dictionary repository)
Primary/Secondary (cmdlet controlled)Lost primary –no newdictionaries deployed
> New-SPEnterpriseSearchTopology> New-SPEnterpriseSearchIndexComponent, New-
SPEnterpriseSearchQueryProcessingComponent, … > Set-SPEnterpriseSearchTopology
Set up your initial search topology- You probably want HAPrepare servers• HW/SW reqs!• Install SharePoint• Add to farm• Start Search
Service Instance• Create SSA
1
Create a new topology object for the SSA
2
Add search components to the topology
Assign components to servers
3
Activate the new topology
This will replace the single-server topology with a scaled-out topology
4
Need more performance or capacity?Scale out your search farm
Create an edit copy of your active topology> New-SPEnterpriseSearchTopology -CloneYou never modify the active topology object
Add before removeAdd components Activate Green state Remove components ActivateEnsures HA during the scale out operation
Small search topologyOne index PartitionPrepared for 10 Mill indexed documents
Fault tolerance 2 physical servers – 4 VMs
Which components on same server?
Search admin db Link db
Analytics dbCrawl db
All SharePoint databases All SharePoint databases
Analytics
Replica ReplicaIndex partition 0
Query Processing Query Processing
Crawl
Admin
Content processing
Analytics
Crawl
Admin
Content processing
Application Server Application Server
Application Server Application Server
Redundant copies of all databases
VM01
VM03
VM02
VM04
Host server A
Host server D
Host server C
Host server B
Crawl
Content Processing
Index
Analytics Processing
Admin
Query Processing
More details: SPC172 - Overview of Capacity Planning, Sizing and High Availability for Search in SharePoint 2013
Index PartitionReplica Replica
Query ProcessingQuery Processing
Demo
Deploying a small search topology with HA
Location of files on disk matters!Local disk for index files!Index operations are I/O demanding
At install timeYou can specify the data folder for all search components (including index location)
Per Index ComponentSeparate data folder for the index filesWhen creating a new index componentNew-SPEnterpriseSearchIndexComponent
-RootDirectory E:\Index
ULS logsPrevent disk contention/floodingSet-SPDiagnosticConfig -LogLocation F:\Logs
Note change from SP 2010 Set index location per SSI not supportedSet-SPEnterpriseSearchServiceInstance
-DefaultIndexLocation
Scale out by re-partitioning the indexWhen each partition exceeds 10 million itemsStateless partitioning using consistent hashingPause query and feedingStart with a symmetric index layout
Index
Index
Index
Index
Content Processing
Partitions
Index
Index
Replicas
Video
Scale out to a two partition search topology
Regional farm 2Regional farm 1
Index
Regional farm 3
Crawler
Crawler
Crawler Index
Index
Regional farm 2Regional farm 1
Index
Office 365 farm
Crawler
Crawler
Crawler Index
Index
Federation
Regional farm 2
Central Search Farm
Regional farm 1
Search service
CrawlerCrawler
Crawler
Crawler
Regional farm 3
Large enterprise: Multi-farm deploymentsCentral search farm Serving multiple content farmsEasier administrationBetter relevancyBetter end-user experience
Distributed deploymentSearch index on each farmSearch results using federationResult blocks in the search result
HybridOn premise and O365 – Federation Stepwise migration to the cloud
When to choose federation?Slow WAN between remote farmsRegulatory restrictionsMost users want search results from local sourcesHybrid (On premise and O365)
Keeping your search topology healthy
Search Diagnostic SourcesSystem/Component stateCentral Admin SSA Search Administration (UI)
Health information on SSA levelHealth state for search topology components
PowerShell: Get-SPEnterpriseSearchStatusSearch component status including detailed diagnostics info for issue resolution
Search health reports (UI)Content feeding performance, query rate/latency
TelemetryLogs and performance countersULS events and associated rollup in SCOM, including alerts
Server Name Admin Crawler Content Analytics Query Index PartitionProcessing Processing Processing 0
Understanding component issuesYellow (degraded)Transient component lifecycle states – Restarts and topology changesIndex replica not in sync
Seeding: Copy index to new replicaCatching up: Journal replay to a replica that is out of sync with latest updates
Red (failed / unknown)Indicates that server, or search component, is unreachableHW/Network issuesCmdlet: Will normally appear as ‘Unknown’.
Get-SPEnterpriseSearchStatus gives more details
Get-SPEnterpriseSearchStatusComponent state for each search componentActive / degraded / failed / unknown (not reachable)Extended info with additional data when yellow/red
Primary/secondary role for stateful HA componentsAdmin component, index component (per partition)
Status for analytics background jobsUsage analytics / Search analytics
Detailed diagnostic data for indexerIndex housekeeping (seeding, master merge, catch up)Diagnostic data for issue resolution
Get-SearchTopologyState.ps1One stop shop for search statusConsolidates data from multiple cmdlets and sources
All data from Get-SPEnterpriseSearchStatusCrawl status (crawling/idle/paused)# of indexed documentsHA status for topologyIndex size alertHost controller dictionary repository
Download: See TechNet Wiki
Demo
Get-SearchTopologyState.ps1
Search Health Reports - QuerySearch Administration DiagnosticsQuery performanceQuery latency Query rate
FederationPer result source
Query/feedingQuery latency / feeding rate correlation
Search Health Reports - Content
IndexContent Processing
Crawler
Search Administration Reports
Usage Database
Crawler Flow
Crawl LogErrors by type
Crawl RateDocuments/sec
By content source
Crawl LatencyTime spent in
content proc. and indexing
Crawl LoadCPU and memory
load
What about my feeding performance?
More details: SPC044 - Crawl and Index all Enterprise Content with SharePoint 2013 Search
What happened tomy document?
Server view Component view
Monitoring using SCOMArchitectural health view
Microsoft.Office.Server.Search
Search Service ApplicationHost Controller
Index Component
Index1
Index2Server Server
Admin Component
Monitoring using SCOMSCOM MonitorsUses events, performance data, and other information to assess the health state
SCOM Monitor examplesAnalytics analysis: failed to start warning – search analytics
Possible cause: Performance issue – analysis takes more than 24h – Scale outIndex: Missing partition
Content processing is not able to reach all partitions – Check component stateSearch Admin Platform Services: Repository Replication
Local repository (search dictionaries) not able to obtain the latest repository version, resulting in no new dictionaries being deployed to this server – Check host controller status (PowerShell)
TechNet articles - http://technet.microsoft.com/en-us/library/ee513082(v=office.15).aspx
Key day-to-day search operation
Your schedule as a search adminDailyCheck search topology UI at start of dayMonitor content freshness and query latency – Search health reportsAny component not green – More details using Get-SPEnterpriseSearchStatus
RegularlyVerify # of documents per partition and disk space on index and analytics componentsProduct backupSecurity updatesCumulative updatesReplace failed HW
Backup and Restore
Everything but the index is in databasesRestore possible to a different deploymentMay use B/R for standby DR locationRestore location must have the same topology
Ballpark figures8 hours to backup 80M document index: 2TB of data, 6 hours to restore
Backup and restoreSharePoint farmRecommended approachPart of SharePoint Backup/Restore
DB-only backup/restorePrimarily when moving DB servers
Volume Shadow Copy (VSS)Customer application controls backupSelective snapshot, including the indexSearch is down for a longer period of time
How does the standard backup work?Consistent across all DBs and indexIndex backup phase 1Backup the index – indexing paused – feeding continues – content updates persisted to the journal
Index backup phase 2Backup the journal – content updates since phase 1 started – feeding paused
Continued operations during backupQueries continue to work throughout the backupCrawl is paused for ~10% of the overall backup time
Patching and updatesSequence is important!For Windows Update, CUs, SPsTo minimize query downtimePassword Roll cannot be sequenced – Consider timing
Define your update groups1st half of components – Ensure HA during patching2nd half of componentsPrimary Admin Component – admin failover
Implies a short query downtime (few seconds)
Search admin db Link db
Analytics dbCrawl db
All SharePoint databases All SharePoint databases
Analytics
Replica ReplicaIndex partition 0
Query Processing Query Processing
Crawl
Admin
Content processing
Analytics
Crawl
Admin
Content processing
Application Server Application Server
Application Server Application Server
Group 1 Group 2 Group 3
Replace a failed server (with HA)
Add replacement server to topology
Clone active topologyAdd componentsActivate topology
1
Wait until added components are green
For index replica: Seeding must be completed
2
Remove old server from topology
Clone, remove, activate
Before you remove server from the farm! 3
Verify Host Controller repository status
Needs action if primary host controller was on the failed server
4
Verify status of primary host controller – in case it was on the failed server> Get-SPEnterpriseSearchHostController
If primary host controller was on the failed server:> Set-SPEnterpriseSearchPrimaryHostController
Survival Guide (TechNet Wiki)http://social.technet.microsoft.com/wiki/contents/articles/14382.survival-guide-sharepoint-2013-and-enterprise-search-sp2013es.aspx
TechNet Documentationhttp://technet.microsoft.com/en-us/library/cc303422(v=office.15).aspx
Related Resources
TechNet contentSearch AdministrationAdminister search in SharePoint Server 2013
Search TopologyScale search for performance and availability in SharePoint Server 2013Technical diagrams for SharePoint 2013Manage the search topology in SharePoint Server 2013Get-SPEnterpriseSearchStatus
Backup/RestoreBack up Search service applications in SharePoint 2013Overview of SharePoint Foundation and the Volume Shadow Copy Service
Search HOLs and events @ SPCHOL031 – Introduction to Search in SharePoint 2013HOL034 – Exploring Search Query Rules in SharePoint 2013HOL032 – Extending the Search experience in SharePoint 2013HOL033 – People Search in SharePoint 2013HOL035 – SharePoint Server 2013 Search Connectors and Using BCS
Meet a Search SME
Ask questions, meet the community and share knowledge!
Mon-Thu @ Exhibit Hall
Hands on Labs
Daily 10:30am-6:30pm @ HOL Lab Lounge
Ask the Experts
Discuss search!
Wed 6:15PM @ Ask the Experts Lounge
Related Search Sessions @ SPC
Wed 3:15pm - SPC049 - Custom Security Trimming for Search in SharePoint 2013Speakers: Morgan Larsson, Sveinar Rasmussen
Wed 3:15pm - SPC150 - Microsoft Early Learning: Moving Search to O365 and Building a Hybrid Experience - Speaker: Rene Sanchez Almaguer
SPC202 - Search Architecture in SharePoint 2013Speakers: Thomas Molbach, Rune Zakariassen
SPC259 - What's New in Search for SharePoint 2013Speakers: Glen Anderson, Cem Aykan
SPC172 - Overview of Capacity Planning, Sizing and High Availability for Search in SharePoint 2013 - Speakers: Barry Waldbaum, Olaf Birkeland
Wed 10:30am - SPC117 - How to Manage and Troubleshoot Search: A Practical Guide Speakers: Darrin Allred, Ed Barnett, Agnes Molnar
Questions?
Evaluate this session now on MySPC using your laptop or mobile device: http://myspc.sharepointconference.com
MySPC
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.