understanding & architecting search in sp 2013/2016

Keys to Understanding & Architecting Search in 2013/2016 On-PremAjay Iyer

Let’s Get Social !Twitter: @spscincinnati

#spscincyMobile: www.spsmobile.comCheck us out on Facebook: https://www.facebook.com/sps

cincy

Submit your feedback! Go to http://SPSMobile.com

He was awesome!

me@somewhere.com

Thank You to our Sponsors!!!Platinum

BronzeSilve

r

Gold

Ajay Iyer

• SharePoint Architect• @shankarajay1• http://sharepointadminstuff.wordpress.com• Solution Architecture, Capacity Planning, Search, Migrations,

Enterprise Content Management, Document Imaging and….. InfoPath

Outline

• Review Search Components & Terminologies

• Gather Requirements for Search (What specifics should you ask?)

• Search Farm Architecture & Design

• Incremental Crawls or Continuous Crawls?

• What’s new in SP2016 Search?

Review of Search Components & Terminologies

Review of Search Components

Search Roles

• Search Administration

• Crawl

• Content Processing

• Analytics Processing

• Index

• Query Processing

Search Databases

• Search Administration DB

• Crawl DB

• Analytics Reporting DB

• Link DB

Other Search-related Terminologies

• No. of items in Index• Search Topology• Index Partitions• Content Sources• Incremental Crawls/Continuous Crawls• Full Crawls• Crawled Properties• Managed Properties

Review of Search Components

Crawl Content Processing Index

Analytics Processing

Query Processing Links

DB

WFEs

Search IndexAnalytics DB

Crawl DB

Crawl DB

Gathering Requirements for Search

Gathering Requirements for Search

• How many web applications & site collections in current environment?• How many total documents?• What are the different document file types? PDFs, TIFFs, DOCX, XLSX,

etc. ?• Are the PDF’s, image-only or searchable?• What’s the frequency of content additions and changes?• How quickly do the users expect their documents to be searchable, after

uploading to SharePoint?• Are there any Line-Of-Business external applications that interface with

SharePoint?

Questions

Gathering Requirements for Search

Questions ResponseNo. of Web Apps 2No. of Site Collections 25Total No. of Documents 3,980,272Document File Types PDFs (mostly), approx. 25,000 office

documents, approx. 3000 TIFFsPDF Types Mostly Searchable PDFs, and approx. 2000

Image-Only PDFsFrequency of content change Approx. 200 documents added per hourHow soon should docs be retrievable from search

10 minutes or less

Sample Response

Search Farm Architecture & Design

Search Farm Architecture & DesignSample Search Farm Design

DB Server

WFE 1

WFE 2

APP 1

APP 2

Office Web Apps

WAWS

Search Farm Architecture & DesignSample Search Farm Design

DB Server

WFE 1

WFE 2

APP 1

APP 2

Office Web Apps

WAWS

CRWLR 1

CRWLR 1

Query component

Query component

Crawl/Search Admin components

Crawl/Search Admin components

Search Farm Architecture & Design

Crawl & Search Admin Servers:• 8 vCPUs• 32GB RAM• 80GB - System Drive• 120GB - Data Drive (SP Binaries, Data, Logs)• 40GB – Drive for Search Index

Search Servers Specifications

Search Farm Architecture & Design

Analytics & Content Processing Servers:• 4 vCPUs• 16GB RAM• 80GB - System Drive• 120GB - Data Drive (SP Binaries, Data, Logs)

Search Servers Specifications

Search Farm Architecture & Design

Query Servers:• 4 vCPUs• 16GB RAM• 80GB - System Drive• 120GB - Data Drive (SP Binaries, Data, Logs)• 40GB – Drive for Search Index

Search Servers Specifications

Search Farm Architecture & Design

Incremental Crawls or Continuous Crawls??

Search Farm Architecture & Design

Don’t forget about your DB servers!

Search Farm Architecture & Design

Incremental, Continuous and Full Crawls add a lot of disk I/O to the DB servers

Possible Issues:• Slow Crawl Rates (dps in the single digits)• Documents getting skipped during indexing• High query latency (takes too long for search results to come back)

Search Farm Architecture & Design

Possible Resolutions:• Check disk I/O on the SQL server data drive

• Move the Search Databases to a different LUN (if virtual) or DISK (if physical)

• Change the Search Performance level from the default “Maximum” to “Partly Reduced” or “Reduced”• Here’s how you fix it using PowerShell

• Check memory utilization on the crawl & query servers• Look for RAM utilization on the msssearch.exe and noderunner.exe services

What’s new in SP2016 Search

What’s New in SP2016 Search

What’s New in SP2016 Search

• Each Search Service Application now supports indexing up to 500 million items (SP2013 supported 250 million items)

• Searching for sensitive documents. E.g. documents with SSNs, credit cards, passport numbers etc. in them, using Data Loss Prevention (DLP)

• Hybrid Search is available in SharePoint 2016 and SharePoint 2013 + August September 2015 CU• Can crawl Office 365 content as well as On-Prem content and get unified results

across both.

Additional Questions?

SharePint

Ajay Iyer

• SharePoint Architect• @shankarajay1• http://sharepointadminstuff.wordpress.com• Solution Architecture, Capacity Planning, Search, Migrations,

Enterprise Content Management, Document Imaging and….. InfoPath