SHAREPOINT SERVER 2013 FARM ARCHITECTURE AND PERFORMANCE Level 300

• Founding Partner of Summit 7 Systems, Inc.

• Summit 7 Systems Lead Architect• Six time Microsoft® MVP for SharePoint

Server• CISSP, Microsoft Certified Trainer, CCNA,

MCITP: SharePoint Server• Author of several SharePoint books by

Microsoft PRESSS• Master SCUBA Diver Trainer• ben.curry@summit7systems.com

BEN CURRY / MANAGING PARTNER

@curryben

http://summit7systems.com/blogs/author/ben-curry/

 

» Summit 7 Systems was chosen by KMWorld Magazine as one of the top 100 Companies that Matter in Knowledge Management along with companies such as Microsoft, Oracle and IBM.

» Summit 7 Systems was named to the 2011 and 2012 CRN Next-Gen 250 List as a company bringing innovative processes, methodologies and models to the solution provider industry.

» Top 1% of Microsoft Partners Worldwide» Summit 7 Systems was named #6 on the 2012 CRN Fast Growth 100 based on our 2009 – 2011 growth of over

930% per year.»~ 50% of Technical Staff hold US Government SECRET Clearances.»Service Disabled Veteran Owned Small Business (SDVOSB).

Summit 7 systems is a premier provider of consulting and implementation services specializing on the Microsoft SharePoint Platform and FAST Enterprise Search.

 

» Summit 7 Systems was chosen by KMWorld Magazine as one of the top 100 Companies that Matter in Knowledge Management along with companies such as Microsoft, Oracle and IBM.

» Summit 7 Systems was named to the 2011 and 2012 CRN Next-Gen 250 List as a company bringing innovative processes, methodologies and models to the solution provider industry.

» Top 1% of Microsoft Partners Worldwide» Summit 7 Systems was named #6 on the 2012 CRN Fast Growth 100 based on our 2009 – 2011 growth of over

930% per year.»~ 50% of Technical Staff hold US Government SECRET Clearances.»Service Disabled Veteran Owned Small Business (SDVOSB).

OUR BACKGROUND

SERVICESSharePoint QuickStartInformation Architecture and Governance DevelopmentUpgrade and MigrationBranding and Design (User Experience)Web Content Management Design and DeploymentSharePoint SearchCustom Workflow or Web Part DevelopmentInfoPath Forms and WorkflowsPerformance Baselines and Best Practices OptimizationMapping Business Process to Software PlatformsCloud Services Design and ProvisioningRemote Support ContractsCompliance Quickstart

SOFTWARE PLATFORMSFAST Enterprise SearchSharePoint 2007SharePoint 2010 SharePoint 2013Office PlatformSitecore

SOLUTION AREASSharePoint Platform SolutionsEnterprise SearchEnterprise Content ManagementInternet / Web Content ManagementExtranet SolutionsIntranet SolutionsBusiness Process ManagementEnterprise Project ManagementExchange Server

SERVICESSharePoint QuickStartInformation Architecture and Governance DevelopmentUpgrade and MigrationBranding and Design (User Experience)Web Content Management Design and DeploymentSharePoint SearchCustom Workflow or Web Part DevelopmentInfoPath Forms and WorkflowsPerformance Baselines and Best Practices OptimizationMapping Business Process to Software PlatformsCloud Services Design and ProvisioningRemote Support ContractsCompliance Quickstart

SOFTWARE PLATFORMSFAST Enterprise SearchSharePoint 2007SharePoint 2010 SharePoint 2013Office PlatformSitecore

SOLUTION AREASSharePoint Platform SolutionsEnterprise SearchEnterprise Content ManagementInternet / Web Content ManagementExtranet SolutionsIntranet SolutionsBusiness Process ManagementEnterprise Project ManagementExchange Server

PLATFORMS, SOLUTIONS & SERVICES

COMMERCIAL PAST PERFORMANCEMEDICAL/PHARMA FINANCE ENERGY/CHEMICAL EDUCATION MANUFACTURING

University of Utah Medical Center

St Jude Children’s Research Hospital

Nektar Therapeutic

Pfizer

OneWest Bank

BBVA Compass

WellPoint

Bank Independent

Otter Tail Power

Nova Chemicals

Champion Technologies

QuikTrip

Kraton

Kentucky Community and Technical College

System

Texas A&M University, Qatar

UNC-Charlotte

Escola de Sao Paulo, Macau, China

Hawker-Beechcraft

Kimberly-Clark

Coca Cola Bottling Consolidated

SC Johnson

Nucor Steel

STATE & LOCAL PAST PERFORMANCEMINNESOTA

DEPARTMENT OF TRANSPORTATION

ALABAMA MEDICAID

KENTUCKY COMMUNITY & TECHNICAL COLLEGE SYSTEM

GALVESTON COUNTY, TEXAS

CITY OF CHARLOTTE,

NORTH CAROLINA

COUNTY OF SANTA CLARA, CALIFORNIA

1 Year

902TS Professional Services Contract

Holder

SharePoint 2007 to BPOS-Dedicated SharePoint 2010

Migration

Business Data Catalog

Development

14 Months

AvePoint DocAve Architecture,

Implementation and Support

Operations and Maintenance

through AnchorPoint

Strategy and GovernanceEmployee Portal

SharePoint 2007 to SharePoint 2010 Migration

SharePoint 2010 to SharePoint 2013 Migration

Business Process Automation

Project Management Office

SiteCore Web Content Management

Operations and Maintenance through AnchorPoint

18 Months

SharePoint 2007 to SharePoint 2010

Migration

SharePoint 2010 Public Farm

Branding, Custom Development and Website Buildout

Operations and Maintenance through

AnchorPoint

4 Months

SharePoint 2010 Extranet

Requirements, Planning and

Design 

  8 Months

SharePoint 2010 Requirements,

Design and Implementation

FEDERAL PAST PERFORMANCE

REDSTONE TEST CENTER

MISSILE DEFENSE AGENCY

SPACE AND MISSILE DEFENSE

COMMAND

NATO MEADS MANAGEMENT

AGENCY

DEPARTMENT OF VETERANS AFFAIRS

INTERNAL REVENUE SERVICE

Sub to ERC and Bering Straits Information Technology

22 Months

Information Architecture Development

System Stabilization

SharePoint 2010 Migration

Sub to Boeing, SRA International and

Trident Group

10 Months

Information Architecture Development

SharePoint Logical and Physical Architecture

Dashboarding and Business

Intelligence

Sub to ASD

2 Months

Custom Workflow Development

SharePoint Branding and UX

Sub to KAI Global

5 Months

SharePoint and K2 Workflow

Development / Troubleshooting

Sub to Dell / Quest Software

9 Months

Quest SharePoint Toolset

Requirements, Design and

Implementation

Sub to Dell / Quest Software

                                9 Months

SharePoint 2007 to SharePoint 2010

Migration

9

AGENDA

SharePoint Server 2010 Farm ArchitectureCapacity PlanningTesting Tools

SHAREPOINT SERVER 2010

• Enterprise Search• PerformancePoint• Excel Calc Services• BCS• InfoPath• Word Conversions• Visio Graphics• Managed Metadata• User Profiles• Web Analytics

• Project Server• FAST for SharePoint• Sandboxed Solutions• Office Web Apps• Web Applications• Usage Logging• Access Services• Secure Store

OLD SCHOOL 2007

WebQueryIncoming Email

WebQueryIncoming Email

Index ServerExcel ServicesDocument Conversions

SQL ServerSQL Server

SHAREPOINT SERVER 2013 FARM

WebQueryIndexIncoming EmailViso Graphics ServiceBCSManaged Metadata

WebQueryIndexIncoming EmailViso Graphics ServiceBCSManaged Metadata

CrawlerExcel ServicesDocument ConversionsProject ServerSecure StorePerformancePointUser Profile SyncUser Profiles

SQL ServerReporting ServicesAnalysis Services

SQL ServerReporting ServicesAnalysis Services

CrawlerExcel ServicesDocument ConversionsProject ServerSecure StorePerformancePointUser ProfilesWeb Analytics

WEB APPLICATION ARCHITECTURE

SiteCollection

SiteCollection

SiteCollection

SiteCollection

http://portal.contoso.com

ContentDatabase

ContentDatabase

ContentDatabase

WEB APPLICATION ARCHITECTURE

SiteCollection

SiteCollection

SiteCollection

SiteCollection

http://portal.contoso.com

ContentDatabase

ContentDatabase

ContentDatabase

SQL Server

ApplicationPool

SERVICE APPLICATION ARCHITECTURE

ServiceApplication

Service Machin

e Instanc

e

Service Machin

e Instanc

e

Service Application Proxy

Web Application

Service Instance Service Instance

MANAGED METADATAMOVING SERVICE ON SERVER

SCALING SEARCH

Crawl Server (component)› Temp Indexing› Connects to Crawl Database

Query Server (component)› Index Partition and Query Component› Can be Primary or Failover

Property DatabaseCrawl DatabaseCrawling Target Server (dedicated host for crawling)

SEARCH ARCHITECTURE

SQL Server - SearchDB- PropertyDB

SQL Server - SearchDB- PropertyDB

Misc Apps- Crawler

Misc Apps- Crawler

Web- Crawl Target

WebQueryIndex

WebQueryIndex

CAPACITY & PERFORMANCE PLANNING

Research and Study› http://technet.microsoft.com/en-us/library/cc262971.aspx

Define your targetsCreate Logical ArchitectureDesign Physical ArchitectureTest and validateLog and MonitorAdjust architecture as needed

CACHING & COMPRESSION

HTTP CompressionBLOB CachingOutput CachingObject Caching

MY TOOLBOX

Internet Explorer Developer ToolbarVisual Round-trip Analyzer

› Free Download› Requires Netmon 2.x

FiddlerDev DashboardVisual Studio Team Suite & Ultimate versions

DEVELOPER DASHBOARD

$DevDashboardSettings = [Microsoft.SharePoint.Administration.SPWebService]::ContentService.DeveloperDashboardSettings;

$DevDashboardSettings.DisplayLevel = 'OnDemand';

$DevDashboardSettings.RequiredPermissions = 'EmptyMask';

$DevDashboardSettings.TraceEnabled = $true;

$DevDashboardsettings.Update()

WEB APPLICATION BOUNDARIESLIMIT MAXIMUM VALUE LIMIT TYPE NOTES

Content database

300 per Web application

Supported

Administrative operations such as creating a new site collection will experience decrease in performance. Use Windows PowerShell to manage the Web application when a large number of content databases are present.

Zone 5 per Web application BoundaryThe number of zones defined for a farm is hard-coded to 5.

Managed path 20 per Web application Supported

Managed paths are cached on the Web server, and CPU resources are used to process incoming requests against the managed path list.

Application Pools

10 per Web Server Supported RAM and Workload

CONTENT DATABASE BOUNDARIES

Content database size200 GB per content database.**4TB w/ restrictions

Supported

+200GB requires disk I/O of .25 IOPS per GB, but 2 IOPS per GB is recommended.

Robust DR strategy

Site collections per content database

2,000 recommended5,000 maximum

Supported

Strongly recommend limiting the number of site collections in a content database to 2,000. *Up to 5,000 site collections in a database are supported.

Remote BLOB Storage (RBS) storage subsystem on Network Attached Storage (NAS)

Time to first byte of any response from the NAS cannot exceed 20 milliseconds 

Boundary

From the time that SharePoint Server 2010 requests a BLOB, until it receives the first byte from the NAS, no more than 20 milliseconds can pass.

VERY LARGE SCALE DEMO

14.4TB content database failed over in 45 seconds…http://blogs.msdn.com/b/pandrew/archive/2011/10/06/video-published-of-the-full-farm-failover-demo-in-the-sharepoint-2011-conference-keynote.aspx

Document Size

# of Docume

nts

LISTS AND LIBRARIES BOUNDARIESFile size 2 GB Boundary The default maximum file size is 50 MB.

Documents 30,000,000 per library SupportedVaries depending on organization, type and size.

Items 30,000,000 per list SupportedThis value may vary depending on the number of columns in the list and the usage of the list.

List view threshold

5,000 ThresholdMaximum number of items that a database operation can simultantouesly process (outside the daily time window)

Subsite 2,000 per site view ThresholdNavigation and management performance will decrease significantly as the number of subsites grows.

SEARCH BOUNDARIESSharePoint search service applications 20 per farm Supported ..that’s more than you need!

Crawl databases and database Items

*10 crawl databases per search service application*25 million items per crawl database

ThresholdThe supported limit is 10 crawl databases per SharePoint Search service application.

Crawl components 16 per search service application Threshold

The total number of crawl components per server must be less than 128/(total query components) .

Index partitions 20 per search service application; 128 total Threshold The boundary for the total number

of index partitions is 128.

Indexed items100 million per search service application; 10 million per index partition

Supported

Crawl log entries 100 million per search application Supported

Property databases 10 per search service application Threshold

Query components 128 per search application ThresholdConcurrent crawls 20 per search application Threshold

MINE 2007 USAGE CONTENT

Users› Browser Read› Browser Write› Outlook RSS› Outlook Sync› OneNote› Offline Clients› Collab› Management

• Operations– Indexing– Search Crawling– Backup/Restore– CooP– Shared Services

DOCUMENT CURRENT CONTENT

Database SizesNumber of content databasesNumber of site collections

› Current Quotas!

Number of Web appsSearch Index and DB sizeUser Profiles

DEFINE TARGETS

Server availabilityWeb application availabilityService Application availabilityResponse TimesFour ‘9s’ is achievable

› (but it is expensive…)

VISUAL STUDIO TESTING FOR IT PROS

Visual Studio 2010 isn’t just for developers!

OVERVIEW OF INFRASTRUCTURE TESTING

Test single server or modify affinity/stickyFirewall ports from controller and agents to controller/agents/serversRemote Registry permissions to tested servers

WHY TEST?

Ensure adequate resourcesTest performance impact of Web partsTest any major configuration change

DON’T GUESS. KNOW!

MY FARM SETUP

SharePoint Server 2010SQL Server 2008 R2Visual Studio Ultimate 2010Windows Server 2008 R2

FE01(Web + Query)

FE02(Web + Query)

App01(Applications)

APP01SQL Server

PREPARE VISUAL STUDIO

Setup Results Database on SQL Server 2005 or 2008Remove default tests if desiredSetup the test controller and agents

BUILD A TESTING 'RIG’

Requires Visual Studio 2010 Ultimate, Test, or Team Suite Editions

› If you don’t see ‘Test’ as an option, you have the wrong version

Must have one controller› Use multiple agents for better real-world load

Create new Project using the ‘Test Project’ templateCreate Load test results store (DB)Set loadtest database as the repository

Tip: Use loadtestresultsrepository.sql to create the repository database:

C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE’

CREATE A TEST PROJECT AND WIRE UP THE REPOSITORY

TEST WEB PART IMPACT

Record Web test for the hosting pageBaseline load time

› Change run settings to test browser version and WAN clients

Load Web partTest response timeShould not be used for production testing, but will give you a good idea of Web part impact

CREATE LOAD TEST

Create load test using one or more Web tests› Choose % mix for your environment› For new implementations or complete

unknowns, use the posted mix on TechNet

Document Load Test Settings› Be sure to use the same settings when

comparing baselines and changes

Create Peak and off-peak baselines

READING THE RESULTS

Processor time and Disk I/O are criticalUse Systems Center Operations Manager management pack documentation for performance counters to monitor during tests

› Or rule author view

Load Test Summary is usefulAdd custom counters for Search, etc.

LAST PERFORMANCE SLIDE

Analyze the ResultsIdentify bottlebecksOptimizeRinse, Lather, & Repeat

LOGGING

Event LogsTrace LogsUsage LogsWeb AnalyticsDeveloper DashboardSCOM

MONITORING

Health ReportsWeb AnalyticsAdministrative ReportsTimer Job StatusSQLIO Benchmark Toolhttp://go.microsoft.com/fwlink/?LinkID=105586

AGGREGATE PERFORMANCE

Add-SPDiagnosticsPerformanceCounter -Category "Processor" -Counter "% Processor Time" -Instance "_Total" -WebFrontEnd

WHAT ARE WE LOOKING FOR?Processor

Processor - % Processor Time Over 75-85%Upgrade processorIncrease number of processorsAdd additional server(s)

Disk

Avg. Disk Queue LengthGradually increasing, system not in a steady state and queue is backing up

Increase number or speed of disksChange array configuration to stripeMove some data to an alternative server

% Idle Time Greater than 90% Increase number of disksMove data to an alternative disk or server

% Free Space Less than 30% Increase number of disksMove data to an alternative disk or server

Memory

Available Mbytes Less than 2GB on a Web server.

Add memory.Note: SQL server available memory will be low, by design, and does not always indicate a problem.

Cache Faults/sec Greater than 1Add memory Increase cache speed or size if possibleMove data to an alternative disk or server

Pages/sec Greater than 10 Add memory

Paging File

% Used and % Used Peak

The server paging file, sometimes called the swap file, holds "virtual" memory addresses on disk. Page faults occur when a process has to stop and wait while required "virtual" resources are retrieved from disk into memory. These will be more frequent if the physical memory is inadequate.

Add memory

NIC

Total Bytes/secOver 40-50% of network capacity. This is the rate at which data is sent and received via the network interface card.

Investigate further by monitoring Bytes received/sec and Bytes Sent/sec.Reassess network interface card speedCheck number, size, and usage of memory buffers

Process

Working Set Greater than 80% of total memory Add memory

% Processor Time Over 75-85%.Increase number of processorsRedistribute workload to additional servers

ASP.NET

Application Pool Recycles Several per day, causing intermittent slowness.Make sure that you have not implemented settings that automatically recycle the application pool unnecessarily throughout the day.

Requests Queued Hundreds or thousands of requests queued.Implement additional Web serversThe default maximum for this counter is 5,000, and you can change this setting in the Machine.config file

Request Wait TimeAs the number of wait events increases, users will experience degraded page rendering performance.

Implement additional Web servers

Requests Rejected Greater than 0 Implement additional Web servers

WHAT ARE WE LOOKING FOR?

PROCESSOR & PROCESSESProcessor

% Processor Time

This shows processor usage over a period of time. If this is consistently too high, you may find performance is adversely affected. Remember to count "Total" in multiprocessor systems. You can measure the utilization on each processor as well, to ensure balanced performance between cores.

Process

- Working SetThis counter indicates the current size (in bytes) of the working set for a given process. This memory is reserved for the process, even if it is not in use.

- % Processor TimeThis counter indicates the percentage of processor time that is used by a given process.

Thread Count (_Total) The current number of threads.

PERFORMANCE COUNTERS

Processor

You should monitor processor performance to ensure that all processor usage does not remain consistently high (over 80 percent) as this indicates that the system would not be able to handle any sudden surges of activity. And that in the common state, you will not see a domino effect if one component failure will bring the remaining components to a malfunctioning state. For example – if you have three Web servers, you should make sure the average CPU across all servers is under 60% so that if one fails, there is still room for the other two to pick up the extra load.

Network Interface Monitor the rate at which data is sent and received via the network interface card. This should remain below 50 percent of network capacity.

Disks and Cache

There are a number of logical disk options that you should monitor regularly. The available disk space is essential in any capacity study, but you should also review the time that the disk is idle. Dependent on the types of applications or services you are running on your servers, you may review disk read and write times. Extended queuing for write or read function will affect performance. The cache has a major impact on read and write operations. You must monitor for increased cache failures.

Memory and Paging File Monitor the amount of physical memory available for allocation. Insufficient memory will lead to excessive use of the page file and an increase in the number of page faults per second.

DISK COUNTERS

Avg. Disk Queue Length

This shows the average number of both read and write requests that were queued for the selected disk during the sample interval. A bigger disk queue length may not be a problem as long as disk reads/writes are not suffering and the system is working in a steady state without expanding queuing.

Avg. Disk Read Queue Length The average number of read requests that are queued.

Avg. Disk Write Queue Length The average number of write requests that are queued.

Disk Reads/sec The number of reads to disk per second.

Disk Writes/sec The number of writes to disk per second.

MEMORY COUNTERS

- Available MbytesThis shows the amount of physical memory available for allocation. Insufficient memory will lead to excessive use of the page file and an increase in the number of page faults per second.

- Cache Faults/sec

This counter shows the rate at which faults occur when a page is sought in the file system cache and is not found. This may be a soft fault, when the page is found in memory, or a hard fault, when the page is on disk.The effective use of the cache for read and write operations can have a significant effect on server performance. You must monitor for increased cache failures, indicated by a reduction in the Async Fast Reads/sec or Read Aheads/sec.

- Pages/secThis counter shows the rate at which pages are read from or written to disk to resolve hard page faults. If this rises, it indicates system-wide performance problems.

Paging File

- % Used and % Used Peak

The server paging file, sometimes called the swap file, holds "virtual" memory addresses on disk. Page faults occur when a process has to stop and wait while required "virtual" resources are retrieved from disk into memory. These will be more frequent if the physical memory is inadequate.

ASP.NETRequests Total The total number of requests since the service was started.

Requests QueuedMicrosoft SharePoint Foundation 2010 provides the building blocks for HTML pages that are rendered in the user browser over HTTP. This counter shows the number of requests waiting to be processed.

Request Wait TimeThe number of milliseconds that the most recent request waited in the queue for processing. As the number of wait events increases, users will experience degraded page-rendering performance.

Requests RejectedThe total number of requests not executed because of insufficient server resources to process them. This counter represents the number of requests that return a 503 HTTP status code, indicating that the server is too busy.

Requests Executing (_Total)

The number of requests currently executing.

Requests/Sec (_Total)

The number of requests executed per second. This represents the current throughput of the application. Under constant load, this number should remain within a certain range, barring other server work (such as garbage collection, cache cleanup thread, external server tools, and so on).

SQL SERVERGeneral Statistics

This object provides counters to monitor general server-wide activity, such as the number of current connections and the number of users connecting and disconnecting per second from computers running an instance of SQL Server.

User ConnectionsThis counter shows the amount of user connections on your instance of SQL Server. If you see this number rise by 500 percent from your baseline, you may see a performance reduction.

Databases

This object provides counters to monitor bulk copy operations, backup and restore throughput, and transaction log activities. Monitor transactions and the transaction log to determine how much user activity is occurring in the database and how full the transaction log is becoming. The amount of user activity can determine the performance of the database and affect log size, locking, and replication. Monitoring low-level log activity to gauge user activity and resource usage can help you to identify performance bottlenecks.

Transactions/secThis counter shows the amount of transactions on a given database or on the entire SQL Server instance per second. This number is to help you create a baseline and to help you troubleshoot issues.

Locks This object provides information about SQL Server locks on individual resource types.Number of Deadlocks/sec This counter shows the number of deadlocks on the SQL Server per second. This should normally be 0.

Average Wait Time (ms) This counter shows the average amount of wait time for each lock request that resulted in a wait.Lock Wait Time (ms) This counter shows the total wait time for locks in the last second.

Lock Waits/sec This counter shows the number of locks per second that could not be satisfied immediately and had to wait for resources.

SQL Statistics

This object provides counters to monitor compilation and the type of requests sent to an instance of SQL Server. Monitoring the number of query compilations and recompilations and the number of batches received by an instance of SQL Server gives you an indication of how quickly SQL Server is processing user queries and how effectively the query optimizer is processing the queries.

Buffer CacheThis object provides counters to monitor how SQL Server uses memory to store data pages, internal data structures, and the procedure cache, as well as counters to monitor the physical I/O as SQL Server reads and writes database pages.

summit7systems

summit7systems.com/blogs

FOR MORE FROM SUMMIT 7 SYSTEMS…

facebook.com/summit7systems

@summit7systems

summit 7 systems